MobileFrontend/Dynamic Sections

This puts down some ideas on how we could reduce the page size we serve to mobile users.

Problem and Motivation
Large pages are a problem on mobile devices for several reasons.


 * On older devices they can cause slowdown (for example Bugzilla::32773
 * Network connectivity can heavily impact download speed and ability, and it is extremely variable
 * Evidence suggested that many users on mobile get the answer they are looking for in the first section (calculated by logging page views against how many people toggled a section). Also sections further down the page are less likely to get loaded - See https://www.mediawiki.org/w/index.php?title=Event_logging/Mobile&oldid=619140#Section_toggles
 * thus lots of unnecessary content was being loaded on initial load see Bugzilla::31011
 * Prevent downloading of images hidden in sections
 * In the current setup if there are 8 photos in the History section of the San Francisco article, these will get loaded regardless of whether that section is ever opened.
 * Smaller payload
 * Firstly many phones are terrible at scrolling large pieces of content. The idea was to split an article into separate pages for each individual section. This way sub content could easily be accessed and read without having to scroll through a long article. Also some ancient phones simply cannot cope with large articles and will crash.
 * Load and run javascript quicker
 * Tests were run on the Barack Obama article (a known large article) and reducing the page to just show sections resulted in a 90% reduction in the time for all client side javascript ro run https://docs.google.com/spreadsheet/ccc?key=0ArbzKvV50qF6dEFzR0pCbUxfVy1ybGVPZU9yRTU0OUE#gid=1

Motivation to hijacking links and loading subsequent pages via javascript
Using the history api the mobile site could be made to feel more like an app

The current site reloads the entire UI on every page load. As well as being wasteful this makes the website feel less responsive then an app where clicking on a link simply loads the page into the UI.

By moving in a direction where all links are hijacked and loaded via ajax the mobile website will feel more snappy. It would be good to run user tests on this to quantify this more but since this has been in beta I have had many positive comments about how this feels nicer to user.

Proposed Solution

 * When accessing a page on the mobile site (e.g. http://m.wikipedia.org/wiki/San%20Francisco) we load the summary of the article and the headings of the sections in that article.
 * Once this is loaded we should load *all* other sections asynchronously so that the user never perceives a delay in anything Tfinc (talk) 18:31, 22 June 2012 (UTC)
 * Section headings for non javascript enabled devices would link to a dedicated section page

For pages with javascript

 * Devices with javascript would enhance these links so that when clicked they load sections dynamically. YuviPanda has done some work on this in the Mobile App so it would be a case of refactoring and reusing that code.
 * On unfolding a section the address bar would be update with a hash - refreshing a page with javascript and this hash would load that section

For pages without javascript

 * Without javascript these links would render only that section e.g http://en.m.wikipedia.org/wiki/San_Francisco/Section:History would load a page with just the history section.
 * Clicking the title of the article (San Francisco) in any given section (e.g. the history section) or pressing the back button would return a user to the main page

J2ME App

 * The J2ME App will use LUIT (Lightweight UI Toolkit) which allows a form of dynamic loading - only the part of the page that needs updating is updated, which minimizes the usage of volatile memory - a similar behavior to pages with Javascript

WAP

 * For WAP or basic feature phones, we can do basically the same as the "without Javascript" approach, above, without any styling. See 31714.

Potential issues

 * Introducing Javascript changes always carries some risk of breaking current functionality, and there have been complications due to recent Javascript changes
 * The sections would fall back to clickable links if an error is thrown during the expanding of a section. So this shouldn't be such a problem
 * Caching issues may arise due to treating sections as pages - more input from Ops needed
 * Logging will be affected, and scripts that filter the logs for data analysis will need to be updated
 * Back behavior will treat sections as back steps - possibly a good thing
 * But not necessarily on the javascript enabled version.
 * Saving pages requires the entire article to be downloaded - or should we change the convention to save only the sections that have been downloaded?
 * This is a solvable problem but we don't save on the mobile site at the moment so this should be a concern for later.

Working with the API
I would appreciate some feedback from MaxSem.

Section headings would need to annotate themselves with their section id to assist in the usage of the api. Currently section ids on a given page are based on their position in the document. The 10th heading in a page will have id 10 however than 10th heading might be a h4 tag hidden within the second h2 tag. With the above proposal the page will only know its top level sections so it will be difficult to know what id to pass to the api without assistance from the server. I suggest this is done by annotating h2 tags with a 'data-section' attribute which contains the id of that section needed to look it up in the api.

One concern I have is whether the id's are reliable. For example take a page with 3 headings 'A', 'B' and 'C'. Imagine content of those sections is only clicked when I load those headings. If I click 'B' a request to the api is made to grab section with id 2. However between me loading the page and clicking 'B' another user might add a new section above 'A' called 'Z'. My worry is that when clicking 'B' it actually grabs section 'A' as the id's have now changed with respect to the document.

Implementation
This is now deployed to the beta. However there is no fallback for non-javascript users https://bugzilla.wikimedia.org/show_bug.cgi?id=42746

To think about

 * How would saving pages would where a page has not been completely downloaded? - see Potential issues above
 * How would this effect a find on page function?
 * Would it make sense to do more on the server in terms of preserving page context so the section requests are not so disruptive to caching and logging?