MobileFrontend/Dynamic Sections

This puts down some ideas on how we could reduce the page size we serve to mobile users.

Problem and Motivation
Large pages are a problem on mobile devices for several reasons.


 * On older devices they can cause slowdown (for example Bugzilla::32773
 * Network connectivity can heavily impact download speed and ability, and it is extremely variable
 * Evidence suggested that many users on mobile get the answer they are looking for in the first section (calculated by logging page views against how many people toggled a section). Also sections further down the page are less likely to get loaded - See https://www.mediawiki.org/w/index.php?title=Event_logging/Mobile&oldid=619140#Section_toggles
 * thus lots of unnecessary content was being loaded on initial load see Bugzilla::31011
 * Prevent downloading of images hidden in sections
 * In the current setup if there are 8 photos in the History section of the San Francisco article, these will get loaded regardless of whether that section is ever opened.
 * Smaller payload
 * Firstly many phones are terrible at scrolling large pieces of content. The idea was to split an article into separate pages for each individual section. This way sub content could easily be accessed and read without having to scroll through a long article. Also some ancient phones simply cannot cope with large articles and will crash.
 * Load and run javascript quicker
 * Tests were run on the Barack Obama article (a known large article) and reducing the page to just show sections resulted in a 90% reduction in the time for all client side javascript ro run https://docs.google.com/spreadsheet/ccc?key=0ArbzKvV50qF6dEFzR0pCbUxfVy1ybGVPZU9yRTU0OUE#gid=1

Motivation to hijacking links and loading subsequent pages via javascript
Using the history api the mobile site could be made to feel more like an app

The current site reloads the entire UI on every page load. As well as being wasteful this makes the website feel less responsive then an app where clicking on a link simply loads the page into the UI.

By moving in a direction where all links are hijacked and loaded via ajax the mobile website will feel more snappy. It would be good to run user tests on this to quantify this more but since this has been in beta I have had many positive comments about how this feels nicer to user.

Implementation
This is now deployed to the beta. However there is no fallback for non-javascript users https://bugzilla.wikimedia.org/show_bug.cgi?id=42746

For pages with javascript

 * When accessing a page on the mobile site (e.g. http://m.wikipedia.org/wiki/San%20Francisco) we load the summary of the article and the headings of the sections in that article.
 * Once this is loaded we s load *all* other sections asynchronously so that the user never perceives a delay in anything Tfinc (talk) 18:31, 22 June 2012 (UTC)
 * Clicking a heading and waiting any length of time for the section content to load is unacceptable. If a page has been loaded all its content should be available from that point on (to deal with situations where a reader is going through a tunnel during the time they want to read section 2 which is closed).
 * Devices with javascript would enhance headings so that when clicked (unfolded) they load sections dynamically and feel instant to the user.
 * On unfolding a section the address bar updates with a hash - refreshing a page with javascript and this hash would load that section
 * Section headings for non javascript enabled devices should link to a dedicated section page (see below) in cases where the connection is unreliable and the request to collect section content fails.

For pages supporting the History API
All subsequent links to articles in a page would be routed via the API. This would give a greater feeling of responsiveness to the mobile site by avoiding reloading of the Chrome.

In future we might making use of a cache manifest so that the mobile site can work offline.

For pages without javascript (TODO)

 * Section headings for non javascript enabled devices should link to a dedicated section page (TODO)
 * Without javascript the headings would behave as links that when clicked would render only that section e.g http://en.m.wikipedia.org/wiki/San_Francisco/Section:History would load a page with just the history section.
 * Clicking the title of the article (San Francisco) in any given section (e.g. the history section) or pressing the back button would return a user to the page with headings and summary

J2ME App

 * The J2ME App will use LUIT (Lightweight UI Toolkit) which allows a form of dynamic loading - only the part of the page that needs updating is updated, which minimizes the usage of volatile memory - a similar behavior to pages with Javascript

WAP

 * For WAP or basic feature phones, we can do basically the same as the "without Javascript" approach, above, without any styling. See 31714.

Potential issues

 * Introducing Javascript changes always carries some risk of breaking current functionality, and there have been complications due to recent Javascript changes
 * The sections would fall back to clickable links if an error is thrown during the expanding of a section. So this shouldn't be such a problem
 * Caching issues may arise due to treating sections as pages - more input from Ops needed
 * Logging will be affected, and scripts that filter the logs for data analysis will need to be updated
 * Back behavior will treat sections as back steps - possibly a good thing
 * But not necessarily on the javascript enabled version.
 * Saving pages requires the entire article to be downloaded - or should we change the convention to save only the sections that have been downloaded?
 * This is a solvable problem but we don't save on the mobile site at the moment so this should be a concern for later.

Working with the API
I would appreciate some feedback from MaxSem.

Section headings would need to annotate themselves with their section id to assist in the usage of the api. Currently section ids on a given page are based on their position in the document. The 10th heading in a page will have id 10 however than 10th heading might be a h4 tag hidden within the second h2 tag. With the above proposal the page will only know its top level sections so it will be difficult to know what id to pass to the api without assistance from the server. I suggest this is done by annotating h2 tags with a 'data-section' attribute which contains the id of that section needed to look it up in the api.

One concern I have is whether the id's are reliable. For example take a page with 3 headings 'A', 'B' and 'C'. Imagine content of those sections is only clicked when I load those headings. If I click 'B' a request to the api is made to grab section with id 2. However between me loading the page and clicking 'B' another user might add a new section above 'A' called 'Z'. My worry is that when clicking 'B' it actually grabs section 'A' as the id's have now changed with respect to the document.

To think about

 * How would saving pages would where a page has not been completely downloaded? - see Potential issues above
 * How would this effect a find on page function?
 * Would it make sense to do more on the server in terms of preserving page context so the section requests are not so disruptive to caching and logging?