User:Bjoern~mediawikiwiki/low bandwidth suggestions

From mediawiki.org

Here are three suggestions to enhance mediawiki. The aim is to make mediawiki more friendly for browsing from poorly connected areas. The three suggestions are:

  • to provide facilities for mediawiki mirroring (via OAI extension or enhancement of the API)
  • to reduce the size of the default skin to meet low-bandwidth accessibility guidelines
  • to provide a very low bandwidth skin and mobile browser detection (that can be overridden by the user).

Those three elements should become part of the mediawiki software itself, and ideally be turned on by default, so that all projects using mediawiki (including wikipedia, other wikimedia projects, wikieducatior, etc) can benefit from them.

Overall, my suggestion would be to set up a working group to look at these issues, to have some discussion, and then implement those features in a way that fits with existing strategies for the mediawiki software development.

Suggestion 1: Provide facilities for mediawiki mirroring[edit]

Wikis are designed to be used through a web browser "online". They can be difficult to use in hybrid online/offline 'fragile connectivity' environments. However, this is exactly the environment users in the developing world find themselves in. So if you become a contributor to wikieducator, but one day your connection fails for a week, your materials become inaccessible for your local teaching. That's unacceptable.

A simple solution would be provided by one-way wiki mirroring: A 'public' wiki is mirrored into a 'local' wiki (on a LAN, or on a national internet exchange point). A simple scenario would be for both wikis to run the same software (i.e. mediawiki), because they would thereby provide the same functionality (such as searching, pdf downloading etc). However, the local wiki need not be editable, but 'edit' links are replaced by links to the public wiki. (However, see synchronisation suggestion below.)

Step 1: Getting the data - Make OAI extension more user friendly, or enhance the API[edit]

In order to be able to mirror a mediawiki locally, you need to be able to get updates from the public wiki. This is possible through the OAI extension. With the extension it is possible to obtain page updates from a remote wiki, and ingest these into a local wiki. However,

  • the extension isn't part of the mediawiki software itself,
  • there are some long-standing issues with it, and
  • installation is far from trivial for the average user.

(For further information, see SMN:Mediawiki/OAI_mirror.)

It would be desirable to update the extension, and move the functionality into the core of the software. An OAI interface is an interesting feature to have, and perhaps it is worth maintaining the OAI functionality.

Another approach might be integrate such functionality into the API. In principle it seems as if the API could do some of this funtionality, but because of performance considerations, it doesn't seem to be trivial to do so. Basically the API is unable to provide changed pages (across all namespaces) between two dates: see this thread ( mailarchive:mediawiki-api/2009-May/001141.html, mailarchive:mediawiki-api/2009-May/001142.html, mailarchive:mediawiki-api/2009-May/001143.html, mailarchive:mediawiki-api/2009-May/001145.html), continued in this thread (mailarchive:mediawiki-api/2009-June/001157.html, mailarchive:mediawiki-api/2009-June/001158.html, mailarchive:mediawiki-api/2009-June/001159.html, mailarchive:mediawiki-api/2009-June/001160.html).

Ideally, the desired functionality would be the ability to obtain

  • a list of all revisions between two dates or revision numbers (for complete mirroring including all revisions)
  • a list of pages changes between two dates or revision number (for mirroring the latest page versions only)

Information about file uploads would also need to be included.

I don't know the details, but it seems that the OAI extension was built specifically to overcome the performace issues, and to provide a format that is specifically tailored to meet mirroring requirements.

To make progress on this, one would need to discuss exactly what page information is needed for reliable mirroring, and then make those changes accordingly.

Step 2: Local wiki - automatically ingest updates into the local wiki.[edit]

The OAI extension also allows one to do this (see SMN:Mediawiki/OAI_mirror). This works by installing the OAI extension also on a local wiki, and configure it to point back at a public wiki. A php script can then be run, that fetches updates from the public wiki, including file uploads.

This is key functionality, that would be a significant enhancement if available out of the box. If one decided to maintain the OAI extension, not much further work is needed to make this work. The main improvements would be in usability.

Alternatively, if one went to an API-based process, the mirroring functionality could be bundled into an extension: It's always a local decision to provide a mirror, so the installation of the extension shouldn't be a major obstacle. However, the installation of the extension would need to be as straight forward as possible, without manual creation of extra tables etc. There would need to be some consideration of resilience, in case of interrupted updates, and the update process would need to be able to handle mirroring of file uploads.

Ideally, one would be able to choose whether to mirror a public wiki to a local wiki with

  • all revisions (i.e. all revisions are fetched on update), or
  • latest revisions only (i.e. intermediate revisions aren't fetched on update).

Add-on: Pragmatic two-way synchronisation[edit]

It would also be desirable to add two-way synchronisation. A simple proof of concept is available here SMN:Mediawiki/OAI_mirror/pragmatic_synchronisation.

Suggestion 2: Optimise MonoBook skin to be in line with low-bandwidth guidelines[edit]

The low-bandwidth guidelines recommend a page size of 25kB, up to 75kB for progressively loading content. An empty mediawiki page is about 100kB, and more if there is text/images.

As images are user contributed, it's hard to ensure full optimisation, but if the empty page size was reduced, it would leave more room for user content. The suggestion is thus to optimise the mediawiki default MonoBook skin to be more in line with low-bandwidth guidelines. This doesn't necessarily mean a major change in look and feel. Yahoo Yslow and Google page speed indicate that some optimisation would be fairly straight forward.

An example for an optimised monobook skin is available here: SMN:Mediawiki_access/Monobmo/Example. I optimised this very crudely, with not regard for appearances. I am not suggesting that we should simplify the layout that much, but it just shows that we can optimise the skin further.

Another low-bandwidth recommendation is to provide the size of links to images and other media. This could be achieved through a 'filesize' parser function, example here: SMN:Mediawiki_file_size

Suggestion 3: Very low bandwidth skin and mobile browser detection[edit]

Mediawiki should come with a no css/js skin[edit]

There should also be a very low bandwidth skin available, such as a skin with css/js removed, or a skin with very basic css/js.

  • This skin SMN:Mediawiki access/Monobmo/nocss is an example for nocss/js skin (or just turn of css/js in your browser :-), that could be used for mobiles when browsing from a small screen, and
  • this very light weight SMN:Mediawiki_access/Monobmo/Example skin, could be a replacement skin for the standard Monobook skin where minimal css/js is desirable.

The suggestion is the mediawiki might come with both of these additional skins.

Make the skin selectable by non-logged-in site visitors[edit]

This skin should be selectable by non-logged-in viewers of a mediawiki: If you are browsing from very low bandwidht, you just turn the low bandwidth skin on, and you get a better experience. Example on SMN wiki: toggle mobile skin.

Provide switching mechanism for mobiles[edit]

The basic no css/js skin should switch on automatically when users are browsing from a 'basic devices' (such as basic phones).

An example for this is provided on this wiki: http://www.sciencemedianetwork.org. The mobile skin is used automatically for mobile phone access, but can be switched by the user (to and from the simple skin), by using the toggle mobile skin link. For an example using the opera mini simulator, see SMN:Mediawiki_access/Mobile/Skin.

What constitutes a basic phone would be selectable by the administrator: On some wikis you may want to provide the simple skin for all mobile phones (including smart phones), or perhaps you only want to display the skin for some very basic phones (with basic native browsers).

It would be nice to have pagination, as is provided by some of the mobile phone interfaces, but that may not be absolutely essential.