ResourceLoader/Version 2 Design Specification

Etherpad: RL2

= Module sources =

Tracking: bug 30022

= Gadget Manager =

Tracking: bug 29398

Use-cases
By having a visual interface, gadget managers will no longer have to learn and cope with yet another syntax (as used on ). The Special-page will be able to log events which will enable users to easily see who changed what and when. They will be able to get that information chronologically for all gadgets (What has been done in the last few days?), or for one particular gadget (What happened to Gadget Foo ?). This is handled through and the history-page of individual Gadget definition pages. Over the last year more and more options have been added to gadgets and resource loader modules. There's over a dozen module properties and already half a dozen Gadget settings (gadget title, description, and specific settings (required user rights, enable by default), module scripts, styles, dependencies, messages, position and more..). Having them in a nicely designed form with responsive javascript interactions and autocompletion will make it easier to understand and extendable without complicating the "Gadget syntax" every time (this syntax will cease to exist). Through this Gadget manager interface, the process of 'Adding' (or 'Creating') a gadget will be as simple as a click. Aside from writing one from scratch another goal of the Gadget Manager is to enable easy exporting and importing of Gadget properties from one wiki to another. Because the gadget definition will have a specified JSON format, it can easily be exported to a different wiki. Special:Export can be used to export the entire gadget and everything it needs (the definition, the title and description messages, the js/css pages and any other interface messages it may use). Note however that the process of exporting/importint a gadget is, although a lot easier in version 2, no longer intended to happen as much within the Wikimedia farm due to the new Shared gadgets feature, which no longer requires the local wikis to "copy" or "reroute" to a gadget from a central place. There will be an InstantCommons-like feature which allows third party wikis to easily flip a switch and get all the popular gadgets, that are actively developed within Wikimedia, onto their own wikis.
 * Usability
 * Tracking
 * Extendable
 * Sharing

Storage of Gadget definition
The definition of each gadget will be stored in a new MediaWiki database table,. mysql> DESCRIBE mw_gadgets; +--+-+--+-+-+---+ +--+-+--+-+-+---+ +--+-+--+-+-+---+
 * Field       | Type                | Null | Key | Default | Extra |
 * gd_id       | varbinary(255)      | NO   | PRI | NULL    |       |
 * gd_blob     | mediumblob          | NO   |     | NULL    |       |
 * gd_shared   | tinyint(1) unsigned | NO   |     | 0       |       |
 * gd_id: Unique id of the gadget. Lowercase [a-z], 0-9 and dashes allowed. Cannot be changed after creation.
 * gd_blob: JSON string containing the module properties and gadget settings
 * gd_shared: Whether this is a shared gadget. Gadgets with gd_shared=1 are exposed through the Repo interface.
 * "global" is somewhat wiki-farm specific. Perhaps "shared" is more appropriate ?
 * Yes, we basically agreed on this a while ago. Hereby updated to avoid further confusion.

The definitions will be saved to a wiki page for versioning, history, watching etc.

Gadget resources
The resources will stay where they are: As pages with revisions in the wiki database. The difference intended to be made as part of the ResourceLoader 2 project is moving these out of the prefixed "Gadget-" scope in the MediaWiki:-namespace (which is intended for messages, not actual wiki content (let alone executable resources). Instead move them to a new Gadget:-namespace only editable by users with the  right.

Messages
The title of the gadget and the description associated with will stay in the MediaWiki:-namespace. The main reason for this is, is to enable easy localization and centralization of the description.

Aside from the title (eg. ) and the description (eg. ) a gadget can have a number of custom made messages. These are handled by the existing system present in MediaWiki which is suited for this already and are loaded through the messages framework in ResourceLoader (associated at module registration and passed to the browser through  /  ).

User rights
To keep track of what user rights are talked about in the specification and task management, a short summary
 * gadgets-edit: Edit gadget resources in the Gadget:-namespace
 * gadgets-definition-create: Creating pages directly in the Gadget definition:-namespace.
 * gadgets-definition-edit: Edit in the Gadget definition:-namespace.
 * gadgets-definition-delete: Deleting pages directly in the Gadget definition:-namespace.

Special:Gadgets

 * Overview of all gadgets: :
 * Permalink to single gadget: :
 * Export-action of a gadget: :


 * Creation, modification, and deletion are possible without JavaScript through the regular WikiPage actions. However the visual editor requires JavaScript.

Related bugs

 * (bug 27561) Register modules client side
 * (bug 27771) Mechanism for loading site wide JavaScript libraries
 * (bug 27281) Add support in the front-end for loading wiki pages as resources

Designs
= Shared gadgets =

One of the most wanted features for gadgets is shared gadgets. Users want to share gadgets, reach a wider audience, and avoid outdated and redundant copies at all costs. By having a "Gadget Repository" this problem will be solved. When a gadget is created on a "gadget repository" wiki (and marked "shared") it will show up on all wikis who have their ForeignGadgetRepo set to that wiki.

For Wikimedia wikis this will most likely be set as default for all wikis (just like Wikimedia Commons is set as ForeignFileRepo). They would probably be stored on mediawiki.org (not Meta-Wiki as they're not Wikimedia-specific). As part of this project (or later) we may create a convenience configuration (like "InstantCommons") to easily allow third party wikis to use mediawiki.org as their foreign repository for gadgets.

Use-cases
No more need to duplicate, distribute, update etc. gadgets will be centrally organized and developed. Due to the nature of ResourceLoader it is simple to get interface messages in the user language. These messages would be fetched from the ForeignGadgetRepo, which means all messages will be central rather than on each wiki separately. Thanks to the fallback principle and nightly LocaliationUpdate, gadgets on mediawiki.org could even be translated on translatewiki.net
 * Sharing
 * Translation
 * Less duplication / Re-use of code:
 * Use-cases for dependancy model:
 * Loading a shared gadget from a foreign repository
 * Loading a gadget on a repository-wiki that depends on other gadget from that wiki
 * Loading a gadget on a local wiki that depends on a gadget from a foreign repository

Preferences

 * Preferences will either get:
 * 1 extra tab ("Gadgets") showing both local and foreign gadgets mixed, and grouped by 'category' with some kind of indication which are foreign and which aren't.
 * 2 extra tabs ("Gadgets", "Shared gadgets") both have their own sections.
 * Why? Perhaps end users should not be aware of it.
 * Why not? Clearer separation, feels better. We need more motivation for "2 tabs" to remain an option.
 * WMF could override the messages with "Shared" with "Global"

Client-side support
Keep track of origin in the client-side registry of ResourceLoader. The internal registry will have an origin-key for each module (just like it has for groups). The execute-function will use the right loader-script as needed. And split up requests per module source.

Support for this needs to be implemented in MediaWiki core.

GadgetRepo

 * Abstract GadgetRepo, implemented as LocalGadgetRepo, ForeignDBGadgetRepo and ForeignApiGadgetRepo


 * Query to mw_gadgets table or api depending on Foreign-method
 * Registered in the start-up module just like the local ones.

ApiQueryGadgets

 * Filter by either gd_id (string) or gd_shared (boolean)
 * Allow multiple ids, if no ids given: Return all
 * Properties:
 * id (gd_id)
 * metadata (gd_blob)
 * title (parse: plain; uselang)
 * title-msgkey
 * description (parse: html; uselang)
 * description-msgkey

See also the format of the metadata

ApiQueryGadgetCategories

 * Category keys
 * Category display title in user language
 * Category member count

= Load queue enhancements =

Use-cases
Modules that style the output from the server directly (ie. not content generated by JavaScript) should load before the body is outputted. This to avoid a "flash of unstyled content". By loading it from the top the styling will be parsed by the browser when the content is parsed, thus no "flash of unstyled content" will appear. Modules that insert, modify or remove elements on the page need to be loaded from the top of the body in a "document ready" wrapper. Otherwise the user could see a short "jump" in the page where something is added, removed or otherwise changed. By loading these from the top they will be able to queue themselves to start "changing" the content as soon as it's loaded.
 * Content styling
 * Content modification

Implementation proposal
A simple top-loading queue was already implemented in MediaWiki 1.18, however to apply these enhancements we need to do some research to the behavior in different browsers of dynamically inserted script and style tags when: And we should research wether or not script tags in the body block the document-ready event, ininitial testing suggests it does in which case we should manually trigger jQuery's document ready event at the last line before  for performance reasons.
 * inserted into the head from the head
 * inserted into the top of the body from the body
 * appended to the body after document ready (this we know works properly)

We need to know whether or not they are asynchronous, load in the right order and block browser execution. Goals (or blocker if not possible)
 * We need requests in head to block/postpone further browser execution
 * We need asynchronous and non-blocking requests in the top of the body that download, parse and execute while browser receives and parses the document.
 * We need to trigger document-ready event at the bottom of the body and not

= Noscript stylesheets =

Use-cases

 * todo

Statistics

 * http://krinkle-tools.grizzdesign.nl/noscript-head/ (Blue JS | Green JS-off)

More numbers: RL2-noscript-head
 * IE6 and higher: OK
 * FF 2 and higher: OK
 * Opera 10 and higher: NO
 * Safari 3 and higher: OK
 * Chrome 10 and higher: OK