ResourceLoader/Version 2 Design Specification

From MediaWiki.org
Jump to: navigation, search
shortcuts:
RL2
RL/V2SPEC

Discussion about already or not-yet presented requirements and/or feature requests should take place on the talk page.

While collecting and interpreting requirements and feature requests from the Wikimedia Foundation staff and developer community some of these requirements were understood to be either mutually exclusive or out of scope of the resources dedicated to this project or the scale of the system. For this reason, this specification is the result of negotiation between multiple parties.

Etherpad: RL2


Module sources[edit | edit source]

Tracking: bug 30022

Gadget Manager[edit | edit source]

Tracking: bug 29398

Use-cases[edit | edit source]

  • Usability
    By having a visual interface, gadget managers will no longer have to learn and cope with yet another syntax (as used on MediaWiki:Gadgets-definition).
  • Tracking
    The Special-page will be able to log events which will enable users to easily see who changed what and when. They will be able to get that information chronologically for all gadgets (What has been done in the last few days?), or for one particular gadget (What happened to Gadget Foo ?). This is handled through Special:RecentChanges/namespace=2302 and the history-page of individual Gadget definition pages.
  • Extendable
    Over the last year more and more options have been added to gadgets and resource loader modules. There's over a dozen module properties and already half a dozen Gadget settings (gadget title, description, and specific settings (required user rights, enable by default), module scripts, styles, dependencies, messages, position and more..). Having them in a nicely designed form with responsive javascript interactions and autocompletion will make it easier to understand and extendable without complicating the "Gadget syntax" every time (this syntax will cease to exist).
  • Sharing
    Through this Gadget manager interface, the process of 'Adding' (or 'Creating') a gadget will be as simple as a click. Aside from writing one from scratch another goal of the Gadget Manager is to enable easy exporting and importing of Gadget properties from one wiki to another. Because the gadget definition will have a specified JSON format, it can easily be exported to a different wiki. Special:Export can be used to export the entire gadget and everything it needs (the definition, the title and description messages, the js/css pages and any other interface messages it may use). Note however that the process of exporting/importing a gadget is, although a lot easier in version 2, no longer intended to happen as much within the Wikimedia farm due to the new Shared gadgets feature, which no longer requires the local wikis to "copy" or "reroute" to a gadget from a central place. There will be an InstantCommons-like feature which allows third party wikis to easily flip a switch and get all the popular gadgets, that are actively developed within Wikimedia, onto their own wikis.

Implementation proposal[edit | edit source]

Storage of Gadget definition[edit | edit source]

The definition of each gadget will be stored in a new MediaWiki database table, mw_gadgets.

mysql> DESCRIBE mw_gadgets;
+--------------+---------------------+------+-----+---------+-------+
| Field        | Type                | Null | Key | Default | Extra |
+--------------+---------------------+------+-----+---------+-------+
| gd_id        | varbinary(255)      | NO   | PRI | NULL    |       |
| gd_blob      | mediumblob          | NO   |     | NULL    |       |
| gd_shared    | tinyint(1) unsigned | NO   |     | 0       |       |
+--------------+---------------------+------+-----+---------+-------+
  • gd_id: Unique id of the gadget. Lowercase [a-z], 0-9 and dashes allowed. Cannot be changed after creation.
  • gd_blob: JSON string containing the module properties and gadget settings
  • gd_shared: Whether this is a shared gadget. Gadgets with gd_shared=1 are exposed through the Repo interface.
    • "global" is somewhat wiki-farm specific. Perhaps "shared" is more appropriate ?
      • Yes, we basically agreed on this a while ago. Hereby updated to avoid further confusion.
{
    "settings": {
        // The key of the category this gadget belongs to.
        // Interface message is "gadgetcategory-{category}"
        "category": "maintenance-tools",
        // The rights required to be able to enable/load this gadget
        "rights": ["delete", "block"],
        // Whether this gadget is enabled by default
        "default": true,
        // Whether or not this gadget is hidden from preferences
        "hidden": false,
        // Whether this gadget is shared
        "shared": true
    },
    // GadgetResourceLoaderModule
    "module": {
        // Scripts and styles are in NS_GADGET
        "scripts": ["foobar.js"],
        "styles": ["foobar.css"],
        "dependencies": ["jquery.ui.tabs", "gadget.awesome"],
        "messages": ["foobar-welcome", "foo-bye", "recentchanges"],
        "position": "top"
    }
}

The definitions will be saved to a wiki page for versioning, history, watching etc.

Gadget resources[edit | edit source]

The resources will stay where they are: As pages with revisions in the wiki database. The difference intended to be made as part of the ResourceLoader 2 project is moving these out of the prefixed "Gadget-" scope in the MediaWiki:-namespace (which is intended for messages, not actual wiki content (let alone executable resources). Instead move them to a new Gadget:-namespace only editable by users with the editgadget right.

Messages[edit | edit source]

The title of the gadget and the description associated with will stay in the MediaWiki:-namespace. The main reason for this is to enable easy localization and centralization of the description.

Aside from the title (eg. MediaWiki:Gadget-foobar) and the description (eg. MediaWiki:Gadget-foobar-description) a gadget can have a number of custom made messages. These are handled by the existing system present in MediaWiki which is suited for this already and are loaded through the messages framework in ResourceLoader (associated at module registration and passed to the browser through mw.loader.implement / mw.messages.set).

User rights[edit | edit source]

To keep track of what user rights are talked about in the specification and task management, a short summary

gadgets-edit
Edit gadget resources in the Gadget:-namespace
gadgets-definition-create
Creating pages directly in the Gadget definition:-namespace.
gadgets-definition-edit
Edit in the Gadget definition:-namespace.
gadgets-definition-delete
Deleting pages directly in the Gadget definition:-namespace.

Special:Gadgets[edit | edit source]

Overview of all gadgets
Special:Gadgets: title=Special:Gadgets [& action=view]
Permalink to single gadget
Special:Gadgets/<id>: title=Special:Gadgets & gadget=<id> [& action=view]
Export-action of a gadget
Special:Gadgets/<id>/export: title=Special:Gadgets & gadget=<id> & action=export
  • Creation, modification, and deletion are possible without JavaScript through the regular WikiPage actions. However the visual editor requires JavaScript.

Related bugs[edit | edit source]

  • (bug 27561) Register modules client side
  • (bug 27771) Mechanism for loading site wide JavaScript libraries
  • (bug 27281) Add support in the front-end for loading wiki pages as resources

Designs[edit | edit source]

Shared gadgets[edit | edit source]

One of the most wanted features for gadgets is shared gadgets. Users want to share gadgets, reach a wider audience, and avoid outdated and redundant copies at all costs. By having a "Gadget Repository" this problem will be solved. When a gadget is created on a "gadget repository" wiki (and marked "shared") it will show up on all wikis who have their ForeignGadgetRepo set to that wiki.

For Wikimedia wikis this will most likely be set as default for all wikis (just like Wikimedia Commons is set as ForeignFileRepo). They would probably be stored on mediawiki.org (not Meta-Wiki as they're not Wikimedia-specific). As part of this project (or later) we may create a convenience configuration (like "InstantCommons") to easily allow third party wikis to use mediawiki.org as their foreign repository for gadgets.

Use-cases[edit | edit source]

  • Sharing
    No more need to duplicate, distribute, update etc. gadgets will be centrally organized and developed.
  • Translation
    Due to the nature of ResourceLoader it is simple to get interface messages in the user language. These messages would be fetched from the ForeignGadgetRepo, which means all messages will be central rather than on each wiki separately. Thanks to the fallback principle and nightly LocaliationUpdate, gadgets on mediawiki.org could even be translated on translatewiki.net
  • Less duplication / Re-use of code:
  • Use-cases for dependancy model:
    • Loading a shared gadget from a foreign repository
    • Loading a gadget on a repository-wiki that depends on other gadget from that wiki
    • Loading a gadget on a local wiki that depends on a gadget from a foreign repository
    • mw.loader.load( 'some gadget from repo' )
  • Use-cases for repository model:
    • Have a local gadget on the repo-wiki that is not shared and has the ability to be hidden and/or enabled by default
    • Have a gadget on the repo-wiki that is shared with foreign wikis. The hidden property will be inherited by foreign wikis, the "enabled by default"-property will not.
    • Ability for a foreign wiki to enable a gadget from the repo-wiki by default
      • (by creating a local gadget with no scripts, a dependency on the remote gadget and enabling it by default)

Implementation[edit | edit source]

Preferences[edit | edit source]

  • Preferences will either get:
    • 1 extra tab ("Gadgets") showing both local and foreign gadgets mixed, and grouped by 'category' with some kind of indication which are foreign and which aren't.
    • 2 extra tabs ("Gadgets", "Shared gadgets") both have their own sections.
      • Why? Perhaps end users should not be aware of it.
      • Why not? Clearer separation, feels better. We need more motivation for "2 tabs" to remain an option.
      • WMF could override the messages with "Shared" with "Global"

Client-side support[edit | edit source]

Keep track of origin in the client-side registry of ResourceLoader. The internal registry will have an origin-key for each module (just like it has for groups). The execute-function will use the right loader-script as needed. And split up requests per module source.

mw.loader.addSource( 'mediawikiwiki', { loadScript: 'http://mediawiki.org/w/load.php' } );

Support for this needs to be implemented in MediaWiki core.


Server-side support[edit | edit source]

GadgetRepo[edit | edit source]

Abstract GadgetRepo, implemented as LocalGadgetRepo, ForeignDBGadgetRepo and ForeignApiGadgetRepo
  • Query to mw_gadgets table or api depending on Foreign-method
  • Registered in the start-up module just like the local ones.

ApiQueryGadgets[edit | edit source]

  • Filter by either gd_id (string) or gd_shared (boolean)
  • Allow multiple ids, if no ids given: Return all
  • Properties:
    • id (gd_id)
    • metadata (gd_blob)
    • title (parse: plain; uselang)
    • title-msgkey
    • description (parse: html; uselang)
    • description-msgkey

See also the format of the metadata

ApiQueryGadgetCategories[edit | edit source]

  • Category keys
  • Category display title in user language
  • Category member count


Load queue enhancements[edit | edit source]

Use-cases[edit | edit source]

  • Content styling
    Modules that style the output from the server directly (ie. not content generated by JavaScript) should load before the body is outputted. This to avoid a "flash of unstyled content". By loading it from the top the styling will be parsed by the browser when the content is parsed, thus no "flash of unstyled content" will appear.
  • Content modification
    Modules that insert, modify or remove elements on the page need to be loaded from the top of the body in a "document ready" wrapper. Otherwise the user could see a short "jump" in the page where something is added, removed or otherwise changed. By loading these from the top they will be able to queue themselves to start "changing" the content as soon as it's loaded.

Implementation proposal[edit | edit source]

A simple top-loading queue was already implemented in MediaWiki 1.18, however to we plan to take it further with more enhancements. For that we need to do some research to the behavior in different browsers of dynamically inserted script and style tags when:

  • inserted into the head from the head
  • inserted into the top of the body from the body
  • appended to the body after document ready (this we know works properly)

And we should research wether or not script tags in the body block the document-ready event, ininitial testing suggests it does in which case we should manually trigger jQuery's document ready event at the last line before </body> for performance reasons.

We need to know whether or not they are asynchronous, load in the right order and block browser execution. Goals (or blocker if not possible)

  • We need requests in head to block/postpone further browser execution
  • We need asynchronous and non-blocking requests in the top of the body that download, parse and execute while browser receives and parses the document.
  • We need to trigger document-ready event at the bottom of the body and not

Noscript stylesheets[edit | edit source]

Use-cases[edit | edit source]

  • todo

Statistics[edit | edit source]

  • IE6 and higher: OK
  • FF 2 and higher: OK
  • Opera 10 and higher: NO
  • Safari 3 and higher: OK
  • Chrome 10 and higher: OK

More numbers: etherpad:RL2-noscript-head

See also[edit | edit source]

ResourceLoader
Documentation Features · Vocabulary · Migration guide (users) · Migration guide (developers) · Developing with ResourceLoader · Default modules · Mobile support
Project information Status updates · Version 1 Design Specification (tasks) · Version 2 Design Specification (tasks) · Requirements
Other JavaScript Deprecations