User:✓/RL 3.0

This page is something like a general overview of my vision of mediawikis javascript infrastructure. It's a working sheet, feel free to contribute and/or discuss.

I've also tried to link nearly every related bugzilla entry (lots of them with contribs of mine, some already closed or even wontfixed) that I could find. I'm pleased to get more links to discussions.

Symbol support vote.svg Pros
The ResourceLoader is a cool thing, no question, and has brought many improvements (see also requirements):
 * it's easy to use for extension developers
 * it has nice features like CSS Parsing and (debugable) minification
 * it encourages the use of JS namespacing

Symbol oppose vote.svg Contras
But there are also lots of drawbacks, especially when used for user scripts - of course the high traffic of anonymous users is handled already. This list may extend the task list of ResourceLoader/Version 2 Design Specification (better: Version 2.1 todo):
 * no way to make full use of ResourceLoader for user-level scripts (i.e. a replacement for the inefficient action=raw) (bug 34958 (Allow creation of ResourceLoader wiki-modules on a user level) )
 * The current workaround is manual dependency resolving, using  (from de:Benutzer:Schnark/js/imagepopups.js)
 * or undocumented  as in User:PerfektesChaos/js/editToolStrIns/debug.js?
 * no support for internationalisation/customisation, the former only when it's a core/extension-added script. See also
 * no support for packaging userpages
 * the load/ready events for modules are not public. Many scripts could use such hooks: de:Benutzer:Schnark/js/wikieditor.js needs to check for existence of, the extra-editbuttons-gadgets uses very queer magic to execute in the right time, a script that extends the functionality of enhancedRecentChanges and expandedWatchList... just an example how I tried to code the last one:
 * Bug 28995 - Public interface for getting loader state of a ResourcerLoader module (mine :-) was a step in the right direction, but not far enough I had to find out.


 * see Bug 27561 - Register modules client side for "create custom wikipage-modules" - thats what my vision is all about :-)
 * would also solve Bug 29001 (Prevent recursive loading of resources) / Bug 26814, since no URLs are loaded any more

deliberations

 * Most scripts are not even needed when loaded. Many begin with something like
 * But still, the whole script is loaded for each request. So two things for better code:
 * define only one function, not two or three event listeners. Don't define a function at all, just write a script which is automatically wrapped in a function.
 * define the rest somwhere else: dependencies, start, requirements


 * contains too much data. This should be split in  and , next to  . Getting a more dynamic page (inline editing, ajax page loader and other stuff), the static site stuff must consist on, while page, "view state" and even user could change
 * we should start providing general hooks for these changes, to prevent uncontrolled proliferation
 * Some script like libraries need to be loaded in the beginning, i.e. usually head. Today that's not easy to control for custom/user libs → bug 27771 (Mechanism for loading site wide JavaScript libraries)
 * the registration array of modules is already very long, many stay unused. If we're going to enable user modules and external modules, we won't be able to provide them all. seems to be reverting bug 25085 (Loader scripts should have access to server-side registration info)
 * What are we going to do with MediaWiki:Common.js/.css, MediaWiki: .js/.css? mw.loader.using will get inefficient, when the module isn't pre-registered. See Bug 30358 (Make client-side loader load site and user modules)
 * It might be needed to queue load requests. The calls to using or load might come from different sources, but when all know there's a order in it they could let the last statement call the go. see Bug 27415 (Implement mw.loader.wait and thus support multiple mw.loader.go's)
 * I have no idea how external modules would work. V2 Design says something about ForeignApiGadgetRepo. Might get crowded when we put all userscripts in :-) But I guess at WMF this would be centralised at bits. Elsewhile, there would need to be an interwiki-api for the module list, the usage statistics and message-receiving.
 * A remote-stuff loading bug is 30022 (Add support for (custom) loadScript sources to ResourceLoader)

solution?
To come to a conclusion: user scripts and gadgets don't differ really from each other. They have the same aim, only gadgets are more popular. Therefore, gadgets may be presented in a selective list, while userscripts need to be searched for and added dynamically by typing their name into the preferences form.

So I'd propose to set up the following infrastructure:
 * We have a Specialpage "DefineResourceLoaderModule", needing no more rights than autoconfirmed.
 * you can define modules there, prefixed with the namespace "user. ."
 * If you have "gadgeteditor" rights (usually in group sysop, maybe an extra group), you can define modules with the prefix "gadget."
 * not sure whether to kick MediaWiki:gadgets-definition or not and show just all gadget modules. I think it's better to remove all the RL-options from it, and add the possibility to define internal and external gadget-modules. I can see no need for a "Gadget Manager", as proposed, in my vision setup.
 * All module settings are public, editable only by those who have the rights ("own" and "edituserjscss" for user modules, "gadgeteditor" for gadgets).
 * a module definition includes:
 * name
 * that identifier with the prefix (shouldn't change?)
 * short description
 * If it's a gadget, that description should be saved in MediaWiki Namespace (as usual, with the "name"), where it can be localised (rights "editinterface", "gadgettranslator"?). Userscripts dont get translations?
 * link to a help page
 * For userscripts, this defaults to the script/stylesheet subpage without suffix.
 * What about user js/css which have subpages/submodules? Should "User:Name/Foo.js/bar.js" be documented at "User:Name/Foo.js/bar" (no back link to "User:Name/Foo" on contentSub), "User:Name/Foo/bar" (backlink to "User:Foo") or "User:Name/Foo" (central documentation)?
 * There is no User:Foo.js. A module "[iw-prefix]user.foo.bar", located usually at User:Foo/bar.js, is considered to be documented at User:Foo/bar. A sub-script should be located at User:Foo/bar/x.js (or User:Foo/bar.js/x.js, sometimes? — you're free to choose the scheme) may be named "user.foo.bar.x"-module, documented at User:Foo/bar[.js]/x. You can change all of these references, set up redirects to centralized docs or do anything else. For example scripts located at User:Foo/js/x.js and User:Foo/js/y.js may be loaded together as module "user.foo.z-framework.init", and be documented at User:Foo/my_library/js or something. --Bergi (talk) 16:46, 2 March 2012 (UTC)
 * activation conditions
 * a set of conditions when to activate the module (see below), might be just a js function, or a boolean variable/expression
 * list of scripts and stylesheets
 * Not to be forgotten :-) Their default names are the module name plus ".js"/".css" suffix, in User- respectively MediaWiki namespace.
 * each script has a start/position: in which loading state the function should be executed:
 * head (yea, please let me define that explicitly - bug 27488 (Implement 'top' position option for ResourceLoader modules) / bug 34958 (Allow creation of ResourceLoader wiki-modules on a user level)  for "Allow user scripts to load from position top")
 * bottom
 * onDOMReady - no need for another jQuery(document).ready-bind, do it in RL
 * onSomethingelse
 * also stylesheets need position information, like "not async". Most stylesheets are to be added for different conditions, yes, but not to be loaded dynamic or inside a json file (sounds crazy, but is done?!). Instead they need to be added to head, where they are loaded (combinded and minimized, of course) to influence static pages. Added serverside, to bring styling options with RL modules to noscript users - don't forget them!
 * modules to be prefetched
 * Some modules may be loaded just in need, like jquery.gui. They are not a dependency of the specified module though, but they're should load info should be registrated.
 * translation
 * and of course, #3 type of resources; not applicable today. See also
 * you can add native systemmessages, maybe even with wildcards - don't use the MediaWiki ns for tool-specific messages. These messages might get loaded from the requester's wiki, even if the module is used as an external one.
 * add the name[s] of [a] page[s], whose iso-named subpages contain a (JSON) key-value-map. These pagenames must be user subpages or MediaWiki pages for gadgets. The data-pages are editable both with "gadgeteditor", "gadgettranslator" (and "editinterface"/"sysop") rights. There might be a nice frontend for declaring new or deprecating message names and translating, instead of editing json files.
 * To be exakt: for each of the translation definition(-set)s you should be able to choose whether the localisation is done on "contentlang" or "uselang". See also 29873 (Add support for content language for messages in ResourceLoader) → 25349 (Resource loader should allow loading messages in nonstandard ways)
 * each of the so-defined modules will build a resource-loader-group. It may be merged to another group when the enhance-other-module load condition is set.
 * How to output that? When a page is requested, ResourceLoader will get all addModules and user-activated modules, resolve their dependencies and save that list. On the flush of the output queue, RL will write a script containing the registration (with group, version, depencies, conditions), topmodules and the load call.
 * It might happen, that using or other dependency methods don't find a requested module registered. So then they will complete their work, then file a request with all unknown modules they found. The answer will include the registrations (just as the modules were added in the beginning), after that the methods can go on checking the conditions and eventually load the modules. Will that extra request get caching problems?


 * As said above, all gadgets are available in the respective section of Special:Preferences as a list, with their description, just to select with a checkbox - as already today. Then there is a pref section "custom modules". Here you can add any valid module names - own userscriptmodules (these already listed?), foreign usermodules, external (interwiki) usermodules and gadgets. A nice autocomplete, and when valid the description shows up :-)
 * The old /common.js, / .js, /common.css and / .css might work on. Or we autoconvert them into user modules, being loaded on-skin and in-head.
 * There should be some statistics, if not even public user settings about module usage. Today it is desired to add a out-commented wikilink behind every importScript call, give the script author a) an indication how widespread it is used and b) the possibility to inform users about breaking changes (especially in customisation). To publish the number of uses should be no problem, and we could offer a select for user modules (X) disable chance to get informed show my usage to script author[s]  I'm proud of using this, show it everybody

The load and execution conditions may contain: → or simply define a js function directly instead of the last three points, as any gui-definable logic won't be complex enough for some things. Otherwise a js function would be built out of that gui-defined logic.
 * nothing, to indicate the module is to be loaded always, or:
 * just a pointer to another module which we want to enhance, i.e. load alongside when it gets loaded
 * a flag whether the module is allowed to load on "secure" pages like Preferences or Login →Bug 10005 (User modules not loaded on pages that disallow module origin "ORIGIN_CORE_INDIVIDUAL") . At least, I think, with today's JS it would be possible to completely simulate such pages anyway, so we might not need that any more
 * the dependencies to be fulfilled before execution (handle onReady as a dependency?)
 * the usual $wgPageState: wgPageName, wgSkin, wgCanonicalSpecialPageName, wgAction, wgNamespaceId etc
 * user.options user.customisation, I mean - options sounds useless
 * user.rights - interesting when I dynamically switch to my bot account

You might want to have a look at Schnarks jsmodules, a nice workaround for RL. In de:Benutzer:Schnark/js/modules.js you find the module definitions: version, link, description, dependencies, and some load condition functions.

customisation
I'm the sort of guy who wants to be able to change everything. From different toolbar icons up to a fully converted skin, it should be 1 script, many options. A really great example is User:PerfektesChaos/js/editToolStrIns. It does not only have everything you want, it runs into any problems you could run in (and survives :-).

I figured out that there are two directions: internationalisation/localisation and generalisation/customisation. You also have two types of variables in your script: messages and options. Sometimes, it might happen that especially external, imported messages need an site-scoped "update". I'm not sure if we could implement that with the same translation mechanism - any onwiki-defintion would be another module, maybe with dependencies? -, but there must to be a way to integrate some customised messages. At last, users would like to set up a script/module to customise messages on a per-user-basis; maybe not even language-specific but for all translations.
 * Messages
 * come from wfGetMessage respectively api.php?action=query&meta=allmessages of the current wiki, if they are system messages; which means that they are automatically fall-backed (unfortunately first from project to MW-default, then from bar to de - I'm not really lucky with that, nor anyone else is) and can be edited in site (wiki) scope. But modules may also store tool-specific messages in their own structures (described above). I'd propose them to be simple, complete JSON files, but when we allowed partial translation with fallbacks that would need an extra API / RL feature.

If the options were just one simple object, you might call  on it, done. If we had subobjects, we might use something like, as this doesn't overwrite object properties. But still it isn't suitable for maps (key-value-pairs), where we might want to remove keys but hold their values, in case the keys are added again by a more specific customisation. Experienced scripters would have made the value Object one setting (to merge in), and the keys another setting, an (ordered!) Array to overwrite/manipulate.
 * Options
 * are any other values that do not need to be translated. That could make it much easier, but it isn't, because options usually are complex objects. The default options can be hardcoded into the module logic, if the script is well-modularized they are exposed as a config object. The situation is quite the same, both at wiki and user scope there are settings to be integrated. But exactly that's what makes it difficult: integrate, not overwrite (as like for messages).

These config manipulations and language-fallback operations have to be handled by every script that does more than serving a library, and is imho the reason why scripts are not translated and shared. Particularly a config object might have more than 3 different setting integrations, e.g. a foreign script is translated to a different wiki, adapted by User:A, improved by User:B and specialised by User:C. And it gets really heavy if the language affects the options...

Especially because RessourceLoader makes the loading of different scripts (containing the setting options) asynchronous, a user script (most specific) can't be sure any more to be the last to manipulate a config object. Therefore, I think RL must include an interface to create such customisable configurations more easy, staying asynchronous and respect the priorities of different manipulations.