ResourceLoader/Package modules

From MediaWiki.org
Jump to navigation Jump to search
shortcut: RL/PKG

A package module is a type of ResourceLoader module that can consist of multiple script files or data exports, accessible to the module's entry point via require() .

To enable this feature in a module, use the 'packageFiles' property of a module descriptor. Traditional modules specify multiple script files with the 'scripts' property, which blindly concatenates them as if they were a single script without ability to separately access them client-side.

Why[edit]

Node.js compatibility[edit]

This feature was originally conceived as a way to make code written for Node.js easier to integrate with ResourceLoader. Libraries that contain multiple files that require() each other, could be registered as a ResourceLoader module without modification.

Private export and import[edit]

The ability to export values from individual files and import them within a module, has also proved useful for modernising JS code in MediaWiki itself, by no longer needing to attach all classes to a public and global object.

Data and config bundling[edit]

The ability for a module to have multiple "files" that remain individually addressable, has also opened the door to support bundling of JSON files and virtual data exports from PHP (as JSON). The proposal to embed config variables in ResourceLoader modules, thus ended up being implemented as virtual files in a package module.

Before this feature existed, embedding config and data in modules was also possible, but required writing your own subclass of ResourceLoaderFileModule, an obscure technique that most developers didn't know about or felt uncomfortable using. Most instances of this technique in MediaWiki core have now been ported to use a virtual file in a package module instead.

A more common technique for exporting values from PHP was to export them with $out->addJsConfigVars()[1] or through the ResourceLoaderGetConfigVars hook, as mw.config keys. This is problematic for performance because:

  • Startup module config vars were exported on all page views for all users (wastes bandwidth cost), and need to be parsed and processed before your actual modules can begin to download (delays interaction), and were only cached for a short time (frequent re-download).
  • OutputPage config vars are in the <head> and block downloading of article text (delays visual rendering), and have to be processed before actual modules can start to begin downloading (delays interaction).

Exports part of a package module, on the other hand:

  • Only downloaded if they are needed, together with the code (saves bandwidth).
  • Only downloaded when they are needed (avoids rendering or interaction delays).
  • Benefit from longer caching times (only re-downloaded if they change, as part of the module).

How it works[edit]

A package file can either be JavaScript code (script), or a JSON blob (data). A file's type is inferred based on its extension (.js or .json). Files can be real files from disk, or dynamic files generated by code. Dynamic files are commonly used to export the values of configuration settings or other data from the server, and in those cases they're typically named config.json or data.json.

Every module has one main file. This is the first file listed, unless another file is explicitly designated as the main file. When the module loads, only the code in the main file is executed. The code in the other files is not executed unless and until it is invoked using require().

To invoke a non-main file, use require( './foo.js' ). The path must start with ./ or ../ to indicate that it's a file in the same module (as opposed to the name of another module), and the path must be relative to the current file. The file suffix .js is required. The return value of require( './foo.js' ) is the value that foo.js assigns to module.exports.

Module definition[edit]

Files can be specified in the 'packageFiles' property in the following ways:

  • 'foo/bar.js' will load a file from the filesystem
  • [ 'name' => 'blah.js', 'file' => 'foo/bar.js' ] will load the file foo/bar.js from the file system, but its alias will be blah.js (meaning it will be known under that name that for the purposes of require())
  • [ 'name' => 'foo/bar.js', 'main' => true ] explicitly designates this file as the main file (the file that is executed first). If no main file is explicitly designated, the first file is the main file
  • [ 'name' => 'blah.js', 'content' => 'console.log( "Hello world" );' ] defines a dynamic file called blah.js, with the specified contents
  • [ 'name' => 'blah.json', 'content' => [ 'hello' => 'world' ] ] defines a dynamic file whose contents are {"hello":"world"}
  • [ 'name' => 'blah.json', 'callback' => function ( ResourceLoaderContext $context, Config $config, array $callbackParams ) { /* ... */ } ] defines a dynamic file whose contents are the result of executing the callback
    • You can also specify a static function as a callback: 'callback' => [ 'MyExtensionHooks', 'generateBlahJson' ] will call MyExtensionHooks::generateBlahJson()
    • For JS files, the callback should return a string. For JSON files, it can return anything that's JSON-serializable (typically an associative array)
    • The callback is executed in the context of a load.php request, so it doesn't know which user is logged in or which page is being viewed; instead, the result is computed once and reused across different users and pages. If you attempt to use the current user in your callback, you will likely get a Sessions are disabled for this entry point error. You should not use RequestContext in this callback. The callback takes three arguments: the ResourceLoaderContext $context object, a Config $config object, and an optional $callbackParams array. The value of callbackParams specified in packageFiles will be passed as $callbackParams.
  • [ 'name' => 'blah.json', 'config' => [ 'LegalTitleChars', 'IllegalFileChars' ] ] defines a dynamic file whose contents are {"LegalTitleChars": "...", "IllegalFileChars": "..."}. Note that this syntax uses config setting names as understood by Config::get() (e.g. 'LegalTitleChars'), not the names of global variables (e.g. 'wgLegalTitleChars')
    • You can use aliases to export configuration variables under different names: 'config' => [ 'NaughtyCharacters' => 'IllegalFileChars' ] will result in {"NaughtyCharacters": "value of $wgIllegalFileChars"}
    • If you need to do more advanced manipulation of config variables, use a callback as described above. In the callback, you can use e.g. $context->getConfig()->get( 'IllegalFileChars' ) to get the value of a config setting.

Base path[edit]

Most package modules set 'localBasePath' to the common directory prefix of the files. This is done for convenience (not having to write resources/src/whatever/ over and over), and to make dynamic files easier to deal with. Without a base path, a dynamic file called config.json would have to be accessed using require( '../../../config.json' ), or it would have to be named resources/src/whatever/config.json so it could be accessed using require( './config.json' ). With a base path, you get the best of both worlds.

Note that, if you set localBasePath, you will also have to set remoteBasePath (for core) or remoteExtPath (for extensions) to match.

Incompatibility with 'scripts'[edit]

If a module uses the 'packageFiles' property, it cannot use the 'scripts' property. Defining a module that uses both properties will throw an exception.

Package modules are also incompatible with 'languageScripts' and 'skinScripts'. Defining these properties won't throw an exception, but they will be ignored. A way of defining language/skin-specific script files for package modules has not yet been developed (and this blocks porting ResourceLoaderLanguageDataModule).

Example uses in real code[edit]

Basic example/illustration[edit]

Module definition (core)[edit]

In Resources.php:

    'mything' => [
        // Make all paths relative to resources/src/mything
        'localBasePath' => "$IP/resources/src/mything",
        'remoteBasePath' => "$wgResourceBasePath/resources/src/mything",
        'packageFiles' => [
            'init.js', // Main file because it's listed first
            'thinglib/index.js',
            'thinglib/formatter.js',
            [ 'name' => 'config.json', 'config' => [ 'UseLongThingFormat' ] ],
            [ 'name' => 'data.json', 'callback' => function ( ResourceLoaderContext $context, Config $config, array $callbackParams ) {
                $language = Language::factory( $context->getLanguage() );
                return [
                    'monthNames' => $language->getMonthNamesArray();
                ];
            } ],
        ],
    ],

Module definition (extension)[edit]

In extension.json:

    "mything": {
        "localBasePath": "modules/mything",
        "remoteExtPath": "MyExtension/modules/mything",
        "packageFiles": [
            "init.js",
            "thinglib/index.js",
            "thinglib/formatter.js",
            {
                "name": "config.json", 
                "config": [ "UseLongThingFormat" ]
            },
            {
                "name": "data.json",
                "callback": "MyExtensionHooks::getMyThingData",
                "callbackParams": {"key1": "value1", "key2": => "value2"}
            }
        ]
    }

In MyExtensionHooks.php:

class MyExtensionHooks {
    // ...
    public static function getMyThingData( ResourceLoaderContext $context, Config $config, array $callbackParams ) {
        $language = Language::factory( $context->getLanguage() );
        return [
            'monthNames' => $language->getMonthNamesArray();
        ];
    }
}

JavaScript[edit]

In init.js:

var thinglib = require( './thinglib/index.js' ),
    monthNames = require( './data.json' ).monthNames,
    config = require( './config.json' ),
    formatter = new thinglib.Formatter( { months: monthNames } );

if ( config.UseLongThingFormat ) {
    formatter.format( /* something */ );
} else {
    // do something else
}

In thinglib/index.js:

var thinglib = {
    Formatter: require( './formatter.js' ) // note this path is relative to the file we're in
    /* otherthing: require( './otherthing.js' ) */
    /* etc */
};

module.exports = thinglib;

In thinglib/formatter.js:

function Formatter( config ) {
    // ...
}

Formatter.prototype.format = function ( /* ... */ ) {
    // ...
};

module.exports = Formatter;

Debugging[edit]

In debug mode (?debug=true), traditional modules load each of their files in a separate request. This feature is partly unavailable to package modules. In debug mode, a package module is still loaded in its own request and without minification, but the files within the same module are bundled together in one request. If you're having trouble finding a file or a piece of code using your browser's developer tools (for example, to place a breakpoint somewhere), you can use the following hack.

Suppose you're trying to find the file named thinglib/formatter.js in the module named mything from the example above, and want to place a breakpoint in it. You can do that as follows:

  1. In your browser console, run mw.loader.moduleRegistry['mything'].script.files['thinglib/formatter.js']. This will print something like f(require,module){...} (in Chrome) or function js() (in Firefox).
  2. In Chrome, this will print something like f(require,module){...}; click on this text. In Firefox, this will print function js() followed by an arrow pointing up and to the right to three horizontal lines; click this icon.
  3. This will take you to the "Scripts" panel . If you see minified code, click the "pretty print" button (at the bottom left of the code pane, the icon looks like {}).
  4. In Chrome, you will now be looking at a line of code highlighted in yellow that looks like "thinglib/formatter.js": function(require,module) {. The code of thinglib/formatter.js begins below this line. In Firefox, you won't be taken to this place automatically, you'll have to search for thinglib/formatter.js':to find it[2].

You can also access the exports of a file (i.e. the value it assigns to module.exports and returned by require('./formatter.js')) by running mw.loader.moduleRegistry['mything'].packageExports['thinglib/formatter.js'] in the console. Often this is a function or an object containing functions, which you can then access using steps 2-4 above.

See also[edit]

Footnotes[edit]

  1. Note that vars that depend on the request context (e.g. the user or the page title) can't be moved to a package module. They can only be exported with OutputPage::addJsConfigVars().
  2. Note that Firefox's pretty printer changes double quotes (") to single quotes ('), so you will have to search for thinglib/formatter.js': with a single quote, even though there's a double quote in the original code generated by ResourceLoader.