ResourceLoader/Package modules

From MediaWiki.org
Jump to navigation Jump to search
shortcut: RL/PKG

A package module is a type of ResourceLoader module that can consist of multiple script files or data exports, accessible to the module's entry point via require() .

To enable this feature in a module, use the 'packageFiles' property of a module descriptor. Traditional modules specify multiple script files with the 'scripts' property, which blindly concatenates them as if they were a single script without ability to separately access them client-side.

Why[edit]

Node.js compatibility[edit]

This feature was originally conceived as a way to make code written for Node.js easier to integrate with ResourceLoader. Libraries that contain multiple files that require() each other, could be registered as a ResourceLoader module without modification.

Private export and import[edit]

The ability to export values from individual files and import them within a module, has also proved useful for modernising JS code in MediaWiki itself, by no longer needing to attach all classes to a public and global object.

Data and config bundling[edit]

The ability for a module to have multiple "files" that remain individually addressable, has also opened the door to support bundling of JSON files and virtual data exports from PHP (as JSON). The proposal to embed config variables in ResourceLoader modules, thus ended up being implemented as virtual files in a package module.

Before this feature existed, embedding config and data in modules was also possible, but required writing your own subclass of ResourceLoaderFileModule, an obscure technique that most developers didn't know about or felt uncomfortable using. Most instances of this technique in MediaWiki core have now been ported to use a virtual file in a package module instead.

A more common technique for exporting values from PHP was to export them with $out->addJsConfigVars()[1] or through the ResourceLoaderGetConfigVars hook, as mw.config keys. This is problematic for performance because:

  • Startup module config vars were exported on all page views for all users (wastes bandwidth cost), and need to be parsed and processed before your actual modules can begin to download (delays interaction), and were only cached for a short time (frequent re-download).
  • OutputPage config vars are in the <head> and block downloading of article text (delays visual rendering), and have to be processed before actual modules can start to begin downloading (delays interaction).

Exports part of a package module, on the other hand:

  • Only downloaded if they are needed, together with the code (saves bandwidth).
  • Only downloaded when they are needed (avoids rendering or interaction delays).
  • Benefit from longer caching times (only re-downloaded if they change, as part of the module).

How it works[edit]

A package file can either be JavaScript code (script), or a JSON blob (data). A file's type is inferred based on its extension (.js or .json). Files can be real files from disk, or dynamic files generated by code. Dynamic files are commonly used to export the values of configuration settings or other data from the server, and in those cases they're typically named config.json or data.json.

Every module has one main file. This is the first file listed, unless another file is explicitly designated as the main file. When the module loads, only the code in the main file is executed. The code in the other files is not executed unless and until it is invoked using require().

To invoke a non-main file, use require( './foo.js' ). The path must start with ./ or ../ to indicate that it's a file in the same module (as opposed to the name of another module), and the path must be relative to the current file. The file suffix .js is required. The return value of require( './foo.js' ) is the value that foo.js assigns to module.exports.

Module definition[edit]

Files can be specified in the 'packageFiles' property in the following ways:

Static files[edit]

Static file from the filesystem[edit]
"packageFiles": [
  "foo/bar.js"
]

The name under which it is made available to require() can be overridden if needed, using a name alias:

"packageFiles": [
  { "name": "blah.js", "file": "foo/bar.js" }
]
Main module file[edit]

By default the first JavaScript file in the packageFiles array will be considered the module's main file (or "entry point").

To set this explicitly instead, use the main attribute. This indicates which of the files executes first and has the control to import other files:

"packageFiles": [
  { "name": "bar.js", "main": true }
]
Dynamic source file[edit]

The callback option can be used to dynamically decide which file to relate to an exported name. Read more about what callbacks can do in the sections further down.

"packageFiles": [
    {
      "name": "foo.js",
      "callback": "MyExtensionHooks::getFooFile"
    }
]
/** @return ResourceLoaderFilePath */
function getFooFile( ResourceLoaderContext $context, Config $config ) {
    $file = $config->get( 'FooSpecialEnabled' ) ? 'foo-special.js' : 'foo.js';
    return new ResourceLoaderFilePath( $file );
}

Virtual files[edit]

Custom content[edit]

Define a virtual JavaScript file, with the specified contents:

"packageFiles": [
    {
      "name": "bar.js",
      "content": "console.log( 'Hello world' );"
    }
]

Define a virtual JSON file, whose contents will serialised as JSON. For example the below would export {"hello": "world"}:

"packageFiles": [
    {
      "name": "bar.json",
      "content": { "hello": "world" }
    }
]

The same works from PHP as well (such as in MediaWiki core's Resources.php file, or from an extension hook):

'packageFiles' => [
	'name' => 'blar.json',
	'content' => [ 'hello' => 'world' ]
]
Generated content[edit]

Define a virtual file whose contents is generated by a callback. For JS files, the callback should return a string. For JSON files, it can return anything that's JSON-serializable (typically an associative array).

"packageFiles": [
    {
      "name": "blah.json",
      "callback": "MyExtensionHooks::generateBlah"
    }
]
/** @return array|string|int|bool Data for JSON */
function generateBlah( ResourceLoaderContext $context, Config $config ) {
    return /* ... */;
}

The callback is executed in the context of a load.php request and cached as part of that module, so it can't know which user is logged-in or which page is being viewed. instead, the result is computed once and re-used across different users and pages. If you did attempt to access a RequestContext or User in your callback, it would likely return Error: Sessions are disabled for this entry point .

The ResourceLoaderContext and Config do offer information that you can vary by, such as by site confguration, current skin, and current interface language.

The callback takes an optional $param, which is set to the value of a "callbackParam" key specified in packageFiles. This allows callbacks to be re-used for multiple purposes:

"packageFiles": [
    {
      "name": "foo.js",
      "callback": "MyExtensionHooks::generateFoo",
      "callbackParam": [ "A", "B", "C" ]
    }
]
/** @return string JavaScript code */
function generateFoo( ResourceLoaderContext $context, Config $config, $param ) {
    return /* ... */;
}

Config files[edit]

This is a shortcut for the common case of generating JSON that exports one or more MediaWiki configuration variables. The below defines virtual file whose contents are {"LegalTitleChars": "...", "IllegalFileChars": "..."}:

[
	'name' => 'blah.json',
	'config' => [ 'LegalTitleChars', 'IllegalFileChars' ]
]

Note that this syntax uses config setting names as understood by Config::get() (e.g. 'LegalTitleChars'), which are without the wg prefix that global variables use (e.g. 'wgLegalTitleChars').

You can also use aliases to export configuration variables under different names:

[
	'name' => 'blah.json',
	'config' => [ 'naughtyChars' => 'IllegalFileChars' ]
]

will result in {"naughtyChars": "value of $wgIllegalFileChars"}.

If you need to do more advanced manipulation of config variables, use a callback as described above. In the callback, you can use e.g. $config->get( 'IllegalFileChars' ) to get the value of a config setting.

Base path[edit]

Most package modules set 'localBasePath' to the common directory prefix of the files. This is done for convenience (not having to write resources/src/whatever/ over and over), and to make dynamic files easier to deal with. Without a base path, a dynamic file called config.json would have to be accessed using require( '../../../config.json' ), or it would have to be named resources/src/whatever/config.json so it could be accessed using require( './config.json' ). With a base path, you get the best of both worlds.

Note that, if you set localBasePath, you will also have to set remoteBasePath (for core) or remoteExtPath (for extensions) to match.

Incompatibility with 'scripts'[edit]

If a module uses the 'packageFiles' property, it cannot use the 'scripts' property. Defining a module that uses both properties will throw an exception.

Package modules are also incompatible with 'languageScripts' and 'skinScripts'. Defining these properties won't throw an exception, but they will be ignored. A way of defining language/skin-specific script files for package modules has not yet been developed (and this blocks porting ResourceLoaderLanguageDataModule).

Example uses in real code[edit]

Basic example/illustration[edit]

Module definition (core)[edit]

In Resources.php:

    'mything' => [
        // Make all paths relative to resources/src/mything
        'localBasePath' => "$IP/resources/src/mything",
        'remoteBasePath' => "$wgResourceBasePath/resources/src/mything",
        'packageFiles' => [
            'init.js', // Main file because it's listed first
            'thinglib/index.js',
            'thinglib/formatter.js',
            [ 'name' => 'config.json', 'config' => [ 'UseLongThingFormat' ] ],
            [ 'name' => 'data.json', 'callback' => function ( ResourceLoaderContext $context, Config $config, array $callbackParams ) {
                $language = Language::factory( $context->getLanguage() );
                return [
                    'monthNames' => $language->getMonthNamesArray();
                ];
            } ],
        ],
    ],

Module definition (extension)[edit]

In extension.json:

    "mything": {
        "localBasePath": "modules/mything",
        "remoteExtPath": "MyExtension/modules/mything",
        "packageFiles": [
            "init.js",
            "thinglib/index.js",
            "thinglib/formatter.js",
            {
                "name": "config.json", 
                "config": [ "UseLongThingFormat" ]
            },
            {
                "name": "data.json",
                "callback": "MyExtensionHooks::getMyThingData",
                "callbackParam": { "key1": "value1", "key2": "value2" }
            }
        ]
    }

In MyExtensionHooks.php:

class MyExtensionHooks {
    // ...
    public static function getMyThingData( ResourceLoaderContext $context, Config $config, array $callbackParams ) {
        $language = Language::factory( $context->getLanguage() );
        return [
            'monthNames' => $language->getMonthNamesArray();
        ];
    }
}

JavaScript[edit]

In init.js:

var thinglib = require( './thinglib/index.js' ),
    monthNames = require( './data.json' ).monthNames,
    config = require( './config.json' ),
    formatter = new thinglib.Formatter( { months: monthNames } );

if ( config.UseLongThingFormat ) {
    formatter.format( /* something */ );
} else {
    // do something else
}

In thinglib/index.js:

var thinglib = {
    Formatter: require( './formatter.js' ) // note this path is relative to the file we're in
    /* otherthing: require( './otherthing.js' ) */
    /* etc */
};

module.exports = thinglib;

In thinglib/formatter.js:

function Formatter( config ) {
    // ...
}

Formatter.prototype.format = function ( /* ... */ ) {
    // ...
};

module.exports = Formatter;

Debugging[edit]

In debug mode (?debug=true), traditional modules load each of their files in a separate request. This feature is partly unavailable to package modules. In debug mode, a package module is still loaded in its own request and without minification, but the files within the same module are bundled together in one request. If you're having trouble finding a file or a piece of code using your browser's developer tools (for example, to place a breakpoint somewhere), you can use the following hack.

Suppose you're trying to find the file named thinglib/formatter.js in the module named mything from the example above, and want to place a breakpoint in it. You can do that as follows:

  1. In your browser console, run mw.loader.moduleRegistry['mything'].script.files['thinglib/formatter.js']. This will print something like f(require,module){...} (in Chrome) or function js() (in Firefox).
  2. In Chrome, this will print something like f(require,module){...}; click on this text. In Firefox, this will print function js() followed by an arrow pointing up and to the right to three horizontal lines; click this icon.
  3. This will take you to the "Scripts" panel . If you see minified code, click the "pretty print" button (at the bottom left of the code pane, the icon looks like {}).
  4. In Chrome, you will now be looking at a line of code highlighted in yellow that looks like "thinglib/formatter.js": function(require,module) {. The code of thinglib/formatter.js begins below this line. In Firefox, you won't be taken to this place automatically, you'll have to search for thinglib/formatter.js':to find it[2].

You can also access the exports of a file (i.e. the value it assigns to module.exports and returned by require('./formatter.js')) by running mw.loader.moduleRegistry['mything'].packageExports['thinglib/formatter.js'] in the console. Often this is a function or an object containing functions, which you can then access using steps 2-4 above.

See also[edit]

Footnotes[edit]

  1. Note that vars that depend on the request context (e.g. the user or the page title) can't be moved to a package module. They can only be exported with OutputPage::addJsConfigVars().
  2. Note that Firefox's pretty printer changes double quotes (") to single quotes ('), so you will have to search for thinglib/formatter.js': with a single quote, even though there's a double quote in the original code generated by ResourceLoader.