ResourceLoader/Package files

shortcut: RL/PKG
From mediawiki.org

Package files can be used with a ResourceLoader module to bundle multiple script files and data exports, accessible client-side via require().

To enable this feature in a module, use the 'packageFiles' property of a module descriptor. Traditional modules specify multiple script files with the 'scripts' property, which blindly concatenates them as if they were a single script without ability to separately access them client-side.

Why[edit]

Node.js compatibility[edit]

This feature was originally conceived as a way to make code written for Node.js easier to integrate with ResourceLoader. Libraries that contain multiple files that require() each other, could be registered as a ResourceLoader module without modification.

Private export and import[edit]

The ability to export values from individual files and import them within a module, has also proved useful for modernising JS code in MediaWiki itself, by no longer needing to attach all classes to a public and global object.

Data and config bundling[edit]

The ability for a module to have multiple "files" that remain individually addressable, has also opened the door to support bundling of JSON files and virtual data exports from PHP (as JSON). The proposal to embed config variables in ResourceLoader modules, thus ended up being implemented as virtual files in the module bundle.

Before this feature existed, embedding config and data in modules was also possible, but required writing your own subclass of ResourceLoaderFileModule, an obscure technique that most developers didn't know about or felt uncomfortable using. Most instances of this technique in MediaWiki core have now been ported to use a virtual file instead.

A more common technique for exporting values from PHP was to export them with $out->addJsConfigVars()[1] or through the ResourceLoaderGetConfigVars hook, as mw.config keys. This is problematic for performance because:

  • Startup module config vars were exported on all page views for all users (wastes bandwidth cost), and need to be parsed and processed before your actual modules can begin to download (delays interaction), and were only cached for a short time (frequent re-download).
  • OutputPage config vars are in the <head> and block downloading of article text (delays visual rendering), and have to be processed before actual modules can start to begin downloading (delays interaction).

Exports bundled using Package files, on the other hand:

  • Only downloaded if they are needed, together with the code (saves bandwidth).
  • Only downloaded when they are needed (avoids rendering or interaction delays).
  • Benefit from longer caching times (only re-downloaded if they change, as part of the module).

How it works[edit]

MediaWiki version:
1.33

A package file can either be JavaScript code (script), or a JSON blob (data). A file's type is inferred based on its extension (.js or .json). Files can be real files from disk, or dynamic files generated by code. Dynamic files are commonly used to export the values of configuration settings or other data from the server, and in those cases they're typically named config.json or data.json.

Every module has one main file. This is the first file listed, unless another file is explicitly designated as the main file. When the module loads, only the code in the main file is executed. The code in the other files is not executed unless and until it is invoked using require().

To invoke a non-main file, use require( './foo.js' ). The path must start with ./ or ../ to indicate that it's a file in the same module (as opposed to the name of another module), and the path must be relative to the current file. The file suffix .js is required. The file may assign a value to module.exports, and this value will be returned by the require( … ) function call.

The main file may also assign a value to module.exports, defining a public interface of the module. Code in other modules may access this value by using require( 'module' ) (without ./ or ../ and .js). The module must be loaded first, as a dependency or with mw.loader.using.

Gadgets[edit]

Since MediaWiki 1.38, Gadgets also support package files. For details refer to Package Gadgets and Extension:Gadgets#Options.

Module definition[edit]

Files can be specified in the 'packageFiles' property in the following ways:

Static files[edit]

Static file from the filesystem[edit]
"packageFiles": [
  "foo/bar.js"
]

The name under which it is made available to require() can be overridden if needed, using a name alias:

"packageFiles": [
  { "name": "blah.js", "file": "foo/bar.js" }
]
Main module file[edit]

By default the first JavaScript file in the packageFiles array will be considered the module's main file (or "entry point").

To set this explicitly instead, use the main attribute. This indicates which of the files executes first and has the control to import other files:

"packageFiles": [
  { "name": "bar.js", "main": true }
]
Dynamic source file[edit]

The callback option can be used to dynamically decide which file to relate to an exported name. Read more about what callbacks can do in the sections further down.

"packageFiles": [
    {
      "name": "foo.js",
      "callback": "MyExtensionHooks::getFooFile"
    }
]
function getFooFile( MediaWiki\ResourceLoader\Context $context, Config $config ): MediaWiki\ResourceLoader\FilePath {
    $file = $config->get( 'FooSpecialEnabled' ) ? 'foo-special.js' : 'foo.js';
    return new MediaWiki\ResourceLoader\FilePath( $file );
}

Virtual files[edit]

Custom content[edit]

Define a virtual JavaScript file, with the specified contents:

"packageFiles": [
    {
      "name": "bar.js",
      "content": "console.log( 'Hello world' );"
    }
]

Define a virtual JSON file, whose contents will serialised as JSON. For example the below would export {"hello": "world"}:

"packageFiles": [
    {
      "name": "bar.json",
      "content": { "hello": "world" }
    }
]

The same works from PHP as well (such as in MediaWiki core's Resources.php file, or from an extension hook):

'packageFiles' => [
    [
    	'name' => 'blar.json',
    	'content' => [ 'hello' => 'world' ],
	],
]
Generated content[edit]

Define a virtual file whose contents is generated by a callback. For JS files, the callback should return a string. For JSON files, it can return anything that's JSON-serializable (typically an associative array).

"packageFiles": [
    {
      "name": "blah.json",
      "callback": "MyExtensionHooks::generateBlah"
    }
]
/** @return array|string|int|bool Data for JSON */
function generateBlah( MediaWiki\ResourceLoader\Context $context, Config $config ) {
    return /* ... */;
}

The callback is executed in the context of a load.php request and cached as part of that module, so it can't know which user is logged-in or which page is being viewed. Instead, the result is computed once and re-used across different users and pages. If you did attempt to access a RequestContext or User in your callback, it would likely return Error: Sessions are disabled for this entry point .

The MediaWiki\ResourceLoader\Context and Config do offer information that you can vary by, such as by site configuration, current skin, and current interface language.

The callback takes an optional $param, which is set to the value of a "callbackParam" key specified in packageFiles. This allows callbacks to be re-used for multiple purposes:

"packageFiles": [
    {
      "name": "foo.js",
      "callback": "MyExtensionHooks::generateFoo",
      "callbackParam": [ "A", "B", "C" ]
    }
]
/** @return string JavaScript code */
function generateFoo( MediaWiki\ResourceLoader\Context $context, Config $config, $param ): string {
    return /* ... */;
}

Common pitfall: using i18n messages in generated content callbacks[edit]

A common source of errors is trying to use wfMessage() in these callbacks. Trying to do this results in a Sessions are disabled for this entry point error. Instead, you should use the $context object passed in as the first parameter, and replace calls to wfMessage() with $context->msg(). Similarly, you should not use $wgLang, use $context->getLanguage() instead.

function generateBlah( MediaWiki\ResourceLoader\Context $context ) {
    // Wrong, causes an error:
    $numUsersMessage = wfMessage( 'how-many-users' )->numParams( $numberOfUsers )->parse();
    $userList = $wgLang->commaList( $users );
    
    // Right:
    $numUsersMessage = $context->msg( 'how-many-users' )->numParams( $numberOfUsers )->parse();
    $userList = $context->getLanguage()->commaList( $users );
}

Config files[edit]

This is a shortcut for the common case of generating JSON that exports one or more MediaWiki configuration variables. The below defines virtual file whose contents are {"LegalTitleChars": "...", "IllegalFileChars": "..."}:

[
	'name' => 'blah.json',
	'config' => [ 'LegalTitleChars', 'IllegalFileChars' ]
]

Note that this syntax uses config setting names as understood by Config::get() (e.g. 'LegalTitleChars'), which are without the wg prefix that global variables use (e.g. 'wgLegalTitleChars').

You can also use aliases to export configuration variables under different names:

[
	'name' => 'blah.json',
	'config' => [ 'naughtyChars' => 'IllegalFileChars' ]
]

will result in {"naughtyChars": "value of $wgIllegalFileChars"}.

If you need to do more advanced manipulation of config variables, use a callback as described above. In the callback, you can use e.g. $config->get( 'IllegalFileChars' ) to get the value of a config setting.

Base path[edit]

Most modules using Package files set 'localBasePath' to the common directory prefix of the files. This is done for convenience (not having to write resources/src/whatever/ over and over), and to make dynamic files easier to deal with. Without a base path, a dynamic file called config.json would have to be accessed using require( '../../../config.json' ), or it would have to be named resources/src/whatever/config.json so it could be accessed using require( './config.json' ). With a base path, you get the best of both worlds.

Note that, if you set localBasePath, you will also have to set remoteBasePath (for core) or remoteExtPath (for extensions) to match.

Incompatibility with 'scripts'[edit]

If a module uses the 'packageFiles' property, it cannot use the 'scripts' property. Defining a module that uses both properties will throw an exception.

Package files are also incompatible with 'languageScripts' and 'skinScripts'. Defining 'languageScripts' won't throw an exception, but the property will be ignored. ('skinScripts' throws an exception just like 'scripts' does.) A way of defining language/skin-specific script files for modules using Package files has not yet been developed (and this blocks porting ResourceLoaderLanguageDataModule).

Example uses in real code[edit]

Basic example/illustration[edit]

MediaWiki version:
1.33

Module definition (core)[edit]

In Resources.php:

    'mything' => [
        // Make all paths relative to resources/src/mything
        'localBasePath' => "$IP/resources/src/mything",
        'remoteBasePath' => "$wgResourceBasePath/resources/src/mything",
        'packageFiles' => [
            'init.js', // Main file because it's listed first
            'thinglib/index.js',
            'thinglib/formatter.js',
            [ 'name' => 'config.json', 'config' => [ 'UseLongThingFormat' ] ],
            [ 'name' => 'data.json', 'callback' => function ( MediaWiki\ResourceLoader\Context $context, Config $config, array $callbackParams ) {
                $language = Language::factory( $context->getLanguage() );
                return [
                    'monthNames' => $language->getMonthNamesArray();
                ];
            } ],
        ],
    ],

Module definition (extension)[edit]

In extension.json:

    "mything": {
        "localBasePath": "modules/mything",
        "remoteExtPath": "MyExtension/modules/mything",
        "packageFiles": [
            "init.js",
            "thinglib/index.js",
            "thinglib/formatter.js",
            {
                "name": "config.json", 
                "config": [ "UseLongThingFormat" ]
            },
            {
                "name": "data.json",
                "callback": "MyExtensionHooks::getMyThingData",
                "callbackParam": { "key1": "value1", "key2": "value2" }
            }
        ]
    }

In MyExtensionHooks.php:

class MyExtensionHooks {
    // ...
    public static function getMyThingData( MediaWiki\ResourceLoader\Context $context, Config $config, array $callbackParams ) {
        $language = Language::factory( $context->getLanguage() );
        return [
            'monthNames' => $language->getMonthNamesArray();
        ];
    }
}

JavaScript[edit]

In init.js:

var thinglib = require( './thinglib/index.js' ),
    monthNames = require( './data.json' ).monthNames,
    config = require( './config.json' ),
    formatter = new thinglib.Formatter( { months: monthNames } );

if ( config.UseLongThingFormat ) {
    formatter.format( /* something */ );
} else {
    // do something else
}

In thinglib/index.js:

var thinglib = {
    Formatter: require( './formatter.js' ) // note this path is relative to the file we're in
    /* otherthing: require( './otherthing.js' ) */
    /* etc */
};

module.exports = thinglib;

In thinglib/formatter.js:

function Formatter( config ) {
    // ...
}

Formatter.prototype.format = function ( /* ... */ ) {
    // ...
};

module.exports = Formatter;

You no longer need to wrap script files in a namespace closure.

Debugging[edit]

In debug mode (?debug=true), traditional file modules load each of their files in a separate request. This feature is partly unavailable to Package files. In debug mode, modules with Package files are still loaded in a separate request and without minification, but individual files within the same module are bundled together in that request. If you're having trouble finding a file or a piece of code using your browser's developer tools (for example, to place a breakpoint somewhere), you can use the following hack.

Suppose you're trying to find the file named thinglib/formatter.js in the module named mything from the example above, and want to place a breakpoint in it. You can do that as follows:

  1. In your browser console, run mw.loader.moduleRegistry['mything'].script.files['thinglib/formatter.js']. This will print something like f(require,module){...} (in Chrome) or function js() (in Firefox).
  2. In Chrome, this will print something like f(require,module){...}; click on this text. In Firefox, this will print function js() followed by an arrow pointing up and to the right to three horizontal lines; click this icon.
  3. This will take you to the "Scripts" panel . If you see minified code, click the "pretty print" button (at the bottom left of the code pane, the icon looks like {}).
  4. In Chrome, you will now be looking at a line of code highlighted in yellow that looks like "thinglib/formatter.js": function(require,module) {. The code of thinglib/formatter.js begins below this line. In Firefox, you won't be taken to this place automatically, you'll have to search for thinglib/formatter.js':to find it[2].

Accessing exports from browser console[edit]

The require function is not available in the global scope, such as in browser console, but alternatives are available for debugging.

To access the exports of a module, use mw.loader.require('mything') instead. This only works with modules (returning the exports of the main file), not with files inside a module.

To access the exports of a file, use mw.loader.moduleRegistry['mything'].packageExports['thinglib/formatter.js'] in the console. Often this is a function or an object containing functions, which you can then access using steps 2-4 above.

See also[edit]

Footnotes[edit]

  1. Note that vars that depend on the request context (e.g. the user or the page title) can't be moved to a package module. They can only be exported with OutputPage::addJsConfigVars().
  2. Note that Firefox's pretty printer changes double quotes (") to single quotes ('), so you will have to search for thinglib/formatter.js': with a single quote, even though there's a double quote in the original code generated by ResourceLoader.