ResourceLoader/Architecture

This page summarizes the many Features of ResourceLoader which together build the rich experience and front-end performance.

Module properties

 * Scripts
 * Styles
 * Messages
 * Dependencies
 * Group

Conditions
Scripts can be conditionally included in a module based on the current production mode, selected skin and user language. This keeps responses as small as possible by only including relevant components while also allowing progressive enhancement and conditional inclusion of features based on the environment.

Such as the  method, which is just a no-op closure in production mode, whereas in debug mode the   module loads additional components that implement a visual console for browsers that don't have one by default.

Embedding
In order to reduce the number of HTTP requests for images used in the interface, ResourceLoader makes use of Data URI embedding. When enabled, images will be automatically base64-encoded and embedded into the stylesheet in-place. While it will make the stylesheet larger, it improves performance by removing the overhead associated with requesting all those additional files. The actual server response (which contains the minified result of all concatenated stylesheets and all embedded images in a single request) uses gzip compression. This enables the response to function a bit like a "super sprite" (more on this below). Regardless of the expansion caused by base64 encoding, the ResourceLoader response is smaller in size than the sum of the individual binary files.

To trigger the embedding the " " annotation is used in a CSS comment. For example:

Another interesting point is that using this technique makes traditional sprites obsolete. The motivation of a sprite is good, it:
 * Reduces HTTP requests by combining multiple images into one.
 * Increases power of compression by providing more sample data in a single file.

But sprites come with a few caveats:
 * Maintenance (if an image needs to be updated, one has to regenerate the entire sprite, update the background positions, etc.)
 * Produces overly complex CSS (unrelated stylesheets would be referencing the same image with arbitrary background positions, it isn't obvious which image is being referred to, like " " vs. " )".
 * ResourceLoader Wikimania 2011.pdf  Imposes restrictions on usage of the image (no freedom in background-repeat, -size or -position as that would "leak" other parts of the sprite).

These caveats aren't the end of the world (sprites are in wide use, clearly they do work). And many front-end systems use sprites, even in a semi-automated fashion (making some of the above caveats less painful). But, using the automated embedding technique: the above caveats don't apply: .. the advantages of sprites also hold up: ... and, in addition to that, it gets even more out of it:
 * No maintenance
 * Clean CSS
 * No restrictions
 * Reduced number of HTTP requests.
 * Increased power of compression
 * No download delay: Once the stylesheet is there, all the images are there as well.
 * When using an interactive state (e.g. ) browsers only start the download once the relevant state is active. By embedding it there will be no flash during the download, as the image is instantly available.
 * The number of requests is even smaller. The CSS and the images are now in the same request.
 * The Gzip compression rate will be even higher (completely unrelated modules can be concatenated and compressed together, without having to worry about anything. Also, this way PNG headers can be compressed even stronger).

Conditions

 * See the section under  for more information

Similar to scripts, the styles property also features the ability to compose the module dynamically based on the environment.

Conditions
As with the other two resource types, the messages are also optimized to load only what is necessary for the requesting environment. Messages will be fetched from MediaWiki's localization framework (including proper fallback to other languages for messages that are not translated yet). And only 1 resulting set of messages is delivered the client, the set that the user will need.

Caching
Modules are composed based on the environmental variables (mode, skin, user language). The result of the generated module package is cached with a timestamp. This timestamp is based on a number of factors. All together, used to ensure proper cache busting when needed while also avoiding unnecessary re-generation of the package when it isn't needed.
 * JavaScript files and CSS files Last modified timestamps from the files on disk.
 * Files referenced in CSS Extracted from the stylesheets, local path resolved with . Last modified timestamps from the files on disk.
 * Localization Last time any of the messages has been updated in the localization storage.
 * Dependencies It works recursively for any modules that this module depends on.

Whenever a module is requested from the server, the above is evaluated. If needed, the cache will be re-generated. All phases of the optimization and packaging have been optimized to be able to run on-demand on the web server. No build scripts, no periodic cron tasks.

JavaScriptMinifier
Although the re-generation of a module package should be relatively rare (since cache is very well controlled), when it does happen it has to perform well from a web server. ResourceLoader features on-demand generation of the module packages. For that reason it doesn't use the famous JSMin.php library (based on Douglas Crockford's JSMin) because it is too slow to run on-demand from the web. Although JSMin.php only takes about 1 second for (which is okay if you're on the command-line), when working on-demand from the web (with potentially dozens of large files lined up to be minified) waiting this long is unacceptable. Especially if potentially hundreds of thousands of requests could come in at the same moment, all finding out that the cache isn't up to date, to avoid a cache stampede.

Instead ResourceLoader uses an implementation of Paul Copperman's JavaScriptMinifier, which is almost 4X as fast. In addition to the speed, time has told that JavaScriptMinifier interprets the JavaScript syntax more correctly and succeeds in situations where JSMin outputs invalid JavaScript. The output size of JavaScriptMinifier is slightly larger than JSMin (about half a kilobyte larger for the jquery.js sample). The reason this is not considered a loss is because it is put in the bigger picture. ResourceLoader doesn't aim to compress as small as can be no matter the cost. Instead it aims for balance, get large gains in a wide range of areas while also featuring instant cache invalidation, fast module generation, a transparent "build"-free environment for the developer, etc. The fact that it could be a little bit smaller then becomes an acceptable trade off.

JavaScriptMinifier is a standalone library. It is free, open-source and available from the MediaWiki repository ([//svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/includes/libs/JavaScriptMinifier.php?view=markup svn] &bull; git)

Balance

 * Batching
 * Groups