Extension:MediaWikiFarm

The extension MediaWikiFarm creates farms of wikis with the following main features:
 * Multiversion: As a standard MediaWiki extension, it turns MediaWiki into a single-version farm; installed "in front of MediaWiki" it creates multiversion farms;
 * Hierarchical configuration: The configuration can be written either in YAML, PHP, or JSON, in a hierarchical manner: default value, default value for the wikis family, value for a single wiki;
 * Performance: Caching, caching, caching; there is one cache per file, caches are written as PHP arrays to benefit from OPcache extension and a final per-wiki LocalSettings.php (in PHP hence) is issued; with this cache architecture, this extension adds a mean time of 165µs per request compared to a standard LocalSettings.php (on a personal computer; PHP7+OPcache).

Detailed features

 * MediaWiki farm: each wiki is identified by its host, with dedicated version, configuration, and data directories;
 * Multiversion farm: each wiki is one of the installed MediaWiki version, with extensions and skins selected for this version, and extensions and skins activated or not for this wiki;
 * Hierarchical configuration: each parameter can be specified at different scales: for every wiki in the farm, for a specific family, or for a single wiki;
 * Multi-files configuration: the configuration can be distributed accross files with possibly different permissions, e.g. all passwords in a secret file, other parameters in a public file;
 * Configuration in YAML, JSON, or PHP: these three file formats are supported (YAML needs an external library);
 * Transparent from MediaWiki point of view: the configuration is a classic LocalSettings.php (after compilation from the configuration files) and the initial bootstraping code is hiding itself as much as possible;
 * Caches: all configuration files and final configuration are cached as PHP arrays, interoperables accross PHP versions and cacheable in OPcache;
 * CLI compatible: a command-line script selects the right wiki just like in the Web version;
 * Multi-farms: multiple farms can be created, each one with its own configuration files, possibly shared accross farms to e.g. create pre-production wikis with slightly different configurations;
 * Syntax-error-proof: syntax errors in configuration files do not crash the farm, just silently ignored when cache is activated;
 * Delayed version switch: during version switch, the version does not change until the maintenance/update.php script is run;
 * Custom 404: it can be displayed a customised HTTP 404 page for missing wikis, written in PHP or HTML;
 * Classical LocalSettings.php: although only visible in the cache directory, there is a classical LocalSettings.php;
 * Whole range of MediaWiki versions: every MediaWiki version from 1.1 to 1.28 works in multiversion farm (small issue for 1.1 and 1.2 still pending);
 * Wide range of PHP versions: every PHP version from 5.2 to 7.1 works (YAML library requires 5.3.2 or 5.5.9, so YAML-to-PHP must be handled separately for old MediaWiki versions);
 * Documented: all functions are documented and a general documentation is available in a subdirectory;
 * Translated: the description on Special:Version is translatable on TranslateWiki although, well, it’s the only string in the user interface;
 * Unit-tested, well: about 100 tests (~256 assertions) run in 10-15 seconds in recent MediaWiki versions with PHP 5.6 to 7.1, totalling 100% code coverage with strict coverage activated and global variables under watch;
 * Performance-tested: a basic testing infrastructure measures the performance in order to be aware of the performance impact of various strategies during development (e.g. the per-wiki LocalSettings.php was added because it was 32% (77µs/239µs) quicker than the previous implementation).

Installation
There are two installation modes:
 * an almost classical installation in the  subdirectory, but limited to single-version MediaWiki farms;
 * a more complex and completely unusual installation for multiversion MediaWiki farms.

These instructions are only minimal; you should read the documentation available in the subdirectory  to understand the goals, the basic concepts, and prepare your naming scheme (the URLs) and directory organisations before you start installing it.

Quick installation

 * Download the MediaWikiFarm extension and copy the directory in the usual subdirectory ;
 * 1) Optionally run Composer in the subdirectory extension (to read YAML files);
 * 2) Create a directory where you want to put your configuration, e.g.  ;
 * 3) Create and configure the main configuration file , e.g. from the sample file;
 * 4) Move your existing LocalSettings.php in this configuration directory;
 * 5) Copy the placeholder LocalSettings.php provided by MediaWikiFarm in your MediaWiki directory.

Multiversion installation
C.f. documentation in the subdirectory

Documentation
An extensive documentation is available in the subdirectory  of the extension. Some basic instructions should be transferred here (installation, basic configuration).

Be aware it is non-trivial to set up a MediaWiki farm; this extension only improves the MediaWiki part, after that it remains farm-type configuration of: DNS, webserver, MySQL, HTTPS, domain names, external services (memcached, Parsoid, *oid…), this is the job of a configuration management tool (puppet, Chef, Ansible, Cfengine, etc.).

Testing
The extension has been tested with a multiversion installation with MediaWiki ranging from 1.1 to 1.28alpha, with PHP 5.2 (MediaWiki 1.1-1.12), PHP 5.6 (MediaWiki 1.13-1.21), PHP 7.0 (MediaWiki >= 1.22), and PHP 7.1RC1 (MediaWiki 1.28alpha), with nginx 1.6.2 with fastcgi module. One minor issue is still pending to run MW 1.1 and 1.2.

There is a unit test suite with about 100 tests (250 assertions). This can be run with the customised MediaWiki PHPUnit (see tests/phpunit/phpunit.php in MW directory). This specific test suite can be run separately with. It should run without error with strict coverage, with a total code coverage of 100%. Only raw scripts and some untestable methods are code-coverage-ignored (4 methods out of 38). Note there is a global function that PHPUnit doesn’t code-coverage, see.

MediaWiki deactivated backup of global variables, but it can be re-enabled in this test suite by modifying the object property in constructor of MediaWikiFarmTestCase (slightly slower); in this case, expectedly-modified global variables are declared at the beginning of the test and any other global variable modified reports the test as risky. If a test is really long, possibly PHPUnit computes the differences of values of global variables between the beginning and the end of the test (count 10 minutes for a heavily modified global state).

A performance test is implemented in subdirectory tests/perfs. It is recommended to use it only in a development environment. To use it, you must have the farm installed and point, in the webserver config, the index.php to tests/perfs/index.php, then launch in command line the script. You must either delete the cache directory just before or give permissions to the script to do it. The first execution creates the cached LocalSettings.php, so it’s normal to see a "high" time in 'config farm', but it then divided by the sample size. With PHP 7.0 and MediaWiki 1.28alpha with the farm cache activated, I have figures (sample size=5000): 171µs for bootstraping farm, 219µs for executing the LS.php of farm (most (~177µs) of it is initialisation of ExtensionRegistry), 221µs for executing the classical LS.php (the two LS.php are strictly the same file, so it is not significant), and 9.984ms for the compilation of configuration (when cache is invalidated).

See also User:Seb35/MediaWiki Archaeology.

Development
Some ideas:
 * reduce the initial bootstraping from 3 files read (farms.yml.php, versions.yml.php, config-version.php) to 2 (metadata.php, config-version.php); additionally, such a file could be used to collect all wikis defined on the farms (will be useful to execute whole-farm operations like update.php or census wikis (e.g. for stats))
 * collect more errors and warnings
 * think about ways to display errors+warnings, either on-wiki, on a central wiki, or on-file
 * handle constants like CACHE_NONE, or more generally PHP syntax to use e.g. references; possibly this would be another config file with list of PHP-enabled parameters (or even a restricted list of PHP syntaxes authorised)
 * add tags for transversal selection of wikis: think about file format, how to store them: lists and/or dictionaries
 * research if it would be possible to handle Composer-installed extensions (see this discussion), better is only the "activation" feature, but more probably it would have to hook Composer to create isolated "autoloading parts for a given wiki" with ideally all other code (MW extensions and Composer libraries) laying in their respective directories -> I have now a rather precise idea of how to implement this properly

Related extensions

 * Extension:Farmer
 * Extension:Simple Farm
 * Extension:WikiFarm
 * Extension:Configure
 * Extension:Shared codebase
 * Extension:CreateWiki