Requests for comment/Extension management with Composer/Farms and Composer

This page is a summary of the various issues and solutions about managing MediaWiki extensions with Composer in a farm.

This page comes for the topic "A way to disable extensions" and from personal (Seb35) research I did for the extension MediaWikiFarm (see references at the end). It should evolve with the state of the art.

Definitions

 * Extension: a MediaWiki extension or skin
 * Composer-managed Extension: Extension installed with the Composer mechanism: adding it in  and running Composer
 * (Extension) Activation:
 * Preliminary, Composer’s initial goal is to manage libraries, which one can define as passive code (e.g. classes or functions: no main function = if you don’t explicitely call it, it will never be executed and hence will have absolutely no effect – other than reserve of symbol names). This corresponds to class autoloading (MediaWiki class autoloader and Composer PSR4/PSR0/classmap/files when the files only contains classes or functions definitions)
 * MediaWiki extensions are generally active code (they register Hooks (term for the various MediaWiki mechanisms like hooks, namespaces registration, AuthManager mechanisms, permissions regitration, etc.) providing features). The extension’s code becomes active when registered in MediaWiki, this can be through:
 * Legacy activation:  the extension, registering the Hooks (possibly indirectly by a function/method)
 * Composer activation: autoloading a file registering the Hooks (possibly indirectly by a function/method)
 * ExtensionRegistry: (since MediaWiki 1.25) direct registration of the Hooks (possibly indirectly by a callback function/method)
 * Wiki: a MediaWiki website, with a dedicated database and URL, and a specific configuration and (Activated) extensions set
 * Wiki farm: a set of Wikis

Constraints

 * Dependencies: Composer-managed MediaWiki extensions have dependencies in their composer.json, these MUST be handled.
 * Single code: in a farm, you typically want to have only one (or a few) MediaWiki codebase (say, 1.27 LTS) to reduce maintenance burden and possibly CPU (Composer computations) and disk space (MW 1.27 = 110 Mio + vendor = 30 Mio), and activate only some extensions depending on the wiki (by the way, it’s what Wikimedia does, the biggest MediaWiki farm)

Issue
You want a Wiki farm with different Wikis, each one with a specific configuration and extensions set.

Consequently, given some MediaWiki extensions are Composer-managed, you want to Activate only some Extensions, and hence find all Dependencies for Activated Extensions.

Solutions
Each solution has its advantages and drawbacks. They are sorted by increasing difficulty, the more you scroll down the non-standard it becomes and the more you need carefully-crafted tools.

1. One MediaWiki installation per Wiki
Each Wiki has a dedicated directly containing a MediaWiki installation with the specific directory corresponding to the Activated Extensions.
 * Details


 * Advantages
 * Each Wiki is completely independent from the others, you can have one LocalSettings per directory or, if you prefer, one central LocalSettings, and there is one composer.json and one  directory per Wiki, this is the ultra-classical installation
 * Each Wiki can have completely different versions for non-Composer-managed Extensions and for Composer-managed Extensions


 * Drawbacks
 * When you want to activate/deactivate an Extension, you have to check if it is Composer-managed, and if so, run Composer to recompute the  directory
 * When you want to change the MediaWiki version globally, you have to use some batch script
 * The size is linear with the number of Wikis (currently about 140 Mio + Extensions, multiplied by the number of Wikis), the CPU is linear with the number of Wikis

2. One MediaWiki installation with one directory per Wiki
All Wikis use the same directory containing a unique MediaWiki installation, but each  directory is computed for the specific Wiki, and the file   directory in a non-standard path
 * Details
 * When you want to activate/deactivate an Extension, you have to check if it is Composer-managed, and if so, run the specific-flavour Composer to recompute the  directory corresponding to the Wiki
 * When you want to change the MediaWiki version globally, you have to use some batch script to compute the specific  directories
 * Wikis have the very same versions for non-Composer-managed Extensions

3. One MediaWiki installation with one directory containing one   subdirectory per Wiki
All Wikis use the same directory containing a unique MediaWiki installation and all Wikis use the same  directory containing classical Composer-managed librairies, Composer-managed Extensions have their code installed with Composer (in the   or   MediaWiki directories); Composer is run as much time as there are combinations of Composer-managed Extensions, and each autoloading set contained in a   directory is moved in a   directory, and the file   contains some logic to include the Composer autoloader corresponding to the Wiki.
 * Details

The specific  directories can be prepared beforehand (scenario A) or when a specific Composer-managed Extensions set is requested (scenario B). All small improvement would be to create symbolic links for identical combinations (when an Extension requires other Extensions).


 * Advantages
 * (A) You can instantly activate the Composer-managed Extensions
 * When you want to change the MediaWiki version globally, you can prepare a unique MediaWiki installation, and put it in production when ready
 * (B) Very few resources are needed


 * Drawbacks
 * Non-standard setup, you have to wrap Composer in some script to create a specific  directories
 * (A) Exponential number of combinations of Composer-managed Extensions = 2^N (see Wikipedia); disk size is not limiting (500 Kio per combination), but CPU time will become limiting, and it is quite a waste of resources only a few combinations are used
 * You have to prepare the various  directories beforehand
 * Wikis have the very same versions for non-Composer-managed Extensions and Composer-managed Extensions

4. One MediaWiki installation with one directory containing one   subdirectory per Extension
Similar to the previous setup, but instead of computing all combinations, you compute only the Composer autoloader for each Composer-managed Extension (idependently of other Composer-managed Extensions). In the file, you include all Extension’s autoloaders corresponding to Activated Extensions: you will load as much Composer autoloaders as there are Activated Extensions (or a bit less if you know some Extensions are – automatically – Activated by others).
 * Details

Possibly a class can be autoloaded by multiple Composer autoloaders, but it is not important, particularly given the corresponding file is always the same. Note that recently (2016), some Extensions (PageForms, SemanticFormsSelect) are Composer-installed but ExtensionRegistry-Activated, so the Activation should correspond to the Composer installation to avoid "all Composer-installed Extensions correctly installed but some Dependencies of Activated Extension are not Activated" (more clearly SemanticFormsSelect Activated but PageForms (Composer Dependency) not Activated).


 * Advantages
 * You can instantly activate the Composer-managed Extensions
 * When you want to change the MediaWiki version globally, you can prepare a unique MediaWiki installation, and put it in production when ready
 * Very few resources are needed: linear (and small) with the number of Extensions and independent of the number of Wikis


 * Drawbacks
 * Non-standard setup, you have to wrap Composer in some script to create a specific  directories
 * You have to prepare the various  directories beforehand
 * Wikis have the very same versions for non-Composer-managed Extensions and Composer-managed Extensions

5. One MediaWiki installation with one directory with a modified autoloader of files, to activate files corresponding to Activated Extensions
On Phabricator T61872#802235 detailled by User:Egel, install Composer in the traditional way, then remove the autoloading of the files (=the entry points of the extensions are always in the files; the other autoloading methods are only for passive code (classes and functions)). Then, extensions are Activated by including the "autoloading files" corresponding to the Activated Extensions. A further improvement would be to run Composer as much times there are Composer-managed Extensions to find Dependencies between Extensions, and next use these informations to Activate coherent sets of Extensions.
 * Details


 * Advantages
 * You can instantly activate the Composer-managed Extensions
 * When you want to change the MediaWiki version globally, you can prepare a unique MediaWiki installation, and put it in production when ready
 * Very very few resources are needed: independent of the number Wikis


 * Drawbacks
 * Non-standard setup, you have to wrap Composer in some script to create the specific  files
 * You have to prepare the various  files beforehand
 * Wikis have the very same versions for non-Composer-managed Extensions and Composer-managed Extensions

Composer-based
To be researched. Possibly create a Composer plugin implementing one of the previous strategy or another approach.

Implementations

 * Phabricator T61872#802235 - ScoutWiki uses the fifth solution
 * Extension:MediaWikiFarm - implementing the forth solution in (after experiments with the 2nd and 3rd solution)