Requests for comment/Improving extension management

This RfC proposes a system for easier extension management through web interfaces, tarball generators and distributors, and dependency management.

Pain points for developers

 * Handling dependencies
 * Depending upon another extension is typically manual and hacky with to avoid load order dependencies.
 * Versioning
 * Versioning is mainly an internal thing, not really exposed anywhere, and not really used by sysadmins. Or is it? As a dev, I typically ignore them unless someone asks me to bump it.
 * Compatibility with core
 * We have REL_ branches, but people usually don't backport every change they make that is compatible
 * Some extension maintainers keep compatibility with older versions of core, but that isn't documented anywhere in a machine readable format for tools like ExtensionDistributor.
 * wfUseMW is hacky as well

Pain points for sysadmins

 * Just because an extension has a REL1_xx branch doesn't mean it actually is compatible with said version
 * Upgrading tons of extensions at once (during a major version bump) is not super friendly and requires a lot of manual steps

Considerations

 * Security: This system should function in an environment where webservers cannot write executable PHP files
 * Ease of use: Shouldn't require a CLI interface, nearly everything should be doable from the web
 * But...for people who like CLI/git, we should still support that workflow.

Dependencies upon MediaWiki core
Extensions can currently specify that they depend upon a specific version of core by using, however that is extremely hacky, and only allows for specifying a lower bound of support. We automatically create REL1_XX branches whenever core is branched, except that there is no actual verification that the extension works with that core version unless the maintainer or a user does so.

Instead of that, I propose that we store the versions that the extension supports in its extension.json file:

Supports is an array of versions that the maintainer has explicitly stated work and are supported. Experimental is used for those are that are in active development and not yet recommended for use. Broken means that it is not compatible with those versions. When an extension drops support for a certain release, it would create the REL1_XX branch at the last commit that did support that release.

The extension distributor would be updated to read from extension.json to see which versions to display.

Dependencies upon Libraries
We have standardized upon using composer as the dependency manager for library dependencies. Extensions would list their depdencies upon external libraries through composer.json files.

For users using the web interface and without CLI access, we would have a web proxy to composer that would take an input of the required dependencies and output a tarball of the vendor directory they need. I've written a quick prototype to demonstrate how this might work.

CLI users would be able to use the composer-merge-plugin to manage library dependencies.

Dependencies upon extensions
There are very few actual dependencies upon other extensions. Many times it's simply dependencies upon common code, that really should be refactored out into a separate library. A good example of this is AbuseFilter depending upon AntiSpoof. AbuseFilter really only needs AntiSpoof's normalization functionality, not the entire extension. That common code should be refactored out into a separate library which both AbuseFilter and AntiSpoof depend upon.

However, for cases where extensions do actually depend upon each other, these constraints can be added in the extension.json file:

This will be processed by the registry at compile time...?

Dependencies

 * Specify in the extension.json file what other extensions are required.
 * Processed by the registry at run-time
 * External libraries can be depended upon via composer.json. But not actual extensions.

Versioning

 * I think we should keep this in its current state, generally internal and exposed if the extension maintainer chooses to use it.

Compatibility with core

 * Extensions define what minimum wmf branch (or stable release) of core they support in the extension.json file. We don't auto-create release branches for each extension.
 * When the developer chooses to drop support for a major version of core, they have the option of creating a REL1_xx branch that users can use.
 * `master` can be compatible with whatever the developer wants it to be compatible with, and it's clear to everyone else what it's compatible with
 * Could we run tests automatically with every version of core it claims to be compatible with???

Implementation

 * SpecialPage to enable/disable extensions (via wfLoadExtension stuff/configdb)
 * Downloading/installing circles around tarballs.
 * ExtensionDistributor preps "bundles" which includes a group of extensions you need for the one you enabled, for your mw version.

Example because I'm not explaining my self properly:
 * User wants to install Translate, running mw1.23.
 * Ticks enable.
 * Fetches "Translate-ULS.1.23.tar.gz" from ED.
 * Unzips
 * Boom, magic

ahhhhhh