Library infrastructure for MediaWiki/status

Last update on: 2014-11-monthly

2014-10-23
This project has been chosen as a Wikimedia Engineering Top Priority project for FY2014-15, Q2.

2014-10-monthly
The project kicked off mid-month with the merge of patches that enable PSR-3-based logging by MediaWiki. These changes are being tested in beta and will begin to roll out to the production cluster in early November with the 1.25wmf6 release branch.

An investigation into the possibility of using a package manager for JavaScript libraries in MediaWiki closed with the consensus opinion that we are not ready to choose a package manager at this time. The frontend standards group will revisit this decision in three to six months. It was agreed however that as far as possible JavaScript libraries should follow the guidelines for library development that are being worked on for PHP code.

Initial work has begun on a Profiler implementation that uses XHProf to collect information about the runtime costs of MediaWiki code. This approach to profiling will enable collection of information on code running on Wikimedia servers without relying on explicit wfProfileIn and wfProfileOut calls. This in turn will make splitting code out of MediaWiki core easier.

2014-11-18
Mid quarter progress update

At (or at least near) the midpoint of our initial three month project, we are making good progress on our major commitments and have picked up some interesting additional work.

Aaron Schulz and Chad Horohoe have joined the team. Both are helping make updates to the Profiler classes used to measure the performance of MediaWiki related to the Better PHP profiling RFC. Profiling was identified early on as a common entanglement for many generally useful utility libraries found in the MediaWiki codebase. The work that is progressing here should enable us to remove many explicit  and   calls in the MediaWiki PHP code while still getting the benefit of performance measurements via the XHProf profiling library.

The cssjanus library has been removed from MediaWiki's core repository and replaced with a Composer managed import from the offical upstream. We have also extracted the CDB library originally written by Tim Starling and published it on Packagist.

Several classes have been moved from includes/utils to includes/libs (ArrayUtils, MapCacheLRU, Cookie/CookieJar) which makes them easy candidates for publication in stand alone libraries in the future. Aaron is working on a list of possible libraries to create from the MediaWiki codebase that would group several useful classes together. This should produce more sustainable projects than having literally dozens of libraries made up of only a single class.

Bryan is continuing to work on structured logging changes and hopes to soon test a Monolog based logging pipeline in Beta to replace the current system. 

2014-11-monthly
Aaron Schulz and Chad Horohoe joined the team. Both are helping make updates to the Profiler classes used to measure the performance of MediaWiki related to the Better PHP profiling RFC. Profiling was identified early on as a common entanglement for many generally useful utility libraries found in the MediaWiki codebase. The work that is progressing here should enable us to remove many explicit  and   calls in the MediaWiki PHP code while still getting the benefit of performance measurements via the XHProf profiling library. Profiling via the XHProf functionality built into HHVM is currently in use in both the beta and production clusters and helping drive some low hanging fruit code improvements.

Bryan is continuing to work on structured logging changes and is testing a Monolog based logging pipeline in Beta to replace the current system. has been deprecated and all usage in the core or MediaWiki replaced with the new  class which was introduced by the PSR-3 logging changes.

The cssjanus library has been removed from MediaWiki's core repository and replaced with a Composer managed import from the official upstream. The lessphp CSS pre-processor which was historically manually copied into MediaWiki's git repository is now imported via Composer.

The CDB library originally written by Tim Starling has been extracted to its own git repository and published on Packagist. Both MediaWiki itself and the "multiversion" scripts that are used to manage the WMF wiki family are now importing CDB via Composer instead of the old practice of keeping two copies of the code updated manually in the respective repositories.

The simplei18n PHP library that was developed for the IEG's Grant review application based on code from the Wikimania Scholarships application was transferred from Bryan's personal github account to the official Wikimedia account.

External dependencies for the BounceHandler and Elastica extensions have been removed from the extension git repositories and replaced with Composer managed imports. For the WMF cluster, these dependent libraries have been added to the mediawiki/vendor.git repository. ExtensionDistributor has been updated to package composer managed dependencies in the tarballs it generates for installing extensions. The php-composer-validate test is now applied to all extensions and skins to validate the syntax of composer.json when changes are uploaded to gerrit.

Several classes have been moved from includes/utils to includes/libs (ArrayUtils, MapCacheLRU, Cookie/CookieJar) which makes them easy candidates for publication in stand alone libraries in the future. Aaron is working on a list of possible libraries to create from the MediaWiki codebase that would group several useful classes together. This should produce more sustainable projects than having literally dozens of libraries made up of only a single class.

