Library infrastructure for MediaWiki/Initial project pitch

This is a project under consideration by the WMF MediaWiki Core team for late 2014. See Wikimedia MediaWiki Core Team/Quarterly review, October 2014 and Wikimedia MediaWiki Core Team/Backlog

What we want to accomplish
Our long term goal is to make a MediaWiki deployment an application that is composed of many small purpose built libraries (internally and/or externally developed) with interfaces that allow individual libraries to be exchanged for others. Using this flexibility, allow easier tuning of major features for various use-cases and easier development of new large features by composition. For example, swap storage of revisions as exists now with some system that has better performance characteristics for the Wikipedia wiki farm.

Why do this?

 * Make life better for new (and experienced) developers by organizing the code into simple components that can be easily understood.
 * Reverses inertia toward ever expanding monolithic core by encouraging developers (in core) to develop their work as reusable modules with clearly-defined interfaces
 * Starts making true unit testing of core viable by having individually-testable units
 * Provides an interim step on the way to service-oriented architecture in a way that is useful independently of that goal
 * Encourages reuse and integration with larger software ecosystem. Done correctly, this will provide a useful means of expanding our development capacity through recruitment of library authors eager to showcase their work on a top 10 website.
 * Share our awesome libraries with others and encourages contributions from them even if they aren't particularly interested in making our sites better.

Near term goals

 * Show a path to decomposition of MediaWiki into smaller pieces.
 * Introduce the wider PHP development community to years of battle tested code contained in MediaWiki as reusable libraries that can be added to any PHP project.

Three month goal

 * Establish the precedent for how libraries should be incorporated into MediaWiki by replacing existing functionality with an existing library
 * Primary candidate: Get structured logging patches finished and merged (Structured Logging RFC)
 * Uses PSR-3 logging interface and Monolog both via Composer
 * Start people converting from wfDebugLog to direct use of PSR-3 interface
 * Add configuration for WMF cluster that uses Monolog to format and ship logs to logstash without using udp2log
 * Establish the precedent for how libraries should be used by demonstrating the split of at least one library out of core
 * Candidate #1: ResourceLoader
 * This is an obvious choice because the Editing team would like it to be independent of core and it would server as a good example of the quality of implementation and design available in the larger MediaWiki codebase.
 * Candidate #2: Extract Profiler into a library that can be used by anyone who wants to instrument a PHP application stack
 * Use Profiler as a test case for developing a full life cycle plan for extracted libraries
 * There is a lot of code that is reusable save for wfDebug / WfProfile calls and could otherwise be extracted
 * Other candidates:
 * CSSJanus
 * CSSMin
 * CLDR parser
 * HashRing
 * Aaron's UUID generator
 * Zip directory reader
 * IPSet
 * PHP JSON sanitizer and pretty printer
 * General considerations:
 * Where do we put the library in git/gerrit?
 * Where do we document the library?
 * Where do we track bugs?
 * How do we promote the use of the library
 * Introduce some dependency management system to MediaWiki which can be used to configure complex objects
 * Complete infrastructure for using composer to manage internal and external dependencies (WMF Composer RFC)
 * TODO: strip this down to something that we're much more confident can be completed by the end of the 2014 calendar year

Out of scope
Complete breakup of MediaWiki core into components

The end goal is not to eliminate MediaWiki but instead to make MediaWiki the platform for creating massive multi-user versioned data-based projects, and make it a platform that is flexible enough to extend in novel ways.