Wikimedia Platform Engineering/MediaWiki Core Team/Quarterly review, October 2013

This is an outline for a review of the MediaWiki Core team which will take place at WMF on October 15.
 * Notes summarizing this review

Team
See Wikimedia MediaWiki Core Team page.

Changes since last review:
 * Bryan Davis started as Senior Software Engineer, on loan to Multimedia and focused on bugfixing and technical debt.
 * Dan Garry started as Product Manager for Platform
 * Ori Livneh moved into the Senior Performance Engineer role

Auth systems / Security
Chris Steipp, Aaron Schulz, and Brad Jorsch did the bulk of the software development on this, with support from Tim Starling on software design, May Tee-Galloway, Brandon Harris, and Jared Zimmerman on user experience, and Chad Horohoe, Ryan Lane and Ori Livneh on the SSL deployment.

We had planned to wrap this up early in the quarter. It unfortunately lasted the duration of the quarter, and still isn't quite finished.


 * SUL - most work on this completed in August. We had not fully anticipated the usability concerns with our initial implementation, so substantial rework was needed.  We additionally were nagged by some pernicious high-priority bugs, the last of which we finally got to the bottom of in September.
 * SSL deployment - significant work by the MediaWiki Core group was not in plan for this quarter, but our decision to deploy this took substantial time away from other aspects of the project.
 * OAuth - this is another case where we underestimated the amount of work that was needed, in particular on usability. After substantial iteration, we have this in a pretty good place.
 * OpenID - We're relying on volunteer development for this. It will also need substantial usability work, so we would like to postpone this work until 2014.

We're happy with where SUL, SSL and OAuth are right now, but want to do better with how we get there for our next project of similar scope.

Search (ElasticSearch) deployment
This project is primarily driven by Chad Horohoe and Nik Everett in MediaWiki Core, along with support from Faidon Liambotis, Peter Youngmeister and Asher Feldman in TechOps.

We have developed the CirrusSearch extension to MediaWiki, which is an application glue layer to ElasticSearch. ElasticSearch is an externally developed search indexing component.

Italian Wiktionary, Catalan Wikipedia and English Wikisource are all running CirrusSearch now. Additionally, we deployed to all "closed" wikis. Further feature refinement and bugfixing are ongoing, with roughly 2 to 3 deployments a week.

Architecture formalization
Tim Starling has largely been spearheading this work, with support from Brion Vibber and Mark Bergsma.

The Architecture guidelines document continues to progress, though there is still substantial work needed to come to consensus on many points of direction, and to make the document a well-organized and well-scoped document. Our RFC list is now much more organized, and there have now been two IRC meetings that have happened in consecutive weeks recently (September 25 and October 2).

DevOps sprint
This was postponed due to work on auth systems.

Upcoming quarter
Personnel allocations are still a little bit up for grabs at this point.

Search (ElasticSearch) deployment
We anticipate we'll be able to complete this deployment by the end of 2013, and disable our previous search technology (lsearchd) in January 2014. Nik Everett and Chad Horohoe will continue on this project through the quarter.

Architecture formalization
Tim Starling will continue to put effort in this area, though will likely spend some time on other projects. We plan to continue with RFC review meetings on IRC, more-or-less weekly.

Our first Architecture summit is planned for January 2014, where we plan to make substantial progress toward consensus on architecture changes planned in RFCs. We plan to make this an annual event.

DevOps sprint
The main areas of focus for this sprint are: We haven't finalized the personnel allocation in this area.
 * git-deploy
 * monitoring / reporting
 * deployment script improvements
 * multi-site awareness
 * Labs related