Wikimedia Platform Engineering/MediaWiki Core Team/Check-ins/20140331

who: Brad, Ori, Aaron, Rob, Greg, Chris, Chad, Nik, Bryan, Dan, Tim regrets: no regrets, no apologies

Bug escalation

 * https://bugzilla.wikimedia.org/show_bug.cgi?id=46014 that dang old inconsistent revision content from google indexers…
 * Brad taking it on
 * Memcached traffic increase: was the result of https://gerrit.wikimedia.org/r/#/c/117231/, but calls into question the strategy of relying on memcache to repeatedly retrieve an ~80k (in the case of enwiki) blob

Quarterly Review
Contenders:

HHVM: continuing work on that

 * Reasons AGAINST: good packages not expected before May; more work than could probably be done in a single quarter.
 * Reasons FOR: not being able to get to a full production roll-out by the end of the quarter does not mean that we don’t easily have a quarter’s worth of work to usefully invest. There are a lot of loose ends from this quarter.

Revamp revision storage (w/Gabriel & co.)

 * Pro: would be a catalyst for refactoring things in core
 * The task: (a) specify an API for storage of revisions; anything implementing that API could be trivially used with MediaWiki. (b) refactor core so that rev storage is an implementation of that API.
 * Have an interface that is bundled with core and that provides all the required functionality.
 * Gabriel’s revision storage platform would be an alternate implementation of that interface.
 * TODO: Ping Gabriel and find out how much intends to do on his own and how a sprint by the core team could assist

Job queue work

 * Bug 46770 “Rewrite jobs-loop.sh in a proper programming language”? [TS]
 * Bug 46770 would be nice. So would more monitoring. I tend to agree with Rob that it’s probably not worth a full quarter (nor do I agree that we should scrap the whole thing in favor of $someNewThing) [CH]
 * Antoine: Isn’t our time better invested in overhauling the whole job queue system?
 * But WHY? Like Rob said...there doesn’t seem to be a compelling case for replacing the whole thing...just making some improvements around the edges. [CH]
 * Not really, after Aaron already spent so much time overhauling it over the last 18 months [TS]
 * +10 [CH]
 * We could use better monitoring of the queue processing, nicer priority systems..

SUL finalization

 * Legoktm should be able to do the engineering work, Dan can do the other loose ends

OpenID connect (goes hand-in-hand with Phabricator)

 * Continuous integration fully tied to Gerrit stream-events / Gerrit comment. Need to either adapt Zuul or rethink the way we do CI.
 * https://secure.phabricator.com/book/phabricator/article/herald/ Herald -> gerrit stream events adapter should be an easy hack.
 * Or we can just get rid of Zuul… (We should, but the ability to kludge things temporarily during a migration would be good.)

Scap

 * Get to “100%” moved to Python

Performance

 * Speed up LocalFile locking behavior (https://gerrit.wikimedia.org/r/#/c/122550/) (Aaron’s patch; Ori to review.)
 * BagOStuff::lock in general should distinguish “locked” from “can’t connect”
 * PDF thumbnail pool counter use (coming soon to TIFFs)
 * GeoIP → cookie fully deployed
 * Ori, Timo & Roan working on frontend performance workshop for Zurich
 * Preliminary notes: https://etherpad.wikimedia.org/p/perfworkshop

HHVM

 * http://en.hhvm.beta.wmflabs.org/wiki/Main_page
 * Missing static assets: https://bugzilla.wikimedia.org/show_bug.cgi?id=63310
 * Probably the result of loading a different set of extensions.
 * Required hacking a package in an ugly way; waiting for a fix in RT 7133
 * Lightning talk for metrics meeting on HHVM with Flow as case-study (Ori + Erik B.)
 * Preliminary notes: http://etherpad.wikimedia.org/p/hhvm-lightning
 * Patches needing review:
 * https://gerrit.wikimedia.org/r/#/c/117362/ (HHVM support for Wikidiff2)
 * https://gerrit.wikimedia.org/r/#/c/120180/ (Add a handler for HHVM fatals)
 * https://gerrit.wikimedia.org/r/#/c/120469/ (HHVM support for FastStringSearch)
 * HHVM 3.0 released. http://hhvm.com/blog/4349/hhvm-3-0-0
 * 3.0 release brings exciting new headaches.
 * HHVM discussed during today’s TechOps meeting; waiting to hear how it went.

Performance profiling for JavaScript code

 * Something Ori will work on, but could be a bona fide project if Timo joins platform

Deployment tooling / RelEng

 * Beta migration to eqiad complete!
 * Upgraded elasticsearch and kibana for logstash cluster in prod and beta
 * Fixed in scap (errors in python scripts were logged but the exit status was not properly returned to the shell)
 * Still need those two config changes done before tomorrow(?)

Search

 * Going to all group1 wikis but commons/meta/incubator as primary on Wednesday
 * Elasticsearch 1.1.0 has been out for a week without another revision planned so I’m verifying against it
 * Nicer highlighter on the horizon.
 * This Thursday we’ll get some performance fixes on enwiki which we’ll use to run another round of performance tests
 * Search page redesign slowing down this sprint while we wait for some designs from Brandon

SecurePoll

 * Going to be moving the date picker into SecurePoll instead of core, apparently
 * Other than that, Brad has been being pulled to other bugs recently

ContactForm for trademark

 * Brad set up a Labs instance
 * We have the mediawiki-core-team project now! If Brad missed giving anyone access, everyone else is an admin and can add you.
 * Dan has pinged Yana for feedback on the form.