Wikimedia Platform Engineering/2011-06 Meeting

Latest meetings:etherpad:PlatformEng-Meeting

Past meetings:etherpad:GenEng-2011-03-23

---

Notes from GenEng-2011-06 meeting: Agenda:etherpad:GenEng-MeetingRoadmapWMF General Engineering

MediaWiki Core

• Tim Starling - now: 1.17.0, code review up next: Disk-backed object cache, Het Deploy, 1.18 deploy
• Chad Horohoe - now: auth proxy for research databases; GSoC mentorship, code review; PHPUnit cleanup. Up next: GSoC, code review & ?
• Sam Reed - now: API bugfixes, code review, GSoC mentorship. Up next: GSoC, code review & ?
• Priyanka Dhanda - now: hetdeploy, shell bugs, GSoC mentorship. Up next: hetdeploy & GSoC mentorship

MediaWiki.next

• Brion Vibber - Now: parser + Visual editor, code review
  • Brion, Trevor, Neil still working on early parts.
    • ParserPlayground gadget will add more of that code soon (brion doing prelim test work with CodeEditor -> brion)
    • More visible update on wikitext-l and blog posts planned for this week (delayed from end of last week) -> brion

   Sumana to help action surfacing the process - to meet with Brion Thursday re getting pages on watchlist, posting to wikitext-l, etc

• • • • editing surface work, some of the early dom tests -> trevor
      • combining dom transforms & planning for the editor communication connection -> neil
  • Erik's still grabbing Wikia folks for intermittent coordination; we expect to at least work with them on implementing plugins and UI extensions in the August-September timeframe, once we have a stabler base. Possibly some direct work with Inez also? (unsure exact status)

Analytics

• Erik Zachte - Wikimedia Report Card 2011
• Nimish Gautam - Wikimedia Report Card 2011

Technical Liaison; Developer Relations (tl;dr)

• Guillaume Paumier - Now: engineering project documentation
• Mark Hershberger - Now: bug triage, code review assignments
• Sumana Harihareswara - Now: GSoC, volunteer coordination. up next: engineering project documentation
• Siebrand Mazeland - Now: translatewiki.net. Up next: more localisation work

Misc :)

• RobLa - Now: Hiring, QA management, engineering project documentation

Hiring

Now:

• QA Lead (September start, but look now): wmf:Job openings/QA Lead
• Systems Engineer - Analytics: wmf:Job openings/Systems Engineer - Data Analytics

Future:

• Security Engineer (job description TBD)

   Focus on MediaWiki security. Lots of code review. Naturally inclined to do code review. We desperately need one of these people.

Please find us candidates for all three of these! Great candidates get hired, we will reshuffle teams where needed.

engineering project documentation

• Monthly reports - going well, need status

   we are very grateful to Guillaume for his informative monthly reports. Reporting needs to be done in a standardized way. Please consider talking to Guillaume so that you can report well and consistently.

• Project pages - not going well, need pages + status

   pretty easy to update these pages.  If you're working on a project that has no project page, go ahead & create it.
   Good examples?
           hetdeploy:Heterogeneous deploymenthetdeploy

   Template page:Template:Create new Wikimedia engineering project page

Some people already like to write weekly reports. But we aren't asking you to do it if you think it's a chore. Make sure projects have status - that's the important thing!

• Roadmap - MW Core has a roadmap here:

 NOT the deceptively namedRoadmap

fiendishly, the correct page isWMF General Engineering#Activities middle column: these are the projects we are trying to march through sequentially

The first thing that's not done yet == our top priority.

It took a while to properly express what MW Core team is doing & accurately capture continuous responsibilities, aux responsibilities, & main projects. It would be great if other projects had similar roadmaps.

Brion: should I have a page ordered roughly like this for the Future group? which is not entirely GenEng. RobLa: that would be great. And if that would mean "work on parser & figure out the rest of the roadmap" that would be respectable!

   ^ brion: reorder thehttp://www.mediawiki.org/wiki/Future page into this

• WMF General Engineering

[half an hour left]

See Wikimedia Foundation engineering project documentation

Now, statuses, as material for Guillaume and as "here's a blocker" signalling

1.17.0

bugzilla:26676MediaWiki 1.17 release tarball

minor nits at this point. nothing major. Tim: able to go to .0 early next week. "Yay", said the team and there was much rejoicing!

Code review

wikitech-l conversation. re 20% time being dedicated. all fulltime engineers dedicate 20% of their time to chores such as code review. Bugfixing, shell requests, documentation(!?), and so on. Please weigh in on the mailing list & encourage others to do the same.

Code review is a core responsibility to us. But everyone needs to consider what it means to make a 20% commitment, and accommodate that in their scheduling. EPMs, team leads, etc.

Siebrand says: IMO a big advantage for code review by feature engineers outside of their area of attention is that they stay (more) in touch with the rest of the code base, keeping them versatile.

Brion on code review philosophy & practice:

   Ideal env: seeing stuff right as it's coming in, fairly fresh, just-written or just-applied, and we have a chance for people more familiar with existing codebase to look at it, see that it is as advertised, check re i18n & testcases, bugs, security.  Ask for fix or - if not immediately fixable, back it out right away with explanation of what's wrong. Get conversation with author, get fixed, get reapplied in fixed state
       Ideal - very new, can grab person online synchronously or quickly
   Backlog code review: more difficult. Many months behind actual change. Lost some context & whatever is not explicitly said in commit comments.  To ask person who wrote it, they have to reach back into memories.  In following 9 months, was this code changed in various ways? very difficult to simply back out old code.  Piles of things have used it.
   This breaks things up & it's harder.
   Brion's preferences: quick feedback loop, keep running as fast as possible.

Tim likes to review code of people who are on IRC at the time, or email them and tell them to get on IRC. [the comment system lets us do async communication now at least, but it's still better when folks can be reached immediately] RobLa: role for Sumana and Mark to highlight when these folks are online & encourage them to ... when people are available, right there ready for a code review, what's the best way to make that known? ...

Siebrand: In the early days we had a policy of "commit while on IRC and stick around for at least 5 mins".

Suggestion is made to create some bot functionality to notify relevant devs about revent commits.

Brion: if we keep up with current code review, this'll naturally happen. The bot already exists in #mediawiki . So we already have the stream of new info RobLa: but it doesn't work re backlog.

So then someone has to start pinging re that revision. 500 right now for 1.18? http://toolserver.org/~robla/crstats/crstats.118phase3.html [based on query to toolserver]

[20 mins left]

what is it that will "definitely help"? lots of extension updates we will immediately mark deferred, not a priority. Chad: but most of those to defer: we have pruned out. The remaining list we must look at.

Mark's next step: ensure chad is right on that -- DONE!

Disk-backed object cache (DBOC)

Tim! delayed due to release, will have time for it now. We will borrow some of Asher's (ops) time to help with this. Hopefully relatively quick. there is a project page!Disk-backed object cache

Heterogeneous Deployment (aka hetdeploy)

Tim has been reviewing code add features, clean it up PD updating deployments on prototype. making sure meeting scripts work Tim & PD to coordinate on what else needs to be done after this, catch edge cases & other tasks in sync with 1.18

RobLa: we have talked about deploying well before 1.18 ... we need a test wiki. test.wikimedia.org runs straight off a machine?? a bad test case.

PD: we should have a separate... separate document root, looks at same code set up another DNS.... Tim & PD to hash this out some more.

PD will update status page ASAP.

MW 1.18 deploy

dependent on hetdeploy & code review.

Siebrand: having some metrics/KPIs would be nice/describe the process better.

Research DB Auth proxy

Chad!

• Erik directing him to work on proj: Wikimedians & external research databases

Ryan Lane also working on this OpenID!!!!!! RyanLane also wants to use this in a few other places Chad waiting for Ryan to have more time after IPv6 & SSL work Chad reviewing OpenID extension to ensure suitability for WMF deploy Mark: someone started maintaining this. Tom Gries Chad: yes. Chad's reviewing that. Siebrand: what's the opinion on the extension? ready? Chad: some more time Brion: a couple of months. Chad agrees

PHPUnit cleanup

• We fixed this today!!

CruiseControl doing builds as it should.

[10 mins left]

GSoC mentorship

• Summer of Code 2011
  • Sam, Priyanka, Sumana and Chad should talk later

API bugfixing

• Moving along!

Commit access

All ur SVN are belong to Sumana.

• now. Moving swiftly.

Job queue monitoring

• RobLa to talk to Tim about postponing this till after virtualization framework so Sam can dev against a play production env -- DONE

trouble getting a test set up? torrus? no, just difficulties with udpprofile CGI versions etc.

Configuration database

Chad: Tim & Chad talked weeks ago. Chad needs to work, merge stuff. backburner.

Quality Assurance

RobLa: in addition to automated stuff & hiring: some other options

Parser

(see updates at top of page) - line 21

Visual editor

(see updates at top of page)

Wikimedia Report Card 2011

we are getting requirements replace existing report card, which requires awful labor-intensive processes "bring this to someplace past 1994" there's a project page. Erik & Nimish

volunteer coordination

Sumana is doing a bunch of stuff. Look at her public page for more info.User:Sumanah/TODO Last week: parser, hiphop, wikimania This week: (missed it)

• Sumana: need to talk to Tim about HipHop timeline -- obsoleted

translatewiki.net

Siebrand: Recenty ended a fantastic Translation Rally https://translatewiki.net/wiki/Thread:Project_talk:Rally-2011-04/Results. This one was for all 20 supported projects. We're continuing the improvement of the Translate extension. Niklas was recently at a KDE hacking event, where focus was on improvement of the page translation feature, which we hope to be able to enable within a few months on a Wikimedia project. Raimond is committing L10n updates for core and extensions on a daily basis. In Berlin, Tim ran usage stats for MediaWiki messages on the cluster. We have an updated set of about 500 MediaWiki messages again that are "most used", so that a translator can make a very visible initial difference for their language with the least possible effort (that may still be major). Everything is running smoothly.

Q by Siebrand: Is any review of Translate with WMF focus currently planned, or will Niklas be doing that himself within the next few months, with some additional review for security?bugzilla:28021 Mark talked with Alolita about getting Roan to review Translate, in July when they are both here.

Q by Siebrand: A while ago, Roan was planned to review Babel. Because of an unfortunate misunderstanding, this was postponed. Is this still on any roadmap?bugzilla:15308 Mark: will be a similar thing, Roan & Trevor to do it at same time maybe (July)? Tim: looked at this at some stage, looked pretty harmless?

   in bug report, comment at bottom, Purodha? is working on this & don't want any review, come back later
   problem - comment from March!
   rollback and branch and so on?
   [inaudible]

revert the new code & use whatever's there? all ext does is - userbox templates for languages, this ext generates those. someone needs to review latest changes & figure out whether they can go in or not. Tim can assign this to someone.

Questions? Rob on vacation next week! Siebrand on vacation 30 July - 17 August Sumana on vacation next week

Here you go, Guillaume!