Wikimedia Release Engineering Team

This is the team responsible for Release Engineering/Management and QA at the Wikimedia Foundation. We predominately use the QA mailing list along with conversations on the appropriate IRC channels (especially, , , and ).

= Status =

= Quarterly Reviews =
 * April 2014
 * February 2014
 * November 2013
 * August 2013

= Wishlist =

We maintain a list of projects that would be great to have done, but are not on our roadmap in in the near term. See the wishlist here.

= Meetings =

See /Meetings for notes from the Showcase meetings of the Release and QA Team.

= April - July '14 Goals Progress =
 * For past quarter's checkins, see the archives.
 * See the WMF Engineering 13-14 goals page for the yearly view.
 * Mid-quarter checkin (June 27th)

Deployment tooling

 * - Process through all (useful) pain points from the Dev/Deploy review session - (Greg)
 * - Integrate HHVM support into our deployment systems - (Bryan, Greg, Mukunda, others from Platform)
 * Waiting on HHVM packaging for Ubuntu.
 * - continue the scap(py) (stretch goal, finally prioritized since it adds a lot to our deployment workflow)
 * All scap scripts have been migrated to python minus one
 * - trebuchet integration conversation (stretch goal)

Phabricator
New project since this quarter
 * Release engineer hired (Mukunda)
 * - SUl/OAuth integration - (Mukunda)
 * Two patches in gerrit:
 * https://gerrit.wikimedia.org/r/#/c/139438/
 * https://gerrit.wikimedia.org/r/#/c/139442/
 * - Security component - (Mukunda)
 * - Puppetization with ops (Chase) - (Mukunda)

Beta cluster

 * - Support HHVM deployment tooling and puppet configuration testing - (Bryan, Antoine, Mukunda)
 * Waiting on HHVM packaging, puppet refactor for Trusty.
 * - Swift cluster in beta (stretch goal)
 * Andrew Bogott/Filippo created preliminary puppet classes that matches labs environment
 * merged last night - https://gerrit.wikimedia.org/r/#/c/137803/
 * - assist language team to deploy cxserver - added mid-quarter - (Antoine) (added mid-quarter)

MediaWiki Release

 * - Successfully support the release of MediaWiki 1.23 - (Antoine, Greg)
 * - Kickoff/complete second RFP - (Greg)
 * - Investigate and create useful release/deployment metrics visualizations - (Greg) - (stretch goal)

Browser tests

 * - Use tags to run builds appropriate to released versions (e.g. don't run master build on test2wiki) - (Chris)
 * dependent on the two below
 * - Retire Cloudbees Jenkins instance - (Željko)
 * - Retire /qa/browsertests repo by porting tests to /mediawiki/core and appropriate extension repos (Chris) (added mid-quarter)
 * - Integrate WMF Jenkins with new WMF SauceLabs account - (Željko, Antoine, Chris)
 * - Use API to create test data at runtime more widely (not just for MobileFrontend but also VisualEditor, Flow, local dev env etc.) - (Chris, Željko) Calling this done for the quarter, but we'll always be using it into the future
 * Used by MobileFrontend
 * Used by VisualEditor
 * Used by smoke tests (part of quarterly goals)
 * - Add browsertests to new repos e.g. GettingStarted - (Chris, Željko)

Engineering Liasion Work

 * - Pair with teams and community members to improve QA practices
 * Language team
 * Wikidata team
 * Commuity members
 * - GSOC mentoring

Vagrant
New project since this quarter
 * Automation Engineer hired (Dan)
 * - Improve Vagrant setup process (especially re browser tests) and clean up Ruby according to style guide
 * - VE browser tests in Vagrant
 * - Ruby style guide

Hiring

 * - Complete hiring and train new Release Engineer (Greg, all)
 * - Complete hiring and train new Automation Engineer (Ruby)  (Chris, all)
 * - Complete hiring of a new QA Tester (Chris, Greg) - added mid-quarter

Quarterly Review Action items

 * - create a plan for browser testing of MediaWiki 1.23 - Chris M
 * gerrit changes ready to be merged
 * TODO: send announcement to wikitech-l post merge
 * - Greg get firm requirements from Antoine, circle back to Mark who'll have an idea of Op's timelime for production Shinken (monitoring system to maybe replace Icinga)
 * no deadline knows on ops side. Might not be high priority
 * ops in charge: Alexandros (Greece based)
 * maybe use it in Beta Cluster as a POC
 * - Figure out how to keep HHVM unit tests from delaying +2 for standard production commits (Antoine, Chris M, Zeljko)

Greg

 * - Process through pain points
 * - WMF product should be consulted on some shellbugs
 * - Successfully support the release of MediaWiki 1.23
 * - Kickoff/complete second RFP
 * - Refine Deployment_tooling/Notes/Deployment_system_requirements with Bryan
 * - Investigate and create useful release/deployment metrics visualizations

Antoine

 * - Support HHVM deployment tooling and puppet configuration testing
 * - Integrate WMF Jenkins with new WMF SauceLabs account
 * - RFC for Jenkins tests isolation
 * wrote and published to ops and engineering lists
 * agreement by End of June
 * POC by End of July
 * - RFC for cross repositories testing and integration job in gate
 * - implementing cross repository cloner (aka have repositories match branches/zuul ref)
 * - assist language team to have cxserver deployed to beta cluster

Mukunda

 * - read all the things
 * - WMF SUL/OAuth support T40
 * - security component support in Phab Upstream T4893, our phab T95
 * local 6 line hack
 * pinging evan regularly on real long term fix
 * - puppet wmflib beginnings

Bryan

 * Scap refactor/python port
 * Repository .git is synchronised fine, but is broken for submodules because of hardcoded gitdir link
 * ✅ Build JSON git info caches during scap
 * ✅ support in GitInfo.php
 * Code is deployed in beta (and prod) but it doesn't work as expected. See notes on bug for where I'm stuck.
 * ✅ Fix sync-common for hosts in wikimedia.org TLD
 * - Integrate HHVM support into our deployment systems
 * ✅ Met with Emir from Facebook to discuss HHVM and deployment in general
 * Next generation deployment tooling
 * Support train deploys
 * ✅ Helped with deployment training hangout

Chris

 * - migrate to WMF Jenkins
 * - create a plan for browser testing of MediaWiki 1.23
 * https://bugzilla.wikimedia.org/show_bug.cgi?id=66272
 * See https://gerrit.wikimedia.org/r/#/c/133507/
 * Chris Steipp had suggestions in email for further tests that map to legacy tests in /qa/browsertests repo
 * After discussion with Rob Lanphier, will port appropriate tests from /qa/browsertests to /mediawiki/core with Dan Duvall over time
 * - Use tags to run builds appropriate to released versions (e.g. don't run master build on test2wiki)
 * Antoine fleshing out a start of this idea at Zurich
 * - Use API to create test data at runtime more widely (not just for MobileFrontend but also VisualEditor, Flow, local dev env etc.)
 * Done for Mobile
 * Done for VisualEditor
 * Done for browser smoke tests
 * - hire new QA Tester

Dan

 * Read all the things
 * - VE broswer tests in Vagrant
 * - MobileFrontend broswer tests in Vagrant

Greg

 * - Process through pain points
 * - config changes not being deployed after merge (RT ticket filed], successfully in Ops' court)
 * - WMF product should be consulted on some shellbugs
 * email sent to WMF Product on 5/15
 * - onboard new Release Engineer
 * ongoing
 * - Successfully support the release of MediaWiki 1.23
 * - Kickoff/complete second RFP
 * - Refine Deployment_tooling/Notes/Deployment_system_requirements with Bryan
 * - Investigate and create useful release/deployment metrics visualizations

Antoine

 * - Figure out how to keep HHVM unit tests from delaying +2 for standard production commits
 * - Support HHVM deployment tooling and puppet configuration testing
 * - Integrate WMF Jenkins with new WMF SauceLabs account
 * - RFC for Jenkins tests isolation
 * - RFC for cross repositories testing and integration job in gate
 * - RFC for cross repositories testing and integration job in gate

Mukunda

 * - read all the things
 * - WMF SUL/OAuth support T40
 * - security component support in Phab Upstream T4893, our phab T95

Bryan

 * Scap refactor/python port
 * ✅ Setup test environment in beta
 * Repository .git is synchronised fine, but is broken for submodules because of hardcoded gitdir link
 * ✅ Build JSON git info caches during scap
 * ✅ support in GitInfo.php
 * Code is deployed in beta (and prod) but it doesn't work as expected. See notes on bug for where I'm stuck.
 * ✅ Use trebuchet to deploy scap scripts
 * Add scap/scap trebuchet target
 * Provision scap scripts using trebuchet
 * ✅ source organization cleanup
 * ✅ Change default console logging verbosity (less noise for deployers)
 * ✅ Convert sync-* to python
 * Extract common base class for sync operations
 * Convert sync-dir and sync-file to python
 * Convert sync-dblist to python
 * Convert sync-docroot to python
 * - Integrate HHVM support into our deployment systems
 * Next generation deployment tooling
 * Support train deploys
 * ✅ Created Train deploy instructions on wikitech based on noted from Sam and experience in March

Chris

 * - create a plan for browser testing of MediaWiki 1.23
 * See https://gerrit.wikimedia.org/r/#/c/133507/
 * Chris Steipp had suggestions in email for further tests that map to legacy tests in /qa/browsertests repo
 * After discussion with Rob Lanphier, will port appropriate tests from /qa/browsertests to /mediawiki/core with Dan Duvall over time
 * - Use tags to run builds appropriate to released versions (e.g. don't run master build on test2wiki)
 * Antoine fleshing out a start of this idea at Zurich
 * - Use API to create test data at runtime more widely (not just for MobileFrontend but also VisualEditor, Flow, local dev env etc.)
 * Done for Mobile
 * Done for VisualEditor
 * Done for browser smoke tests
 * - hire new QA Tester

Dan

 * Read all the things