Wikimedia Release Engineering Team/Checkin archive/20131022

Presents: Chris S, Chris M. Zeljko, Andre, Antoine, Greg (by phone), Sam

Improve our deployment process

 * git-deploy
 * http://git-annex.branchable.com/devblog/day_39__git-recover-repository/
 * Greg'll be testing out git-deploy more in the coming week, reducing noise specifically, as a goal
 * automation
 * Flow tests looking nice right now. Not adding new ones for now, while some UI bugs get sorted
 * Željko spending significant time with Language team on test design for new tests and re-engineering existing tests
 * VisualEditor tests are turning up:
 * performance problems in Chrome compared to Firefox
 * behavior problems in Chrome compared to Firefox (VE pages in Chrome have different contents than in FF)
 * anomalous interactions between VE and ULS
 * probably bugs in WebDriver itself unfortunately. Right now cursor tests are flaky for every platform/browser except for FF/beta labs.  Chrome builds all have performance problems and test2wiki builds all have VE+ULS interaction problems
 * monitoring
 * https://ganglia.wikimedia.org/latest/?r=day&cs=&ce=&tab=v&vn=VisualEditor (from Ori)
 * Other examples; https://ganglia.wikimedia.org/latest/ -> on top click View

Take the Beta Cluster to the next level

 * monitoring of fatals, errors, performance
 * Hopefully beta will Just Work without SSL pending this setting change merged: https://gerrit.wikimedia.org/r/#/c/90670/1
 * getting actual certs remains important
 * https://bugzilla.wikimedia.org/48501 (RT 5184, nothing interesting there).
 * automated support for Messages for Flow (and for any extension in beta but not in production) is working nicely
 * add more automated tests for eg the API
 * sync articles/gadgets from production https://bugzilla.wikimedia.org/show_bug.cgi?id=49779
 * feed experiences/gained knowledge of Beta Cluster automation up to production automation
 * small steps: see the comments from Antoine and me on wikitech-l Oct 20/21 under "Optimizing the deployment train schedule"

Better align QA effort with high profile features

 * Apply model of Language/Mobile embedded QA to a new feature team (specifically VisualEditor)
 * Željko paired with Jeff Hall on test tools
 * Chris invited Jeff to pair on an in-depth look at the current issues for VisualEditor browser tests, set for Friday afternoon.
 * Chris preparing to set some first charters for a VisualEditor exploratory tester: behavior and performance in Chrome is our biggest potential issue right now.
 * Željko extracted code shared between repositores into a Ruby gem: https://github.com/wikimedia/mediawiki-selenium
 * more code needs to be extracted
 * Include more user contributed code testing (eg: Gadgets)
 * How is Tomislav doing? continuing along?
 * The new test for the ProveIt gadget pointed up an additional problem with VisualEditor: see the attachment for https://bugzilla.wikimedia.org/show_bug.cgi?id=55801
 * The new test for the ProveIt gadget also points out an issue with ChromeDriver. Because ProveIt always floats over the edit page, Selenium tests in Chrome always fail: https://bugzilla.wikimedia.org/show_bug.cgi?id=55751 . This is arguably a bug in ChromeDriver (under discussion now by W3C) but it's also questionable behavior from ProveIt.
 * Gadget sync from production is bug https://bugzilla.wikimedia.org/49779
 * Increase capacity through community training for browser tests
 * Right now training the Language team is the highest priority.
 * pairing with Niklas and Amir weekly
 * The other high priority is training Jeff and bring on Rummana, pending all the stuff for that.
 * But we still have ongoing high-signal discussions on the QA list. Community is not dead.
 * Google Code-in looks like it's 99% going to happen.

Enhance deployment train
Please reply to thread "[Wikitech-l] Optimizing the deployment train schedule" http://lists.wikimedia.org/pipermail/wikitech-l/2013-October/072535.html

Misc

 * MediaWiki release
 * Jenkins job is under https://integration.wikimedia.org/ci/job/mediawiki-core-release/ should be triggered when a ref is updated or by triggering it manually and filling in ZUUL_REF something like ref/tags/1.22.0. Release script is in mediawiki/tools/release.git : /make-release/make-release.py
 * Markus working on a PHPUnit script to test out the resulting tarball (try to install, run unit tests, chec
 * Signing tarball
 * should be done manually for now, the CI Jenkins is not secure at all. Potential plan: create a new isolated Jenkins box restricted to a bunch of people, could be used to releae Debian packages / Maven java packages or MediaWiki release.