Wikidata deployment/guide

Configuration changes
Wikidata configuration is in https://git.wikimedia.org/summary/operations/mediawiki-config.git


 * wmf-config/Wikibase.php
 * wmf-config/InitialiseSettings.php
 * wmf-config/CommonSettings.php

People listed on https://gerrit.wikimedia.org/r/#/admin/groups/21,members can approve changes and deploy them.

Normally aude or hoo handles these, but one can also ask Reedy, Greg G. or sign up for a SWAT deploy (where someone will deploy for you).

There no deploys on Fridays (and weekends) except for emergencies.


 * Puppet changes

dispatch changes and some other stuff is run as a cron job. Cron jobs are configured in puppet, in "manifests/misc/maintenance.pp"

https://git.wikimedia.org/blob/operations%2Fpuppet.git/55e76106e0e976c5d56ccb2a54921ad90bbe764d/manifests%2Fmisc%2Fmaintenance.pp

Any cron jobs that you want to stop need to be marked as "absent" and do not get removed from puppet manifest.

To change a cron job, add a new cron job and mark the old one as "absent".

Make a new deployment branch of Wikidata build
On command line with latest master of Wikibase, replacing branch name in example with appropriate branch name:

Note: It's also possible to make a branch of Wikibase using gerrit interface.

Then

Update mediawiki/tools/release.git

 * Before a new core MediaWiki branch gets cut on Thursday, update Wikidata branch in the  file in http://git.wikimedia.org/summary/mediawiki%2Ftools%2Frelease.git then submit to gerrit and make sure Reedy or whoever is deploying sees / approves.  (aude and hoo can also approve it)   This can be done on Wednesday (ideally) or Thursday morning UTC.
 * If not done in time, then we need to update Wikidata submodule in the MediaWiki core deployment branch and ensure the submodule update gets deployed.

Document stuff

 * Ahead of time, document important changes and bug fixes that are in the new branch on Wikidata deployment. This helps keep a record of what changed /when, things to announce to the community, and be on look out for any issues after deployment related to those changes.
 * On Wikidata deployment, important to document any deployment notes (e.g. config changes needed).

To help search for things to document, you can search commit messages via git log, with the --grep option:

Deployment time

 * Make sure all config changes and other todos in the deployment notes section of Wikidata deployment (for the relevant branch) are done.
 * Wikibase etc. automatically gets deployed along with new branches of MediaWiki core at the regularly scheduled core deployment times (Thursdays for test.wikidata / test2, Tuesdays for wikidata/wikivoyage/etc., Thursdays for Wikipedias). See Wikidata deployment and Deployments for schedules and details.  Reedy or another WMF person will deploy.
 * When the wikis get switched to new version of core + Wikibase / Wikidata build, then check the wikis to be sure things are good.
 * At deployment time, hang out on #wikidata and #wikimedia-operations irc channel in case problems occur with our stuff.
 * On Wikidata deployment, mark the deployment as done.
 * Inform Lydia that deployment is done and she will handle announcements and communications to the community.

Deploy hotfix
Hot fixes are critical bug fixes / fix regressions and are to be avoided, if possible, or kept to a minimum.

If a hot fix is needed for deployment, then follow these steps:


 * cherry pick changes into Wikibase.git deployment branch (hint: gerrit has a cherry pick button)
 * wait for WikidataJenkins to approve (and manually verify also, as needed, to be sure!)
 * +2 code review and wmf jenkins will merge
 * then update Wikidata.git deployment branch, per steps below. (you might need to wait 5 minutes or so for packagist / composer to find the new change in Wikibase.git)
 * (be sure to double check the submodule commit hash for Wikidata.git is correct)
 * Poke aude, hoo, Reedy, Greg G, or another deployer or sign up for a SWAT deploy (where someone will deploy for you).
 * Once deployed, double check test.wikidata / test2.wikipedia/ wikidata etc. to verify the hotfix
 * Hang out on #wikimedia-operations and #wikidata irc channels in case problems occur

Note: there no deploys on Fridays (and weekends) except for emergencies.

Make a Wikidata build for master / beta deployment
Builds for beta are generated daily on a labs instance, which is in the Wikidata-build project. Both WikidataJenkins and wmf-jenkins will run tests on any new builds. Once they both approve, then someone needs to +2 code review and then jenkins will merge the new build. Builds for master branch of Wikidata.git will be deployed automatically to beta.

Make a Wkidata build for branches / production deployment
You can use the WikidataBuilder tool, which is essentially a wrapper around composer and uses grunt.

Setup a build_config directory for the branch in your local copy of WikidataBuilder:
 * 1) Setup
 * Copy Wikidata_master directory
 * Change composer.json to require "dev-BRANCHNAME" (e.g. "dev-mw1.23-wmf21") of Wikibase instead of "dev-master"
 * In config.js, change update: BUILD_DIR: 'Wikidata_master' to BUILD_DIR: 'Wikidata_BRANCHNAME' (e.g. BUILD_DIR: 'Wikidata_mw1.23-wmf21' )

From base directory of WikidataBuilder, run:
 * 2) Make a build


 * 3a) Make new branch for Wikidata.git


 * 3b) Update Wikidata.git branch

Both WikidataJenkins and wmf jenkins will run tests on the build. Be patient and wait for them.
 * 4) Wait for jenkins!

Once both the jenkins approve and after you check the changes on gerrit look sane, then you can +2 code review. jenkins (wmf) will then merge the update, running gate-and-submit with full tests again.
 * 5) +2 code review and merge

Update core submodule

 * See how to update extension submodule in MediaWiki core deployment branch: How to deploy code