SQL/XML Dumps/Becoming a dumps co-maintainer/Access

Being a maintainer and developer for the dumps is a full stack job; we do capacity planning, coordinate orders of hardware, manage puppet configuration, write, package and deploy the code, monitor hardware and software issues, deal with upgrades of MediaWiki, PHP and Debian, and acommodate new requests for dumps. As such, there are a number of systems to which you will need specific kinds of access.

Merging, testing and deploying code
To be a co-maintainer of the dumps with the ability to run all the tests you need to run, and merge all the patches in all the repos and deploy them, you will need:

Puppet

 * +2 in the puppet repo
 * requires ops group in ldap, or someone with it to merge for you (hnowlan, ariel)
 * root on puppetmaster to deploy puppet patches
 * requires +2 in puppet, as these must be done together
 * root on target hosts (dumpsdata, labstore1006,7, snapshot hosts) to disable, enable and run puppet
 * needs new group dumps-roots in puppet with specific sudo privileges
 * ability to run Puppet compiler jobs
 * requires the wmf group in ldap, which all WMF developers should have

dumps code (python)

 * +2 in python dumps repo
 * requires membership in ops ldap group, or someone who does (hnowlan, ariel)
 * deployment group on deployment hosts for scap
 * requires membership in platform-engineering group in puppet
 * admin access to deployment-prep for testing
 * requires getting an existing admin to add you via horizon

dumps code (C utils)

 * +2 in mwbzutils repo
 * root on apt repo host to add packages
 * root on target hosts (snapshot hosts) to install/update packages

dumps code (MediaWiki core, ActiveAbstract extension)

 * +2 in core and ActiveAbstract repos
 * deployment group on deployment hosts for deployment of backports

Server maintenance
Sometimes we need to upgrade to a new OS, add or replace servers, and so on. For these tasks you will need:


 * root access to the cumin hosts, for re-imaging and other commands
 * root access to the management interface of the snapshot and dumpsdata hosts, for re-imaging, etc
 * root access to the snapshot and dumpsdata hosts, to
 * access to icinga to downtime hosts/services or ack alerts
 * access to grafana and other monitoring services, to check the state of the servers
 * +2 in operations/dumps/scap repo for adding or removing targets from the dumps scap list

Watching for incoming issues
You'll need to be able to get warnings about issues as they come up, and notifications of patches to review.


 * membership in the ops-dumps@wikimedia.org mail alias, for various statistics and exception reports
 * get added automatically as a reviewer to relevant gerrit changes in MediaWiki
 * become a member of (but not necessarily watch) the Phabricator dumps-generation project
 * subscribe to and become a moderator of the xmldatadumps-l mailing list