GitLab/Roadmap

The migration of code hosting, code review, continuous integration, and existing delivery mechanisms for all Wikimedia developed projects is a large undertaking that will necessarily proceed in stages. Each stage involves mirroring a new set of repositories; communication and on-boarding a new cohort of users; and configuring new sets of permissions.

1. Groundwork
Consultation, documentation, vetting contractors, and buying hardware

Groups involved
Technical community, Engineering Productivity, Site Reliability Engineering, Security, WMF Legal, GitLab contractor

Activities

 * GitLab community consultation
 * Document code review and continuous integration workflows to include GitLab
 * Produce a roadmap
 * Engage with WMF Legal for ToS/Privacy Policy updates
 * Begin hiring process for GitLab contractor
 * Identify and estimate hardware use for GitLab components

2. Setup
Install and provision GitLab, create runbooks for upgrade and common administration tasks.

Groups involved
Engineering Productivity, Site Reliability Engineering, Datacenter operations, Security, GitLab contractor

Activities

 * Build a pre-production testing environment
 * Explore and establish login and authentication options
 * Rack, install, and setup GitLab for production
 * Establish deployment and upgrade cadence

3. Utility project migration
Now that we have a solid foundation we can start migrating some early adopter repositories with limited stakeholders. We expect to get to this point, with a few of these repositories migrated, by the end of June 2021.

Groups involved
Engineering Productivity, Site Reliability Engineering, Security

Activities

 * Implementation of the Gerrit Privilege Policy on GitLab.
 * Mirror code from Gerrit to GitLab.
 * Ensure continuous integration triggers and runs on GitLab hosts.
 * Adapt notification bots (IRC/Slack) for GitLab.
 * Adapt integrations with Phabricator
 * Migrate code review to GitLab for these repos.
 * Test out and build (if necessary) support for dependent patchset workflows.
 * Set these Gerrit repositories to read-only.
 * Reach out to WMCS re:striker

4. Deployment Pipeline services
From the learnings of the earlier adopters we can start to migrate more repositories more quickly, specifically those services deployed via the Deployment Pipeline.

Groups involved
Engineering Productivity, Site Reliability Engineering, Platform Engineering, Security

Activities

 * Build GitLab workers for secure Docker image creation.
 * Migrate relevant bots for these repositories
 * Plan with SRE and Security:
 * Continuous build and integration for each merge request for each repository.
 * Continuous delivery of image artifacts to production infrastructure.
 * Migrate code review to GitLab for these repos.
 * Set these Gerrit repositories to read-only.

5. Everything except MediaWiki+Extensions+Skins
Leaving MediaWiki for after the bulk of the repositories are moved allows us to learn a lot about the idiosyncrasies of the new system and anticipate needed changes for MediaWIki.

This includes Phabricator Diffusion repositories created for WMCS users via Striker.

Groups involved
Engineering Productivity, Product and Technology department teams

Activities

 * Migrate relevant bots to these repositories
 * Migrate code review to GitLab for these repos.
 * Set these Gerrit/Diffusion repositories to read-only.

6. MediaWiki+Extensions+Skins
The final large group migration. This migration should be the best informed from what we’ve learned and the improvements we’ve made along the way.

Groups involved
Engineering Productivity, Product and Technology department teams

Activities

 * Re-implementation of developer and release tools:
 * Ensure that deployment tooling has support for GitLab.
 * Retool metric collection to support GitLab.
 * Migrate code review to GitLab for these repos.
 * Set these Gerrit repositories to read-only.

7. Decommission Gerrit
We have migrated all active code review to GitLab and now we can put in place a read-only/static view of Gerrit for historical URL preservation purposes. This is a big project itself.

Groups involved
Engineering Productivity, Site Reliability Engineering

Activities

 * Gerrit becomes read-only
 * Create static dumps of historic reviews with reasonable efforts to preserve URLs
 * Important URLs: Change-Id, SHA1, unique ID, anchors to review comments(?)
 * Gerrit servers decommissioned
 * Ask GitHub repos to move to GitLab