GitLab/Roadmap
The migration of code hosting, code review, continuous integration, and existing delivery mechanisms for all Wikimedia developed projects is a large undertaking that will necessarily proceed in stages. Each stage involves mirroring a new set of repositories; communication and on-boarding a new cohort of users; and configuring new sets of permissions.
1. Groundwork[edit]
Consultation, documentation, vetting contractors, and buying hardware
Groups involved[edit]
Technical community, Engineering Productivity, Site Reliability Engineering, Security, WMF Legal, GitLab contractor
Activities[edit]
- GitLab community consultation
- Document code review and continuous integration workflows to include GitLab
- Produce a roadmap
- Engage with WMF Legal for ToS/Privacy Policy updates
- Begin hiring process for GitLab contractor
- Identify and estimate hardware use for GitLab components
2. Setup[edit]
Install and provision GitLab, create runbooks for upgrade and common administration tasks.
Groups involved[edit]
Engineering Productivity, Site Reliability Engineering, Datacenter operations, Security, GitLab contractor
Activities[edit]
- Build a pre-production testing environment
- Explore and establish login and authentication options
- Rack, install, and setup GitLab for production
- Establish deployment and upgrade cadence
3. Utility project migration[edit]
Now that we have a solid foundation we can start migrating some early adopter repositories with limited stakeholders. We expect to get to this point, with a few of these repositories migrated, by the end of June 2021.
Groups involved[edit]
Engineering Productivity, Site Reliability Engineering, Security
Activities[edit]
- Implementation of the Gerrit Privilege Policy on GitLab.
- Mirror code from Gerrit to GitLab.
- Ensure continuous integration triggers and runs on GitLab hosts.
- Adapt notification bots (IRC/Slack) for GitLab.
- Adapt integrations with Phabricator
- Migrate code review to GitLab for these repos.
- Test out and build (if necessary) support for dependent patchset workflows.
- Set these Gerrit repositories to read-only.
4. Deployment Pipeline services[edit]
From the learnings of the earlier adopters we can start to migrate more repositories more quickly, specifically those services deployed via the Deployment Pipeline.
Groups involved[edit]
Engineering Productivity, Site Reliability Engineering, Platform Engineering, Security
Activities[edit]
- Build GitLab workers for secure Docker image creation.
- Migrate relevant bots for these repositories
- Plan with SRE and Security:
- Continuous build and integration for each merge request for each repository.
- Continuous delivery of image artifacts to production infrastructure.
- Migrate code review to GitLab for these repos.
- Set these Gerrit repositories to read-only.
5. Everything except MediaWiki+Extensions+Skins[edit]
Leaving MediaWiki for after the bulk of the repositories are moved allows us to learn a lot about the idiosyncrasies of the new system and anticipate needed changes for MediaWIki.
Groups involved[edit]
Engineering Productivity, Product and Technology department teams
Activities[edit]
- Migrate relevant bots to these repositories
- Migrate code review to GitLab for these repos.
- Set these Gerrit repositories to read-only.
6. MediaWiki+Extensions+Skins[edit]
The final large group migration. This migration should be the best informed from what we’ve learned and the improvements we’ve made along the way.
Groups involved[edit]
Engineering Productivity, Product and Technology department teams
Activities[edit]
- Re-implementation of developer and release tools:
- Ensure that deployment tooling has support for GitLab.
- Retool metric collection to support GitLab.
- Migrate code review to GitLab for these repos.
- Set these Gerrit repositories to read-only.
7. Decommission Gerrit[edit]
We have migrated all active code review to GitLab and now we can put in place a read-only/static view of Gerrit for historical URL preservation purposes. This is a big project itself.
Groups involved[edit]
Engineering Productivity, Site Reliability Engineering
Activities[edit]
- Gerrit becomes read-only
- Create static dumps of historic reviews with reasonable efforts to preserve URLs
- Important URLs: Change-Id, SHA1, unique ID, anchors to review comments(?)
- Gerrit servers decommissioned