GitLab/Roadmap

From mediawiki.org
Jump to navigation Jump to search

The migration of code hosting, code review, continuous integration, and existing delivery mechanisms for all Wikimedia developed projects is a large undertaking that will necessarily proceed in stages. Each stage involves mirroring a new set of repositories; communication and on-boarding a new cohort of users; and configuring new sets of permissions.

1. Groundwork[edit]

Consultation, documentation, vetting contractors, and buying hardware

Groups involved[edit]

Technical community, Engineering Productivity, Site Reliability Engineering, Security, WMF Legal, GitLab contractor

Activities[edit]

  • GitLab community consultation
  • Document code review and continuous integration workflows to include GitLab
  • Produce a roadmap
  • Engage with WMF Legal for ToS/Privacy Policy updates
  • Begin hiring process for GitLab contractor
  • Identify and estimate hardware use for GitLab components

2. Setup[edit]

Install and provision GitLab, create runbooks for upgrade and common administration tasks.

Groups involved[edit]

Engineering Productivity, Site Reliability Engineering, Datacenter operations, Security, GitLab contractor

Activities[edit]

  • Build a pre-production testing environment
  • Explore and establish login and authentication options
  • Rack, install, and setup GitLab for production
  • Establish deployment and upgrade cadence

3. Utility project migration[edit]

Now that we have a solid foundation we can start migrating some early adopter repositories with limited stakeholders. We expect to get to this point, with a few of these repositories migrated, by the end of June 2021.

Groups involved[edit]

Engineering Productivity, Site Reliability Engineering, Security

Activities[edit]

  • Implementation of the Gerrit Privilege Policy on GitLab.
  • Mirror code from Gerrit to GitLab.
  • Ensure continuous integration triggers and runs on GitLab hosts.
  • Adapt notification bots (IRC/Slack) for GitLab.
  • Adapt integrations with Phabricator
  • Migrate code review to GitLab for these repos.
    • Test out and build (if necessary) support for dependent patchset workflows.
  • Set these Gerrit repositories to read-only.

4. Deployment Pipeline services[edit]

From the learnings of the earlier adopters we can start to migrate more repositories more quickly, specifically those services deployed via the Deployment Pipeline.

Groups involved[edit]

Engineering Productivity, Site Reliability Engineering, Platform Engineering, Security

Activities[edit]

  • Build GitLab workers for secure Docker image creation.
  • Migrate relevant bots for these repositories
  • Plan with SRE and Security:
    • Continuous build and integration for each merge request for each repository.
    • Continuous delivery of image artifacts to production infrastructure.
  • Migrate code review to GitLab for these repos.
  • Set these Gerrit repositories to read-only.

5. Everything except MediaWiki+Extensions+Skins[edit]

Leaving MediaWiki for after the bulk of the repositories are moved allows us to learn a lot about the idiosyncrasies of the new system and anticipate needed changes for MediaWIki.

Groups involved[edit]

Engineering Productivity, Product and Technology department teams

Activities[edit]

  • Migrate relevant bots to these repositories
  • Migrate code review to GitLab for these repos.
  • Set these Gerrit repositories to read-only.

6. MediaWiki+Extensions+Skins[edit]

The final large group migration. This migration should be the best informed from what we’ve learned and the improvements we’ve made along the way.

Groups involved[edit]

Engineering Productivity, Product and Technology department teams

Activities[edit]

  • Re-implementation of developer and release tools:
    • Ensure that deployment tooling has support for GitLab.
    • Retool metric collection to support GitLab.
  • Migrate code review to GitLab for these repos.
  • Set these Gerrit repositories to read-only.

7. Decommission Gerrit[edit]

We have migrated all active code review to GitLab and now we can put in place a read-only/static view of Gerrit for historical URL preservation purposes. This is a big project itself.

Groups involved[edit]

Engineering Productivity, Site Reliability Engineering

Activities[edit]

  • Gerrit becomes read-only
  • Create static dumps of historic reviews with reasonable efforts to preserve URLs
    • Important URLs: Change-Id, SHA1, unique ID, anchors to review comments(?)
  • Gerrit servers decommissioned