Gerrit/Advanced usage

This page describes the workflow for how MediaWiki core and extensions developers use git and gerrit. (To come: git-review docs here provided we use it.)



The diagram on the right is an accurate description of what our git workflow will look like. (Transcription is welcome!) To understand how our gerrit workflow will work, see these pages:


 * https://labsconsole.wikimedia.org/wiki/Git#Review
 * http://wiki.openstack.org/GerritWorkflow
 * http://source.android.com/source/life-of-a-patch.html

Also useful (if you subtract away GitHub stuff):
 * http://wiki.openstack.org/GerritJenkinsGithub

The gerrit review workflow might change; Roan & Chad are investigating using git-review, a tool that would help us better use git and gerrit together. If you'd like to try it out, we're seeking test subjects! Also, John Du Hart would like for us to consider using Phabricator instead.

Easier access to commit
Everyone will be able to push to everything. There will be no distinction between core and extensions! Since code doesn't automatically make it into the "master" repo until pulled, there's no danger of bad commits unless someone pulls irresponsibly. Thus, for many of the git repositories, only a small group will be able to pull changes into the repository that the Wikimedia Foundation hosts. (This repository is the one that we assume most developers will set as their master.)
 * This is like how ops does it. Ops, core MW and deployed extensions, "gated trunk" model - needs review before merge.  For any other extensions, you get an option of doing a straight push.  ("post-commit review")

Install git
Install git on your computer.

Get the right permissions
Log in to https://labsconsole.wikimedia.org, verify that you can log in, then log in at https://gerrit.wikimedia.org with the same username and password. Click Settings and go to SSH Public Keys and paste in your public key (usually  on Linux) so our git repository will recognize you. (We aim to fix this soon by simply pulling the key from LDAP.)


 * Teacher: give the user a login using these instructions.

Clone the repository
Step 2 is to clone the repository in question that you're wanting to make a change on. Do this in your command line by:

git clone ssh:// @gerrit.wikimedia.org:29418/mediawiki/core.git
 * 1) Create or move to the directory on your machine that you do your development in. "wikimedia-git-repos" might be reasonable.
 * 2) typing (replacing mediawiki/core.git with mediawiki/extensions/FooBar.git for an extension or any other part of the repository)

Go have a sip of coffee while you wait for a minute :)


 * If you're doing the examples tutorial with Sumana, use this:

git clone ssh://USERNAME@gerrit.wikimedia.org:29418/test/mediawiki/extensions/examples.git DESCRIPTIVE-NAME

Prepare to work with gerrit
In order to properly work with gerrit, you need to have a pre-commit hook that adds a "change id" to your commit summary so gerrit understands how to work with it.

If you are on a Mac and do not have Macports installed, or you are on Windows, go to "Manual setup".

Git-review (easy!)
Install git-review, a tool to simplify working with Gerrit repositories so you don't have to remember some pretty confusing commands. Do those steps, then run: git review -s In your cloned copy to setup to work with Gerrit. It will probably ask you for your commit username. Then it will automatically install the pre-commit hook.

Manual setup
If installing git-review is not feasible for you, you'll need to download the hook and place it in the right directory in your cloned copy of the repository.

Change to the repository directory (for example, .  Then wget https://gerrit.wikimedia.org/r/tools/hooks/commit-msg > .git/hooks/

or if you don't have wget: curl https://gerrit.wikimedia.org/r/tools/hooks/commit-msg > .git/hooks/commit-msg

If neither of those works, download from https://gerrit.wikimedia.org/r/tools/hooks/commit-msg commit-msg and move it to the .git/hooks/ directory.

You also need to ensure the hook is executable. In Linux you do this with: chmod u+x .git/hooks/commit-msg

Next, add an alias to simplify the command to push changes to Gerrit for review. You can do this by executing the following from your repository clone: git config alias.push-for-review "push origin HEAD:refs/for/master" (The refs/for/ is gerrit magic and must not be omitted. However, you may adapt master to point to the remote branch that you want to commit to. E.g.: When trying to push to the remote branch Foo use refs/for/Foo.)

Commit a change
Make some changes in your working directory like you usually would, have fun! Commit them to your clone by doing:

git commit -a -m "COMMIT MESSAGE HERE"

(don't forget to git add any new files).

Tip: Using git commit -a -m "Commit message here" is nice because you can write the commit message right there in the command. But you can also do git commit -a -m for longer commit messages, and then it'll open a text editor for you to write the commit summary in.

Push your change to gerrit
If you installed git review, the command to push changes to gerrit is very simple: git review

If you had to install the hook manually, you can push your changes by doing: git push-for-review

Note: Some extensions do not require review, in which case you can just git push origin master to push your changes past gerrit straight into the repository.

SSH
If you get the error Permission denied (publickey). fatal: The remote end hung up unexpectedly Then you're not logged in to your ssh key right now. Solution: do  to make it prompt you for the passphrase for your key and add it to the active keychain. Then you can check what's in your keychain with. Then try pushing for review again.

The fingerprint of the Gerrit server is

dc:e9:68:7b:99:1b:27:d0:f9:fd:ce:6a:2e:bf:92:e1

so you can say yes when it asks you to add that fingerprint to the known hosts file.

Email doesn't match
TODO: help people understand how to granularly config their email and name settings in git and/or gerrit per-repo to make gerrit accept their changes remote: ERROR: committer email address (email) remote: ERROR: does not match your user account. Fix: add your secondary email address in Gerrit, and make sure you click the confirmation link in the email Gerrit sends you. Then try pushing again.

How we review code
The things we look for in code won't change, but the workflow will be different than it was before the Git switch.

Review before merge
It's important to us to have a review-before-merge workflow, for MediaWiki core and also for any extension we deploy. We will also offer that option to any extensions author who wants it for their extension. (The default is to use the current model we've been using with Subversion -- pushed and automatically merged.) For this decision we will defer to whoever is considered the primary author on the extension.

The one exception is i18n commits, which will be able to be pushed without review.

Who can review?
Who will have the ability to do code review?

We are using gerrit to manage code review. Anyone who had a Subversion account will get a gerrit account, and anyone can get a gerrit account. Within gerrit, anyone can comment on commits and signal their criticisms and approvals. Anyone can give a nonbinding "+1" to any commit (this is like "inspection" and "signoff" in the Subversion code review interface). However, for any given repository ("project"), only a small group of people will have the ability to approve code within gerrit and merge it into the repository. (Within gerrit, this superapproval is a "+2" even though that's a misleading name, because two +1 approvals DO NOT add up to a +2.) These people are "project owners".

Even within a project, we can also specify particular branches that only specific people can pull into.

MediaWiki core
We are going to maintain a "WMF" branch of mediawiki/core.git. We will use submodules for deployed extensions, and can pull from master as regularly as we want for deployments. At the start of the migration to git, the people with code review access to this branch are going to be the people who have the ability to deploy code to Wikimedia Foundation servers. See this list in gerrit and look for the lists of people under the headings "admins::mortals" or "admins::roots". Once we have the groups properly set up, gerrit will offer a list of the "project owners" for this branch. Every member of the Wikimedia Foundation operations (systems administration) team will also be in the project owners group insofar as they have code review rights globally, but in practice will rarely review code. We may add some existing code reviewers to this project owners group; we will figure out that final group by 7 March 2012.

At the start of the migration, we anticipate that this list of project owners for the WMF branch will also be the list of project owners for the master branch. However, eventually, we will add to the list of project owners for master, using as criteria the number and quality of developers' previous commits and code reviews.

No procedure has yet been defined for adding and removing people from the project owners groups. There will be a procedure by 7 March 2012.

MediaWiki extensions that the Wikimedia Foundation deploys
Same procedure as for MediaWiki core, and the same project owner groups.

Other MediaWiki extensions
Every extension author can choose between two choices here: the gated-trunk/push-for-review model, and a straight push model. For any given extension, we will honor the wishes of the person/s listed as the main author on the extension's mediawiki.org page.


 * The gated-trunk/push-for-review is the model that we are using for MediaWiki core, as mentioned above. A project owners group (plus the abovementioned project owners group for MediaWiki core) will be able to "+2" (approve and merge) changes to their extensions.  The extension author(s) will be able to define a project owners group and add others to it.


 * The straight push model is similar to how we did things in Subversion; anyone can suggest a change and submit a pull request, and it will automatically be approved and merged.

We could define groups to make this easier for batches of extensions (e.g. SMW developers). Chad will offer your community a choice. Please let Chad what you would like.

Other projects
Same procedure as for "other MediaWiki extensions" above.

How to comment on, review, and merge code in Gerrit
Anyone can comment on code in Gerrit.


 * Make sure you have a gerrit.wikimedia.org login. If you don't know, try logging in at labsconsole.wikimedia.org; the username and password should be the same.  If you can't, ask in #mediawiki for someone to help.
 * Log in to Gerrit. If you know the changeset you want to look at (URL will look like https://gerrit.wikimedia.org/r/#change,2713 or https://gerrit.wikimedia.org/r/#change,2713 ), go to that. Otherwise, use the search box and try searching for the author ("Owner") or project.
 * The changeset has a few important areas, links and buttons:
 * Reviewers. 'gerrit2' is the autoreviewer. gerrit2 auto-verifies anything that passes the Jenkins tests.  If it passes, you see a green checkmark. If it fails, a red X.  A changeset needs a green checkmark before anyone can merge it. (In the future, the autoreviewer proxy usernames will change to be more indicative, "jenkins" or "lint".)
 * Add reviewer (manually ping someone to request their review. It'll show up in their gerrit stream)
 * Side-by-side diff button, where you can double-click on a line and comment on that line! Then, click "Up to change" to go back to the changeset
 * Review button (leave an overall comment, view inline comments from the diff)
 * Abandon Change button (you'll see this if you wrote this diff. This action removes the diff from the merge queue, but leaves it in gerrit for archival purposes)

If you are one of the project owners, you'll also see:
 * Abandon Change button
 * on the Review page, additional Code Review options to +2 (approve) or -2 (veto) a diff, and a Publish And Submit button (publish your comment and merge diff into the branch, in 1 step)
 * Submit Patch Set 1 button (merge -- only useful if you or someone else has already given a +2 approval to the diff, but not merged it)

How to create a repository ("project")
See "Request a new Git repository". There's a form to fill out. It should get processed very quickly (within a couple of days).