Git/Workflow

This page describes the workflow for how MediaWiki core and extensions developers will use Git, git-review, and Gerrit.

Ideal Git workflow for MediaWiki based on ^demon's ideal Git workflow

The diagram on the right is an accurate description of what our Git workflow looks like. (Transcription is welcome!)

To understand our workflow, see below.

Justification

Easier submission of code

Get an account!

In the old workflow, developers had to send their patch through Bugzilla or email it, then have someone with proper access rights commit it. Extensions developers were prevented from committing to MediaWiki core.

With Git, everyone commits in their local repository. Then everyone is allowed to submit their code for review to Gerrit. There is no more distinction between core and extensions access rights. Since code does not automatically make it into the WMF repository, there's no danger of bad commits unless someone merges irresponsibly. Thus, for many of the Git repositories hosted by the WMF, only a small group of people needs the access rights to merge (apply) the code. See "Who can review?".

This workflow has proven successful for the WMF operation team. That model is also known as "gated trunk" - code needs review before merge - and will be the default model for WMF hosted git repository. Extensions can optionally choose to adopt a straight push model in which code is reviewed post-commit. This is much more like the Subversion model in use before 2012.

If you are wondering who is going to review your code, jump straight to "Who can review?"

Better branching

Git supports branching and merging far better than Subversion did. We are encouraging people to use branches a lot.

With git-review, you can group branches as a "topic" to help with keeping feature development straight. We aim to encourage people to name branches appropriately. If you are fixing bug n, make a "bug n" branch! Look in Gerrit at the top of changesets to see examples. git-review should detect the "bug XXX" in your commit summary and automatically create a topic branch for you.

Tags vs branches: branches are for forking, to maintain a separate feature branch. This is like traditional branching. Tags are more lightweight, designed to point to an individual commit. You can delete a tag, but they are not mutable. Designed for tagging releases. "As of this commit, this was exactly release 1.2.3" -- useful if you aim to tell people to check out specific versions.

Gerrit and Git deal with merge conflicts pretty well. In cases of merge conflicts, the code reviewer gets a notice from gerrit that the merge couldn't go through, and the final merge step fails, but all the code review work will be untouched. To fix the issue, you will have to manually rebase your branch on top of the master branch and resubmit your change.

Getting set up

If you are on Windows, follow labsconsole:Help:Git#Windows.

Install & configure Git

Install Git on your computer.

• Linux: see http://evgeny-goldin.com/wiki/Git#Linux
• Windows: users can download msysgit (http://code.google.com/p/msysgit/) which includes a version of git and some nifty "Git Bash" tools that act like a basic UNIX shell.
  • Select the "OpenSSH" option in the installer to use ssh-agent.exe. It can instead be configured to work with plink/pageant as may be convenient (provided it doesn't cause any problems).
  • It may also be useful to install TortoiseGit as a complimentary, shell-integrated, GUI but code can only be pushed with the "git review" command via the command line or Git Bash.
• Mac OSX:
  • Using MacPorts: 'port install git-core'. For using SVN bidirectional bridge, 'port install git-core +svn'
• FreeBSD:
  • Using pkg_add: pkg_add -rv git

Then do

git config -l

to check that you have a name and email address configured; if you don't, tell Git your username and e-mail address so gerrit can attach it to your accounts:

git config --global user.email "foo@example.com"
git config --global user.name "mrfoo"

Get the right permissions

To clone our code repository properly, so as to easily submit a patch back to us, you need a login. We manage this via a unified LDAP login shared between Gerrit and Labsconsole. Get a login!

Log in to https://labsconsole.wikimedia.org/, verify that you can log in, then log in at https://gerrit.wikimedia.org/ with the same username and password. Click Settings, go to SSH Public Keys, and paste in your public key (usually ~/.ssh/id_rsa.pub on Linux) so our git repository will recognize you. (We aim to fix this soon by simply pulling the key from LDAP.)

If you have a few different email addresses or SSH keys, go into Gerrit's settings and add them to your account.

Teacher: give the user a login using these instructions.

Setup SSH shortcut (optional)

It's easier to access the repository if one does not need to specify full path and the port number. You can just edit your ~/.ssh/config file and add

Host review
Hostname gerrit.wikimedia.org
Port 29418
User yourusername

From now on you can just use shortened commands like:

git clone ssh://review/mediawiki/core.git   

instead of longish

git clone ssh://username@gerrit.wikimedia.org:29418/mediawiki/core.git

Clone the repository

Now, you need to clone the code repository in question that you want to make a change on. For a list of all projects in Gerrit, please see the project list in Gerrit, or get a plain-text list with:

ssh -p 29418 gerrit.wikimedia.org gerrit ls-projects

Create or move to the directory on your machine that you do your development in. If you're making one, wikimedia-git-repos is a good name.

mkdir wikimedia-git-repos
cd wikimedia-git-repos

Now, clone the repository.

Otherwise you type something like this (replacing mediawiki/core.git with mediawiki/extensions/FooBar.git for an extension or any other part of the repository):

git clone ssh://<USERNAME>@gerrit.wikimedia.org:29418/mediawiki/core.git

Go have a sip of coffee while you wait for a minute :)

Note: Notes are not yet been restored for core, as of March 27, 2012

Once cloned, get commit notes which will nicely indicate the old SVN version numbers.

cd core && git fetch origin refs/notes/commits:refs/notes/commits

Prepare to work with gerrit

In order to properly work with gerrit, you need to have a pre-commit hook that adds a "change id" to your commit summary. That unique ID will let Gerrit track your commit. Since Git commit identification is based upon a sha1 of the commit id, whenever you amend a commit, the sha1 will change and gerrit would lost track of it!

git-review

Linux

Install git-review, a tool to simplify working with Gerrit repositories so you don't have to remember some pretty confusing commands.

Debian

Users of Debian or related distros (e.g., Ubuntu, Mint) can do

sudo apt-get install python-pip || sudo easy_install pip
sudo pip install git-review

If you do not have easy_install set up, try

sudo apt-get install python-setuptools

Then run:

git review -s

in your cloned copy to setup to work with Gerrit. It will probably ask you for your commit username. Then it will automatically install the pre-commit hook.

Gentoo

emerge dev-python/pip
pip install git-review

Windows

• Install pip using these instructions [1]:
  • Download and install the latest easy installer for Windows: the .exe at the bottom of http://pypi.python.org/pypi/setuptools (make sure it is the right version for your Python; possibly upgrade Python first)
  • Download and uncompress the latest pip version from here: http://pypi.python.org/pypi/pip#downloads (make sure it is the right version for your Python; possibly upgrade Python first)
  • Go to the uncompressed pip directory and run python setup.py install.
  • Add your c:\Python2x\Scripts to the system path (replacing "X" with the actual python version). Different directories are delimited only by a ";", so do not add whitespace.
• Run pip install git-review
• Create git-review.bat in some PATH-accessible directory containing the following line (replacing "X" with the actual python version):

@python c:\Python2x\Scripts\git-review %*

• Make sure you can connect to gerrit properly via SSH.
  • Start Git Bash.
  • Get ssh-agent running if not done already via eval `ssh-agent`.
  • Add your key to the agent if not done so already via ssh-add path/to/key. Make sure the key is in OpenSSH format (.ppk keys can be exported to this format in PuTTyGen).
  • Run ssh <username>@gerrit.wikimedia.org -p 29418. It should give a Gerrit welcome message and then abort.
• In Git Bash, go to the directory of the fresh MediaWiki clone and run git-review -s. This should install a git hook called "commit-msg" under .git/hooks. Check that the file is there. From now on, commits will have a change-id appended to the summary, which Gerrit will need in order to accept any commits for review.
  • If you get "permission denied (publickey)" no matter what, you have two options. The first is the run scp -P 29418 -v <username>@gerrit.wikimedia.org:hooks/commit-msg . from within the repo's .git/hooks directory every time you want to run "git-review" on a repo for the first time. The second is to try patching "c:\Python2x\Scripts\git-review" to make git-review work properly:

index 0275f47..335f304 100644
--- "a/C:\\Python26\\Scripts\\git-review"
+++ "b/C:\\Python26\\Scripts\\git-review"
@@ -134,7 +134,10 @@ def set_hooks_commit_msg(remote, target_file):
             scp_cmd += hostname
         else:
             scp_cmd += "%s@%s" % (username, hostname)
-        scp_cmd += ":hooks/commit-msg %s" % target_file
+        # msysgit hacks to work around scp.exe (--aaron)
+        rel_target_path = str.replace( ".\%s" % os.path.relpath( target_file ), "\\", "/" )
+        scp_cmd += ":hooks/commit-msg %s" % rel_target_path
+        # end hackery
         (status, scp_output) = run_command_status(scp_cmd)
         if status != 0:
             print "Problems encountered installing commit-msg hook"

Mac OS X via Terminal

Mac OS X comes with Python (for now) but not the installation programs supported by git and git-review.

1. Open Terminal and change to a directory you're comfortable downloading test Git packages to (such as Downloads or Sites)
2. Download and install the OSX Installer for Git
3. Install pip (Note: Already included in some older versions of Mac OS X):
   • sudo easy_install pip
4. Install git-review:
   • sudo pip install git-review

Manual setup

If installing git-review is not feasible for you, you will use an alternate way to communicate with Gerrit.

First, you need to download a pre-commit hook script and place it in the right directory in your cloned copy of the repository. The script is available from https://gerrit.wikimedia.org/r/tools/hooks/commit-msg and must be placed in the repository sub directory .git/hooks/

1) With a browser:

Download the script from the repo using "Save As ..." then browse to wikimedia-git-repos/examples/.git/hooks/. Voilà!

2) With wget:

Change to the repository directory (for example, cd wikimedia-git-repos/examples/

wget  -P .git/hooks https://gerrit.wikimedia.org/r/tools/hooks/commit-msg

3) With curl:

curl https://gerrit.wikimedia.org/r/tools/hooks/commit-msg > .git/hooks/commit-msg

You also need to ensure the hook is executable. In Linux you do this with:

chmod u+x .git/hooks/commit-msg

When ever you commit a change locally, the hook script will generate a unique Change-Id for you.

Next, add an alias to simplify the command to push changes to Gerrit for review. You can do this by executing the following from your repository clone (for example, within wikimedia-git-repos/examples/):

git config --global alias.push-for-review "push origin HEAD:refs/for/master"

(The refs/for/ is Gerrit magic and must not be omitted. However, you may adapt master to point to the remote branch that you want to commit to. E.g.: When trying to push to the remote branch Foo use refs/for/Foo.)

How to submit a patch

Update master

Make sure that your master branch (the branch created when you initially cloned the repository) is up to date:

git pull --rebase origin master

Create a branch

First, create a local branch for your new change. Give the branch a short but reasonably descriptive name.

git checkout -b BRANCHNAME master

This will create a new branch (BRANCHNAME) from 'master' and check it out for you. This is equivalent to doing

git branch BRANCHNAME master
git checkout BRANCHNAME

Use a descriptive branch name. If you are working on a bug, include its id, for example "2012/bug12345".

Make and commit your change

Usual way

Change the code in the examples directory in some way; have fun! Review the changes you have made by using git status, it will show all modified files. To prepare submitting file, you should add your change to the index (the staging area between your working copy and your local repositroy), that is done by using the git add command.

While doing so you can always review the change already added to the staging area by invoking git status and look at the diff with git diff --cached.

Once you are happy with the change list, you can add them to your local repository by using git commit. You will then be prompted with your favorite editor to add descriptive message for this commit. This is what other people will see when you will later push that commit to another repository.

Then check the changes you've made, within the file(s) and within the directory:

git diff
git status

You can repeat this step over and over until you have a set of changes that you want to have pushed to the master branch. One of the cool things about git is that when you git commit, you are committing LOCALLY. This means you can commit as often as you like without potentially screwing things up for another developer on the project, unlike in SVN where you would want to be very careful that the changes you commit would not cause things to break.

Hence the workflow is something like:

# Add change:
$ git add <some file>
# Verify list of files added to the staging area
$ git status
# Review diff of changes staged:
$ git diff --cached
# repeat till happy
$ git commit
<edit commit message>

Commit message guidelines

Crafting the commit message is a very important step of your development work. The message is usually the first thing other developers will see, and the first line is used by various tools as a summary of your commit message. You will thus have a first line which is meaningful and short (≤50 characters) and then a detailed message fully explaining your patch, what you did, your design choice, possible culprit to look at, any test you could have done. Remember that people very often review code that they never tried to run, so provide relevant context and suggest ways to test the change. Bytes are cheap, so just write!

If your change is about fixing a bug, put that bug's number at the very beginning of the summary line in parentheses:

(bug 43546) Rephrasing unclear button labels

Since that first line is used by our Gerrit review tool to generate emails, that will make life easier to people receiving email notifications. Gerrit will also link the bug number automatically to the Bugzilla report.

Whenever you want to refer to another modification, avoid using the commit sha1 which might change until it is merged. You should refer to either the Change ID or the Change number instead. Those will be stable in time and are not altered when someone submit a new patch set on top of your commit.

Following the first summary line, add an empty line. That is how lots of programs interacting with git will actually detect the summary line.

Wrap your message body at 72 characters. This will ensure Gerrit will correctly display it and make the message suitable for display in any mail clients. Some people receiving emails notification might be using a text client such as mutt.

The full format should thus be:

(bug 1234) Summary line used as a subject usually

Rest of the message detailing your change, can be a long message if
needed, for example to explain your design considerations.

Please split your paragraphs with new lines :-)

The rest is pretty much free form, you can even use ascii art since
commit messages are usually displayed using a monospace font.

      ,           ,  
     /             \
    ((__-^^-,-^^-__))
     `-_---' `---_-'
      `--|o` 'o|--'
         \  `  /
          ): :(
          :o_o:
           "-"
ASCII Copyright (C) 2012, Free Software Foundation, Inc.
Published under the GPL v2

Rendering example in the vim editor with syntax coloring:

Which, once pushed to Gerrit, will roughly look like:

And generate an IRC notification:

See also

Other explanations at:

• http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html
• http://en.wikibooks.org/wiki/Git/Introduction#Good_commit_messages

One line shortcut

A shortcut it to ask git commit to directly add everything from your working copy, this is done using the -a parameter. So as a shortcut for the above workflow, one could use :

git commit -a -m "COMMIT MESSAGE HERE"

Tip: using git commit -a -m "Commit message here" is nice because you can write the commit message right there in the command. But you can also do git commit -a for longer commit messages, and then it'll open a text editor for you to write the commit summary in.

Prepare to push your change set to Gerrit

Before your changes can be merged into master, they must undergo review in Gerrit.

But first, it's a good idea to synchronize your change set with any changes that may have occurred in master while you've been working. From within the branch you've been working on, excute the following command:

git pull origin master
git rebase master

git pull will update the code in your local copy of the master branch. Then, git rebase will temporarily set aside the changes you've made in your branch, apply all of the changes that have happend in master to your working branch, then merge all of the changes you've made back into the branch. Doing this will help avoid future merge conflicts. Plus, it gives you an opportunity to test your changes against the latest code in master.

Once you are satisfied with your change set and you've rebased against master, you are ready to push your code to Gerrit for review.

Push your change set to Gerrit

If you installed git review, the command to push changes to Gerrit is very simple:

git review

If you had to install the hook manually, you can push your changes by doing:

git push-for-review

Upon success, you'll get a confirmation and a link to the changeset in Gerrit.

If your commit addresses a bug in Bugzilla, please comment on that bug to note that the commit is in the merge queue, and link to its changeset in Gerrit.

Amending a change

Sometimes, you might need to amend a submitted change. You can amend your own changes as well as changes submitted by someone else, as long as the change hasn't been merged yet.

If you have git-review, checkout the change like this:

git review -d <change number>

If you don't have git-review, checkout the change like this (you can find this line in Gerrit, on the change, in Download -> checkout, ssh):

git fetch ssh://<username>@gerrit.wikimedia.org:29418/mediawiki/ <ref> && git checkout FETCH_HEAD

Make changes, git add the files as needed, then commit the change (ensuring you are amending the commit):

git add some/dir/file.txt
git commit --amend

NOTE: DO NOT use the -m flag to specify a commit summary: that will override the previous summary and regenerate the Change-Id. Instead, use the editor to change the commit summary if needed, and keep the Change-Id line intact.

Push the change

git review -R

The -R is important here. It tells git-review to not rebase your change against master, which clutters diffs between patch set 1 and 2.

Merging your amend back into your branch

After you have amended your change, you may want to merge it back into your local branch.

You can do this by going to the Gerrit change in question.

Here is an example:

https://gerrit.wikimedia.org/r/#/c/7669/4

Go to the Download section and copy cherry pick.

We will select patch set 4.

Switch back to your branch. You will be in your review branch where you just made your change.

Note: use the branch relevant to your change number.

git checkout mingle-fr-2012-59

Paste in the cherry pick and merge any conflicts.

git fetch ssh://<username>@gerrit.wikimedia.org:29418/mediawiki/extensions/DonationInterface refs/changes/69/7669/4 && git cherry-pick FETCH_HEAD

Perform a git add on the modified files.

git add payflowpro_gateway/payflowpro.adapter.php

Do not forget to check your status and run a diff.

git diff

You should see there are no differences:

diff --cc payflowpro_gateway/payflowpro.adapter.php
index d7e510a,738c9df..0000000
--- a/payflowpro_gateway/payflowpro.adapter.php
+++ b/payflowpro_gateway/payflowpro.adapter.php

Then commit the changes:

git commit -m 'Merging patch set 4.'

[mingle-fr-2012-59 4e82e5a] Merging patch set 4.
 1 files changed, 3 insertions(+), 3 deletions(-)

Submitting a change to a branch for review

I would like to propose moving change 556c5cf464b9103b04b247ed7dd7ee3051e9aef6 that was approved to master for REL1_19 branch.

Here it goes, using git cherry-pick:

Script started on Sat Mar 24 18:31:53 2012
$ git checkout master
Already on 'master'
$ git show 556c5cf464b9103b04b247ed7dd7ee3051e9aef6
[commit info snipped]
$ git checkout REL1_19
Switched to branch 'REL1_19'
$ git cherry-pick -n 556c5cf464b9103b04b247ed7dd7ee3051e9aef6
$ git diff
$ git diff --cached
[diff output snipped]
$ git commit -m "Merge 556c5cf464b9103b04b247ed7dd7ee3051e9aef6 for REL1_19 branch"
[REL1_19 dd97c81] Merge 556c5cf464b9103b04b247ed7dd7ee3051e9aef6 for REL1_19 branch
 3 files changed, 17 insertions(+), 6 deletions(-)
$ git push gerrit HEAD:refs/for/REL1_19
Counting objects: 15, done.
Delta compression using up to 8 threads.
Compressing objects:  12% (1/8)   
Compressing objects:  25% (2/8)   
Compressing objects:  37% (3/8)   
Compressing objects:  50% (4/8)   
Compressing objects:  62% (5/8)   
Compressing objects:  75% (6/8)   
Compressing objects:  87% (7/8)   
Compressing objects: 100% (8/8)   
Compressing objects: 100% (8/8), done.
Writing objects:  12% (1/8)   
Writing objects:  25% (2/8)   
Writing objects:  37% (3/8)   
Writing objects:  50% (4/8)   
Writing objects:  62% (5/8)   
Writing objects:  75% (6/8)   
Writing objects:  87% (7/8)   
Writing objects: 100% (8/8)   
Writing objects: 100% (8/8), 1.03 KiB, done.
Total 8 (delta 7), reused 0 (delta 0)
remote: Resolving deltas:   0% (0/7)   
remote: Resolving deltas:   0% (0/7)
remote: 
remote: New Changes:
remote:   https://gerrit.wikimedia.org/r/3689
remote: 
To ssh://saper@review:29418/mediawiki/core.git
 * [new branch]      HEAD -> refs/for/REL1_19
$
Script done on Sat Mar 24 18:34:00 2012

Thus, gerrit change is created for review.

acting on remote branches

By default, your local clone will only have a local master branch set up to track the remote master branch. Tracking means that whenever you fetch objects from the remote repository, git status or git branch will be able to tell you how up-to-date is your local branch, which is very useful. So, whenever you want to regularly act on a remote branch (lets says REMOTE_BRANCH, you want to setup a one locally (REMOTE_BRANCH too to easily remember about it) that track it (with -t).

git branch -vv will give the full details:

$ git clone ...
$ git checkout -b REL1_19 -t origin/REL1_19
$ git branch -vv
  REL1_19 3b2bfd3 [gerrit/REL1_19: ahead 1] .gitreview for REL1_19 branch
* master  13169c8 [gerrit/master: behind 1] * (bug 34212) ApiBlock/ApiUnblock a[...]
$

From bug 35456 : git-review accepts, as an optional argument, the branch name to interact with. When that argument is not specified, it falls back to look for the defaultbranch parameter in a .gitreview file at the root of the repository.

Every branch should have a .gitreview having a correct defaultbranch value. For mediawiki/core.git, else people will have to use something like : git-review BRANCH_NAME.

Committing to non master

To make a change to 1.17, create a tag, and push that tag:

git checkout -b REL1_17 origin/REL1_17

<make code changes>

git add <files-changed>
git commit
git push origin REL1_17
git tag 1.17.3
git push --tags

Partial revert of previous commit

git show <commit> -- <path> | git apply -R

<commit> Can be find in gerrit patch view in small letters next to text Patch Set N. Then push for review normally.

Troubleshooting

SSH and "permission denied (publickey)"

If you get the error

Permission denied (publickey).
fatal: The remote end hung up unexpectedly

Then you're not logged in to your ssh key right now. Solution: do ssh-add ~/.ssh/id_rsa to make it prompt you for the passphrase for your key and add it to the active keychain. Then you can check what's in your keychain with ssh-add -l. Then try pushing for review again.

The fingerprint of the Gerrit server is

dc:e9:68:7b:99:1b:27:d0:f9:fd:ce:6a:2e:bf:92:e1

so you can say yes when it asks you to add that fingerprint to the known hosts file.

Keep in mind that gerrit listens on port 29418 and if for some reason you forgot to specify port number, you might be hitting "normal" SSH daemon listening on port 22 (this one has RSA key fingerprint b5:e9:dc:b2:dd:6e:70:f7:18:8a:dc:a3:5d:ab:99:4d).

To check whether SSH connectivity and public key authentication work you can use:

 $ ssh -p 29418 username@gerrit.wikimedia.org

  ****    Welcome to Gerrit Code Review    ****

  Hi username, you have successfully connected over SSH.

  Unfortunately, interactive shells are disabled.
  To clone a hosted Git repository, use:

  git clone ssh://username@gerrit.wikimedia.org:29418/REPOSITORY_NAME.git

  Connection to gerrit.wikimedia.org closed.

"[remote rejected] master -> master" and "failed to push some refs"

If you forget to add HEAD:refs/for/master as an alias, you will receive something along the lines of:

$ git push
Counting objects: 5, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (3/3), 709 bytes, done.
Total 3 (delta 1), reused 0 (delta 0)
remote: Resolving deltas:   0% (0/1)
To ssh://username@gerrit.wikimedia.org:29418/test/mediawiki/extensions/examples
 ! [remote rejected] master -> master (prohibited by Gerrit)
error: failed to push some refs to 'ssh://username@gerrit.wikimedia.org:29418/test/mediawiki/extensions/examples'

This means you tried to commit to branch "master" instead of submitting your changes for review.

Email doesn't match

TODO: help people understand how to granularly config their email and name settings in Git and/or Gerrit per-repo to make gerrit accept their changes

remote: ERROR:  committer email address (email)
remote: ERROR:  does not match your user account.

Fix: add your secondary email address in Gerrit, and make sure you click the confirmation link in the email Gerrit sends you. Then try pushing again.

Working tree is dirty

Note: if upon doing 'git review' you receive a message "Working tree is dirty" try doing 'git add' for the file(s) changed (or created), then 'git commit', and then 'git review'. This was seen on OSX with an older git client.

missing Change-Id in commit message

Note: if upon doing 'git review' you receive a message about 'missing Change-Id', then your /.git/hooks/commit-msg is probably incorrect. it should look something like:

CHANGE_ID_AFTER="Bug|Issue"
MSG="$1"

# Check for, and add if missing, a unique Change-Id
#
add_ChangeId() {
        clean_message=`sed -e '
                /^diff --git a\/.*/{
                        s///
                        q
                }
                /^Signed-off-by:/d
                /^#/d
        ' "$MSG" | git stripspace`
        if test -z "$clean_message"
        then
                return
        fi

You will also get a missing Change-ID message when trying to merge (git cherry-pick) some change from git that does not have Change-ID. It seems that the hook isn't called by cherry-pick, but it is fortunately called by git commit -c some-commit-id.

In the example below we will be moving trivial change bd7a268e4be2da23ba0b9943c3b0ba1ac88294dc from git master in the mediawiki/core project (it came via SVN trunk) to REL1_19.

Sample session to exhibit this problem:

Script started on Sun Mar 25 01:51:46 2012
$ git checkout REL1_19
Switched to branch 'REL1_19'
$ git log --pretty=oneline -n 1
9074142447ab0bd06f4e6a7ca7c9d8c70d33d109 * (bug 35449) Removed double call to OutputPage::setRobotPolicy() in
SpecialWatchlist::execute() (the other one is hidden in SpecialPage::setHeaders()) * Special:Watchlist no
longer sets links to feed when the user is anonymous
$ git show bd7a268e4be2da23ba0b9943c3b0ba1ac88294dc

commit bd7a268e4be2da23ba0b9943c3b0ba1ac88294dc
Author: Marcin Cieślak <saper@users.mediawiki.org>
Date:   Wed Mar 14 00:36:11 2012 +0000
 
    Cosmetic improvements to PostreSQL updater output
 
    * Don't WARN on sequences already existing
    * Align dots nicely to the rest

diff --git a/includes/installer/PostgresUpdater.php b/includes/installer/PostgresUpdater.php
index d1fc6f7..d4412cb 100644
--- a/includes/installer/PostgresUpdater.php
+++ b/includes/installer/PostgresUpdater.php
@@ -394,7 +394,7 @@ END;
 
        protected function renameSequence( $old, $new ) {
                if ( $this->db->sequenceExists( $new ) ) {
-                       $this->output( "WARNING sequence $new already exists\n" );
+                       $this->output( "...sequence $new already exists.\n" );
                        return;

(...full trivial diff to one file omitted...)

$ git cherry-pick -x bd7a268e4be2da23ba0b9943c3b0ba1ac88294dc
[REL1_19 a354acd] Cosmetic improvements to PostreSQL updater output
 Author: Marcin Cieślak <saper@users.mediawiki.org>
 1 files changed, 18 insertions(+), 18 deletions(-)
$ git log HEAD ^FETCH_HEAD

commit a354acd879c3dd840e7be1e3c6d6fc78d696631d
Author: Marcin Cieślak <saper@users.mediawiki.org>
Date:   Wed Mar 14 00:36:11 2012 +0000
 
    Cosmetic improvements to PostreSQL updater output
 
    * Don't WARN on sequences already existing
    * Align dots nicely to the rest
    (cherry picked from commit bd7a268e4be2da23ba0b9943c3b0ba1ac88294dc)

$ git push gerrit HEAD:refs/for/REL1_19/PostgreSQL

Counting objects: 9, done.
Delta compression using up to 8 threads.
Compressing objects:  20% (1/5)   
Compressing objects:  40% (2/5)   
Compressing objects:  60% (3/5)   
Compressing objects:  80% (4/5)   
Compressing objects: 100% (5/5)   
Compressing objects: 100% (5/5), done.
Writing objects:  20% (1/5)   
Writing objects:  40% (2/5)   
Writing objects:  60% (3/5)   
Writing objects:  80% (4/5)   
Writing objects: 100% (5/5)   
Writing objects: 100% (5/5), 750 bytes, done.
Total 5 (delta 4), reused 0 (delta 0)
remote: Resolving deltas:   0% (0/4)   
remote: Resolving deltas:   0% (0/4)
remote: ERROR: missing Change-Id in commit message
remote: Suggestion for commit message:
remote: Cosmetic improvements to PostreSQL updater output
remote: 
remote: * Don't WARN on sequences already existing
remote: * Align dots nicely to the rest
remote: (cherry picked from commit bd7a268e4be2da23ba0b9943c3b0ba1ac88294dc)
remote: 
remote: Change-Id: Ia354acd879c3dd840e7be1e3c6d6fc78d696631d
To ssh://saper@review:29418/mediawiki/core.git
 ! [remote rejected] HEAD -> refs/for/REL1_19/PostgreSQL (missing Change-Id in commit message)
error: failed to push some refs to 'ssh://saper@review:29418/mediawiki/core.git'
$ 
 
Script done on Sun Mar 25 01:54:08 2012

To fix this, use -n (don't commit) option to git cherry-pick:

git cherry-pick -n bd7a268e4be2da23ba0b9943c3b0ba1ac88294dc
git commit -c bd7a268e4be2da23ba0b9943c3b0ba1ac88294dc

Unfortunately, if you want to add the original commit ID to the message (as done by git cherry-pick -x) you have to add it yourself.

The change above has been submitted as 3c88c61f1b7e36d5d374a42bb0f50783ab5391a4 for REL1_19 review.

Don't push

If you attempt to do 'git push' after doing 'git commit' you may receive a response 'Everything up-to-date'. You have not pushed to the branch. You have to do 'git review' to move your changes to gerrit, and only from gerrit will the branch be updated. This seems to be a side effect of checking out master as a branch as of February 2012.

In some projects (e.g. test/) it is possible to do 'git push' instead of 'git review' and have the push succeed. It is probably better not to do that, as it confuses those who find your changes later and don't know where they came from.

git review complains about multiple commits

If git review asks you if you really want to submit multiple commits, and lists a bunch of unrelated commits from different branches, try this:

 git fetch --all
 git remote update

Fixing unsatisfied dependencies (rebase changes)

Example for https://gerrit.wikimedia.org/r/#change,5154

git review -d Ie6e3c9be # Otherwise cherry-pick doesn't find the commit
git checkout -b notificationsenderaddressfix origin/master
git cherry-pick 05f3235a26302eea8d68b46c2d215c783c8e6237 # In small letters between Patchset N and (gitweb) link.
git commit --amend # Added a note
git review

Maybe also have look at Gerrit/resolve conflict

Splitting a commit in smaller one

Mixing out changes is probably not the best to do nowadays :-D You could split the patch by using something like:

git-review -d 4381
# Move back to parent of patch set 1
git rebase 0d3c396^
# Apply patch set 1 changes without generating a commit
git cherry-pick -n 0d3c396

You should now have all changes not staged for commit. Create a topic branch and select the chunks you want by to craft a new commit.

git checkout -b language_converter
git add --patch

You will then be asked what to do with each of the chunk. `?` gives you an help menu, `y` stage the chunk, `s` split it in smaller parts, `n` do not stage it. Then `git diff --cached` to review change and as usual `git commit && git-review -f`

How we review code

The things we look for in code won't change, but the workflow is different than it was before the Git switch.

Review before merge

It's important to us to have a review-before-merge workflow, for MediaWiki core and also for any extension we deploy. We will also offer that option to any extensions author who wants it for their extension. (The default is to use the current model we've been using with Subversion -- pushed and automatically merged.) For this decision we will defer to whoever is considered the primary author on the extension. See Git/Gerrit project ownership.

The one exception is i18n commits, which will be able to be pushed without review.

Who can review? Gerrit project owners

Who has the ability to do code review?

We use gerrit to manage code review. Anyone can ask for a Gerrit account (Get an account!). Within Gerrit, anyone can comment on commits and signal their criticisms and approvals. Anyone can give a nonbinding "+1" to any commit (this is like "inspection" and "signoff" in the old Subversion code review interface). However, for any given repository ("Gerrit project"), only a small group of people will have the ability to approve code within Gerrit and merge it into the repository. (Within gerrit, this superapproval is a "+2" even though that's a misleading name, because two +1 approvals DO NOT add up to a +2.) These people are "Gerrit project owners". To learn about becoming a Gerrit project owner, see Git/Gerrit project ownership.

Even within a Gerrit project, we can also specify particular branches that only specific people can pull into.

To make a new Project Owner:

• Create a group
• Give it ownership of a Project
• Anyone in that group can now add more owners via https://gerrit.wikimedia.org/r/#/admin/projects/ (but we prefer to keep that process public via Git/Gerrit project ownership)
• Click Groups
• As long as you are a member of the group, you can edit the group
• example: https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki,access

MediaWiki core

We are maintaining a "WMF" branch of mediawiki/core.git. We use submodules for deployed extensions, and can pull from master as regularly as we want for deployments. At the start of the migration to git, the project owners of this branch are going to be the people who have the ability to deploy code to Wikimedia Foundation servers. gerrit will offer a list of the "Gerrit project owners" for this branch, except for the Operations (system administration) group, which is an LDAP group. Every member of the Wikimedia Foundation operations team will also be in the Gerrit project owners group insofar as they have code review rights globally, but in practice will rarely review code. We may add some existing code reviewers to this Gerrit project owners group. Details; you can request to be added.

At the start of the migration, this list of Gerrit project owners for the WMF branch is also the list of Gerrit project owners for the master branch. However, eventually, we will add to the list of Gerrit project owners for master, using as criteria the number and quality of developers' previous commits and code reviews.

Details and procedure for adding and removing people from the Gerrit project owners groups.

MediaWiki has release branches (19 so far) for core, and master (the default branch on trunk). Example: [2]. ("Heads" is gitweb's term for branches.) MediaWiki core and WMF-deployed extensions will be tagging releases just as we did in Subversion, except they'll be Git tags instead of SVN tags. Any other extension will make its own decisions regarding tagging.

MediaWiki extensions that the Wikimedia Foundation deploys

Same procedure as for MediaWiki core, and the same Gerrit project owner groups.

Other MediaWiki extensions

Every extension author can choose between two choices here: the gated-trunk/push-for-review model, and a straight push model. For any given extension, we will honor the wishes of the person/s listed as the main author on the extension's mediawiki.org page.

The gated-trunk/push-for-review is the model that we are using for MediaWiki core, as mentioned above. A Gerrit project owners group (plus the above mentioned Gerrit project owners group for MediaWiki core) will be able to "+2" (approve and merge) changes to their extensions. The extension author(s) will be able to define a Gerrit project owners group and add others to it.

The straight push model is similar to how we did things in Subversion; anyone can suggest a change and submit a pull request, and it will automatically be approved and merged.

We could define groups to make this easier for batches of extensions (e.g. SMW developers). Chad will offer your community a choice. Please let Chad what you would like via Git/New repositories.

Other Gerrit projects

Same procedure as for "other MediaWiki extensions" above.

How to comment on, review, and merge code in Gerrit

A sample changeset, with annotations

Side-by-side diff

Review screen

Anyone can comment on code in Gerrit.

Viewing and commenting on code

• Make sure you have a https://gerrit.wikimedia.org login (Get an account!). If you don't know, try logging in at https://labsconsole.wikimedia.org; the username and password should be the same. If you can't, ask in #mediawiki^connect for someone to help.
• Log in to Gerrit. If you know the changeset you want to look at (URL will look like https://gerrit.wikimedia.org/r/#change,2713 ), go to that. Otherwise, use the search box and try searching. There is no fulltext search in Gerrit, but you can search by author ("Owner"), Gerrit project, branch, changesets you've starred, etc.
• The changeset has a few important fields, links and buttons:

• Reviewers. 'jenkins-bot' is the autoreviewer that auto-verifies anything that passes the Jenkins tests. If it passes, you see a green checkmark. If it fails, a red X. A changeset needs a green checkmark before anyone can merge it.

• Add reviewer (manually ping someone to request their review. It'll show up in their Gerrit stream)

• Side-by-side diff button:

• Opens the diff in a new tab. You can double-click on a line and comment on that line, then save a draft comment! Then, click "Up to change" to go back to the changeset.

• Abandon Change button (you'll see this if you wrote this diff. This action removes the diff from the merge queue, but leaves it in Gerrit for archival purposes)
• Review button:

• On this page you can leave an overall comment, view inline comments from the diff that are still in draft form and awaiting publication, and signal your thoughts on the commit. You can signal approval with a "+1" and disapproval with a "-1". These numbers are nonbinding, won't cause merges or rejections, and have no formal effect on the code review. To publish your draft inline comments plus your comments on the diff as a whole, click Publish.

Formally reviewing and merging or rejecting code

If you are one of the Gerrit project owners, you'll also see:

A Gerrit review screen with approval and veto options

• Abandon Change button
• on the Review page, additional Code Review options to +2 (approve) or -2 (veto) a diff, and a Publish And Submit button (publish your comment and merge diff into the branch, in 1 step)
• Submit Patch Set 1 button (merge -- only useful if you or someone else has already given a +2 approval to the diff, but not merged it)

And once you've merged something into the example Gerrit project you'll see it in https://gerrit.wikimedia.org/r/gitweb?p=test/mediawiki/extensions/examples.git;a=summary .

If you merge a commit that references a Bugzilla bug, please go to that bug and mark it RESOLVED: FIXED and reference the merge ID.

Teacher: for this section, add the participant to the project owner group for the examples Gerrit project at https://gerrit.wikimedia.org/r/#/admin/groups/17,members . Add the "Member" by email or wiki username.

Troubleshooting

Your change could not be merged due to a path conflict

If you get the error "Your change could not be merged due to a path conflict" when submitting a change set in Gerrit, you need to resolve the conflict in a manner very similar to how you would in SVN.

1. Make sure your master branch is up to date: git pull master
2. Create and switch to a new branch in which to checkout the change set with the conflict: git checkout -b BRANCHNAME
3. Checkout the conflicting change set into this branch. You can copy/paste the correct command from the 'Download' section in the Gerrit review. It will look something like this: git fetch ssh://awjrichards@gerrit.wikimedia.org:29418/mediawiki/extensions/MobileFrontend refs/changes/38/3538/2 && git checkout FETCH_HEAD
4. Rebase against master (this should give you a conflict warning. this is OK!): git rebase master
5. Open the file(s) with the conflict, and resolve them. They will be denoted the same way they are in SVN (with <<<<<< and >>>>>>)
6. Once you have finished resolving the conflicts, you need to git add the file again. For instance: git add HtmlFormatter.php
7. Tell the rebase to continue now that the issue is resolved: git rebase --continue
8. Push the resolved conflict to Gerrit for review: git review
9. Re-review the change set in Gerrit, and then submit the changes to be merged to master.

Your change requires a recursive merge to resolve

If you get the error "Your change requires a recursive merge to resolve", you need to rebase the change set against master.

1. Make sure your master branch is up to date: git pull master
2. Create and switch to a new branch in which to checkout the change set with the conflict: git checkout -b BRANCHNAME
3. Checkout the conflicting change set into this branch. You can copy/paste the correct command from the 'Download' section in the Gerrit review. It will look something like this: git fetch ssh://awjrichards@gerrit.wikimedia.org:29418/mediawiki/extensions/MobileFrontend refs/changes/14/3414/3 && git checkout FETCH_HEAD
4. Rebase against master: git rebase master
5. Push the change to Gerrit for review: git review
6. Re-review the change set in Gerrit, and then submit the changes to be merged to master.

How to create a repository ("Gerrit project")

See "Request a new Git repository". There's a form to fill out. It should get processed very quickly (within a couple of days).

See also

Also useful are these pages:

• Git and gerrit in Wikimedia labs
• An intro-to-git "training mission"
• An introduction to git-svn for Subversion/SVK users and deserter by Sam Vilain
• GitMagic A git guide with translations - recommended
• A git branching model
• git: the simple guide
• Git Community Book will take you gently into Git internals. Actually it is very hard to "get" git until knowing something about how it works internally. After this, everything become simple, just hidden without convoluted and anti-productive user interface.
• How we're going to use Git: slides by Brion Vibber, as presented autumn 2011
• Audio from Git presentation by Brion Vibber, autumn 2011
• Notes from Brion's presentation
• Git's website with documentation
• MediaWiki Git Guide (MGG) - selection of relevant Git pages in the MW Virtual Library; for PDF generation or printing
• A very useful Git guide

Also useful (if you subtract away GitHub stuff):

• http://wiki.openstack.org/GerritJenkinsGithub

This might change. If you'd like to help, we're seeking test subjects! Also, John Du Hart would like for us to consider using Phabricator instead; we will consider this in June 2012.

• Gerrit bugs that matter
• https://gerrit.wikimedia.org/r/#q,status:open,n,z
• Download from Git
• Git-review
• labsconsole:Help:Access
• Gerrit