Continuous integration/Jenkins

Jenkins is a Java tool used to handle recurring tasks such as running tests or building packages. The Wikimedia foundation install is available at https://integration.mediawiki.org/ci/

The tool is permanently connected to our review tool (Gerrit) and can be made to react on changes submitted to Gerrit. A typical example, is running MediaWiki unit tests whenever a change is submitted to the mediawiki/core git repository.

There are two sources for job configuration: the main repository at integration/jenkins and Jenkins job builder (mwext-*, mediawiki-core-.*, operations-puppet-validate).

MediaWiki core
Change sent to mediawiki/core trigger the MediaWiki-GIT-Fetching job. It is responsible for fetching the latest master version and apply the change set. It then triggers a hierarchy of tests, whenever a test fail, the build will be considered as failing and a notification is sent to Gerrit for the change author to be notified.


 * 1) MediaWiki-Universal-Linter : does a php lint on all modified files.
 * 2) MediaWiki-Tests-* : tests running one of the PHPUnit groups. All those tests are run even if one of them fail.
 * 3) When those tests have run successfully, a copy of the installation is made available publicly and a script send a test request to TestSwarm.

Nightly snapshots
They are being generated at 3am UTC via the Jenkins job "Nightly - MediaWiki core". The ant target is nightly-mediawiki-core and is really straightforward : it get the latest version of master and uses git-archive to generate a zip file. It is then copied under /org/mediawiki/integration/nightly hierarchy which is publicly available via https://integration.mediawiki.org/nightly/.

The latest snapshot is always available at https://integration.mediawiki.org/nightly/mediawiki/core/mediawiki-latest.zip (it is a symbolic link to the latest snapshot).

Android applications
The WMF produces three Android application (WikipediaMobile, WLMMobile and WiktionaryMobile), hosted on GitHub. A pre commit hook notifies Jenkins whenever a change is submitted in GitHub. The ping back URL is  https://integration.mediawiki.org/ci/github-webhook/ . Jenkins will then fetch the branch the commit was made in and build the application.

Jenkins expects an ant script named build.xml in the root directory and will then execute ant debug. Once it completes a build, it copies the generated apk under /srv/org/mediawiki/integration/*Mobile</tt> to make it available publicly. For example, the WLMMobile application at https://integration.mediawiki.org/WLMMobile/nightly/.

The mobile team is informed of build completion via an IRC bot idling in their irc channel irc://irc.freenode.net/#wikimedia-mobile

Status

 * October 2012
 * Jobs have been ported to support triggering via the Zuul daemon.
 * Parsoid team has been setting up a job reporting their Javascript parser test results


 * September 2012
 * a few extensions (TitleBlacklist...) have Jenkins CI tests, they are tested separately from core MediaWiki.
 * we have an experimental job to test mediawiki/core.git separately against Wikidata + Wikibase extension


 * July 2012
 * tests run on the gallium</tt> server.
 * Jenkins does not run any tests for extensions yet.
 * Longterm, we will want to move Jenkins to the labs.

Re-triggering testing a change on Jenkins
Sometimes Jenkins messes up a test and needs to be re-triggered. To manually re-trigger testing a change on Jenkins, comment 'recheck' on the Gerrit change.

Configuring a Jenkins job to get changes from Gerrit
Have a look at the Gerrit Trigger jenkins plugin page.

You will then have to set up the gitplugin:
 * set the URL of repository to the https:// URL (anonymous) of your git repository.
 * refspec to $GERRIT_REFSPEC
 * branch to $GERRIT_BRANCH
 * Choosing strategy to Gerrit Trigger

Debugging and development
Because the configuration files are somewhat WMF-centric, it takes some time to get everything working locally. There is a script that does this for you, or you can do it manually.

Debug Git plugin
Start Jenkins with Java option: -Dhudson.plugins.git.GitSCM.verbose="true"

Automatic installation
curl https://raw.github.com/valhallasw/wikimedia-mkjenkins/master/mkjenkins.sh | bash</tt>

To make the install go faster, it helps to have a mediawiki-core checkout in ~/src/mediawiki-core - if this repository exists, it will make a local clone. If it doesn't, it will download from gerrit instead (slow!).

Manual installation

 * git clone https://gerrit.wikimedia.org/r/p/integration/jenkins.git ~/.jenkins</tt>
 * ~/.jenkins</tt> is the default jenkins configuration directory
 * WM-specific configuration patch I - ln -s $HOME/.jenkins /var/lib/jenkins</tt>
 * because some jobs assume jenkins is installed in /var/lib/jenkins</tt>
 * Download jenkins and place it in ~/.jenkins
 * install the following plugins (download into ~/.jenkins/plugins</tt>):
 * git
 * ansicolor
 * notification
 * timestamper
 * build-timeout
 * xunit
 * Download Jenkins job builder and configuration - see Continuous_integration/Jenkins job builder for more information. You don't need a password when you install Jenkins locally.
 * WM-specific configuration patch II - Patch the JBB configuration that depends on Zuul (see the mkjenkins script for a diff)
 * If you already have a checkout of mediawiki-core: git clone --mirror -l -- your_existing_checkout /var/lib/jenkins/git/mw-core-bare</tt>. Otherwise, git clone --mirror -- https://gerrit.wikimedia.org/r/p/mediawiki/core.git /var/lib/jenkins/git/mw-core-bare</tt>
 * Start Jenkins: cd ~/.jenkins && java -jar jenkins.war& </tt>
 * When Jenkins is running, install the JBB jobs: rm -f $HOME/.cache/jenkins_jobs/jenkins_jobs_cache.yml && jenkins-jobs --conf jenkins_jobs.ini update config/</tt>.

There are some plans to make a Vagrant environment that handles all this for you, but this is not yet the case. (09:54, 13 December 2012 (UTC))