Continuous integration/Jenkins

Jenkins is a Java tool used to handle recurring tasks such as running tests or building packages. The Wikimedia foundation install is available at https://integration.mediawiki.org/ci/

The tool is permanently connected to our review tool (Gerrit) and can be made to react on changes submitted to Gerrit. A typical example, is running MediaWiki unit tests whenever a change is submitted to the mediawiki/core git repository.

There are two sources for job configuration: the main repository at integration/jenkins and Jenkins job builder (mwext-*, mediawiki-core-.*, operations-puppet-validate).

MediaWiki core
Change sent to mediawiki/core trigger the MediaWiki-GIT-Fetching job. It is responsible for fetching the latest master version and apply the change set. It then triggers a hierarchy of tests, whenever a test fail, the build will be considered as failing and a notification is sent to Gerrit for the change author to be notified.


 * 1) MediaWiki-Universal-Linter : does a php lint on all modified files.
 * 2) MediaWiki-Tests-* : tests running one of the PHPUnit groups. All those tests are run even if one of them fail.
 * 3) When those tests have run successfully, a copy of the installation is made available publicly and a script send a test request to TestSwarm.

Nightly snapshots
They are being generated at 3am UTC via the Jenkins job "Nightly - MediaWiki core". The ant target is nightly-mediawiki-core and is really straightforward : it get the latest version of master and uses git-archive to generate a zip file. It is then copied under /org/mediawiki/integration/nightly hierarchy which is publicly available via https://integration.mediawiki.org/nightly/.

The latest snapshot is always available at https://integration.mediawiki.org/nightly/mediawiki/core/mediawiki-latest.zip (it is a symbolic link to the latest snapshot).

Android applications
The WMF produces three Android application (WikipediaMobile, WLMMobile and WiktionaryMobile), hosted on GitHub. A pre commit hook notifies Jenkins whenever a change is submitted in GitHub. The ping back URL is  https://integration.mediawiki.org/ci/github-webhook/ . Jenkins will then fetch the branch the commit was made in and build the application.

Jenkins expects an ant script named build.xml in the root directory and will then execute ant debug. Once it completes a build, it copies the generated apk under /srv/org/mediawiki/integration/*Mobile</tt> to make it available publicly. For example, the WLMMobile application at https://integration.mediawiki.org/WLMMobile/nightly/.

The mobile team is informed of build completion via an IRC bot idling in their irc channel irc://irc.freenode.net/#wikimedia-mobile

Status

 * October 2012
 * Jobs have been ported to support triggering via the Zuul daemon.
 * Parsoid team has been setting up a job reporting their Javascript parser test results


 * September 2012
 * a few extensions (TitleBlacklist...) have Jenkins CI tests, they are tested separately from core MediaWiki.
 * we have an experimental job to test mediawiki/core.git separately against Wikidata + Wikibase extension


 * July 2012
 * tests run on the gallium</tt> server.
 * Jenkins does not run any tests for extensions yet.
 * Longterm, we will want to move Jenkins to the labs.

Re-triggering testing a change on Jenkins
Sometimes Jenkins messes up a test and needs to be re-triggered. To manually re-trigger testing a change on Jenkins, use http://integration.mediawiki.org/ci/gerrit_manual_trigger. You will need to be logged in using your labsconsole account. Enter a Gerrit search term to search for a patchset to retrigger (for example search for the change number) then check the patchset(s) you want to get retriggered and submit.

Configuring a Jenkins job to get changes from Gerrit
Have a look at the Gerrit Trigger jenkins plugin page.

You will then have to set up the gitplugin:
 * set the URL of repository to the https:// URL (anonymous) of your git repository.
 * refspec to $GERRIT_REFSPEC
 * branch to $GERRIT_BRANCH
 * Choosing strategy to Gerrit Trigger