Continuous integration/Tutorials/Adding basic checks
This page is outdated. It was written for an older version of MediaWiki and may not apply to the most recent version. If you have checked or updated this page and found the content to be suitable, please remove this notice. .
This tutorial shows up how to add a very basic check in Jenkins, we will base this tutorial on
operations/dns.git which hold Wikimedia DNS configuration. The aim is to validate the syntax of the DNS zone files.
What we want to achieve:
Whenever a patch is send against
operations/dns.git we want to expand the gdnsd templates and have it validate the syntax. When no error is detected, Jenkins should vote Verified +2 to let people merge the change after review, if something is wrong, we would vote Verified -1 to let people know the syntax is invalid.
- gdnsd installed on all production Jenkins slaves. This is achieved in puppet is out of the scope of this tutorial
- a local installation of Jenkins Job Builder, you want to read our step by step guide to install Jenkins Job Builder
- a copy of the Jenkins Job Builder from https://gerrit.wikimedia.org/r/integration/jenkins-job-builder-config.git
Editing Jenkins configuration
operations/ repositories are configured inside
operations-misc.yaml. We will need to create a new Jenkins project that gives the Gerrit repository name and list the jobs we want to attach to it.
- project: # Parameters that will be passed to job templates name: 'operations-dns'
Now lets create our job template. It needs: - a name so we can reference it in the project definition - a few default parameters which let the job be triggered by Zuul - we then add the shell commands we want to execute:
- job-template: # The name of our template. Will be referenced to in the project: name: 'operations-dns-lint' # Default settings to let Zuul trigger the job against the submitted patchset: defaults: use-zuul triggers: - zuul # The shell commands. $WORKSPACE is an environnement variable that contains # the full path to where Jenkins cloned the repository with the submitted patch builders: - shell: | # Prepare the zone files mkdir -p "$WORKSPACE"/chroot/zones authdns-gen-zones "$WORKSPACE"/templates "$WORKSPACE"/chroot/zones # lint them gdnsd -d "$WORKSPACE"
Now we want to edit the project definition to reference this template.
- project: name: 'operations-dns' jobs: # Jobs we want to generate using the parameters above - 'operations-dns-lint' # job referenced there
Now have Jenkins Job Builder to generate the configuration using
jenkins-jobs test -o - config/ operations-dns-lint. It will parse the YAML definition, realize the
operations-dns-lint job and write out on standard output the XML Jenkins job definition. If something went wrong, you want to double check your YAML definition, usually the errors are related to a wrong indentation (use two spaces for indentation).
The command is:
$ jenkins-jobs --conf jenkins_jobs.ini test -o - config/ operations-dns-lint // snip XML output
The most important things to check are: - the git repository name - the builders section (the actual commands that would be run).
The git configuration is
<scm class="hudson.plugins.git.GitSCM"> <configVersion>2</configVersion> <userRemoteConfigs> <hudson.plugins.git.UserRemoteConfig> <name>origin</name> <refspec>$ZUUL_REF</refspec> <url>/srv/ssd/zuul/git/$ZUUL_PROJECT</url> </hudson.plugins.git.UserRemoteConfig> </userRemoteConfigs> <branches> <hudson.plugins.git.BranchSpec> <name>$ZUUL_COMMIT</name> </hudson.plugins.git.BranchSpec> </branches> <!-- snip various git settings --> </scm>
The repository URL is
/srv/ssd/zuul/git/$ZUUL_PROJECT which trailling part is the gerrit project. The Gerrit project name $ZUUL_PROJECT, the refspec $ZUUL_REF and branch specifier $ZUUL_COMMIT are provided by Zuul and let the Jenkins git plugin to fetch the patchset.
Then verify the shell commands you want to execute under the
builders section. Note that some characters needs to be escaped (such as double quotes) since it is an XML file:
<builders> <hudson.tasks.Shell> <command># Prepare the zone files mkdir -p "$WORKSPACE"/chroot/zones authdns-gen-zones "$WORKSPACE"/templates "$WORKSPACE"/chroot/zones # lint them gdnsd -d "$WORKSPACE"</command> </hudson.tasks.Shell> </builders>
Once happy, you can deploy the configuration on Jenkins:
$ jenkins-jobs --conf jenkins_jobs.ini update config/ operations-dns-lint
If all went well, the job should be available at https://integration.wikimedia.org/ci/job/operations-dns-lint/ :
When clicking the configure link, you can review the final configuration. Browsing down at the bottom of that page you will see the actual shell commands that would be run:
Editing Zuul configuration
The configuration is held in
integration/zuul-config.git repository which has a single file: layout.yaml.
projects section add gerrit repository name and trigger the
operations-dns-lint in the
check-voter pipeline. This pipeline will make Zuul vote Verified +2 on success and Verified -1 on failure:
projects: - name: operations/dns check-voter: - operations-dns-lint
You will need the change to be deployed on Zuul server and have Zuul reloaded. It will then clone the repository and wait for changes received from Gerrit. From there, it would trigger the Jenkins job with your patchset and report back in Gerrit the status of the zone linting.