User:Hashar/CI/docgen

Random content related to documentation generation with Jenkins. This is more of a drop box for Hashar ongoing work, will be refactored whenever something is actually working.

Using the repo ssh://integration-jenkins2.pmtpa.wmflabs:29418/mediawiki/core.git

on tag addition
Tag something:

git tag v0.0 69ce0db && git push --tags

Zuul receives the event:

2013-01-17 08:53:21,766 DEBUG gerrit.GerritWatcher: Received data from Gerrit event stream: {u'refUpdate': {u'newRev': u'69ce0dbb92f25cd51c1ec867fcdda32fb6f549da', u'oldRev': u'0000000000000000000000000000000000000000', u'project': u'mediawiki/core', u'refName': u'refs/tags/v0.0'}, u'submitter': {u'email': u'hashar@free.fr', u'name': u'Hashar'}, u'type': u'ref-updated'}

On tag deletion
Deleting the tag:

git push gerrit :refs/tags/v0.0

Zuul receives the event:

2013-01-17 09:05:10,912 DEBUG gerrit.GerritWatcher: Received data from Gerrit event stream: {u'refUpdate': {u'newRev': u'0000000000000000000000000000000000000000', u'oldRev': u'69ce0dbb92f25cd51c1ec867fcdda32fb6f549da', u'project': u'mediawiki/core', u'refName': u'refs/tags/v0.0'}, u'submitter': {u'email': u'hashar@free.fr', u'name': u'Hashar'}, u'type': u'ref-updated'}

On a tag deletion
2013-01-17 09:05:11,031 DEBUG zuul.JenkinsCallback: Received data from Jenkins: {u'build': {u'full_url': u'http://10.4.0.172:8080/ci/job/mediawiki-core-docgeneration/1/', u'number': 1, u'parameters': {u'GERRIT_NEWREV': u'0000000000000000000000000000000000000000', u'GERRIT_OLDREV': u'69ce0dbb92f25cd51c1ec867fcdda32fb6f549da', u'GERRIT_PROJECT': u'mediawiki/core', u'GERRIT_REFNAME': u'refs/tags/v0.0', u'UUID': u'a8b657c6f2644031b4d9a331a9a25f69', u'ZUUL_NEWREV': u'0000000000000000000000000000000000000000', u'ZUUL_OLDREV': u'69ce0dbb92f25cd51c1ec867fcdda32fb6f549da', u'ZUUL_PIPELINE': u'publish', u'ZUUL_PROJECT': u'mediawiki/core', u'ZUUL_REF': u'refs/tags/v0.0', u'ZUUL_REFNAME': u'refs/tags/v0.0', u'ZUUL_SHORT_NEWREV': u'0000000', u'ZUUL_UUID': u'a8b657c6f2644031b4d9a331a9a25f69'}, u'phase': u'STARTED', u'url': u'job/mediawiki-core-docgeneration/1/'}, u'name': u'mediawiki-core-docgeneration', u'url': u'job/mediawiki-core-docgeneration/'}

In the Jenkins interface, the build show up with:

Triggered by reference: refs/tags/v0.0 Old revision: 69ce0dbb92f25cd51c1ec867fcdda32fb6f549da New revision: 0000000000000000000000000000000000000000 Pipeline: publish

Jenkins console:

09:05:11 Started by user Zuul triggering system 09:05:11 Building in workspace /var/lib/jenkins/jobs/mediawiki-core-docgeneration/workspace 09:05:11 Checkout:workspace / /var/lib/jenkins/jobs/mediawiki-core-docgeneration/workspace - hudson.remoting.LocalChannel@17e34309 09:05:11 Using strategy: Default 09:05:11 Wiping out workspace first. 09:05:11 Cloning the remote Git repository 09:05:11 Cloning repository /var/lib/zuul/git/mediawiki/core 09:05:11 git --version 09:05:11 git version 1.7.9.5 09:05:11 Fetching upstream changes from /var/lib/zuul/git/mediawiki/core 09:05:11 No candidate revisions 09:05:11 ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job. 09:05:11 Finished: FAILURE

Tag addition
Data from Jenkins:

2013-01-17 09:13:13,020 DEBUG zuul.JenkinsCallback: Received data from Jenkins: {u'build': {u'full_url': u'http://10.4.0.172:8080/ci/job/mediawiki-core-docgeneration/3/', u'number': 3, u'parameters': {u'GERRIT_NEWREV': u'c14584808ad15ca999658551a9f7d124fd9ccf97', u'GERRIT_OLDREV': u'0000000000000000000000000000000000000000', u'GERRIT_PROJECT': u'mediawiki/core', u'GERRIT_REFNAME': u'refs/tags/v1.0', u'UUID': u'6a2aa3b1c6bf49f989078782150eabdf', u'ZUUL_NEWREV': u'c14584808ad15ca999658551a9f7d124fd9ccf97', u'ZUUL_OLDREV': u'0000000000000000000000000000000000000000', u'ZUUL_PIPELINE': u'publish', u'ZUUL_PROJECT': u'mediawiki/core', u'ZUUL_REF': u'refs/tags/v1.0', u'ZUUL_REFNAME': u'refs/tags/v1.0', u'ZUUL_SHORT_NEWREV': u'c145848', u'ZUUL_UUID': u'6a2aa3b1c6bf49f989078782150eabdf'}, u'phase': u'STARTED', u'url': u'job/mediawiki-core-docgeneration/3/'}, u'name': u'mediawiki-core-docgeneration', u'url': u'job/mediawiki-core-docgeneration/'}

Build information is:

Triggered by reference: refs/tags/v1.0 Old revision: 0000000000000000000000000000000000000000 New revision: c14584808ad15ca999658551a9f7d124fd9ccf97 Pipeline: publish

The console for the failed build: 09:13:13 Started by user Zuul triggering system 09:13:13 Building in workspace /var/lib/jenkins/jobs/mediawiki-core-docgeneration/workspace 09:13:13 Checkout:workspace / /var/lib/jenkins/jobs/mediawiki-core-docgeneration/workspace - hudson.remoting.LocalChannel@17e34309 09:13:13 Using strategy: Default 09:13:13 Wiping out workspace first. 09:13:13 Cloning the remote Git repository 09:13:13 Cloning repository /var/lib/zuul/git/mediawiki/core 09:13:13 git --version 09:13:13 git version 1.7.9.5 09:13:13 Fetching upstream changes from /var/lib/zuul/git/mediawiki/core 09:13:13 No candidate revisions 09:13:13 ERROR: Couldn't find any revision to build. Verify the repository and branch configuration for this job. 09:13:13 Finished: FAILURE

The Git plugin has: /var/lib/zuul/git/mediawiki/core. Refspec: $ZUUL_REF (= refs/tags/v1.0) Branch specificier: $ZUUL_BRANCH (NOT DEFINED :/)

From scratch
/var/lib/zuul/git/mediawiki/core has no tag

In local repo tag a commit: git tag v1.0.0 69ce0db

Push tags to gerrit: $ git push --tags If you are having access problems, please see: https://labsconsole.wikimedia.org/wiki/Access#Accessing_public_and_private_instances Total 0 (delta 0), reused 0 (delta 0) remote: Processing changes: refs: 1, done Killed by signal 1. To ssh://hashar@integration-jenkins2.pmtpa.wmflabs:29418/mediawiki/core.git * [new tag]        v1.0.0 -> v1.0.0

Zuul receives: {u'refUpdate': {u'newRev': u'69ce0dbb92f25cd51c1ec867fcdda32fb6f549da', u'oldRev': u'0000000000000000000000000000000000000000', u'project': u'mediawiki/core', u'refName': u'refs/tags/v1.0.0'}, u'submitter': {u'email': u'hashar@free.fr', u'name': u'Hashar'}, u'type': u'ref-updated'}

Jenkins equivalent: {u'build': {u'full_url': u'http://10.4.0.172:8080/ci/job/mediawiki-core-docgeneration/5/', u'number': 5, u'parameters': {u'GERRIT_NEWREV': u'69ce0dbb92f25cd51c1ec867fcdda32fb6f549da', u'GERRIT_OLDREV': u'0000000000000000000000000000000000000000', u'GERRIT_PROJECT': u'mediawiki/core', u'GERRIT_REFNAME': u'refs/tags/v1.0.0', u'UUID': u'5022a26a4b2d417096652340142eaa6f', u'ZUUL_NEWREV': u'69ce0dbb92f25cd51c1ec867fcdda32fb6f549da', u'ZUUL_OLDREV': u'0000000000000000000000000000000000000000', u'ZUUL_PIPELINE': u'publish', u'ZUUL_PROJECT': u'mediawiki/core', u'ZUUL_REF': u'refs/tags/v1.0.0', u'ZUUL_REFNAME': u'refs/tags/v1.0.0', u'ZUUL_SHORT_NEWREV': u'69ce0db', u'ZUUL_UUID': u'5022a26a4b2d417096652340142eaa6f'}, u'phase': u'STARTED', u'url': u'job/mediawiki-core-docgeneration/5/'}, u'name': u'mediawiki-core-docgeneration', u'url': u'job/mediawiki-core-docgeneration/'} Aka with a ZUUL_NEWREF and ZUUL_REF. Note there is no ZUUL_BRANCH in post jobs.

Git plugin verbose output: 09:39:41 Started by user Zuul triggering system 09:39:41 Building in workspace /var/lib/jenkins/jobs/mediawiki-core-docgeneration/workspace 09:39:41 Checkout:workspace / /var/lib/jenkins/jobs/mediawiki-core-docgeneration/workspace - hudson.remoting.LocalChannel@7b5e236a 09:39:41 Using strategy: Default 09:39:41 Wiping out workspace first. 09:39:41 Cloning the remote Git repository 09:39:41 Cloning repository /var/lib/zuul/git/mediawiki/core 09:39:41 [workspace] $ git --version 09:39:41 git --version 09:39:41 git version 1.7.9.5 09:39:41 $ git clone --progress -o origin /var/lib/zuul/git/mediawiki/core /var/lib/jenkins/jobs/mediawiki-core-docgeneration/workspace 09:39:41 Fetching upstream changes from /var/lib/zuul/git/mediawiki/core 09:39:41 [workspace] $ git fetch -t /var/lib/zuul/git/mediawiki/core refs/tags/v1.0.0 09:39:41 ERROR: Problem fetching from origin / origin - could be unavailable. Continuing anyway. 09:39:41 �[8mha:AAAAWB+LCAAAAAAAAABb85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWtSE4tKMnMz/PLL0ldFVf2c+b/lb5MDAwVRQxSaBqcITRIIQMEMIIUFgAAckCEiWAAAAA=�[0mhudson.plugins.git.GitException: Command "git fetch -t /var/lib/zuul/git/mediawiki/core refs/tags/v1.0.0" returned status code 128: 09:39:41 stdout: 09:39:41 stderr: fatal: Couldn't find remote ref refs/tags/v1.0.0 09:39:41 fatal: The remote end hung up unexpectedly

The important parts are:

[workspace] $ git fetch -t /var/lib/zuul/git/mediawiki/core refs/tags/v1.0.0 ... returned status code 128: stderr: fatal: Couldn't find remote ref refs/tags/v1.0.0

Looking at the origin /var/lib/zuul/git/mediawiki/core, then git tag -l is empty :/

$ ls -l /var/lib/zuul/git/mediawiki/core/.git/refs/tags/ total 0 $

Zuul did not update the repository on reference updates. Should be done before triggering the job. The issue has been fixed in Feb 2013 by Antoine with upstream change https://review.openstack.org/#/c/19928/.

New change having just lint (no merge)
Pushed a change with only a -lint job (aka no merge). Git plugin output: 10:18:42 Started by user Zuul triggering system 10:18:42 Building in workspace /var/lib/jenkins/jobs/mediawiki-core-lint/workspace 10:18:42 Checkout:workspace / /var/lib/jenkins/jobs/mediawiki-core-lint/workspace - hudson.remoting.LocalChannel@7b5e236a 10:18:42 Using strategy: Default 10:18:42 Last Built Revision: Revision 3740c47747d47ee444dc9335091daf00a08f23dc (origin/master) 10:18:42 Wiping out workspace first. 10:18:42 Cloning the remote Git repository 10:18:42 Cloning repository /var/lib/zuul/git/mediawiki/core 10:18:42 [workspace] $ git --version 10:18:42 git --version 10:18:42 git version 1.7.9.5 10:18:42 $ git clone --progress -o origin /var/lib/zuul/git/mediawiki/core /var/lib/jenkins/jobs/mediawiki-core-lint/workspace 10:18:42 Fetching upstream changes from /var/lib/zuul/git/mediawiki/core 10:18:42 [workspace] $ git fetch -t /var/lib/zuul/git/mediawiki/core refs/zuul/master/Z30e30ecf23a14881a01ccfec52cbc465 10:18:42 getCandidateRevisions(false,master,,,,remoteUrls=[/var/lib/zuul/git/mediawiki/core],buildsByBranchName={origin/master=Build #43 of Revision 3740c47747d47ee444dc9335091daf00a08f23dc (origin/master)},lastBuild=Build #43 of Revision 3740c47747d47ee444dc9335091daf00a08f23dc (origin/master)]) considering branches to build 10:18:42 [workspace] $ git tag -l master 10:18:42 Qualifying master as a branch in repository origin -> origin/master 10:18:42 [workspace] $ git rev-parse origin/master^{commit} 10:18:42 rev-parse origin/master -> AnyObjectId[e686c49b4454cce5e0f41fd166cac5b7144d6380] 10:18:42 Found a new commit AnyObjectId[e686c49b4454cce5e0f41fd166cac5b7144d6380] to be built on origin/master 10:18:42 Commencing build of Revision e686c49b4454cce5e0f41fd166cac5b7144d6380 (origin/master) 10:18:42 Checking out Revision e686c49b4454cce5e0f41fd166cac5b7144d6380 (origin/master) 10:18:42 [workspace] $ git checkout -f e686c49b4454cce5e0f41fd166cac5b7144d6380 10:18:42 [workspace] $ git tag -a -f -m "Jenkins Build #44" jenkins-mediawiki-core-lint-44 10:18:42 [workspace] $ git rev-list 3740c47747d47ee444dc9335091daf00a08f23dc 10:18:42 [workspace] $ git whatchanged --no-abbrev -M --pretty=raw 3740c47747d47ee444dc9335091daf00a08f23dc..e686c49b4454cce5e0f41fd166cac5b7144d6380 10:18:42 [workspace] $ /bin/sh -xe /tmp/hudson243757277796990374.sh 10:18:42 + export 10:18:42 export BUILD_ID='2013-01-17_10-18-42' 10:18:42 export BUILD_NUMBER='44' 10:18:42 export BUILD_TAG='jenkins-mediawiki-core-lint-44' 10:18:42 export BUILD_URL='http://10.4.0.172:8080/ci/job/mediawiki-core-lint/44/' 10:18:42 export EXECUTOR_NUMBER='5' 10:18:42 export GERRIT_BRANCH='master' 10:18:42 export GERRIT_CHANGES='mediawiki/core:master:refs/changes/17/17/3' 10:18:42 export GERRIT_PROJECT='mediawiki/core' 10:18:42 export GIT_BRANCH='master' 10:18:42 export GIT_COMMIT='e686c49b4454cce5e0f41fd166cac5b7144d6380' 10:18:42 export HOME='/var/lib/jenkins' 10:18:42 export HUDSON_COOKIE='967cf198-29de-4457-b8a9-68e180bd0da8' 10:18:42 export HUDSON_HOME='/var/lib/jenkins' 10:18:42 export HUDSON_SERVER_COOKIE='94834190ccb2c0f0ac2034b6057350d8' 10:18:42 export HUDSON_URL='http://10.4.0.172:8080/ci/' 10:18:42 export JENKINS_HOME='/var/lib/jenkins' 10:18:42 export JENKINS_SERVER_COOKIE='94834190ccb2c0f0ac2034b6057350d8' 10:18:42 export JENKINS_URL='http://10.4.0.172:8080/ci/' 10:18:42 export JOB_NAME='mediawiki-core-lint' 10:18:42 export JOB_URL='http://10.4.0.172:8080/ci/job/mediawiki-core-lint/' 10:18:42 export LANG='en_US.UTF-8' 10:18:42 export LD_LIBRARY_PATH='/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64/server:/usr/lib/jvm/java-6-openjdk-amd64/jre/lib/amd64:/usr/lib/jvm/java-6-openjdk-amd64/jre/../lib/amd64' 10:18:42 export LOGNAME='jenkins' 10:18:42 export MAIL='/var/mail/jenkins' 10:18:42 export NODE_LABELS='master' 10:18:42 export NODE_NAME='master' 10:18:42 export NODE_PATH='/usr/lib/nodejs:/usr/share/javascript' 10:18:42 export PATH='/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games' 10:18:42 export PWD='/var/lib/jenkins/jobs/mediawiki-core-lint/workspace' 10:18:42 export SHELL='/bin/bash' 10:18:42 export SHLVL='1' 10:18:42 export TERM='xterm-256color' 10:18:42 export USER='jenkins' 10:18:42 export UUID='1716b1396c674cb8af2b61dd8cf65344' 10:18:42 export WORKSPACE='/var/lib/jenkins/jobs/mediawiki-core-lint/workspace' 10:18:42 export ZUUL_BRANCH='master' 10:18:42 export ZUUL_CHANGE='17' 10:18:42 export ZUUL_CHANGES='mediawiki/core:master:refs/changes/17/17/3' 10:18:42 export ZUUL_CHANGE_IDS='17,3' 10:18:42 export ZUUL_COMMIT='e686c49b4454cce5e0f41fd166cac5b7144d6380' 10:18:42 export ZUUL_PATCHSET='3' 10:18:42 export ZUUL_PIPELINE='check' 10:18:42 export ZUUL_PROJECT='mediawiki/core' 10:18:42 export ZUUL_REF='refs/zuul/master/Z30e30ecf23a14881a01ccfec52cbc465' 10:18:42 export ZUUL_UUID='1716b1396c674cb8af2b61dd8cf65344' 10:18:42 export _='/usr/bin/daemon' 10:18:42 [_shared] $ ant -file build.xml -DZUUL_BRANCH=master -DZUUL_CHANGE_IDS=17,3 -DUUID=1716b1396c674cb8af2b61dd8cf65344 -DZUUL_UUID=1716b1396c674cb8af2b61dd8cf65344 -DZUUL_PROJECT=mediawiki/core -DZUUL_REF=refs/zuul/master/Z30e30ecf23a14881a01ccfec52cbc465 -DZUUL_PIPELINE=check -DGERRIT_BRANCH=master -DZUUL_PATCHSET=3 -DGERRIT_CHANGES=mediawiki/core:master:refs/changes/17/17/3 -DZUUL_COMMIT=e686c49b4454cce5e0f41fd166cac5b7144d6380 -DZUUL_CHANGES=mediawiki/core:master:refs/changes/17/17/3 -DZUUL_CHANGE=17 -DGERRIT_PROJECT=mediawiki/core php-lint 10:18:42 Buildfile: /var/lib/jenkins/jobs/_shared/build.xml 10:18:43 10:18:43 git-files-changed: 10:18:43     [echo] Files changed according to git: 10:18:43     [echo] -- 10:18:43     [echo] README 10:18:43     [echo] -- 10:18:44     [exec] Result: 1 10:18:44     [exec] Result: 1 10:18:45     [echo] .sql file changed? false 10:18:45 10:18:45 php-lint: 10:18:45     [echo] Blindly running `php -l` on modified PHP files: 10:18:45 10:18:45 BUILD SUCCESSFUL 10:18:45 Total time: 2 seconds 10:18:45 [workspace] $ /bin/sh -xe /tmp/hudson4155017858026670472.sh 10:18:45 + /var/lib/jenkins/bin/wmfgrunt --cwd /var/lib/jenkins/jobs/mediawiki-core-lint/workspace jshint 10:18:45 Running "jshint" task 10:18:45 OK 10:18:45 10:18:45 Done, without errors. 10:18:45 Finished: SUCCESS

That works since Zuul crafted a ref for us and that is the one given to the git plugin:

git fetch -t /var/lib/zuul/git/mediawiki/core refs/zuul/master/Z30e30ecf23a14881a01ccfec52cbc465

Need to find a way to make Zuul to update the repo for tags too.

Zuul related debug log:

2013-01-17 10:18:41,570 DEBUG zuul.IndependentPipelineManager: Launching jobs for change  2013-01-17 10:18:41,570 DEBUG zuul.Repo: Resetting repository /var/lib/zuul/git/mediawiki/core 2013-01-17 10:18:41,795 DEBUG zuul.Repo: Checking out master 2013-01-17 10:18:41,830 DEBUG zuul.Repo: Merging refs/changes/17/17/3 2013-01-17 10:18:42,063 DEBUG zuul.IndependentPipelineManager: Found job mediawiki-core-lint for change  2013-01-17 10:18:42,064 INFO zuul.Jenkins: Launch job mediawiki-core-lint for change  with dependent changes [] 2013-01-17 10:18:42,104 DEBUG zuul.IndependentPipelineManager: Adding build  of job mediawiki-core-lint to change 
 * 1) Below Zuul initialize the local repository:
 * 1) And craft the change for us:
 * 1) Then trigger the job: