Continuous integration/Zuul/gearman upgrade
notes dropped by Antoine "hashar" Musso to prepare the migration of Zuul to a version using Gearman
Preparation
[edit]- Make sure Zuul gearman port is firewalled out and only access from localhost Gerrit change 94136 Done
- Python dependencies are installed on gallium Gerrit change 93454 Done
- Verify the configuration files contains gearman configuration Gerrit change 93457 Done
- python-voluptuous version is below 0.7.x Done
Upgrading
[edit]- shutdown Zuul (sudo /etc/init.d/zuul stop) Done
- bump
master
branch to the version in labs: git push -f gerrit labs:master Done - tag new version (tagged: wmf-deploy-20131120) Done
- make sure the branch has the dependency hack Gerrit change 93452 (it is in
labs
branch already anyway). Done - on gallium update the code as root:
cd /usr/local/src/zuul git remote update git reset --hard origin/master git tag wmf-deploy-`date +%Y%m%d` HEAD git push --tags
Done
Then actually install the new version:
HTTP_PROXY=. HTTPS_PROXY=. python setup.py install
If it fails, it is surely because a python dependency is missing or the source code is missing the hack Gerrit change 93452. python-voluptuous should be below 0.7 since we don't have a later version in our apt repository yet. Done
Minor issue Antoine had to manually install the python-pip
package to be able to run the install. Package removed after upgrade.
Restarting
[edit]Update Zuul layout configuration by merging in Gerrit change 93590 and pulling in /etc/zuul/wikimedia as user jenkins. Done
Minor issue had to delete old versions because pip did not found the new version but was still relying on the previous one. Aka: rm -fR /usr/local/lib/python2.7/dist-packages/zuul*
. Result:
$ zuul-server --version Zuul version: 6241272
Validate the layout using the new code base: Done
zuul-server -c /etc/zuul/zuul.conf -l /etc/zuul/wikimedia/layout.yaml -t
If all went fine, start Zuul!! Done
sudo /etc/init.d/zuul start
Look at the logs in /var/log/zuul/zuul.log or /var/log/zuul/debug.log
Enable Gearman
[edit]Enable Gearman in Jenkins https://integration.wikimedia.org/ci/configure then check 'Enable Gearman'. It needs to point to 127.0.0.1:4370. The Test Connection button would confirm a connection to Zuul daemon.
Trigger a job. Done
Success!FAILURE :-(
Restropective to be posted on wikitech-l. Basically the Jenkins gearman plugin triggers job on any slave, regardless of their jobs restrictions. That is an upstream bug.