Continuous integration/Quibble/nan
Appearance
Quibble is a Python script for setting up a MediaWiki instance and running various tests against it.
It works by cloning MediaWiki core and several extensions, installing dependencies, creating the database, and running one or more test commands.
Iân-sin ua̍t-to̍k
[edit]- Introducing Quibble, Antoine Musso (April 2018), phabricator.wikimedia.org.
- https://doc.wikimedia.org/quibble/
Tshòng-kiàn hām pòo-sū sin-ê Quibble pán-pún
[edit]This is moderately fiddly and stressful, and should be done when you have a fair amount of time free and CI is relatively quiet; Fridays are commonly picked.
Creating the release
[edit]See RELEASE.rst in the repository.
Creating images
[edit]This covers the creation of Docker images using the Quibble git tag (example)
- In integration-config's
dockerfiles
directory, manually edit theDockerfile.template
for the primary Quibble image to specify the new version (as of 2022-02-02, this is forquibble-buster
). That is done in the pip3 install command line. - Use docker-pkg to create the appropriate cascade of
changelog
updates, e.g.docker-pkg -c dockerfiles/config.yaml --info update --reason "Bump quibble to 0.0.44" --version 0.0.44 quibble-buster dockerfiles/
- Build the images locally to ensure they still build:
docker-pkg -c dockerfiles/config.yaml --info build --use-cache dockerfiles/
- Check the locally-built images to ensure that quibble correctly updated:
./dockerfiles/debug-image quibble-buster-php72
and thenhead /usr/local/bin/quibble
and see that the correct version number is shown. - Create this update as a change, submit it to code review, and wait for it to be merged.
- Create and publish the new images on the CI production server:
./fab deploy_docker
(remember to!log
this action in IRC) – 🐌 this will take a while, perhaps an hour or more
Update Jenkins jobs
[edit]Switch CI jobs over to the new image (example)
- In integration-config's
jjb
directory, update the Jenkins job-builder YAML files to specify the new docker images:sed -i '' s/:0.0.42/:0.0.43/' jjb*
- Verify all Quibble images have been updated:
./utils/docker-updates
- Verify that these jobs build correctly:
./jjb-test -o output/
- Get a list of all jobs which will be updated by the change; there will be around 150 as of 2020-06-04.
- Push these updates as a commit to code review.
- Based on your knowledge of the changes in quibble, pick a likely simple, infrequent, fast-running job to manually change over, and push it:
./jjb-update 'quibble-vendor-mysql-php72-docker'
- Manually trigger run a run of this job through the Jenkins Web interface, and carefully watch the output to ensure it works as it used to where it should, and in a different way where Quibble's changes should change things.
- If you spot an error, rollback the job to the definition in master immediately rather than debugging live; remember that other people's workflows depend on CI continuing to work.
- Repeat this with increasingly major/high-profile jobs until you are satisfied that you or someone else on IRC would have noticed if the jobs were broken.
- Update the rest of the jobs in your list.
- Merge the commit as deployed.
- You most probably want to verify that a
recheck
onmediawiki/core
andmediawiki/extensions/Wikibase
pass fully. - Continue to monitor CI for a while, in case things blow up despite your hard work. If it does, revert everything.