Continuous integration/Quibble

Jump to navigation Jump to search

Quibble is a Python script for setting up a MediaWiki instance and running various tests against it.

It works by cloning MediaWiki core and several extensions, installing dependencies, creating the database, and running one or more test commands.

Further reading[edit]

Creating and deploying a new Quibble release[edit]

This is moderately fiddly and stressful, and should be done when you have a fair amount of time free and CI is relatively quiet; Fridays are commonly picked.

  1. Create the new release tag (example):
    1. Run quibble's master branch locally to ensure that you're confident the basics work; this will be imperfect, but is a good start.
    2. Determine the new version number
    3. Update CHANGELOG.rst, replacing the placeholder section with a list of every commit since the previous release
    4. Create this update as a change, submit it to code review, and wait for it to be merged
    5. Once merged, tag the tip of the branch with git tag -s 0.0.43 -m 'Signed 0.0.43 release' (updated with the appropriate version number) and the publish with git push --tags.
    6. Update CHANGELOG.rst to put in the next release's placeholder section.
  2. Create the images using the tag (example)
    1. In integration-config's dockerfiles directory, manually edit the Dockerfile.template for the primary quibble image or images to specify the new version (as of 2020-06-04, this is for quibble-stretch and quibble-buster).
    2. Use docker-pkg to create the appropriate cascade of changelog updates, e.g. docker-pkg -c dockerfiles/config.yaml --info update --reason "Bump quibble to 0.0.44" --version 0.0.44 quibble-buster dockerfiles/
    3. Build the images locally to ensure they still build: docker-pkg -c dockerfiles/config.yaml --info build --use-cache dockerfiles/
    4. Check the locally-built images to ensure that quibble correctly updated: ./dockerfiles/debug-image quibble-buster-php72 and then head /usr/local/bin/quibble and see that the correct version number is shown.
    5. Create this update as a change, submit it to code review, and wait for it to be merged.
    6. Create and publish the new images on the CI production server: tox -e fabric -- deploy_docker (remember to !log this action in IRC) – 🐌 this will take a while, perhaps half an hour or more
  3. Switch CI jobs over to the new image (example)
    1. In integration-config's jjb directory, update the Jenkins job-builder YAML files to specify the new docker images: sed -i '' s/0.0.42/0.0.43/' jjb*
    2. Verify that these jobs build correctly: tox -e jenkins-jobs -- test config/jjb/ -o output/
    3. Get a list of all jobs which will be updated by the change; there will be around 150 as of 2020-06-04.
    4. Push these updates as a commit to code review.
    5. Based on your knowledge of the changes in quibble, pick a likely simple, infrequent, fast-running job to manually change over, and push it: tox -e jenkins-jobs -- --conf jenkins_jobs.ini update ./jjb/ 'quibble-vendor-mysql-php72-docker'
    6. Manually trigger run a run of this job through the Jenkins Web interface, and carefully watch the output to ensure it works as it used to where it should, and in a different way where quibble's changes should change things.
    7. If you spot an error, rollback the job to the definition in master immediately rather than debugging live; remember that other people's workflows depend on CI continuing to work.
    8. Repeat this with increasingly major/high-profile jobs until you are satisfied that you or someone else on IRC would have noticed if the jobs were broken.
    9. Update the rest of the jobs in your list.
    10. Merge the commit as deployed.
    11. Continue to monitor CI for a while, in case things blow up despite your hard work. If it does, revert everything.

External links[edit]