Continuous integration/Qemu

Agent
To provision a new agent, follow Nova Resource:Integration/Setup to set up a Qemu worker agent for Jenkins.

The following step to create  is not yet puppetized: agent$ ssh-keygen -t rsa -b 4096


 * Don't use any actual e-mail address (leave the default to your shell name and agent hostname).
 * Don't set a pass phrase.
 * Name it like " ", for example.
 * Use sudo to move  and   to.
 * Set, so that it is available as read-only for the Jenkins user.

Create Qemu image
Create a Qemu-compatible vm.img that has the same Debian major version installed as the outer Qemu agent VM. Then: apt-get install libguestfs-tools
 * 1) Install the needed tool on the outer VM host

truncate -s 20G ./out.img virt-resize --expand /dev/sda1 ./vm.img ./out.img
 * 1) Create a new disk, and resize the old one into the new one

virt-filesystems --long --parts --blkdevs -h -a ./out.img
 * 1) Verify the new disk size

Launch for remote control
This is how Jenkins jobs launch the VM. This is different from the above provisioning workflow. Rather than getting an interactive shell directly from Qemu, we we launch the Qemu with an SSH port exposed, and login that way: you@agent$ qemu-system-x86_64 -device virtio-net,netdev=user.0 -netdev user,id=user.0,hostfwd=tcp::4293-:22 -smp 4 -m 4096 -nographic vm.img … you@agent$ install -m 600 /srv/vm-images/sshkey_qemu_root_v1 root.key you@agent$ ssh -i ./root.key -p 4293 root@localhost … root@img# … root@img# exit
 * 1) Terminal 1 (launch guest VM)
 * 1) Terminal 2 (create copy of key, then open shell via ssh)

Create new snapshot version
Launch a snapshot manually to create changes to the base image. Never launch or modify a snapshot directly. Always copy first. you@agent$ cp /path/to/thing-to-change.img ~/vm.img you@agent$ qemu-system-x86_64 -m 4096 -nographic vm.img Wait for the bootloader to pass, then once " " appears, enter " ".

A shell will appear shortly. Make your changes, then run  to log out from the VM.

Once back in the bootscreen, use  to exit from Qemu and return to the agent.

The vm.img file is now an updated snaphot, ready for publishing.

Publish new snapshot

 * 1) Before publishing a new snaphot, verify that you are able to launch the VM and connect to it over SSH using the  "Launch for remote control" steps above.
 * 2) Use sudo to move the img file from your home directory to . For example, , or  , where the sequence letter is for revisions of the same base.
 * 3) Set , so that it is available as read-only for the Jenkins user.

qemu-debian10buster
Current version:

Provisioning:


 * Snapshot of Debian 10 Buster with.
 * Grub configured to use a serial console.


 * Run


 * Required packages:
 * Optional packages:
 * Preload for mw-cli jobs (T248779):
 * Ensure no containers are running:
 * edit, and set   (The nano editor is pre-installed).
 * mkdir
 * copy contents from host file  to guest   (E.g. create the file with nano and copy the pub file contents from the other terminal tab)
 * run
 * copy contents from host file  to guest   (E.g. create the file with nano and copy the pub file contents from the other terminal tab)
 * run

Jenkins Jobs
The Jenkins jobs currently use https://gerrit.wikimedia.org/g/integration/config/+/master/jjb/qemu-run.bash

After publishing a new image, update this script to point to the new image version.

When making changes to this script, you will need to regenerate the Jenkins jobs that embed this script for the changes to take effect. (see Continuous integration/Jenkins job builder).