User:DWalden (WMF)/Thumbor migration

Project documentation[edit]

For Phabricator board, team members involved, etc. see 2022_Thumbor_updates.

The Phabricator board is: https://phabricator.wikimedia.org/tag/thumbor_migration/

Feature documentation[edit]

Thumbor allows users to see resized versions of media hosted on a wiki (e.g. Commons). For example, for thumbnails.

It can be used for any media including images, videos, PDFs, etc.

Test documentation[edit]

Where to test it[edit]

Thumbor is available on the beta and production environments. However, to test the latest version you need to install it locally.

How to install locally[edit]

git clone https://gerrit.wikimedia.org/r/operations/software/thumbor-plugins/
cd thumbor-plugins

Create a thumbor.conf in the same directory. Here is the one I currently use phab:P37284.

If you want to run it in docker (recommended):

blubber .pipeline/blubber.yaml dev > Dockerfile
make build

Otherwise, if you want to run it on "bare metal":

python3 -m pip install -r requirements.txt
python3 setup.py build
python3 setup.py install --user
thumbor --port 8800 --conf=thumbor.conf -a wikimedia_thumbor.app.App

Thumbor should then be running on port 8800.

To check it is running, visit: http://localhost:8800/thumbor/unsafe/320x/https://upload.wikimedia.beta.wmflabs.org/wikipedia/en/1/1b/CuteGoat.jpg

(See also README.)

Coverage criteria[edit]

Types of media (PNG, JPG, OGV, webm, 3D, PDF, DjVU, SVG, etc.)

Oracles[edit]

Previous versions of thumbor (e.g. those installed on beta or production)

Risks[edit]

Quality criteria[edit]

Techniques[edit]

Blink testing
Automatic comparison (e.g. using imagemagick's compare or GIMP)

Test data[edit]

Any media (including video, pdf) on Commons can be passed to Thumbor. There are lots of places to find images.

https://commons.wikimedia.beta.wmflabs.org/wiki/Special:NewFiles (allows you to filter by types of media, e.g. vector, bitmap, video)
https://commons.wikimedia.beta.wmflabs.org/wiki/Special:MIMESearch (allows you to filter by mime type)
allimages API (see script)

Test tools and scripts[edit]

Script to compare an instance of thumbor running locally with thumbor on beta: https://gitlab.wikimedia.org/dwalden/thumbor-testing/-/blob/main/comparison_prototype.sh
- You should change line 31 with output from the below script
Script to find lots of media for testing: https://gitlab.wikimedia.org/dwalden/thumbor-testing/-/blob/main/allimages.py

Logs and debugging[edit]

To see more detailed logs on the command line, run with the -d -l debug flags, like:

thumbor -d -l debug --port 8800 --conf=thumbor.conf -a wikimedia_thumbor.app.App

Current testing status[edit]

Test strategy[edit]

Compare the thumbnails generated by the latest version of thumbor with the older version running on beta.