Talk:Requests for comment/Standardized thumbnails sizes

Browser scaling

Asking the browser to rescale an image will normally yield an ugly result, so I don't think it is an option unless you have a strategy to quickly eliminate all odd-sized images. Elimination seems to be difficult with templates and associated CSS tuned to exact dimensions.

It might be worth tackling large thumbnails first as those are the most expensive and relatively rare. Gabriel Wicke (GWicke) (talk) 22:17, 7 February 2013 (UTC)

Browsers are pretty good about scaling these days, and lots of people browse with their browser zoomed or on a phone or tablet that's scaling everything anyway. I wouldn't worry too much, but it's probably wise to do some testing in various browsers. brion (talk) 00:41, 8 February 2013 (UTC)

@Brion: Your hi-res image serving has been very effective. But editors do notice blurriness and complain at VPT when they accidentally change their zoom on desktop browsers, so optimal rendering at 100% zoom on desktop browsers should continue to be a key requirement for the foreseeable future.

The proposal has the following drawbacks:

1. Browser scaling of a server-scaled image will generally produce a worse result than a single scaling, unless the cache-scaled image is more than about 200% the display dimensions; but serving 200%+ images for browsers to downscale is unlikely to be acceptable to low-bandwidth users, so quality or speed would be sacrificed.
2. Many images are used as icons, where precise pixel control is necessary.
3. Serving images at higher-than-displayed resolutions would increase bandwidth.
4. There is no evidence that the current flexibility is causing adverse results.
5. Conversely, the proposal would still permit numerous different sizes to be cached for any uploaded image. So the thumbnail servers would remain just as vulnerable to DOS requests for rare-but-permitted large scalings of numerous different images.

Therefore served sizes should be identical to rendered size (as now) or at least the lesser of original size and 200% of the intended size (but only if low-bandwidth users do not object).

If DOS mitigation is an intended result, it requires a completely different approach. (Possibilities might be indentifying pathological request patterns; or serving unscaled images for uncached thumbnails that have not been generated within a set interval.) — Richardguk (talk) 04:37, 9 February 2013 (UTC)

As I said on the engineering mailing list, client-side scaling will produce linear artifacts if the destination size is close to the source size. If there are in fact technical issues with the way thumbnail caching is handled at present, somebody should tell me exactly what those issues are, and why the potential solutions I have suggested will not work. Tim Starling (talk) 04:12, 8 February 2013 (UTC)

URLs compatibility

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

I assume this will still be compatible with our current way of specifying thumbnail sizes, except that it will return the closest match? That is, as a client, I do not need to have any knowledge of 'allowed' sizes. If I request a thumbnail for 310px, I should automatically get a 320px thumbnail and some way of knowing it was rounded up. This should be at the thumb scaler level rather than the API level, so that'll allow clients to continue building thumbnail URLs with knowledge of just the filename without adding an extra network request. Yuvipanda (talk) 07:45, 8 February 2013 (UTC)

We're suckers for backwards compatibility so that's probably what we'd end up doing.

Or, if Tim's position prevails we'll keep making arbitrary thumbs, we just won't store them outside of cache. Either way I suspect we've got URL compatibility. brion (talk) 16:25, 8 February 2013 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Caching strategy

The RFC states the second part of the problem as: "the thumbnails are hard to cache properly since we have to cache a copy of each of the sizes". But there might be a better caching strategy than simply refusing to cache non-standard sizes.

Potential thumbnail strategies:

Serving

(A) serve arbitrary sizes as requested;

(B) serve only preferred sizes (rounding non-standard sizes according to some algorithm).

Storing

(1) store every requested thumbnail indefinitely;

(2) store only preferred-size thumbnails indefinitely.

Caching

(i) retain indefinitely (ad hoc deletion);

(ii) periodically delete the longest-unserved thumbnails (LRU), regenerating and recaching them if re-requested.

The status quo seems to be A.1.i. The proposal seems to be to switch to B.2.i or possibly A.2.i (I think Brion implies above that A.2 is "Tim's position").

But to prevent excessive thumbnail file storage, all that is needed is a better caching strategy. "Least Recently Used" seems like a sound but simple and non-disruptive basis for predicting which thumbnails will not be requested again.

So, instead of inconveniencing users by switching serving strategy or storing strategy, why not just switch caching strategy from (i) to (ii)? — Richardguk (talk) 05:49, 9 February 2013 (UTC)

Seems to be a good amendment to the "possible solution" part. Would you mind enhancing it with your above text? We still have to expose the exact problem though. Antoine "hashar" Musso (talk) 22:58, 11 February 2013 (UTC)

Snapping to an approved size without client side scaling

An option discussed briefly in person (and I believe on the Etherpad) today was to snap to an approved size without client-side scaling. In other words, if you put:

[[File:some-image.jpg|thumb|199px]]

you get something like:

<img width="200" height="244" src="some-image-200px.jpg>

The 199 is silently treated as 200 everywhere. This can definitely still affect layouts (particularly when the difference is greater), but will not cause new client-side scaling. Superm401 - Talk 08:36, 11 September 2013 (UTC)

That probably won't work well when trying to use fixed vertical sizes matched on different widths (think of photo galleries, perhaps?), or small sizes where slight pixel counts will dramatically change the size. brion (talk) 18:14, 11 September 2013 (UTC)

Potential client: Extension:MultimediaViewer

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Hi there,

We've been struggling with a similar issue in the Multimedia team. If we want Extension:MultimediaViewer to scale properly, we should probably bucket our requests for thumbnails to a specified list of sizes. This may be a way to test different solutions to this problem without necessarily figuring out a canonical solution in core.

Basically the idea would be to try bucketing the thumbnail requests to a set of 10-20 predetermined sizes, then either requesting the nearest one above the ideal size and scaling down on the client, or requesting the nearest one below and showing the image in a container with a frame.

If we can be of any use in testing possible solutions, we'd love suggestions as to implementation details. MarkTraceur (talk) 01:07, 7 November 2013 (UTC)

Let's start by just devising some buckets and using them explicitly in MultimediaViewer, then if we find that successful we can start building it into the backend?

The traditional default selectable File: page display sizes appear to be some classic full-screen sizes:

• 320x240
• 640x480
• 800x600 (default)
• 1024x768
• 1280x1024 (slightly different aspect ratio)

Some things to consider:

• do these fit default aspect ratios well?
• do we also need to bucket display density variants?
• what about dealing with panoramic images? (very long or tall aspect ratios, work best when requesting a larger total size in the longer dimension and allowing some kind of scroll/panning) Should we set size by the long dimension, or the short dimension ('show horizontal panoramas at 600px high, however wide they are' or 'show vertical panoramas at 800px wide, however tall they are)? brion (talk) 19:47, 7 November 2013 (UTC)

I'll probably add 1920x1080 just to make sure we're supporting everyone, and use the full-size image for anything above that.

When it comes to panoramic images, I guess we'll need to determine that based on the aspect ratio of the thumbnail on the page - AFAIK it's guaranteed to preserve the original ratio, so that should be OK, but we'll determine whether the ratio is e.g. 3 or 1/3, and if so, use the height or width as the only limiting factor, then enable scrolling.

Good plan :) MarkTraceur (talk) 19:58, 7 November 2013 (UTC)

Be very, very careful about using "full-size" image. Sometimes that's a 4MP or 8MP photo, but sometimes it's a 100MB TIFF file. And of course sometimes it requires rotation for display in browser, or transformation to a JPG, PNG, etc. Generally... never EVER use the original file as anything but a download link, unless the API happens to return you the original file when you ask for a rendered thumbnail... brion (talk) 20:13, 7 November 2013 (UTC)

I mean, the only time that would happen is if the screen size were bigger than 1920x1080. And that's...unlikely.

I could add one more size that's double that, I guess. Maybe two more (one double, one quadruple) just to be safe. MarkTraceur (talk) 20:25, 7 November 2013 (UTC)

Not that unlikely; I have one of those monitors at home, and yes, I do use it full-screen in 1:1 resolution. :-) Jdforrester (WMF) (talk) 20:35, 7 November 2013 (UTC)

You'll have people with non-retina screens in the 2560x1440 range running those big 27-30" monitors.

You'll also have people with retina screens where the screen density bumps you up from 1280x800 or 1440x900 CSS pixels to 2560x1600 or 2880x1800.

Common densities are going to be in the 1.0, 1.5, and 2.0 ranges, but others may appear (especially if you have a funny zoom, etc) brion (talk) 20:08, 8 November 2013 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Meaningful statistics

Concerning the goal to minimize cpu load in generating an excessive amount of thumbnails for a single image, the presented statistics of different thumbnail sizes isn't necessarily meaningful. If each of the images would only ever have exactly one thumbnail generated, it would not harm anyone if the thumbnails all have various and weird sizes. The main problem is that one image has dozens of different thumbnails; so a statistics like "1234 images have 7 thumbnails" would be more meaningful to calculate the possible impact in saving of CPU/storage for additionally generated thumbnails.

Concerning possible sizes of thumbnails, if the space used by them is an issue in any way, I would suggest that there is a predefined list of possible sizes, and an algorithm which takes as input N desired thumbnail sizes, and outputs M thumbnail sizes to be generated/kept, so that at most M thumbnails would be kept.

For the list of those N thumbnail sizes, some suggested ones should be compared with the presented sizes statistics to assess one that matches the best in terms of "minimal amount of scaling needed". 213.61.9.75 16:57, 15 November 2013 (UTC)

Needed for MobileFrontend

Several features in MobileFrontend are now surfacing thumbnails in association with article lists: search, nearby, related articles, watchlist, etc. In order for these features to work smoothly we would really like to have standardized, pre-rendered thumbnails. Kaldari (talk) 21:36, 17 December 2013 (UTC)

Still relevant?

The following discussion is closed. Please do not modify it. Subsequent comments should be made on the appropriate discussion page. No further edits should be made to this discussion.

---

Is this RfC still relevant, or is it superseded by other, more recent RfCs? Sharihareswara (WMF) (talk) 15:15, 23 April 2014 (UTC)

This was discussed further at wikitech-l "Larger thumbnails for MediaWiki / Wikimedia wikis?" from April 9-14, and mentioned in this other thread "Image scaling proposal: server-side mip-mapping" (part 1 from April 30 and part 2 up till May 13 (None of which is in Gmane?!)) - I'm not sure what the resolution was though. Quiddity (WMF) (talk) 17:04, 21 May 2014 (UTC)

In short, relevant but outdated. There are conversations about generating some thumbnails on upload and maybe using them to generate the rest of the thumbnails. It's still in the research phase, I think. Tgr (WMF) (talk) 18:08, 21 May 2014 (UTC)

Ping :) Is this still Antoine's RfC or is it something the multimedia group has taken over? If the latter, could you change authorship in the infobox? Thanks! Sharihareswara (WMF) (talk) 16:58, 17 June 2014 (UTC)

We had several mailing list discussion in 2012 / beginning of 2013 regarding optimizing the thumbnails rendering. That RFC is merely a summary of the discussions and is intended to avoid repeating ourself on each discussion. I am not leading the RFC by any mean, would be nice to have the new multimedia team to take leadership there. Antoine "hashar" Musso (talk) 10:29, 18 June 2014 (UTC)

We're awaiting Ops input. Sharihareswara (WMF) (talk) 19:40, 21 July 2014 (UTC)

The discussion above is closed. Please do not modify it. No further edits should be made to this discussion.

Ops discussion

Gilles said:

We're not looking to standardize the sizes at the moment, we're just considering rendering specific sizes at upload time or shortly after. More specifically, the buckets that Media Viewer is using. Which currently get generated when the first person uses Media Viewer on a given image.

Gilles had also mentioned in email:

Has the swift capacity been increased yet thanks to the new hardware? If so, could we resume the discussion of "pre"generating specific thumbnail sizes at upload time?

Media Viewer could benefit greatly from this performance-wise. As seen on this graph, the launch to all wikis affected the average considerably, since users started hitting a lot of images that didn't have Media Viewer-sized thumbnails yet: http://multimedia-metrics.wmflabs.org/dashboards/mmv#overall_network_performance-graphs-tab

Thumbnailing improvements are still in the works on our end, and the idea of not using swift anymore for those is definitely on the team's radar (we've started working on more modest thumbnailing improvements at this point), but if the capacity is there, we might as well improve the average image load time for our users, even if the ever-increasing swift use for thumbnail is still a problem itself.

In reply, Mark Bergsma said:

Yes, our Swift capacity expansion has completed. Although the capacity expansion had been planned with reduced storage for thumbs in mind for the future, work that hasn’t started yet, we should have space for this available right now and it shouldn’t be problematic provided that we monitor it well. So please go ahead with this for newly uploaded images - we’ll see how it goes.

Filippo will look into whether Swift’s TTL feature is usable for us these days; hopefully we can use it to reduce storage of unused thumbs / thumb sizes until we move them out of Swift completely.

(about choosing something other than swift in the future)

Yeah. As you know there’s a fair amount of desire to change the way thumb handling & storage works, support more variants (sizes, quality), etc by various teams. I’m starting to think it would be good for us to organise a sprint on this very topic, get the multimedia team, relevant Ops people and some other interested developers from other teams and really dive into these problems. Sharihareswara (WMF) (talk) 23:07, 1 August 2014 (UTC)

Template:Largethumb

Note that nlwiki (for example) wishes to have *two* standard thumbnail sizes on the wiki, and is currently using the {{largethumb}} template to do this. This causes various problems for VE.

The solution is probably some more flexible method of applying named "standard image styles" to an image, which would include a default size parameter. Then nlwiki would use something like [[File:Foo.jpg|style=large]] instead of a bespoke template. cscott (talk) 19:33, 26 February 2015 (UTC)

Generate statistics for most requested non-existing thumb widths

To add more useful thumb widths to $wgUploadThumbnailRenderMap I generate a list of all thumb widths that are created because the thumb didn't exist yet.

Maybe it's useful to someone: MediaTransformOutput.php > hasFile()

	public function hasFile() {		
		// Generate statistics for most requested non-existing thumb widths
		// includes x1.5 and x2
		global $wgUploadThumbnailRenderMap;
		if( $this->path != null // not when thumb size is >= original and original is returned
		    && !$this->isError() // only if thumb is really created
		    && !in_array( $this->width, $wgUploadThumbnailRenderMap ) // not if is in job list to be created
		  ) {	
			$file = '/path/to/thumbs_not_existing_on_request';
			$current = file_get_contents($file);
			$current .= $this->width . "\n";
			file_put_contents($file, $current);
		}
		
		// If TRANSFORM_LATER, $this->path will be false.
		// Note: a null path means "use the source file".
		return ( !$this->isError() && ( $this->path || $this->path === null ) );
	}

After some hours / days grab the most requested thumb sizes:

sort /path/to/thumbs_not_existing_on_request | uniq -c | sort -rn | head -n 30

Example after some hours:

99 240
60 170
6 320
...  

>> Adding 240 and 170 to $wgUploadThumbnailRenderMap might be worth it. Subfader (talk) 13:46, 13 September 2015 (UTC)