Reading/Multimedia/Architecture/Tech Debt Backlog

Loosely prioritized list of technical debt projects related to the Multimedia team.


 * 1) Chunked upload. Session and general more reliability
 * 2) Upload class (more extensible. The chunked uploading case is very ugly)
 * 3) deadlocks and long held locks in filebackend / file class when moving/deleting/uploading file.
 * 4) current image thumbnail should have timestamp in it for better caching clear
 * 5) Partial failures in filebackend (If things fail, either nothing should happen or everything should happen. Sometimes pages get deleted but not the file, or vice veras)
 * 6) Various swift layer improvements/thumbnailing cache (making thumbs cached at a different layer, maybe. Making content at an sha1 place and pretty urls only to user)


 * API upload is all kind of bad. everything under the directory /upload would be nice to rewrite.
 * though please be sure to test well with Upload Wizard and other API consumers, it's very easy to accidentally break edge cases (e.g. blacklist failures, duplicate checks, yadda yadda yadda)
 * backlink invalidation in commons would be nice to fix (22390)
 * Redirect of old filename on file move would be nice. (for hotlinkers like the wmf blog, and due to lack of purging cache of commons client wikis)
 * this is hard to implement as a general MediaWiki feature without changing to run files through PHP by default, but could be done with our special file backends...
 * Our use case is probably the most important. I don't think third parties use file redirects that much. Patch https://gerrit.wikimedia.org/r/#/c/80135/
 * versioned urls for thumbnails
 * large file support in general - deadlocks abound
 * way FileRepo stores files is bad in general...requires pessimistic in general (see Requests for comment/Refactor on File-FileRepo-MediaHandler)
 * Why chunked upload should be fixed:
 * Upload actions that go through the job queue should be more reliable.
 * Suggest that long term stop using session data for this (people log in, log out, etc).
 * possible several pieces here: fix job queue to be sane, fix upload jobs to not (ab?)use session storage, and ?
 * HTCP purges need active monitoring - DONE