FileBackend design considerations

This project page is for ideas about the FileBackend class rewrite.

The FileBackend project is a refactoring project which will split off backend operations from the FileRepo class hierarchy. The idea is to separate metadata storage from file storage, so that metadata features such as Commons (Foreign*Repo) can be implemented independently from file storage features (Swift etc.), allowing the two to be combined in arbitrary ways.

Applications brainstorming

 * Filesystem
 * Swift
 * Azure
 * Amazon S3

Backend considerations
Current architecture assumes:
 * We can use FS path-like names (called storage paths).
 * If a backend can't do this, it can always normalize them in some fashion.
 * We can list objects/files that have a path starting with a prefix
 * If a backend can't do this, then it will need DB tracking get the lists.

Locking considerations
There is a LockManager heirarchy for handling locks.
 * Includes DBFileLockManager class
 * We can use DB-based locking or make a subclass to use some distrubuted lock manger that's already out there. Things like FSRepo or backend uses without involving `image` table rows really need this, since there is no locking at all currently. LocalRepo tries to do some locking by locking `image` table rows.

Performance considerations
On backends without native file/object renaming support, copy+delete has to be used. This kind of sucks for super large files.

FileRepo integration

 * FSRepo and LocalRepo assume an FS backend. We need to change that to remove that assumption but also maintain backwards compatibility, which is a bit tricky.

Other integration

 * thumb.php
 * image_auth.php
 * thumb_handler.php