Extension:SwiftCloudFiles

From MediaWiki.org
Jump to: navigation, search
MediaWiki extensions manual - list
Crystal Clear action run.png
SwiftCloudFiles

Release status: stable

Implementation File repository
Description Includes and registers the php-cloudfiles API (https://github.com/rackerlabs/php-cloudfiles), allowing MediaWiki to interface with an OpenStack Swift storage backend.
Author(s) Aaron Schulztalk
License No license specified
Download
Tags
Swift, OpenStack, Rackspace, Cloudfiles

Translate the SwiftCloudFiles extension if possible

Check usage and version matrix; code metrics
Bugs: list open list all report

SwiftCloudFiles uses the Swift Backend that is built into the new wiki core (1.20+). Uploaded files are stored across a swift files container system that allows for a multiple point-of-failure configuration and reduces the risk of data corruption or loss. In a typical wiki configuration, files that are uploaded live in a single that a wiki administrator must maintain. When using a single server, that server provides a single point of failure to all uploaded files. In today's fast-paced world, that can slow down an administrator significantly when he is responsible for syncing, backing-up and or restoring files. With this extension, a wiki administrator can build a private swift instance or utilize a system such as Rackspace Cloud Files for their uploads. The configuration examples below are operational with a Rackspace CloudFiles instance, however they should work with a private swift instance as well. Jwestyp (talk) 21:42, 27 June 2012 (UTC)

Warning Warning: The php-cloudfiles bindings are no longer being maintained and their use is deprecated and are being replaced with the php-opencloud bindings. The php-cloudfiles bindings will not be available after August 1, 2013. See http://php-opencloud.com for more info.


Installation[edit | edit source]

Prerequisites[edit | edit source]

Required PHP extensions:

  • curl (apt-get install php5-curl)
  • fileinfo
  • mb_string


Extension[edit | edit source]

  1. Use the download links above to download the package.
  2. Place in your extensions directory, i.e. http://www.domain.com/w/extensions/.
  3. You will need to create private credentials file
  4. You will need to add configuration settings to the LocalSettings.php file.


Private Credentials File[edit | edit source]

Below is an example of the wgAuth.php file that you would have included in a private folder. This folder should be located outside the webroot on your server. If you webroot is located at '/var/www/html/' then a good place for the private folder at '/var/www/private/'.

<?php
$wgSwiftConf = array(
    // This the the url for your authentication server.
    // For Rackspace Cloud Files, this would be: auth.api.rackspacecloud.com
    //     The current version of SwiftCloudFiles and php-cloudfiles appears to be incompatible with
    //     the newer identity.api.rackspacecloud.com/v1.0
    'authUrl' => 'auth.api.rackspacecloud.com',

    // Some swift authentications require a username to be used with the
    // account or tenant id. (i.e. 'username:tenant')
    'user' => 'Username',

    // The API Key for used for authentication
    'key' => 'API Key' );
?>

Standard LocalSettings.php configuration example:[edit | edit source]

require( "$IP/extensions/SwiftCloudFiles/SwiftCloudFiles.php" );
require( "/var/www/private/wgauth.php" ); // Location of private credentials file.

#$wgHashedUploadDirectory = false;

$wgFileBackends[]  =  array(
        'name'                => 'localSwift',
        'class'               => 'SwiftFileBackend',
        'lockManager'         => 'nullLockManager',
        'swiftUseCDN'         => true,
        'parallelize'         => 'implicit',

        //These three entries are taken from the wgauth.php private file.
        'swiftAuthUrl'        => $wgSwiftConf['authUrl'],
        'swiftUser'           => $wgSwiftConf['user'],
        'swiftKey'            => $wgSwiftConf['key']
);

$wgLocalFileRepo  =  array(
        'class'              => 'LocalRepo',
        'name'               => 'local',
        'backend'            => 'localSwift',
        'scriptDirUrl'       => $wgScriptPath,
        'scriptExtension'    => $wgScriptExtension,
        'url'                => 'http://cdn.cdnprovider.com', // Public file location url ->Maybe not needed?<-
        'hashLevels'         => 0,
        'deletedHashLevels'  => 0,
        'zones'              => array(      
            //Change container and url values to match your Swift provider configuration settings
            'public'  =>  array( 'container' =>  'public', 'url' => 'http://cdn.cdnprovider.com' ),
            'thumb'   =>  array( 'container' =>  'thumb',  'url' => 'http://cdn.cdnprovider.com' ),
            'temp'    =>  array( 'container' =>  'temp',   'url' => 'http://cdn.cdnprovider.com' ),
            'deleted' =>  array( 'container' =>  'deleted' ),    // deleted items don't have a URL
        )
);

Using: $wgHashedUploadDirectory, $wgFileBackends, $wgLocalFileRepo

Settings information[edit | edit source]

from <MediaWiki Install Directory>/includes/filebackend/SwiftFileBackend.php

        /**
         * @see FileBackendStore::__construct()
         * Additional $config params include:
         *   - swiftAuthUrl       : Swift authentication server URL
         *   - swiftUser          : Swift user used by MediaWiki (account:username)
         *   - swiftKey           : Swift authentication key for the above user
         *   - swiftAuthTTL       : Swift authentication TTL (seconds)
         *   - swiftTempUrlKey    : Swift "X-Account-Meta-Temp-URL-Key" value on the account.
         *                          Do not set this until it has been set in the backend.
         *   - swiftAnonUser      : Swift user used for end-user requests (account:username).
         *                          If set, then views of public containers are assumed to go
         *                          through this user. If not set, then public containers are
         *                          accessible to unauthenticated requests via ".r:*" in the ACL.
         *   - swiftUseCDN        : Whether a Cloud Files Content Delivery Network is set up
         *   - swiftCDNExpiry     : How long (in seconds) to store content in the CDN.
         *                          If files may likely change, this should probably not exceed
         *                          a few days. For example, deletions may take this long to apply.
         *                          If object purging is enabled, however, this is not an issue.
         *   - swiftCDNPurgable   : Whether object purge requests are allowed by the CDN.
         *   - shardViaHashLevels : Map of container names to sharding config with:
         *                             - base   : base of hash characters, 16 or 36
         *                             - levels : the number of hash levels (and digits)
         *                             - repeat : hash subdirectories are prefixed with all the
         *                                        parent hash directory names (e.g. "a/ab/abc")
         *   - cacheAuthInfo      : Whether to cache authentication tokens in APC, XCache, ect.
         *                          If those are not available, then the main cache will be used.
         *                          This is probably insecure in shared hosting environments.
         *   - rgwS3AccessKey     : Ragos Gateway S3 "access key" value on the account.
         *                          Do not set this until it has been set in the backend.
         *                          This is used for generating expiring pre-authenticated URLs.
         *                          Only use this when using rgw and to work around
         *                          http://tracker.newdream.net/issues/3454.
         *   - rgwS3SecretKey     : Ragos Gateway S3 "secret key" value on the account.
         *                          Do not set this until it has been set in the backend.
         *                          This is used for generating expiring pre-authenticated URLs.
         *                          Only use this when using rgw and to work around
         *                          http://tracker.newdream.net/issues/3454.
         */


from <MediaWiki Install Directory>/includes/filebackend/FileBackend.php (this is inherited from SwiftFileBackend.php)

        /**
         * Create a new backend instance from configuration.
         * This should only be called from within FileBackendGroup.
         *
         * $config includes:
         *   - name        : The unique name of this backend.
         *                   This should consist of alphanumberic, '-', and '_' characters.
         *                   This name should not be changed after use (e.g. with journaling).
         *                   Note that the name is *not* used in actual container names.
         *   - wikiId      : Prefix to container names that is unique to this backend.
         *                   If not provided, this defaults to the current wiki ID.
         *                   It should only consist of alphanumberic, '-', and '_' characters.
         *                   This ID is what avoids collisions if multiple logical backends
         *                   use the same storage system, so this should be set carefully.
         *   - lockManager : Registered name of a file lock manager to use.
         *   - fileJournal : File journal configuration; see FileJournal::factory().
         *                   Journals simply log changes to files stored in the backend.
         *   - readOnly    : Write operations are disallowed if this is a non-empty string.
         *                   It should be an explanation for the backend being read-only.
         *   - parallelize : When to do file operations in parallel (when possible).
         *                   Allowed values are "implicit", "explicit" and "off".
         *   - concurrency : How many file operations can be done in parallel.
         *
         * @param $config Array
         * @throws MWException
         */

Further Info[edit | edit source]

For more information on the configuration settings used in this extension, visit Manual:Configuration_settings