Extension:Simple Farm

Simple Farm is a simple MediaWiki farm extension allowing to run several wikis with just one physical MediaWiki directory. The extension loads a certain wiki by either a virtual script path (using a tiny bit of mod-rewrite) or by using sub-domains (requiring some more mod-rewrite). It is also possible to mix these methods for the same farm, not all members have to use the same method. Furthermore, 'Simple Farm' comes with a maintenance script which allows to maintain several farm members at the same time, e.g. doing database updates for all or certain farm members by executing just one command. A certain farm can also be chosen via command-line access, using an environment variable.

How it Works [edit]

Simple Farm kicks in before the MediaWiki has initialized all of its components. At this stage the wiki has not even made a connection to its database, global variables and functions are not all initialized, though most Extensions might already be registered. At this stage, Simple Farm takes over initialization to load a wiki from a certain database and set required global configuration variables accordingly. After that job is done, it is even possible to load further global settings according to one farm member's specific needs.

Basically the same happens in maintenance mode via the command-line. There is one script which will take commands which should be executed for one or more specific farm members. Then the script will connect to one wiki to have Mediawiki's maintenance functionality available, and from there it will handle the commands for each farm member's wiki independently.

Setup and Installation [edit]

First of all, you have to download the code and place the SimpleFarm directory within your MediaWiki extensions directory. Then add the following code near the bottom of your LocalSettings.php file:

# Simple Farm
require_once( "$IP/extensions/SimpleFarm/SimpleFarm.php" );
require_once( "$IP/FarmMembers.php" );
SimpleFarm::init();
/* ... Member Settings ... */

FarmMembers.php would be a new php file with the necessary farm setup you had to create within your MediaWiki installation directory. The content of that file will be explained next. The comment at the bottom is just a placeholder for additional per farm member configuration. Finally, you won't get around applying some rewrite rules to your wikis environment.

Setting-up Farm Members [edit]

This section will introduce available settings to control the farms behavior and to register members. This configuration might be in a separate php file so you won't have it all over your LocalSettings.php.

$egSimpleFarmMembers is the config varariable to define all the members of the wiki farm. It is an array containing a sub-array per farm member. The following associative Array keys are obligatory:

db: The farms database name
name: The name of the farm member wiki. This value will end up in $wgSitename.

Furthermore, one of the following methods has to be chosen:

addresses: one or more (as array of strings) server names associated with the wiki. Can be used for sub-domain based farm members like farm1.mw.org, farm2.mw.org... , farmX.mw.org If this method is chosen, $wgScriptPath must be set in LocalSettings.php. Even if this method is not chosen, the addresses key should contain one address anyway.

and/or:

scriptpath: Virtual script path to the particular wiki directory. Can be used for a rewrite rules based wiki farm setup. This will set the $wgScriptPath variable automatically. There should be a .htaccess file in the parent directory of the farm to redirect all paths to the farms MediaWiki directory. Example: mw.org/farm1, mw.org/farm2... , mw.org/farmX where for the first member the value had to be /farm1, for the second /farm2 and so on.

The following key is optional:

maintain

Flag whether or not the wiki is in maintaining mode right now. The following flags are allowed:

SimpleFarm::MAINTAIN_OFF / false: Normal mode, no maintenance is going on right now.
SimpleFarm::MAINTAIN_SIMPLE: Blocks simple browser access to the wiki but accessing the wiki with maintain url parameter still is possible.
SimpleFarm::MAINTAIN_STRICT: Blocks all attempts to access the wiki except for command-line based maintenance.
SimpleFarm::MAINTAIN_TOTAL: Block all attempts to access the wiki, even command-line maintenance.

Example for address-based setup [edit]

A valid address-based setup in a FarmMembers.php could look like this:

<?php
# Farm members:
$egSimpleFarmMembers = array(
        array(
                'name' => 'FruitMix-Wiki',
                'db' => 'fruitfarm',
                'addresses' => array( 'fruitmix.fruitfarm.org', 'allfruits.fruitfarm.org' ),
                'maintain' => false,
        ),      
        array(
                'name' => 'Apple-Wiki',
                'db' => 'fruitfarm_apples',
                'addresses' => 'apples.fruitfarm.org',
                'maintain' => false,
        ),      
        array(
                'name' => 'Peaches-Wiki',
                'db' => 'fruitfarm_peaches',
                'addresses' => 'peaches.fruitfarm.org',
                'maintain' => SimpleFarm::MAINTAIN_SIMPLE,
        ),
);

Example for scriptpath-based setup [edit]

A valid scriptpath-based setup in a FarmMembers.php could look like this:

<?php
# Farm members:
$egSimpleFarmMembers = array(
        array(
                'name' => 'FruitMix-Wiki',
                'db' => 'fruitfarm',
                'addresses' => 'fruitfarm.org',
                'scriptpath' => '/fruitmix',
                'maintain' => false,
        ),      
        array(
                'name' => 'Apple-Wiki',
                'db' => 'fruitfarm_apples',
                'addresses' => 'fruitfarm.org',
                'scriptpath' => '/apples',
                'maintain' => false,
        ),      
        array(
                'name' => 'Peaches-Wiki',
                'db' => 'fruitfarm_peaches',
                'addresses' => 'peaches.fruitfarm.org',
                'scriptpath' => '/peaches',
                'maintain' => SimpleFarm::MAINTAIN_SIMPLE,
        ),
);

Settings per Member [edit]

Simple Farm will just set the most important configuration variables, you might want to set certain configuration variables according to the chosen farm member. There are several ways of doing that, the two most convenient are presented here.

Using a Configuration File per Member [edit]

You can simply add the following lines to your LocalSettings.php after the SimpleFarm::init(); line.

# Allow custom config files per wiki in 'wikiconfigs' sub-directory:
if( file_exists( "$IP/wikiconfigs/$wgDBname.php" ) ) {
        include( "$IP/wikiconfigs/$wgDBname.php" );
}

Now you can create a new directory wikiconfigs within your MediaWiki installation directory which can contain several files named after each wikis database name + .php. In these files you can apply settings just like in Localsettings.php with the exception that they will only apply for a particular farm member.

Using $egSimpleFarmMembers [edit]

It is possible to add further, custom keys to each farm member array within $egSimpleFarmMembers. Within LocalSettings.php they can then be accessed via SimpleFarmMember::getCfgOption(). It is recommended to put a prefix in front of these additional keys (e.g. cfg_) or to simply use the name of the global configuration variable they will end up in. This will prevent complications with future option keys added by Simple Farm extension.

Wherever $egSimpleFarmMembers is defined:

$egSimpleFarmMembers = array(
        array(
                'name' => 'FruitMix-Wiki',
                'db' => 'fruitfarm',
                'addresses' => array( 'fruitmix.fruitfarm.org', 'allfruits.fruitfarm.org' ),
                'maintain' => false,
                'wgEnableEmail' => true,
                'wgMaxUploadSize' => 1024*1024*10, // 10MB
                'wgEmergencyContact' => 'fruitmix@fruitfarm.org',
        ),
        /* further farm members ... */
);

In LocalSettings.php (after the SimpleFarm::init(); line.) these additional keys can be accessed:

$cFM = SimpleFarm::getActiveMember();
 
# get active farm member options and set them:
$wgEnableEmail = $cFM->getCfgOption( 'wgEnableEmail' );
$wgMaxUploadSize = $cFM->getCfgOption( 'wgMaxUploadSize' );
$wgEmergencyContact = $cFM->getCfgOption( 'wgEmergencyContact' );
 
# Initialize 'Semantic MediaWiki'
enableSemantics( $cFM->getFirstAddress() );
 
unset( $cFM );

In case one of the accessed keys is not defined for one farm, false will be returned! Instead of using $cFM->getCfgOption( 'option' ) a prettier $cFM( 'option' ) could be used if PHP 5.3 is running on the server.

Setting-up Rewrite Rules [edit]

The webservers rewrite rules have to direct any web request to a particular wiki farm member to the physical directory of the Simple Farm wiki installation. Rewrite rules and sytanx depends on the webservers software. The following examples are for the widely spread Apache HTTP Server. The .htaccess file for these code examples has to be in the same directory the wiki directory is stored in, not within the wiki directory.

For addresses based farm members [edit]

This rewrite example works if you use the addresses based $egSimpleFarmMembers configuration. So this would be the way to direct urls like: farm1.mw.org, farm2.mw.org... , farmX.mw.org to the MediaWiki directory.

Important: This assumes that $wgScriptPath is set to /w and $wgArticlePath is set to /wiki/$1

# (c) 2011-2012 by Stephan Jauernick (http://stejau.org)
# Distributed under ISC license
RewriteEngine on
 
# crazy magic:
# this piece checks if we have mess with wiki stuff
RewriteCond %{REQUEST_FILENAME} wiki$
# next rule sets a env var to a value which can be used to check if files are in shared code base
RewriteRule ^/?wiki/(.*)$ - [E=REQUEST_FILENAME_W:%{DOCUMENT_ROOT}/w/$1]
# check if file is existent in shared code base
RewriteCond %{ENV:REQUEST_FILENAME_W} -d [OR]
RewriteCond %{ENV:REQUEST_FILENAME_W} -f
# if so: rewrite
RewriteRule ^/?wiki/(.*)$ /w/$1 [PT,L,QSA]
 
# standard rewrite magic:
RewriteRule ^/?wiki/[Ii]ndex\.php/(.*)$ /w/index.php?title=$1 [PT,L,QSA,B]
RewriteRule ^/?wiki/(.*)$ /w/index.php?title=$1 [PT,L,QSA,B]
RewriteRule ^/?wiki$ /w/index.php [PT,L,QSA,B]
RewriteRule ^/?images/(.*)$ /w/images/$1 [PT,L,QSA,B]
RewriteRule ^wiki/images/(.*)$ /w/images/$1 [PT,L,QSA,B]
RewriteCond %{REQUEST_URI} ^/$
RewriteRule ^/?$ /w/index.php

Many thanks to Stephan Jauernick for this set of rules.

For scriptpath based farm members [edit]

The following rewrite example works if your farm member setup in $egSimpleFarmMembers uses the scriptpath key to define where the wiki lays virtually. So this can be used to direct urls like: mw.org/farm1, mw.org/farm2... , mw.org/farmX to one directory.

Important: The name of the physical wiki directory can't be used as a farm members virtual path name. If you want a path name wiki for one of the farm members, you have to rename the physical MediaWiki drectory into something else, e.g. mw and adjust the path within the rewrite rules as well.

RewriteEngine On
RewriteBase /
 
# Redirect virtual wiki path to physical wiki path. 'Simple Farm' can choose the farm member by the original virtual path then
# The physical path in this case is 'wiki', there can be no wiki accessible using this path.
RewriteRule ^(?!wiki(?:/|$))[^/]+(?:/(.*))?$ wiki/$1
 
# No '/' allowed within the wiki path when using the rule above. If you want paths using '/', use something like:
# RewriteRule ^(?:fruits|fruits/apple|fruits/strawberry)(?:/(.*))?$  wiki/$1

Configuration Variables [edit]

Besides the required setup, there are some optional configuration variables controlling some of Subpage Funs behavior:

$egSimpleFarmMembers: Please see the #Setting-up Farm Members section for detail information.

$egSimpleFarmMainMemberDB: Database name of one of the $wgSimpleFarmMembers wikis. If null (default), it will be set when SimpleFarm::init() was called. The default value is the first key of $wgSimpleFarmMembers then. This main member is important for maintenance since the generic maintenance script will connect to the main member first to have full basic MediaWiki maintenance support.

$egSimpleFarmIgnoredDomainPrefixes: In case some farm members work sub-domain wise, this array can contain sub-domain prefixes which will be ignored when choosing to which farm member the given sub-domain name belongs to. By default, this is set to array( 'www' ), which means in something like http://www.member1.myfarm.com the www.member1 part would be considered the string to choose the farm member from but since www is set to be ignored, only member1 will be left. Of course, the same can be achieved by using rewrite rules.

$egSimpleFarmWikiSelectEnvVarName: Name of the environment variable used to select a wiki via command-line access if only one wiki should be selected directly instead of using the provided maintenace script. By default the name of this environment variable is WIKI.

$egSimpleFarmErrorNoMemberFoundCallback: Allows to define a PHP callback function which should be called in case no wiki has been found. This can happen in case somebody is using an url which would require a non-existing farm member.
Note: It would be nice if we could use the hook system of MediaWiki instead, but at this stage, the hook system has not even initialized. There might be a more complex system for this in the future.

Affected Configuration Variables [edit]

There are several global configuration variables which will be affected by Simple Farm. Some might just be conditionally be affected, others in any case. Be aware, that setting any of these variables before SimpleFarm::init(); might overwrite their value!

The following variables are affected:

$wgSitename        = SimpleFarmMember::getName(); // depends on farm member configuration key 'name'
$wgDBname          = SimpleFarmMember::getDB(); // depends on farm member configuration key 'db'
$wgScriptPath      = SimpleFarmMember::getScriptPath(); // in case of address-based members this is the initial $wgScriptPath value, otherwise configuration key 'scriptpath'
$wgUploadDirectory = "{$IP}/images/images_{$wgDBname}";
$wgUploadPath      = "{$wgScriptPath}/images/images_{$wgDBname}";
$wgLogo            = "{$wgScriptPath}/images/logos/{$wgDBname}.png";
$wgFavicon         = "{$wgScriptPath}/images/logos/{$wgDBname}.ico";

Farm Maintenance [edit]

Mass Maintenance [edit]

To make maintenance tasks like updating all database tables easy throughout the farm, Simple Farm introduces a new maintenance script to select all or certain farm members and run another (MW core) maintenance script on all of these farm members.

By default the location of that Script within the MediaWiki installation is extensions/SimpleFarm/maintenance/maintainFarm.php

Besides MW default maintenance script parameters (such as --help), the script allows the following parameters:

--farmexclude: Comma-separated list of farm members (by database names) to deselect for farming.
--farmonly: Comma-separated list of farm members (by database names) for doing some farming on.
--farmpreview: A preview mode. If set, this will output all selected wikis for farming without running any actual farming script on them.

It requires one argument, if not in preview mode, argument 1 which should be set to the shell command that should run on all selected farm members.

Examples:

php maintainFarm.php --farmexclude="fruitfarm_apples, fruitfarm_peaches" --farmpreview
php maintainFarm.php "php ../../../maintenance/update.php --quick" --farmonly=fruitfarm

Single Farm Maintenance [edit]

For command-line access accessing just one single farm, it is easiest to use the command-line while setting the environment variable for selecting a farm member. For example:

WIKI=fruitfarm_peaches php update.php

If the environment variable is not set, the main farm member (set in $egSimpleFarmMainMemberDB) will be chosen for the maintenance action.

Extension:Simple Farm

Contents

How it Works [edit]

Setup and Installation [edit]

Setting-up Farm Members [edit]

Example for address-based setup [edit]

Example for scriptpath-based setup [edit]

Settings per Member [edit]

Using a Configuration File per Member [edit]

Using $egSimpleFarmMembers [edit]

Setting-up Rewrite Rules [edit]

For addresses based farm members [edit]

For scriptpath based farm members [edit]

Configuration Variables [edit]

Affected Configuration Variables [edit]

Farm Maintenance [edit]

Mass Maintenance [edit]

Single Farm Maintenance [edit]

See Also [edit]

Navigation menu

Personal tools

Namespaces

Variants

Views

Actions

Search

Navigation

Support

MediaWiki.org

Development

Print/export

Toolbox