User:DWalden (WMF)/Wiki family

From mediawiki.org

Setup[edit]

The below steps should create:

Some problems:

  • Logins often won't work. I don't know why. Just keep retrying until they do. This problem seems to have sorted itself out.
  • I don't know if the job runner will work properly. This works as long as you run the appropriate commands in #Recommended way of starting the wiki family at the bottom of the page.
  • I think it might be accessing the database as a root user.

Clone all the necessary repos: 

git clone "https://gerrit.wikimedia.org/r/mediawiki/core" family
cd family
git clone "https://gerrit.wikimedia.org/r/mediawiki/skins/Vector" skins/Vector
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/CentralAuth" extensions/CentralAuth
git clone "https://gerrit.wikimedia.org/r/mediawiki/extensions/AntiSpoof" extensions/AntiSpoof

Create a docker-compose.override.yml with this in it: 

version: '3.7'
services:
  mariadb-main:
    image: 'bitnami/mariadb:latest'
    volumes:
      - mariadbdata:/bitnami/mariadb
    environment:
      - MARIADB_REPLICATION_MODE=master
      - MARIADB_REPLICATION_USER=repl_user
      - MARIADB_REPLICATION_PASSWORD=repl_password
      - MARIADB_ROOT_PASSWORD=main_root_password
      - MARIADB_USER=my_user
      - MARIADB_PASSWORD=my_password
      - MARIADB_DATABASE=enwiki
  mariadb-replica:
    image: 'bitnami/mariadb:latest'
    depends_on:
      - mariadb-main
    volumes:
      - mariadbdatarep:/bitnami/mariadb
    environment:
      - MARIADB_REPLICATION_MODE=slave
      - MARIADB_REPLICATION_USER=repl_user
      - MARIADB_REPLICATION_PASSWORD=repl_password
      - MARIADB_MASTER_HOST=mariadb-main
      - MARIADB_MASTER_PORT_NUMBER=3306
      - MARIADB_MASTER_ROOT_PASSWORD=main_root_password
volumes:
  mariadbdata:
    driver: local
  mariadbdatarep:
    driver: local

Create .env file: 

MW_SCRIPT_PATH=/w
MW_SERVER=http://localhost:8080
MW_DOCKER_PORT=8080
MEDIAWIKI_USER=Admin
MEDIAWIKI_PASSWORD=dockerpass
XDEBUG_CONFIG=
XDEBUG_ENABLE=true
XHPROF_ENABLE=true
MW_DOCKER_UID=1000
MW_DOCKER_GID=1000

In the mediawiki core repo, run these commands to start the docker container and install PHP libraries: 

docker compose up -d
mv composer.local.json-sample composer.local.json
docker compose exec mediawiki composer update

Run this command to access the database: 

docker compose exec mariadb-main mariadb -u root -pmain_root_password

Run these SQL commands to create the databases: 

CREATE DATABASE loginwiki;
GRANT ALL ON loginwiki.* TO 'my_user'@'%' IDENTIFIED BY 'my_password';
CREATE DATABASE centralauth;
GRANT ALL ON centralauth.* TO 'my_user'@'%' IDENTIFIED BY 'my_password';

Exit the database and run these commands: 

source .env
docker compose exec mediawiki php maintenance/install.php --dbname=enwiki --dbuser=my_user --dbpass=my_password --dbserver=mariadb-main --server="${MW_SERVER}" --scriptpath="${MW_SCRIPT_PATH}" --lang en --pass ${MEDIAWIKI_PASSWORD} mediawiki ${MEDIAWIKI_USER}
mv LocalSettings.php LocalSettings_enwiki.php
docker compose exec mediawiki php maintenance/install.php --dbname=loginwiki --dbuser=my_user --dbpass=my_password --dbserver=mariadb-main --server="${MW_SERVER}" --scriptpath="${MW_SCRIPT_PATH}" --lang en --pass ${MEDIAWIKI_PASSWORD} mediawiki ${MEDIAWIKI_USER}

Run this command: 

docker compose exec mariadb-main mariadb -u root -pmain_root_password -e "CREATE TABLE centralauth.objectcache LIKE enwiki.objectcache;"

Add this to the end of LocalSettings.php: 

wfLoadSkin( 'Vector' );

wfLoadExtension( 'CentralAuth' );
wfLoadExtension( 'AntiSpoof' );

$wgSharedDB = 'centralauth';
$wgSharedTables = array( 'objectcache' );
$wgCentralAuthSessionCacheType = CACHE_DB;

if ( defined( 'MW_DB' ) ) {
    $wgDBname = MW_DB;
} elseif ( isset($_SERVER['SERVER_NAME']) && count(explode(".", $_SERVER['SERVER_NAME'])) > 1 ) {
    $wgDBname = explode(".", $_SERVER['SERVER_NAME'])[0];
} else {
    $wgDBname = 'loginwiki';
}

$wgConf->settings = [
    'wgServer' => [
        # If you want to allow also usage of https, just use '//localhost'
        #  and set 'http://localhost' at 'wgCanonicalServer'
        'default' => 'http://localhost:8080',
    ],

    'wgCanonicalServer' => [
        'default' => 'http://localhost:8080',
    ],

    'wgScriptPath' => [
        'default' => '/w',
    ],

    'wgArticlePath' => [
        'default' => '/wiki/index.php/$1',
    ],

    'wgSitename' => [
        'default' => 'Wikipedia',
    ],

    'wgLanguageCode' => [
        'default' => 'en',
    ],
];

$wgLocalDatabases = [
    'enwiki',
    'loginwiki',
];

foreach ( $wgLocalDatabases as $db ) {
    $wgConf->settings['wgServer'][$db] = 'http://' . $db . '.localhost:8080';
    $wgConf->settings['wgCanonicalServer'][$db] = 'http://' . $db . '.localhost:8080';
    $wgConf->settings['wgSitename'][$db] = $db;
}

$wgConf->wikis = $wgLocalDatabases;
$wgConf->suffixes = [ 'wiki' ];
$wgConf->localVHosts = [ 'localhost' ];

function efGetSiteParams( $conf, $wiki ) {
    $site = null;
    $lang = null;
    foreach( $conf->suffixes as $suffix ) {
        if ( substr( $wiki, -strlen( $suffix ) ) === $suffix ) {
            $site = $suffix;
            $lang = substr( $wiki, 0, -strlen( $suffix ) );
            break;
        }
    }
    return [
        'suffix' => $site,
        'lang' => $lang,
        'params' => [
            'lang' => $lang,
            'site' => $site,
            'wiki' => $wiki,
        ],
        'tags' => [],
    ];
}

$wgConf->suffixes = $wgLocalDatabases;
$wgConf->siteParamsCallback = 'efGetSiteParams';
$wgConf->extractAllGlobals( $wgDBname );
$dockerMainDb = [
	'host' => "mariadb-main",
	'dbname' => $wgDBname,
	'user' => 'root',
	'password' => 'main_root_password',
	'type' => "mysql",
	'flags' => DBO_DEFAULT,
	'load' => 0,
];
$dockerReplicaDb = [
	'host' => "mariadb-replica",
	'dbname' => $wgDBname,
	'user' => 'root',
	'password' => 'main_root_password',
	'type' => "mysql",
	'flags' => DBO_DEFAULT,
	'max lag' => 60,
	'load' => 1,
];
// Integration tests fail when run with replication, due to not having the temporary tables.
if ( !defined( 'MW_PHPUNIT_TEST' ) ) {
	$wgDBservers = [ $dockerMainDb, $dockerReplicaDb ];
} else {
	$wgDBserver = $dockerMainDb['host'];
	$wgDBuser = $dockerMainDb['user'];
	$wgDBpassword = $dockerMainDb['password'];
	$wgDBtype = $dockerMainDb['type'];
}

$wgCentralAuthDatabase = 'centralauth';
$wgCentralAuthCookies = true;
$wgCentralAuthAutoMigrate = true;
#$wgCentralAuthCookieDomain = '.example.org';
$wgCentralAuthLoginWiki = 'loginwiki';

Run these commands: 

docker compose exec mediawiki php maintenance/sql.php --wikidb centralauth extensions/CentralAuth/schema/mysql/tables-generated.sql
docker compose exec mediawiki php maintenance/sql.php --wikidb enwiki extensions/AntiSpoof/sql/mysql/tables-generated.sql
docker compose exec mediawiki php maintenance/sql.php --wikidb loginwiki extensions/AntiSpoof/sql/mysql/tables-generated.sql
docker compose exec mediawiki php maintenance/sql.php --wikidb centralauth extensions/AntiSpoof/sql/mysql/tables-generated.sql
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass0.php --wiki loginwiki
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass0.php --wiki enwiki
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass1.php --wiki loginwiki
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass1.php --wiki enwiki

It should now all be setup. Go to http://enwiki.localhost:8080/wiki/Main_Page.

Adding a new wiki[edit]

Create a new database for the wiki. Run this command to access the database: 

docker compose exec mariadb-main mariadb -u root -p

Run these SQL commands to create the databases: 

CREATE DATABASE <wiki database name>;
GRANT ALL ON <wiki database name>.* TO 'my_user'@'%' IDENTIFIED BY 'my_password';

mv LocalSettings.php LocalSettings_old.php
source .env
docker compose exec mediawiki php maintenance/install.php --dbname=<wiki database name> --dbuser=my_user --dbpass=my_password --dbserver=mariadb-main --server="${MW_SERVER}" --scriptpath="${MW_SCRIPT_PATH}" --lang en --pass ${MEDIAWIKI_PASSWORD} mediawiki ${MEDIAWIKI_USER}
mv LocalSettings_old.php LocalSettings.php

Add the new wiki's database name to $wgLocalDatabases in LocalSettings.php. 

docker compose exec mediawiki php maintenance/sql.php --wikidb <wiki database name> extensions/AntiSpoof/sql/mysql/tables-generated.sql
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass0.php --wiki <wiki database name>
docker compose exec mediawiki php extensions/CentralAuth/maintenance/migratePass1.php --wiki <wiki database name>
docker compose exec mediawiki php maintenance/run.php --wiki <wiki database name> update.php --quick

(Optional) Add the wiki database name to the script in #Recommended_way_of_starting_the_wiki_family.

Access the new wiki via http://<wiki database name>.localhost:8080.

Having a different configuration per wiki[edit]

If you want to add specific configuration options that are only for certain wikis, add something like this to you LocalSettings.php: 

if ( $wgDBname == 'loginwiki' ) {
// Config only for loginwiki
} elseif ( $wgDBname == 'enwiki' ) {
// Config only for enwiki
} elseif ( $wgDBname == '<wiki database name>' ) {
// Config only for other wiki
}

You could also potentially do some of this with Manual:$wgConf.

Recommended way of starting the wiki family[edit]

Recommended way of updating all the code, installing php dependencies, updating databases and starting this family of wikis. You can save this to a file locally and run like sh <filename> or you can copy and paste this into your terminal. Should be ran from the mediawiki core repository: 

#!/bin/bash

for dir in ./ skins/*/ extensions/*/
do
    cd $dir
    git checkout master
    git pull
    cd -
done

docker compose up -d
docker compose exec mediawiki composer update

# When you create a new wiki, add its database name to the end of this list
for wiki in loginwiki enwiki
do
    docker compose exec mediawiki php maintenance/run.php --wiki $wiki update.php --quick
    docker compose exec -d mediawiki-jobrunner php maintenance/run.php --wiki $wiki runJobs.php --wait
done
Retrieved from "https://www.mediawiki.org/w/index.php?title=User:DWalden_(WMF)/Wiki_family&oldid=6409410"