Manual:Backing up a wiki

It is important to make regular backups of the data in your wiki. This page provides an overview of the backup process for a typical MediaWiki wiki; you will probably want to devise your own backup scripts or schedule to suit the size of your wiki and your individual needs.

Overview
MediaWiki stores important data in two places:
 * Database : Pages and their contents, users and their preferences, metadata, search index, etc.
 * File System : Software configuration files, custom skins, extensions, images (inc. deleted images) etc.

Consider making the Wiki read-only before creating the backup - see Manual:$wgReadOnly. This makes sure all parts of your backup are consistent (some of your installed extensions may write data nonetheless).

File transfer
Unless you have direct access to the server hosting the wiki, (and even then) you will have to choose a method for transferring files:
 * Secure copy with SCP or WinSCP
 * SSH File Transfer Protocol (SFTP)
 * Using a FTP client.
 * The hosting company might provide a file manager interface via the web browser, check with your provider.
 * Some other method, there is a list of these at List of file transfer protocols

Database
Most of the critical data in the wiki is stored in the database, which is typically straightforward to back up. When using the default MySQL backend, the database can be dumped into a script file which can be used later to recreate the database and all the data in it from scratch.

Mysqldump from the command line
The most convenient way to create a dump file of the database you want to back up, is to use the standard MySQL dump tool mysqldump from the command line. Be sure to get the parameters right or you may have difficulty restoring the database.

First insert the following line into LocalSettings.php $wgReadOnly = 'Dumping Database, Access will be restored shortly'; this can be removed as soon as the dump is completed.

Example of the command to run on the Linux/UNIX shell: mysqldump -h hostname -u userid --password --default-character-set=whatever dbname > backup.sql Substituting hostname, userid, password, dbname etc. as appropriate. If no character set is specified, mysqldump uses utf8, and earlier versions used latin1. Your wiki's database might be using binary. Check in the LocalSettings.php file to find out which (usually under $wgDBTableOptions DEFAULT CHARSET). Otherwise mysql might dump according to the server's file system and not the wiki's. The dbname can also be found in LocalSettings.php under $wgDBname.

Other parameters might be useful such as --quote-names       : Quote identifiers within backtick characters --hex-blob         : Dump binary columns using hexadecimal notation

See mysqldump for a full list of command line paremeters.

The output from mysqldump can instead be piped to gzip, for a smaller output file, as follows mysqldump -h hostname -u userid --password dbname | gzip > backup.sql.gz

A similar mysqldump command can be used to produce xml output instead, by including the --xml parameter. mysqldump -h hostname -u userid --password --xml dbname > backup.xml and to compress the file with a pipe to gzip mysqldump -h hostname -u userid --password --xml dbname | gzip > backup.xml.gz

Remember to also backup the file system components of the wiki that might be required, eg. images, logo, and extensions.

Running mysqldump with Cron
Cron is the time-based job scheduler in Unix-like computer operating systems. Cron enables users to schedule jobs (commands or shell scripts) to run periodically at certain times or dates.

A sample command that you may run from a crontab may look like this:  nice -n 19 mysqldump -u $USER --password=$PASSWORD $DATABASE -c | nice -n 19 gzip -9 > ~/backup/wiki-$DATABASE-$(date '+%Y%m%d').sql.gz

The  lowers the priority of the process.

Use valid values for $USER, $PASSWORD, $DATABASE. This will write a backup file with the weekday in the filename so you would have a rolling set of backups. If you want to save the files and extensions as well, you might want to use this one.

If you want add this task in Cron through Cpanel then you must escape the character "%"

/usr/bin/mysqldump -u $USER --password=$PASSWORD $DATABASE -c | /bin/gzip > ~/backup/wiki-$DATABASE-$(date '+\%Y\%m\%d').sql.gz

Else you get error:

/bin/sh: -c: line 0: unexpected EOF while looking for matching `'' /bin/sh: -c: line 1: syntax error: unexpected end of file

Tables
Under close examination one finds that some of the tables dumped have various degrees of temporariness. So to save disk space (beyond just gziping), although those tables need to be present in a proper dump, their data does not. However, under certain circumstances the disadvantage of having to rebuild all this data may outweigh the saving in disk space (for example, on a large wiki where restoration speed is paramount).

See a mailing list thread about the topic.

Character set
You can see which character set your tables are using with a mysql statement like  (including the semicolon). The last line will include a DEFAULT CHARSET clause.

If the last line does not include a DEFAULT CHARSET clause then there is another way if you know that nobody has changed the character set of the database server since it was installed and the wiki's database was created using the default character set of the database. The STATUS command displays the database server's default character set next to Server characterset:. Here is an example output:

mysql> status - - - - - - - - - mysql Ver 12.22 Distrib 4.0.20a, for Win95/Win98 (i32) Connection id:         13601 Current database: Current user:          root@localhost SSL:                   Not in use Server version:        4.0.20a-nt Protocol version:      10 Connection:            localhost via TCP/IP Client characterset:   latin1 Server characterset:   latin1 TCP port:              3306 Uptime:                27 days 4 hours 58 min 26 sec

Use the option --default-character-set=latin1</tt> on the mysqldump</tt> command line to avoid the conversion if you find it set to "latin1</tt>".

Like this: <pre style="white-space:pre-wrap;white-space:-moz-pre-wrap;white-space:-pre-wrap;white-space:-o-pre-wrap;word-wrap:break-word;overflow:auto;"> nice -n 19 mysqldump -u $USER -p$PASSWORD --default-character-set=$CHARSET $DATABASE -c | nice -n 19 gzip -9 > ~/backup/wiki-sql-$(date '+%a').sql.gz

Also one can try --default-character-set=binary. “Convert latin1 to UTF-8 in MySQL” on Gentoo Linux Wiki has more information.

Latin-1 to UTF-8 conversion
When you want to upgrade from a rather old Mediawiki installation with Latin-1 to UTF-8 which might be tricky depending on your operating system and MySQL settings - in my example from Mediawiki 1.5 (2004) to 1.15.1 (2009) - perform the following steps as found in the article Convert a MySQL DB from latin1 to UTF8 and further adapted to Mediawiki specialities (DBNAME is the name of your wiki database): mysqldump -u root -p --opt --default-character-set=latin1 --skip-set-charset DBNAME > DBNAME.sql

Then use sed to change character settings latin1 to utf8: <pre style="white-space:pre-wrap;white-space:-moz-pre-wrap;white-space:-pre-wrap;white-space:-o-pre-wrap;word-wrap:break-word;overflow:auto;"> sed -e 's/character set latin1 collate latin1_bin/character set utf8 collate utf8_bin/g' -e 's/CHARSET=latin1/CHARSET=utf8/g' DBNAME.sql > DBNAME2.sql

Every character in UTF-8 needs up to 3 bytes, thus it is necessary to decrease one key which is done with the following command Relevant sources: A further problem which prevents reimporting the database was the math table (ERROR line 389: Duplicate entry  for key 1'' when trying to import the mysqldump). I solved it by simply deleting the math table content, as this is only a cache and need not to be imported when upgrading.
 * Bug 1322 "Specified key was too long" (MySQL error) during installation"
 * MediaWiki maintenance/tables.sql
 * I found different MediaWiki versions already using different cl_sortkey length and took this into account below
 * "Specified key was too long; max key length is 1000 bytes": "Truncate so that the cl_sortkey key fits in 1000 bytes"

sed -e 's/`cl_sortkey` varchar(255)/`cl_sortkey` varchar(70)/gi' DBNAME2.sql > DBNAME21.sql sed -e 's/`cl_sortkey` varchar(86)/`cl_sortkey` varchar(70)/gi' DBNAME21.sql > DBNAME22.sql sed -e 's/`cl_sortkey`(128)/`cl_sortkey`(70)/gi' DBNAME22.sql > DBNAME23.sql sed -e '/^INSERT INTO `math/d' DBNAME23.sql > DBNAME3.sql

From here I then created a new database DBNEW and then imported the dumpfile mysql -u root -p -e "create database DBNEW" mysql -u root -p --default-character-set=utf8 DBNEW < DBNAME3.sql

Now start a fresh MediaWiki installation and use your new wiki database name DBNEW - actually the UTF-8 converted copy of your untouched old DBNAME wiki - and the database copy will be automatically upgraded to the recent MediaWiki database scheme. Several successful conversions from MediaWiki 1.5 to MediaWiki 1.15.1 under PHP 5.2.12 (apache2handler) and MySQL 4.1.13 have been made.

Latin-1 to UTF-8 conversion under Windows
Tested under WindowsXP. Mediawiki 1.13.2 dumped under EasyPHP 1.8.0.1. Converted with Chargo 0.8.1. Imported to XAMPP 1.7.3. Updated to Mediawiki 1.15.1.
 * 1) Dump your Database as usual.
 * 2) Convert your Database using the character set conversion utility Charco
 * 3) Replace all latin1 thru utf8 inside the dump.
 * 4) Import the dump into a new DB or overwrite the old.
 * 5) Ready

Latin-1 to UTF-8 conversion under Mac

 * 1) First export your Database as usual, separated into schema and data. You can use the terminal command mysqldump</tt>, which the official installer places in /usr/local/mysql/</tt>. Note that in the following lines, the lack of spaces between -u and username and -p and password is deliberate:
 * 2) *./mysqldump --default-character-set=latin1 --skip-set-charset -d -uuser -ppassword DBNAME > ~/db_schema.sql</tt>
 * 3) *./mysqldump --default-character-set=latin1 --skip-set-charset -t -uuser -ppassword DBNAME > ~/db_data.sql</tt>
 * 4) The database exports are now in your personal folder. Convert both exports with Charco from ISOlatin1 to UTF-8. Append "_utf8" to the output file names and fix the .txt</tt> extension that Charco enforces back to .sql</tt>.
 * 5) Open the file ~/db_schema_utf8.sql</tt> with Text Editor and replace each "DEFAULT CHARSET=latin1" phrase with "DEFAULT CHARSET=utf8"
 * 6) Make a new database using Sequel Pro, with encoding "UTF-8 Unicode (utf8)".
 * 7) * Import the ~/db_schema_utf8.sql</tt> file into your new database
 * 8) * Import the ~/db_data_utf8.sql</tt> file into your new database
 * 9) * ensure that your wiki user has access to the new database by adding a relevant line in your MYSQL database in the DB table
 * 10) Change the variable $wgDBname in your LocalSettings.php</tt> to reflect the name of the new database. Then test if everything works. If not, flip back to the old database and try a different method.
 * 11) (Optional) Delete the old database, and (also optional) rename the new database to the old database and revert the change in $wgDBname.

This sequence was adapted from Khelll's Blog, and used for Mediawiki 1.19.2 and MySQL 5.1.57. It will fix encodings that already show up as garbled under an updated wiki installation as well.

Repairing corrupted character sets
In case your database's character set got corrupted (see warning above), an easy way to fix the corrupted characters and remedy the situation for future backups has been posted in this source

Directly changing all latin1-encoded columns to UTF-8 won't help, as MySQL will just transform the erroneous characters directly. The remedy is to change the wrongly encoded latin1 string type (char/varchar/TEXT) into a binary type (binary/varbinary/BLOB). A conversion into a UTF8-encoded string type (char/varchar/TEXT) will then fix all your previously erroneous characters to their proper representation.

In short: latin1 char/varchar/TEXT -> binary/varbinary/BLOB -> UTF8 char/varchar/TEXT

Also don't forget to change the default charset for your database and the single tables to UTF-8, so your character sets won't get corrupted again.

PostgreSQL
You can use the pg_dump</tt> tool to back up a MediaWiki PostgreSQL database. For example: pg_dump mywiki > mywikidump.sql will dump the mywiki</tt> database to mywikidump.sql.

To restore the dump: psql mywiki -f mywikidump.sql

You may also want to dump the global information, e.g. the database users:

pg_dumpall --globals > postgres_globals.sql

SQLite
See Manual:SQLite

phpMyAdmin
Turn wiki to read only by adding  to LocalSettings.php.

Open the browser to your phpadmin link, login, choose the wiki database. (Check LocalSettings.php if you're not sure). Select Export. Make sure all items under Export are highlighted, and make sure Structure is highlighted (it's important to maintain the table structure). Optionally check Add DROP TABLE to delete existing references when importing. Make sure Data is checked. Select zipped. Then click on GO and save the backup file.

Remove  from LocalSettings.php

Remember to also backup the file system components of the wiki that might be required, eg. images, logo, and extensions.

File system
MediaWiki stores other components of the wiki in the file system where this is more appropriate than insertion into the database, for example, site configuration files (LocalSettings.php</tt>, AdminSettings.php</tt>), image files (including deleted images, thumbnails and rendered math and SVG images, if applicable), skin customisations, extension files, etc.

The best method to back these up is to place them into an archive file, such as a <tt>.tar</tt> file, which can then be compressed if desired. On Windows, applications such as WinZip or 7-zip can be used if preferred.

For Linux variants, assuming the wiki is stored in /srv/www/htdocs/wiki tar zcvhf wikidata.tgz /srv/www/htdocs/wiki

It should be possible to backup the entire "wiki" folder in "htdocs" if using XAMPP.

XML dump
It is also a good idea to create an XML dump in addition to the database dump. XML dumps contain the content of the wiki (wiki pages with all their revisions), without the site-related data (it does not contain user accounts, image metadata, logs, etc). XML dumps are independent of the database structure, and can be imported into future (and even past) versions of MediaWiki. They are also less likely to cause problems with character encoding, and can readily be processed by third party tools, which makes them a good fallback should your main database dump become unusable, and also as a means of redistributing content en masse.

To create an XML dump, use the command-line tool <tt>dumpBackup.php</tt>, located in the <tt>maintenance</tt> directory of your MediaWiki installation. Run the command as <tt>php dumpBackup.php</tt> without any arguments to display a brief description of the syntax. You need to specify whether you want a full dump of the complete history of every page, or just the current contents of each page. Prior to MediaWiki 1.16: If an attempt to use  fails with a message about insufficient  permissions, ensure that you have a properly configured   file. Instructions on creating the file are at Manual:AdminSettings.php.

You can also create an XML dump for a specific set of pages online, using the Special:Export, although attempting to dump large quantities of pages through this interface will usually time out.

To import an XML dump into a wiki, use the command-line tool <tt>importDump.php</tt>. For a small set of pages, you can also use the Special:Import page via your browser (per default, this is restricted to the sysop group). As an alternative to <tt>dumpBackup.php</tt> and <tt>importDump.php</tt>, you can use MWDumper, which is faster, but requires a Java runtime environment.

See Manual:Importing XML dumps for more information.

Without shell access to the server
If you have no shell access, then use the WikiTeam Python script dumpgenerator.py from a DOS, Unix or Linux command-line. To run the script see the WikiTeam tutorial.

See also Meta:Data dumps

Scripts

 * Unofficial backup script by User:Duesentrieb; creates a backup of all files, a database dump, and an xml dump.
 * Unofficial backup script by User:Duesentrieb, User:Kaotic added the ability for the script to place the wiki into read only mode during the database dump.
 * Unofficial backup script by User:Duesentrieb, User:Megam0rf added a few lines to circumvent Kaotic's problem with.
 * Unofficial backup script by User:Duesentrieb, User:Robkam set option for default character-set to binary, and other minor tweaks.
 * Unofficial backup script by Flominator; creates a backup of all files and the database, with optional backup rotation.
 * User:Darizotas/MediaWiki Backup Script for Windows - a script for backing up a Windows MediaWiki install. Note: Has no restore feature.
 * Unofficial web-based backup script, mw_tools, by Wanglong (allwiki.com); you can use it to back up your database, or use the backup files to recover the database, the operation is very easy.
 * WikiTeam tools - if you do not have server access (e.g. your wiki is in a free wikifarm), you can generate an XML dump and an image dump using WikiTeam tools (see some saved wikis)
 * Fullsitebackup