User:OrenBochman/Installation

I have been trying to do some development on media wiki search. It turns out that setting up an environment is not so simple. I decided to document the process in case I need to do it again and for other peoples benefit.

=MediaWiki Development Environment Setup On Windows=

XAMPP Application Stack Installation
Installed latest XAMPP. That means:
 * XAMPP 1.7.4
 * Apache 2.2.17
 * MySQL 5.5.8
 * PHP 5.3.5
 * phpMyAdmin 3.3.9
 * FileZilla FTP Server 0.9.37
 * Tomcat 7.0.3 (with mod_proxy_ajp as connector)

To enable eaccelerator edit php\php.ini and uncomment ";zend_extension = "\xampp\php\ext\php_eaccelerator.dll"

However the binary is not available in the XAMPP distribution and needs to be downloaded separately. This is no easy task. You need to check what type of php installation you have then. The answers is available from the PHP info page in the XAMPP main page.
 * what is the PHP version?
 * Is your version built as ThreadSafe?
 * which version of VisualStuio it was built with?

Problem: Apache won't start

 * 1) Skype blocked port 80 (resolved)
 * 2) had to change User Account Control (UAC) via the control panel security settings which blocked control panel from starting Apache (resolved).
 * the database base (local link)

Media Wiki installation

 * 1) decompressed MediaWiki software archive version 1.17 to D:\xampp\htdocs\mediawiki
 * 2) browsed to http://localhost/mediawiki/ and followed instructions
 * 3) however it is necessary to use binary representation and not UTF=8 in db otherwise mwdumper will fail

Main & Status Pages

 * Main page(local)
 * Version Page (local)

Changing Capitalization Settings

 * 1) editing LocalSettings.php and adding at the end and then $wgCapitalLinks=false;
 * 2) running cleanUpCaps.php from the command line (this took about 12 hours for 3 million entries)

Extensions
require( "extensions/ParserFunctions/ParserFunctions.php"); $wgUseAjax = true; require_once("{$IP}/extensions/CategoryTree/CategoryTree.php"); require_once("$IP/extensions/CharInsert/CharInsert.php"); require_once("$IP/extensions/ImageMap/ImageMap.php"); require_once ("$IP/extensions/LabeledSectionTransclusion/lsth.php"); require_once ("$IP/extensions/SyntaxHighlight_GeSHi/SyntaxHighlight_GeSHi.php");
 * 1) Since My main purpose is Development related (bots and indexing) I wanted to introduce sufficent extetions to allow decent dumping
 * 2) consult ...
 * 3) download
 * 4) decompress to
 * 5) edit LocalSettings.php adding require( "extensions/ParserFunctions/ParserFunctions.php"); to the end
 * 1) char insert
 * 1) image map
 * 1) require_once Labeled Section Transclusion
 * 1) syntax highlighting
 * 1) oren code end

DIF
get the diff utility and edit LocalSettings.php adding $wgDiff = 'C:/Server/xampp/htdocs/MW/bin/GnuWin32/bin/diff.exe'; $wgDiff3 = 'C:/Server/xampp/htdocs/MW/bin/GnuWin32/bin/diff3.exe'; the alternative is to do nothing. Dif would be available but slower.
 * 1) Path to the GNU diff3 utility. Used for conflict resolution.

Extensions and Dumping
It turns out that dumping a MediaWiki via the dumphtml command-line extension is not compatible with some of the other extensions. It is prudent to turn such extensions off for making static dumps. The worst culprit is the syntax highlighting extension, which is not important in the main Wiktionary pages, but useful when developing scripts in user namespace.

Importing Contents

 * 1) importing via ... will take forever and has problems with the term Wiktionary in the inter-wiki namespace table
 * 2) importing via mwdumper with some db modification took about 12 hours.
 * 3) unfortunately it crashed every time requiring one to drop the db and reinstall.
 * 4) to speed things up I removed the indexes from the Text Page and Revisions table and later re introduced them

phpmyadmin can be used to import sql dumps - but it cannot import realty big ones for this another application is required. []

Fixing Capitalization
add to LocalSettings.php:

$wgCapitalLinks=false;

next run maintenece/cleanupCaps.php (12 hours)

DB time outs
Once imported there appeared to be a crash. This was due to database time outs. Some were caused by simultaneous db imports of required sql tables. However even when these were done the time outs persisted.

The solution involved five days of diagnostic ad various attempts at patching things up. The actual solution came from: resolution: installed zend e-accelerator from [] and time outs have stopped.
 * Turning on traces and diagnostics.
 * Trying to dump pages via the dumpHtml extention.
 * Enabling Zend-Eaccelerator seems to have reduced the problem. Once done time outs no longer occurred on most pages.
 * Increasing the database time out from 30 seconds to 60.

Importing SQL Dumps
Running the rebuild script should restore the DB to fully functional status. However this script runs three tasks each taking an order of magnitude longer than its predecessor. The recreation of links seems to be impractical in a large project. Also there is no indication of progress.

I wanted to have a fully functioning version of Wiktionary at this point (perhaps without the pictures) to allow a static dump which could be used to make an offline version.

Diagnostics

 * 1) run rebuild:
 * 2) Then the wiki decided to crash. It gives the error: "Fatal error: Maximum execution time of 30 seconds exceeded in D:\xampp\htdocs\mediawiki\includes\db\DatabaseMysql.php on line 23"
 * 3) It turns out that the problem is not a crash but a slow response on many pages
 * 4) random link works
 * 5) pages generated by random link work too
 * 6) Switching to a second empty db with different table suffix works fine too

More Problems
Pages were often littered with malformed extraneous tags (which if properly written would never be shown by a browser Resolution: install Tidy.

IRC

 * I initialy used a web access. After some time I decided to use IRC client software.
 * I chose chatzilla. It is a fire fox extension.
 * http://chatzilla.hacksrus.com/faq/
 * freenode network has some channels
 * mediawiki for media wiki discussions
 * wikimedia-tech for server/cluster related discussions (good place to report down time
 * wiktionary for wiktionary issues
 * Openzim for the static dump format
 * Kiwix for the static dump format

MySQL & Optimizing

 * http://dom.as/2007/01/26/mediawiki-performance-tuning/ from domas' blog
 * http://www.mediawiki.org/wiki/Manual:Performance_tuning RTFM
 * User:Robchurch/Performance_tuning has more resources
 * Manual:MySQL currently a stub