Extension:Zend Search Lucene for MediaWiki

MediaWiki extensions manual - list

Zend Search Lucene for MediaWiki Release status: stable
Implementation	Search, Special page, Hook
Description	Zend Search Lucene for MediaWiki. The OTHER Extended Search Solution for MediaWiki. Supports an unlimited number of Wiki-Instances / Lucene Indexes as well as password restricted Wikis.
Author(s)	PSL, Wiki-Dienstleister
Last version	2.1 (June 01, 2011)
MediaWiki	1.16-1.20 / PHP >=5.2.3
License	GPL Licence
Download	ZSL for MediaWiki Change Log
Check usage and version matrix

Description

Zend Search Lucene for MediaWiki adds better search capabilities to MediaWiki, includes a huge list of external file formats and aims to bring enhanced knowledge management features into your Wiki application. This engine is mainly designed for science and business collaborations. Unlike many other extended search solutions it depends on nothing else than PHP and an up-to-date configured web server. This means:

• No Java, Python, C++ etc.
• No additional compilations
• No trouble with unending dependency issues
• No daemons

The plugin is based on the native Lucene port of the Zend Framework. Therefore it provides the same enterprise level search features as the famous Java-based original Apache Lucene. Zend Search Lucene for MediaWiki can be tested and compared (thru utilizing a new special page for search results), before defining it as the complete replacement search engine on a wiki.

Notable Features

- word search
- phrase search
- wildcard search
- fuzzy search
- suggest with 4 modes
- category filter
- namespace filter
- user preferences
- handles an unlimited numbers of Wiki instances/indexes
- runs on *nix and windows based systems
- supports restricted wikis
- includes a huge list of searchable file formats
- enhanced knowledge management features
- automatic skin switches
- ready for customizing, extendable OOP
- commercial premium support if needed

Requirements

- PHP >=5.2.3, mbstring enabled (default)
- PHP Zend Framework 1.11
- Cron (recommended)

Compatibility

This extension has been successfully checked against the following MediaWiki versions. Others may work too.

• 1.20 - works - (Jaideraf)
• 1.17 - works - (Ucananduwill)
• 1.16 - works - (steviex2)

The extension has been shown to work with the following languages and supports UTF-8.

• English - works (help page currently german only) - all versions - (steviex2)
• German - works - all versions - (steviex2)

Installation Instructions

Step 1 - Install / Download Zend Framework

Download Zend Framework. Unpack and copy the contents of the download file to a webserver folder (commonly not below web root). Zend Framework install is done!

Step 2 - Configure Zend Search Lucene for MediaWiki

Download and extract the extensions PslZslAdmin and PslZendSearchLucene to your Wiki(s) extension directory. Move the files PslZendSearchLuceneIndexer.php and PslZendSearchLuceneIndexerConfig.php to a server directory above web root. Edit the marked parts of the file PslZendSearchLuceneIndexerConfig.php as described in it.

Step 3 - Run PslZendSearchLuceneIndexer.php

Run the indexer to prepare for searching (you can do this with extension PslZslAdmin too):
Unix-Example:

/usr/bin/php -c /etc/php5/cli/php.ini /ZSearch/PslZendSearchLuceneIndexer.php

Windows-Example:

c:\xampp\php\php.exe -f C:\xampp\htdocs\psl-suche\PslZendSearchLuceneIndexer.php

Make sure to replace the paths to match your installation!

Note: You can automate the PslZendSearchLuceneIndexer.php script call with a cron job- scheduled task.

Step 4 - Extension Installation/Configuration - Local Settings.php

Add the following directives to your LocalSettings.php

Note: this is only an example config. If you have any trouble visit the talk page

/* Configuration Zend Search Lucene for MediaWiki - Start */
$PslDomainDir                   = "sysdoc";
$PslPhpExecutionStringUnix      = "/usr/bin/php -c /etc/php5/cli/php.ini ";
$PslMaintenancePath             = "/var/www/vhosts/indi.knowledgeserver.de/subdomains/".$PslDomainDir."/httpdocs/maintenance/";
$PslXmlPath                     = "/var/www/vhosts/indi.knowledgeserver.de/subdomains/".$PslDomainDir."/".$PslDomainDir."_current.xml";
 
$wgPslZslAdminUseAutoReIndex    = false;
$wgPslZslAdminDefaultEmail      = "info@gibtesdochnicht.de";//email for debug
$wgPslZslAdminDumpString        = $PslPhpExecutionStringUnix.$PslMaintenancePath."dumpBackup.php --current --quiet --uploads > ".$PslXmlPath;
$wgPslZslAdminMediaDir          = "/var/www/vhosts/indi.knowledgeserver.de/subdomains/".$PslDomainDir."/public/";//could vary depending on img_auth.php
$wgPslZslAdminReIndexString     = $PslPhpExecutionStringUnix."/PSL_ADD_ONS/PslZendSearchLuceneIndexer.php ".$PslXmlPath." wikidb_".$PslDomainDir." ".$PslMaintenancePath."dumpBackup.php";
require_once( "$IP/extensions/PslZslAdmin/PslZslAdmin.php");
 
$wgSearchType                      = 'PslZendSearchLucene';
$wgPslEnableSuggestions            = true;//enables suggestions
$wgPslEnableStopWords              = false;//enables stop words
$wgPslStopWords                    = array('aber','als','am','an');
$wgPslImagePath                    = "https://".$_SERVER['HTTP_HOST'].$wgScriptPath."/extensions/PslZendSearchLucene/";
 
$wgPslWikiUrl                      = "https://".$PslDomainDir.".indi.knowledgeserver.de/index.php/";
$wgPslEntriesPerPage               = 20;
$wgPslUtf8DecodeResults            = false;//utf8-hint for related display issues, (play around with this if needed)
$wgPslIndexDir                     = "/PSL_ADD_ONS/psl_search_indexes/wikidb_".$PslDomainDir;//index directory
$wgPslZendLibraryDir               = "/PSL_ADD_ONS/ZF/library/";//path to zend framework library
$wgPslEnablePopularSearches        = true;//requires table-create rights for MediaWikis db-account
$wgPslPopularSearchesHistory       = 365;//data remains 365 days
$wgPslProtectPopularSearches       = false;//
$wgPslHighlightColor               = "#ff6900";
$wgPslEnabaleDebugMode             = false;//debug mode
$wgPslEnableSuggestions            = true;//enables suggestions
$wgPslEnableFileSearch             = true;//enables file search
$wgPslEnablePsIpTracking           = false;//enables ip tracking for geo lacation services etc. (currently not implemented)
$wgPslEnableAnonKey                = true;//anonymous key for science
$wgPslHistoryEntries               = 30;//history entries per page
$wgPslHistoryMiniStat              = true;
 
$wgHiddenPrefs[] = 'searchlimit';//this entries are disabling no more needed user preferences of the old/default search
$wgHiddenPrefs[] = 'contextlines';
$wgHiddenPrefs[] = 'contextchars';
$wgHiddenPrefs[] = 'disablesuggest';
$wgHiddenPrefs[] = 'searcheverything';
$wgHiddenPrefs[] = 'searchnamespaces';
$wgPslEnableUserInHistory          = false;//enhanced knowledge management feature could tackle your country specific law!
require_once( "$IP/extensions/PslZendSearchLucene/PslZendSearchLucene.php");
/* Configuration Zend Search Lucene for MediaWiki - End */

Note: Make sure to replace the paths to match your installations and server configs!

Mode Of Operation

By default, this extension will not run as overwrite for the built-in search engine, but instead provide a new Special Page called Extended Search: ZSL for MediaWiki or Special:PslZendSearchLucene. This allows users to evaluate search results by comparing the built-in search vs. PslZendSearchLucene search.

If the performance is acceptable to replace the built-in search engine, Zend Search Lucene for MediaWiki can easily be configured to be the default search engine. To do so, modify LocalSettings.php and add this before the require_once line that includes the extension:

$wgSearchType = 'PslZendSearchLucene';

Now, the standard search method will use PslZendSearchLucene by default. Note: when used in this way, the extension preserves the functionality of the Go and Search buttons.

Troubleshooting

If you get something like this:

Warning: PslZendSearchLucene::require_once(Zend/Search/Lucene.php) [pslzendsearchlucene.require-once]: failed to open stream: No such file or 
directory in /var/www/vhosts/my.sexyserver.no/subdomains/sysdoc/httpdocs/extensions/PslZendSearchLucene/PslZendSearchLucene_body.php on line 92

... you may have to modify your PHP open_basedir-directive to point to your Zend Framework Library. A common example could be to edit an apache vhost.conf:

<Directory /var/www/vhosts/my.sexyserver.no/subdomains/sysdoc/httpdocs>
    <IfModule mod_php4.c>
         php_admin_value open_basedir "/PSL_ADD_ONS/ZF/:/tmp"
    </IfModule>
 
    <IfModule mod_php5.c>
          php_admin_value open_basedir "/PSL_ADD_ONS/ZF/:/tmp"
    </IfModule>
</Directory>

...and reload it on shell:

/usr/local/psa/admin/sbin/websrvmng --reconfigure-vhost --vhost-name=my.sexyserver.gov

ToDo

• Search within other exotic formats
• Translations for Helppage
• index->delete(); while increment update

Revisions

• v2.1 - 03.05.2011
  • hot fix "quote", hot fix "dead links"
• v2.0 - 03.05.2011
  • multiple features, minor fixes, performance optimizations
• v0.9.3 - 09.03.2011
  • implementing of stop words, help system (german)
• v0.9.2 - 07.03.2011
  • ranking options, refactoring (performance), utf8-bug fix
• v0.9.1 - 04.03.2011
  • redirect bug fix (default search engine)
• v0.9.0 - 22.02.2011
  • some enhanced features, default search switch
• v0.8 - 21.02.2011
  • some enhanced features, debug mode
• v0.7 - 20.02.2011
  • some enhanced features, search field select
• v0.6 - 18.02.2011
  • bug fix redirect
• v0.5 - 14.01.2011
  • bug fix mismatch of uppercase strings
• v0.4 - 19.01.2011
  • some enhanced features, phrase search
• v0.3 - 23.01.2011
  • index engine for multiple wikis
• v0.2 - 16.01.2011
  • some enhanced features, highligting etc.
• v0.1 - 20.01.2011
  • initial release (RFC)

Announcements

08.02.2013
Zend cancelled ZSL for the 2.0 branch of the Zend Framework
I will maintain this project untill the seperate ZF 1.x branch disapears.
01.06.2011
Version 2.1 released

For further support see also

• webserver-management.de

See also

• In depth description of Zend Search Lucene for MediaWiki (German)