Thread:Project:Support desk/Problem crawling MediaWiki with Search Server / Sharepoint

Hi all,

I am trying to crawl a MediaWiki (MediaWiki 1.17.0 PHP 5.3.3-7+squeeze1 (apache2handler) MySQL 5.1.49-3) with Microsoft Search Server 2010 (basically Sharepoint), but unfortunately I cannot make any progress.

I am trying to start at the main page, although Special:AllPages would be better (but is not indexable :

I know that,
 * My crawl must be case-sensitive (I implemented it on Sharepoint)
 * The special pages are no-index, no-follow per se

I already checked,
 * that $wgNoFollowLinks = false; is in the localsettings.php
 * the robots.txt which allows crawling
 * $wgDefaultRobotPolicy which is index, follow
 * $wgNamespaceRobotPolicies which does not contain the NS which I'd like to crawl (not 100% sure)

Btw. - I already tried an OpenSearch-Approach but MediaWiki does not support returning OpenSearch RSS-feeds for search results... Oh and I searched the web like crazy, seems like I am not the only one who has this particular challenge...

Numerous days trying to crawl MediaWiki - I'd really appreciate help :)

Kind regards,

MMF