Manual:Short URL/Prevent bots from crawling index.php
Prevent bots from crawling index.php[edit]
If you are using short URLs (see note below), you can make sure that search engines only index actual wiki pages, without indexing action views (such as edit or history pages, with URLs in the form index.php?title=Main_page&action=edit).
Create a file named robots.txt in the root of your MediaWiki installation with the following content.
User-agent: * Disallow: /index.php
Note: Creating a robots.txt file in the root directory of your MediaWiki installation with Disallow: /index.php without creating a short URL first will block all pages from being indexed. This is because your MediaWiki page will still have a index.php in the title, which the robots.txt file will "disallow".
If you are using long URLs, follow the link below.