Manual:$wgDefaultRobotPolicy

Details
Allows specifying the default robot policy for all pages on the wiki. The default is to encourage indexing and following of links. This can be overridden on a ns>Special:MyLanguage/Manual:$wgNamespaceRobotPolicies|per-namespace and/or article>Special:MyLanguage/Manual:$wgArticleRobotPolicies|per-article basis.

e.g.

would forbid indexing and following of links on all pages outside the main namespace.

External link

 * [http://www.robotstxt.org/meta.html About the Robots    tag] (www.robotstxt.org)