Manual:$wgDefaultRobotPolicy

Details
Allows specifying the default robot policy for all pages on the wiki. The default is to encourage indexing and following of links. This can be overridden on a per-namespace and/or per-article basis.

e.g. would forbid indexing and following of links on all pages outside the main namespace.

External link

 * About the Robots &lt;META&gt; tag (www.robotstxt.org)