Manual:Robot policy

From mediawiki.org
Jump to navigation Jump to search

Robot policy is used to tell web crawlers and other tools what to exclude.

Configuration settings[edit]