Thread:User talk:^demon/Gerrit crawling

Why all disallowed? It's not like we are so rich as regards search as to ignore the search engines... I suppose it's because gerrit/gitweb is so slow and crawling would kill it? Isn't throttling possible? Thank you very much,