# for more information see <http://en.wikipedia.org/wiki/Robots.txt>

User-agent: *
Disallow:
Crawl-delay: 10
Request-rate: 1/10
Visit-time: 1800-2359

User-agent: *
Disallow:
Crawl-delay: 10
Request-rate: 1/10
Visit-time: 0000-0600