Errol King's activity

Jim Gilliam responded to Robot.txt with completed 2012-01-27 07:25:00 -0800

Robot.txt

Do we have the ability to post Robot.txt files? I want to use it to block google and other search engines from crawling my site while its in development. Thanks.

Official response from Jim Gilliam completed

There is now a checkbox under Websites > Settings for "Allow search engines" - uncheck that and your robots.txt will not allow Google or other search engines to index it.