Slow down, but not stop, serving pages to bot that doesn't respect robots.txt delay
nginx-forum at nginx.us
Fri Aug 12 09:33:57 UTC 2011
Thank you, but i want to serve the real content to the bot not an error
Is this possible?
Posted at Nginx Forum: http://forum.nginx.org/read.php?2,213704,213727#msg-213727
More information about the nginx