Slow down, but not stop, serving pages to bot that doesn't respect robots.txt delay

rastrano nginx-forum at nginx.us
Fri Aug 12 09:33:57 UTC 2011


Thank you, but i want to serve the real content to the bot not an error
page.

Is this possible?

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,213704,213727#msg-213727



More information about the nginx mailing list