Can block spider access to all files, include .php file?

filebackup nginx-forum at nginx.us
Mon Jun 22 09:45:30 MSD 2009


I had found if I blocked all files, include .php file, I can't open my website.

location ~* .*\.*$ {
if ($http_user_agent ~* baidu|spider) {
rewrite ^/ http://www.mydomain.com/robots.txt;
}
}

If I setup this rule into my nginx.conf file, I can't open my website. So, how can I block spider access to all files?

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,3236,3236#msg-3236






More information about the nginx mailing list