How to block fake google spider and fake web browser access?

[Francis Daly]

On Tue, May 05, 2015 at 09:07:41AM -0400, meteor8488 wrote:

Hi there,

>  I tried to use "deny" to deny access from an IP. But it seems that it can
> still access my server.
> 
> In my http part:
> 
> deny  69.85.92.0/23;
> deny  69.85.93.235;

A request comes in to nginx. nginx chooses one server{} block in its
configuration to handle it. nginx chooses one location{} block in that
server{} configuration to handle it. Only configuration directives in,
or inherited into, that location{} are relevant.

(If you use any rewrite-module directives, things may be different.)

>  69.85.93.235 - - [05/May/2015:19:44:22 +0800] "GET /thread-1251687-1-1.html
> HTTP/1.0" 302 154 "http://www.example.com" "Mozilla/5.0 (compatible;
> Baiduspider/2.0; +http://www.baidu.com/search/spider.html)"
> "123.125.71.107"

What is the one location{} that handles this request? What "allow" and
"deny" directives are in that location{}? And in the enclosing server{}?


Can you provide a complete nginx.conf that shows the behaviour you report?

(It doesn't have to be your production config. Something smaller
that shows this problem on a test machine, may make obvious where the
problem is.)

Thanks,

	f
-- 
Francis Daly        francis at daoine.org