fake googlebots

Rainer Duffner rainer at ultra-secure.de
Sun Sep 25 22:06:31 UTC 2016

> Am 25.09.2016 um 23:58 schrieb lists at lazygranch.com:
> I got a spoofed googlebot hit. It was easy to detect since there were
> probably a hundred requests that triggered my hacker detection map
> scheme. Only two requests received a 200 return and both were harmless.
> 200 - - [25/Sep/2016:17:45:23 +0000] "GET / HTTP/1.1" 847 "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "-"
> For the fake googlebot:
> # host
> Host not found: 3(NXDOMAIN)
> For a real googlebot:
> # host
> domain name pointer crawl-66-249-69-184.googlebot.com.
> IP2location shows it is a Chinese ISP:
> 3(NXDOMAIN)http://www.ip2location.com/
> Nginx has a reverse DNS module:
> https://github.com/flant/nginx-http-rdns
> I see it has a 10.1 issue:
> https://github.com/flant/nginx-http-rdns/issues/8
> Presuming this bug gets fixed, does anyone have code to verify
> googlebots? Or some other method?

Sorry to be so blunt - but what’s the point?

You can also password-protect your site and give the credentials only to your friends.
Problem solved.

Most of the traffic of the web these days is created by bots (unless you’re a popular shop or offer original, often updated content for popular topics, then you’ll actually get visitors).

If it’s not the Big G, it might be bing or baidu or yandex or some other bot.

More information about the nginx mailing list