r1ch+nginx at teamliquid.net
Mon Jun 11 10:00:35 UTC 2018
This is almost certainly not Google as they obey robots.txt. The & to &
conversion is another sign of a poor quality crawler. Check the RDNS and
you will find it's probably some IP faking Google UA, I suggest blocking at
On Fri, Jun 8, 2018 at 1:57 AM shiz <nginx-forum at forum.nginx.org> wrote:
> Recently, Google has started spidering my website and in addition to normal
> pages, appended "&" to all urls, even the pages excluded by robots.txt
> e.g. page.php?page=aaa -> page.php?page=aaa&
> Any idea how to redirect/rewrite this?
> Posted at Nginx Forum:
> nginx mailing list
> nginx at nginx.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the nginx