limit_req per subnet?

lists at lists at
Wed Dec 14 19:15:44 UTC 2016

‎They claim to obey robots.txt. They also claim to to use consecutive IP addresses.
Some dated posts (2011) indicate semrush uses AWS. I block all of AWS IP space and can say I've never seen a semrush bot. So that might be a solution. I got the AWS IP space from some Amazon Web page. 

I get a bit of kick back about blocking things that are not eyeballs like colos and VPS, but it works for me.  I only block after seeing a hacking attempt(s) in my logs.

  Original Message  
From: Grant
Sent: Wednesday, December 14, 2016 10:31 AM
To: nginx at
Reply To: nginx at
Subject: Re: limit_req per subnet?

> I am curious what is the request uri they was hitting. Was it a dynamic page
> or file or a static one.

It was semrush and it was all manner of dynamic pages.

- Grant
nginx mailing list
nginx at

More information about the nginx mailing list