Aborting malicious requests

[lists at lazygranch.com]

On Tue, 20 Mar 2018 13:03:09 +0000
"Friscia, Michael" <michael.friscia at yale.edu> wrote:

> This is great, thank you again, this is a huge jumpstart!

Per NIST best practices, you should limit the HTML verbs that you
allow. A very simple website can run on just GET and HEAD. Here is how
you 444 websites trying to POST for example to your website. In this
case, only GET and HEAD are allowed.

        if ($request_method !~ ^(GET|HEAD)$ ) {
            return 444;

You might as well trap bad agents. Basically whatever isn't a browser.
I found a list on github and have been adding new ones as I get
pestered.

https://paste.fedoraproject.org/paste/FI-IRICSJy1SR5mwBZxVDQ/
I called this file mapbadagentslarge. Use the same basic scheme. This
list is overkill, but it doesn't seem to slow down nginx. What you want
to avoid are the scrapers like nutch.

        if ($badagent)     { return 444; }

I also block bad referrals. Porn sites for instance. If a bad site links
to your site, at least you can return a 403 (not 444) and google won't
consider the link in its algorithm. You can request an incognito
browser and look at them, preferably in private. I've clicked on the
occasional odd referral only to have porn pop up my screen while at a
coffee shop. Blocking referrals will lower your google rank.

https://paste.fedoraproject.org/paste/6ZLa10-4L9KocFNJiNG~pw/

        if ($bad_referer)  { return 403; }

If you are using encryption AND if you are mapping http requests to
https, you should do these maps in both the http and https blocks. It
doesn't make sense to go through the encryption process just to tell
the IP to take a hike.

What you do with the 444 entries in the access.log is up to you. You
can do nothing and probably be fine. I have scripts to get the bad IPs
and if they have no "eyes", I block them in the firewall. Determining
if they have no eyes is time consuming. You can feed the IP to
ip2location.com. A few of the IPs assigned to data centers really go to
ISPs. ISPs have eyes, so you don't want to block them. You can get the
IP space assigned to the entity with bgp.he.net.