Country banning

[Jim Ohlstein]

Igor Sysoev wrote:
> On Sat, Aug 29, 2009 at 01:58:11PM -0400, Jim Ohlstein wrote:
>
>   
>> Igor Sysoev wrote:
>>     
>>> On Sat, Aug 29, 2009 at 11:41:16AM -0400, Jim Ohlstein wrote:
>>>
>>>  
>>>       
>>>> We're dealing with a high degree of fraud from certain countries and 
>>>> would like to simply ban all IP's from those countries.
>>>>
>>>> I seem to recall reading here that using the Geo module is more 
>>>> efficient for this purpose than the GeoIP module.
>>>>
>>>> Currently I have the following in nginx.conf:
>>>>
>>>> geo $country {
>>>>   include geo.conf;
>>>> }
>>>>
>>>> where geo.conf is generated from MaxMind country lite csv database using 
>>>> geo2nginx.pl supplied with nginx.
>>>>
>>>> In the site config I have multiple if statements like:
>>>>
>>>> server {
>>>>   ...
>>>>
>>>>   if ($country = XX) {
>>>>       return 403;
>>>>   }
>>>>
>>>>   if ($country = YY) {
>>>>       return 403;
>>>>   }
>>>>
>>>>   if ($country = ZZ) {
>>>>       return 403;
>>>>   }
>>>>
>>>> ...
>>>>
>>>> }
>>>>
>>>> Is this more efficient than using GeoIP module? Is there a more 
>>>> efficient way of doing this?
>>>>    
>>>>         
>>> Yes:
>>>
>>> # convert MaxMind's blocks to nginx format:
>>> ./nginx.pl GeoLiteCity_20090701/GeoLiteCity-Blocks.csv > maxmind.conf
>>>
>>> # convert MaxMand location to country names:
>>> ./country.pl GeoLiteCity_20090701/GeoLiteCity-Location.csv maxmind.conf > 
>>> countries.conf
>>>
>>> # set forbidden countries to 1, ignore others:
>>> perl -ne 'print "$1 1\n" if /^(\S+) (US|RU|CN|...);$/' < countries.conf > 
>>> networks.conf
>>>
>>> # aggregate networks:
>>> ./compress.pl networks.conf > forbidden.conf
>>>
>>> Then use it:
>>>
>>> geo $forbidden {
>>>    default  0;
>>>    include  forbidden.conf;
>>> }
>>>
>>> server {
>>>     if ($forbidden) {
>>>         return 403;
>>>     }
>>>
>>>
>>> nginx.pl:
>>> -----------
>>> #!/usr/bin/perl -w
>>>
>>> use Net::CIDR::Lite;
>>> use strict;
>>> use warnings;
>>>
>>> while (<>) {
>>>    if (/^"([^"]+)","([^"]+)","([^"]+)"/){
>>>        my($start, $end, $region) = ($1, $2, $3);
>>>        my $cidr = Net::CIDR::Lite->new(ip($start) . "-" . ip($end));
>>>        print((join " $region;\n", $cidr->list), " $region;\n");
>>>    }
>>> }
>>>
>>> sub ip {
>>>    my $n = shift;
>>>    return (($n >> 24) & 0xff) . "." .
>>>           (($n >> 16) & 0xff) . "." .
>>>            (($n >> 8) & 0xff) . "." .
>>>                   ($n & 0xff);
>>> }
>>> -----------
>>>
>>> country.pl:
>>> -----------
>>> #!/usr/bin/perl -w
>>>
>>> use warnings;
>>> use strict;
>>>
>>> my %country;
>>>
>>> while (<>) {
>>>    if (/^(\d+),"([^"]+)","([^"]*)"/) {
>>>        $country{$1} = $2;
>>>        next;
>>>    }
>>>
>>>    if (/^\S+ \d+;$/) {
>>>        last;
>>>    }
>>> }
>>>
>>> do {
>>>    if (/^(\S+) (\d+);$/) {
>>>        print "$1 $country{$2};\n";
>>>    } else {
>>>        print STDERR;
>>>    }
>>>
>>> } while (<>);
>>> -----------
>>>
>>> compress.pl:
>>> -----------
>>> #!/usr/bin/perl -w
>>>
>>> use Net::CIDR::Lite;
>>> use strict;
>>> use warnings;
>>>
>>> my %cidr;
>>>
>>> while (<>) {
>>>    if (/^(\S+) (\S+);/) {
>>>        my($net, $region) = ($1, $2);
>>>        if (!defined $cidr{$region}) {
>>>            $cidr{$region} = Net::CIDR::Lite->new;
>>>        }
>>>        $cidr{$region}->add($net);
>>>    }
>>> }
>>>
>>> for my $region (sort { $a <=> $b } keys %cidr) {
>>>    print((join " $region;\n", $cidr{$region}->list), " $region;\n");
>>> }
>>> -----------
>>>
>>>  
>>>       
>> Thank you Igor. Unfortunately forbidden.conf comes up as an empty file. 
>> I have confirmed the three perl files are as above and I have run it twice.
>>
>> [jim at saturn geo]# ls -lh
>> total 195M
>> -rwx------ 1 jim jim      422 Aug 29 13:15 compress.pl
>> -rw-r--r-- 1 jim jim       91M Aug 29 13:49 countries.conf
>> -rwx------ 1 jim jim      334 Aug 29 13:15 country.pl
>> -rw-r--r-- 1 jim jim       0 Aug 29 13:50 forbidden.conf
>> drwxr-xr-x 2 jim jim      4.0K Aug 29 13:25 GeoLiteCity_20090801
>> -rw-r--r-- 1 jim jim       104M Aug 29 13:48 maxmind.conf
>> -rw-r--r-- 1 jim jim       156K Aug 29 13:50 networks.conf
>> -rwx------ 1 jim jim      497 Aug 29 13:14 nginx.pl
>>     
>
> Sorry, I mistaked:
>
> -perl -ne 'print "$1 1\n" ...
> +perl -ne 'print "$1 1;\n" ...
>
>
>   

Thank you again Igor. It's working like a charm and loads are down a bit 
as well.

I'm assuming that "forbidden.conf" is loaded in memory and so if I 
replace it then I need to reload nginx. Correct?


Jim