nginx behind load balancer
敬张
911csj at gmail.com
Sat Nov 26 20:44:46 UTC 2011
$remote_addr(OSI 3layer) and $http_x_forwarded_for(OSI 7layer) of these two
variables is different. behind LB server may be you can use config like
this.
http {
map $proxy_add_x_forwarded_for $xff_pass {
default "xff403";
~^61\.152\.90\.8 "xffpass";
~^61\.172\.241\. "xffpass";
limit_req_zone $http_x_forwarded_for zone=one:10m rate=200r/s;
...
server {
if ($xff_pass !~* xffpass) {
return 403;
}
limit_req zone=one;
...
}
}
2011/11/26 Rami Essaid <rami.essaid at gmail.com>
> As a quick update, it looks like this has happened before. The load
> balancer bounces off of several internal IP's sometimes and nginx picks the
> last one only. Does anyone know of a workaround to remove the last two
> trusted IPs from the x-forwarded-for header?
>
> http://forum.nginx.org/read.php?11,26102,214069
>
> Rami
>
> On Fri, Nov 25, 2011 at 4:54 PM, Rami Essaid <rami.essaid at gmail.com>wrote:
>
>> In looking at the $proxy_add_x_forwarded_for variable I believe that the
>> Loadbalancer is in fact passing the variables but nginx is taking the wrong
>> value?
>>
>> Here is what i get from the variables.
>> $proxy_add_x_forwarded_for: 217.27.244.18, 10.160.43.200, 10.160.43.200
>> $remote_addr: 10.160.43.200
>>
>> Does this mean that nginx is taking the last value in the
>> x-forwarded-for?
>>
>> To answer your other question, yes, the Amazon plenty of sockets
>> available to connect to nginx.
>>
>> The reason we added this was to be able to utilize Amazon's auto scaling
>> feature for redundancy and scalability. We have not been able to test
>> performance yet since we are still trying to make it work but as soon as we
>> do I will be sure to report back.
>>
>>
>> On Fri, Nov 25, 2011 at 4:19 PM, Stefan Caunter <stef at caunter.ca> wrote:
>>
>>> On Fri, Nov 25, 2011 at 12:37 PM, Rami Essaid <rami.essaid at gmail.com>
>>> wrote:
>>> > Thanks for the help Maxim! We disabled our limit_req and that seemed
>>> to
>>> > have fixed the problem. Looking at the logs it seems that only 1/3 of
>>> the
>>> > requests are correctly getting the new IP assigned via the realIP
>>> module,
>>> > the remainder are still logging the load balancer IP. This probably
>>> is more
>>> > of an issue with the amazon load balancer but do you have any idea on
>>> what
>>> > may be going on?
>>> > Also, where would you recommend as a place to start tracking and
>>> fixing the
>>> > other issue?
>>> >
>>>
>>> You have not established that load balancer is setting X-Forwarded-For
>>> or some other header to pass Real IP.
>>>
>>> Does the amazon pseudo device have enough sockets available to connect
>>> to nginx?
>>>
>>> What do you expect from adding this layer? Were you hitting
>>> performance limits? Has it improved or degraded performance?
>>>
>>> >
>>> >
>>> > On Fri, Nov 25, 2011 at 12:14 PM, Maxim Dounin <mdounin at mdounin.ru>
>>> wrote:
>>> >>
>>> >> Hello!
>>> >>
>>> >> On Fri, Nov 25, 2011 at 09:54:14AM -0500, Rami Essaid wrote:
>>> >>
>>> >> > Hi Maxim,
>>> >> >
>>> >> > We implemented to module and still had some trouble. A lot of the
>>> >> > connections would return " 503 Service Temporarily Unavailable".
>>> Our
>>> >> > configuration works fine without the load balancer but then gives
>>> these
>>> >> > 503
>>> >> > errors behind the load balancer.
>>> >>
>>> >> nginx itself will only return 503 if it hits either limit_conn or
>>> >> limit_req.
>>> >>
>>> >> If you see this returned by nginx, and it only happens with load
>>> >> balancer, this may indicate you've not configured realip module
>>> >> properly (or your load balancer doesn't provide appropriate
>>> >> headers) and you are hitting per-ip limits configured due to all
>>> >> requests appear to be from load balancer.
>>> >>
>>> >> Check if client's ip logged is really client's one, not an ip of
>>> >> your load balancer.
>>> >>
>>> >> > Looking into the error logs I notice a lot of these errors both
>>> with and
>>> >> > without the load balancer "connect() failed (111: Connection
>>> refused)
>>> >> > while
>>> >> > connecting to upstream". Could this be the reason that we are
>>> having
>>> >> > issues?
>>> >>
>>> >> Unlikely, but It's a good idea to track and fix this in any case.
>>> >>
>>> >> Maxim Dounin
>>> >>
>>> >> >
>>> >> > Thanks!
>>> >> > Rami
>>> >> >
>>> >> >
>>> >> > On Mon, Nov 21, 2011 at 7:46 AM, Maxim Dounin <mdounin at mdounin.ru>
>>> >> > wrote:
>>> >> >
>>> >> > > Hello!
>>> >> > >
>>> >> > > On Mon, Nov 21, 2011 at 07:40:49AM -0500, Rami Essaid wrote:
>>> >> > >
>>> >> > > > Thanks Maxim,
>>> >> > > >
>>> >> > > > This looks like exactly what we need. In your experience does
>>> this
>>> >> > > resolve
>>> >> > > > most issues behind a load balancer?
>>> >> > >
>>> >> > > Yes.
>>> >> > >
>>> >> > > Maxim Dounin
>>> >> > >
>>> >> > > > On Mon, Nov 21, 2011 at 7:38 AM, Maxim Dounin <
>>> mdounin at mdounin.ru>
>>> >> > > wrote:
>>> >> > > >
>>> >> > > > > Hello!
>>> >> > > > >
>>> >> > > > > On Mon, Nov 21, 2011 at 07:25:39AM -0500, Rami Essaid wrote:
>>> >> > > > >
>>> >> > > > > > Hi Guys,
>>> >> > > > > >
>>> >> > > > > > This weekend for scalability we tried putting our nginx
>>> servers
>>> >> > > behind
>>> >> > > > > > amazon's elastic load balancers and came across a road
>>> block: it
>>> >> > > does not
>>> >> > > > > > transparently pass the user IP and header information to
>>> nginx.
>>> >> > > > > > This
>>> >> > > > > caused
>>> >> > > > > > issues with several pieces of nginx we use including the IP
>>> >> > > > > > allow /
>>> >> > > deny
>>> >> > > > > > rules, the limit_req module, and the limit_con module. Has
>>> >> > > > > > anyone
>>> >> > > > > > successfully put nginx behind a load balancer? Any ideas
>>> on how
>>> >> > > > > > to
>>> >> > > make
>>> >> > > > > > this work?
>>> >> > > > >
>>> >> > > > > http://wiki.nginx.org/HttpRealIpModule
>>> >> > > > >
>>> >> > > > > Maxim Dounin
>>> >> > > > >
>>> >> > > > > _______________________________________________
>>> >> > > > > nginx mailing list
>>> >> > > > > nginx at nginx.org
>>> >> > > > > http://mailman.nginx.org/mailman/listinfo/nginx
>>> >> > > > >
>>> >> > >
>>> >> > > > _______________________________________________
>>> >> > > > nginx mailing list
>>> >> > > > nginx at nginx.org
>>> >> > > > http://mailman.nginx.org/mailman/listinfo/nginx
>>> >> > >
>>> >> > > _______________________________________________
>>> >> > > nginx mailing list
>>> >> > > nginx at nginx.org
>>> >> > > http://mailman.nginx.org/mailman/listinfo/nginx
>>> >> > >
>>> >>
>>> >> > _______________________________________________
>>> >> > nginx mailing list
>>> >> > nginx at nginx.org
>>> >> > http://mailman.nginx.org/mailman/listinfo/nginx
>>> >>
>>> >> _______________________________________________
>>> >> nginx mailing list
>>> >> nginx at nginx.org
>>> >> http://mailman.nginx.org/mailman/listinfo/nginx
>>> >
>>> >
>>> > _______________________________________________
>>> > nginx mailing list
>>> > nginx at nginx.org
>>> > http://mailman.nginx.org/mailman/listinfo/nginx
>>> >
>>>
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>
>>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20111127/d7abbaec/attachment.html>
More information about the nginx
mailing list