Resolver not re-resolving new ip address of an AW ELB

[Nishikubo Minoru]

Hello,

I checked my old note and configured as follows, but I doubt myself that
will help you...

- Configured the resolver of internal reverse proxy to internal ALB/ELB to
AWS dedicated DNS server.
- Configured the resolver cache 30 seconds.
- Set proxy_pass arguments to ALB/ELB endpoint names.

Here is my old note:

The resolver 172.31.0.2 is the address of AWS dedicated DNS server. I
checked the response time of resolve accross the availability zone with
dig, the AWS dedicated DNS server will respond without difference in spite
of our cache resolver.

TTL of DNS record in AWS is 60seconds, ALB(ELB) will scale in/out and
update their address, so the internal resolver in nginx will keep 30
seconds, that is valid=30s with resolver directive.

resolver 172.31.0.2 valid=30s;
resolver_timeout 10s;
set $backendelb "
backendelb-87654321.elasticloaodbalancing.region.amazonaws.com";
location / { proxy_pass http://$backendelb; }

On Sat, Oct 28, 2017 at 3:58 AM, RKGood <nginx-forum at forum.nginx.org> wrote:

> Hello,
>
> We are trying to find a solution from past couple of days but nothing seems
> to work so far. Our server in prod going down everyday when the AWS ELB
> changes ip address. Our's little complex proxy (don't ask me why we have to
> do this :), there is a strong reason for it), our clients will send
> requests
> apache (legacy), apache proxies to nginx (new) and nginx decides whether
> proxy back to apache or serve the request with new micro services. Resolver
> re-resolves the new micro services (internal alb) ip address but fail to
> re-resolve the legacy apache (has a ELB with route 53 entry in front). We
> are using https endpoint to proxy apache request. The request flows thru
> ELB
> (legacy) -> Apache -> ELB (new) -> nginx -> ELB (legacy) -> apache
>
> Can you please provide feedback on what are we doing wrong, this is only
> happening in production. Our load is normal few fundred requests per
> second.
> We aren't able to simulate it in test environment.
>
> Here is the configuration:
>
>
> user nginx;
> worker_processes  auto;
> worker_rlimit_nofile 5120;
> error_log /var/log/nginx/error.log warn;
> pid /var/run/nginx.pid;
>
> events {
>   worker_connections  2048;
> }
>
> http {
>   include    /etc/nginx/mime.types;
>   default_type text/plain;
>
>   resolver 10....2 172.16.0.23 valid=30s ipv6=off;
>   resolver_timeout 5s;
>
>   log_format main '$proxy_protocol_addr - [$status] - [$request_time] -
> [$upstream_response_time] - $server_name $upstream_addr $request';
>
>   access_log /var/log/nginx/error.log main;
>
>   rewrite_log on;
>
>   client_body_timeout             60s;
>   client_header_timeout           30s;
>   send_timeout                    60s;
>   sendfile     off;
>
>   tcp_nodelay on;
>   tcp_nopush   on;
>   reset_timedout_connection on;
>
>   server_names_hash_bucket_size 128;
>   client_body_buffer_size 64k;
>   client_max_body_size 10m;
>
>   server {
>
>         listen        443 ssl proxy_protocol default_server;
>     server_name  mydomain.com;
>     ssl_certificate      mydomain.crt;
>     ssl_certificate_key  mydomain.key;
>
>     set $alb_upstream aws-internal-alb;
>     set $apache_upstream legacy.domain.com;
>
>     proxy_buffers 8 24k;
>     proxy_buffer_size 2k;
>     proxy_http_version 1.1;
>     proxy_set_header Connection "";
>     proxy_set_header X-Real-IP       $proxy_protocol_addr;
>     proxy_next_upstream off;
>
>
>     location /services/(migrated1|migrated2)/ {
>
>         proxy_set_header Host $host;
>         proxy_connect_timeout 302;
>         proxy_read_timeout 302;
>
>         rewrite /services/(.*) /$1?$args break;
>         proxy_pass http://alb_upstream;
>     }
>
>     location /services/ {
>         proxy_set_header x-nginx-rejected true;
>         proxy_set_header Host legacy.domain.com;
>         proxy_connect_timeout 302;
>         proxy_read_timeout 302;
>
>         rewrite /services/(.*) /$1?$args break;
>         proxy_pass https://$apache_upstream;
>     }
>
>   }
> }
>
> Thanks in advance.
> RK
>
> Posted at Nginx Forum: https://forum.nginx.org/read.
> php?2,277101,277101#msg-277101
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20171031/443e1fba/attachment.html>