<div dir="ltr">Hello,<div><br></div><div>I checked my old note and configured as follows, but I doubt myself that will help you... </div><div><br></div><div><div>- Configured the resolver of internal reverse proxy to internal ALB/ELB to AWS dedicated DNS server.</div><div>- Configured the resolver cache 30 seconds.</div><div>- Set proxy_pass arguments to ALB/ELB endpoint names.</div><div><br></div><div>Here is my old note:</div><div><br></div><div>The resolver 172.31.0.2 is the address of AWS dedicated DNS server. I checked the response time of resolve accross the availability zone with dig, the AWS dedicated DNS server will respond without difference in spite of our cache resolver.</div><div><br></div><div>TTL of DNS record in AWS is 60seconds, ALB(ELB) will scale in/out and update their address, so the internal resolver in nginx will keep 30 seconds, that is valid=30s with resolver directive.</div><div><br></div><div>resolver 172.31.0.2 valid=30s;</div><div>resolver_timeout 10s;</div><div>set $backendelb "<a href="http://backendelb-87654321.elasticloaodbalancing.region.amazonaws.com">backendelb-87654321.elasticloaodbalancing.region.amazonaws.com</a>";</div><div>location / { proxy_pass http://$backendelb; }</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Oct 28, 2017 at 3:58 AM, RKGood <span dir="ltr"><<a href="mailto:nginx-forum@forum.nginx.org" target="_blank">nginx-forum@forum.nginx.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<br>
<br>
We are trying to find a solution from past couple of days but nothing seems<br>
to work so far. Our server in prod going down everyday when the AWS ELB<br>
changes ip address. Our's little complex proxy (don't ask me why we have to<br>
do this :), there is a strong reason for it), our clients will send requests<br>
apache (legacy), apache proxies to nginx (new) and nginx decides whether<br>
proxy back to apache or serve the request with new micro services. Resolver<br>
re-resolves the new micro services (internal alb) ip address but fail to<br>
re-resolve the legacy apache (has a ELB with route 53 entry in front). We<br>
are using https endpoint to proxy apache request. The request flows thru ELB<br>
(legacy) -> Apache -> ELB (new) -> nginx -> ELB (legacy) -> apache<br>
<br>
Can you please provide feedback on what are we doing wrong, this is only<br>
happening in production. Our load is normal few fundred requests per second.<br>
We aren't able to simulate it in test environment.<br>
<br>
Here is the configuration:<br>
<br>
<br>
user nginx;<br>
worker_processes auto;<br>
worker_rlimit_nofile 5120;<br>
error_log /var/log/nginx/error.log warn;<br>
pid /var/run/nginx.pid;<br>
<br>
events {<br>
worker_connections 2048;<br>
}<br>
<br>
http {<br>
include /etc/nginx/mime.types;<br>
default_type text/plain;<br>
<br>
resolver 10....2 172.16.0.23 valid=30s ipv6=off;<br>
resolver_timeout 5s;<br>
<br>
log_format main '$proxy_protocol_addr - [$status] - [$request_time] -<br>
[$upstream_response_time] - $server_name $upstream_addr $request';<br>
<br>
access_log /var/log/nginx/error.log main;<br>
<br>
rewrite_log on;<br>
<br>
client_body_timeout 60s;<br>
client_header_timeout 30s;<br>
send_timeout 60s;<br>
sendfile off;<br>
<br>
tcp_nodelay on;<br>
tcp_nopush on;<br>
reset_timedout_connection on;<br>
<br>
server_names_hash_bucket_size 128;<br>
client_body_buffer_size 64k;<br>
client_max_body_size 10m;<br>
<br>
server {<br>
<br>
listen 443 ssl proxy_protocol default_server;<br>
server_name <a href="http://mydomain.com" rel="noreferrer" target="_blank">mydomain.com</a>;<br>
ssl_certificate mydomain.crt;<br>
ssl_certificate_key mydomain.key;<br>
<br>
set $alb_upstream aws-internal-alb;<br>
set $apache_upstream <a href="http://legacy.domain.com" rel="noreferrer" target="_blank">legacy.domain.com</a>;<br>
<br>
proxy_buffers 8 24k;<br>
proxy_buffer_size 2k;<br>
proxy_http_version 1.1;<br>
proxy_set_header Connection "";<br>
proxy_set_header X-Real-IP $proxy_protocol_addr;<br>
proxy_next_upstream off;<br>
<br>
<br>
location /services/(migrated1|<wbr>migrated2)/ {<br>
<br>
proxy_set_header Host $host;<br>
proxy_connect_timeout 302;<br>
proxy_read_timeout 302;<br>
<br>
rewrite /services/(.*) /$1?$args break;<br>
proxy_pass <a href="http://alb_upstream" rel="noreferrer" target="_blank">http://alb_upstream</a>;<br>
}<br>
<br>
location /services/ {<br>
proxy_set_header x-nginx-rejected true;<br>
proxy_set_header Host <a href="http://legacy.domain.com" rel="noreferrer" target="_blank">legacy.domain.com</a>;<br>
proxy_connect_timeout 302;<br>
proxy_read_timeout 302;<br>
<br>
rewrite /services/(.*) /$1?$args break;<br>
proxy_pass https://$apache_upstream;<br>
}<br>
<br>
}<br>
}<br>
<br>
Thanks in advance.<br>
RK<br>
<br>
Posted at Nginx Forum: <a href="https://forum.nginx.org/read.php?2,277101,277101#msg-277101" rel="noreferrer" target="_blank">https://forum.nginx.org/read.<wbr>php?2,277101,277101#msg-277101</a><br>
<br>
______________________________<wbr>_________________<br>
nginx mailing list<br>
<a href="mailto:nginx@nginx.org">nginx@nginx.org</a><br>
<a href="http://mailman.nginx.org/mailman/listinfo/nginx" rel="noreferrer" target="_blank">http://mailman.nginx.org/<wbr>mailman/listinfo/nginx</a><br>
</blockquote></div><br></div>