<html><head><meta http-equiv="content-type" content="text/html; charset=us-ascii"></head><body style="overflow-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">Hey Maxim,<div><br></div><div>You are absolutely right, I totally forgot about the cache_lock. I have listed our settings below.</div><div><br></div><div>The reason we are using the cache_lock is to save the backend application to not get 100's of requests when a stale item is invalid. Even if we have use_stale updating, we notice that only the first request will use the stale item, the following requests will do a new request even though there is already a background request going on to refresh the stale item. (This does not happen if we set keepalive to 0, where new connections are being used, but has the performance degradation as mentioned.) This was the reasoning for the cache_lock, but that gives the issue about the 500ms lock, while the item might already be refreshed after 100ms.</div><div><br></div><div>So is there an option to make the 500ms in the <a href="http://ngx_http_file_cache.ch">ngx_http_file_cache.c</a> configurable? Are there any downsides to that? Or is there a better alternative</div><div><br></div><div>Our config:</div><div><p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; min-height: 14px;"><span style="font-kerning: none"></span><br></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">add_header X-Cache-Status $upstream_cache_status always;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; min-height: 14px;"><span style="font-kerning: none"></span><br></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none"># tag::proxy_cache[]</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_cache content-proxy-7b49d8c897-62gk9;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_cache_background_update on;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_cache_bypass $arg_nocache;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_cache_lock on;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_cache_use_stale updating error timeout invalid_header http_500 http_502 http_503 http_504;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_cache_valid 404 5m;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none"># end::proxy_cache[]</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; min-height: 14px;"><span style="font-kerning: none"></span><br></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none"># tag::upstream[]</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none"># end::upstream[]</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; min-height: 14px;"><span style="font-kerning: none"></span><br></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none"># This header will be used to indicate a request coming from nginx</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none"># With this header set we can return 503 only to nginx when a server is in maintenance mode</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_set_header X-Content-Cache $hostname;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; min-height: 14px;"><span style="font-kerning: none"></span><br></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_ssl_server_name on;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal; min-height: 14px;"><span style="font-kerning: none"></span><br></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_read_timeout 10;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_connect_timeout 10;</span></p>
<p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">proxy_send_timeout 10;</span></p><p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none"><br></span></p><p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">Greets,</span></p><p style="margin: 0px; font-style: normal; font-variant-caps: normal; font-stretch: normal; line-height: normal; font-size-adjust: none; font-kerning: auto; font-variant-alternates: normal; font-variant-ligatures: normal; font-variant-numeric: normal; font-variant-east-asian: normal; font-variant-position: normal; font-feature-settings: normal; font-optical-sizing: auto; font-variation-settings: normal;"><span style="font-kerning: none">Roy</span></p></div><div><div><br><blockquote type="cite"><div>On 23 Mar 2023, at 18:44, Maxim Dounin <mdounin@mdounin.ru> wrote:</div><br class="Apple-interchange-newline"><div><div>Hello!<br><br>On Thu, Mar 23, 2023 at 09:26:48AM +0100, Roy Teeuwen wrote:<br><br><blockquote type="cite">We are using NGINX as a proxy / caching layer for a backend <br>application. Our backend has a relatively slow response time, <br>ranging between the 100 to 300ms. We want the NGINX proxy to be <br>as speedy as possible, to do this we have implemented the <br>following logic:<br><br>- Cache all responses for 5 mins (based on cache control <br>headers)<br>- Use stale cache for error's on the backend<br>- Do a background update for stale cache<br><br>The last part has an issue, namely if a first request reaches <br>nginx, it will trigger a background request, but other requests <br>for the same resource will be locked until this background <br>request is finished instead of still returning the stale cache <br>that is available. This is caused by the fact that there is a <br>keepalive on the connection, which locks all subsequent requests <br>until the background request is finished.<br></blockquote><br>Could you please clarify what you are trying to describe?<br><br>Keepalive on the connection might delay handling of subsequent <br>requests on the same connection (and not other requests to the <br>same resource).<br><br>Other requests to the same resource might be delayed by the <br>proxy_cache_lock (https://nginx.org/r/proxy_cache_lock), but it is <br>not something in your description, and it only works for new cache <br>elements and has no effect when there is a stale cache item. <br><br><blockquote type="cite">The issue that we are facing in this situation is that the <br>locking is very long, namely 500ms hardcoded. I think it is <br>caused by this:<br>https://github.com/nginx/nginx/blob/master/src/core/ngx_connection.c#L703<br></blockquote><br>This looks completely unrelated. A 500ms delay can be seen with <br>proxy_cache_lock as previously mentioned, see here:<br><br>http://hg.nginx.org/nginx/file/tip/src/http/ngx_http_file_cache.c#l455<br><br>But again, it is not expected to appear in the use case you <br>describe.<br><br><blockquote type="cite">This means that our relatively slow backend of 100-200ms <br>actually gets worse than better.<br><br><br>Is it an option to make this 500ms a configurable setting <br>instead of 500ms? Are there any downsides to making this 500ms <br>lower? I'd be willing to see if we can contribute this.<br><br><br>Another option that I'd tried is to set the keepalive to 0, so <br>that every request is a new connection. In small amounts of <br>requests this actually seemed to solve the issue, but the moment <br>that we went to a real life situation, this degraded the <br>performance massively, so we had to revert this<br></blockquote><br>First of all, it might be a good idea to better understand what <br>is the issue you are seeing.<br><br>Also make sure that you have "proxy_cache_use_stale updating" <br>enabled (https://nginx.org/r/proxy_cache_use_stale). It is <br>designed exactly for the use case you describe, and works quite <br>well in most use cases.<br><br>-- <br>Maxim Dounin<br>http://mdounin.ru/<br>_______________________________________________<br>nginx-devel mailing list<br>nginx-devel@nginx.org<br>https://mailman.nginx.org/mailman/listinfo/nginx-devel<br></div></div></blockquote></div><br></div></body></html>