<div dir="ltr">Hi Maxim,<div><br></div><div>Sorry for my late reply.</div><div><br></div><div>We use Nginx as CDN Cache. And it is normal case that the responses are </div><div>different from different origin servers. For instance: the origin server could be </div><div>the customer server or some other CDN provider Cache server. The response </div><div>header could be different.</div><div><br></div><div>In race condition, I found the inconsistency between disk file and metadata in</div><div>shared memory. </div><div>When (c->uniq != of.uniq) happens, I saw " too long header" in error log.</div><div><br></div><div>To fix this, I picked the ngx_pagesize to initiate c->body_start again. then the "too long header" </div><div>is gone from error log.</div><div><br></div><div>I will try your patches next week. Thanks!</div>
<div><br></div><br><div class="gmail_quote"><div dir="ltr">On Wed, Nov 28, 2018 at 4:45 AM Maxim Dounin <<a href="mailto:mdounin@mdounin.ru" target="_blank">mdounin@mdounin.ru</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Hello!<br>
<br>
On Mon, Nov 26, 2018 at 06:11:33PM -0800, Yongguang Bai wrote:<br>
<br>
> # HG changeset patch<br>
> # User Yongguang Bai <<a href="mailto:yongguang.bai@quantil.com" target="_blank">yongguang.bai@quantil.com</a>><br>
> # Date 1543276278 28800<br>
> # Mon Nov 26 15:51:18 2018 -0800<br>
> # Node ID 4d48e1f3ebf6f0634e8e2cbd057d9fcfbea9d988<br>
> # Parent a7ff19afbb14795fef14f599a304d0ad21052b70<br>
> Mail: nginx_error:cache file has too long header<br>
> <br>
> This error is printed when cached file is expired and the<br>
> response is changed in proxied server, and Nginx is under<br>
> heavy load.<br>
> <br>
> diff -r a7ff19afbb14 -r 4d48e1f3ebf6 src/http/ngx_http_file_cache.c<br>
> --- a/src/http/ngx_http_file_cache.c Mon Nov 26 18:29:56 2018 +0300<br>
> +++ b/src/http/ngx_http_file_cache.c Mon Nov 26 15:51:18 2018 -0800<br>
> @@ -376,6 +376,10 @@<br>
> ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0,<br>
> "http file cache fd: %d", of.fd);<br>
> <br>
> + if(c->uniq != of.uniq){<br>
> + c->body_start = ngx_pagesize;<br>
> + }<br>
> +<br>
> c->file.fd = of.fd;<br>
> c->file.log = r->connection->log;<br>
> c->uniq = of.uniq;<br>
<br>
This change looks wrong. In particular, ngx_pagesize, which is <br>
usually 4k, can be smaller than existing c->body_start, and this <br>
change will make things worse instead of fixing anything.<br>
<br>
Also, could you please elaborate what exactly you are trying to <br>
fix?<br>
<br>
There is a known race which may lead to "cache file has too long <br>
header" error - and, actually the error was added to ensure that <br>
this race is properly handled if happens, see here:<br>
<br>
<a href="http://hg.nginx.org/nginx/rev/6f97afc238de" rel="noreferrer" target="_blank">http://hg.nginx.org/nginx/rev/6f97afc238de</a><br>
<a href="http://mailman.nginx.org/pipermail/nginx-devel/2011-September/001287.html" rel="noreferrer" target="_blank">http://mailman.nginx.org/pipermail/nginx-devel/2011-September/001287.html</a><br>
<br>
Note that this race can only result in problems if for some reason <br>
the backend returns responses with different headers nearly at the <br>
same time. This may indicate that there is something wrong with <br>
the backend.<br>
<br>
Also, there is a known bug with multiple variants which also can <br>
lead to the same error. The patch can be found here:<br>
<br>
<a href="http://mailman.nginx.org/pipermail/nginx-devel/2018-January/010774.html" rel="noreferrer" target="_blank">http://mailman.nginx.org/pipermail/nginx-devel/2018-January/010774.html</a><br>
<br>
If you are hitting this bug, please test if the patch works for <br>
you.<br>
<br>
-- <br>
Maxim Dounin<br>
<a href="http://mdounin.ru/" rel="noreferrer" target="_blank">http://mdounin.ru/</a><br>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_5722740231418330061gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><table border="0" cellpadding="0" cellspacing="0" width="100%" style="font-family:Arial,Tahoma,sans-serif;color:rgb(102,102,102)"><tbody><tr><td colspan="2"><p style="font-size:12px;margin:10px 0px;padding:0px">Best Regards,</p><p style="margin:10px 0px;padding:0px"><font size="3"><b>Yongguang Bai</b></font></p><p style="font-size:12px;margin:10px 0px;padding:0px">Engineering Manager</p></td></tr><tr><td height="0"><img src="https://www.quantil.com/quantil-emailfooter-logo.gif" alt="Quantil"><p style="font-size:11px;color:rgb(212,32,38);margin:0px;padding:0px">Connecting users with content...it's that simple.</p></td></tr><tr><td style="font-size:12px"><p style="margin:10px 0px 0px;padding:0px">Direct: +1-408-421-6265<br>Office: +1-888-847-9851<br> </p></td></tr><tr><td><a href="https://twitter.com/Team_Quantil" target="_blank"><img width="20" height="20" src="https://www.quantil.com/outline-color-twitter-96.png" alt="Tweeter" style="width:20px;height:20px;margin-left:0px"></a> <a href="https://plus.google.com/+Quantil_team/" target="_blank"><img width="20" height="20" src="https://www.quantil.com/outline-color-googleplus-96.png" alt="Google Plus" style="width:20px;height:20px;margin-left:10px"></a> <a href="https://www.linkedin.com/company/quantil" target="_blank"><img width="20" height="20" src="https://www.quantil.com/outline-color-linkedin-96.png" alt="Linked In" style="width:20px;height:20px;margin-left:10px"></a> </td></tr><tr></tr><tr><td style="max-width:125px"><br></td></tr></tbody></table></div></div></div></div></div></div></div>