SSL_shutdown() failed (SSL: ... bad write retry)

[Maxim Dounin]

Hello!

On Thu, Aug 20, 2020 at 09:30:37AM -0400, vergil wrote:

> Maxim Dounin Wrote:
> -------------------------------------------------------
> > Do you see any other errors on the same connection before the 
> > SSL_shutdown() error?  As suggested previously, somethig relevant 
> > might be logged at the "info" level.  Note that seeing info-level 
> > error messages will probably require error logging to be 
> > reconfigured, much like with debug.
> > 
> > If there is nothing, I'm afraid the only solution would be to try 
> > to catch a debugging log related to these errors.  As previously 
> > suggested, this can be done without too much load by using the 
> > debug_connection with relatively large CIDR blocks and waiting for 
> > the error to happen from with a client from one of these blocks.
> > 
> > -- 
> > Maxim Dounin
> > http://mdounin.ru/
> > _______________________________________________
> > nginx mailing list
> > nginx at nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx
> 
> 
> Good day.
> 
> I've change log level from notice to info and there's indeed one new message
> related to HTTP/2 problem.
> 
> 2020/08/20 15:59:31 [info] 32305#32305: *1982005 client timed out (110:
> Connection timed out) while processing HTTP/2 connection, client: XXX,
> server: XXX:443
> 2020/08/20 15:59:36 [crit] 32305#32305: *1982005 SSL_shutdown() failed (SSL:
> error:1409F07F:SSL routines:ssl3_write retry) while processing HTTP/2
> connection, client: XXX, server: XXX:443
> 
> I don't know if this will help.

Thanks, I think I have an idea about what's going on here.  Likely 
these are read timeouts, which can interfere with writes in 
HTTP/2, causing SSL_shutdown() errors.  Please try the following 
patch:

# HG changeset patch
# User Maxim Dounin <mdounin at mdounin.ru>
# Date 1597950898 -10800
#      Thu Aug 20 22:14:58 2020 +0300
# Node ID f95e76e9144773a664271c3e91e4cb6df3bc774a
# Parent  7015f26aef904e2ec17b4b6f6387fd3b8298f79d
HTTP/2: connections with read timeouts marked as timed out.

In HTTP/2, closing a connection because of a read timeout might happen
when there are unfinished writes, resulting in SSL_shutdown() errors.
Fix is to mark such connections with the c->timedout flag to avoid sending
SSL shutdown.

diff --git a/src/http/v2/ngx_http_v2.c b/src/http/v2/ngx_http_v2.c
--- a/src/http/v2/ngx_http_v2.c
+++ b/src/http/v2/ngx_http_v2.c
@@ -346,6 +346,7 @@ ngx_http_v2_read_handler(ngx_event_t *re
 
     if (rev->timedout) {
         ngx_log_error(NGX_LOG_INFO, c->log, NGX_ETIMEDOUT, "client timed out");
+        c->timedout = 1;
         ngx_http_v2_finalize_connection(h2c, NGX_HTTP_V2_PROTOCOL_ERROR);
         return;
     }


-- 
Maxim Dounin
http://mdounin.ru/