SSL_shutdown() failed (SSL: ... bad write retry)

[vergil]

Maxim Dounin Wrote:
-------------------------------------------------------
> Hello!
> 
> On Thu, Aug 13, 2020 at 12:11:54PM -0400, vergil wrote:
> 
> > Maxim Dounin Wrote:
> > -------------------------------------------------------
> > > Hello!
> > > 
> > > On Thu, Aug 13, 2020 at 11:39:36AM -0400, vergil wrote:
> > > 
> > > > This one was hard to catch.
> > > > 
> > > > I've captured one error with 30 seconds delta before and after
> the
> > > event.
> > > > Where can i attach log file for you? There's 400K messages, so i
> > > cannot
> > > > simple put it here.
> > > 
> > > Attaching the log to the message into the mailing list should 
> > > work, but I'm not sure it's supported by the (obsolete) forum 
> > > interface you are using.  If not, you may put the log at a 
> > > convinient place and provide a link here, or attach it to a 
> > > ticket on trac.nginx.org, or email to me privetely.
> > 
> > I've attached log file to our S3 public storage. You can download it
> through
> > this link:
> >
> https://drive-public-eu.s3.eu-central-1.amazonaws.com/nginx/nginx-debu
> g.csv
> > 
> > A note: this is a CSV format from our logging system. I can try to
> extract
> > logs in original format if you need.
> 
> Thanks, but this doesn't seem to contain anything related to the 
> SSL_shutdown() except the message itself:
> 
> "2020-08-13T15:19:03.279Z","7","shmtx lock",
> "2020-08-13T15:19:03.279Z","7","shmtx lock",
> "2020-08-13T15:19:03.279Z","7","timer delta: 0",
> "2020-08-13T15:19:03.280Z","2","SSL_shutdown() failed (SSL:
> error:1409F07F:SSL routines:ssl3_write_pending:bad write retry) while
> closing request, client: XXX.XXX.XXX.XXX, server:
> XXX.XXX.XXX.XXX:443","9140"
> "2020-08-13T15:19:03.280Z","7","epoll: fd:322 ev:0005
> d:00007F0A0FCDDEB0",
> "2020-08-13T15:19:03.280Z","7","epoll: fd:54 ev:0004
> d:00007F0A0FCDFAC9",
> 
> And nothing else in the log saying "SSL_shutdow()", while there 
> should be a debug messages like "SSL_shutdown: -1" and 
> "SSL_get_error: ..." right before the message, and nothing at all 
> related to the connection 9140.
> 
> It looks like the debug logging is only enabled on the global 
> level, but disabled at http or server level.  Please see the part 
> starting at "Note that redefining the log without also specifying 
> the debug level will disable the debugging log" in the "A 
> debugging log" article 
> (http://nginx.org/en/docs/debugging_log.html).
> 
> -- 
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

So... Bad news: i cannot capture the event when full debug enabled. Server
cannot handle the load and our service partially down at that time.

What can i say is that this problem reveal itself on all servers with new
nginx version.

I'll send you the link privately where you can get our (some-redacted)
config files.

Regards,
Alexander.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,289087,289112#msg-289112