[PATCH 0 of 2] KTLS / SSL_sendfile() support

[Maxim Dounin]

Hello!

On Tue, Oct 12, 2021 at 07:47:36PM +0300, Sergey Kandaurov wrote:

[...]

> > - What about SSL_sendfile(), early data and write blocking?
> >  Ref. c->ssl->write_blocked, 7431:294162223c7c by pluknet at .
> >  Looks like it is not a problem with SSL_sendfile(), but needs
> >  further checking.
> > 
> 
> On that particular one.
> 
> Indeed, it should not be an issue, since KTLS bypasses OpenSSL internals.

My concern here is alert dispatching and flushing part of the 
SSL_sendfile() function.  It still does a lot in various OpenSSL 
write code path, and I'm not sure it cannot trigger the same 
OpenSSL issue if blocking happens at wrong moment.

On the other hand, this is unlikely, and probably we can ignore 
this anyway.

> For the record, I've reproduced the original issue fixed in 294162223c7c.
> For example, it could be reading discarded body sent separately in 1-RTT.
> Even with the fix backed out, reading with blocked sendfile works fine.
> 
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL buf copy: 246
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL to write: 246
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL_write_early_data: 1, 246
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL to sendfile: @0 1048576
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL_sendfile: 45056
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL to sendfile: @45056 1003520
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL_sendfile: 40960
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL to sendfile: @86016 962560
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL_sendfile: 61440
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL to sendfile: @147456 901120
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL_sendfile: -1
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL_get_error: 3
> 2021/10/12 16:15:53 [debug] 38707#0: *2 http write filter 0000000802259660
> 2021/10/12 16:15:53 [debug] 38707#0: *2 http copy filter: -2 "/file?"
> 2021/10/12 16:15:53 [debug] 38707#0: *2 http finalize request: -2, "/file?" a:1,
>  c:2
> 2021/10/12 16:15:53 [debug] 38707#0: *2 event timer add: 13: 60000:707289850
> 2021/10/12 16:15:53 [debug] 38707#0: *2 kevent set event: 13: ft:-2 fl:0025
> 2021/10/12 16:15:53 [debug] 38707#0: timer delta: 1
> 2021/10/12 16:15:53 [debug] 38707#0: worker cycle
> 2021/10/12 16:15:53 [debug] 38707#0: kevent timer: 60000, changes: 1
> 2021/10/12 16:15:53 [debug] 38707#0: kevent events: 1
> 2021/10/12 16:15:53 [debug] 38707#0: kevent: 13: ft:-1 fl:0020 ff:00000000 d:138 ud:0000000802328841
> 2021/10/12 16:15:53 [debug] 38707#0: *2 http run request: "/file?"
> 2021/10/12 16:15:53 [debug] 38707#0: *2 http read discarded body
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL_read_early_data: 2, 0
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL_read: 10
> 2021/10/12 16:15:53 [debug] 38707#0: *2 SSL_read: avail:128
> 
> For comparison (and to make sure I'm testing it right),
> disabling sendfile on unfixed nginx would reintroduce an error:
> 
> 2021/10/12 16:33:41 [debug] 42445#0: *2 SSL_read_early_data: 2, 0
> 2021/10/12 16:33:41 [alert] 42445#0: *2 ignoring stale global SSL error (SSL: error:0A00010F:SSL routines::bad length) while sending response to client, client: 127.0.0.1, server: localhost, request: "GET /file HTTP/1.1", host: "localhost"
> 2021/10/12 16:33:41 [debug] 42445#0: *2 SSL_read: -1
> 2021/10/12 16:33:41 [debug] 42445#0: *2 SSL_get_error: 5

It would be great to make a test (may be disabled by default 
and/or with some comments on tuning needed to reproduce) for the 
original issue, to make sure we'll be able to check possible 
future OpenSSL fixes, if any.

-- 
Maxim Dounin
http://mdounin.ru/