Running nginx-quic in a real environment

[Maxim Dounin]

Hello!

On Thu, Sep 29, 2022 at 05:53:52PM +0400, Sergey Kandaurov wrote:

> 
> > On 28 Sep 2022, at 01:04, Maxim Dounin <mdounin at mdounin.ru> wrote:
> > 
> > Hello!
> > 
> > On Tue, Sep 27, 2022 at 04:05:54PM +0400, Sergey Kandaurov wrote:
> > 
> >>> On 27 Sep 2022, at 14:11, João Sousa Andrade <joao.andrade at protocol.ai> wrote:
> >>> 
> >>> Thank you for the clarification Sergey. 
> >>> 
> >>> We have been running http3 in production for the past couple of weeks. There's something we have noticed which I'm not entirely sure as to what is causing it.
> >>> 
> >>> We have been getting lots of errors of the form: "[error] 34#34: *338736 quic no available client ids for new path while handling decrypted packet, client: $IP, server: 0.0.0.0:443". I tried looking through the code to little avail. I'm wondering what's causing these errors. Is it something which could be tweaked through configuration?
> >> 
> >> This happens when QUIC connection continues over a new network path,
> >> known as QUIC connection migration, which should by done by switching
> >> to a new connection ID, but the client didn't previously supply enough
> >> Connection ID to chose from.  Normally both peers maintain a set of
> >> unused Connection IDs, which may be needed not only for migration,
> >> but also due to NAT rebinding.  Assuming that peers that initiate
> >> connection migration maintain enough connection IDs, a likely reason
> >> of the network path change failure as seen in the above error can be
> >> due to NAT rebinding with the client implementation that doesn't a notion
> >> of connection migration so it didn't sent NEW_CONNECTION_ID frames.
> >> In the case of NAT rebinding the server should send probing frames
> >> over a new network path using next available client Connection ID,
> >> but there were no any, as seen in the above error.
> >> Hope that helps.
> > 
> 
> At least Firefox doesn't send NEW_CONNECTION_ID, which normally
> happens after handshake completion, and, as additionally received
> in another, private report, this leads to such errors behind NAT.
> 
> > Shouldn't it be at the "info" level, much like other 
> > client-related errors?
> > 
> 
> So indeed it may have sense to change logging level.
> 
> # HG changeset patch
> # User Sergey Kandaurov <pluknet at nginx.com>
> # Date 1664459599 -14400
> #      Thu Sep 29 17:53:19 2022 +0400
> # Branch quic
> # Node ID 7d78208f141b382a55bea3f7c1e66471b0c53937
> # Parent  a931e690475ee59387af517de60845b4b4307d28
> QUIC: "info" logging level on insufficient client connection ids.
> 
> Apparently, this error is reported on NAT rebinding if client didn't
> previously send NEW_CONNECTION_ID to supply additional connection ids.
> 
> diff --git a/src/event/quic/ngx_event_quic_migration.c b/src/event/quic/ngx_event_quic_migration.c
> --- a/src/event/quic/ngx_event_quic_migration.c
> +++ b/src/event/quic/ngx_event_quic_migration.c
> @@ -309,7 +309,7 @@ ngx_quic_set_path(ngx_connection_t *c, n
>      /* new path requires new client id */
>      cid = ngx_quic_next_client_id(c);
>      if (cid == NULL) {
> -        ngx_log_error(NGX_LOG_ERR, c->log, 0,
> +        ngx_log_error(NGX_LOG_INFO, c->log, 0,
>                        "quic no available client ids for new path");
>          /* stop processing of this datagram */
>          return NGX_DONE;

Looks good.

-- 
Maxim Dounin
http://mdounin.ru/