<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-2022-jp">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"PingFang SC";
panose-1:2 11 4 0 0 0 0 0 0 0;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"\@PingFang SC";}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:10.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="en-CN" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">Thanks for reply.
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">I$B!G(Bm agree that a better framework is needed to implement different congestion algorithms.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">And the current implementation may have a little problem, for example it should be better for
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">passing acknowledge event (including the ack of multiple packets and the losts) instead of pass each
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">packet info to congestion control algorithms.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">As you suggested, $B!H(Bmoved rtt and congestion control variables to ngx_quic_path_t structure$B!I(B is separate patch now.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">And this patch is prerequisite for multipath quic.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""># HG changeset patch<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""># User Yu Zhu <lishu.zy@alibaba-inc.com><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""># Date 1670831223 -28800<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""># Mon Dec 12 15:47:03 2022 +0800<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""># Branch quic<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""># Node ID 8723d4282f6d6a5b67e271652f46d79ee24dfb39<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""># Parent b87a0dbc1150f415def5bc1e1f00d02b33519026<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">QUIC: moved rtt and congestion control variables to ngx_quic_path_t structure.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">As rfc9001, section 6. Loss Detection:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> Loss detection is separate per packet number space, unlike RTT measurement<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> and congestion control, because RTT and congestion control are properties<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> of the path, whereas loss detection also relies upon key availability.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">No functional changes.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">diff -r b87a0dbc1150 -r 8723d4282f6d src/event/quic/ngx_event_quic.c<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">--- a/src/event/quic/ngx_event_quic.c Tue Oct 25 12:52:09 2022 +0400<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+++ b/src/event/quic/ngx_event_quic.c Mon Dec 12 15:47:03 2022 +0800<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -263,15 +263,6 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_queue_init(&qc->free_frames);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->avg_rtt = NGX_QUIC_INITIAL_RTT;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->rttvar = NGX_QUIC_INITIAL_RTT / 2;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->min_rtt = NGX_TIMER_INFINITE;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->first_rtt = NGX_TIMER_INFINITE;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- /*<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- * qc->latest_rtt = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- */<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc->pto.log = c->log;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc->pto.data = c;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc->pto.handler = ngx_quic_pto_handler;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -311,12 +302,6 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc->streams.client_max_streams_uni = qc->tp.initial_max_streams_uni;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc->streams.client_max_streams_bidi = qc->tp.initial_max_streams_bidi;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->congestion.window = ngx_min(10 * qc->tp.max_udp_payload_size,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- ngx_max(2 * qc->tp.max_udp_payload_size,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- 14720));<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->congestion.ssthresh = (size_t) -1;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->congestion.recovery_start = ngx_current_msec;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> if (pkt->validated && pkt->retried) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc->tp.retry_scid.len = pkt->dcid.len;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc->tp.retry_scid.data = ngx_pstrdup(c->pool, &pkt->dcid);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">diff -r b87a0dbc1150 -r 8723d4282f6d src/event/quic/ngx_event_quic_ack.c<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">--- a/src/event/quic/ngx_event_quic_ack.c Tue Oct 25 12:52:09 2022 +0400<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+++ b/src/event/quic/ngx_event_quic_ack.c Mon Dec 12 15:47:03 2022 +0800<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -29,7 +29,7 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">} ngx_quic_ack_stat_t;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">-static ngx_inline ngx_msec_t ngx_quic_lost_threshold(ngx_quic_connection_t *qc);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+static ngx_inline ngx_msec_t ngx_quic_lost_threshold(ngx_quic_path_t *path);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">static void ngx_quic_rtt_sample(ngx_connection_t *c, ngx_quic_ack_frame_t *ack,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> enum ssl_encryption_level_t level, ngx_msec_t send_time);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">static ngx_int_t ngx_quic_handle_ack_frame_range(ngx_connection_t *c,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -48,11 +48,11 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> /* RFC 9002, 6.1.2. Time Threshold: kTimeThreshold, kGranularity */<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">static ngx_inline ngx_msec_t<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">-ngx_quic_lost_threshold(ngx_quic_connection_t *qc)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ngx_quic_lost_threshold(ngx_quic_path_t *path)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">{<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_msec_t thr;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- thr = ngx_max(qc->latest_rtt, qc->avg_rtt);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ thr = ngx_max(path->latest_rtt, path->avg_rtt);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> thr += thr >> 3;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> return ngx_max(thr, NGX_QUIC_TIME_GRANULARITY);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -179,21 +179,23 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> enum ssl_encryption_level_t level, ngx_msec_t send_time)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">{<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_msec_t latest_rtt, ack_delay, adjusted_rtt, rttvar_sample;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_quic_path_t *path;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_quic_connection_t *qc;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path = qc->path;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> latest_rtt = ngx_current_msec - send_time;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->latest_rtt = latest_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->latest_rtt = latest_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- if (qc->min_rtt == NGX_TIMER_INFINITE) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->min_rtt = latest_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->avg_rtt = latest_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->rttvar = latest_rtt / 2;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->first_rtt = ngx_current_msec;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ if (path->min_rtt == NGX_TIMER_INFINITE) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->min_rtt = latest_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->avg_rtt = latest_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->rttvar = latest_rtt / 2;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->first_rtt = ngx_current_msec;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> } else {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->min_rtt = ngx_min(qc->min_rtt, latest_rtt);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->min_rtt = ngx_min(path->min_rtt, latest_rtt);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ack_delay = (ack->delay << qc->ctp.ack_delay_exponent) / 1000;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -203,18 +205,18 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> adjusted_rtt = latest_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- if (qc->min_rtt + ack_delay < latest_rtt) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ if (path->min_rtt + ack_delay < latest_rtt) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> adjusted_rtt -= ack_delay;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->avg_rtt += (adjusted_rtt >> 3) - (qc->avg_rtt >> 3);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- rttvar_sample = ngx_abs((ngx_msec_int_t) (qc->avg_rtt - adjusted_rtt));<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->rttvar += (rttvar_sample >> 2) - (qc->rttvar >> 2);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->avg_rtt += (adjusted_rtt >> 3) - (path->avg_rtt >> 3);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ rttvar_sample = ngx_abs((ngx_msec_int_t) (path->avg_rtt - adjusted_rtt));<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->rttvar += (rttvar_sample >> 2) - (path->rttvar >> 2);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_log_debug4(NGX_LOG_DEBUG_EVENT, c->log, 0,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> "quic rtt sample latest:%M min:%M avg:%M var:%M",<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- latest_rtt, qc->min_rtt, qc->avg_rtt, qc->rttvar);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ latest_rtt, path->min_rtt, path->avg_rtt, path->rttvar);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">}<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -317,7 +319,7 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- cg = &qc->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ cg = &qc->path->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> blocked = (cg->in_flight >= cg->window) ? 1 : 0;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -428,13 +430,15 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_uint_t i, nlost;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_msec_t now, wait, thr, oldest, newest;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_queue_t *q;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_quic_path_t *path;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_quic_frame_t *start;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_quic_send_ctx_t *ctx;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_quic_connection_t *qc;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path = qc->path;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> now = ngx_current_msec;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- thr = ngx_quic_lost_threshold(qc);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ thr = ngx_quic_lost_threshold(path);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> /* send time of lost packets across all send contexts */<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> oldest = NGX_TIMER_INFINITE;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -471,7 +475,7 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> break;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- if (start->last > qc->first_rtt) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ if (start->last > path->first_rtt) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> if (oldest == NGX_TIMER_INFINITE || start->last < oldest) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> oldest = start->last;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -519,8 +523,8 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- duration = qc->avg_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- duration += ngx_max(4 * qc->rttvar, NGX_QUIC_TIME_GRANULARITY);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ duration = qc->path->avg_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ duration += ngx_max(4 * qc->path->rttvar, NGX_QUIC_TIME_GRANULARITY);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> duration += qc->ctp.max_ack_delay;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> duration *= NGX_QUIC_PERSISTENT_CONGESTION_THR;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -535,7 +539,7 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_quic_connection_t *qc;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- cg = &qc->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ cg = &qc->path->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> cg->recovery_start = ngx_current_msec;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> cg->window = qc->tp.max_udp_payload_size * 2;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -656,7 +660,7 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- cg = &qc->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ cg = &qc->path->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> blocked = (cg->in_flight >= cg->window) ? 1 : 0;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -721,7 +725,7 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> if (ctx->largest_ack != NGX_QUIC_UNSET_PN) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> q = ngx_queue_head(&ctx->sent);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> f = ngx_queue_data(q, ngx_quic_frame_t, queue);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- w = (ngx_msec_int_t) (f->last + ngx_quic_lost_threshold(qc) - now);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ w = (ngx_msec_int_t) (f->last + ngx_quic_lost_threshold(qc->path) - now);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> if (f->pnum <= ctx->largest_ack) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> if (w < 0 || ctx->largest_ack - f->pnum >= NGX_QUIC_PKT_THR) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -777,17 +781,19 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">ngx_quic_pto(ngx_connection_t *c, ngx_quic_send_ctx_t *ctx)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">{<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_msec_t duration;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_quic_path_t *path;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_quic_connection_t *qc;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path = qc->path;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> /* RFC 9002, Appendix A.8. Setting the Loss Detection Timer */<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- duration = qc->avg_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ duration = path->avg_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- duration += ngx_max(4 * qc->rttvar, NGX_QUIC_TIME_GRANULARITY);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ duration += ngx_max(4 * path->rttvar, NGX_QUIC_TIME_GRANULARITY);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> duration <<= qc->pto_count;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- if (qc->congestion.in_flight == 0) { /* no in-flight packets */<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ if (path->congestion.in_flight == 0) { /* no in-flight packets */<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> return duration;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">diff -r b87a0dbc1150 -r 8723d4282f6d src/event/quic/ngx_event_quic_connection.h<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">--- a/src/event/quic/ngx_event_quic_connection.h Tue Oct 25 12:52:09 2022 +0400<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+++ b/src/event/quic/ngx_event_quic_connection.h Mon Dec 12 15:47:03 2022 +0800<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -80,6 +80,14 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">};<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+typedef struct {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ size_t in_flight;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ size_t window;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ size_t ssthresh;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_msec_t recovery_start;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+} ngx_quic_congestion_t;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">struct ngx_quic_path_s {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_queue_t queue;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> struct sockaddr *sockaddr;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -96,6 +104,15 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> uint64_t seqnum;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_str_t addr_text;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> u_char text[NGX_SOCKADDR_STRLEN];<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_msec_t first_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_msec_t latest_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_msec_t avg_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_msec_t min_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_msec_t rttvar;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_quic_congestion_t congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> unsigned validated:1;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> unsigned validating:1;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> unsigned limited:1;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -143,14 +160,6 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">} ngx_quic_streams_t;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> <o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">-typedef struct {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- size_t in_flight;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- size_t window;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- size_t ssthresh;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- ngx_msec_t recovery_start;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">-} ngx_quic_congestion_t;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">/*<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> * RFC 9000, 12.3. Packet Numbers<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> *<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -218,12 +227,6 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_event_t path_validation;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_msec_t last_cc;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- ngx_msec_t first_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- ngx_msec_t latest_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- ngx_msec_t avg_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- ngx_msec_t min_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- ngx_msec_t rttvar;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">-<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_uint_t pto_count;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_queue_t free_frames;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -237,7 +240,6 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">#endif<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_quic_streams_t streams;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- ngx_quic_congestion_t congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> off_t received;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">diff -r b87a0dbc1150 -r 8723d4282f6d src/event/quic/ngx_event_quic_migration.c<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">--- a/src/event/quic/ngx_event_quic_migration.c Tue Oct 25 12:52:09 2022 +0400<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+++ b/src/event/quic/ngx_event_quic_migration.c Mon Dec 12 15:47:03 2022 +0800<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -135,17 +135,26 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> /* address did not change */<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> rst = 0;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->avg_rtt = prev->avg_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->rttvar = prev->rttvar;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->min_rtt = prev->min_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->first_rtt = prev->first_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->latest_rtt = prev->latest_rtt;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_memcpy(&path->congestion, &prev->congestion,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ sizeof(ngx_quic_congestion_t));<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> if (rst) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- ngx_memzero(&qc->congestion, sizeof(ngx_quic_congestion_t));<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_memzero(&path->congestion, sizeof(ngx_quic_congestion_t));<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->congestion.window = ngx_min(10 * qc->tp.max_udp_payload_size,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->congestion.window = ngx_min(10 * qc->tp.max_udp_payload_size,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_max(2 * qc->tp.max_udp_payload_size,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> 14720));<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->congestion.ssthresh = (size_t) -1;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- qc->congestion.recovery_start = ngx_current_msec;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->congestion.ssthresh = (size_t) -1;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->congestion.recovery_start = ngx_current_msec;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> }<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> /*<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -217,6 +226,21 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> path->addr_text.len = ngx_sock_ntop(sockaddr, socklen, path->text,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> NGX_SOCKADDR_STRLEN, 1);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->avg_rtt = NGX_QUIC_INITIAL_RTT;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->rttvar = NGX_QUIC_INITIAL_RTT / 2;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->min_rtt = NGX_TIMER_INFINITE;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->first_rtt = NGX_TIMER_INFINITE;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ /*<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ * qc->latest_rtt = 0<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ */<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->congestion.window = ngx_min(10 * qc->tp.max_udp_payload_size,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ ngx_max(2 * qc->tp.max_udp_payload_size,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ 14720));<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->congestion.ssthresh = (size_t) -1;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ path->congestion.recovery_start = ngx_current_msec;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ngx_log_debug2(NGX_LOG_DEBUG_EVENT, c->log, 0,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> "quic path seq:%uL created addr:%V",<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> path->seqnum, &path->addr_text);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">diff -r b87a0dbc1150 -r 8723d4282f6d src/event/quic/ngx_event_quic_output.c<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">--- a/src/event/quic/ngx_event_quic_output.c Tue Oct 25 12:52:09 2022 +0400<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+++ b/src/event/quic/ngx_event_quic_output.c Mon Dec 12 15:47:03 2022 +0800<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -87,7 +87,7 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> c->log->action = "sending frames";<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- cg = &qc->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ cg = &qc->path->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> in_flight = cg->in_flight;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -135,8 +135,8 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> static u_char dst[NGX_QUIC_MAX_UDP_PAYLOAD_SIZE];<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- cg = &qc->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> path = qc->path;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ cg = &path->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> while (cg->in_flight < cg->window) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -223,7 +223,7 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- cg = &qc->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ cg = &qc->path->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> while (!ngx_queue_empty(&ctx->sending)) {<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">@@ -336,8 +336,8 @@<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> static u_char dst[NGX_QUIC_MAX_UDP_SEGMENT_BUF];<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> qc = ngx_quic_get_connection(c);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">- cg = &qc->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> path = qc->path;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">+ cg = &path->congestion;<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""> ctx = ngx_quic_get_send_ctx(qc, ssl_encryption_application);<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">--<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New"">Yu Zhu<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">Roman Arutyunyan <arut@nginx.com><br>
<b>Date: </b>Wednesday, December 7, 2022 at 23:05<br>
<b>To: </b>nginx-devel@nginx.org <nginx-devel@nginx.org><br>
<b>Subject: </b>Re: QUIC: reworked congestion control mechanism.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt">Hi,<br>
<br>
Thanks for the path.<br>
<br>
On Tue, Dec 06, 2022 at 02:35:37PM +0000, </span><span lang="ZH-CN" style="font-size:11.0pt;font-family:"PingFang SC",sans-serif">$B<k1'(B</span><span style="font-size:11.0pt"> wrote:<br>
> Hi,<br>
> <br>
> # HG changeset patch<br>
> # User Yu Zhu <lishu.zy@alibaba-inc.com><br>
> # Date 1670326031 -28800<br>
> # Tue Dec 06 19:27:11 2022 +0800<br>
> # Branch quic<br>
> # Node ID 9a47ff1223bb32c8ddb146d731b395af89769a97<br>
> # Parent 1a320805265db14904ca9deaae8330f4979619ce<br>
> QUIC: reworked congestion control mechanism.<br>
> <br>
> 1. move rtt measurement and congestion control to struct ngx_quic_path_t<br>
> because RTT and congestion control are properities of the path.<br>
<br>
I think this part should be moved out to a separate patch. <br>
<br>
> 2. introduced struct "ngx_quic_congestion_ops_t" to wrap callback functions<br>
> of congestion control and extract the reno algorithm from ngx_event_quic_ack.c.<br>
<br>
The biggest question about this part is how extensible is this approach? <br>
We are planning to implement more congestion control algorithms in the future <br>
and need a framework that would allow us to do that. <br>
<br>
Even CUBIC needs more data fields that we have now, and BBR will prooably <br>
need much more than that. Not sure how we'll add those data fields considering <br>
the proposed modular design. Also, we need to make sure the API is enough for <br>
future algorithms. <br>
<br>
I suggest that we finish the first part which moves congestion control <br>
to the path object. Then, until we have at least one other congestion <br>
control algorithm supported, it's hard to come up with a good API for it. <br>
I this we can postpone the second part until then. <br>
<br>
Also, I think CUBIC can be hardcoded into Reno without modular redesign of the <br>
code. <br>
<br>
> No functional changes.<br>
<br>
[..]<br>
<br>
> diff -r 1a320805265d -r 9a47ff1223bb src/event/quic/congestion/ngx_quic_reno.c<br>
> --- /dev/null Thu Jan 01 00:00:00 1970 +0000<br>
> +++ b/src/event/quic/congestion/ngx_quic_reno.c Tue Dec 06 19:27:11 2022 +0800<br>
> @@ -0,0 +1,133 @@<br>
> +<br>
> +/*<br>
> + * Copyright (C) Nginx, Inc.<br>
> + */<br>
> +<br>
> +<br>
> +#include <ngx_config.h><br>
> +#include <ngx_core.h><br>
> +#include <ngx_event.h><br>
> +#include <ngx_event_quic_connection.h><br>
> +<br>
> +<br>
> +static void ngx_quic_reno_on_init(ngx_connection_t *c, ngx_quic_congestion_t *cg);<br>
> +static ngx_int_t ngx_quic_reno_on_ack(ngx_connection_t *c, ngx_quic_frame_t *f);<br>
> +static ngx_int_t ngx_quic_reno_on_lost(ngx_connection_t *c, ngx_quic_frame_t *f);<br>
> +<br>
> +<br>
> +ngx_quic_congestion_ops_t ngx_quic_reno = {<br>
> + ngx_string("reno"),<br>
> + ngx_quic_reno_on_init,<br>
> + ngx_quic_reno_on_ack,<br>
> + ngx_quic_reno_on_lost<br>
> +};<br>
> +<br>
> +<br>
> +static void<br>
> +ngx_quic_reno_on_init(ngx_connection_t *c, ngx_quic_congestion_t *cg)<br>
> +{<br>
> + ngx_quic_connection_t *qc;<br>
> +<br>
> + qc = ngx_quic_get_connection(c);<br>
> +<br>
> + cg->window = ngx_min(10 * qc->tp.max_udp_payload_size,<br>
> + ngx_max(2 * qc->tp.max_udp_payload_size,<br>
> + 14720));<br>
> + cg->ssthresh = (size_t) -1;<br>
> + cg->recovery_start = ngx_current_msec;<br>
> +}<br>
> +<br>
> +<br>
> +static ngx_int_t<br>
> +ngx_quic_reno_on_ack(ngx_connection_t *c, ngx_quic_frame_t *f)<br>
> +{<br>
> + ngx_msec_t timer;<br>
> + ngx_quic_path_t *path;<br>
> + ngx_quic_connection_t *qc;<br>
> + ngx_quic_congestion_t *cg;<br>
> +<br>
> + qc = ngx_quic_get_connection(c);<br>
> + path = qc->path;<br>
<br>
What if the packet was sent on a different path?<br>
<br>
> +<br>
> + cg = &path->congestion;<br>
> +<br>
> + cg->in_flight -= f->plen;<br>
> +<br>
> + timer = f->last - cg->recovery_start;<br>
> +<br>
> + if ((ngx_msec_int_t) timer <= 0) {<br>
> + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,<br>
> + "quic congestion ack recovery win:%uz ss:%z if:%uz",<br>
> + cg->window, cg->ssthresh, cg->in_flight);<br>
> +<br>
> + return NGX_DONE;<br>
> + }<br>
> +<br>
> + if (cg->window < cg->ssthresh) {<br>
> + cg->window += f->plen;<br>
> +<br>
> + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,<br>
> + "quic congestion slow start win:%uz ss:%z if:%uz",<br>
> + cg->window, cg->ssthresh, cg->in_flight);<br>
> +<br>
> + } else {<br>
> + cg->window += qc->tp.max_udp_payload_size * f->plen / cg->window;<br>
> +<br>
> + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,<br>
> + "quic congestion avoidance win:%uz ss:%z if:%uz",<br>
> + cg->window, cg->ssthresh, cg->in_flight);<br>
> + }<br>
> +<br>
> + /* prevent recovery_start from wrapping */<br>
> +<br>
> + timer = cg->recovery_start - ngx_current_msec + qc->tp.max_idle_timeout * 2;<br>
> +<br>
> + if ((ngx_msec_int_t) timer < 0) {<br>
> + cg->recovery_start = ngx_current_msec - qc->tp.max_idle_timeout * 2;<br>
> + }<br>
> +<br>
> + return NGX_OK;<br>
> +}<br>
> +<br>
> +<br>
> +static ngx_int_t<br>
> +ngx_quic_reno_on_lost(ngx_connection_t *c, ngx_quic_frame_t *f)<br>
> +{<br>
> + ngx_msec_t timer;<br>
> + ngx_quic_path_t *path;<br>
> + ngx_quic_connection_t *qc;<br>
> + ngx_quic_congestion_t *cg;<br>
> +<br>
> + qc = ngx_quic_get_connection(c);<br>
> + path = qc->path;<br>
<br>
Same here.<br>
<br>
> +<br>
> + cg = &path->congestion;<br>
> +<br>
> + cg->in_flight -= f->plen;<br>
> + f->plen = 0;<br>
> +<br>
> + timer = f->last - cg->recovery_start;<br>
> +<br>
> + if ((ngx_msec_int_t) timer <= 0) {<br>
> + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,<br>
> + "quic congestion lost recovery win:%uz ss:%z if:%uz",<br>
> + cg->window, cg->ssthresh, cg->in_flight);<br>
> +<br>
> + return NGX_DONE;<br>
> + }<br>
> +<br>
> + cg->recovery_start = ngx_current_msec;<br>
> + cg->window /= 2;<br>
> +<br>
> + if (cg->window < qc->tp.max_udp_payload_size * 2) {<br>
> + cg->window = qc->tp.max_udp_payload_size * 2;<br>
> + }<br>
> +<br>
> + cg->ssthresh = cg->window;<br>
> +<br>
> + ngx_log_debug3(NGX_LOG_DEBUG_EVENT, c->log, 0,<br>
> + "quic congestion lost win:%uz ss:%z if:%uz",<br>
> + cg->window, cg->ssthresh, cg->in_flight);<br>
> +<br>
> + return NGX_OK;<br>
> +}<br>
<br>
[..]<br>
<br>
--<br>
Roman Arutyunyan<br>
_______________________________________________<br>
nginx-devel mailing list -- nginx-devel@nginx.org<br>
To unsubscribe send an email to nginx-devel-leave@nginx.org<o:p></o:p></span></p>
</div>
</div>
</body>
</html>