[RFC] event/openssl: Add dynamic record size support for serving ssl trafic

chen gzchenym at 126.com
Wed May 13 05:11:30 UTC 2015


the constant value here is optimal for most cases, like another patch that optimizing ssl initial write buffer size for large certificate, I think having a knob at conf file did not make any difference.





On 2015-05-13 12:57 , SplitIce Wrote:


Good Job.


Perhaps rather than changing the constants, they could be exposed as configuration options?


On Wed, May 13, 2015 at 12:28 PM, chen <gzchenym at 126.com> wrote:

1) we will have that fixed
2) no api is exposed by openssl that we can use to trigger a FLUSH, use SSL_write is what we can do. If we inspect the data using wireshark, you will find out that one SSL_write we result in one ssl record.
3) there are some old linux box that are still using IW4,


To Q2 specifically, BIO_flush we disrupt the internal state of ssl layer? And it will be better if we let ssl layer itself handle the bio stuff. 







At 2015-05-05 21:39:40, "chen" <gzchenym at 126.com> wrote:

Hi list:
This is v1 of the patchset the implementing the feature SSL Dynamic Record Sizing, inspiring by Google Front End (https://www.igvita.com/2013/10/24/optimizing-tls-record-size-and-buffering-latency/) .
There are 3 conditions, if true at the same time, may trigger SSL_write to send small record over the link, hard coded 1400 bytes at this time to keep it fit into MTU size. We just send out 3 of this small record at most to reduce framing overhead when serving large object, that is enough for browser to discovery other dependency of the page at top of html file. If the buffer chain is smaller than 4096 bytes, it will not justify the overhead of sending small record. After idle for 60s(hard coded at this moment), start all over again.


Any comments is welcome.


Regard
YM


hg export tip
# HG changeset patch
# User YM Chen <gzchenym at 126.com>
# Date 1430828974 -28800
# Node ID 31bfe6403c340bdc4c04e8e87721736c07bceef8
# Parent  162b2d27d4e1ce45bb9217d6958348c64f726a28
[RFC] event/openssl: Add dynamic record size support for serving ssl trafic

SSL Dynamic Record Sizing is a long sought after feature for website that serving
huge amount of encrypted traffic. The rational behide this is that SSL record should
not overflow the congestion window at the beginning of slow-start period and by doing
so, we can let the browser decode the first ssl record within 1 rtt and establish other
connections to fetch other resources that are referenced at the top of the html file.

diff -r 162b2d27d4e1 -r 31bfe6403c34 src/event/ngx_event_openssl.c
--- a/src/event/ngx_event_openssl.c     Wed Apr 29 14:59:02 2015 +0300
+++ b/src/event/ngx_event_openssl.c     Tue May 05 20:29:34 2015 +0800
@@ -1508,6 +1508,11 @@
     ngx_uint_t   flush;
     ssize_t      send, size;
     ngx_buf_t   *buf;
+    ngx_msec_t   last_sent_timer_diff;
+    ngx_uint_t   loop_count;
+
+    last_sent_timer_diff = ngx_current_msec - c->ssl->last_write_msec;
+    loop_count = 0;
 
     if (!c->ssl->buffer) {
 
@@ -1517,7 +1522,13 @@
                 continue;
             }
 
-            n = ngx_ssl_write(c, in->buf->pos, in->buf->last - in->buf->pos);
+            size = in->buf->last - in->buf->pos;
+
+            if(last_sent_timer_diff > 1000*60 && loop_count < 3 && size > 4096) {
+                size = 1400;
+            }
+
+            n = ngx_ssl_write(c, in->buf->pos, size);
 
             if (n == NGX_ERROR) {
                 return NGX_CHAIN_ERROR;
@@ -1532,8 +1543,11 @@
             if (in->buf->pos == in->buf->last) {
                 in = in->next;
             }
+
+            loop_count ++;
         }
 
+        c->ssl->last_write_msec = ngx_current_msec;
         return in;
     }
 
@@ -1614,9 +1628,14 @@
         if (size == 0) {
             buf->flush = 0;
             c->buffered &= ~NGX_SSL_BUFFERED;
+            c->ssl->last_write_msec = ngx_current_msec;
             return in;
         }
 
+        if(last_sent_timer_diff > 1000*60 && loop_count < 3 && size > 4096) {
+            size = 1400;
+        }
+
         n = ngx_ssl_write(c, buf->pos, size);
 
         if (n == NGX_ERROR) {
@@ -1633,14 +1652,18 @@
             break;
         }
 
-        flush = 0;
-
-        buf->pos = buf->start;
-        buf->last = buf->start;
+        if(buf->last == buf->pos) {
+            flush = 0;
+
+            buf->pos = buf->start;
+            buf->last = buf->start;
+        }
 
         if (in == NULL || send == limit) {
             break;
         }
+
+        loop_count++;
     }
 
     buf->flush = flush;
@@ -1652,6 +1675,7 @@
         c->buffered &= ~NGX_SSL_BUFFERED;
     }
 
+    c->ssl->last_write_msec = ngx_current_msec;
     return in;
 }
 
diff -r 162b2d27d4e1 -r 31bfe6403c34 src/event/ngx_event_openssl.h
--- a/src/event/ngx_event_openssl.h     Wed Apr 29 14:59:02 2015 +0300
+++ b/src/event/ngx_event_openssl.h     Tue May 05 20:29:34 2015 +0800
@@ -51,6 +51,8 @@
     ngx_buf_t                  *buf;
     size_t                      buffer_size;
 
+    ngx_msec_t                  last_write_msec;
+
     ngx_connection_handler_pt   handler;
 
     ngx_event_handler_pt        saved_read_handler;










_______________________________________________
nginx-devel mailing list
nginx-devel at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx-devel


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20150513/b4f1533c/attachment-0001.html>


More information about the nginx-devel mailing list