different Memory consumption for H1 and H2

[Muhui Jiang]

HI

Thanks for your great answer. you mentioned that sendfile() is to copy
between kernel space and userland. I am curious, why this whole process
don't need to malloc any memory? Could you please explain more on the
detail implementation of the sendfile(). Many Thanks

Regards
Muhui

2017-05-11 23:11 GMT+08:00 Maxim Dounin <mdounin at mdounin.ru>:

> Hello!
>
> On Thu, May 11, 2017 at 10:32:41PM +0800, Muhui Jiang wrote:
>
> > Recently, I did an experiment to test the memory consumption of nginx. I
> > request a large static zip file. I explored the debug information of
> nginx.
> >
> > For H2, below is a part of the log, I noticed that every time server will
> > allocate 65536 bytes, I increase the connection number, I noticed that
> the
> > server's memory consumption will reach to a threshhold and then increase
> > very slowly:
>
> [...]
>
> > 2017/05/11 04:54:20 [debug] 29451#0: *10499 http output filter
> > "/image/test.zip?"
> > 2017/05/11 04:54:20 [debug] 29451#0: *10499 http copy filter:
> > "/image/test.zip?"
> > 2017/05/11 04:54:20 [debug] 29451#0: *10499 malloc:
> 0000000002699A80:65536
> > 2017/05/11 04:54:20 [debug] 29451#0: *10499 read: 14, 0000000002699A80,
> > 65536, 0
>
> [...]
>
> > 2017/05/11 04:54:20 [debug] 29451#0: *10499 http2 frame out:
> > 00000000026155F0 sid:1 bl:0 len:1
> > 2017/05/11 04:54:20 [debug] 29451#0: *10499 SSL buf copy: 9
> > 2017/05/11 04:54:20 [debug] 29451#0: *10499 SSL buf copy: 1
> > 2017/05/11 04:54:20 [debug] 29451#0: *10499 SSL to write: 138
> > 2017/05/11 04:54:20 [debug] 29451#0: *10499 SSL_write: 138
> > 2017/05/11 04:54:20 [debug] 29451#0: *10499 http2:1 DATA frame
> > 00000000026155F0 was sent
>
> [...]
>
> > For H/1.1, below is a part of the debug log, no malloc is noticed during
> > the send file process. And even when I increase the connection number to
> a
> > very large value, the result shows nginx's memory consumption is still
> very
> > low. :
>
> [...]
>
> > 2017/05/11 22:29:06 [debug] 29451#0: *11015 http write filter limit 0
> > 2017/05/11 22:29:06 [debug] 29451#0: *11015 sendfile: @72470952 584002
> > 2017/05/11 22:29:06 [debug] 29451#0: *11015 sendfile: 260640 of 584002
>
> [...]
>
> > Hope to get your comments and what are the difference of nginx's memory
> > allocation mechanisms between HTTP/2.0 and HTTP/1.1. Many Thanks
>
> The difference is due to sendfile(), which is used in case of
> plain HTTP, and can't be used with SSL-encrypted connections.
> HTTP/2 is normally used with SSL encryption, so it is usually not
> possible to use sendfile() with HTTP/2.
>
> When sendfile() is not available or switched off, nginx uses
> output_buffers (http://nginx.org/r/output_buffers) to read a file
> from disk, and then writes these buffers to the connection.
>
> When it is possible to use the sendfile(), nginx does not try to
> read contents of static files it returns, but simply calls
> sendfile().  This is usually most effecient approach , as it
> avoids additional buffers and copying between kernel space and
> userland.  Unfortunately, it is not available when using HTTPS
> (including HTTP/2).
>
> --
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170513/575d48f5/attachment.html>