Is it possible that nginx will not buffer the client body?
Weibin Yao
yaoweibin at gmail.com
Mon Feb 25 02:13:42 UTC 2013
Can you show me your configure? It works for me with nginx-1.2.7.
Thanks.
2013/2/22 Pasi Kärkkäinen <pasik at iki.fi>
> On Fri, Feb 22, 2013 at 11:25:24AM +0200, Pasi Kärkkäinen wrote:
> > On Fri, Feb 22, 2013 at 10:06:11AM +0800, Weibin Yao wrote:
> > > Use the patch I attached in this mail thread instead, don't use the
> pull
> > > request patch which is for tengine.Â
> > > Thanks.
> > >
> >
> > Oh sorry I missed that attachment. It seems to apply and build OK.
> > I'll start testing it.
> >
>
> I added the patch on top of nginx 1.2.7 and enabled the following options:
>
> client_body_postpone_sending 64k;
> proxy_request_buffering off;
>
> after that connections through the nginx reverse proxy started failing
> with errors like this:
>
> [error] 29087#0: *49 upstream prematurely closed connection while reading
> response header from upstream
> [error] 29087#0: *60 upstream sent invalid header while reading response
> header from upstream
>
> And the services are unusable.
>
> Commenting out the two config options above makes nginx happy again.
> Any idea what causes that? Any tips how to troubleshoot it?
>
> Thanks!
>
> -- Pasi
>
>
> >
> > > 2013/2/22 Pasi KÀrkkÀinen <[1]pasik at iki.fi>
> > >
> > > On Fri, Jan 18, 2013 at 10:38:21AM +0200, Pasi KÀrkkÀinen wrote:
> > > > On Thu, Jan 17, 2013 at 11:15:58AM +0800, ?????? wrote:
> > > > > Â Â Yes. It should work for any request method.
> > > > >
> > > >
> > > > Great, thanks, I'll let you know how it works for me. Probably
> in two
> > > weeks or so.
> > > >
> > >
> > > Hi,
> > >
> > > Adding the tengine pull request 91 on top of nginx 1.2.7 doesn't
> work:
> > >
> > > cc1: warnings being treated as errors
> > > src/http/ngx_http_request_body.c: In function
> > > 'ngx_http_read_non_buffered_client_request_body':
> > > src/http/ngx_http_request_body.c:506: error: implicit declaration
> of
> > > function 'ngx_http_top_input_body_filter'
> > > make[1]: *** [objs/src/http/ngx_http_request_body.o] Error 1
> > > make[1]: Leaving directory `/root/src/nginx/nginx-1.2.7'
> > > make: *** [build] Error 2
> > >
> > > ngx_http_top_input_body_filter() cannot be found from any .c/.h
> files..
> > > Which other patches should I apply?
> > >
> > > Perhaps this?
> > > [2]
> https://github.com/cfsego/limit_upload_rate/blob/master/for-nginx.patch
> > >
> > > Thanks,
> > > -- Pasi
> > >
> > > >
> > > > > Â Â 2013/1/16 Pasi KÃ*â*¬rkkÃ*â*¬inen <[1][3]pasik at iki.fi>
> > > > >
> > > > > Â Â Â On Sun, Jan 13, 2013 at 08:22:17PM +0800, ??????
> wrote:
> > > > > Â Â Â > Ã* Â Ã* This patch should work between nginx-1.2.6
> and
> > > nginx-1.3.8.
> > > > > Â Â Â > Ã* Â Ã* The documentation is here:
> > > > >
> > > > > Â Â Â > Ã* Â Ã* ## client_body_postpone_sending ##
> > > > > Â Â Â > Ã* Â Ã* Syntax: **client_body_postpone_sending**
> `size`
> > > > > Â Â Â > Ã* Â Ã* Default: 64k
> > > > > Â Â Â > Ã* Â Ã* Context: `http, server, location`
> > > > > Â Â Â > Ã* Â Ã* If you specify the
> `proxy_request_buffering` or
> > > > > Â Â Â > Ã* Â Ã* `fastcgi_request_buffering` to be off,
> Nginx will
> > > send the body
> > > > > Â Â Â to backend
> > > > > Â Â Â > Ã* Â Ã* when it receives more than `size` data or
> the
> > > whole request body
> > > > > Â Â Â has been
> > > > > Â Â Â > Ã* Â Ã* received. It could save the connection and
> reduce
> > > the IO number
> > > > > Â Â Â with
> > > > > Â Â Â > Ã* Â Ã* backend.
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* ## proxy_request_buffering ##
> > > > > Â Â Â > Ã* Â Ã* Syntax: **proxy_request_buffering** `on |
> off`
> > > > > Â Â Â > Ã* Â Ã* Default: `on`
> > > > > Â Â Â > Ã* Â Ã* Context: `http, server, location`
> > > > > Â Â Â > Ã* Â Ã* Specify the request body will be buffered
> to the
> > > disk or not. If
> > > > > Â Â Â it's off,
> > > > > Â Â Â > Ã* Â Ã* the request body will be stored in memory
> and sent
> > > to backend
> > > > > Â Â Â after Nginx
> > > > > Â Â Â > Ã* Â Ã* receives more than
> `client_body_postpone_sending`
> > > data. It could
> > > > > Â Â Â save the
> > > > > Â Â Â > Ã* Â Ã* disk IO with large request body.
> > > > > Â Â Â >
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* Â Ã* Â Note that, if you
> specify it
> > > to be off, the nginx
> > > > > Â Â Â retry mechanism
> > > > > Â Â Â > Ã* Â Ã* with unsuccessful response will be broken
> after
> > > you sent part of
> > > > > Â Â Â the
> > > > > Â Â Â > Ã* Â Ã* request to backend. It will just return 500
> when
> > > it encounters
> > > > > Â Â Â such
> > > > > Â Â Â > Ã* Â Ã* unsuccessful response. This directive also
> breaks
> > > these
> > > > > Â Â Â variables:
> > > > > Â Â Â > Ã* Â Ã* $request_body, $request_body_file. You
> should not
> > > use these
> > > > > Â Â Â variables any
> > > > > Â Â Â > Ã* Â Ã* more while their values are undefined.
> > > > > Â Â Â >
> > > > >
> > > > > Â Â Â Hello,
> > > > >
> > > > > Â Â Â This patch sounds exactly like what I need aswell!
> > > > > Â Â Â I assume it works for both POST and PUT requests?
> > > > >
> > > > > Â Â Â Thanks,
> > > > >
> > > > > Â Â Â -- Pasi
> > > > >
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Hello!
> > > > > Â Â Â > Ã* Â Ã* Â Ã* @yaoweibin
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* If you are eager for this
> feature, you
> > > could try my
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* patch:
> > > [2][2][4]https://github.com/taobao/tengine/pull/91.
> > > > > Â Â Â This patch has
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* been running in our production
> servers.
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* Â Ã* what's the nginx version your patch
> based on?
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Thanks!
> > > > > Â Â Â > Ã* Â Ã* Â Ã* On Fri, Jan 11, 2013 at 5:17 PM, Ã**Ã*
> > > Ã**Ã*°Ã*±Ã*³
> > > > > Â Â Â <[3][3][5]yaoweibin at gmail.com> wrote:
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* I know nginx team are working on
> it. You
> > > can wait for it.
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* If you are eager for this
> feature, you
> > > could try my
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* patch:
> > > [4][4][6]https://github.com/taobao/tengine/pull/91.
> > > > > Â Â Â This patch has
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* been running in our production
> servers.
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* 2013/1/11 li zJay
> > > <[5][5][7]zjay1987 at gmail.com>
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* Hello!
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* is it possible that nginx
> will not
> > > buffer the client
> > > > > Â Â Â body before
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* handle the request to
> upstream?
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* we want to use nginx as a
> reverse
> > > proxy to upload very
> > > > > Â Â Â very big file
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* to the upstream, but the
> default
> > > behavior of nginx is to
> > > > > Â Â Â save the
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* whole request to the local
> disk
> > > first before handle it
> > > > > Â Â Â to the
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* upstream, which make the
> upstream
> > > impossible to process
> > > > > Â Â Â the file on
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* the fly when the file is
> uploading,
> > > results in much high
> > > > > Â Â Â request
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* latency and server-side
> resource
> > > consumption.
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* Thanks!
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã*
> > > _______________________________________________
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* nginx mailing list
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã* [6][6][8]nginx at nginx.org
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Â Ã*
> > > [7][7][9]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* --
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Weibin Yao
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* Developer @ Server Platform Team
> of
> > > Taobao
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã*
> > > _______________________________________________
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* nginx mailing list
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã* [8][8][10]nginx at nginx.org
> > > > > Â Â Â > Ã* Â Ã* Â Ã* Â Ã*
> > > [9][9][11]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* Â Ã*
> > > _______________________________________________
> > > > > Â Â Â > Ã* Â Ã* Â Ã* nginx mailing list
> > > > > Â Â Â > Ã* Â Ã* Â Ã* [10][10][12]nginx at nginx.org
> > > > > Â Â Â > Ã* Â Ã* Â Ã*
> > > [11][11][13]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* --
> > > > > Â Â Â > Ã* Â Ã* Weibin Yao
> > > > > Â Â Â > Ã* Â Ã* Developer @ Server Platform Team of Taobao
> > > > > Â Â Â >
> > > > > Â Â Â > References
> > > > > Â Â Â >
> > > > > Â Â Â > Ã* Â Ã* Visible links
> > > > > Â Â Â > Ã* Â Ã* 1. mailto:[12][14]zjay1987 at gmail.com
> > > > > Â Â Â > Ã* Â Ã* 2.
> > > [13][15]https://github.com/taobao/tengine/pull/91
> > > > > Â Â Â > Ã* Â Ã* 3. mailto:[14][16]yaoweibin at gmail.com
> > > > > Â Â Â > Ã* Â Ã* 4.
> > > [15][17]https://github.com/taobao/tengine/pull/91
> > > > > Â Â Â > Ã* Â Ã* 5. mailto:[16][18]zjay1987 at gmail.com
> > > > > Â Â Â > Ã* Â Ã* 6. mailto:[17][19]nginx at nginx.org
> > > > > Â Â Â > Ã* Â Ã* 7.
> > > [18][20]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â Â Â > Ã* Â Ã* 8. mailto:[19][21]nginx at nginx.org
> > > > > Â Â Â > Ã* Â Ã* 9.
> > > [20][22]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â Â Â > Ã* Â 10. mailto:[21][23]nginx at nginx.org
> > > > > Â Â Â > Ã* Â 11.
> > > [22][24]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > >
> > > > > Â Â Â > _______________________________________________
> > > > > Â Â Â > nginx mailing list
> > > > > Â Â Â > [23][25]nginx at nginx.org
> > > > > Â Â Â > [24][26]
> http://mailman.nginx.org/mailman/listinfo/nginx
> > > > >
> > > > > Â Â Â _______________________________________________
> > > > > Â Â Â nginx mailing list
> > > > > Â Â Â [25][27]nginx at nginx.org
> > > > > Â Â Â [26][28]
> http://mailman.nginx.org/mailman/listinfo/nginx
> > > > >
> > > > > Â Â --
> > > > > Â Â Weibin Yao
> > > > > Â Â Developer @ Server Platform Team of Taobao
> > > > >
> > > > > References
> > > > >
> > > > > Â Â Visible links
> > > > > Â Â 1. mailto:[29]pasik at iki.fi
> > > > > Â Â 2. [30]https://github.com/taobao/tengine/pull/91
> > > > > Â Â 3. mailto:[31]yaoweibin at gmail.com
> > > > > Â Â 4. [32]https://github.com/taobao/tengine/pull/91
> > > > > Â Â 5. mailto:[33]zjay1987 at gmail.com
> > > > > Â Â 6. mailto:[34]nginx at nginx.org
> > > > > Â Â 7. [35]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â Â 8. mailto:[36]nginx at nginx.org
> > > > > Â Â 9. [37]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â 10. mailto:[38]nginx at nginx.org
> > > > > Â 11. [39]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â 12. mailto:[40]zjay1987 at gmail.com
> > > > > Â 13. [41]https://github.com/taobao/tengine/pull/91
> > > > > Â 14. mailto:[42]yaoweibin at gmail.com
> > > > > Â 15. [43]https://github.com/taobao/tengine/pull/91
> > > > > Â 16. mailto:[44]zjay1987 at gmail.com
> > > > > Â 17. mailto:[45]nginx at nginx.org
> > > > > Â 18. [46]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â 19. mailto:[47]nginx at nginx.org
> > > > > Â 20. [48]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â 21. mailto:[49]nginx at nginx.org
> > > > > Â 22. [50]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â 23. mailto:[51]nginx at nginx.org
> > > > > Â 24. [52]http://mailman.nginx.org/mailman/listinfo/nginx
> > > > > Â 25. mailto:[53]nginx at nginx.org
> > > > > Â 26. [54]http://mailman.nginx.org/mailman/listinfo/nginx
> > > >
> > > > > _______________________________________________
> > > > > nginx mailing list
> > > > > [55]nginx at nginx.org
> > > > > [56]http://mailman.nginx.org/mailman/listinfo/nginx
> > > >
> > > > _______________________________________________
> > > > nginx mailing list
> > > > [57]nginx at nginx.org
> > > > [58]http://mailman.nginx.org/mailman/listinfo/nginx
> > >
> > > _______________________________________________
> > > nginx mailing list
> > > [59]nginx at nginx.org
> > > [60]http://mailman.nginx.org/mailman/listinfo/nginx
> > >
> > > --
> > > Weibin Yao
> > > Developer @ Server Platform Team of Taobao
> > >
> > > References
> > >
> > > Visible links
> > > 1. mailto:pasik at iki.fi
> > > 2.
> https://github.com/cfsego/limit_upload_rate/blob/master/for-nginx.patch
> > > 3. mailto:pasik at iki.fi
> > > 4. https://github.com/taobao/tengine/pull/91
> > > 5. mailto:yaoweibin at gmail.com
> > > 6. https://github.com/taobao/tengine/pull/91
> > > 7. mailto:zjay1987 at gmail.com
> > > 8. mailto:nginx at nginx.org
> > > 9. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 10. mailto:nginx at nginx.org
> > > 11. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 12. mailto:nginx at nginx.org
> > > 13. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 14. mailto:zjay1987 at gmail.com
> > > 15. https://github.com/taobao/tengine/pull/91
> > > 16. mailto:yaoweibin at gmail.com
> > > 17. https://github.com/taobao/tengine/pull/91
> > > 18. mailto:zjay1987 at gmail.com
> > > 19. mailto:nginx at nginx.org
> > > 20. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 21. mailto:nginx at nginx.org
> > > 22. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 23. mailto:nginx at nginx.org
> > > 24. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 25. mailto:nginx at nginx.org
> > > 26. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 27. mailto:nginx at nginx.org
> > > 28. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 29. mailto:pasik at iki.fi
> > > 30. https://github.com/taobao/tengine/pull/91
> > > 31. mailto:yaoweibin at gmail.com
> > > 32. https://github.com/taobao/tengine/pull/91
> > > 33. mailto:zjay1987 at gmail.com
> > > 34. mailto:nginx at nginx.org
> > > 35. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 36. mailto:nginx at nginx.org
> > > 37. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 38. mailto:nginx at nginx.org
> > > 39. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 40. mailto:zjay1987 at gmail.com
> > > 41. https://github.com/taobao/tengine/pull/91
> > > 42. mailto:yaoweibin at gmail.com
> > > 43. https://github.com/taobao/tengine/pull/91
> > > 44. mailto:zjay1987 at gmail.com
> > > 45. mailto:nginx at nginx.org
> > > 46. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 47. mailto:nginx at nginx.org
> > > 48. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 49. mailto:nginx at nginx.org
> > > 50. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 51. mailto:nginx at nginx.org
> > > 52. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 53. mailto:nginx at nginx.org
> > > 54. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 55. mailto:nginx at nginx.org
> > > 56. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 57. mailto:nginx at nginx.org
> > > 58. http://mailman.nginx.org/mailman/listinfo/nginx
> > > 59. mailto:nginx at nginx.org
> > > 60. http://mailman.nginx.org/mailman/listinfo/nginx
> >
> > > _______________________________________________
> > > nginx mailing list
> > > nginx at nginx.org
> > > http://mailman.nginx.org/mailman/listinfo/nginx
> >
> > _______________________________________________
> > nginx mailing list
> > nginx at nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
--
Weibin Yao
Developer @ Server Platform Team of Taobao
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20130225/b971ea35/attachment-0001.html>
More information about the nginx
mailing list