[PATCH 1 of 1] Test proxy_unfinished: limit buffers to force buffering on no proxy temp

[Maxim Dounin]

Hello!

On Tue, Apr 15, 2014 at 05:51:25PM +0300, Markus Linnala wrote:

> On 15.04.2014 16:04, Maxim Dounin wrote:
> >Hello!
> >
> >On Tue, Apr 15, 2014 at 03:26:26PM +0300, Markus Linnala wrote:
> >
> >># HG changeset patch
> >># User Markus Linnala <Markus.Linnala at cybercom.com>
> >># Date 1397505734 -10800
> >>#      Mon Apr 14 23:02:14 2014 +0300
> >># Node ID cf54213675d59b801bee34328223c28f992cf8c7
> >># Parent  74a015aad3521ffef6b404cdaa9eafd991824301
> >>Test proxy_unfinished: limit buffers to force buffering on no proxy temp
> >>
> >>With some systems (Fedora-20 Linux 3.13) socket buffers are huge and
> >>nginx does not use temp files when proxying. Set buffer sizes to small
> >>values to make use of temp files more probable. Currently there is no
> >>way to force temp buffers.
> >>
> >>no proxy temp should fail on 1.5.2 and work okay on later versions.
> >>
> >>sub_filter forces chunked encoding and so it is possible to notice
> >>problem because last chunk is 0 chunk when there is error.
> >>
> >>Only use limited buffers on 'no proxy temp' as other tests fail
> >>because tests take too long.
> >>
> >>http://trac.nginx.org/nginx/ticket/541
> >
> >[...]
> >
> >Have you tried something simplier, like
> >
> >--- a/proxy_unfinished.t
> >+++ b/proxy_unfinished.t
> >@@ -50,7 +50,7 @@ http {
> >                         keys_zone=one:1m;
> >
> >      server {
> >-        listen       127.0.0.1:8080;
> >+        listen       127.0.0.1:8080 sndbuf=4k;
> >          server_name  localhost;
> >
> >          location / {
> >
> >
> >?
> >
> 
> I did not try plain 4k.
> 
> I did try smaller buffers and 'big (un)?finished un' tests were failing
> because it took too longer than 5s to do test. Now I ran tests again and if
> sndbuf is less than 4k or maybe some buffer size you get really bad
> throughoutput.
> 
> not ok 13 - big unfinished un
> 
> #   Failed test 'big unfinished un'
> #   at proxy_unfinished.t line 152.
> #                   undef
> #     doesn't match '(?^s:unfinished)'
> not ok 14 - big finished un
> 
> #   Failed test 'big finished un'
> #   at proxy_unfinished.t line 153.
> #                   undef
> #     doesn't match '(?^s:finished)'
> 
> 
> If sndbuf is more than about 36k test seems to fail sometimes.
> 
> One way to force creating of temp_file is to use proxy_store, but then test
> does not work as it does not start to response before there is error.
> 
> With my patch nginx tries to create temp-file when it has sent out 24k of a
> response. With sndbuf=4k it tries to create temp file around 90k. I see this
> by running prove under strace.
> 
> strace -ff -o foo -s 200 -tt prove
> 
> Extra sleep 0.5s or lowering proxy buffers size does not seem to affect the
> no proxy temp test.

There are reasons for both sleep and low proxy buffers:

- sleep is to make sure that client isn't reading too fast to make 
  buffering unneeded;

- low proxy buffers are to reduce buffering in nginx itself.

I've committed a patch with sndbuf=32k and a regular expression 
change to silently allow full responses.

http://hg.nginx.org/nginx-tests/rev/915ef26ac6eb

-- 
Maxim Dounin
http://nginx.org/