Sharing data when download the same object from upstream

Alex Garzão alex.garzao at
Fri Aug 30 18:53:55 UTC 2013

Hello Anatoli,

> I think this is asynchronous and if the upstream is faster than the
> downstream it save the data to cached file faster and the downstream gets
> the data from the file instead of the mem buffers.

In this case, I don't need to worry about upstream/downstream speed. Very good!

> I have the same but in ordered array (simple implementation). Anyway the
> rbtree will do the same. But this structure should be in shared memory
> because all workers should know which files are currently in downloading
> from upstream state. The should exist in tmp directory.

About shared memory, I did. Thanks.

>> My second doubt is: Could I just fix ngx_event_pipe to send to all
>> requests (instead of to send to one request)? And, if true,
>> ngx_http_output_filter can be used to send a big chunk at first time
>> (300 MB or more) and little chunks after that?
> Use smaller chunks.

Ok. In really, I tried to send a big chunk with
ngx_http_output_filter, but, in some cases, it returns NGX_AGAIN. I
looked at all places in NGINX where this function is called, but seems
to me that, when it returns NGX_AGAIN,
ngx_http_upstream_finalize_request is called, and it deals with
buffers not sent. In my approach I call ngx_http_output_filter for
each chunk, but this not working. I think that I can't call
ngx_http_output_filter more than once per request. Or Can I ?

About use smaller chunks. I will adjust to address this. Thanks.

> You are on a right way. Just keep digging. Do not forget to turn off this
> features when you have flv or mp4 seek, partial requests and content-ecoding
> different than identity because you will send broken files to the browsers.

Ok. Thanks in advance for your help Anatoli.


Alex Garzão
Projetista de Software
Azion Technologies
alex.garzao (at)

More information about the nginx-devel mailing list