Sharing data when download the same object from upstream

[Alex Garzão]

Hello Anatoli,

> I think this is asynchronous and if the upstream is faster than the
> downstream it save the data to cached file faster and the downstream gets
> the data from the file instead of the mem buffers.

In this case, I don't need to worry about upstream/downstream speed. Very good!

> I have the same but in ordered array (simple implementation). Anyway the
> rbtree will do the same. But this structure should be in shared memory
> because all workers should know which files are currently in downloading
> from upstream state. The should exist in tmp directory.

About shared memory, I did. Thanks.

>> My second doubt is: Could I just fix ngx_event_pipe to send to all
>> requests (instead of to send to one request)? And, if true,
>> ngx_http_output_filter can be used to send a big chunk at first time
>> (300 MB or more) and little chunks after that?
>>
>
> Use smaller chunks.

Ok. In really, I tried to send a big chunk with
ngx_http_output_filter, but, in some cases, it returns NGX_AGAIN. I
looked at all places in NGINX where this function is called, but seems
to me that, when it returns NGX_AGAIN,
ngx_http_upstream_finalize_request is called, and it deals with
buffers not sent. In my approach I call ngx_http_output_filter for
each chunk, but this not working. I think that I can't call
ngx_http_output_filter more than once per request. Or Can I ?

About use smaller chunks. I will adjust to address this. Thanks.

> You are on a right way. Just keep digging. Do not forget to turn off this
> features when you have flv or mp4 seek, partial requests and content-ecoding
> different than identity because you will send broken files to the browsers.

Ok. Thanks in advance for your help Anatoli.

Regards.

--
Alex Garzão
Projetista de Software
Azion Technologies
alex.garzao (at) azion.com