question about filter

[Anatoli Marinov]

Hello mates,
I continued to work on the issue I found in my module. I wrote another 
simpler module that just collect first 128 KB of each served file and 
write them in a file.
Unfortunately for one and the same served file I found different content 
in dump files and this is very confusing situation. Maybe my function 
that collect separated buffers from the chains is wrong?
The function skip only ngx_buf_special flagged buffers ....
Should I skip something else?

The test module is attached;

Thanks in advance
Anatoli Marinov

On 06/15/2011 02:16 PM, Anatoli Marinov wrote:
> Hello Colleagues,
>
> I am trying to write a filter that should buffer some data coming from 
> upstream. When the filter has collected enough data (for example 3 
> MB), special function will estimate the desired limit rate related to 
> the data from the buffer (the files are mp4 so limit rate will be set 
> according to the mp4 bitrate).
> In most cases it works but sometime for one and the same file I got 
> different data in the buffer ?!?!?!. The data is similar but invalid 
> so the mp4 parsing function cannot use it.
> When the file exists in the local cache everything is ok. The issue 
> appears only when the file comes from upstream.
>
> What could be the reason to receive different data in the filter?
> I am using nginx 1.0.2 on 64b machine with linux.
>
> My example code:
>
> static void ngx_http_throttling_collect(ngx_http_request_t *r,
>         ngx_http_throttling_ctx_t *ctx, ngx_chain_t *in)
> {
>     ngx_chain_t *ch_in;
>     ngx_buf_t *buf;
>     ngx_int_t read_bytes, in_b_size, local_b_space, len;
>
>     ch_in = in;
>
>     for(ch_in = in; ch_in != NULL; ch_in = ch_in->next) {
>
>         if(ch_in->buf == NULL) {
>             continue;
>         }
>
>         buf = ch_in->buf;
>
>         if(ngx_buf_special(buf)) {
>             NGX_HTTP_LLOG(r->connection->log, "r[%p] buff is 
> special\n", r);
>             continue;
>         }
>
>         /* try to recognize the resource */
>         if(ctx->buff_type == THROTLING_UNKNOWN) {
>             if(buf->temp_file == 0) {
>                 ctx->buff_type = THROTLING_USE_FILE;
>             } else {
>                 ctx->buff_type = THROTLING_USE_MEM;
>             }
>         }
>
>         if(ctx->buff_type == THROTLING_USE_FILE) {
>             read_bytes = read(buf->file->fd, ctx->b.last,
>                     NGX_HTTP_THRT_BUFF_SIZE);
>
>             if(read_bytes != NGX_HTTP_THRT_BUFF_SIZE) {
>                 ctx->state = THROTTLING_CANNOT_GET_FORMAT_INFO;
>                 r->limit_rate = ctx->lconf->default_rate_limit;
>                 r->limit_rate_after = r->limit_rate *  
> ctx->lconf->full_speed_sec;
>                 return;
>             }
>
>             ctx->b.last += read_bytes;
>             ngx_http_throttling_process_buffer(r, ctx);
>         } else {
>             in_b_size = buf->last - buf->start;
>             local_b_space = NGX_HTTP_THRT_BUFF_SIZE -
>                     CDN_BUFF_DATA_LEN(&ctx->b);
>
>             if(local_b_space == 0) {
>                 ngx_http_throttling_process_buffer(r, ctx);
>                 return;
>             }
>
>             len = local_b_space < in_b_size ? local_b_space
>                     : in_b_size;
>
>             if(len) {
>                 ctx->b.last = ngx_copy(ctx->b.last, buf->start, len);
>             }
>         }
>     }
> }
>
>
> Thanks for all ideas in advance.
>
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://nginx.org/mailman/listinfo/nginx-devel

-------------- next part --------------
A non-text attachment was scrubbed...
Name: ngx_http_test_module.c
Type: text/x-csrc
Size: 10904 bytes
Desc: not available
URL: <http://nginx.org/pipermail/nginx-devel/attachments/20110616/3b625ccc/attachment.c>