question about filter

[Anatoli Marinov]

Hello Colleagues,

I am trying to write a filter that should buffer some data coming from 
upstream. When the filter has collected enough data (for example 3 MB), 
special function will estimate the desired limit rate related to the 
data from the buffer (the files are mp4 so limit rate will be set 
according to the mp4 bitrate).
In most cases it works but sometime for one and the same file I got 
different data in the buffer ?!?!?!. The data is similar but invalid so 
the mp4 parsing function cannot use it.
When the file exists in the local cache everything is ok. The issue 
appears only when the file comes from upstream.

What could be the reason to receive different data in the filter?
I am using nginx 1.0.2 on 64b machine with linux.

My example code:

static void ngx_http_throttling_collect(ngx_http_request_t *r,
         ngx_http_throttling_ctx_t *ctx, ngx_chain_t *in)
{
     ngx_chain_t *ch_in;
     ngx_buf_t *buf;
     ngx_int_t read_bytes, in_b_size, local_b_space, len;

     ch_in = in;

     for(ch_in = in; ch_in != NULL; ch_in = ch_in->next) {

         if(ch_in->buf == NULL) {
             continue;
         }

         buf = ch_in->buf;

         if(ngx_buf_special(buf)) {
             NGX_HTTP_LLOG(r->connection->log, "r[%p] buff is 
special\n", r);
             continue;
         }

         /* try to recognize the resource */
         if(ctx->buff_type == THROTLING_UNKNOWN) {
             if(buf->temp_file == 0) {
                 ctx->buff_type = THROTLING_USE_FILE;
             } else {
                 ctx->buff_type = THROTLING_USE_MEM;
             }
         }

         if(ctx->buff_type == THROTLING_USE_FILE) {
             read_bytes = read(buf->file->fd, ctx->b.last,
                     NGX_HTTP_THRT_BUFF_SIZE);

             if(read_bytes != NGX_HTTP_THRT_BUFF_SIZE) {
                 ctx->state = THROTTLING_CANNOT_GET_FORMAT_INFO;
                 r->limit_rate = ctx->lconf->default_rate_limit;
                 r->limit_rate_after = r->limit_rate *  
ctx->lconf->full_speed_sec;
                 return;
             }

             ctx->b.last += read_bytes;
             ngx_http_throttling_process_buffer(r, ctx);
         } else {
             in_b_size = buf->last - buf->start;
             local_b_space = NGX_HTTP_THRT_BUFF_SIZE -
                     CDN_BUFF_DATA_LEN(&ctx->b);

             if(local_b_space == 0) {
                 ngx_http_throttling_process_buffer(r, ctx);
                 return;
             }

             len = local_b_space < in_b_size ? local_b_space
                     : in_b_size;

             if(len) {
                 ctx->b.last = ngx_copy(ctx->b.last, buf->start, len);
             }
         }
     }
}


Thanks for all ideas in advance.