Buffers Chain

[Maxim Dounin]

Hello!

On Thu, May 22, 2014 at 03:21:56PM +0100, Paulo Silva wrote:

> Hi,
> Let me introduce the Elephant in the room. My name is Paulo and I'm
> from Portugal.
> I am starting developing nginx modules and I'm having a great time
> with buffers chain :D
> 
> I'm using nginx as a reverse proxy and my goal is to log the full
> response body (HTML) on the server.
> 
> I did start with a filter module with a body_filter and my approach is
> as follow:
> 
> 1. Inspect given ngx_chain_t *in linked list to understand if nginx
> has the full response (last_buf)
> 2. On each body_filter call I'm updating a linked list which glues
> given ngx_chain_t *in to the previous ones.
> 3. As soon as I detect the last_buf, I iterate my internal linked list
> (and each linked ngx_chain_t lists), dumping buffers content.
> 
> Although I'm reaching the last_buf, I'm missing some bytes.

In no particular order:

- Make sure you are allocating your own ngx_chain_t structures.  
  Trying to modify ngx_chain_t structures got in *in chain is a 
  common mistake.

- If you are passing buffers to next filters, it's incorrect to 
  assume that buffer contents will be available later (a buffer 
  contents will be eventually sent to a client, and then the 
  buffer can be reused by its owner).

- If you are not passing buffers to next filters, it's incorrect 
  to assume that you'll get full response (e.g., with 
  "output_buffers 1 64", you'll get only first 64 bytes of a 
  response).

- If you didn't set r->filter_need_in_memory, it's incorrect to 
  assume that buffers will be in memory.

To log a response body, you'll have to copy contents of all 
buffers sent though your filter.  Just saving links to buffers is 
certainly not enough.

-- 
Maxim Dounin
http://nginx.org/