Usage of the shadow field in ngx_buf_t?

[Brian Pane]

On Sat, Dec 5, 2009 at 4:06 PM, Brian Pane <brianp at brianp.net> wrote:
[...]
> Thanks.  What is the right thing to do if a body filter consumes
> only part of a buf and sets aside the rest for later?

By the way, I should note that the best solution I've come up with
so far is:

- In the body filter, make a shallow copy of the chain using
  ngx_chain_add_copy.
- Each chain link in this copy points to one of the buffers
  from the original chain passed into the body filter.
- Leave the "in" chain unmodified, so that the ngx_output_chain
  code puts it in the busy list.
- In subsequent calls to the body filter, once the data is finally
  consumed, set buf->pos=buf->last.  When the current invocation
  of the output body filters finishes, ngx_output_chain will call
  ngx_chain_update_chains, which will move the busy bufs
  to the free list (since buf->pos is finally equal to buf->last
  for each buf).

There's a problem, though.  This technique works when
one filter does it.  But if two or more filters do the same
thing, though, then there's a problem: the first filter in the
chain might set buf->pos=buf->last because it has finished
its processing on the buffer, but the next filter might want
to set aside some or all of the buffer until it receives more
data.  The two filters thus end up needing to set buf->pos
to two different values for the same buf.

As a workaround, I suppose each filter that sets aside
buffers could make a copy of each buf, pass the copy
on to the next filter, and then (after the next filter returns)
update the original buf based on how much of the copy
buf the next filters have consumed.  (Could I use the
buf->shadow field in the copy buf to hold the pointer
to the original?)

But it seems like this is a problem that other people
may have solved already, so if there's an existing
solution, I'd rather use that instead of inventing a
new design.

Thanks,
-Brian