Buffer reuse like gzip filter module, with pre-configured number of buffers

[Maxim Dounin]

Hello!

On Tue, May 31, 2022 at 12:57:46PM -0400, hanzhai wrote:

[...]

> rc = ngx_http_next_body_filter(r, ch); // rc got NGX_OK
> if (rc != NGX_OK && rc != NGX_AGAIN) {
>     goto failed;
> }
> 
> ngx_chain_update_chains(r->pool, &ctx->free, &ctx->busy, &ch,
>                         (ngx_buf_tag_t) &ngx_http_my_filter_module);
> 
> ngx_chain_t *cl = ch;
> while (cl) {
>     ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, "buf sz: %z",
> ngx_buf_size(cl->buf)); // output: buf sz: 65536
>     cl = cl->next;
> }
> 
> // reuse the buffer b if possible
> 
> 
> Explanation:
> 
> I created a buffer b with 64k, then I copied 64k data, then I called
> ngx_http_next_body_filter to send the first 64k data to the client, then I
> want to reuse the buffer b to send the next 64k data and so on.
> 
> The question is after ngx_http_next_body_filter, ctx->free is still NULL,
> because ngx_buf_size(b) got 65536 which means that the buffer b cannot be
> recycled. My guess even if the next_body_filter returns an NGX_OK, it
> doesn't guarantee any of the buffers inside the ngx_chain_t can be reused.
> 
> What should I do if I want to reuse the 64k buffer?

The idea is that buffers which aren't yet sent will be referenced 
in ctx->busy, and once fully sent, they will be moved to ctx->free 
and can be reused.  If your body filter was called and there are 
no buffers in ctx->free (and you've exhausted all the configured 
buffers), you simply call ngx_http_next_body_filter() with NULL 
chain and then call ngx_chain_update_chains() to see if there are 
any buffers which can be reused.  And if there are still no 
buffers to use, you simply return NGX_AGAIN to wait for additional 
write events.  Check the ctx->nomem case in the gzip filter for 
additional details.

-- 
Maxim Dounin
http://mdounin.ru/