Memory use flares up sharply, how to troubleshoot?

[Maxim Dounin]

Hello!

On Mon, Jul 21, 2014 at 05:44:45PM -0400, gthb wrote:

> Hi,
> 
> I finally reproduced this, with debug logging enabled --- I found the
> problematic request in the error log preceding the kill signal, saying it
> was being buffered to a temporary file:
> 
>     2014/07/21 11:39:39 [warn] 21182#0: *32332838 an upstream response is
> buffered to a temporary file /var/cache/nginx/uwsgi_temp/9/90/0000186909
> while reading upstream, client: x.x.x.x, server: foo.com, request: "GET
> /api/nasty/troublemaker.csv?el=xyzzy!a:b&dates_as_dates=1 HTTP/1.1",
> upstream: "uwsgi://123.45.67.89:3003", host: "foo.com"
>     2014/07/21 11:41:18 [alert] 16885#0: worker process 21182 exited on
> signal 9
> 
> and retrying that request reproduces the problem, nginx growing in size
> without bound. (The request never made it to the access log because of the
> OOM kill, which is why my previous testing didn't reproduce it)

[...]

> These extra lines *never* appear in the healthy requests, so I imagine they
> point to the problem (but I am not at all familiar with Nginx debug output);
> in particular those "write new buf" lines look relevant; they are output
> right after ngx_alloc_chain_link is called.

The lines in question are just sending the response to the client 
via the response body filter chain.

> All the possibly relevant Nginx config:

I don't see anything obviously wrong in the config, but again - I 
strongly recommend you to post _full_ configuration.  Or, better 
yet, if you are able to reproduce the problem with a reduced 
config, post it instead (and corresponding debug log).

[...]

> Does this narrow down the problem? Can I provide anything further?

No.  Please show the debug log.

-- 
Maxim Dounin
http://nginx.org/