upgrading binary failed - execve - too long argument list

[Mathew Heard]

Maxim,

Out of curiosity I checked some of my servers with "ss -l -t -p | grep
nginx | wc -l"

I'm at 15k at most. Large numbers of worker processes make it relatively
easy to hit the ~20k limit it seems.

With Regards,
Mathew

On Wed, 5 May 2021 at 01:53, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Fri, Apr 30, 2021 at 11:14:48AM +0200, Charlie Kilo wrote:
>
> > correction.. if i count with
> > ss -l -t |grep http | wc -l
> > we have around 58340 listening sockets.. at least on that machine..
>
> Since "listen ... reuseport;" implies a separate listening socket
> for each worker process, that's expected with 50 worker processes
> and 1200 "listen ... reuseport" in the configuration.
>
> > On Fri, Apr 30, 2021 at 8:27 AM Charlie Kilo <krikkiteer at gmail.com>
> wrote:
> >
> > > Thanks a lot for the hints so far.. to provide the further info and
> answer
> > > the questions..
> > >
> > > getconf ARG_MAX shows 2097152
> > > ulimit -s shows 8192
> > > setting it to unlimited, doesn't change anything (also not with
> prlimit)
> > > wc -c /proc/<pid>/environ shows 1949
> > >
> > > it seems on a regular machine we have around 1200 listening sockets
> and do indeed use "reuseport"
>
> Thanks, so all seem to be using the defaults.
>
> Looking more closely at Linux limits, it seems that you are
> hitting the MAX_ARG_STRLEN limit.  It limits single argument
> length (and a single environment variable length), and hardcoded
> to be 32 pages, that is, almost always 128k.
>
> With 128k limit nginx should be able to pass about 20k listening
> sockets, so with 58340 sockets you are well above the limit.
>
> An obvious workaround would be to reduce the amount of listening
> sockets - either by dropping the "reuseport", or by reducing the
> the number of listening sockets (normally just one on the wildcard
> address is enough), or by reducing the number of worker processes.
>
> It would be interesting to know why you are using such a large
> number of listening sockets with reuseport enabled.  If the use
> case is reasonable, we probably should consider implementing some
> workaround for the 128k limit.
>
> --
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20210505/6e702bc5/attachment.htm>