posix_memalign(16, 16384) failed (12: Cannot allocate memory)
mdounin at mdounin.ru
Fri Sep 23 16:07:28 UTC 2016
On Fri, Sep 23, 2016 at 05:20:56AM -0400, JohnCarne wrote:
> [root at web1 ~]# nginx -v
> nginx version: nginx/1.11.4
> We are now after 13 days we observer suddenly in nginx logs this in an
> intempestive manner, and causing nginx to reload, causing slow down on
> server : posix_memalign(16, 16384) failed (12: Cannot allocate memory)
> This happens after our upgrade to last nginx version through nDeploy.
> I called in nginx sysadmin, Ndeploy sysadmin too, and finally cloudlinux
> support which made an incredible job investigating the issue over 7 days by
> enabling multiple kernel debug tools to find out what is going on.
> All nginx/linux settings has been tweaked/verified. Issue can't be solved,
> and about 5 guys has broken their head on the issue, without being able to
> solve. We know all the basic, even advanced, and experts were in.
Just a basic hint, in case you haven't tried it yet: re-compile
nginx without any 3rd party modules, and check if it helps.
> Cloudlinux support says this is the cause, and you need nginx expert to find
> out why nginx beheave likes this :
> From the information we collected it appears that nginx is really changing
> his ulimits:
> # grep nginx /home/abackupnomem3.log | tail
> nginx-792752  5438179.898678: setrlimit: (sys_setrlimit+0x63/0x70
> Conclusion is that nginx manage those rlimits. This is not a solution, but a
> way for you where to dig more.
The setrlimit() call is used by nginx to manage some limits it
knows about and configured to manage. In particular, it is used
directive as seen in your config, and for the worker_rlimit_core
directive. Details about the directives can be found here:
They set RLIMIT_CORE and RLIMIT_NOFILE limits, nothing more, and
have nothing to do with the memory allocation errors you see.
More information about the nginx