posix_memalign(16, 16384) failed (12: Cannot allocate memory)

[JohnCarne]

Hello,

[root at web1 ~]# nginx -v
nginx version: nginx/1.11.4

We are now after 13 days we observer suddenly in nginx logs this in an
intempestive manner, and causing nginx to reload, causing slow down on
server : posix_memalign(16, 16384) failed (12: Cannot allocate memory)

This happens after our upgrade to last nginx version through nDeploy.

I called in nginx sysadmin, Ndeploy sysadmin too, and finally cloudlinux
support which made an incredible job investigating the issue over 7 days by
enabling multiple kernel debug tools to find out what is going on.

All nginx/linux settings has been tweaked/verified. Issue can't be solved,
and about 5 guys has broken their head on the issue, without being able to
solve. We know all the basic, even advanced, and experts were in.

Cloudlinux support says this is the cause, and you need nginx expert to find
out why nginx beheave likes this :

>From the information we collected it appears that nginx is really changing
his ulimits:
# grep nginx /home/abackupnomem3.log | tail
nginx-792752 [009] 5438179.898678: setrlimit: (sys_setrlimit+0x63/0x70

Conclusion is that nginx manage those rlimits. This is not a solution, but a
way for you where to dig more.


This was added : ulimit -q unlimited  in etc/init.d/nginx :

start() {
    echo -n $"Starting $prog: "
    ulimit -n 64000
    ulimit -q unlimited
    daemon --pidfile=${pidfile} ${nginx} -c ${conffile}
    RETVAL=$?
    echo
    [ $RETVAL = 0 ] && touch ${lockfile}
    return $RETVAL
}


Anyone has a clue ?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,269787,269787#msg-269787