Debugging CPU usage in Nginx

[Brad Patton]

Hi all,

About 2 months ago the CPU usage on 1 of our servers started going crazy.  All of a sudden, Nginx itself started using about 6x the CPU power to serve requests with no increase in traffic at all.  It's a dramatic difference and it's directly attributed to Nginx, not PHP, Mariadb, or anything else.  I can show you a lot of graphs and data showing the problem if needed so let me know if you would like to see more.  🙂

The only OS updates during the week prior to the issue starting were to PHP.  We're currently running Nginx version 1.27.0 and we were on 1.25.5 when it started.  In addition to those updates, we've run several test builds where we disabled certain modules just randomly trying to isolate the issue.

I was starting to think that the issue had something to do with our configuration, even though that didn't change at the start of the issue either, but then I ran a test in a Fedora 40 container and it solved the problem.  Unfortunately, I could only run that test temporarily, but the results were clear.  The same exact build with the same exact configuration works fine in Fedora 40 but takes about 6x the CPU power to serve the same requests in Centos Stream 9.

That makes me think there is either some bug in Centos 9 or a bug in Nginx that only happens in Centos 9.  Does anyone have any thoughts on how to find out exactly what in Nginx is causing the much higher CPU usage?

If not, would you like to see more information about the problem?  If so, what?

Thanks!
Brad

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20240812/48db8f59/attachment.htm>