NGINX using 100% of the server CPU when testing with Blitz.io

[julianfernandes]

Yichun Zhang (agentzh) Wrote:
-------------------------------------------------------
> Hello!
> 
> On Wed, Nov 26, 2014 at 11:15 AM, julianfernandes wrote:
> > Running Blitz.io on it the server is getting absolutely murdered by
> the
> > NGINX worker processes, which each one using 100% CPU according to
> top and
> > htop.
> 
> 100% CPU usage problems are usually trivial (and also fun) to solve
> with the right tools. For example, the on-CPU flame graph tool like
> this:
> 
>     https://github.com/openresty/nginx-systemtap-toolkit#sample-bt
> 
> Linux's "perf top" can be useful here as well.
> 
> Well, just FYI :)
> 
> Regards,
> -agentzh
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

I will look into those, thank you for the suggestion!

:)


Robert Paprocki Wrote:
-------------------------------------------------------
> Hi,
> 
> Can you provide the mailing list with the output of 'nginx -V' and a
> debug log when a worker process is pegged at 100%. See
> http://nginx.org/en/docs/debugging_log.html for info on debug logs.
> 
> I also wonder why your supplied config has the following:
> 
> limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s ;
> limit_req_status 444 ;
> 
> Are you using this anywhere?
> 
> On 11/26/2014 11:15 AM, julianfernandes wrote:
> > I'm facing an issue here and after six days of looking around I
> decided to
> > ask for some help here, after all a million heads can think a lot
> better
> > than one.
> > 
> > I have a Ubuntu 14.04 server setup with NGINX, HHVM, PHP5-FPM (as a
> backup),
> > Percona MySQL, Memcached (which will be replaced by Redis). I have
> > fastcgi_cache setup for WordPress and object caching done over
> memcached.
> > All cool and dandy in theory, but not in practice.
> > 
> > This is a RamNode OpenVZ SSD VPS with 2GB of RAM and an Intel Xeon
> E5 with
> > two cores for my VPS. I also tested on a KVM SSD VPS with 2GB of RAM
> and an
> > Intel Xeon E3 with four cores for my VPS.
> > 
> > Running Blitz.io on it the server is getting absolutely murdered by
> the
> > NGINX worker processes, which each one using 100% CPU according to
> top and
> > htop. I usually run with the following pattern:
> > 
> >     --pattern 999-1000:60 https://www.geeksune.com/blog/hello-world/
> > 
> > That makes makes CPU go to the roof and according to Blitz.io this
> is the
> > result of that:
> > 
> >> 135 HITS WITH 57,734 ERRORS & 234 TIMEOUTS
> > 
> > Obviously that isn't good. RAM usage stay under 250MB all the time
> and it
> > seems that all those requests from Blitz.io are hitting the cache,
> as seen
> > here:
> > 
> >> 54.232.204.19 - HIT [23/Nov/2014:19:06:32 -0200] "GET / HTTP/1.1"
> 200 7632
> > "-" "blitz.io; 20bed746e60d4e53401c0c04d245c4eb at 189.101.124.196"
> > 
> > Notice the HIT at the start. I set a new log format and added
> > $upstream_cache_status to it.
> > 
> > A similar setup on the same machine works just fine with Blitz.io,
> so there
> > is definitely something wrong with my NGINX setup and it seems
> related to
> > fastcgi_cache. I have the same results every time, even with just
> PHP5-FPM
> > with Zend.
> > 
> > Does anyone have a clue about what is happening? My configuration
> files look
> > like this:
> > 
> >  - /etc/nginx/nginx.conf - http://paste.ubuntu.com/9236266/
> >  - /etc/nginx/sites-available/geeksune.com -
> > http://paste.ubuntu.com/9236282/
> >  - /etc/nginx/conf.d/includes/ssl.inc -
> http://paste.ubuntu.com/9236298/
> >  - /etc/nginx/conf.d/includes/security.inc -
> > http://paste.ubuntu.com/9236321/
> >  - /etc/nginx/conf.d/includes/caching.inc -
> > http://paste.ubuntu.com/9236353/
> >  - /etc/nginx/conf.d/includes/locations.inc -
> > http://paste.ubuntu.com/9236366/
> > 
> > Thanks in advance.
> > 
> > :)
> > 
> > Posted at Nginx Forum:
> http://forum.nginx.org/read.php?2,255158,255158#msg-255158
> > 
> > _______________________________________________
> > nginx mailing list
> > nginx at nginx.org
> > http://mailman.nginx.org/mailman/listinfo/nginx
> > 
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

I was going to use those two lines for wp-login.php protection on WordPress
later on, but I will probably remove them and create something with
fail2ban. I did removed those lines for testing, but it did not solve the
issue.

Here is the output of nginx -V:
- http://paste.ubuntu.com/9268513/

And this is the error.log with debug:
- http://paste.ubuntu.com/9268765/

That is a gigantic file, but vim could handle it just fine. It seems SSL is
being mentioned everywhere on the file, so my next test will be with HTTP
instead of HTTPS. Can you see anything from that?

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,255158,255181#msg-255181