2 of 16 cores are constantly maxing out - how to balance the load?

[Raffael Vogler]

Hey Lucas, your assumption seems to be correct. According to
/proc/interrupts the following stats are significantly higher for those two
cores (CPU5, CPU6 - 0-based indexing):

- CPU5: xen-percpu-ipi       callfuncsingle5
- CPU6: xen-percpu-ipi       callfuncsingle6
- CPU5: xen-pirq-msi-x     eth0-TxRx-0
- CPU6: xen-pirq-msi-x     eth0-TxRx-1
- CPU5,6: TLB shootdowns
- CPU5,6: Hypervisor callback interrupts

Is this something that can and should be optimized or is it simply a matter
of fact due to the high load on the (fixed) available network card capacity?


On Thu, Jan 11, 2018 at 11:16 AM, Lucas Rolff <lucas at lucasrolff.com> wrote:

> If it’s the same two cores, it might be another process that uses the same
> two cores and thus happens to max out.
>
> One very likely possibility would be interrupts from e.g. networking. You
> can check /proc/interrupts to see where interrupts from the network happens.
>
>
>
> *From: *nginx <nginx-bounces at nginx.org> on behalf of Raffael Vogler <
> raffael.vogler at yieldlove.com>
> *Reply-To: *"nginx at nginx.org" <nginx at nginx.org>
> *Date: *Thursday, 11 January 2018 at 11.14
> *To: *"nginx at nginx.org" <nginx at nginx.org>
> *Subject: *2 of 16 cores are constantly maxing out - how to balance the
> load?
>
>
>
> Hello!
>
> I have nginx with php-fpm running on a 16 core Ubuntu 16.04 instance. The
> server is handling more than 10 million requests per hour.
>
> https://imgur.com/a/iRZ7V
>
> As you can see on the htop screenshot cores 6 and 7 are maxed out and
> that's the case constantly - even after restarting nginx those two cores
> stay at that level.
>
> I wonder why is that so and how to balance the load more evenly?
>
> Also I'm curious to know whether this might indicate a performance
> relevant issue or if it is most likely harmless and just looks odd.
>
> > cat /etc/nginx/nginx.conf | grep -v '^\s*#'
>
>
>
> user www-data;
>
> worker_processes auto;
>
> pid /run/nginx.pid;
>
> events {
>
>         worker_connections 768;
>
> }
>
> http {
>
>         sendfile on;
>
>         tcp_nopush on;
>
>         tcp_nodelay on;
>
>         keepalive_timeout 65;
>
>         types_hash_max_size 2048;
>
>         include /etc/nginx/mime.types;
>
>         default_type application/octet-stream;
>
>         ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
>
>         ssl_prefer_server_ciphers on;
>
>         access_log /var/log/nginx/access.log;
>
>         error_log /var/log/nginx/error.log;
>
>         gzip on;
>
>         gzip_disable "msie6";
>
>         include /etc/nginx/conf.d/*.conf;
>
>         include /etc/nginx/sites-enabled/*;
>
> }
>
> Thanks
>
> Raffael
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>



-- 
Raffael Vogler, Chief Technology Officer

Yieldlove GmbH
Neuer Pferdemarkt 1, 20359 Hamburg

www.yieldlove.com

raffael.vogler at yieldlove.com
XING <https://www.xing.com/profile/Raffael_Vogler> - LinkedIn
<https://www.linkedin.com/in/raffaelvogler>
Skype: joyofdata

Registernummer: HRB 127559; Registergericht: Amtsgericht Hamburg; USt-ID:
DE815426709; Geschäftsführung:Benjamin Gries, Timo Hagenow, Ivan Tomic
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20180111/5ce069fe/attachment.html>