[PATCH 2 of 2] add support for setting worker processes and CPU affinity automatically

[Valentin V. Bartenev]

On Friday 13 January 2012 14:04:11 Joshua Zhu wrote:
> Hi,
> 
> On Fri, Jan 13, 2012 at 1:53 AM, Valentin V. Bartenev <ne at vbart.ru> wrote:
> > On Thursday 12 January 2012 19:37:15 Joshua Zhu wrote:
> [snip]
> 
> Thank you very much for the review.
> 
>  You use "ngx_ncpu" global variable and for Linux you set it by
> 
> > sysconf(_SC_NPROCESSORS_ONLN).
> > 
> > On freeBSD and MacOS systems it set by sysctls(hw.ncpu), and it is equal
> > to 1 on others.
> 
> Setting CPU affinity is available on Linux systems only. Please correct me
> if I'm wrong.
> 

You're right.

But I generally said about "ngx_ncpu", that also affects "auto" setting in your 
patch for the "worker_processes" directive.

> > Which means that your "auto" settings are very inaccurate. Sometimes the
> > "worker_processes" directive will be equal to 2 * cores or more,
> > sometimes it will be equal to 1, and if you're lucky, then it will be
> > equal to real CPU cores.
> 
> This is mostly caused by the hyper-threading feature of some CPUs. I don't
> think this is an issue if hyper-threading is turned on. Sure we need to
> find a better way to detect the number of CPUs on a system.
> 

Yeap, we need to find a better way. And I don't think the solution will be simple 
and it is worth to be implemented in a web server. 

> > So, I don't think that this patch makes life easier. In fact, it creates
> > a false sense of optimal worker_processes setting. Even if ngx_ncpu
> > would be always equal to real hardware cores number, it wouldn't be
> > optimal for every
> > use cases.
> 
> It's not 'optimal', but 'automatic'. This patch still supports setting
> worker processes and CPU affinities mannually as what the standard Nginx
> does ;)

So, when you need some automation for configuration and deployment, the better 
solution would be using proper software for that task, isn't it? Like Puppet,
for example.

wbr, Valentin V. Bartenev