High Load configuration question

[Volodymyr Kostyrko]

On 11.01.2010 20:58, Tronman wrote:

> Still, I am curious as to what you meant by "your module is taking focus and not giving it back", also if anyone else has something to add feel free to chime in!

Let's look at this from whole other point.

There are two major models to create multitask programs - thread model 
and event model. Each have their own drawbacks and advantages. Event 
model is far more supreme then threading when you need to wait for many 
events or you have very high context switch rate. Threading however is 
better for computational tasks because any thread can be preempted at 
any time and threads can execute in parallel on multiple cpus.

Event model means that program specifies all of events it want to wait 
for and asks kernel to send signal when any of them occurs. After that 
program processes specified event (reading data, sending data) and 
returns back to waiting for another kernel signal. When event is been 
processed all other events are just postponed and none of them can 
preempt running code. This can sound as bad choice, but when you need to 
read data it's better to make it fast and at once, without switching 
threads. Switching threads always means loosing a part of processor 
cache and branching tree. So giving running process 2ms more is 
generally better than preempting it now and loosing 1ms on context switch.

Nginx is built a top of that principle. No execution loop should block 
for relatively big time (I'd say 100ms is waaaay too big) and should use 
it's time at max.

So there may be too different problems with your module:

1. You try to compute something. It's not good and actually it's better 
to be done in other process. Or you can reformat the loop to process one 
hunk of data at a time and signal the dispatcher that you would like to 
work more.

2. You try to perform blocking operations (like reading files) yourself 
without giving this job to nginx. Your code is opening file, then 
waiting for the first block to appear, then waiting for the second 
block... This can take a lot time.

-- 
Sphinx of black quartz judge my vow.