Curious problem with PHP

[Ian]

Hi all,

I have been investigating a performance problem with my 
nginx/init-fastcgi/PHP set up and I'm not having much success. All ideas 
gratefully received.

It appears as if the PHP worker processes return their reply (in 100ms 
or so), and then hang about for 500-700ms before being able to handle 
the next message. This results in worker starvation under remarkably low 
loads.

The application is a chat room. I am using the Nginx HTTP push module. 
POSTed messages are passed to PHP, which uses its CURL module to write 
to the /publish page. PHP also talks to MySQL for data storage.
Javascript fetches from the /subscribe address, updates the screen and 
requests again. GETs from /subscribe do not involve PHP.

I know the delay is after the reply, because Firebug shows the response 
time is 100 to 200ms - as expected. Ping times are 100ms plus).

With 5 PHP worker processes and three people paste/sending as fast as 
they can, it is very easy to overload the system.  (I can nearly do it 
on my own).  Each browser stores the POSTs waiting for the previous 
reply thus each browser can only have one message in PHP at a time. Thus 
there cannot be more than three messages in the PHP system at once.  So 
how can 5 workers be overloaded?

I have stopped and counted 5 of my own messages coming through over the 
next 7-8 seconds.  It appears from comments made on the phone, that all 
messages arrive together - just late.  No messages are dropped.

So it appears that worker starvation is happening - but not long 
response times. Server load is also slight (machine load is pushed up 
from 0.3% to 2% out of 400% - 4CPUs).

Conclusion: the worker processes are hanging about after sending their 
reply for some considerable time (guess 500-700ms) before they can 
handle a new message.

In an effort to identify what might cause the PHP worker process to hang 
about I have:
      released the session as early as possible - no change.
      switched the MySQL processing from a transaction to auto-commit - 
this made things slightly worse.
      doubled the VPS memory - this had minimal effect but removes the 
possibility of swapping
      attempted to use keep-alive between nginx and the fastcgi process 
(and failed. Gave up as this will probably going to make things worse - 
no keep alive is the default).

Using nginx 1.4.4 , PHP 5.3.10

Has anyone got an idea as what might be happening? I'm stumped.

Thanks

Ian