FCGI.pm ?

[mike]

On Fri, Mar 6, 2009 at 5:22 PM, Roger Hoover <roger.hoover at gmail.com> wrote:
>
>
> On Thu, Mar 5, 2009 at 4:15 PM, mike <mike503 at gmail.com> wrote:
>>
>> I guess I would be more than happy then to stop using php-fpm and use
>> this new tool.
>>
>> However I do like the benefits of php-fpm like the ability to override
>> some php.ini directives.
>
> This seems like something that still appropriate for PHP's FCGI wrapper,
> even if that wrapper no longer needs to manage pools of processes.

I want management of pools so I can separate by uid/gid, limits of
children, and if applicable adaptive spawning limits

I believe php-fpm uses libevent, there's gotta be a way to make a
middleware style daemon for this that leverages a large amount of
php-fpm style concepts and even daemonizing code..

>> Perhaps we should get Grzegorz (fcgiwrap), Andrei (php-fpm) and others
>> together for this.
>
> I'd love to see if there's interest in this idea.


>> A generic fastcgi to (php|cgi|ruby|python|whatever) interface ?
>
> A language-agnostic FastCGI process manager?  One process manager to rule
> them all and in darkness bind them. :)
>
>>
>> On Thu, Mar 5, 2009 at 12:26 PM, Roger Hoover <roger.hoover at gmail.com>
>> wrote:
>> >
>> >
>> > On Thu, Mar 5, 2009 at 9:56 AM, mike <mike503 at gmail.com> wrote:
>> >>
>> >> Sure - you mean anything that nginx (or any other server) speaks
>> >> fastcgi
>> >> to
>> >
>> > Yes!
>> >
>> >>
>> >> webserver <-> this process manager <-> code (perl cgi, or anything
>> >> else) ?
>> >>
>> >> Wouldn't this also be able to replace php-fpm then?
>> >
>> > Ideally, yes.  Even with PHP's unique lifecycle (all objects discarded
>> > after
>> > each request), I don't see a reason why process pool management needs to
>> > be
>> > built in to PHP FastCGI handling.  Each PHP process could manage the
>> > request
>> > lifecycle for itself while the process pool management could be handled
>> > generically, just like any other language.
>> >
>> >>
>> >> It seems like there are already ways using Tomcat/etc. for java,
>> >> php-fpm for PHP, ruby and python have managers, but CGI does not. Can
>> >> you give an example of what other things you'd want this to manage?
>> >
>> > In short, I'd be happy if it exactly matched the process management
>> > features
>> > of mod_fastcgi for Apache.  mod_fastcgi does both FastCGI proxying and
>> > process management.  Nginx + other event driven webservers implement the
>> > FastCGI proxying piece but not the process management piece.
>> >
>> > You're right that each language seems to have a different way of
>> > managing
>> > processes.  This is appealing when all the infrastructure you run is in
>> > a
>> > single language.  If you run ruby, you do it "the ruby way".  However,
>> > the
>> > organizations I've worked for need to run code in different languages,
>> > for
>> > various reasons including aquisition or migration.  I think everyone
>> > would
>> > benefit from a language agnostic process manage similar to mod_fastcgi
>> > but
>> > not tied to Apache.
>> >
>> >>
>> >> On Thu, Mar 5, 2009 at 9:22 AM, Roger Hoover <roger.hoover at gmail.com>
>> >> wrote:
>> >> > This is an interesting idea.  I'd like to see a generic process
>> >> > manager
>> >> > come
>> >> > out of the effort though, not just one that only works for wrapping
>> >> > CGI.  It
>> >> > would be a great separation of concerns.   The CGI wrapper could
>> >> > focus
>> >> > on a
>> >> > single task: accepting FastCGI requests and forking CGI processes to
>> >> > handle
>> >> > them.  Process pool management could be handled by a generic FastCGI
>> >> > process
>> >> > manager, which could manage fcgiwrap pools and any other type of
>> >> > FastCGI
>> >> > processes.
>> >> >
>> >> > On Wed, Mar 4, 2009 at 9:05 PM, mike <mike503 at gmail.com> wrote:
>> >> >>
>> >> >> I am going to start a cgi-fpm project. The goals will be aligned
>> >> >> like
>> >> >> php-fpm except slightly modified for the differences with cgi and
>> >> >> fastcgi.
>> >> >> It might not be able to support adaptive spawning without some sort
>> >> >> of
>> >> >> api
>> >> >> or tools but at least will make management easier and not require
>> >> >> third
>> >> >> party tools. It will basically enhance fcgiwrap with php-fpm style
>> >> >> configuration and hooks for external control for things like an
>> >> >> nginx
>> >> >> module.
>> >> >>
>> >> >> The annoyances with cgi and fastcgi can be discussed and hopefully
>> >> >> addressed.
>> >> >> Thoughts?
>> >> >> Also this could just be an nginx module too. But it would add some
>> >> >> weight
>> >> >> and require suexec type stuff. So probably not a good idea.
>> >> >> Let me throw together a quick list of ideas.
>> >> >> On Mar 4, 2009, at 12:34 PM, Roger Hoover <roger.hoover at gmail.com>
>> >> >> wrote:
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Wed, Mar 4, 2009 at 11:51 AM, mike <mike503 at gmail.com> wrote:
>> >> >>>
>> >> >>> On Wed, Mar 4, 2009 at 9:03 AM, Roger Hoover
>> >> >>> <roger.hoover at gmail.com>
>> >> >>> wrote:
>> >> >>>
>> >> >>> >>  - dynamic pool size management (keep 1-5 running depending on
>> >> >>> >> load;
>> >> >>> >>   this will require congestion notifications from the web
>> >> >>> >> server,
>> >> >>> >> like
>> >> >>> >>   you said)
>> >> >>> >
>> >> >>> > Functionality was recently added to supervisord to modify it's
>> >> >>> > configuration
>> >> >>> > dynamically through the XML-RPC api so this is matter of
>> >> >>> > implementing
>> >> >>> > the
>> >> >>> > load logic in an nginx plugin and making calls to supervisord to
>> >> >>> > add
>> >> >>> > and
>> >> >>> > subtract from the pool.
>> >> >>>
>> >> >>> While I would like to keep my software stack low, this sounds like
>> >> >>> a
>> >> >>> neat benefit. Would just need to define hard upper limits, and how
>> >> >>> long to wait or whatever to kill spare/unused children (like
>> >> >>> apache, I
>> >> >>> suppose)
>> >> >>>
>> >> >>> Personally I would like to see a daemon that does this in itself.
>> >> >>> Leverages the fcgiwrap code + adds on features. I suppose it would
>> >> >>> have to be 'aware' of how many connections it was servicing per
>> >> >>> pool
>> >> >>> which Grzegorz makes it sound like can be very hard... but then it
>> >> >>> could manage things dynamically.
>> >> >>>
>> >> >>> request comes in -> depending on what port/socket/etc. it checks
>> >> >>> the
>> >> >>> pool, determines if any children are open (if more needed, spawn
>> >> >>> like
>> >> >>> apache, maybe log a notice in the log), changes to proper uid/gid
>> >> >>> if
>> >> >>> configured, then executes the fastcgi stuff, if it gets back an
>> >> >>> error,
>> >> >>> determine whether or not to log it, pass it back with the same http
>> >> >>> code, do both, etc..
>> >> >>>
>> >> >>> etc.
>> >> >>
>> >> >> The approach you describe assumes that the parent process can
>> >> >> intercept
>> >> >> socket connections as they come in.  I don't think this is possible
>> >> >> within
>> >> >> the constraints of the FastCGI spec.  Each FastCGI process is forked
>> >> >> with
>> >> >> file descriptor 0 pointing to a shared FastCGI socket and each child
>> >> >> process
>> >> >> just calls accept() on that socket.  The OS is responsible to
>> >> >> determining
>> >> >> which process in the pool accepts each request so there's no way for
>> >> >> the
>> >> >> parent process to keep track of which child is taking which request.
>> >> >> Unless
>> >> >> that information can be retrieved from the kernel, I think the only
>> >> >> place
>> >> >> that load logic can be implemented is in an nginx module.
>> >> >>
>> >> >>>
>> >> >>> I don't understand enough about sockets, C, threading/forking/event
>> >> >>> models/etc. to see if that is even an option but it seems like it
>> >> >>> could be done, just not sure if it would be way too slow or not?
>> >> >>>
>> >> >>
>> >> >
>> >> >
>> >>
>> >
>> >
>>
>
>