Issue with 3rd-party memc and eval modules

[agentzh]

On Wed, Feb 24, 2010 at 9:52 PM, Markus Jelsma <markus at buyways.nl> wrote:
> Yes, i can provide a snippet. In the end i have decided to abandon the project
> of doing it in configuration. It is extremely hard to try and write imperative
> code in a declarative language; although we do have the if operator [1],
> working with it is more troublesome than ever before ;)
>

Sigh. I'm sure there's some bug in one of those modules that you were
using. I'm afraid you'll also have a hard time in writing a module
yourself because the nginx's core does not have a nice API ;)

> We will now try and build a module for our purpose, at least, i hope so.
>
> Anyway, here is a snippet to reproduce a non -closing connection using eval in
> a named location called by echo_exec - it gets complicated indeed.
>

I've reproduced this connection hang using the following (slightly
modified) version of config:

   location /test
   {
     echo_exec @initialize;
   }

   location @initialize
   {
     eval_override_content_type 'text/plain';

     eval $id
     {
       #rewrite ^(.*)$ /id;
       proxy_pass http://127.0.0.1:$server_port/id;
     }
     echo $id;
   }
   location /id {
        echo hi;
   }

Our Test::Nginx::Socket module reports a timeout that the nginx server
does not close the client connection in time.

But I've found a work-around by avoid using named location:

   location /test
   {
     echo_exec /initialize;
   }

   location /initialize
   {
     internal;
     eval_override_content_type 'text/plain';

     eval $id
     {
       #rewrite ^(.*)$ /id;
       proxy_pass http://127.0.0.1:$server_port/id;
     }
     echo $id;
   }
   location /id {
        echo hi;
   }

Then GET /test gives the output as expected:

     hi

It seems to be a named location issue. According to the implementation
in the nginx core, "named location" is quite buggy. And we've run into
some other issues earlier as well. So my suggestion is to use the
"internal" directive combined with normal locations for such kind of
things.

>
> Even this little piece of code gave me a hard time; imagine how some 50 lines
> using various eval blocks, proxy_pass and memc_pass directives and many evil
> if operators ;)
>

I do understand the frustration. Without considerable knowledge to the
nginx internals, it's quite difficult to diagnose issues when things
go wrong.

> Anyway, thanks for all the support on the way!
>

You're always welcome to report detailed issues to the list and please
don't give up config file scripting ;)

>
> [1]: http://wiki.nginx.org/IfIsEvil
>

Well, the "if" directive in the http rewrite module is not that evil
as long as we know how it works. I believe the evilness of "if" has
been exaggerated. It just not works the way you're used to in most of
other languages like C and Perl.

We've been enjoying "if" in our nginx.conf for something big:

    http://agentzh.org/misc/nginx.conf

I've given some hints regarding "if"'s behavior here:

   http://forum.nginx.org/read.php?2,25197,25197

It's not that hard to predict how it works ;)

On the other hand, I'd say named locations are indeed evil because it
does not clear the request object like normal locations and may lead
to consequences that are *very* hard to predict.

Cheers,
-agentzh