Buffers Chain

[Paulo Silva]

On Mon, May 26, 2014 at 9:09 AM, SplitIce <mat999 at gmail.com> wrote:
> As in blocking send and connect? I don't know the specifics of Unix Sockets,
> but don't they block when the buffer fills (I know FIFO queues do)?
>

Sorry, I don't fully understand your question.
I was expecting that with the SOCK_NONBLOCK it would not block.

What would be your approach?
Do you know about any nginx internal mechanism to accomplish this goal
(get the upstream response body out of nginx)?

>
> On Mon, May 26, 2014 at 9:22 AM, Paulo Silva <pauloasilva at gmail.com> wrote:
>>
>> Hi,
>> I'm not sure whether I will face problems with other filters modifying
>> the response body after mine, but for know I'm comfortable as I can
>> rebuild the full response body just iterating buffers chains.
>>
>> As I said before I'm using nginx as reverse proxy and my main goal is
>> to pass the upstream (proxy_pass) response to another local process
>> (relative to nginx).
>>
>> I am benchmarking Unix sockets and Shared memory as IPC.
>> I did it already for Unix Sockets and with my prototype the nginx
>> "performance" dropped for half the number of requests per second. Of
>> course I'm doing something really bad.
>>
>> Is it OK to use socket/connect/send from inside an nginx module?
>>
>> I would be glad to hear from you.
>> Thanks,
>>
>> On Fri, May 23, 2014 at 2:50 PM, Paulo Silva <pauloasilva at gmail.com>
>> wrote:
>> > Because I don't have deep knowledge of nginx internal and I can not
>> > find a proper resource about it, the best I can do and with what I am
>> > comfortable is with body_filter.
>> >
>> > Do you think I can notice whether all other 3rd party module filters
>> > finish modifying the ngx_chain_t *in ?
>> >
>> >
>> >
>> > On Fri, May 23, 2014 at 2:41 PM, Maxim Dounin <mdounin at mdounin.ru>
>> > wrote:
>> >> Hello!
>> >>
>> >> On Fri, May 23, 2014 at 02:17:27PM +0100, Paulo Silva wrote:
>> >>
>> >>> Hi,
>> >>>
>> >>> On Fri, May 23, 2014 at 12:58 PM, Maxim Dounin <mdounin at mdounin.ru>
>> >>> wrote:
>> >>> > Hello!
>> >>> >
>> >>> > On Fri, May 23, 2014 at 11:57:20AM +0100, Paulo Silva wrote:
>> >>> >
>> >>> >> there is other option than modify the auto/modules file?
>> >>> >>
>> >>> >> According to my goal (capture the full request response body) I
>> >>> >> would
>> >>> >> say that my module must run right before the postpone.
>> >>> >
>> >>> > Before the postpone filter you'll get subrequest bodies in your
>> >>> > filter, which is probably not what you want (the postpone filter
>> >>> > is to glue subrequest together, correctly ordered).
>> >>> >
>> >>> >> Am I supposed to modify the auto/modules like follows?
>> >>> >>
>> >>> >> if [ $HTTP_POSTPONE = YES ]; then
>> >>> >>     HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES
>> >>> >> $HTTP_POSTPONE_FILTER_MODULE"
>> >>> >>     HTTP_SRCS="$HTTP_SRCS $HTTP_POSTPONE_FILTER_SRCS"
>> >>> >> fi
>> >>> >>
>> >>> >> # insert my module here!
>> >>> >>
>> >>> >> if [ $HTTP_SSI = YES ]; then
>> >>> >>     have=NGX_HTTP_SSI . auto/have
>> >>> >>     HTTP_FILTER_MODULES="$HTTP_FILTER_MODULES
>> >>> >> $HTTP_SSI_FILTER_MODULE"
>> >>> >>     HTTP_DEPS="$HTTP_DEPS $HTTP_SSI_DEPS"
>> >>> >>     HTTP_SRCS="$HTTP_SRCS $HTTP_SSI_SRCS"
>> >>> >> fi
>> >>> >>
>> >>> >>
>> >>> >> I did check my modules config file and I did realize that it is
>> >>> >> "queued" as HTTP_AUX_FILTER_MODULES. There are different queues for
>> >>> >> core modules and addons?
>> >>> >
>> >>> > The HTTP_AUX_FILTER_MODULES is a generic queue, and it's the
>> >>> > only one currently officially supported for 3rd party modules.
>> >>> >
>> >>> > If you want your filter to be called right before/after postpone
>> >>> > filter, it should be relatively safe to put it into the
>> >>> > HTTP_POSTPONE_FILTER_MODULE variable though (and may be with some
>> >>> > additional checks to make sure the postpone filter is enabled, or
>> >>> > just a code to enable it unconditionally).
>> >>> >
>> >>>
>> >>> And this is also valid when compiling nginx with the --add-module
>> >>> flag?
>> >>> How does config file look like?
>> >>>
>> >>> My knowledge is restricted to Emiller's Guide To Nginx Module
>> >>> Development (http://www.evanmiller.org/nginx-modules-guide.html) and a
>> >>> few debugging hours.
>> >>
>> >> Uhm, looking again into auto/modules I think I was wrong, and
>> >> modifying the HTTP_POSTPONE_FILTER_MODULE variable won't work
>> >> (added module config scripts are executed later on), you should
>> >> modify HTTP_FILTER_MODULES variable instead, and put your module
>> >> into a proper position.
>> >>
>> >> Note that the "config" file of a module is just a shell script,
>> >> and you are free to do more or less anything there.
>> >>
>> >> --
>> >> Maxim Dounin
>> >> http://nginx.org/
>> >>
>> >> _______________________________________________
>> >> nginx-devel mailing list
>> >> nginx-devel at nginx.org
>> >> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>> >
>> >
>> >
>> > --
>> > Paulo A. Silva
>> > http://tech.pauloasilva.com
>> > http://linkedin.com/in/devpauloasilva/
>>
>>
>>
>> --
>> Paulo A. Silva
>> http://tech.pauloasilva.com
>> http://linkedin.com/in/devpauloasilva/
>>
>> _______________________________________________
>> nginx-devel mailing list
>> nginx-devel at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx-devel
>
>
>
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel



-- 
Paulo A. Silva
http://tech.pauloasilva.com
http://linkedin.com/in/devpauloasilva/