Video Streaming using non http backend, Ref ngx_drizzle

[agentzh]

Hello!

On Mon, Jun 4, 2012 at 10:01 AM, Sammy Raul <sammyraul1 at gmail.com> wrote:
> I am trying to stream video it can be mp4, flv anything using nginx.
>
> The video streams in the form of 1024 size will be available from the
> backend non-http server.
>

I think this can be done trivially via ngx_lua module while still
achieving good performance. Here is a small example that demonstrates
how to meet your requirements with a little Lua:

    location /api {
        content_by_lua '
            local sock, err = ngx.socket.tcp()
            if not sock then
                ngx.log(ngx.ERR, "failed to get socket: ", err)
                ngx.exit(500)
            end

            sock:settimeout(1000)  -- 1 sec

            local ok, err = sock:connect("some.backend.host", 12345)
            if not ok then
                ngx.log(ngx.ERR, "failed to connect to upstream: ", err)
                ngx.exit(502)
            end

            local bytes, err = sock:send("some query")
            if not bytes then
                ngx.log(ngx.ERR, "failed to send query: ", err)
                ngx.exit(502)
            end

            while true do
                local data, err, partial = sock:receive(1024)
                if not data then
                    if err == "closed" then
                        if partial then
                            ngx.print(partial)
                            ngx.eof()
                            ngx.exit(ngx.OK)
                        end
                    else
                        ngx.log(ngx.ERR, "error reading data: ", err)
                        ngx.exit(502)
                    end
                else
                    ngx.print(data)
                    ngx.flush(true)
               end
            end
        ';
    }

See the documentation for details:

    http://wiki.nginx.org/HttpLuaModule

> For achieveing this I followed the ngx_http_drizzle source.
>
> I wrote an upstream handler and followed most of the source code from
> ngx_http_drizzle.
>

As the author of ngx_drizzle, I suggest you start from trying out
ngx_lua. Customizing ngx_drizzle for your needs requires a *lot* of
work. The C approach should only be attempted when Lua is indeed too
slow for your purpose, which is not very likely for many applications
though.

Also, please note that ngx_drizzle does not support strict
non-buffered data output. So, for downstream connections that are slow
to write, data will still accumulate in RAM without control. On the
other hand, the ngx_lua sample given above does not suffer from this
issue.

> I have few questions or to be more precise I did not understood how the
> output from drizzle is being streamed to the client.
>
> 1) In ngx_http_drizzle_output.c the function ngx_http_drizzle_submit_mem is
> the place where it is setting the output filter, Is it also sending the
> response i.e the stream to the client at this point, or it is some other
> function?
>

Nope. Sending output buffers to the output filter chain is done by the
ngx_http_drizzle_output_bufs function.

> 2) What I need to do to send my video contents to the client, I followed the
> drizzle example but setting output and sending stream to the client, how I
> can achieve this. I have 1024B avaialble at one point and I want to send
> this to the client till the backend server has no stream to send and the
> client should be able to play the content.
>

Basically, you can call the ngx_http_output_filter function, just as
other nginx upstream modules.

> 3) Is it possible to send the video stream to the client with the browser.
>

I do not quite follow this question.

Best regards,
-agentzh