keepalive seems not to work

[Łukasz Tasz]

Hi all,

I fixed my issue with a simple application. Solution could be named "not
named upstream support".
idea is that there is proxy_pass to upstream, and upstream is forwarding
request to my local service (upstream server 127.0.0.1:1981)
Under localhost 127.0.0.1:1981 my simple application is making requests to
any server, and keeps connection alive. Control of all connection is under
nginx - keepalive #value; keepalive_requests... app looks like:

> #!/bin/env python3
> import sanic
> import aiohttp
> app = sanic.Sanic("gadula")
> app.config.REQUEST_TIMEOUT = 60
> app.config.KEEP_ALIVE_TIMEOUT = 60
> aiohttp_timeout = aiohttp.ClientTimeout(
>     total=None,
>     connect=None,
>     sock_connect=10.0,
>     sock_read=120.0
> )
> @app.route("/<path:path>")
> async def router(request, path):
>     try:
>         async with app.session.get(request.url, raise_for_status=True,
>                                    timeout=aiohttp_timeout) as r:
>             # TODO: Consider using Sanic's FileStreaming feature.
>             return sanic.response.raw(await r.read())
>     except aiohttp.ClientResponseError as e:
>         return sanic.response.json({
>             'url': request.url,
>             'error': str(e),
>         }, status=e.status)
>     except Exception as e:
>         return sanic.response.json({
>             'url': request.url,
>             'error': str(e),
>         }, status=500)
> @app.listener('before_server_start')
> async def setup(app, loop):
>     app.session = aiohttp.ClientSession()
> @app.listener('after_server_stop')
> async def setup(app, loop):
>     await app.session.close()
> app.run(host="127.0.0.1", port=1981, debug=False, access_log=False)
>
app is written in asynchronous mode, so can serve many nginx workers and
generally it does the job, now I have nginx configuration which keeps
konnection from *any* clients to *any* server.
question, why it cannot be done without any intermediate application??
any comments, suggestions are welcome!!

regards
Łukasz Tasz
RTKW


czw., 8 paź 2020 o 12:16 Łukasz Tasz <lukasz at tasz.eu> napisał(a):

> Hi,
> sucha setup is on 1stproxy, there I have upstream defined to second proxy
> and it works - connection is reused.
> problem is that it is chain of forward proxy with caching, and
> your-server.com including port is different  - service:port is dynamic.
> I'm asking for it, because with firefox (set global proxy to my second
> proxy) I go to some blabla.your-server.com connection is kept
> in a meaning that on server side I see only one established connection,
> and all of requests that I make from firefox are made over this keept
> connection. Problem starts when I will use chain of proxies (all nginx).
>
> regards
> Łukasz Tasz
> RTKW
>
>
> czw., 8 paź 2020 o 11:43 Marcin Wanat <marcin.wanat at gmail.com> napisał(a):
>
>>
>> On Thu, Oct 8, 2020 at 11:36 AM Łukasz Tasz <lukasz at tasz.eu> wrote:
>>
>>> Hi all,
>>>
>>> can I expect that proxy_pass will keep connection to remote server that
>>> is being proxied?
>>>
>>> when I'm using setup client -> proxy -> server it looks to work
>>> but when I'm using:
>>> client -> 1stProxy_upstream -> proxy -> server
>>> connection between 1stProxy and proxy is being kept thanks to keepalive
>>> 100, but proxy makes new connection every new request, very simple setup:
>>>
>>> http {
>>>   server {
>>>     listen 8080;
>>>     location / {
>>>       keepalive_disable none;
>>>       keepalive_requests 1000;
>>>       keepalive_timeout 300s;
>>>       proxy_cache proxy-cache;
>>>       proxy_cache_valid 200 302 301 30m;
>>>       proxy_cache_valid any 1m;
>>>       proxy_cache_key $scheme://$http_host$request_uri;
>>>       proxy_pass $scheme://$http_host$request_uri;
>>>       proxy_http_version 1.1;
>>>       proxy_set_header Connection "";
>>>     }
>>>   }
>>> }
>>> I would expect that when client connects proxy and it works then it
>>> should also works when proxy connects upstream proxy....
>>>
>>
>> For keepalive in upstream proxy you shoud use upstream configuration
>> block and configure keepalive in it:
>>
>> upstream backend {
>>    zone backend 1m;
>>    server your-server.com;
>>    keepalive 128;
>> }
>>
>>   server {
>>     listen 8080;
>>     location / {
>>       proxy_cache proxy-cache;
>>       proxy_cache_valid 200 302 301 30m;
>>       proxy_cache_valid any 1m;
>>       proxy_cache_key $scheme://$http_host$request_uri;
>>       proxy_pass $scheme://backend$request_uri;
>>       proxy_http_version 1.1;
>>       proxy_set_header Connection "";
>>     }
>>   }
>>
>> --
>> Marcin Wanat
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20201019/1c8660d7/attachment.htm>