From nginx-forum at forum.nginx.org  Sat Jul  1 02:14:55 2017
From: nginx-forum at forum.nginx.org (ptcell)
Date: Fri, 30 Jun 2017 22:14:55 -0400
Subject: ngx_http_sub_module causes requests to hang on a simple match.
Message-ID: <75023c41f6cd332bcf9dc70c4a23cd05.NginxMailingListEnglish@forum.nginx.org>

I've built with the sub filter enabled and I'm finding it hangs requests if
there is a match.   It is a very simple substitution/replace.   I've
resorted to following the request in GDB and the sub module completes and
calls the next body filter (which in my case appears to be the charset
module).   I have no other odd modules enabled other than using threads with
a thread pool size of two (shouldn't matter, right?).  Pausing all the
threads in GDB shows no obvious place it is hanging.

If I change the match string to something that doesn't match anything, the
request works fine.

Here is my config:

        location / {
            root   html;
            index  index.html index.htm;
            sub_filter '</title>' 'xxx</title>';
            sub_filter_once on;
        }

nginx -V
nginx version: nginx/1.7.11
built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 
configure arguments: --with-http_sub_module --with-debug --with-threads
--with-cc-opt='-O0 -g'

Thanks!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275245,275245#msg-275245


From soracchi at multidialogo.it  Sat Jul  1 12:04:08 2017
From: soracchi at multidialogo.it (Andrea Soracchi)
Date: Sat, 1 Jul 2017 14:04:08 +0200 (CEST)
Subject: Strange issue after nginx update
In-Reply-To: <20170630163426.GB3000@daoine.org>
References: <7255090.1031.1498662518798.JavaMail.sorry@sorry-Dell-System-XPS-L322X>
 <3523972.333.1498730493247.JavaMail.sorry@sorry-Dell-System-XPS-L322X>
 <CAKWQeyhny35Udo44Lfhp+Jr3JGudFeAZAGhLwmXUFPaxXaUMTw@mail.gmail.com>
 <1514956751.155023111.1498779972105.JavaMail.zimbra@netbuilder.it>
 <CAKWQeyiuNvRiigCZSUZb9tGKj6jC9LM3BoeBVGHb=iO4Q51+4w@mail.gmail.com>
 <30825680.131.1498809638062.JavaMail.sorry@sorry-Dell-System-XPS-L322X>
 <15750487.938.1498831830132.JavaMail.sorry@sorry-Dell-System-XPS-L322X>
 <20170630163426.GB3000@daoine.org>
Message-ID: <471079259.161284295.1498910648639.JavaMail.zimbra@netbuilder.it>

Hi Francis, 

the problem is with the last ubuntu 16.04 LTS with the php 5.6 ( repository ondrej/php5-5.6). 

I try to downgrade to ubuntu 14.04 LTS... 

Thanks, 

Andrea 




ANDREA SORACCHI 
+39 329 0512704 
System Engineer 


+39 0521 24 77 91 
soracchi at netbuilder.it 


Da: "Francis Daly" <francis at daoine.org> 
A: "nginx" <nginx at nginx.org> 
Inviato: Venerd?, 30 giugno 2017 18:34:26 
Oggetto: Re: Strange issue after nginx update 

On Fri, Jun 30, 2017 at 04:10:32PM +0200, Andrea Soracchi wrote: 

Hi there, 

you suggest that with the old unknown version of nginx and the old 
unknown version of php, things worked. 

Now with the new updated version of nginx and the new updated version 
of php, things fail. 

And with the same updated version of nginx and a different updated 
version of php, things work again. 

The only pair there where you have one piece the same and one piece 
different, is new nginx and two different versions of php - one works, 
one fails. 

So you probably want to compare the php versions and their configuration 
to see what is different. 


(If you have one version of php and two versions of nginx showing one 
set working and one set failing, then you should compare the nginx 
versions for differences. But that is not what you have reported here, 
if I am reading it right.) 

Good luck with it, 

f 
-- 
Francis Daly francis at daoine.org 
_______________________________________________ 
nginx mailing list 
nginx at nginx.org 
http://mailman.nginx.org/mailman/listinfo/nginx 

-- 
Questo messaggio e' stato analizzato ed e' risultato non infetto. 
This message was scanned and is believed to be clean. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170701/1854e455/attachment.html>

From vbart at nginx.com  Sun Jul  2 12:26:53 2017
From: vbart at nginx.com (Valentin V. Bartenev)
Date: Sun, 02 Jul 2017 15:26:53 +0300
Subject: ngx_http_sub_module causes requests to hang on a simple match.
In-Reply-To: <75023c41f6cd332bcf9dc70c4a23cd05.NginxMailingListEnglish@forum.nginx.org>
References: <75023c41f6cd332bcf9dc70c4a23cd05.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <1762729.p3EqeqSY3a@vbart-workstation>

On Friday 30 June 2017 22:14:55 ptcell wrote:
> I've built with the sub filter enabled and I'm finding it hangs requests if
> there is a match.   It is a very simple substitution/replace.   I've
> resorted to following the request in GDB and the sub module completes and
> calls the next body filter (which in my case appears to be the charset
> module).   I have no other odd modules enabled other than using threads with
> a thread pool size of two (shouldn't matter, right?).  Pausing all the
> threads in GDB shows no obvious place it is hanging.
> 
> If I change the match string to something that doesn't match anything, the
> request works fine.
> 
> Here is my config:
> 
>         location / {
>             root   html;
>             index  index.html index.htm;
>             sub_filter '</title>' 'xxx</title>';
>             sub_filter_once on;
>         }
> 
> nginx -V
> nginx version: nginx/1.7.11
> built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 
> configure arguments: --with-http_sub_module --with-debug --with-threads
> --with-cc-opt='-O0 -g'
> 
> Thanks!
> 

This is very old version of nginx.  First of all, you should update up to
the supported version.  There are a bunch of bugs have been fixed.

  wbr, Valentin V. Bartenev


From nginx-forum at forum.nginx.org  Mon Jul  3 06:46:39 2017
From: nginx-forum at forum.nginx.org (Vishnu Priya Matha)
Date: Mon, 03 Jul 2017 02:46:39 -0400
Subject: Issues with limit_req_zone.
In-Reply-To: <20170509224250.GB10157@daoine.org>
References: <20170509224250.GB10157@daoine.org>
Message-ID: <a731775ab01d7b7e4aea3711e8d26732.NginxMailingListEnglish@forum.nginx.org>

Then how does the burst_size play a role here ? How is the burst_size be
calculated ? 

Since requests_per_sec  is 100/s => 1 request per 0.01 sec - then does that
mean 50 is also 50 per 0.01 sec or is it 1 per 0.02 sec ?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,274089,275256#msg-275256


From nginx-forum at forum.nginx.org  Mon Jul  3 08:57:31 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Mon, 03 Jul 2017 04:57:31 -0400
Subject: set_real_ip_from,
 real_ip_header directive in ngx_http_realip_module
In-Reply-To: <20170629153302.GL55433@mdounin.ru>
References: <20170629153302.GL55433@mdounin.ru>
Message-ID: <59f46e8d9a8e4fc1abc1686d1f5a087e.NginxMailingListEnglish@forum.nginx.org>

hi Maxim,

Thinks for you reply.
i got a problem on http_realip_module, as what you said, duplicate addresses
occurred in that header.
if i want to get the real ip for access limiting, and append the last hop
proxy address in X-Forwarded-Fro header at the same time, what should i do?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,272653,275258#msg-275258


From shahzaib.cb at gmail.com  Mon Jul  3 10:45:44 2017
From: shahzaib.cb at gmail.com (shahzaib mushtaq)
Date: Mon, 3 Jul 2017 15:45:44 +0500
Subject: Client timed out errors !!
Message-ID: <CAD3xhrNnNNhkKJjb89DcGikwg+8NKrRfTTemOXrf2f5OdW8s1w@mail.gmail.com>

Hi,

We're seeing following info logs during serving mp4 videos via nginx :

2017/07/03 15:42:10 [info] 14725#100906: *964419 client timed out (60:
Operation timed out) while sending mp4 to client,

Is there anything we can do to fix it ?

Thanks in advance !!

Shahzaib
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170703/fe20ba21/attachment.html>

From georgi at serversolution.info  Mon Jul  3 13:03:31 2017
From: georgi at serversolution.info (Georgi Georgiev)
Date: Mon, 3 Jul 2017 16:03:31 +0300
Subject: Custom error_log format
Message-ID: <473E91C4-2D31-4797-990D-23083A0CB118@serversolution.info>

Hello,
I would like to have / to add the $request_id variable in the error_log, but I read that the only possible way is to add it to the source code. Has anyone here an experience with that, which file and what should I add? Or some other workaround?

From mdounin at mdounin.ru  Mon Jul  3 13:09:19 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 3 Jul 2017 16:09:19 +0300
Subject: set_real_ip_from,
 real_ip_header directive in ngx_http_realip_module
In-Reply-To: <59f46e8d9a8e4fc1abc1686d1f5a087e.NginxMailingListEnglish@forum.nginx.org>
References: <20170629153302.GL55433@mdounin.ru>
 <59f46e8d9a8e4fc1abc1686d1f5a087e.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170703130919.GT55433@mdounin.ru>

Hello!

On Mon, Jul 03, 2017 at 04:57:31AM -0400, foxgab wrote:

> Thinks for you reply.
> i got a problem on http_realip_module, as what you said, duplicate addresses
> occurred in that header.
> if i want to get the real ip for access limiting, and append the last hop
> proxy address in X-Forwarded-Fro header at the same time, what should i do?

There are two basic options:

1. Avoid using the realip module.  You can still do access checks 
using the "if" directive - and, for example, appropriate geo 
blocks.

2. Avoid using $proxy_add_x_forwarded_for.  You may add the 
original address yourself, using the $realip_remote_addr variable 
and appropriate map{} blocks.

Alternatively, you may want to rethink your setup to simply avoid 
one or the another.

-- 
Maxim Dounin
http://nginx.org/

From peter_booth at me.com  Mon Jul  3 15:01:15 2017
From: peter_booth at me.com (Peter Booth)
Date: Mon, 03 Jul 2017 11:01:15 -0400
Subject: ngx_http_sub_module causes requests to hang on a simple match.
In-Reply-To: <1762729.p3EqeqSY3a@vbart-workstation>
References: <75023c41f6cd332bcf9dc70c4a23cd05.NginxMailingListEnglish@forum.nginx.org>
 <1762729.p3EqeqSY3a@vbart-workstation>
Message-ID: <1048653C-5D61-4EAD-8AC0-F9FD597AC514@me.com>

What happens if you simplify the match string to only contain characters? Something like 

>> sub_filter 'xxx' 'yyy';

Can it ever do a substitute?

Sent from my iPad

> On Jul 2, 2017, at 8:26 AM, Valentin V. Bartenev <vbart at nginx.com> wrote:
> 
>> On Friday 30 June 2017 22:14:55 ptcell wrote:
>> I've built with the sub filter enabled and I'm finding it hangs requests if
>> there is a match.   It is a very simple substitution/replace.   I've
>> resorted to following the request in GDB and the sub module completes and
>> calls the next body filter (which in my case appears to be the charset
>> module).   I have no other odd modules enabled other than using threads with
>> a thread pool size of two (shouldn't matter, right?).  Pausing all the
>> threads in GDB shows no obvious place it is hanging.
>> 
>> If I change the match string to something that doesn't match anything, the
>> request works fine.
>> 
>> Here is my config:
>> 
>>        location / {
>>            root   html;
>>            index  index.html index.htm;
>>            sub_filter '</title>' 'xxx</title>';
>>            sub_filter_once on;
>>        }
>> 
>> nginx -V
>> nginx version: nginx/1.7.11
>> built by gcc 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 
>> configure arguments: --with-http_sub_module --with-debug --with-threads
>> --with-cc-opt='-O0 -g'
>> 
>> Thanks!
>> 
> 
> This is very old version of nginx.  First of all, you should update up to
> the supported version.  There are a bunch of bugs have been fixed.
> 
>  wbr, Valentin V. Bartenev
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170703/24dbbb09/attachment.html>

From nelsonmarcos at gmail.com  Mon Jul  3 20:09:28 2017
From: nelsonmarcos at gmail.com (Nelson Marcos)
Date: Mon, 3 Jul 2017 17:09:28 -0300
Subject: Does Nginx supports If-Range ?
Message-ID: <CAJaC8Ys-PN=QMrMw7HuOX=6NBF7S3PYtOAyFVhM-ttKKNNqqzA@mail.gmail.com>

Hello everyone!

I don't know if it is an expected behaviour or a bug:


Scenario 1(OK): If I perform a request *with the header Range*,  Nginx
serves the *partial content(HTTP 206)*.

Scenario 2 (NOT OK): If I perform a request *with the header Range AND the
header "If-Range" *with the Etag, Nginx serves the *entire file*(200). Why
not serve the partial content if its cached version matches the If-Range
header?

In both scenarios the file is already cached.

Here is my conf: https://pastebin.com/gQQ0GSg6

Here are my requests and my files: https://pastebin.com/rxLwYaSK


The error happened on my server(Nginx 1.10.2) but I was also able to
reproduce it on my Macbook (nginx 1.12.0).

Thanks for any help :)

Kinds,
NM
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170703/8e3c5de7/attachment.html>

From zchao1995 at gmail.com  Tue Jul  4 01:25:34 2017
From: zchao1995 at gmail.com (Zhang Chao)
Date: Mon, 3 Jul 2017 18:25:34 -0700
Subject: Does Nginx supports If-Range ?
In-Reply-To: <CAJaC8Ys-PN=QMrMw7HuOX=6NBF7S3PYtOAyFVhM-ttKKNNqqzA@mail.gmail.com>
References: <CAJaC8Ys-PN=QMrMw7HuOX=6NBF7S3PYtOAyFVhM-ttKKNNqqzA@mail.gmail.com>
Message-ID: <CAPr95BjNxe6ymBpCq_OnfTFJo6wOqMUy9=4xBkhPiZ9qHndYBw@mail.gmail.com>

Hi!

I don?t know if it is an expected behaviour or a bug:

Scenario 1(OK): If I perform a request with the header Range, Nginx serves
the partial content(HTTP 206).

Scenario 2 (NOT OK): If I perform a request with the header Range AND the
header ?If-Range? with the Etag, Nginx serves theentire file(200). Why not
serve the partial content if its cached version matches the If-Range header?

Make sure that the ETag value in the header If-Range is same as the entity
ETag, don?t forget the double quotation marks. Maybe you need to support
the simple demo.




On 4 July 2017 at 04:10:01, Nelson Marcos (nelsonmarcos at gmail.com) wrote:

Hello everyone!

I don't know if it is an expected behaviour or a bug:


Scenario 1(OK): If I perform a request *with the header Range*,  Nginx
serves the *partial content(HTTP 206)*.

Scenario 2 (NOT OK): If I perform a request *with the header Range AND the
header "If-Range"* with the Etag, Nginx serves the *entire file*(200). Why
not serve the partial content if its cached version matches the If-Range
header?

In both scenarios the file is already cached.

Here is my conf: https://pastebin.com/gQQ0GSg6

Here are my requests and my files: https://pastebin.com/rxLwYaSK


The error happened on my server(Nginx 1.10.2) but I was also able to
reproduce it on my Macbook (nginx 1.12.0).

Thanks for any help :)

Kinds,
NM

_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170703/5b7af5b6/attachment-0001.html>

From nginx-forum at forum.nginx.org  Tue Jul  4 07:33:36 2017
From: nginx-forum at forum.nginx.org (charlesparasa)
Date: Tue, 04 Jul 2017 03:33:36 -0400
Subject: prxy_buffering
Message-ID: <87cb3e2d7901cd9d269a45a8ac0173a8.NginxMailingListEnglish@forum.nginx.org>

I am trying to check the functionality of proxy_buffering . can you please
provide me with some sample test scenario. 
Basically I want to visualise the behaviours of nginx when proxy_buffering
is on and off .
And also please share what are the parameter gets effected when
proxy_buffering is on or off .

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275275,275275#msg-275275


From nginx-forum at forum.nginx.org  Tue Jul  4 08:01:44 2017
From: nginx-forum at forum.nginx.org (guruprasads)
Date: Tue, 04 Jul 2017 04:01:44 -0400
Subject: Nginx Tuning
Message-ID: <3679e1e4c5dd08e1950ac67d7f01ca05.NginxMailingListEnglish@forum.nginx.org>

Hi,

I am trying to tune nginx server.
I want to restrict number of client connection per server and restrict
bandwidth.
I tried 
worker_connections 2; 
for max connections in nginx.conf file. 
but its connecting only after worker_connection value set to 7.

my conf file look like below.

user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

# Load dynamic modules. See /usr/share/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;

events {
    worker_connections 7;
}

thanks.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275276,275276#msg-275276


From vfclists at gmail.com  Tue Jul  4 11:20:52 2017
From: vfclists at gmail.com (vfclists .)
Date: Tue, 4 Jul 2017 12:20:52 +0100
Subject: Is there a module that can prettify the page output before it is send
 to the requesting client?
Message-ID: <CAE3mkNCOQ0bePmnLOpssO__8+KjggWbp+EBni33OT_T17QVU4g@mail.gmail.com>

Is there a module that can prettify the HTML before it gets sent to the
client? Some think like that must be run before the zip stage.

-- 
Frank Church

=======================
http://devblog.brahmancreations.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170704/ba66c340/attachment.html>

From mdounin at mdounin.ru  Tue Jul  4 12:21:14 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 4 Jul 2017 15:21:14 +0300
Subject: Does Nginx supports If-Range ?
In-Reply-To: <CAJaC8Ys-PN=QMrMw7HuOX=6NBF7S3PYtOAyFVhM-ttKKNNqqzA@mail.gmail.com>
References: <CAJaC8Ys-PN=QMrMw7HuOX=6NBF7S3PYtOAyFVhM-ttKKNNqqzA@mail.gmail.com>
Message-ID: <20170704122114.GW55433@mdounin.ru>

Hello!

On Mon, Jul 03, 2017 at 05:09:28PM -0300, Nelson Marcos wrote:

> I don't know if it is an expected behaviour or a bug:
> 
> 
> Scenario 1(OK): If I perform a request *with the header Range*,  Nginx
> serves the *partial content(HTTP 206)*.
> 
> Scenario 2 (NOT OK): If I perform a request *with the header Range AND the
> header "If-Range" *with the Etag, Nginx serves the *entire file*(200). Why
> not serve the partial content if its cached version matches the If-Range
> header?
> 
> In both scenarios the file is already cached.
> 
> Here is my conf: https://pastebin.com/gQQ0GSg6
> 
> Here are my requests and my files: https://pastebin.com/rxLwYaSK

The "ETag" header in the response is invalid, as well as 
"If-Range" in the request.  Quoting from the second link:

: Etag: 345a2dd5c8f22e9ffaf250151ea820df
: If-Range: 345a2dd5c8f22e9ffaf250151ea820df

In both cases entity tag should be in double quotes, see 
https://tools.ietf.org/html/rfc2616#section-14.19.

Fixing your backend to return correct ETag will make things work.  
Alternatively, you can use Last-Modified date in the If-Range 
request header instead.

-- 
Maxim Dounin
http://nginx.org/

From r1ch+nginx at teamliquid.net  Tue Jul  4 12:23:57 2017
From: r1ch+nginx at teamliquid.net (Richard Stanway)
Date: Tue, 4 Jul 2017 14:23:57 +0200
Subject: Nginx Tuning
In-Reply-To: <3679e1e4c5dd08e1950ac67d7f01ca05.NginxMailingListEnglish@forum.nginx.org>
References: <3679e1e4c5dd08e1950ac67d7f01ca05.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <CAKWQeyhau=4Me-4D3tkfwY6uGOFzRLHRWVpNT_X6ewsAFKQrMA@mail.gmail.com>

You shouldn't be changing worker_connections, this is the total number of
connections (of any type) permitted per worker.

Take a look at the documentation at http://nginx.org/en/docs/

Of interest to you are
http://nginx.org/en/docs/http/ngx_http_limit_conn_module.html and
http://nginx.org/en/docs/http/ngx_http_core_module.html#limit_rate

On Tue, Jul 4, 2017 at 10:01 AM, guruprasads <nginx-forum at forum.nginx.org>
wrote:

> Hi,
>
> I am trying to tune nginx server.
> I want to restrict number of client connection per server and restrict
> bandwidth.
> I tried
> worker_connections 2;
> for max connections in nginx.conf file.
> but its connecting only after worker_connection value set to 7.
>
> my conf file look like below.
>
> user nginx;
> worker_processes auto;
> error_log /var/log/nginx/error.log;
> pid /run/nginx.pid;
>
> # Load dynamic modules. See /usr/share/nginx/README.dynamic.
> include /usr/share/nginx/modules/*.conf;
>
> events {
>     worker_connections 7;
> }
>
> thanks.
>
> Posted at Nginx Forum: https://forum.nginx.org/read.
> php?2,275276,275276#msg-275276
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170704/fac97395/attachment.html>

From mdounin at mdounin.ru  Tue Jul  4 12:27:47 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 4 Jul 2017 15:27:47 +0300
Subject: Nginx Tuning
In-Reply-To: <3679e1e4c5dd08e1950ac67d7f01ca05.NginxMailingListEnglish@forum.nginx.org>
References: <3679e1e4c5dd08e1950ac67d7f01ca05.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170704122746.GX55433@mdounin.ru>

Hello!

On Tue, Jul 04, 2017 at 04:01:44AM -0400, guruprasads wrote:

> Hi,
> 
> I am trying to tune nginx server.
> I want to restrict number of client connection per server and restrict
> bandwidth.
> I tried 
> worker_connections 2; 
> for max connections in nginx.conf file. 
> but its connecting only after worker_connection value set to 7.
> 
> my conf file look like below.
> 
> user nginx;
> worker_processes auto;
> error_log /var/log/nginx/error.log;
> pid /run/nginx.pid;
> 
> # Load dynamic modules. See /usr/share/nginx/README.dynamic.
> include /usr/share/nginx/modules/*.conf;
> 
> events {
>     worker_connections 7;
> }
> 
> thanks.

The worker_connections directive is to control number of internal 
connection structures in nginx.  It should not be set to low 
values, as these structures are used in many places in nginx - 
including listening sockets, client connections, connections to 
upstream servers, and so on.  And shortage of these structures 
will result in fatal errors.  Instead, you may want to use large 
numbers if you want to handle reasonable load, as default 512 is 
very small in the modern world.

If you want to limit connections, consider using the limit_conn 
module instead see here:

http://nginx.org/en/docs/http/ngx_http_limit_conn_module.html

To restrict bandwidth on a per-connection basis, there is 
the limit_rate directive (http://nginx.org/r/limit_rate).

-- 
Maxim Dounin
http://nginx.org/

From nelsonmarcos at gmail.com  Tue Jul  4 14:29:45 2017
From: nelsonmarcos at gmail.com (Nelson Marcos)
Date: Tue, 4 Jul 2017 11:29:45 -0300
Subject: Does Nginx supports If-Range ?
In-Reply-To: <20170704122114.GW55433@mdounin.ru>
References: <CAJaC8Ys-PN=QMrMw7HuOX=6NBF7S3PYtOAyFVhM-ttKKNNqqzA@mail.gmail.com>
 <20170704122114.GW55433@mdounin.ru>
Message-ID: <CAJaC8Yun0Yjp9dp-5DvCCkaVzhNqkhzJz9Lm_OXX_vxo5YvMtw@mail.gmail.com>

Thanks Zhang and Maxim!

I'm checking how to fix that on my backend.

Kind,
NM

2017-07-04 9:21 GMT-03:00 Maxim Dounin <mdounin at mdounin.ru>:

> Hello!
>
> On Mon, Jul 03, 2017 at 05:09:28PM -0300, Nelson Marcos wrote:
>
> > I don't know if it is an expected behaviour or a bug:
> >
> >
> > Scenario 1(OK): If I perform a request *with the header Range*,  Nginx
> > serves the *partial content(HTTP 206)*.
> >
> > Scenario 2 (NOT OK): If I perform a request *with the header Range AND
> the
> > header "If-Range" *with the Etag, Nginx serves the *entire file*(200).
> Why
> > not serve the partial content if its cached version matches the If-Range
> > header?
> >
> > In both scenarios the file is already cached.
> >
> > Here is my conf: https://pastebin.com/gQQ0GSg6
> >
> > Here are my requests and my files: https://pastebin.com/rxLwYaSK
>
> The "ETag" header in the response is invalid, as well as
> "If-Range" in the request.  Quoting from the second link:
>
> : Etag: 345a2dd5c8f22e9ffaf250151ea820df
> : If-Range: 345a2dd5c8f22e9ffaf250151ea820df
>
> In both cases entity tag should be in double quotes, see
> https://tools.ietf.org/html/rfc2616#section-14.19.
>
> Fixing your backend to return correct ETag will make things work.
> Alternatively, you can use Last-Modified date in the If-Range
> request header instead.
>
> --
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170704/0bf4266d/attachment.html>

From finid at vivaldi.net  Tue Jul  4 14:37:37 2017
From: finid at vivaldi.net (Dan Edwards)
Date: Tue, 04 Jul 2017 09:37:37 -0500
Subject: map configuration
Message-ID: <64754e6db825ef109c65f6d48ddb9d38@vivaldi.net>

Hello:

Need help understanding this piece of Nginx configuration:


===
map $http_upgrade $connection_upgrade {
     default upgrade;
     '' close;
}
===


What does the last line mean?

TIA,



-- 
-Dan Edwards-

From michael.salmon at ericsson.com  Tue Jul  4 14:50:37 2017
From: michael.salmon at ericsson.com (Michael Salmon)
Date: Tue, 4 Jul 2017 14:50:37 +0000
Subject: map configuration
In-Reply-To: <64754e6db825ef109c65f6d48ddb9d38@vivaldi.net>
References: <64754e6db825ef109c65f6d48ddb9d38@vivaldi.net>
Message-ID: <921F4EE1-E49C-4D8E-BB83-F0767DB98A1A@ericsson.com>

The map is similar to $connection_upgrade = ($http_upgrade == '') ? 'close' : 'upgrade';

Each line in a map is a match and a value with default being used if nothing matches.

/Michael Salmon
SE KI73 03 366 (OLC: 9FFVCX53+C5Q8)
+46 722 184 909

> On 4 Jul 2017, at 16:37:37, Dan Edwards <finid at vivaldi.net> wrote:
> 
> Hello:
> 
> Need help understanding this piece of Nginx configuration:
> 
> 
> ===
> map $http_upgrade $connection_upgrade {
>    default upgrade;
>    '' close;
> }
> ===
> 
> 
> What does the last line mean?
> 
> TIA,
> 
> 
> 
> -- 
> -Dan Edwards-
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx


From peter_booth at me.com  Tue Jul  4 16:55:55 2017
From: peter_booth at me.com (Peter Booth)
Date: Tue, 04 Jul 2017 12:55:55 -0400
Subject: Nginx Tuning
In-Reply-To: <3679e1e4c5dd08e1950ac67d7f01ca05.NginxMailingListEnglish@forum.nginx.org>
References: <3679e1e4c5dd08e1950ac67d7f01ca05.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <026BE68B-9992-4DF2-9AE0-934165918154@me.com>

What is your ultimate goal here? What are you wanting to prevent?

Sent from my iPhone

> On Jul 4, 2017, at 4:01 AM, guruprasads <nginx-forum at forum.nginx.org> wrote:
> 
> Hi,
> 
> I am trying to tune nginx server.
> I want to restrict number of client connection per server and restrict
> bandwidth.
> I tried 
> worker_connections 2; 
> for max connections in nginx.conf file. 
> but its connecting only after worker_connection value set to 7.
> 
> my conf file look like below.
> 
> user nginx;
> worker_processes auto;
> error_log /var/log/nginx/error.log;
> pid /run/nginx.pid;
> 
> # Load dynamic modules. See /usr/share/nginx/README.dynamic.
> include /usr/share/nginx/modules/*.conf;
> 
> events {
>    worker_connections 7;
> }
> 
> thanks.
> 
> Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275276,275276#msg-275276
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

From peter_booth at me.com  Tue Jul  4 17:45:33 2017
From: peter_booth at me.com (Peter Booth)
Date: Tue, 04 Jul 2017 13:45:33 -0400
Subject: Is there a module that can prettify the page output before it is
 send to the requesting client?
In-Reply-To: <CAE3mkNCOQ0bePmnLOpssO__8+KjggWbp+EBni33OT_T17QVU4g@mail.gmail.com>
References: <CAE3mkNCOQ0bePmnLOpssO__8+KjggWbp+EBni33OT_T17QVU4g@mail.gmail.com>
Message-ID: <C0E770F4-65BE-40FC-A32E-35CFBD0194E4@me.com>

Depends on your definition of pretty and what you want to achieve. Are you looking for pretty for a human reader or for a browser?

Google's pagespeed module comes in both apache and nginx flavors and applies a bunch of page optimization transformations to the page and embedded resources. I've seen it reduce download times for an untuned site from 6 seconds to 1.2. But the HTML that's returned has been "prettified" for a browser not for a person to view.

Peter

Sent from my iPhone

> On Jul 4, 2017, at 7:20 AM, vfclists . <vfclists at gmail.com> wrote:
> 
> Is there a module that can prettify the HTML before it gets sent to the client? Some think like that must be run before the zip stage.
> 
> -- 
> Frank Church
> 
> =======================
> http://devblog.brahmancreations.com
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170704/73bad003/attachment.html>

From lists at lazygranch.com  Tue Jul  4 19:51:24 2017
From: lists at lazygranch.com (lists at lazygranch.com)
Date: Tue, 04 Jul 2017 12:51:24 -0700
Subject: Nginx Tuning
In-Reply-To: <026BE68B-9992-4DF2-9AE0-934165918154@me.com>
References: <3679e1e4c5dd08e1950ac67d7f01ca05.NginxMailingListEnglish@forum.nginx.org>
 <026BE68B-9992-4DF2-9AE0-934165918154@me.com>
Message-ID: <20170704195124.5693526.49368.32245@lazygranch.com>

I'd suggest the online guides on anti-DDoSing for NGINX. They cover limiting the number of connections, etc. Of course in reality these schemes would just limit some some kid in the basement from flooding your server rather than a real DDoS attack. But better than nothing, plus what is in those guides is effective for taming aggressive download managers that try to open multiple connections.?


? Original Message ?
From: Peter Booth
Sent: Tuesday, July 4, 2017 9:56 AM
To: nginx at nginx.org
Reply To: nginx at nginx.org
Subject: Re: Nginx Tuning

What is your ultimate goal here? What are you wanting to prevent?

Sent from my iPhone

> On Jul 4, 2017, at 4:01 AM, guruprasads <nginx-forum at forum.nginx.org> wrote:
> 
> Hi,
> 
> I am trying to tune nginx server.
> I want to restrict number of client connection per server and restrict
> bandwidth.
> I tried 
> worker_connections 2; 
> for max connections in nginx.conf file. 
> but its connecting only after worker_connection value set to 7.
> 
> my conf file look like below.
> 
> user nginx;
> worker_processes auto;
> error_log /var/log/nginx/error.log;
> pid /run/nginx.pid;
> 
> # Load dynamic modules. See /usr/share/nginx/README.dynamic.
> include /usr/share/nginx/modules/*.conf;
> 
> events {
> worker_connections 7;
> }
> 
> thanks.
> 
> Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275276,275276#msg-275276
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

From vfclists at gmail.com  Tue Jul  4 21:32:13 2017
From: vfclists at gmail.com (vfclists .)
Date: Tue, 4 Jul 2017 22:32:13 +0100
Subject: Is there a module that can prettify the page output before it is
 send to the requesting client?
In-Reply-To: <C0E770F4-65BE-40FC-A32E-35CFBD0194E4@me.com>
References: <CAE3mkNCOQ0bePmnLOpssO__8+KjggWbp+EBni33OT_T17QVU4g@mail.gmail.com>
 <C0E770F4-65BE-40FC-A32E-35CFBD0194E4@me.com>
Message-ID: <CAE3mkNDQR0YdMZ0wen2ojee3UVjrJ_B6sfvo-WJfKYx7e_WT+g@mail.gmail.com>

On 4 July 2017 at 18:45, Peter Booth <peter_booth at me.com> wrote:

> Depends on your definition of pretty and what you want to achieve. Are you
> looking for pretty for a human reader or for a browser?
>
> Google's pagespeed module comes in both apache and nginx flavors and
> applies a bunch of page optimization transformations to the page and
> embedded resources. I've seen it reduce download times for an untuned site
> from 6 seconds to 1.2. But the HTML that's returned has been "prettified"
> for a browser not for a person to view.
>
> Peter
>
> Sent from my iPhone
>
> On Jul 4, 2017, at 7:20 AM, vfclists . <vfclists at gmail.com> wrote:
>
> Is there a module that can prettify the HTML before it gets sent to the
> client? Some think like that must be run before the zip stage.
>
> --
> Frank Church
>
> =======================
> http://devblog.brahmancreations.com
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>

I need one for a human reader.

-- 
Frank Church

=======================
http://devblog.brahmancreations.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170704/70f7d553/attachment.html>

From nginx-forum at forum.nginx.org  Wed Jul  5 05:41:23 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Wed, 05 Jul 2017 01:41:23 -0400
Subject: proxy_set_header directives don't inherit from father context
Message-ID: <49978e7f9faecb3bd48f97f8d6384958.NginxMailingListEnglish@forum.nginx.org>

i found if i didn't configure any proxy_set_header directives in a context,
it will inherit those directives from father context automatic, but if i set
one in current context,  the inheritance won't work.
my configration is like bellow:

http {
    proxy_set_header X-Forwarded-For $remote_addr;
    proxy_set_header X-Forwarded-Proto $scheme;
    proxy_set_header Host $host;
    server {
        listen 80;
        server_name example.com;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        location / {
             proxy_pass http://huiju_nginx_ppr1;
       }
}

i expected the X-Forwarded-For and X-Forwarded-Proto header will be set, but
it didn't.
what happend?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275301,275301#msg-275301


From nginx-forum at forum.nginx.org  Wed Jul  5 05:48:28 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Wed, 05 Jul 2017 01:48:28 -0400
Subject: map configuration
In-Reply-To: <921F4EE1-E49C-4D8E-BB83-F0767DB98A1A@ericsson.com>
References: <921F4EE1-E49C-4D8E-BB83-F0767DB98A1A@ericsson.com>
Message-ID: <cb2deff3851c9c17b702c640d9639dc7.NginxMailingListEnglish@forum.nginx.org>

why no just pass the $http_upgrade and $http_connection headers to the
backend? like conf bellow:
proxy_set_header Upgrade $http_upgrade
proxy_set_header Connection $http_Connection

and if i set the Connection header to "upgrade" permanently, does it make
any harm?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275283,275302#msg-275302


From me at nanaya.pro  Wed Jul  5 05:57:51 2017
From: me at nanaya.pro (nanaya)
Date: Wed, 05 Jul 2017 14:57:51 +0900
Subject: proxy_set_header directives don't inherit from father context
In-Reply-To: <49978e7f9faecb3bd48f97f8d6384958.NginxMailingListEnglish@forum.nginx.org>
References: <49978e7f9faecb3bd48f97f8d6384958.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <1499234271.126662.1030709392.27C41195@webmail.messagingengine.com>

Hi,

On Wed, Jul 5, 2017, at 14:41, foxgab wrote:
> i expected the X-Forwarded-For and X-Forwarded-Proto header will be set,
> but
> it didn't.
> what happend?
> 


Quite a few of additive configs (proxy_set_header, add_header) are only
inherited if there's no same directive set at current block.

>From documentation [1]:

> These directives are inherited from the previous level if and only if there are no proxy_set_header directives defined on the current level.

[1]
http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_set_header

From nginx-forum at forum.nginx.org  Wed Jul  5 05:59:52 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Wed, 05 Jul 2017 01:59:52 -0400
Subject: map configuration
In-Reply-To: <64754e6db825ef109c65f6d48ddb9d38@vivaldi.net>
References: <64754e6db825ef109c65f6d48ddb9d38@vivaldi.net>
Message-ID: <c015cf6a1dcc960462a9b6f368887f63.NginxMailingListEnglish@forum.nginx.org>

that means, if the value of $http_upgrade is null or that variable doesn't
exist, the value of $connection_upgrade will set to "close", or it will set
to "upgrade" in the other cases.

the $http_upgrade is the value of the "Upgrade" header in the client
request.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275283,275304#msg-275304


From francis at daoine.org  Wed Jul  5 07:11:53 2017
From: francis at daoine.org (Francis Daly)
Date: Wed, 5 Jul 2017 08:11:53 +0100
Subject: Issues with limit_req_zone.
In-Reply-To: <a731775ab01d7b7e4aea3711e8d26732.NginxMailingListEnglish@forum.nginx.org>
References: <20170509224250.GB10157@daoine.org>
 <a731775ab01d7b7e4aea3711e8d26732.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170705071153.GC3000@daoine.org>

On Mon, Jul 03, 2017 at 02:46:39AM -0400, Vishnu Priya Matha wrote:

Hi there,

> Then how does the burst_size play a role here ? How is the burst_size be
> calculated ? 

"burst" means, roughly, "let this many happen quickly before fully
enforcing the one-per-period rule".

> Since requests_per_sec  is 100/s => 1 request per 0.01 sec - then does that
> mean 50 is also 50 per 0.01 sec or is it 1 per 0.02 sec ?

100/s means, roughly, "handle 1, then no more until 0.01s has passed".

With that config, "burst=50" means (again, roughly), "handle (up to)
50, then no more until (up to) 0.5s has passed".

You maintain the overall rate, but allow it be short-term exceeded
followed by a quiet period.

Test it by setting rate=1/s and burst=5.

Send in 10 requests very quickly.

See when they are handled.

Wait a while.

Send in 10 requests, the next one as soon as the previous one gets
a response.

See when they are handled.

	f
-- 
Francis Daly        francis at daoine.org

From francis at daoine.org  Wed Jul  5 07:20:17 2017
From: francis at daoine.org (Francis Daly)
Date: Wed, 5 Jul 2017 08:20:17 +0100
Subject: proxy_set_header directives don't inherit from father context
In-Reply-To: <1499234271.126662.1030709392.27C41195@webmail.messagingengine.com>
References: <49978e7f9faecb3bd48f97f8d6384958.NginxMailingListEnglish@forum.nginx.org>
 <1499234271.126662.1030709392.27C41195@webmail.messagingengine.com>
Message-ID: <20170705072017.GD3000@daoine.org>

On Wed, Jul 05, 2017 at 02:57:51PM +0900, nanaya wrote:
> On Wed, Jul 5, 2017, at 14:41, foxgab wrote:

Hi there,

> > i expected the X-Forwarded-For and X-Forwarded-Proto header will be set,
> > but it didn't.
> > what happend?

Your expectation was wrong.

> Quite a few of additive configs (proxy_set_header, add_header) are only
> inherited if there's no same directive set at current block.

Correct; except I'd say that *all* directives are inherited by
replacement, or not inherited.

There are some pairs of directives where setting one will cause the
other not to be inherited either, but they should be clearly related to
each other.

> > These directives are inherited from the previous level if and only if there are no proxy_set_header directives defined on the current level.

The person writing the documentation gets to decide what should be there;
I think it would be more useful to make it clear that *everything* works
that way, apart from the few exceptions that are not inherited at all
(which are, in the main, the *_pass directives).

	f
-- 
Francis Daly        francis at daoine.org

From francis at daoine.org  Wed Jul  5 07:31:24 2017
From: francis at daoine.org (Francis Daly)
Date: Wed, 5 Jul 2017 08:31:24 +0100
Subject: prxy_buffering
In-Reply-To: <87cb3e2d7901cd9d269a45a8ac0173a8.NginxMailingListEnglish@forum.nginx.org>
References: <87cb3e2d7901cd9d269a45a8ac0173a8.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170705073124.GE3000@daoine.org>

On Tue, Jul 04, 2017 at 03:33:36AM -0400, charlesparasa wrote:

Hi there,

> I am trying to check the functionality of proxy_buffering . can you please
> provide me with some sample test scenario. 

The documentation is at http://nginx.org/r/proxy_buffering

Is there something specific that is not clear, after reading that?

> Basically I want to visualise the behaviours of nginx when proxy_buffering
> is on and off .

There is the client, which talks to nginx; and nginx, which talks to
the http or https upstream.

The link from the client to nginx can be fast or slow (or variable). The
link from nginx to upstream can be fast or slow. Draw a picture of what
happens if one link is fast and the other link is slow, with and without
nginx buffering the response.

Often, the desired aim is that upstream closes the connection to nginx
as quickly as possible, in order that it can handle a new request.

	f
-- 
Francis Daly        francis at daoine.org

From nginx-forum at forum.nginx.org  Wed Jul  5 07:47:01 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Wed, 05 Jul 2017 03:47:01 -0400
Subject: what's the difference between proxy_buffer_size and proxy_buffers
Message-ID: <6147fc2bfdac051945e0e0f1d139fb70.NginxMailingListEnglish@forum.nginx.org>

to explain the proxy_buffer_size directive, the document says:

"Sets the size of the buffer used for reading the first part of the response
received from the proxied server"

what does "first part" mean? that is the first packet or what?

what's the relationship and difference between between proxy_buffer_size and
proxy_buffers?
and how these two directives effect the limitation of
proxy_busy_buffers_size?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275308,275308#msg-275308


From smntov at gmail.com  Wed Jul  5 19:55:24 2017
From: smntov at gmail.com (ST)
Date: Wed, 05 Jul 2017 22:55:24 +0300
Subject: response headers show wrong time
Message-ID: <1499284524.3985.75.camel@gmail.com>

Hello,

response headers show wrong time:

Date: Wed, 05 Jul 2017 19:44:37 GMT

while system time is set to 22:44:37 (output of the 'date' from command
line).

Any ideas where I can set headers' date to system time?

Thank you in advance!
ST


From r at roze.lv  Wed Jul  5 21:33:26 2017
From: r at roze.lv (Reinis Rozitis)
Date: Thu, 6 Jul 2017 00:33:26 +0300
Subject: response headers show wrong time
In-Reply-To: <1499284524.3985.75.camel@gmail.com>
References: <1499284524.3985.75.camel@gmail.com>
Message-ID: <000001d2f5d6$5634c0f0$029e42d0$@roze.lv>

> Hello,
> 
> response headers show wrong time:
> 
> Date: Wed, 05 Jul 2017 19:44:37 GMT
> 
> while system time is set to 22:44:37 (output of the 'date' from command line).
> 
> Any ideas where I can set headers' date to system time?

It's not a wrong time just your system has a GMT+3 timezone or EEST (Eastern European Summer Time) but nginx sends the date in GMT (or UTC) timezone  which is defined in HTTP protocol RFC (http://www.w3.org/Protocols/rfc2616/rfc2616-sec3.html#sec3.3.1)

So there is nothing to set or change.

rr


From nginx-forum at forum.nginx.org  Thu Jul  6 02:34:07 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Wed, 05 Jul 2017 22:34:07 -0400
Subject: "aio on" unsupported on centos6.8
Message-ID: <117d6cf73405377e9ec67e1027eb5cac.NginxMailingListEnglish@forum.nginx.org>

it said ""aio on" is unsupported on this platform in
/usr/local/nginx/conf/nginx.conf:25" when i apply my conf.

see version info bellow:

[root]# /usr/local/nginx/sbin/nginx -v
nginx version: nginx/1.10.3

[root]# lsb_release -a
LSB Version:   
:base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch
Distributor ID: CentOS
Description:    CentOS release 6.8 (Final)
Release:        6.8
Codename:       Final

[root]# uname -a
Linux nginx-ppr1-public-a1 2.6.32-696.3.2.el6.x86_64 #1 SMP Tue Jun 20
01:26:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275320,275320#msg-275320


From nginx-forum at forum.nginx.org  Thu Jul  6 06:19:09 2017
From: nginx-forum at forum.nginx.org (c0nw0nk)
Date: Thu, 06 Jul 2017 02:19:09 -0400
Subject: Nginx Auth Module auth_basic and Flooding/DoS/DDoS
Message-ID: <e5d906151bab97fa6e8a01f473cc4fec.NginxMailingListEnglish@forum.nginx.org>

Here is my config :

http {

limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;
limit_conn_zone $binary_remote_addr zone=addr:10m;

server {


location /secured/ {

auth_basic "secured area";
auth_basic_user_file conf/htpasswd;

limit_req zone=one burst=5;
limit_conn addr 1;

}


}




My question is with the nginx auth module should i still need to protect
that area from flooding / ddos or will the auth module denying access be
enough.

Would like to know how well the auth module processes and does compared to
the limit_req or limit_conn module and if i should keep those in my
configuration or remove them since the auth module could be already doing
all the work.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275321,275321#msg-275321


From mdounin at mdounin.ru  Thu Jul  6 11:55:41 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 6 Jul 2017 14:55:41 +0300
Subject: "aio on" unsupported on centos6.8
In-Reply-To: <117d6cf73405377e9ec67e1027eb5cac.NginxMailingListEnglish@forum.nginx.org>
References: <117d6cf73405377e9ec67e1027eb5cac.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170706115541.GG55433@mdounin.ru>

Hello!

On Wed, Jul 05, 2017 at 10:34:07PM -0400, foxgab wrote:

> it said ""aio on" is unsupported on this platform in
> /usr/local/nginx/conf/nginx.conf:25" when i apply my conf.
> 
> see version info bellow:
> 
> [root]# /usr/local/nginx/sbin/nginx -v
> nginx version: nginx/1.10.3
> 
> [root]# lsb_release -a
> LSB Version:   
> :base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch
> Distributor ID: CentOS
> Description:    CentOS release 6.8 (Final)
> Release:        6.8
> Codename:       Final
> 
> [root]# uname -a
> Linux nginx-ppr1-public-a1 2.6.32-696.3.2.el6.x86_64 #1 SMP Tue Jun 20
> 01:26:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

Make sure nginx you are using was compiled with AIO support 
("--with-file-aio", use "nginx -V" to see configure arguments).

-- 
Maxim Dounin
http://nginx.org/

From nginx-forum at forum.nginx.org  Fri Jul  7 02:43:44 2017
From: nginx-forum at forum.nginx.org (rlx01)
Date: Thu, 06 Jul 2017 22:43:44 -0400
Subject: nginx -s reload terminates connections
Message-ID: <635f48c5d7f72deb6fe8d84a5e0ce252.NginxMailingListEnglish@forum.nginx.org>

Hi,

Using the official 1.12.0 package on debian 9 and the 1.12.0 ports package
on FreeBSD 11-RC1, calling nginx -s reload drops connections.

It's fairly easy to reproduce, I just install the fresh package from
wherever, and serving the default index.html, I run wrk, and in the middle
of the run I call 'nginx -s reload' without actually changing anything:

$ wrk -d 30 -c 100 -t 20 http://nginx-test01.lan/
Running 30s test @ http://nginx-test01.lan/
  20 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    10.06ms    4.10ms  53.02ms   79.07%
    Req/Sec   503.04    161.36   757.00     53.20%
  300801 requests in 30.05s, 243.82MB read
  Socket errors: connect 0, read 100, write 0, timeout 0
Requests/sec:  10008.67
Transfer/sec:      8.11MB

You can see 100 socket read errors in this example.

I tried this with the h2o (2.2.2) server just to see what happens and there
is no error:

$ wrk -d 30 -c 100 -t 20 http://h2o-test01.lan/
Running 30s test @ http://nginx-test01.lan/
  20 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     4.06ms    1.83ms  56.08ms   76.57%
    Req/Sec     1.24k   724.32     6.61k    97.67%
  741569 requests in 30.02s, 599.01MB read
Requests/sec:  24699.67
Transfer/sec:     19.95MB

--

The same issue occurs when I test HTTP/2 with 'h2load'. The h2load test just
stops as soon as I call 'nginx -s reload'.

Is this expected behaviour? If so, is there some configuration directive
that would fix it?

In my real world usage, I have a nginx server running on the edge and it
proxies (HTTP/1.1) to a different nginx server. If I call 'nginx -s reload'
on the internal nginx the external nginx shows "upstream prematurely closed
connection while reading response header from upstream" if it's under heavy
load.

Thanks!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275328,275328#msg-275328


From nginx-forum at forum.nginx.org  Fri Jul  7 05:30:59 2017
From: nginx-forum at forum.nginx.org (JackB)
Date: Fri, 07 Jul 2017 01:30:59 -0400
Subject: nginx -s reload terminates connections
In-Reply-To: <635f48c5d7f72deb6fe8d84a5e0ce252.NginxMailingListEnglish@forum.nginx.org>
References: <635f48c5d7f72deb6fe8d84a5e0ce252.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <e55de460a15fa22cbedea4cdd78c00f9.NginxMailingListEnglish@forum.nginx.org>

With a plain standard configuration I can confirm behavior of wrk on Ubuntu
16.04 (Linux 4.4.0):

Running without reload during run:

$ wrk -d 30 -c 100 -t 20 http://localhsot/
Running 30s test @ http://localhost/
  20 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.45ms    0.89ms  29.75ms   86.77%
    Req/Sec     2.07k   279.09     5.32k    78.41%
  1241732 requests in 30.10s, 1.45GB read
Requests/sec:  41248.07
Transfer/sec:     49.25MB


Calling reload one time during run:

$ wrk -d 30 -c 100 -t 20 http://localhost/
Running 30s test @ http://localhost/
  20 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.34ms    8.76ms  91.81ms   85.10%
    Req/Sec     1.22k     0.86k    5.43k    58.83%
  688865 requests in 30.10s, 822.49MB read
  Socket errors: connect 0, read 39, write 0, timeout 0
Requests/sec:  22887.01
Transfer/sec:     27.33MB


Could not verify a connection termination with a reload during a long
running request. Might be a "problem" with wrk?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275328,275329#msg-275329


From nginx-forum at forum.nginx.org  Fri Jul  7 05:55:34 2017
From: nginx-forum at forum.nginx.org (rlx01)
Date: Fri, 07 Jul 2017 01:55:34 -0400
Subject: nginx -s reload terminates connections
In-Reply-To: <e55de460a15fa22cbedea4cdd78c00f9.NginxMailingListEnglish@forum.nginx.org>
References: <635f48c5d7f72deb6fe8d84a5e0ce252.NginxMailingListEnglish@forum.nginx.org>
 <e55de460a15fa22cbedea4cdd78c00f9.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <75417f3e77a575eedaf74831266e6c02.NginxMailingListEnglish@forum.nginx.org>

Hi,

It's not a problem specific to wrk as we see this with heavy production
traffic where nginx drops real requests. This is just the simplest possible
test case. :)

It also fails with siege, h2load, ab, etc.

Thanks!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275328,275330#msg-275330


From nginx-forum at forum.nginx.org  Fri Jul  7 09:49:18 2017
From: nginx-forum at forum.nginx.org (JackB)
Date: Fri, 07 Jul 2017 05:49:18 -0400
Subject: nginx -s reload terminates connections
In-Reply-To: <75417f3e77a575eedaf74831266e6c02.NginxMailingListEnglish@forum.nginx.org>
References: <635f48c5d7f72deb6fe8d84a5e0ce252.NginxMailingListEnglish@forum.nginx.org>
 <e55de460a15fa22cbedea4cdd78c00f9.NginxMailingListEnglish@forum.nginx.org>
 <75417f3e77a575eedaf74831266e6c02.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <08fe441ce862adbd519bb3144d4a3c9d.NginxMailingListEnglish@forum.nginx.org>

Tested ab and confirmed terminated requests with reloads during run
(complete output pasted at the end).

Failed requests:        180
   (Connect: 0, Receive: 24, Length: 78, Exceptions: 78)

ab reports reports terminated connections while receiving. This should not
happen at all when executing a reload.

But:
If I run ab without keepalive connections, it is not possible for me to get
failed requests by reloading nginx. Maybe the problem is related with
persistent connections.

Does your edge nginx use keepalive connections to access the internal
nginx?




$ ab -n 1000000 -c 20 -k -r http://localhost/
This is ApacheBench, Version 2.3 <$Revision: 1706008 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 10.10.1.171 (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
Completed 1000000 requests
Finished 1000000 requests


Server Software:        nginx
Server Hostname:        127.0.0.1
Server Port:            80

Document Path:          /
Document Length:        973 bytes

Concurrency Level:      20
Time taken for tests:   17.916 seconds
Complete requests:      1000000
Failed requests:        180
   (Connect: 0, Receive: 24, Length: 78, Exceptions: 78)
Keep-Alive requests:    994974
Total transferred:      1251877604 bytes
HTML transferred:       972924106 bytes
Requests per second:    55815.82 [#/sec] (mean)
Time per request:       0.358 [ms] (mean)
Time per request:       0.018 [ms] (mean, across all concurrent requests)
Transfer rate:          68236.90 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.0      0       3
Processing:     0    0   0.4      0      20
Waiting:        0    0   0.4      0      20
Total:          0    0   0.4      0      20

Percentage of the requests served within a certain time (ms)
  50%      0
  66%      0
  75%      0
  80%      0
  90%      1
  95%      1
  98%      1
  99%      1
 100%     20 (longest request)

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275328,275339#msg-275339


From joan.tomas at marfeel.com  Fri Jul  7 09:52:48 2017
From: joan.tomas at marfeel.com (=?UTF-8?Q?Joan_Tom=c3=a0s_i_Buliart?=)
Date: Fri, 7 Jul 2017 11:52:48 +0200
Subject: NGINX stale-while-revalidate cluster
Message-ID: <7d98c323-cbdd-b158-e4b2-260681324ca0@marfeel.com>

Hi,

We are implementing an stale-while-revalidate webserver cluster with NGINX.

We are using the new proxy_cache_background_update to answer request as 
soon as possible while NGINX updates the content from the origin in the 
background. This solution works perfectly when the requests for the same 
object are served by the same NGINX server (when we have only one server 
or when we have a previous load balancer that classifies the requests).

In our scenario we have a round robin load balancer (ELB) and we need to 
scale the webservers layer. So, as a consequence, only the Nginx that 
receive the request updates the cache content while the others keep the 
old version. This means that, we can send old versions of content due to 
the content not being updated on all the webservers. The problem 
accentuates when we put a CDN in front of the webservers.

We are thinking on developing something that once an Nginx instance 
updates its cache would let know all other instances to get a copy of 
the newest content. We are thinking about processing NGINX logs and, 
when it detects a MISS, EXPIRED or UPDATING cache status, it makes a 
HEAD request to the other NGINXs on the cluster to force the 
invalidation of this content.


Do any of you have dealt with this problem or a similar one?

We have also tried the post_action but it is blocking the client request 
until it completes. It is not clear for us which would be the best 
approach. The options that we are considering are:

- NGINX module
- LUA script
- External script that process syslog entries from NGINX

What would be your recommendation?

Many thanks in advance,


-- 
Joan Tom?s-Buliart
www.marfeel.com <http://www.marfeel.com>
Joan Tom?s-Buliart
+34 931 785 950
www.marfeel.com <http://www.marfeel.com>
Discover our referral program!! 
<http://blog.marfeel.com/earn-money-marfeel-referral-program/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170707/4c07d540/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: olhgllimkponlhjm.gif
Type: image/gif
Size: 4180 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170707/4c07d540/attachment-0001.gif>

From lucas at slcoding.com  Fri Jul  7 10:12:35 2017
From: lucas at slcoding.com (Lucas Rolff)
Date: Fri, 07 Jul 2017 12:12:35 +0200
Subject: NGINX stale-while-revalidate cluster
In-Reply-To: <7d98c323-cbdd-b158-e4b2-260681324ca0@marfeel.com>
References: <7d98c323-cbdd-b158-e4b2-260681324ca0@marfeel.com>
Message-ID: <3E009A4E-0D3A-49FF-B7CC-D1EA1A53AEDA@slcoding.com>

Instead of doing round robin load balancing why not do a URI based load balancing? Then you ensure your cached file is only present on a single machine behind the load balancer.

Sure there will be moments where this is not the case ? let's assume that a box goes down, and traffic will switch, but in that case I'd as a "post task" take the moment from when the machine went down, until it came online again, find all requests that expired in the meantime, and flush it to ensure the entry is updated on the machine that had been down in the meantime.

It will still require some work, but at least over time your "overhead" should be less.

From:  nginx <nginx-bounces at nginx.org> on behalf of Joan Tom?s i Buliart <joan.tomas at marfeel.com>
Reply-To:  <nginx at nginx.org>
Date:  Friday, 7 July 2017 at 11.52
To:  <nginx at nginx.org>
Subject:  NGINX stale-while-revalidate cluster

    
 

Hi,
 
 We are implementing an stale-while-revalidate webserver cluster with NGINX. 
 
 We are using the new proxy_cache_background_update to answer request as soon as possible while NGINX updates the content from the origin in the background. This solution works perfectly when the requests for the same object are served by the same NGINX server (when we have only one server or when we have a previous load balancer that classifies the requests). 
 
 In our scenario we have a round robin load balancer (ELB) and we need to scale the webservers layer. So, as a consequence, only the Nginx that receive the request updates the cache content while the others keep the old version. This means that, we can send old versions of content due to the content not being updated on all the webservers. The problem accentuates when we put a CDN in front of the webservers.
 
 We are thinking on developing something that once an Nginx instance updates its cache would let know all other instances to get a copy of the newest content. We are thinking about processing NGINX logs and, when it detects a MISS, EXPIRED or UPDATING cache status, it makes a HEAD request to the other NGINXs on the cluster to force the invalidation of this content.
 
 


 Do any of you have dealt with this problem or a similar one?
 
 
 

We have also tried the post_action but it is blocking the client request until it completes. It is not clear for us which would be the best approach. The options that we are considering are:
 
 - NGINX module
 - LUA script
 - External script that process syslog entries from NGINX
 
 What would be your recommendation?
 
 
 Many thanks in advance,
 
 
  
-- 
  Joan Tom?s-Buliart 

 
 Joan Tom?s-Buliart
 +34 931 785 950
 www.marfeel.com
 Discover our referral program!!
 
 
_______________________________________________ nginx mailing list nginx at nginx.org http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170707/6d747837/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: olhgllimkponlhjm.gif
Type: image/gif
Size: 4180 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170707/6d747837/attachment.gif>

From joan.tomas at marfeel.com  Fri Jul  7 10:30:28 2017
From: joan.tomas at marfeel.com (=?UTF-8?Q?Joan_Tom=c3=a0s_i_Buliart?=)
Date: Fri, 7 Jul 2017 12:30:28 +0200
Subject: NGINX stale-while-revalidate cluster
In-Reply-To: <3E009A4E-0D3A-49FF-B7CC-D1EA1A53AEDA@slcoding.com>
References: <7d98c323-cbdd-b158-e4b2-260681324ca0@marfeel.com>
 <3E009A4E-0D3A-49FF-B7CC-D1EA1A53AEDA@slcoding.com>
Message-ID: <d4abb9d8-f4a2-c2ba-a7e0-f3bb05bc7846@marfeel.com>

Hi Lucas

On 07/07/17 12:12, Lucas Rolff wrote:
> Instead of doing round robin load balancing why not do a URI based 
> load balancing? Then you ensure your cached file is only present on a 
> single machine behind the load balancer.

Yes, we considered this option but it forces us to deploy and maintain 
another layer (LB+NG+AppServer). All cloud providers have round robin 
load balancers out-of-the-box but no one provides URI based load 
balancer. Moreover, in our scenario, our webservers layer is quite 
dynamic due to scaling up/down.

Best,

Joan

From mdounin at mdounin.ru  Fri Jul  7 12:28:48 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Fri, 7 Jul 2017 15:28:48 +0300
Subject: nginx -s reload terminates connections
In-Reply-To: <08fe441ce862adbd519bb3144d4a3c9d.NginxMailingListEnglish@forum.nginx.org>
References: <635f48c5d7f72deb6fe8d84a5e0ce252.NginxMailingListEnglish@forum.nginx.org>
 <e55de460a15fa22cbedea4cdd78c00f9.NginxMailingListEnglish@forum.nginx.org>
 <75417f3e77a575eedaf74831266e6c02.NginxMailingListEnglish@forum.nginx.org>
 <08fe441ce862adbd519bb3144d4a3c9d.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170707122847.GL55433@mdounin.ru>

Hello!

On Fri, Jul 07, 2017 at 05:49:18AM -0400, JackB wrote:

> Tested ab and confirmed terminated requests with reloads during run
> (complete output pasted at the end).
> 
> Failed requests:        180
>    (Connect: 0, Receive: 24, Length: 78, Exceptions: 78)
> 
> ab reports reports terminated connections while receiving. This should not
> happen at all when executing a reload.
> 
> But:
> If I run ab without keepalive connections, it is not possible for me to get
> failed requests by reloading nginx. Maybe the problem is related with
> persistent connections.
> 
> Does your edge nginx use keepalive connections to access the internal
> nginx?

TWIMC:

https://trac.nginx.org/nginx/ticket/1022#comment:1

TL;DR: when using keepalive connections, clients are expected to 
be prepared that a connection can be closed.

-- 
Maxim Dounin
http://nginx.org/

From frank.dias at prodea.com  Fri Jul  7 13:24:59 2017
From: frank.dias at prodea.com (Frank Dias)
Date: Fri, 7 Jul 2017 13:24:59 +0000
Subject: NGINX stale-while-revalidate cluster
Message-ID: <6e27a978-84b1-4e04-8110-b1d3b563cf8f@email.android.com>

Have you thought about using a shared file system for the cache. This way all the nginx 's are looking at the same cached content.

On Jul 7, 2017 5:30 AM, Joan Tom?s i Buliart <joan.tomas at marfeel.com> wrote:
Hi Lucas

On 07/07/17 12:12, Lucas Rolff wrote:
> Instead of doing round robin load balancing why not do a URI based
> load balancing? Then you ensure your cached file is only present on a
> single machine behind the load balancer.

Yes, we considered this option but it forces us to deploy and maintain
another layer (LB+NG+AppServer). All cloud providers have round robin
load balancers out-of-the-box but no one provides URI based load
balancer. Moreover, in our scenario, our webservers layer is quite
dynamic due to scaling up/down.

Best,

Joan
_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

This message is confidential to Prodea unless otherwise indicated or apparent from its nature. This message is directed to the intended recipient only, who may be readily determined by the sender of this message and its contents. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient:(a)any dissemination or copying of this message is strictly prohibited; and(b)immediately notify the sender by return message and destroy any copies of this message in any form(electronic, paper or otherwise) that you have.The delivery of this message and its information is neither intended to be nor constitutes a disclosure or waiver of any trade secrets, intellectual property, attorney work product, or attorney-client communications. The authority of the individual sending this message to legally bind Prodea is neither apparent nor implied,and must be independently verified.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170707/41ae28b9/attachment.html>

From peter_booth at me.com  Fri Jul  7 13:39:03 2017
From: peter_booth at me.com (Peter Booth)
Date: Fri, 07 Jul 2017 09:39:03 -0400
Subject: NGINX stale-while-revalidate cluster
In-Reply-To: <6e27a978-84b1-4e04-8110-b1d3b563cf8f@email.android.com>
References: <6e27a978-84b1-4e04-8110-b1d3b563cf8f@email.android.com>
Message-ID: <CD3424CB-4C96-4DC5-B931-642E0BF98CBE@me.com>

You could do that but it would be bad. Nginx' great performance is based on serving files from a local Fisk and the behavior of a Linux page cache. If you serve from a shared (nfs) filsystem then every request is slower. You shouldn't slow down the common case just to increase cache hit rate.

Sent from my iPhone

> On Jul 7, 2017, at 9:24 AM, Frank Dias <frank.dias at prodea.com> wrote:
> 
> Have you thought about using a shared file system for the cache. This way all the nginx 's are looking at the same cached content.
> 
> On Jul 7, 2017 5:30 AM, Joan Tom?s i Buliart <joan.tomas at marfeel.com> wrote:
> Hi Lucas
> 
> On 07/07/17 12:12, Lucas Rolff wrote:
> > Instead of doing round robin load balancing why not do a URI based 
> > load balancing? Then you ensure your cached file is only present on a 
> > single machine behind the load balancer.
> 
> Yes, we considered this option but it forces us to deploy and maintain 
> another layer (LB+NG+AppServer). All cloud providers have round robin 
> load balancers out-of-the-box but no one provides URI based load 
> balancer. Moreover, in our scenario, our webservers layer is quite 
> dynamic due to scaling up/down.
> 
> Best,
> 
> Joan
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> This message is confidential to Prodea unless otherwise indicated or apparent from its nature. This message is directed to the intended recipient only, who may be readily determined by the sender of this message and its contents. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient:(a)any dissemination or copying of this message is strictly prohibited; and(b)immediately notify the sender by return message and destroy any copies of this message in any form(electronic, paper or otherwise) that you have.The delivery of this message and its information is neither intended to be nor constitutes a disclosure or waiver of any trade secrets, intellectual property, attorney work product, or attorney-client communications. The authority of the individual sending this message to legally bind Prodea is neither apparent nor implied,and must be independently verified.
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170707/93e47347/attachment.html>

From enderulusoy at gmail.com  Fri Jul  7 13:39:47 2017
From: enderulusoy at gmail.com (ender ulusoy)
Date: Fri, 7 Jul 2017 16:39:47 +0300
Subject: How to handle 500 Error on upstream itself,
 While Nginx handle other 5xx errors
Message-ID: <CAESV79ZsNF=gcoEjsy-jZrW++p--HPqcFtLVrNvxKhsyTH6QRQ@mail.gmail.com>

We have a NGINX reverse proxy (clustered) with 45 upstreams (22 domains, 20
subdomains, 11 apps).

Some of our projects hosts apis for some users globally.

Our developers designed special custom 500 responses for special cases and
want to show that messages only for tomcat_api upstream.

They want to serve 500 pages and handle exceptions on tomcat_api upstream
members not on NGINX. But we want to handle all other 5xx errors on NGINX.

As I can not succeeded need a hand right now.

Here is the server configuration follows, could you please help?

Thank you.

server {
  server_name api.hotcoldwarm.com;

  location / {
    add_header X-Frame-Options SAMEORIGIN;
    add_header X-Content-Type-Options nosniff;
    add_header X-XSS-Protection "1; mode=block";
    add_header x-redirect-uri hotcoldwarm;
    add_header Strict-Transport-Security "max-age=31536000;
includeSubdomains; always";
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header Host $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://tomcat_api;
    proxy_next_upstream error timeout http_502 http_503 http_504 http_404;
    proxy_connect_timeout   1;
  }
    listen 80;
    listen 443 ssl;
    ssl_certificate /etc/rapidssl/live/api.hotcoldwarm.com/fullchain.pem;
    ssl_certificate_key /etc/rapidssl/live/api.hotcoldwarm.com/privkey.pem;
    include /etc/rapidssl/options-ssl-nginx.conf;
    if ($scheme != "https") {
        return 301 https://$host$request_uri;
    }

}
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170707/b175e49a/attachment-0001.html>

From owen at nginx.com  Fri Jul  7 14:04:31 2017
From: owen at nginx.com (Owen Garrett)
Date: Fri, 7 Jul 2017 15:04:31 +0100
Subject: NGINX stale-while-revalidate cluster
In-Reply-To: <CD3424CB-4C96-4DC5-B931-642E0BF98CBE@me.com>
References: <6e27a978-84b1-4e04-8110-b1d3b563cf8f@email.android.com>
 <CD3424CB-4C96-4DC5-B931-642E0BF98CBE@me.com>
Message-ID: <2B21C7BA-49CB-424F-BE01-70598ECFBCED@nginx.com>

There are a couple of options described here that you could consider if you want to share your cache between NGINX instances:

https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-1/ <https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-1/> describes a sharded cache approach, where you load-balance by URI across the NGINX cache servers.  You can combine your front-end load balancers and back-end caches onto one tier to reduce your footprint if you wish

https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-2/ <https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-2/> describes an alternative HA (shared) approach that replicates the cache so that there?s no increased load on the origin server if one cache server fails.

It?s not possible to share a cache across instances by using a shared filesystem (e.g. nfs).

---
owen at nginx.com
Skype: owen.garrett
Cell: +44 7764 344779

> On 7 Jul 2017, at 14:39, Peter Booth <peter_booth at me.com> wrote:
> 
> You could do that but it would be bad. Nginx' great performance is based on serving files from a local Fisk and the behavior of a Linux page cache. If you serve from a shared (nfs) filsystem then every request is slower. You shouldn't slow down the common case just to increase cache hit rate.
> 
> Sent from my iPhone
> 
> On Jul 7, 2017, at 9:24 AM, Frank Dias <frank.dias at prodea.com <mailto:frank.dias at prodea.com>> wrote:
> 
>> Have you thought about using a shared file system for the cache. This way all the nginx 's are looking at the same cached content.
>> 
>> On Jul 7, 2017 5:30 AM, Joan Tom?s i Buliart <joan.tomas at marfeel.com <mailto:joan.tomas at marfeel.com>> wrote:
>> Hi Lucas
>> 
>> On 07/07/17 12:12, Lucas Rolff wrote:
>> > Instead of doing round robin load balancing why not do a URI based 
>> > load balancing? Then you ensure your cached file is only present on a 
>> > single machine behind the load balancer.
>> 
>> Yes, we considered this option but it forces us to deploy and maintain 
>> another layer (LB+NG+AppServer). All cloud providers have round robin 
>> load balancers out-of-the-box but no one provides URI based load 
>> balancer. Moreover, in our scenario, our webservers layer is quite 
>> dynamic due to scaling up/down.
>> 
>> Best,
>> 
>> Joan
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx <http://mailman.nginx.org/mailman/listinfo/nginx>
>> 
>> This message is confidential to Prodea unless otherwise indicated or apparent from its nature. This message is directed to the intended recipient only, who may be readily determined by the sender of this message and its contents. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient:(a)any dissemination or copying of this message is strictly prohibited; and(b)immediately notify the sender by return message and destroy any copies of this message in any form(electronic, paper or otherwise) that you have.The delivery of this message and its information is neither intended to be nor constitutes a disclosure or waiver of any trade secrets, intellectual property, attorney work product, or attorney-client communications. The authority of the individual sending this message to legally bind Prodea is neither apparent nor implied,and must be independently verified.
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx <http://mailman.nginx.org/mailman/listinfo/nginx>_______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170707/760dc9f7/attachment.html>

From mailinglisten at simonhoenscheid.de  Fri Jul  7 15:55:20 2017
From: mailinglisten at simonhoenscheid.de (mailinglisten at simonhoenscheid.de)
Date: Fri, 07 Jul 2017 17:55:20 +0200
Subject: Writing Header to Logfile
Message-ID: <4e40909103dc1d416e2e64e33432b1d5@simonhoenscheid.de>

Hello List,

I would like to log a header which is send with the incoming request 
into a custom log field. How can this be done?

Kind Regards
Simon

From nginx-forum at forum.nginx.org  Fri Jul  7 19:41:11 2017
From: nginx-forum at forum.nginx.org (rlx01)
Date: Fri, 07 Jul 2017 15:41:11 -0400
Subject: nginx -s reload terminates connections
In-Reply-To: <20170707122847.GL55433@mdounin.ru>
References: <20170707122847.GL55433@mdounin.ru>
Message-ID: <731deb6691e29829de520f9f33d92c99.NginxMailingListEnglish@forum.nginx.org>

Hi Maxim,

Thank you for the link to the ticket.

IMHO this "during configuration reload existing connections are closed as
soon as they are idle" is kind of misleading as the connection is not idle
since it's being slammed with requests, it's "idle" in the sense that nginx
processed the last request that was in the "queue" for that keepalive
connection.

As this keeps being reported, maybe it would be useful (?) if there was an
option to "drain" keepalive connections (with a hard time limit) before
terminating them.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275328,275369#msg-275369


From nginx-forum at forum.nginx.org  Fri Jul  7 21:28:34 2017
From: nginx-forum at forum.nginx.org (rlx01)
Date: Fri, 07 Jul 2017 17:28:34 -0400
Subject: nginx -s reload terminates connections
In-Reply-To: <731deb6691e29829de520f9f33d92c99.NginxMailingListEnglish@forum.nginx.org>
References: <20170707122847.GL55433@mdounin.ru>
 <731deb6691e29829de520f9f33d92c99.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <27bf3deaa6cbaa60ddbafb1d8173d9ad.NginxMailingListEnglish@forum.nginx.org>

I found worker_shutdown_timeout and the code for graceful shutdown was
fairly easy to follow, so we've patched it to do what we want.

Thanks!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275328,275370#msg-275370


From zchao1995 at gmail.com  Sat Jul  8 05:32:11 2017
From: zchao1995 at gmail.com (Zhang Chao)
Date: Fri, 7 Jul 2017 22:32:11 -0700
Subject: Writing Header to Logfile
In-Reply-To: <4e40909103dc1d416e2e64e33432b1d5@simonhoenscheid.de>
References: <4e40909103dc1d416e2e64e33432b1d5@simonhoenscheid.de>
Message-ID: <CAPr95BhroMQBHevxaR01QmE=QjMJLgWuV=kFu_B8jnoQ4fHDuQ@mail.gmail.com>

I would like to log a header which is send with the incoming request into a
custom log field. How can this be done?

you mean the request header?




On 7 July 2017 at 23:55:36, mailinglisten at simonhoenscheid.de (
mailinglisten at simonhoenscheid.de) wrote:

Hello List,

I would like to log a header which is send with the incoming request
into a custom log field. How can this be done?

Kind Regards
Simon
_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170707/b3d5453e/attachment-0001.html>

From joan.tomas at marfeel.com  Sat Jul  8 13:00:40 2017
From: joan.tomas at marfeel.com (=?UTF-8?Q?Joan_Tom=c3=a0s_i_Buliart?=)
Date: Sat, 8 Jul 2017 15:00:40 +0200
Subject: NGINX stale-while-revalidate cluster
In-Reply-To: <2B21C7BA-49CB-424F-BE01-70598ECFBCED@nginx.com>
References: <6e27a978-84b1-4e04-8110-b1d3b563cf8f@email.android.com>
 <CD3424CB-4C96-4DC5-B931-642E0BF98CBE@me.com>
 <2B21C7BA-49CB-424F-BE01-70598ECFBCED@nginx.com>
Message-ID: <75465980-150b-3fd0-e43c-6cc246d746c4@marfeel.com>

Thanks Owen!

We considered all the options on these 2 documents but, on our 
environment in which is important to use stale-while-revalidate, all of 
them have, at least, one of these drawbacks: or it adds a layer in the 
fast path to the content or it can't guarantee that one request on a 
stale content will force the invalidation off all the copies of this object.

That is the reason for which we are looking for a "background" 
alternative to update the content.

Many thanks in any case,

Joan

On 07/07/17 16:04, Owen Garrett wrote:
> There are a couple of options described here that you could consider 
> if you want to share your cache between NGINX instances:
>
> https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-1/ describes 
> a sharded cache approach, where you load-balance by URI across the 
> NGINX cache servers.  You can combine your front-end load balancers 
> and back-end caches onto one tier to reduce your footprint if you wish
>
> https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-2/ describes 
> an alternative HA (shared) approach that replicates the cache so that 
> there?s no increased load on the origin server if one cache server fails.
>
> It?s not possible to share a cache across instances by using a shared 
> filesystem (e.g. nfs).
>
> ---
> owen at nginx.com <mailto:owen at nginx.com>
> Skype: owen.garrett
> Cell: +44 7764 344779
>
>> On 7 Jul 2017, at 14:39, Peter Booth <peter_booth at me.com 
>> <mailto:peter_booth at me.com>> wrote:
>>
>> You could do that but it would be bad. Nginx' great performance is 
>> based on serving files from a local Fisk and the behavior of a Linux 
>> page cache. If you serve from a shared (nfs) filsystem then every 
>> request is slower. You shouldn't slow down the common case just to 
>> increase cache hit rate.
>>
>> Sent from my iPhone
>>
>> On Jul 7, 2017, at 9:24 AM, Frank Dias <frank.dias at prodea.com 
>> <mailto:frank.dias at prodea.com>> wrote:
>>
>>> Have you thought about using a shared file system for the cache. 
>>> This way all the nginx 's are looking at the same cached content.
>>>
>>> On Jul 7, 2017 5:30 AM, Joan Tom?s i Buliart <joan.tomas at marfeel.com 
>>> <mailto:joan.tomas at marfeel.com>> wrote:
>>>
>>>     Hi Lucas
>>>
>>>     On 07/07/17 12:12, Lucas Rolff wrote:
>>>     > Instead of doing round robin load balancing why not do a URI
>>>     based
>>>     > load balancing? Then you ensure your cached file is only
>>>     present on a
>>>     > single machine behind the load balancer.
>>>
>>>     Yes, we considered this option but it forces us to deploy and
>>>     maintain
>>>     another layer (LB+NG+AppServer). All cloud providers have round
>>>     robin
>>>     load balancers out-of-the-box but no one provides URI based load
>>>     balancer. Moreover, in our scenario, our webservers layer is quite
>>>     dynamic due to scaling up/down.
>>>
>>>     Best,
>>>
>>>     Joan
>>>     _______________________________________________
>>>     nginx mailing list
>>>     nginx at nginx.org <mailto:nginx at nginx.org>
>>>     http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>>
>>> This message is confidential to Prodea unless otherwise indicated or 
>>> apparent from its nature. This message is directed to the intended 
>>> recipient only, who may be readily determined by the sender of this 
>>> message and its contents. If the reader of this message is not the 
>>> intended recipient, or an employee or agent responsible for 
>>> delivering this message to the intended recipient:(a)any 
>>> dissemination or copying of this message is strictly prohibited; 
>>> and(b)immediately notify the sender by return message and destroy 
>>> any copies of this message in any form(electronic, paper or 
>>> otherwise) that you have.The delivery of this message and its 
>>> information is neither intended to be nor constitutes a disclosure 
>>> or waiver of any trade secrets, intellectual property, attorney work 
>>> product, or attorney-client communications. The authority of the 
>>> individual sending this message to legally bind Prodea is neither 
>>> apparent nor implied,and must be independently verified.
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170708/9b3b6bcf/attachment.html>

From peter_booth at me.com  Sat Jul  8 13:30:15 2017
From: peter_booth at me.com (Peter Booth)
Date: Sat, 08 Jul 2017 09:30:15 -0400
Subject: NGINX stale-while-revalidate cluster
In-Reply-To: <75465980-150b-3fd0-e43c-6cc246d746c4@marfeel.com>
References: <6e27a978-84b1-4e04-8110-b1d3b563cf8f@email.android.com>
 <CD3424CB-4C96-4DC5-B931-642E0BF98CBE@me.com>
 <2B21C7BA-49CB-424F-BE01-70598ECFBCED@nginx.com>
 <75465980-150b-3fd0-e43c-6cc246d746c4@marfeel.com>
Message-ID: <D0B2CE77-DD4E-4D0E-BE48-5AF0EBF8B4C2@me.com>

Perhaps it would help if, rather than focus on the specific solution that you are wanting, you instead explained your specific problem and business context?

What is driving your architecture? Is it about protecting a backend that doesn't scale or more about reducing latencies?

How many different requests are there that might be cached? What are the backend calls doing? How do cached objects expire? How long does a call to the backend take? 
Why is it OK to return a stale version of X to the first client but not OK to return a stale version to a second requester?

Imagine a scenario where two identical requests arrive from different clients and hit different web servers. Is it OK for both requests to be satisfied with a stale resource?

It's very easy for us to make incorrect assumptions about all of these questions because of our own experiences.

Peter

Sent from my iPhone

> On Jul 8, 2017, at 9:00 AM, Joan Tom?s i Buliart <joan.tomas at marfeel.com> wrote:
> 
> Thanks Owen!
> 
> We considered all the options on these 2 documents but, on our environment in which is important to use stale-while-revalidate, all of them have, at least, one of these drawbacks: or it adds a layer in the fast path to the content or it can't guarantee that one request on a stale content will force the invalidation off all the copies of this object.
> 
> That is the reason for which we are looking for a "background" alternative to update the content.
> 
> Many thanks in any case,
> 
> Joan
> 
>> On 07/07/17 16:04, Owen Garrett wrote:
>> There are a couple of options described here that you could consider if you want to share your cache between NGINX instances:
>> 
>> https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-1/ describes a sharded cache approach, where you load-balance by URI across the NGINX cache servers.  You can combine your front-end load balancers and back-end caches onto one tier to reduce your footprint if you wish
>> 
>> https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-2/ describes an alternative HA (shared) approach that replicates the cache so that there?s no increased load on the origin server if one cache server fails.
>> 
>> It?s not possible to share a cache across instances by using a shared filesystem (e.g. nfs).
>> 
>> ---
>> owen at nginx.com
>> Skype: owen.garrett
>> Cell: +44 7764 344779
>> 
>>> On 7 Jul 2017, at 14:39, Peter Booth <peter_booth at me.com> wrote:
>>> 
>>> You could do that but it would be bad. Nginx' great performance is based on serving files from a local Fisk and the behavior of a Linux page cache. If you serve from a shared (nfs) filsystem then every request is slower. You shouldn't slow down the common case just to increase cache hit rate.
>>> 
>>> Sent from my iPhone
>>> 
>>> On Jul 7, 2017, at 9:24 AM, Frank Dias <frank.dias at prodea.com> wrote:
>>> 
>>>> Have you thought about using a shared file system for the cache. This way all the nginx 's are looking at the same cached content.
>>>> 
>>>> On Jul 7, 2017 5:30 AM, Joan Tom?s i Buliart <joan.tomas at marfeel.com> wrote:
>>>> Hi Lucas
>>>> 
>>>> On 07/07/17 12:12, Lucas Rolff wrote:
>>>> > Instead of doing round robin load balancing why not do a URI based 
>>>> > load balancing? Then you ensure your cached file is only present on a 
>>>> > single machine behind the load balancer.
>>>> 
>>>> Yes, we considered this option but it forces us to deploy and maintain 
>>>> another layer (LB+NG+AppServer). All cloud providers have round robin 
>>>> load balancers out-of-the-box but no one provides URI based load 
>>>> balancer. Moreover, in our scenario, our webservers layer is quite 
>>>> dynamic due to scaling up/down.
>>>> 
>>>> Best,
>>>> 
>>>> Joan
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx at nginx.org
>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>> 
>>>> This message is confidential to Prodea unless otherwise indicated or apparent from its nature. This message is directed to the intended recipient only, who may be readily determined by the sender of this message and its contents. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient:(a)any dissemination or copying of this message is strictly prohibited; and(b)immediately notify the sender by return message and destroy any copies of this message in any form(electronic, paper or otherwise) that you have.The delivery of this message and its information is neither intended to be nor constitutes a disclosure or waiver of any trade secrets, intellectual property, attorney work product, or attorney-client communications. The authority of the individual sending this message to legally bind Prodea is neither apparent nor implied,and must be independently verified.
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx at nginx.org
>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>> 
>> 
>> 
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170708/862cdd33/attachment-0001.html>

From liulantao at gmail.com  Sat Jul  8 17:33:54 2017
From: liulantao at gmail.com (Liu Lantao)
Date: Sun, 9 Jul 2017 01:33:54 +0800
Subject: Writing Header to Logfile
In-Reply-To: <4e40909103dc1d416e2e64e33432b1d5@simonhoenscheid.de>
References: <4e40909103dc1d416e2e64e33432b1d5@simonhoenscheid.de>
Message-ID: <018D0189-FB36-4D41-900A-ED704F20DD85@gmail.com>

According to nginx document (http://nginx.org/en/docs/http/ngx_http_core_module.html#var_http_):

---
$http_name
arbitrary request header field; the last part of a variable name is the field name converted to lower case with dashes replaced by underscores
???

if the request header name is 'X-Custom-Header', then the variable name is ?$http_x_custom_header?.



> On Jul 7, 2017, at 11:55 PM, mailinglisten at simonhoenscheid.de wrote:
> 
> Hello List,
> 
> I would like to log a header which is send with the incoming request into a custom log field. How can this be done?
> 
> Kind Regards
> Simon
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx


From joan.tomas at marfeel.com  Sat Jul  8 20:28:54 2017
From: joan.tomas at marfeel.com (=?UTF-8?Q?Joan_Tom=c3=a0s_i_Buliart?=)
Date: Sat, 8 Jul 2017 22:28:54 +0200
Subject: NGINX stale-while-revalidate cluster
In-Reply-To: <D0B2CE77-DD4E-4D0E-BE48-5AF0EBF8B4C2@me.com>
References: <6e27a978-84b1-4e04-8110-b1d3b563cf8f@email.android.com>
 <CD3424CB-4C96-4DC5-B931-642E0BF98CBE@me.com>
 <2B21C7BA-49CB-424F-BE01-70598ECFBCED@nginx.com>
 <75465980-150b-3fd0-e43c-6cc246d746c4@marfeel.com>
 <D0B2CE77-DD4E-4D0E-BE48-5AF0EBF8B4C2@me.com>
Message-ID: <bfdda3d1-c1bb-b54e-02b1-90c383348ca3@marfeel.com>

Hi Peter,


yes, it's true. I will try to explain our problem better.

We provide a mobile solution for newspaper and media groups. With this 
kind of partners, it is easy to have a peak of traffic. We prefer to 
give stale content (1 or 2 minutes stale content, not more) instead of 
block the request for some seconds (the time that our tomcat back-end 
could expend to crawl our customers desktop site and generate the new 
content). As I tried to explain in my first e-mail, the 
proxy_cache_background_update 
<http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_background_update> 
works ok while the number of servers is fix and the LB in front of them 
does a URI load balancer.

The major problem appears when the servers has to scale up and scale 
down. Imagine that the URL1 is cache by server 1. All the request for 
URL1 are redirected to Server1 by the LB. Suddenly, the traffic raise up 
and a new server is added. The LB will remap the request in order to 
send some URLs to server 2. The URL1 is one of this group of URL that 
goes to server 2. Some hours later, the traffic goes down and the server 
2 is removed. In this situation, the new request that arrive to Server 1 
asking for URL1 will receive the version of some hours before (not some 
minutes). This is what we are trying to avoid.

Many thanks for all your feedback and suggestions,


Joan

Joan Tom?s-Buliart
On 08/07/17 15:30, Peter Booth wrote:
> Perhaps it would help if, rather than focus on the specific solution 
> that you are wanting, you instead explained your specific problem and 
> business context?
>
> What is driving your architecture? Is it about protecting a backend 
> that doesn't scale or more about reducing latencies?
>
> How many different requests are there that might be cached? What are 
> the backend calls doing? How do cached objects expire? How long does a 
> call to the backend take?
> Why is it OK to return a stale version of X to the first client but 
> not OK to return a stale version to a second requester?
>
> Imagine a scenario where two identical requests arrive from different 
> clients and hit different web servers. Is it OK for both requests to 
> be satisfied with a stale resource?
>
> It's very easy for us to make incorrect assumptions about all of these 
> questions because of our own experiences.
>
> Peter
>
> Sent from my iPhone
>
> On Jul 8, 2017, at 9:00 AM, Joan Tom?s i Buliart 
> <joan.tomas at marfeel.com <mailto:joan.tomas at marfeel.com>> wrote:
>
>> Thanks Owen!
>>
>> We considered all the options on these 2 documents but, on our 
>> environment in which is important to use stale-while-revalidate, all 
>> of them have, at least, one of these drawbacks: or it adds a layer in 
>> the fast path to the content or it can't guarantee that one request 
>> on a stale content will force the invalidation off all the copies of 
>> this object.
>>
>> That is the reason for which we are looking for a "background" 
>> alternative to update the content.
>>
>> Many thanks in any case,
>>
>> Joan
>>
>> On 07/07/17 16:04, Owen Garrett wrote:
>>> There are a couple of options described here that you could consider 
>>> if you want to share your cache between NGINX instances:
>>>
>>> https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-1/ describes 
>>> a sharded cache approach, where you load-balance by URI across the 
>>> NGINX cache servers.  You can combine your front-end load balancers 
>>> and back-end caches onto one tier to reduce your footprint if you wish
>>>
>>> https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-2/ describes 
>>> an alternative HA (shared) approach that replicates the cache so 
>>> that there?s no increased load on the origin server if one cache 
>>> server fails.
>>>
>>> It?s not possible to share a cache across instances by using a 
>>> shared filesystem (e.g. nfs).
>>>
>>> ---
>>> owen at nginx.com <mailto:owen at nginx.com>
>>> Skype: owen.garrett
>>> Cell: +44 7764 344779
>>>
>>>> On 7 Jul 2017, at 14:39, Peter Booth <peter_booth at me.com 
>>>> <mailto:peter_booth at me.com>> wrote:
>>>>
>>>> You could do that but it would be bad. Nginx' great performance is 
>>>> based on serving files from a local Fisk and the behavior of a 
>>>> Linux page cache. If you serve from a shared (nfs) filsystem then 
>>>> every request is slower. You shouldn't slow down the common case 
>>>> just to increase cache hit rate.
>>>>
>>>> Sent from my iPhone
>>>>
>>>> On Jul 7, 2017, at 9:24 AM, Frank Dias <frank.dias at prodea.com 
>>>> <mailto:frank.dias at prodea.com>> wrote:
>>>>
>>>>> Have you thought about using a shared file system for the cache. 
>>>>> This way all the nginx 's are looking at the same cached content.
>>>>>
>>>>> On Jul 7, 2017 5:30 AM, Joan Tom?s i Buliart 
>>>>> <joan.tomas at marfeel.com <mailto:joan.tomas at marfeel.com>> wrote:
>>>>>
>>>>>     Hi Lucas
>>>>>
>>>>>     On 07/07/17 12:12, Lucas Rolff wrote:
>>>>>     > Instead of doing round robin load balancing why not do a URI
>>>>>     based
>>>>>     > load balancing? Then you ensure your cached file is only
>>>>>     present on a
>>>>>     > single machine behind the load balancer.
>>>>>
>>>>>     Yes, we considered this option but it forces us to deploy and
>>>>>     maintain
>>>>>     another layer (LB+NG+AppServer). All cloud providers have
>>>>>     round robin
>>>>>     load balancers out-of-the-box but no one provides URI based load
>>>>>     balancer. Moreover, in our scenario, our webservers layer is
>>>>>     quite
>>>>>     dynamic due to scaling up/down.
>>>>>
>>>>>     Best,
>>>>>
>>>>>     Joan
>>>>>     _______________________________________________
>>>>>     nginx mailing list
>>>>>     nginx at nginx.org <mailto:nginx at nginx.org>
>>>>>     http://mailman.nginx.org/mailman/listinfo/nginx
>>>>>
>>>>>
>>>>> This message is confidential to Prodea unless otherwise indicated 
>>>>> or apparent from its nature. This message is directed to the 
>>>>> intended recipient only, who may be readily determined by the 
>>>>> sender of this message and its contents. If the reader of this 
>>>>> message is not the intended recipient, or an employee or agent 
>>>>> responsible for delivering this message to the intended 
>>>>> recipient:(a)any dissemination or copying of this message is 
>>>>> strictly prohibited; and(b)immediately notify the sender by return 
>>>>> message and destroy any copies of this message in any 
>>>>> form(electronic, paper or otherwise) that you have.The delivery of 
>>>>> this message and its information is neither intended to be nor 
>>>>> constitutes a disclosure or waiver of any trade secrets, 
>>>>> intellectual property, attorney work product, or attorney-client 
>>>>> communications. The authority of the individual sending this 
>>>>> message to legally bind Prodea is neither apparent nor implied,and 
>>>>> must be independently verified.
>>>>> _______________________________________________
>>>>> nginx mailing list
>>>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>>
>>>
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170708/a8404149/attachment-0001.html>

From ng23 at firemail.cc  Sun Jul  9 17:43:05 2017
From: ng23 at firemail.cc (Johan Andersson)
Date: Sun, 09 Jul 2017 19:43:05 +0200
Subject: Flushing responses in nginx modules
Message-ID: <7063b3f64e0aad4bed1fe3ece15f832d@firemail.cc>

Hi everyone,

I have some issues writing my nginx modules.

I am on Debian Stretch, installed nginx with the default configuration, 
and took the hello_world module. It works without a hitch. Then I 
changed the handler to send three "hello world" responses, and sleep for 
one second between each response.

However, when I look at the result in my browser, the page loads, pauses 
for three seconds, and then displays all three "hello world" messages at 
once.

Actually I was flushing each response, so I expected each "hello world" 
message to appear one after the other, with one second pause between 
them.

Am I doing something wrong? Is this event the correct way to achieve 
this? All functions return NGX_OK. This is my code:

static ngx_int_t ngx_http_hello_world_handler(ngx_http_request_t *r)
{
     ngx_buf_t *b;
     ngx_chain_t out;
     ngx_int_t result;

     r->headers_out.content_type.len = sizeof("text/html") - 1;
     r->headers_out.content_type.data = (u_char *) "text/html";
     r->headers_out.status = NGX_HTTP_OK;
     //r->headers_out.content_length_n = sizeof(ngx_hello_world);
     ngx_http_send_header(r);

     for(int i = 0; i < 3; i++)
     {
         b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t));

         out.buf = b;
         out.next = NULL;

         b->pos = ngx_hello_world;
         b->last = ngx_hello_world + sizeof(ngx_hello_world);
         b->memory = 1;
         b->flush = 1;
         b->last_buf = (i == 2);

         result = ngx_http_output_filter(r, &out);
         ngx_http_send_special(r, NGX_HTTP_FLUSH);

         sleep(1);
     }

     return result;
}

Cheers
Johann

From sca at andreasschulze.de  Sun Jul  9 20:38:01 2017
From: sca at andreasschulze.de (A. Schulze)
Date: Sun, 9 Jul 2017 22:38:01 +0200
Subject: Flushing responses in nginx modules
In-Reply-To: <7063b3f64e0aad4bed1fe3ece15f832d@firemail.cc>
References: <7063b3f64e0aad4bed1fe3ece15f832d@firemail.cc>
Message-ID: <de8488e3-448d-c8d8-621b-df46fc8159b8@andreasschulze.de>



Am 09.07.2017 um 19:43 schrieb Johan Andersson:

> Actually I was flushing each response, so I expected each "hello world" message to appear one after the other, with one second pause between them.
You may have a look at https://github.com/openresty/echo-nginx-module
As far as I know they solved similar problems.

Andreas

From peter_booth at me.com  Sun Jul  9 20:58:59 2017
From: peter_booth at me.com (Peter Booth)
Date: Sun, 09 Jul 2017 16:58:59 -0400
Subject: NGINX stale-while-revalidate cluster
In-Reply-To: <bfdda3d1-c1bb-b54e-02b1-90c383348ca3@marfeel.com>
References: <6e27a978-84b1-4e04-8110-b1d3b563cf8f@email.android.com>
 <CD3424CB-4C96-4DC5-B931-642E0BF98CBE@me.com>
 <2B21C7BA-49CB-424F-BE01-70598ECFBCED@nginx.com>
 <75465980-150b-3fd0-e43c-6cc246d746c4@marfeel.com>
 <D0B2CE77-DD4E-4D0E-BE48-5AF0EBF8B4C2@me.com>
 <bfdda3d1-c1bb-b54e-02b1-90c383348ca3@marfeel.com>
Message-ID: <3EC8F0DB-E170-4535-83C7-506D5FD19A70@me.com>

stale-while-revalidate is awesome, but it might not be the optimal tool here. It came out of Yahoo!, 
 the sixth largest website in the world, who used a small number of caching proxies. In their context  
most content is served hot from cache. A cloud deployment typically means a larger number of VMs 
that are each a fraction of  a physical server. Great for fine grained control but a problem for cache hit rates. 
So if you have much less traffic than Yahoo spread across a larger number of web servers then your hit rates
will suffer. What hit rates do you see today?

Dynamic scale out isn?t very compatible with caching reverse proxies. 
Can you separate the caching reverse proxy functionality from the other functionality
and keep the number of caches constant, whilst scaling out the web servers?

Your give the example of a few hour old page being served because of a scale out event. Is that the most
 common case of cache misses in your context or is it unpopular  pages and quiet times of the day?
Are these also served stale even when your server count is static?

Finally, if the root of the problem is serving very stale content, could you simply delete that content 
throughout the day? A script that finds and removes all cached files older than five minutes wouldn?t
 take long to run.

Peter


> On 8 Jul 2017, at 4:28 PM, Joan Tom?s i Buliart <joan.tomas at marfeel.com> wrote:
> 
> Hi Peter,
> 
> 
> yes, it's true. I will try to explain our problem better.
> 
> We provide a mobile solution for newspaper and media groups. With this kind of partners, it is easy to have a peak of traffic. We prefer to give stale content (1 or 2 minutes stale content, not more) instead of block the request for some seconds (the time that our tomcat back-end could expend to crawl our customers desktop site and generate the new content). As I tried to explain in my first e-mail, the proxy_cache_background_update <http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_background_update> works ok while the number of servers is fix and the LB in front of them does a URI load balancer.
> 
> The major problem appears when the servers has to scale up and scale down. Imagine that the URL1 is cache by server 1. All the request for URL1 are redirected to Server1 by the LB. Suddenly, the traffic raise up and a new server is added. The LB will remap the request in order to send some URLs to server 2. The URL1 is one of this group of URL that goes to server 2. Some hours later, the traffic goes down and the server 2 is removed. In this situation, the new request that arrive to Server 1 asking for URL1 will receive the version of some hours before (not some minutes). This is what we are trying to avoid.
> 
> Many thanks for all your feedback and suggestions,
> 
> 
> Joan
> On 08/07/17 15:30, Peter Booth wrote:
>> Perhaps it would help if, rather than focus on the specific solution that you are wanting, you instead explained your specific problem and business context?
>> 
>> What is driving your architecture? Is it about protecting a backend that doesn't scale or more about reducing latencies?
>> 
>> How many different requests are there that might be cached? What are the backend calls doing? How do cached objects expire? How long does a call to the backend take? 
>> Why is it OK to return a stale version of X to the first client but not OK to return a stale version to a second requester?
>> 
>> Imagine a scenario where two identical requests arrive from different clients and hit different web servers. Is it OK for both requests to be satisfied with a stale resource?
>> 
>> It's very easy for us to make incorrect assumptions about all of these questions because of our own experiences.
>> 
>> Peter
>> 
>> Sent from my iPhone
>> 
>> On Jul 8, 2017, at 9:00 AM, Joan Tom?s i Buliart <joan.tomas at marfeel.com <mailto:joan.tomas at marfeel.com>> wrote:
>> 
>>> Thanks Owen!
>>> 
>>> We considered all the options on these 2 documents but, on our environment in which is important to use stale-while-revalidate, all of them have, at least, one of these drawbacks: or it adds a layer in the fast path to the content or it can't guarantee that one request on a stale content will force the invalidation off all the copies of this object.
>>> 
>>> That is the reason for which we are looking for a "background" alternative to update the content.
>>> 
>>> Many thanks in any case,
>>> 
>>> Joan
>>> 
>>> On 07/07/17 16:04, Owen Garrett wrote:
>>>> There are a couple of options described here that you could consider if you want to share your cache between NGINX instances:
>>>> 
>>>> https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-1/ <https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-1/> describes a sharded cache approach, where you load-balance by URI across the NGINX cache servers.  You can combine your front-end load balancers and back-end caches onto one tier to reduce your footprint if you wish
>>>> 
>>>> https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-2/ <https://www.nginx.com/blog/shared-caches-nginx-plus-cache-clusters-part-2/> describes an alternative HA (shared) approach that replicates the cache so that there?s no increased load on the origin server if one cache server fails.
>>>> 
>>>> It?s not possible to share a cache across instances by using a shared filesystem (e.g. nfs).
>>>> 
>>>> ---
>>>> owen at nginx.com <mailto:owen at nginx.com>
>>>> Skype: owen.garrett
>>>> Cell: +44 7764 344779
>>>> 
>>>>> On 7 Jul 2017, at 14:39, Peter Booth <peter_booth at me.com <mailto:peter_booth at me.com>> wrote:
>>>>> 
>>>>> You could do that but it would be bad. Nginx' great performance is based on serving files from a local Fisk and the behavior of a Linux page cache. If you serve from a shared (nfs) filsystem then every request is slower. You shouldn't slow down the common case just to increase cache hit rate.
>>>>> 
>>>>> Sent from my iPhone
>>>>> 
>>>>> On Jul 7, 2017, at 9:24 AM, Frank Dias <frank.dias at prodea.com <mailto:frank.dias at prodea.com>> wrote:
>>>>> 
>>>>>> Have you thought about using a shared file system for the cache. This way all the nginx 's are looking at the same cached content.
>>>>>> 
>>>>>> On Jul 7, 2017 5:30 AM, Joan Tom?s i Buliart <joan.tomas at marfeel.com <mailto:joan.tomas at marfeel.com>> wrote:
>>>>>> Hi Lucas
>>>>>> 
>>>>>> On 07/07/17 12:12, Lucas Rolff wrote:
>>>>>> > Instead of doing round robin load balancing why not do a URI based 
>>>>>> > load balancing? Then you ensure your cached file is only present on a 
>>>>>> > single machine behind the load balancer.
>>>>>> 
>>>>>> Yes, we considered this option but it forces us to deploy and maintain 
>>>>>> another layer (LB+NG+AppServer). All cloud providers have round robin 
>>>>>> load balancers out-of-the-box but no one provides URI based load 
>>>>>> balancer. Moreover, in our scenario, our webservers layer is quite 
>>>>>> dynamic due to scaling up/down.
>>>>>> 
>>>>>> Best,
>>>>>> 
>>>>>> Joan
>>>>>> _______________________________________________
>>>>>> nginx mailing list
>>>>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>>>>> http://mailman.nginx.org/mailman/listinfo/nginx <http://mailman.nginx.org/mailman/listinfo/nginx>
>>>>>> 
>>>>>> This message is confidential to Prodea unless otherwise indicated or apparent from its nature. This message is directed to the intended recipient only, who may be readily determined by the sender of this message and its contents. If the reader of this message is not the intended recipient, or an employee or agent responsible for delivering this message to the intended recipient:(a)any dissemination or copying of this message is strictly prohibited; and(b)immediately notify the sender by return message and destroy any copies of this message in any form(electronic, paper or otherwise) that you have.The delivery of this message and its information is neither intended to be nor constitutes a disclosure or waiver of any trade secrets, intellectual property, attorney work product, or attorney-client communications. The authority of the individual sending this message to legally bind Prodea is neither apparent nor implied,and must be independently verified.
>>>>>> _______________________________________________
>>>>>> nginx mailing list
>>>>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>>>>> http://mailman.nginx.org/mailman/listinfo/nginx <http://mailman.nginx.org/mailman/listinfo/nginx>_______________________________________________
>>>>> nginx mailing list
>>>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>>>> http://mailman.nginx.org/mailman/listinfo/nginx <http://mailman.nginx.org/mailman/listinfo/nginx>
>>>> 
>>>> 
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>>> http://mailman.nginx.org/mailman/listinfo/nginx <http://mailman.nginx.org/mailman/listinfo/nginx>
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>> http://mailman.nginx.org/mailman/listinfo/nginx <http://mailman.nginx.org/mailman/listinfo/nginx>
>> 
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx <http://mailman.nginx.org/mailman/listinfo/nginx>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170709/876d29c7/attachment-0001.html>

From sdnetwork at gmail.com  Sun Jul  9 21:26:47 2017
From: sdnetwork at gmail.com (Arnaud Le-roy)
Date: Sun, 9 Jul 2017 23:26:47 +0200
Subject: Nginx Proxy seems to send twice the same request to the backend
Message-ID: <4A8C8A80-EF80-4CD3-9C0C-240279B64E06@gmail.com>


Hello,

i encountered a strange behaviour with nginx, my backend seems to receive twice the same request from nginx proxy, to be sure that it's not the client that send two request i have had an uuid params to each request.

when the problem occurs in nginx log i found one request in success in access.log

x.x.x.x - - [09/Jul/2017:09:18:33 +0200] "GET /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428 HTTP/1.1" 200 2 "-" "-"

and an other one than generate this log in error.log :

2017/07/09 09:18:31 [error] 38111#38111: *4098505 upstream prematurely closed connection while reading response header from upstream, client: x.x.x.x, server: x.x.com, request: "GET /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428 HTTP/1.1", upstream: "http://172.16.0.11:9092/query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428", host: "x.x.com"

on my backend i can see two request with the same uuid (the two succeed)

{"pid":11424,"level":"info","message":"[API] AUTH1 /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428","timestamp":"2017-07-09 09:18:31.861Z"}
{"pid":11424,"level":"info","message":"[API] AUTH1 /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428","timestamp":"2017-07-09 09:18:33.196Z"}

The client is a node program so i'm sure that it sends only one request with the same uuid (no thread problem ;) 
the nginx serve as simple proxy (no load balancing)

[nginx.conf]

user www-data;
worker_processes  8;
worker_rlimit_nofile 8192;
pid /run/nginx.pid;

events {
	worker_connections 1024;
	# multi_accept on;
}

http {
    
    upstream api {
         keepalive 100;
	 server 172.16.0.11:9092;
    }

    include       mime.types;
    default_type  application/octet-stream;    
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    
    proxy_buffering    off;
    proxy_buffer_size  128k;
    proxy_buffers 100  128k;
    proxy_http_version 1.1;
    
    ### timeouts ###
    resolver_timeout        6;
    client_header_timeout   30;
    client_body_timeout     600;
    send_timeout            10;
    keepalive_timeout       65 20;
    proxy_read_timeout      600;
   
    server {
        listen 443 ssl;
        listen [::]:443 ssl;
        server_name x.x.com;
        include /etc/nginx/nginx_ssl.conf;
	
        client_max_body_size 200M;

        location / {
            proxy_next_upstream off;
            proxy_pass http://api;
            proxy_redirect http://api/ https://$host/;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Connection "";
        }
    }
}

The backend is a simple node server.

the problem occurs randomly, and it happens for sure on nginx/1.10.3 and nginx/1.13.2 on debian/jessie

After some days of research, i found that if i remove the keepalive 100 from upstream configuration there is no longer the problem but i don't understand why ? Maybe somebody can explain me what could hapen ? maybe a misunderstanding about some configuration on keep alive ?

For me it seems to be a problem on nginx, if you can't explain with these information, i can send some debug (nginx-debug) log to you.

thanks in advance.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170709/65b170a0/attachment.html>

From zchao1995 at gmail.com  Mon Jul 10 02:04:37 2017
From: zchao1995 at gmail.com (Zhang Chao)
Date: Sun, 9 Jul 2017 21:04:37 -0500
Subject: Flushing responses in nginx modules
In-Reply-To: <7063b3f64e0aad4bed1fe3ece15f832d@firemail.cc>
References: <7063b3f64e0aad4bed1fe3ece15f832d@firemail.cc>
Message-ID: <CAPr95BgCOaSdVjuMHs+_spOGGaiPq2dqkwsyyYmZ+m_i0xqZAA@mail.gmail.com>

Hello!

You mustn?t use standard sleep function for it will block Nginx?s events
loop, alternatively, you need to put your write event to a timer, set the
proper handler when the timer expires.
BTW, you should always check the return value of ngx_http_send_header and
ngx_http_output_filter.


On 10 July 2017 at 01:43:46, Johan Andersson (ng23 at firemail.cc) wrote:

Hi everyone,

I have some issues writing my nginx modules.

I am on Debian Stretch, installed nginx with the default configuration,
and took the hello_world module. It works without a hitch. Then I
changed the handler to send three "hello world" responses, and sleep for
one second between each response.

However, when I look at the result in my browser, the page loads, pauses
for three seconds, and then displays all three "hello world" messages at
once.

Actually I was flushing each response, so I expected each "hello world"
message to appear one after the other, with one second pause between
them.

Am I doing something wrong? Is this event the correct way to achieve
this? All functions return NGX_OK. This is my code:

static ngx_int_t ngx_http_hello_world_handler(ngx_http_request_t *r)
{
ngx_buf_t *b;
ngx_chain_t out;
ngx_int_t result;

r->headers_out.content_type.len = sizeof("text/html") - 1;
r->headers_out.content_type.data = (u_char *) "text/html";
r->headers_out.status = NGX_HTTP_OK;
//r->headers_out.content_length_n = sizeof(ngx_hello_world);
ngx_http_send_header(r);

for(int i = 0; i < 3; i++)
{
b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t));

out.buf = b;
out.next = NULL;

b->pos = ngx_hello_world;
b->last = ngx_hello_world + sizeof(ngx_hello_world);
b->memory = 1;
b->flush = 1;
b->last_buf = (i == 2);

result = ngx_http_output_filter(r, &out);
ngx_http_send_special(r, NGX_HTTP_FLUSH);

sleep(1);
}

return result;
}

Cheers
Johann
_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170709/151e6eb4/attachment.html>

From nginx-forum at forum.nginx.org  Mon Jul 10 02:55:02 2017
From: nginx-forum at forum.nginx.org (motor)
Date: Sun, 09 Jul 2017 22:55:02 -0400
Subject: Upstream HTTP/2 support
Message-ID: <102558b0b56ca135655aa9fbf8644d4d.NginxMailingListEnglish@forum.nginx.org>

The following post is based on the assumptions that 1. upstream HTTP/2
support is not available in nginx at the moment, and 2. said feature is not
firmly excluded from nginx's future road map :)

We've been internally using an nginx patch (authored by me) which provides
this feature in our nginx instances that front-end etcd servers.
Please let me know if this patch can be a candidate for review and potential
merge.

In case the answer is yes, a heads up: said implementation has been subject
to a few 'compromises' due to internal deadlines, so I might need a fair bit
of technical guidance/discussion to iron out some design/implementation
issues.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275391,275391#msg-275391


From gfrankliu at gmail.com  Mon Jul 10 04:56:01 2017
From: gfrankliu at gmail.com (Frank Liu)
Date: Sun, 9 Jul 2017 21:56:01 -0700
Subject: Upstream HTTP/2 support
In-Reply-To: <102558b0b56ca135655aa9fbf8644d4d.NginxMailingListEnglish@forum.nginx.org>
References: <102558b0b56ca135655aa9fbf8644d4d.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <CAOXPdC-A8+wZKJ3uH2rky5xr_iijym=mujaLCx0Z3jo3um9Zng@mail.gmail.com>

This subject has been discussed here:
https://trac.nginx.org/nginx/ticket/923
I think it is a good idea to have this support.

On Sun, Jul 9, 2017 at 7:55 PM, motor <nginx-forum at forum.nginx.org> wrote:

> The following post is based on the assumptions that 1. upstream HTTP/2
> support is not available in nginx at the moment, and 2. said feature is not
> firmly excluded from nginx's future road map :)
>
> We've been internally using an nginx patch (authored by me) which provides
> this feature in our nginx instances that front-end etcd servers.
> Please let me know if this patch can be a candidate for review and
> potential
> merge.
>
> In case the answer is yes, a heads up: said implementation has been subject
> to a few 'compromises' due to internal deadlines, so I might need a fair
> bit
> of technical guidance/discussion to iron out some design/implementation
> issues.
>
> Posted at Nginx Forum: https://forum.nginx.org/read.
> php?2,275391,275391#msg-275391
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170709/b38a6cb9/attachment-0001.html>

From mdounin at mdounin.ru  Mon Jul 10 12:07:58 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 10 Jul 2017 15:07:58 +0300
Subject: NGINX stale-while-revalidate cluster
In-Reply-To: <bfdda3d1-c1bb-b54e-02b1-90c383348ca3@marfeel.com>
References: <6e27a978-84b1-4e04-8110-b1d3b563cf8f@email.android.com>
 <CD3424CB-4C96-4DC5-B931-642E0BF98CBE@me.com>
 <2B21C7BA-49CB-424F-BE01-70598ECFBCED@nginx.com>
 <75465980-150b-3fd0-e43c-6cc246d746c4@marfeel.com>
 <D0B2CE77-DD4E-4D0E-BE48-5AF0EBF8B4C2@me.com>
 <bfdda3d1-c1bb-b54e-02b1-90c383348ca3@marfeel.com>
Message-ID: <20170710120758.GM55433@mdounin.ru>

Hello!

On Sat, Jul 08, 2017 at 10:28:54PM +0200, Joan Tom?s i Buliart wrote:

> Hi Peter,
> 
> 
> yes, it's true. I will try to explain our problem better.
> 
> We provide a mobile solution for newspaper and media groups. With this 
> kind of partners, it is easy to have a peak of traffic. We prefer to 
> give stale content (1 or 2 minutes stale content, not more) instead of 
> block the request for some seconds (the time that our tomcat back-end 
> could expend to crawl our customers desktop site and generate the new 
> content). As I tried to explain in my first e-mail, the 
> proxy_cache_background_update 
> <http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_background_update> 
> works ok while the number of servers is fix and the LB in front of them 
> does a URI load balancer.
> 
> The major problem appears when the servers has to scale up and scale 
> down. Imagine that the URL1 is cache by server 1. All the request for 
> URL1 are redirected to Server1 by the LB. Suddenly, the traffic raise up 
> and a new server is added. The LB will remap the request in order to 
> send some URLs to server 2. The URL1 is one of this group of URL that 
> goes to server 2. Some hours later, the traffic goes down and the server 
> 2 is removed. In this situation, the new request that arrive to Server 1 
> asking for URL1 will receive the version of some hours before (not some 
> minutes). This is what we are trying to avoid.

This situation is exactly why "stale-while-revalidate=<time>" has 
the "=<time>" in it: to only allow stale content for some limited 
time.  If the response is more stale than the specified time, it 
is not used, and the request is passed to the upstream server 
instead.  That is, the version of some hours before will not be 
returned, exactly as you want to.

Similar behaviour can be achieved by using the "inactive=" 
parameter of the proxy_cache_path directive (for example, when 
using "proxy_cache_use_stale updating").  Responses which were not 
requested (and hence not updated) for the specified time will be 
removed from cache, and so won't be returned in such situation.  
This is slightly less flexible though, as you can't control the 
time on a per-response basis.

-- 
Maxim Dounin
http://nginx.org/

From mdounin at mdounin.ru  Mon Jul 10 12:18:14 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 10 Jul 2017 15:18:14 +0300
Subject: How to handle 500 Error on upstream itself,
 While Nginx handle other 5xx errors
In-Reply-To: <CAESV79ZsNF=gcoEjsy-jZrW++p--HPqcFtLVrNvxKhsyTH6QRQ@mail.gmail.com>
References: <CAESV79ZsNF=gcoEjsy-jZrW++p--HPqcFtLVrNvxKhsyTH6QRQ@mail.gmail.com>
Message-ID: <20170710121814.GN55433@mdounin.ru>

Hello!

On Fri, Jul 07, 2017 at 04:39:47PM +0300, ender ulusoy wrote:

> We have a NGINX reverse proxy (clustered) with 45 upstreams (22 domains, 20
> subdomains, 11 apps).
> 
> Some of our projects hosts apis for some users globally.
> 
> Our developers designed special custom 500 responses for special cases and
> want to show that messages only for tomcat_api upstream.
> 
> They want to serve 500 pages and handle exceptions on tomcat_api upstream
> members not on NGINX. But we want to handle all other 5xx errors on NGINX.
> 
> As I can not succeeded need a hand right now.

Errors interception in the proxy module works as follows:

- Errors are only intercepted if proxy_intercept_errors is set to 
  "on", see http://nginx.org/r/proxy_intercept_errors.  When 
  errors are not intercepted, all errors returned by upstream are sent 
  to the client as is.

- With "proxy_intercept_errors on", nginx will only intercept 
  errors with error_page configured.

That is, if you want to intercept all 5xx errors but not 500, 
consider configuration like this:

    proxy_pass ...
    proxy_intercept_errors on;
    error_page 501 502 503 504 505 /errors/5xx.html;

This will intercept all 5xx errors except 500.

For more information, see the documentation of the directives here:

http://nginx.org/r/proxy_intercept_errors
http://nginx.org/r/error_page

-- 
Maxim Dounin
http://nginx.org/

From mdounin at mdounin.ru  Mon Jul 10 12:51:53 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 10 Jul 2017 15:51:53 +0300
Subject: Upstream HTTP/2 support
In-Reply-To: <102558b0b56ca135655aa9fbf8644d4d.NginxMailingListEnglish@forum.nginx.org>
References: <102558b0b56ca135655aa9fbf8644d4d.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170710125153.GP55433@mdounin.ru>

Hello!

On Sun, Jul 09, 2017 at 10:55:02PM -0400, motor wrote:

> The following post is based on the assumptions that 1. upstream HTTP/2
> support is not available in nginx at the moment, and 2. said feature is not
> firmly excluded from nginx's future road map :)
> 
> We've been internally using an nginx patch (authored by me) which provides
> this feature in our nginx instances that front-end etcd servers.
> Please let me know if this patch can be a candidate for review and potential
> merge.
> 
> In case the answer is yes, a heads up: said implementation has been subject
> to a few 'compromises' due to internal deadlines, so I might need a fair bit
> of technical guidance/discussion to iron out some design/implementation
> issues.

Feel free to post the patch to nginx-devel@ mailing list.  Some 
generic recommendations can be found here:

http://nginx.org/en/docs/contributing_changes.html

As of now another patcheset to introduce HTTP/2 support is being 
reviewed, as available here:

http://mailman.nginx.org/pipermail/nginx-devel/2017-June/010209.html

It also have quite a few "compromises" though, so nobody knows 
which one is better, at least without seeing your version.

-- 
Maxim Dounin
http://nginx.org/

From kworthington at gmail.com  Mon Jul 10 13:05:09 2017
From: kworthington at gmail.com (Kevin Worthington)
Date: Mon, 10 Jul 2017 09:05:09 -0400
Subject: [nginx-announce] nginx-1.13.2
In-Reply-To: <20170627150422.GF55433@mdounin.ru>
References: <20170627150422.GF55433@mdounin.ru>
Message-ID: <CAGo79UXK89JfyEggdzvwLXmJf7gK2PUJXK1AEm5Vp2tNtePboA@mail.gmail.com>

Hello Nginx users,

Now available: Nginx 1.13.2 for Windows
https://kevinworthington.com/nginxwin1132
(32-bit and 64-bit versions)

These versions are to support legacy users who are already using Cygwin
based builds of Nginx. Officially supported native Windows binaries are at
nginx.org.

Announcements are also available here:
Twitter http://twitter.com/kworthington
Google+ https://plus.google.com/+KevinWorthington/

Thank you,
Kevin
--
Kevin Worthington
kworthington *@* (gmail]  [dot} {com)
http://kevinworthington.com/
http://twitter.com/kworthington
https://plus.google.com/+KevinWorthington/


On Tue, Jun 27, 2017 at 11:04 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Changes with nginx 1.13.2                                        27 Jun
> 2017
>
>     *) Change: nginx now returns 200 instead of 416 when a range starting
>        with 0 is requested from an empty file.
>
>     *) Feature: the "add_trailer" directive.
>        Thanks to Piotr Sikora.
>
>     *) Bugfix: nginx could not be built on Cygwin and NetBSD; the bug had
>        appeared in 1.13.0.
>
>     *) Bugfix: nginx could not be built under MSYS2 / MinGW 64-bit.
>        Thanks to Orgad Shaneh.
>
>     *) Bugfix: a segmentation fault might occur in a worker process when
>        using SSI with many includes and proxy_pass with variables.
>
>     *) Bugfix: in the ngx_http_v2_module.
>        Thanks to Piotr Sikora.
>
>
> --
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx-announce mailing list
> nginx-announce at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-announce
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170710/63d4a0f9/attachment.html>

From mdounin at mdounin.ru  Mon Jul 10 13:49:35 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 10 Jul 2017 16:49:35 +0300
Subject: Nginx Proxy seems to send twice the same request to the backend
In-Reply-To: <4A8C8A80-EF80-4CD3-9C0C-240279B64E06@gmail.com>
References: <4A8C8A80-EF80-4CD3-9C0C-240279B64E06@gmail.com>
Message-ID: <20170710134935.GQ55433@mdounin.ru>

Hello!

On Sun, Jul 09, 2017 at 11:26:47PM +0200, Arnaud Le-roy wrote:

> i encountered a strange behaviour with nginx, my backend seems 
> to receive twice the same request from nginx proxy, to be sure 
> that it's not the client that send two request i have had an 
> uuid params to each request.
> 
> when the problem occurs in nginx log i found one request in 
> success in access.log
> 
> x.x.x.x - - [09/Jul/2017:09:18:33 +0200] "GET /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428 HTTP/1.1" 200 2 "-" "-"
> 
> and an other one than generate this log in error.log :
> 
> 2017/07/09 09:18:31 [error] 38111#38111: *4098505 upstream prematurely closed connection while reading response header from upstream, client: x.x.x.x, server: x.x.com, request: "GET /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428 HTTP/1.1", upstream: "http://172.16.0.11:9092/query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428", host: "x.x.com"
> 
> on my backend i can see two request with the same uuid (the two 
> succeed)
> 
> {"pid":11424,"level":"info","message":"[API] AUTH1 /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428","timestamp":"2017-07-09 09:18:31.861Z"}
> {"pid":11424,"level":"info","message":"[API] AUTH1 /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428","timestamp":"2017-07-09 09:18:33.196Z"}
> 
> The client is a node program so i'm sure that it sends only one 
> request with the same uuid (no thread problem ;) the nginx serve 
> as simple proxy (no load balancing)

[...]

>     upstream api {
>          keepalive 100;
> 	 server 172.16.0.11:9092;
>     }

[...]

>         location / {
>             proxy_next_upstream off;
>             proxy_pass http://api;

[...]

> The backend is a simple node server.
> 
> the problem occurs randomly, and it happens for sure on 
> nginx/1.10.3 and nginx/1.13.2 on debian/jessie
> 
> After some days of research, i found that if i remove the 
> keepalive 100 from upstream configuration there is no longer the 
> problem but i don't understand why ? Maybe somebody can explain 
> me what could hapen ? maybe a misunderstanding about some 
> configuration on keep alive ?

With keepalive connections, it is possible that server will close 
the connection and the client will start sending a request a the 
same time.  This case is specifically outlined in RFC 2616,
https://tools.ietf.org/html/rfc2616#section-8.1.4

   A client, server, or proxy MAY close the transport connection at any
   time. For example, a client might have started to send a new request
   at the same time that the server has decided to close the "idle"
   connection. From the server's point of view, the connection is being
   closed while it was idle, but from the client's point of view, a
   request is in progress.

   This means that clients, servers, and proxies MUST be able to recover
   from asynchronous close events. Client software SHOULD reopen the
   transport connection and retransmit the aborted sequence of requests
   without user interaction so long as the request sequence is
   idempotent (see section 9.1.2).

In such situation nginx will re-try the request, as suggested by 
the standard.  In nginx before 1.9.13 this happened even with 
"proxy_next_upstream off".

Though in nginx 1.9.13 this code was rewritten as part of the 
introduction of non-idempotent methods handling in 
proxy_next_upstream.  And in nginx 1.9.13+ re-trying the request 
is not expected to happen unless "proxy_next_upstream error" is 
also configured.  As such, the exact configuration looks strange 
combined with the versions you claim.

Could you please double-check that the behaviour you describe 
appears when using nginx 1.13.2 and with proxy_next_upstream 
switched off?

Just in case, I've tested it here using the following simple nginx 
configuration:

    upstream foo {
        server 127.0.0.1:8081;
        keepalive 10;
    }

    server {
        listen 8080;

        location / {
            proxy_pass http://foo;
            proxy_set_header Connection "";
            proxy_http_version 1.1;
            proxy_next_upstream off;
        }
    }

    server {
        listen 8081;

        if ($connection_requests ~ 2) {
           return 444;
        }

        return 200 ok;
    }

As expected, it returns 502 when using "proxy_next_upstream off" 
and re-tries the request if proxy_next_upstream is not switched 
off.

Note well that if duplicate requests introduce a problem, you may 
want to reconsider logic of your backend.  At least idempotent 
requests can be freely retried, and your backend is expected to 
handle this.  (In practice this often happens even with 
non-idempotent requests too.)

-- 
Maxim Dounin
http://nginx.org/

From lists at viaduct-productions.com  Tue Jul 11 01:20:14 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Mon, 10 Jul 2017 21:20:14 -0400
Subject: nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)
Message-ID: <D0CA07CB-FDC4-40EC-948B-7B184A93EA8D@viaduct-productions.com>

Hi there. 

Looking to get port 80 serving.  Changed to root, but the error keeps the user from running:

nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /usr/local/etc/nginx/nginx.conf:2
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)
nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed

Is there any way to get nginx working as root?  Is this a bad idea?  I just don?t like running as port 8080.  It complicates things down the road as I want to implement SSL (port 443) later on.  Port numbers also get into my code later on.  

Any advice appreciated how I can get this on normal ports 80 and 443.  

_____________
Rich in Toronto @ VP







From zchao1995 at gmail.com  Tue Jul 11 01:24:19 2017
From: zchao1995 at gmail.com (Zhang Chao)
Date: Mon, 10 Jul 2017 18:24:19 -0700
Subject: nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)
In-Reply-To: <D0CA07CB-FDC4-40EC-948B-7B184A93EA8D@viaduct-productions.com>
References: <D0CA07CB-FDC4-40EC-948B-7B184A93EA8D@viaduct-productions.com>
Message-ID: <CAPr95BgubtMjBw2o505ohcxuAX5L0cjRp=LumEP-LfKFS9RmTg@mail.gmail.com>

Hi!

Only the root can bind the ports small than 1024. You should start your
nginx service with the sudo prefix.


On 11 July 2017 at 09:20:40, Viaduct Lists (lists at viaduct-productions.com)
wrote:

Hi there.

Looking to get port 80 serving. Changed to root, but the error keeps the
user from running:

nginx: [warn] the "user" directive makes sense only if the master process
runs with super-user privileges, ignored in
/usr/local/etc/nginx/nginx.conf:2
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)
nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed

Is there any way to get nginx working as root? Is this a bad idea? I just
don?t like running as port 8080. It complicates things down the road as I
want to implement SSL (port 443) later on. Port numbers also get into my
code later on.

Any advice appreciated how I can get this on normal ports 80 and 443.

_____________
Rich in Toronto @ VP






_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170710/0493d10a/attachment-0001.html>

From lists at lazygranch.com  Tue Jul 11 01:27:32 2017
From: lists at lazygranch.com (lists at lazygranch.com)
Date: Mon, 10 Jul 2017 18:27:32 -0700
Subject: nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)
In-Reply-To: <CAPr95BgubtMjBw2o505ohcxuAX5L0cjRp=LumEP-LfKFS9RmTg@mail.gmail.com>
References: <D0CA07CB-FDC4-40EC-948B-7B184A93EA8D@viaduct-productions.com>
 <CAPr95BgubtMjBw2o505ohcxuAX5L0cjRp=LumEP-LfKFS9RmTg@mail.gmail.com>
Message-ID: <20170711012732.5693526.75579.32700@lazygranch.com>

An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170710/e1e63dfa/attachment.html>

From nginx-forum at forum.nginx.org  Tue Jul 11 09:40:17 2017
From: nginx-forum at forum.nginx.org (gmacar)
Date: Tue, 11 Jul 2017 05:40:17 -0400
Subject: Please help me with $http_referer
Message-ID: <4aab7d20abcc1f0f467e92531104be9e.NginxMailingListEnglish@forum.nginx.org>

Hello everyone, I have read the manual
https://www.nginx.com/blog/creating-nginx-rewrite-rules/ but unfortunately I
didn't manage to solve my (simple) problem. What I need to do is: if a
visitor wants to read www.example.com/requested_page.html and comes from
Google or Bing, redirect it to a specific webpage. Example (pseudocode):

if http_referer is Google OR Bing
return https://www.example.com/other_page.html (instead of
requested_page.html)

How should I edit my current nginx.conf?
Thank you

%% Begin of nginx.conf %%

worker_processes 1;
daemon off;

error_log <%= ENV["APP_ROOT"] %>/nginx/logs/error.log;
events { worker_connections 1024; }

http {
  log_format cloudfoundry '$http_x_forwarded_for - $http_referer -
[$time_local] "$request" $status $body_bytes_sent';
  access_log <%= ENV["APP_ROOT"] %>/nginx/logs/access.log cloudfoundry;
  default_type application/octet-stream;
  include mime.types;
  sendfile on;
  gzip on;
  tcp_nopush on;
  keepalive_timeout 30;

  server {
    listen <%= ENV["PORT"] %>;
    server_name localhost;

    location ~ /\.ht { deny  all; }
    location / {
      root <%= ENV["APP_ROOT"] %>/public;
      index index.html index.htm Default.htm;
    }
    
    error_page 404 /404.html;
  }
}

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275402,275402#msg-275402


From ng23 at firemail.cc  Tue Jul 11 14:57:33 2017
From: ng23 at firemail.cc (Johan Andersson)
Date: Tue, 11 Jul 2017 16:57:33 +0200
Subject: Flushing responses in nginx modules
In-Reply-To: <CAPr95BgCOaSdVjuMHs+_spOGGaiPq2dqkwsyyYmZ+m_i0xqZAA@mail.gmail.com>
References: <7063b3f64e0aad4bed1fe3ece15f832d@firemail.cc>
 <CAPr95BgCOaSdVjuMHs+_spOGGaiPq2dqkwsyyYmZ+m_i0xqZAA@mail.gmail.com>
Message-ID: <22fdd15cca2696838552585d2605337b@firemail.cc>

Hi Andreas and Zhang,

Thank you for your hint with the http_echo_module! I read through their 
code to get a hang of how the event loop and the event handling actually 
works.

If I replace the hello_world command in my config files with the 
echo/echo_flush/echo_sleep commands, everything works as expected.

If I use my modified hello_world module (code below), I still get a 
three second pause and then all three "hello world" at once.

So I do not think that my configuration (which is the default Debian 
Stretch configuration) is at fault.

I pasted the whole debug log here: https://pastebin.com/raw/uwuK4UJB

When I look into the debug log, I see four writev lines corresponding to 
the initial header and the three "helloworld" outputs.

So I think the socket gets its data, but perhaps I am missing some magic 
socket options? Which would be strange, as I cannot see the 
http_echo_module doing such a thing.

This is my current code (all error handling omitted -- I will take care 
of that):

struct ngx_http_hello_world_ctx
{
     int counter;
     ngx_event_t event;
};

static int numberOfMessages = 3;

static ngx_int_t ngx_http_hello_world_handler(ngx_http_request_t *r)
{
     struct ngx_http_hello_world_ctx * ctx = ngx_http_get_module_ctx(r, 
ngx_http_hello_world_module);

     if(ctx == NULL)
     {
         ctx = ngx_pcalloc(r->pool, sizeof(struct 
ngx_http_hello_world_ctx));
         ngx_http_set_ctx(r, ctx, ngx_http_hello_world_module);
     }

     ctx->counter = 0;
     ctx->event.data = r;
     ctx->event.handler = ngx_http_hello_world_event_handler;
     ctx->event.log = r->connection->log;

     r->headers_out.content_type.len = sizeof("text/html") - 1;
     r->headers_out.content_type.data = (u_char *) "text/html";
     r->headers_out.status = NGX_HTTP_OK;
     ngx_http_send_header(r);

     r->main->count++; // Increments reference count
     ngx_add_timer(&ctx->event, 0);

     return ngx_http_output_filter(r, NULL);
}

static void ngx_http_hello_world_event_handler(ngx_event_t *ev)
{
     ngx_http_request_t * r = ev->data;
     struct ngx_http_hello_world_ctx * ctx = ngx_http_get_module_ctx(r, 
ngx_http_hello_world_module);

     if(ctx->counter < numberOfMessages)
     {
         ngx_buf_t *b;
         ngx_chain_t out;

         b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t));

         out.buf = b;
         out.next = NULL;

         b->pos = ngx_hello_world;
         b->last = ngx_hello_world + sizeof(ngx_hello_world);
         b->memory = 1;
         b->flush = 1;
         b->last_buf = (ctx->counter == numberOfMessages);

         ngx_http_output_filter(r, &out);
         ngx_http_send_special(r, NGX_HTTP_FLUSH);

         ctx->counter++;

         if(ctx->counter == numberOfMessages)
         {
             ctx->counter = 0;
             ngx_http_send_special(r, NGX_HTTP_LAST);
             ngx_http_finalize_request(r, NGX_OK); // Decrements 
reference count
         }
         else
         {
             ngx_add_timer(&ctx->event, 1000);
         }
     }
}

Cheers
Johan

On 2017-07-10 04:04, Zhang Chao wrote:
> Hello!
> 
> You mustn?t use standard sleep function for it will block Nginx?s
> events loop, alternatively, you need to put your write event to a
> timer, set the proper handler when the timer expires.
> BTW, you should always check the return value of ngx_http_send_header
> and ngx_http_output_filter.
> 
> On 10 July 2017 at 01:43:46, Johan Andersson (ng23 at firemail.cc) wrote:
> 
>> Hi everyone,
>> 
>> I have some issues writing my nginx modules.
>> 
>> I am on Debian Stretch, installed nginx with the default
>> configuration,
>> and took the hello_world module. It works without a hitch. Then I
>> changed the handler to send three "hello world" responses, and sleep
>> for
>> one second between each response.
>> 
>> However, when I look at the result in my browser, the page loads,
>> pauses
>> for three seconds, and then displays all three "hello world"
>> messages at
>> once.
>> 
>> Actually I was flushing each response, so I expected each "hello
>> world"
>> message to appear one after the other, with one second pause between
>> 
>> them.
>> 
>> Am I doing something wrong? Is this event the correct way to achieve
>> 
>> this? All functions return NGX_OK. This is my code:
>> 
>> static ngx_int_t ngx_http_hello_world_handler(ngx_http_request_t *r)
>> 
>> {
>> ngx_buf_t *b;
>> ngx_chain_t out;
>> ngx_int_t result;
>> 
>> r->headers_out.content_type.len = sizeof("text/html") - 1;
>> r->headers_out.content_type.data = (u_char *) "text/html";
>> r->headers_out.status = NGX_HTTP_OK;
>> //r->headers_out.content_length_n = sizeof(ngx_hello_world);
>> ngx_http_send_header(r);
>> 
>> for(int i = 0; i < 3; i++)
>> {
>> b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t));
>> 
>> out.buf = b;
>> out.next = NULL;
>> 
>> b->pos = ngx_hello_world;
>> b->last = ngx_hello_world + sizeof(ngx_hello_world);
>> b->memory = 1;
>> b->flush = 1;
>> b->last_buf = (i == 2);
>> 
>> result = ngx_http_output_filter(r, &out);
>> ngx_http_send_special(r, NGX_HTTP_FLUSH);
>> 
>> sleep(1);
>> }
>> 
>> return result;
>> }
>> 
>> Cheers
>> Johann
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

From mdounin at mdounin.ru  Tue Jul 11 15:46:20 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 11 Jul 2017 18:46:20 +0300
Subject: nginx-1.13.3
Message-ID: <20170711154620.GX55433@mdounin.ru>

Changes with nginx 1.13.3                                        11 Jul 2017

    *) Security: a specially crafted request might result in an integer
       overflow and incorrect processing of ranges in the range filter,
       potentially resulting in sensitive information leak (CVE-2017-7529).


-- 
Maxim Dounin
http://nginx.org/

From mdounin at mdounin.ru  Tue Jul 11 15:46:44 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 11 Jul 2017 18:46:44 +0300
Subject: nginx-1.12.1
Message-ID: <20170711154644.GB55433@mdounin.ru>

Changes with nginx 1.12.1                                        11 Jul 2017

    *) Security: a specially crafted request might result in an integer
       overflow and incorrect processing of ranges in the range filter,
       potentially resulting in sensitive information leak (CVE-2017-7529).


-- 
Maxim Dounin
http://nginx.org/

From mdounin at mdounin.ru  Tue Jul 11 15:47:46 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 11 Jul 2017 18:47:46 +0300
Subject: nginx security advisory (CVE-2017-7529)
Message-ID: <20170711154745.GF55433@mdounin.ru>

Hello!

A security issue was identified in nginx range filter.  A specially
crafted request might result in an integer overflow and incorrect
processing of ranges, potentially resulting in sensitive information
leak (CVE-2017-7529).

When using nginx with standard modules this allows an attacker to
obtain a cache file header if a response was returned from cache.
In some configurations a cache file header may contain IP address
of the backend server or other sensitive information.

Besides, with 3rd party modules it is potentially possible that
the issue may lead to a denial of service or a disclosure of
a worker process memory.  No such modules are currently known though.

The issue affects nginx 0.5.6 - 1.13.2.
The issue is fixed in nginx 1.13.3, 1.12.1.

For older versions, the following configuration can be used
as a temporary workaround:

    max_ranges 1;

Patch for the issue can be found here:

http://nginx.org/download/patch.2017.ranges.txt


-- 
Maxim Dounin
http://nginx.org/

From lists at viaduct-productions.com  Tue Jul 11 17:17:56 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Tue, 11 Jul 2017 13:17:56 -0400
Subject: nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)
In-Reply-To: <20170711012732.5693526.75579.32700@lazygranch.com>
References: <D0CA07CB-FDC4-40EC-948B-7B184A93EA8D@viaduct-productions.com>
 <CAPr95BgubtMjBw2o505ohcxuAX5L0cjRp=LumEP-LfKFS9RmTg@mail.gmail.com>
 <20170711012732.5693526.75579.32700@lazygranch.com>
Message-ID: <B7935C21-55D6-46CE-8429-8B5A23804632@viaduct-productions.com>

nginx.conf sets the user and admin, but that coughs up an error when trying to run as root. This is why it?s so confusing.  


> On Jul 10, 2017, at 9:27 PM, lists at lazygranch.com wrote:
> 
> I don't have server access at the moment, but I think nginx under FreeBSD runs under user www. 


_____________
Rich in Toronto @ VP







From kworthington at gmail.com  Tue Jul 11 17:23:44 2017
From: kworthington at gmail.com (Kevin Worthington)
Date: Tue, 11 Jul 2017 13:23:44 -0400
Subject: [nginx-announce] nginx-1.13.3
In-Reply-To: <20170711154628.GY55433@mdounin.ru>
References: <20170711154628.GY55433@mdounin.ru>
Message-ID: <CAGo79UXoNO+V81aR8Y2B19jWiwabNdct6JtfChGJXue0-dGpyg@mail.gmail.com>

Hello Nginx users,

Now available: Nginx 1.13.3 for Windows
https://kevinworthington.com/nginxwin1133
(32-bit and 64-bit versions)

These versions are to support legacy users who are already using Cygwin
based builds of Nginx. Officially supported native Windows binaries are at
nginx.org.

Announcements are also available here:
Twitter http://twitter.com/kworthington
Google+ https://plus.google.com/+KevinWorthington/

Thank you,
Kevin
--
Kevin Worthington
kworthington *@* (gmail]  [dot} {com)
http://kevinworthington.com/
http://twitter.com/kworthington
https://plus.google.com/+KevinWorthington/

On Tue, Jul 11, 2017 at 11:46 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Changes with nginx 1.13.3                                        11 Jul
> 2017
>
>     *) Security: a specially crafted request might result in an integer
>        overflow and incorrect processing of ranges in the range filter,
>        potentially resulting in sensitive information leak (CVE-2017-7529).
>
>
> --
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx-announce mailing list
> nginx-announce at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-announce
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170711/9be6d437/attachment.html>

From nginx-forum at forum.nginx.org  Tue Jul 11 21:45:15 2017
From: nginx-forum at forum.nginx.org (c0nw0nk)
Date: Tue, 11 Jul 2017 17:45:15 -0400
Subject: nginx security advisory (CVE-2017-7529)
In-Reply-To: <20170711154745.GF55433@mdounin.ru>
References: <20170711154745.GF55433@mdounin.ru>
Message-ID: <28d5df52fe11dbb58985f00c199599d3.NginxMailingListEnglish@forum.nginx.org>

Couldn't you use 

max_ranges 0;

To disable byte range support completely.

Also won't setting the value of ranges to max_ranges 1; break pseudo
streaming in HTML5 video apps etc. ?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275424,275437#msg-275437


From agentzh at gmail.com  Tue Jul 11 22:35:34 2017
From: agentzh at gmail.com (Yichun Zhang (agentzh))
Date: Tue, 11 Jul 2017 15:35:34 -0700
Subject: [ANN] OpenResty 1.11.2.4 released (and new Linux package repositories)
Message-ID: <CAB4Tn6PPCGQYRpLqRgh56TcJ-e-+M05QO3VbzQFErUAhOrmN9w@mail.gmail.com>

Hi folks,

OpenResty 1.11.2.4 is just released to include the latest nginx
security fix in its range filter module (CVE-2017-7529).

You can download this version's source tarball and Win32 binary from
the following page:

    https://openresty.org/en/download.html

Pre-built Linux binary packages for this release can also be obtained
from OpenResty's official Yum and Apt repositories:

    https://openresty.org/en/linux-packages.html

The complete change log since the last (formal) release, 1.11.2.3:

  * bugfix: applied nginx's official security fix for an issue in the
range filter (CVE-2017-7529).

The next OpenResty formal release will be 1.11.2.5, which will include
a lot of new features developed in the last couple of months. We're
still busy testing it.

Now OpenResty provides official APT repositories for Ubuntu 14.04 ~
17.10 and Debian 7 ~ 9:

    https://openresty.org/en/linux-packages.html

    https://openresty.org/en/deb-packages.html

Furthermore, we've migrated our original Yum repositories from Fedora
Copr servers to our own openresty.org. Please migrate to our new Yum
repositories if you are using the old Copr repositories:

   http://openresty.org/en/linux-packages.html#centos

Also, we provide separate Yum repositories for RHEL and CentOS. And we
no longer require the EPEL repository for any of our packages.

Additionally, we provide official Yum repository for Amazon Linux x86_64:

    https://openresty.org/en/linux-packages.html#amazon-linux

The HTML version of the change log with lots of helpful hyper-links
can be browsed here:

    https://openresty.org/en/changelog-1011002.html

OpenResty is a full-fledged web platform
by bundling the standard Nginx core, LuaJIT, lots of
3rd-party Nginx modules and Lua libraries, as well as most of their external
dependencies. See OpenResty's homepage for details:

    https://openresty.org/

OpenResty is supported by the OpenResty Software Foundation, OpenResty
Inc., and the global community.

Thanks!
-agentzh

From nginx-forum at forum.nginx.org  Tue Jul 11 22:56:35 2017
From: nginx-forum at forum.nginx.org (darylwang)
Date: Tue, 11 Jul 2017 18:56:35 -0400
Subject: nginx security advisory (CVE-2017-7529)
In-Reply-To: <20170711154745.GF55433@mdounin.ru>
References: <20170711154745.GF55433@mdounin.ru>
Message-ID: <aea1e7cd22a6a660cf8ac7ebf3ea60eb.NginxMailingListEnglish@forum.nginx.org>

Is there any available proof of concept or other test for this exploit? I'm
applying the patch to our systems and would like some way to check that the
fix is effective.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275424,275439#msg-275439


From fabio.ancona at gmail.com  Wed Jul 12 07:18:53 2017
From: fabio.ancona at gmail.com (Fabio Ancona)
Date: Wed, 12 Jul 2017 09:18:53 +0200
Subject: Help to build nginx withe specific OpenSSL shared library
Message-ID: <CAMtgXyqRJ9vkUfQA_JQ7gHi1qFTXZtGSG7fO=JADkzsQy+qTCw@mail.gmail.com>

Hi.
I have problem to link specific OpenSSL shared library during "configure"
to build nginx from source code on Linux.
On my system I have two version of already built OpenSSL: one in "/opt/ssl"
(version 1.1.0f) and the other one on "/opt/ssldevel" (version 1.1.1 devel
that supports TLS 1.3 draft 18).
The system's OpenSSL library points to the version 1.1.0f in "/opt/ssl" via
the "/etc/ld.so.conf" file:


root at server:~# cat /etc/ld.so.conf
/opt/ssl/lib
/usr/local/lib
include /etc/ld.so.conf.d/*.conf


root at server:~# ldconfig -v
ldconfig: Path `/usr/local/lib' given more than once
ldconfig: Path `/lib/arm-linux-gnueabihf' given more than once
ldconfig: Path `/usr/lib/arm-linux-gnueabihf' given more than once
/opt/ssl/lib:
        libcrypto.so.1.1 -> libcrypto.so.1.1
        libssl.so.1.1 -> libssl.so.1.1

I want to build nginx with OpenSSL shared library not using the system
OpenSSL library (so that available at "/opt/ssl/lib") but using the shared
library in "/opt/ssldevel/lib".

See the shared library available on "/opt/ssldevel/lib":

root at server:/opt/ssldevel/lib# ls -la
total 6588
drwxr-xr-x 4 root root    4096 Jul 11 12:22 .
drwxr-xr-x 9 root root    4096 Jul 11 12:22 ..
drwxr-xr-x 2 root root    4096 Jul 11 12:22 engines-1.1
-rw-r--r-- 1 root root 3363402 Jul 11 12:22 libcrypto.a
lrwxrwxrwx 1 root root      16 Jul 11 12:22 libcrypto.so -> libcrypto.so.1.1
-rwxr-xr-x 1 root root 2305732 Jul 11 12:22 libcrypto.so.1.1
-rw-r--r-- 1 root root  585004 Jul 11 12:22 libssl.a
lrwxrwxrwx 1 root root      13 Jul 11 12:22 libssl.so -> libssl.so.1.1
-rwxr-xr-x 1 root root  468330 Jul 11 12:22 libssl.so.1.1
drwxr-xr-x 2 root root    4096 Jul 11 12:22 pkgconfig



I have tried various combinations of parameter in the "configure" options,
but after building nginx always will use the OpenSSL library (shared)
available on "/opt/ssl/lib". I have checked it with the command "ldd".

There are my "configure" versions that I have tried but without success
because nginx will always go to use the library available at "/opt/ssl/lib"
(I paste only the OpenSSL part):



configure --with-cc-opt="-I /opt/ssldevel/include" --with-ld-opt="-L
/opt/ssldevel/lib -Wl,-rpath -lssl -lcrypto -ldl" --with-http_ssl_module
--with-openssl-opt=enable-tls1_3

configure --with-cc-opt="-I /opt/ssldevel/include" --with-ld-opt="-L
/opt/ssldevel/lib" --with-http_ssl_module --with-openssl-opt=enable-tls1_3



Please, what are the correct parameter to give to "configure" to make nginx
to point to specific path of OpenSSL shared library (so not using the
system's one)?

Many thanks for your support.

Best Regards.

Fabio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170712/1b7869eb/attachment.html>

From sca at andreasschulze.de  Wed Jul 12 08:39:49 2017
From: sca at andreasschulze.de (A. Schulze)
Date: Wed, 12 Jul 2017 10:39:49 +0200
Subject: Help to build nginx withe specific OpenSSL shared library
In-Reply-To: <CAMtgXyqRJ9vkUfQA_JQ7gHi1qFTXZtGSG7fO=JADkzsQy+qTCw@mail.gmail.com>
Message-ID: <20170712103949.Horde.eqWYMugwrnoBugonc4FnJIQ@andreasschulze.de>


Fabio Ancona:

> I want to build nginx with OpenSSL shared library not using the system
> OpenSSL library (so that available at "/opt/ssl/lib") but using the shared
> library in "/opt/ssldevel/lib".

I use to patch the file "auto/lib/openssl/conf"
I picked up a section/platform I never will build for and modifiy
"ngx_feature", "ngx_feature_path" and "ngx_feature_libs"



Description: find my openssl libraries with nodefault names
Author: A. Schulze
---
This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
Index: nginx-1.13.3/auto/lib/openssl/conf
===================================================================
--- nginx-1.13.3.orig/auto/lib/openssl/conf
+++ nginx-1.13.3/auto/lib/openssl/conf
@@ -97,15 +97,15 @@ else

          if [ $ngx_found = no ]; then

-            # MacPorts
+            # my-openssl

-            ngx_feature="OpenSSL library in /opt/local/"
-            ngx_feature_path="/opt/local/include"
+            ngx_feature="my-openssl library in /usr/lib/"
+            ngx_feature_path="/usr/include"

              if [ $NGX_RPATH = YES ]; then
-                ngx_feature_libs="-R/opt/local/lib -L/opt/local/lib  
-lssl -lcrypto $NGX_LIBDL"
+                ngx_feature_libs="-R/usr/lib -L/usr/lib -lssl-my  
-lcrypto-my $NGX_LIBDL"
              else
-                ngx_feature_libs="-L/opt/local/lib -lssl -lcrypto $NGX_LIBDL"
+                ngx_feature_libs="-L/usr/lib -lssl-my -lcrypto-my $NGX_LIBDL"
              fi

              . auto/feature


Andreas


From fabio.ancona at gmail.com  Wed Jul 12 12:00:21 2017
From: fabio.ancona at gmail.com (Fabio Ancona)
Date: Wed, 12 Jul 2017 14:00:21 +0200
Subject: Help to build nginx withe specific OpenSSL shared library
In-Reply-To: <20170712103949.Horde.eqWYMugwrnoBugonc4FnJIQ@andreasschulze.de>
References: <CAMtgXyqRJ9vkUfQA_JQ7gHi1qFTXZtGSG7fO=JADkzsQy+qTCw@mail.gmail.com>
 <20170712103949.Horde.eqWYMugwrnoBugonc4FnJIQ@andreasschulze.de>
Message-ID: <CAMtgXyp6Q5XOfr4oDL35aZPd8kz2M-JWmGCiitnzR1qDti4ffg@mail.gmail.com>

Hi.
Thanks for your suggestion.
But I just found an easier way (using the "configure") to build nginx with
OpenSSL shared library not using the system OpenSSL library but using the
shared library in a specific path (OpenSSL already built in "/opt/ssldevel"
in my case).
I set the "--with-ld-opt=" in this way:

--with-ld-opt="-L /opt/ssldevel/lib -ldl -Wl,-rpath,/opt/ssldevel/lib"

So the configure will be:

configure --with-cc-opt="-I /opt/ssldevel/include" --with-ld-opt="-L
/opt/ssldevel/lib -ldl -Wl,-rpath,/opt/ssldevel/lib" --with-http_ssl_module
--with-openssl-opt=enable-tls1_3

In this way nginx will use the shared OpenSSL library to the specific path
"/opt/ssldevel", see here:

root at server:~# ldd /usr/local/nginx/sbin/nginx
        linux-vdso.so.1 =>  (0x7ef4f000)
        /usr/lib/arm-linux-gnueabihf/libcofi_rpi.so (0x76fd8000)
        libdl.so.2 => /lib/arm-linux-gnueabihf/libdl.so.2 (0x76fbb000)
        libpthread.so.0 => /lib/arm-linux-gnueabihf/libpthread.so.0
(0x76f9c000)
        libcrypt.so.1 => /lib/arm-linux-gnueabihf/libcrypt.so.1 (0x76f65000)
        libssl.so.1.1 => /opt/ssldevel/lib/libssl.so.1.1 (0x76efe000)
        libcrypto.so.1.1 => /opt/ssldevel/lib/libcrypto.so.1.1 (0x76d17000)
        libz.so.1 => /usr/local/lib/libz.so.1 (0x76cf6000)
        libperl.so.5.14 => /usr/lib/libperl.so.5.14 (0x76ba5000)
        libm.so.6 => /lib/arm-linux-gnueabihf/libm.so.6 (0x76b34000)
        libc.so.6 => /lib/arm-linux-gnueabihf/libc.so.6 (0x76a00000)
        /lib/ld-linux-armhf.so.3 (0x54aee000)
        libgcc_s.so.1 => /lib/arm-linux-gnueabihf/libgcc_s.so.1 (0x769d8000)


I hope that it's OK also in your point of view (without introducing other
issues).
Thanks.


Fabio




>
> I use to patch the file "auto/lib/openssl/conf"
> I picked up a section/platform I never will build for and modifiy
> "ngx_feature", "ngx_feature_path" and "ngx_feature_libs"
>
>
>
> Description: find my openssl libraries with nodefault names
> Author: A. Schulze
> ---
> This patch header follows DEP-3: http://dep.debian.net/deps/dep3/
> Index: nginx-1.13.3/auto/lib/openssl/conf
> ===================================================================
> --- nginx-1.13.3.orig/auto/lib/openssl/conf
> +++ nginx-1.13.3/auto/lib/openssl/conf
> @@ -97,15 +97,15 @@ else
>
>          if [ $ngx_found = no ]; then
>
> -            # MacPorts
> +            # my-openssl
>
> -            ngx_feature="OpenSSL library in /opt/local/"
> -            ngx_feature_path="/opt/local/include"
> +            ngx_feature="my-openssl library in /usr/lib/"
> +            ngx_feature_path="/usr/include"
>
>              if [ $NGX_RPATH = YES ]; then
> -                ngx_feature_libs="-R/opt/local/lib -L/opt/local/lib
> -lssl -lcrypto $NGX_LIBDL"
> +                ngx_feature_libs="-R/usr/lib -L/usr/lib -lssl-my
> -lcrypto-my $NGX_LIBDL"
>              else
> -                ngx_feature_libs="-L/opt/local/lib -lssl -lcrypto
> $NGX_LIBDL"
> +                ngx_feature_libs="-L/usr/lib -lssl-my -lcrypto-my
> $NGX_LIBDL"
>              fi
>
>              . auto/feature
>
>
> Andreas
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170712/35dc6372/attachment.html>

From mdounin at mdounin.ru  Wed Jul 12 12:01:32 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 12 Jul 2017 15:01:32 +0300
Subject: nginx security advisory (CVE-2017-7529)
In-Reply-To: <28d5df52fe11dbb58985f00c199599d3.NginxMailingListEnglish@forum.nginx.org>
References: <20170711154745.GF55433@mdounin.ru>
 <28d5df52fe11dbb58985f00c199599d3.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170712120132.GR55433@mdounin.ru>

Hello!

On Tue, Jul 11, 2017 at 05:45:15PM -0400, c0nw0nk wrote:

> Couldn't you use 
> 
> max_ranges 0;
> 
> To disable byte range support completely.

Disabling ranges completely will mitigate the issue as well.  But 
as the issue only affects requests with multiple ranges, it is not 
needed, "max_ranges 1;" is enough.

> Also won't setting the value of ranges to max_ranges 1; break pseudo
> streaming in HTML5 video apps etc. ?

No, pseudo streaming generally uses requests with a single range, 
and these are allowed with "max_ranges 1;".  Requests with 
multiple ranges are very rare in practice (AFAIK, they are used 
by Adobe Acrobat and MS Office, but I've never heard of anything 
more popular than that).

-- 
Maxim Dounin
http://nginx.org/

From zchao1995 at gmail.com  Wed Jul 12 12:51:28 2017
From: zchao1995 at gmail.com (Zhang Chao)
Date: Wed, 12 Jul 2017 05:51:28 -0700
Subject: Flushing responses in nginx modules
In-Reply-To: <22fdd15cca2696838552585d2605337b@firemail.cc>
References: <7063b3f64e0aad4bed1fe3ece15f832d@firemail.cc>
 <CAPr95BgCOaSdVjuMHs+_spOGGaiPq2dqkwsyyYmZ+m_i0xqZAA@mail.gmail.com>
 <22fdd15cca2696838552585d2605337b@firemail.cc>
Message-ID: <CAPr95Bjv95Nauk=hhYtjdAqXWNHVxGAWiiCyL++tZP+9X1yTxQ@mail.gmail.com>

Hi!

I think you can debug your code with gdb, maybe some other modules or
filters prevent sending the data.


On 11 July 2017 at 22:57:50, Johan Andersson (ng23 at firemail.cc) wrote:

Hi Andreas and Zhang,

Thank you for your hint with the http_echo_module! I read through their
code to get a hang of how the event loop and the event handling actually
works.

If I replace the hello_world command in my config files with the
echo/echo_flush/echo_sleep commands, everything works as expected.

If I use my modified hello_world module (code below), I still get a
three second pause and then all three "hello world" at once.

So I do not think that my configuration (which is the default Debian
Stretch configuration) is at fault.

I pasted the whole debug log here: https://pastebin.com/raw/uwuK4UJB

When I look into the debug log, I see four writev lines corresponding to
the initial header and the three "helloworld" outputs.

So I think the socket gets its data, but perhaps I am missing some magic
socket options? Which would be strange, as I cannot see the
http_echo_module doing such a thing.

This is my current code (all error handling omitted -- I will take care
of that):

struct ngx_http_hello_world_ctx
{
int counter;
ngx_event_t event;
};

static int numberOfMessages = 3;

static ngx_int_t ngx_http_hello_world_handler(ngx_http_request_t *r)
{
struct ngx_http_hello_world_ctx * ctx = ngx_http_get_module_ctx(r,
ngx_http_hello_world_module);

if(ctx == NULL)
{
ctx = ngx_pcalloc(r->pool, sizeof(struct
ngx_http_hello_world_ctx));
ngx_http_set_ctx(r, ctx, ngx_http_hello_world_module);
}

ctx->counter = 0;
ctx->event.data = r;
ctx->event.handler = ngx_http_hello_world_event_handler;
ctx->event.log = r->connection->log;

r->headers_out.content_type.len = sizeof("text/html") - 1;
r->headers_out.content_type.data = (u_char *) "text/html";
r->headers_out.status = NGX_HTTP_OK;
ngx_http_send_header(r);

r->main->count++; // Increments reference count
ngx_add_timer(&ctx->event, 0);

return ngx_http_output_filter(r, NULL);
}

static void ngx_http_hello_world_event_handler(ngx_event_t *ev)
{
ngx_http_request_t * r = ev->data;
struct ngx_http_hello_world_ctx * ctx = ngx_http_get_module_ctx(r,
ngx_http_hello_world_module);

if(ctx->counter < numberOfMessages)
{
ngx_buf_t *b;
ngx_chain_t out;

b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t));

out.buf = b;
out.next = NULL;

b->pos = ngx_hello_world;
b->last = ngx_hello_world + sizeof(ngx_hello_world);
b->memory = 1;
b->flush = 1;
b->last_buf = (ctx->counter == numberOfMessages);

ngx_http_output_filter(r, &out);
ngx_http_send_special(r, NGX_HTTP_FLUSH);

ctx->counter++;

if(ctx->counter == numberOfMessages)
{
ctx->counter = 0;
ngx_http_send_special(r, NGX_HTTP_LAST);
ngx_http_finalize_request(r, NGX_OK); // Decrements
reference count
}
else
{
ngx_add_timer(&ctx->event, 1000);
}
}
}

Cheers
Johan

On 2017-07-10 04:04, Zhang Chao wrote:
> Hello!
>
> You mustn?t use standard sleep function for it will block Nginx?s
> events loop, alternatively, you need to put your write event to a
> timer, set the proper handler when the timer expires.
> BTW, you should always check the return value of ngx_http_send_header
> and ngx_http_output_filter.
>
> On 10 July 2017 at 01:43:46, Johan Andersson (ng23 at firemail.cc) wrote:
>
>> Hi everyone,
>>
>> I have some issues writing my nginx modules.
>>
>> I am on Debian Stretch, installed nginx with the default
>> configuration,
>> and took the hello_world module. It works without a hitch. Then I
>> changed the handler to send three "hello world" responses, and sleep
>> for
>> one second between each response.
>>
>> However, when I look at the result in my browser, the page loads,
>> pauses
>> for three seconds, and then displays all three "hello world"
>> messages at
>> once.
>>
>> Actually I was flushing each response, so I expected each "hello
>> world"
>> message to appear one after the other, with one second pause between
>>
>> them.
>>
>> Am I doing something wrong? Is this event the correct way to achieve
>>
>> this? All functions return NGX_OK. This is my code:
>>
>> static ngx_int_t ngx_http_hello_world_handler(ngx_http_request_t *r)
>>
>> {
>> ngx_buf_t *b;
>> ngx_chain_t out;
>> ngx_int_t result;
>>
>> r->headers_out.content_type.len = sizeof("text/html") - 1;
>> r->headers_out.content_type.data = (u_char *) "text/html";
>> r->headers_out.status = NGX_HTTP_OK;
>> //r->headers_out.content_length_n = sizeof(ngx_hello_world);
>> ngx_http_send_header(r);
>>
>> for(int i = 0; i < 3; i++)
>> {
>> b = ngx_pcalloc(r->pool, sizeof(ngx_buf_t));
>>
>> out.buf = b;
>> out.next = NULL;
>>
>> b->pos = ngx_hello_world;
>> b->last = ngx_hello_world + sizeof(ngx_hello_world);
>> b->memory = 1;
>> b->flush = 1;
>> b->last_buf = (i == 2);
>>
>> result = ngx_http_output_filter(r, &out);
>> ngx_http_send_special(r, NGX_HTTP_FLUSH);
>>
>> sleep(1);
>> }
>>
>> return result;
>> }
>>
>> Cheers
>> Johann
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170712/b0fd1e66/attachment-0001.html>

From sca at andreasschulze.de  Wed Jul 12 13:43:26 2017
From: sca at andreasschulze.de (A. Schulze)
Date: Wed, 12 Jul 2017 15:43:26 +0200
Subject: Help to build nginx withe specific OpenSSL shared library
In-Reply-To: <CAMtgXyp6Q5XOfr4oDL35aZPd8kz2M-JWmGCiitnzR1qDti4ffg@mail.gmail.com>
References: <CAMtgXyqRJ9vkUfQA_JQ7gHi1qFTXZtGSG7fO=JADkzsQy+qTCw@mail.gmail.com>
 <20170712103949.Horde.eqWYMugwrnoBugonc4FnJIQ@andreasschulze.de>
 <CAMtgXyp6Q5XOfr4oDL35aZPd8kz2M-JWmGCiitnzR1qDti4ffg@mail.gmail.com>
Message-ID: <20170712154326.Horde.JkAXnprDqazFIjtFRt1ebjB@andreasschulze.de>


Fabio Ancona:

> I set the "--with-ld-opt=" in this way

> I hope that it's OK also in your point of view (without introducing  
> other issues).

if it works for you, it's fine.
For me that didn't work months/versions ago because my openssl library
use the same path but other library names.


I'm not aware (not tested) nginx support my setup without patching /today/
but it would be helpful.

Andreas


From nginx-forum at forum.nginx.org  Wed Jul 12 16:26:41 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Wed, 12 Jul 2017 12:26:41 -0400
Subject: proxy_set_header directives don't inherit from father context
In-Reply-To: <20170705072017.GD3000@daoine.org>
References: <20170705072017.GD3000@daoine.org>
Message-ID: <17f8fff100fad6ad8833e71e3f7a0a61.NginxMailingListEnglish@forum.nginx.org>

what's gone happen if i set multiple proxy_cache_valid directives in
different levels?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275301,275458#msg-275458


From nginx-forum at forum.nginx.org  Wed Jul 12 20:57:13 2017
From: nginx-forum at forum.nginx.org (aledbf)
Date: Wed, 12 Jul 2017 16:57:13 -0400
Subject: nginScript question
Message-ID: <7a4b3366a3443261d2418bd3aca5782a.NginxMailingListEnglish@forum.nginx.org>

Hi,

It is possible to make a HTTP request with nginScript? I am trying to avoid
adding the lua module just for this
Thanks in advance

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275459,275459#msg-275459


From lists at viaduct-productions.com  Wed Jul 12 21:13:54 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Wed, 12 Jul 2017 17:13:54 -0400
Subject: FreeBSD www root Path
Message-ID: <1BC796CF-0838-4C1D-8450-340CDF2C4920@viaduct-productions.com>

Just wondering where the best location is for www on FreeBSD.

The default nginx.conf reports /usr/local/www/nginx

But then it reports this:

[Wed Jul 12 05:10:09 rich at neb /usr/local/www/nginx] ll
dr-xr-xr-x  2 root  wheel    5 Jul  7 10:23 .
drwxr-xr-x  3 root  wheel    4 Jul  7 10:23 ..
-rw-r--r--  1 root  wheel  537 Jul  7 10:23 50x.html
-rw-r--r--  1 root  wheel    1 Jul  7 10:23 EXAMPLE_DIRECTORY-DONT_ADD_OR_TOUCH_ANYTHING
-rw-r--r--  1 root  wheel  612 Jul  7 10:23 index.html
[Wed Jul 12 05:10:11 rich at neb /usr/local/www/nginx] pwd
/usr/local/www/nginx

So without any direct documentation I can find, I?d like to choose a proper path.  I?m hoping /usr/local/www/vhost1, /usr/local/www/vhost2, etc.  

Cheers


_____________
Rich in Toronto @ VP







From nginx-forum at forum.nginx.org  Thu Jul 13 01:42:04 2017
From: nginx-forum at forum.nginx.org (martinzhou)
Date: Wed, 12 Jul 2017 21:42:04 -0400
Subject: nginx security advisory (CVE-2017-7529)
In-Reply-To: <20170712120132.GR55433@mdounin.ru>
References: <20170712120132.GR55433@mdounin.ru>
Message-ID: <f9845f6b1484b121eb1e9767171a639b.NginxMailingListEnglish@forum.nginx.org>

Maxim Dounin Wrote:
-------------------------------------------------------
> Hello!
> 
> On Tue, Jul 11, 2017 at 05:45:15PM -0400, c0nw0nk wrote:
> 
> > Couldn't you use 
> > 
> > max_ranges 0;
> > 
> > To disable byte range support completely.
> 
> Disabling ranges completely will mitigate the issue as well.  But 
> as the issue only affects requests with multiple ranges, it is not 
> needed, "max_ranges 1;" is enough.
> 
> > Also won't setting the value of ranges to max_ranges 1; break pseudo
> > streaming in HTML5 video apps etc. ?
> 
> No, pseudo streaming generally uses requests with a single range, 
> and these are allowed with "max_ranges 1;".  Requests with 
> multiple ranges are very rare in practice (AFAIK, they are used 
> by Adobe Acrobat and MS Office, but I've never heard of anything 
> more popular than that).
> 
> -- 
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx


I found that in some cases (when the browser is requesting for a mp3 file),
the HTTP header will be formed as "Range: bytes=1-100, 200-100". I'm
wondering if we set "max_ranges 0;" or "max_ranges 1;" in the config, it
will cause the failure of loading such files. 

Also, I'm wondering if I've already set a comparatively "big" number after
the "max_ranges", for example, "max_ranges 100;", do I still need to adjust
the number to a low value (e.g. "1" or "2")?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275424,275462#msg-275462


From nginx-forum at forum.nginx.org  Thu Jul 13 04:26:35 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Thu, 13 Jul 2017 00:26:35 -0400
Subject: realip disabled if header contains port
Message-ID: <e18b197d77f231cfb3bbb1a245a197e8.NginxMailingListEnglish@forum.nginx.org>

hello,

i found http_realip_module doesn't work if the realip_header contains client
port.
my config is like bellow:
http {
    real_ip_header      X-Real-IP;
    set_real_ip_from    0.0.0.0/0;
    server {...}
}

the X-Real-IP header in the request is:
X-Real-IP: 123.123.123.123

but the $remote_addr variable in the log turn out to be the address of last
hop proxy.
is anything wrong?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275463,275463#msg-275463


From nginx-forum at forum.nginx.org  Thu Jul 13 04:28:19 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Thu, 13 Jul 2017 00:28:19 -0400
Subject: realip disabled if header contains port
In-Reply-To: <e18b197d77f231cfb3bbb1a245a197e8.NginxMailingListEnglish@forum.nginx.org>
References: <e18b197d77f231cfb3bbb1a245a197e8.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <6898f5acd8254db4d2638175c47a0275.NginxMailingListEnglish@forum.nginx.org>

foxgab Wrote:
-------------------------------------------------------
> hello,
> 
> i found http_realip_module doesn't work if the realip_header contains
> client port.
> my config is like bellow:
> http {
>     real_ip_header      X-Real-IP;
>     set_real_ip_from    0.0.0.0/0;
>     server {...}
> }
> 
> the X-Real-IP header in the request is:
> X-Real-IP: 123.123.123.123
> 
> but the $remote_addr variable in the log turn out to be the address of
> last hop proxy.
> is anything wrong?

sorry, the X-Real-IP header is like:  X-Real-IP: 123.123.123.123:4321

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275463,275464#msg-275464


From nginx-forum at forum.nginx.org  Thu Jul 13 07:37:29 2017
From: nginx-forum at forum.nginx.org (Myoungho)
Date: Thu, 13 Jul 2017 03:37:29 -0400
Subject: Debug log print and conf
Message-ID: <6fdc6c34a58bb69aa4e48dab153b9a4a.NginxMailingListEnglish@forum.nginx.org>

Hello, Im newbie in nginx module developing and now studying how nginx
module works using 'hello_world' module.

I have two questions.

My nginx.conf is; 

        location ~ ^/test1/.*$ {
           error_log /usr/local/nginx/logs/error-helloworld.log debug;
           hello entry1;
        }
 
       location ~ ^/test2/.*$ {
           error_log /usr/local/nginx/logs/error-helloworld.log debug;
           hello  entry2;
        }


1. Where can I see my logs printed by ngx_log_debug*?
   I added following line in handler function.
   ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "hello
world");
   but there is no result in nginx error log nor my_module_error_log.
   and when I use ngx_log_debug1 () function like as follows, there is no
HTTP response. 
   ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, "hello
world[%s]", r->args.data);
   ( in confiure, I used --with-debug )

2. How can I get 'entry1' or 'entry2' string in my handler?
my command arrary are here;
static ngx_command_t ngx_http_hello_commands[] = {
{ ngx_string("hello"),
NGX_HTTP_LOC_CONF|NGX_CONF_TAKE1,
ngx_conf_set_str_slot,
NGX_HTTP_LOC_CONF_OFFSET,
offsetof(ngx_http_hello_loc_conf_t, name),
&ngx_http_hello_p },
ngx_null_command
};

static char *
ngx_http_helloworld(ngx_conf_t *cf, void *post, void *data)
{
	ngx_http_core_loc_conf_t *clcf;
	clcf = ngx_http_conf_get_module_loc_conf(cf, ngx_http_core_module);
	clcf->handler = ngx_http_hello_handler;
	ngx_str_t *name = data; // i.e., first field of ngx_http_hello_loc_conf_t

	if (ngx_strcmp(name->data, "") == 0) {
		return NGX_CONF_ERROR;
	}
	hello_string.data = name->data; // --> It's always 'entry2'
	hello_string.len = ngx_strlen(hello_string.data);
        
	return NGX_CONF_OK;
}

whenever I request "http://~~/test/" or "http://~~/test2/"  the hello_string
is always 'entry2'.

Thanks for any helps. 

Myoungho.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275465,275465#msg-275465


From nginx-forum at forum.nginx.org  Thu Jul 13 11:19:36 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Thu, 13 Jul 2017 07:19:36 -0400
Subject: keepalive connections cache between 2 proxy
Message-ID: <491bc29f63632b1f78e35cc275528ad7.NginxMailingListEnglish@forum.nginx.org>

hello,

my app will keep a long connection with server, while server may push
message to client, two levels nginx proxy for the upstream servers, does
keepalive directive support this model?

the documentation of the keepalive directive said: 
"For HTTP, the proxy_http_version directive should be set to '1.1' and the
'Connection' header field should be cleared."

why clear that header?

if client or server close the connection, will the connection between two
nginx still keepalive?

will nginx creat connection with upstream servers even if there's no request
received?
will nginx pass requests from different clients to upstream in one
keep-alive connection?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275466,275466#msg-275466


From xeioex at nginx.com  Thu Jul 13 12:15:23 2017
From: xeioex at nginx.com (Dmitry Volyntsev)
Date: Thu, 13 Jul 2017 15:15:23 +0300
Subject: nginScript question
In-Reply-To: <7a4b3366a3443261d2418bd3aca5782a.NginxMailingListEnglish@forum.nginx.org>
References: <7a4b3366a3443261d2418bd3aca5782a.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <98aad8c8-8885-58e5-649d-fa8796d7cf96@nginx.com>



On 12.07.2017 23:57, aledbf wrote:
> Hi,
>
> It is possible to make a HTTP request with nginScript?
>

Hi,

Unfortunately, this is not possible so far. But, we are going to add 
such functionality in the future. You can find what is currently 
possible here http://nginx.org/en/docs/http/ngx_http_js_module.html

From sdnetwork at gmail.com  Thu Jul 13 12:18:34 2017
From: sdnetwork at gmail.com (Arnaud Le-roy)
Date: Thu, 13 Jul 2017 14:18:34 +0200
Subject: Nginx Proxy seems to send twice the same request to the backend
In-Reply-To: <20170710134935.GQ55433@mdounin.ru>
References: <4A8C8A80-EF80-4CD3-9C0C-240279B64E06@gmail.com>
 <20170710134935.GQ55433@mdounin.ru>
Message-ID: <2BA068B8-C448-4EF9-9635-AB2497E0BE51@gmail.com>

Thank you very much for you explanation, it is now clear for me !

a little question about the error :

>> 2017/07/09 09:18:31 [error] 38111#38111: *4098505 upstream prematurely closed connection while reading response header from upstream, client: x.x.x.x, server: x.x.com, request: "GET /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428 

can you confirm that is my node server that close the connection, for me it seems strange because it process the two request so from my point of view it can't timeout ?

> Le 10 juil. 2017 ? 15:49, Maxim Dounin <mdounin at mdounin.ru> a ?crit :
> 
> Hello!
> 
> On Sun, Jul 09, 2017 at 11:26:47PM +0200, Arnaud Le-roy wrote:
> 
>> i encountered a strange behaviour with nginx, my backend seems 
>> to receive twice the same request from nginx proxy, to be sure 
>> that it's not the client that send two request i have had an 
>> uuid params to each request.
>> 
>> when the problem occurs in nginx log i found one request in 
>> success in access.log
>> 
>> x.x.x.x - - [09/Jul/2017:09:18:33 +0200] "GET /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428 HTTP/1.1" 200 2 "-" "-"
>> 
>> and an other one than generate this log in error.log :
>> 
>> 2017/07/09 09:18:31 [error] 38111#38111: *4098505 upstream prematurely closed connection while reading response header from upstream, client: x.x.x.x, server: x.x.com, request: "GET /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428 HTTP/1.1", upstream: "http://172.16.0.11:9092/query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428", host: "x.x.com"
>> 
>> on my backend i can see two request with the same uuid (the two 
>> succeed)
>> 
>> {"pid":11424,"level":"info","message":"[API] AUTH1 /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428","timestamp":"2017-07-09 09:18:31.861Z"}
>> {"pid":11424,"level":"info","message":"[API] AUTH1 /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428","timestamp":"2017-07-09 09:18:33.196Z"}
>> 
>> The client is a node program so i'm sure that it sends only one 
>> request with the same uuid (no thread problem ;) the nginx serve 
>> as simple proxy (no load balancing)
> 
> [...]
> 
>>    upstream api {
>>         keepalive 100;
>> 	 server 172.16.0.11:9092;
>>    }
> 
> [...]
> 
>>        location / {
>>            proxy_next_upstream off;
>>            proxy_pass http://api <http://api/>;
> 
> [...]
> 
>> The backend is a simple node server.
>> 
>> the problem occurs randomly, and it happens for sure on 
>> nginx/1.10.3 and nginx/1.13.2 on debian/jessie
>> 
>> After some days of research, i found that if i remove the 
>> keepalive 100 from upstream configuration there is no longer the 
>> problem but i don't understand why ? Maybe somebody can explain 
>> me what could hapen ? maybe a misunderstanding about some 
>> configuration on keep alive ?
> 
> With keepalive connections, it is possible that server will close 
> the connection and the client will start sending a request a the 
> same time.  This case is specifically outlined in RFC 2616,
> https://tools.ietf.org/html/rfc2616#section-8.1.4 <https://tools.ietf.org/html/rfc2616#section-8.1.4>
> 
>   A client, server, or proxy MAY close the transport connection at any
>   time. For example, a client might have started to send a new request
>   at the same time that the server has decided to close the "idle"
>   connection. From the server's point of view, the connection is being
>   closed while it was idle, but from the client's point of view, a
>   request is in progress.
> 
>   This means that clients, servers, and proxies MUST be able to recover
>   from asynchronous close events. Client software SHOULD reopen the
>   transport connection and retransmit the aborted sequence of requests
>   without user interaction so long as the request sequence is
>   idempotent (see section 9.1.2).
> 
> In such situation nginx will re-try the request, as suggested by 
> the standard.  In nginx before 1.9.13 this happened even with 
> "proxy_next_upstream off".
> 
> Though in nginx 1.9.13 this code was rewritten as part of the 
> introduction of non-idempotent methods handling in 
> proxy_next_upstream.  And in nginx 1.9.13+ re-trying the request 
> is not expected to happen unless "proxy_next_upstream error" is 
> also configured.  As such, the exact configuration looks strange 
> combined with the versions you claim.
> 
> Could you please double-check that the behaviour you describe 
> appears when using nginx 1.13.2 and with proxy_next_upstream 
> switched off?
> 
> Just in case, I've tested it here using the following simple nginx 
> configuration:
> 
>    upstream foo {
>        server 127.0.0.1:8081;
>        keepalive 10;
>    }
> 
>    server {
>        listen 8080;
> 
>        location / {
>            proxy_pass http://foo <http://foo/>;
>            proxy_set_header Connection "";
>            proxy_http_version 1.1;
>            proxy_next_upstream off;
>        }
>    }
> 
>    server {
>        listen 8081;
> 
>        if ($connection_requests ~ 2) {
>           return 444;
>        }
> 
>        return 200 ok;
>    }
> 
> As expected, it returns 502 when using "proxy_next_upstream off" 
> and re-tries the request if proxy_next_upstream is not switched 
> off.
> 
> Note well that if duplicate requests introduce a problem, you may 
> want to reconsider logic of your backend.  At least idempotent 
> requests can be freely retried, and your backend is expected to 
> handle this.  (In practice this often happens even with 
> non-idempotent requests too.)
> 
> -- 
> Maxim Dounin
> http://nginx.org/ <http://nginx.org/>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org <mailto:nginx at nginx.org>
> http://mailman.nginx.org/mailman/listinfo/nginx <http://mailman.nginx.org/mailman/listinfo/nginx>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170713/d6a5af75/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature_alr.png
Type: image/png
Size: 18398 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170713/d6a5af75/attachment-0001.png>

From mdounin at mdounin.ru  Thu Jul 13 13:24:53 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 13 Jul 2017 16:24:53 +0300
Subject: Nginx Proxy seems to send twice the same request to the backend
In-Reply-To: <2BA068B8-C448-4EF9-9635-AB2497E0BE51@gmail.com>
References: <4A8C8A80-EF80-4CD3-9C0C-240279B64E06@gmail.com>
 <20170710134935.GQ55433@mdounin.ru>
 <2BA068B8-C448-4EF9-9635-AB2497E0BE51@gmail.com>
Message-ID: <20170713132453.GA55433@mdounin.ru>

Hello!

On Thu, Jul 13, 2017 at 02:18:34PM +0200, Arnaud Le-roy wrote:

> a little question about the error :
> 
> >> 2017/07/09 09:18:31 [error] 38111#38111: *4098505 upstream prematurely closed connection while reading response header from upstream, client: x.x.x.x, server: x.x.com, request: "GET /query?uid=b85cc8a4-b9cd-4093-aea5-95c0ea1391a6_428 
> 
> can you confirm that is my node server that close the 
> connection, for me it seems strange because it process the two 
> request so from my point of view it can't timeout ?

The error indicate that your backend server closed the connection.  
That's more less all nginx knows, so if you want to find out why 
your backend did this - consider looking into the backend.

-- 
Maxim Dounin
http://nginx.org/

From lists at viaduct-productions.com  Thu Jul 13 13:37:08 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Thu, 13 Jul 2017 09:37:08 -0400
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
Message-ID: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>

Hi folks.  Trying to get this FreeBSD nginx installation set up.  

FreeBSD 11.1-RC1
nginx version: nginx/1.12.0

3 vhosts on this box.  nginx.conf tests show the following: 

[Wed Jul 12 06:08:41 rich at neb /var/log/nginx] nginx -t
nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: [emerg] open() "/var/run/nginx.pid" failed (13: Permission denied)
nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed

Here are those permissions (644):

-rw-r--r--   1 root  wheel        4 Jul 11 15:59 nginx.pid

Not sure why nginx is not happy with those restrictions.  It has read/write permissions errors on a process ID file.  Confusing.

Any ideas how I can troubleshoot this?  That nginx.pid permissions issue is halting the whole server.  

Cheers

_____________
Rich in Toronto @ VP







From mdounin at mdounin.ru  Thu Jul 13 14:13:34 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 13 Jul 2017 17:13:34 +0300
Subject: nginx security advisory (CVE-2017-7529)
In-Reply-To: <f9845f6b1484b121eb1e9767171a639b.NginxMailingListEnglish@forum.nginx.org>
References: <20170712120132.GR55433@mdounin.ru>
 <f9845f6b1484b121eb1e9767171a639b.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170713141334.GB55433@mdounin.ru>

Hello!

On Wed, Jul 12, 2017 at 09:42:04PM -0400, martinzhou wrote:

> Maxim Dounin Wrote:
> -------------------------------------------------------
> > Hello!
> > 
> > On Tue, Jul 11, 2017 at 05:45:15PM -0400, c0nw0nk wrote:
> > 
> > > Couldn't you use 
> > > 
> > > max_ranges 0;
> > > 
> > > To disable byte range support completely.
> > 
> > Disabling ranges completely will mitigate the issue as well.  But 
> > as the issue only affects requests with multiple ranges, it is not 
> > needed, "max_ranges 1;" is enough.
> > 
> > > Also won't setting the value of ranges to max_ranges 1; break pseudo
> > > streaming in HTML5 video apps etc. ?
> > 
> > No, pseudo streaming generally uses requests with a single range, 
> > and these are allowed with "max_ranges 1;".  Requests with 
> > multiple ranges are very rare in practice (AFAIK, they are used 
> > by Adobe Acrobat and MS Office, but I've never heard of anything 
> > more popular than that).
> 
> I found that in some cases (when the browser is requesting for a mp3 file),
> the HTTP header will be formed as "Range: bytes=1-100, 200-100". I'm

AFAIK, no general-purpose browsers do this, at least no popular 
ones.  Some music players may do so though.

> wondering if we set "max_ranges 0;" or "max_ranges 1;" in the config, it
> will cause the failure of loading such files. 

Full response with code 200 will be returned to the client.  This 
is valid response as per RFC, and all HTTP-complaint clients are 
expected to understand it and handle it properly.  Also, this is 
what happens regularly when a server does not support range 
requests, so is highly unlikely to break any clients.

I wouldn't recommend using "max_range 0;" though, as it will 
disable single-range requests as well, and this means that 
download resumption and seeking won't work.

> Also, I'm wondering if I've already set a comparatively "big" number after
> the "max_ranges", for example, "max_ranges 100;", do I still need to adjust
> the number to a low value (e.g. "1" or "2")?

For the workaround to work, multi-range requests need to be 
disabled.  That is, you should use "max_ranges 1;".

-- 
Maxim Dounin
http://nginx.org/

From tkadm30 at yandex.com  Thu Jul 13 14:25:32 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Thu, 13 Jul 2017 10:25:32 -0400
Subject: Duplicated response body with fastcgi
Message-ID: <2096f882-1f94-6b98-4de5-446b7d7753fa@yandex.com>

Hi,
I'm trying to setup a Django app with nginx using fastcgi. Here's my config:

# configuration file /etc/nginx/nginx.conf:
user www-data;
worker_processes 4;
pid /run/nginx.pid;

events {
     worker_connections 512;
     multi_accept on;
     use epoll;
}

http {

     ##
     # Basic Settings
     ##

     sendfile on;
     tcp_nopush on;
     tcp_nodelay on;
     keepalive_timeout 80;
     types_hash_max_size 2048;
     # server_tokens off;

     # server_names_hash_bucket_size 64;
     # server_name_in_redirect off;

     include /etc/nginx/mime.types;
     default_type application/octet-stream;

     ##
     # SSL Settings
     ##

     ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
     ssl_prefer_server_ciphers on;

     ##
     # Logging Settings
     ##

     access_log /var/log/nginx/access.log;
     error_log /var/log/nginx/error.log;

     ##
     # Gzip Settings
     ##

     gzip off;
     gzip_disable "msie6";

     # gzip_vary on;
     # gzip_proxied any;
     # gzip_comp_level 6;
     # gzip_buffers 16 8k;
     # gzip_http_version 1.1;
     # gzip_types text/plain text/css application/json 
application/javascript text/xml application/xml application/xml+rss 
text/javascript;

     ##
     # Virtual Host Configs
     ##

     include /etc/nginx/conf.d/development.conf;
     #include /etc/nginx/sites-enabled/*;
}


#mail {
#    # See sample authentication script at:
#    # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
#    # auth_http localhost/auth.php;
#    # pop3_capabilities "TOP" "USER";
#    # imap_capabilities "IMAP4rev1" "UIDPLUS";
#
#    server {
#        listen     localhost:110;
#        protocol   pop3;
#        proxy      on;
#    }
#
#    server {
#        listen     localhost:143;
#        protocol   imap;
#        proxy      on;
#    }
#}

# configuration file /etc/nginx/mime.types:

types {
     text/html                             html htm shtml;
     text/css                              css;
     text/xml                              xml;
     image/gif                             gif;
     image/jpeg                            jpeg jpg;
     application/javascript                js;
     application/atom+xml                  atom;
     application/rss+xml                   rss;

     text/mathml                           mml;
     text/plain                            txt;
     text/vnd.sun.j2me.app-descriptor      jad;
     text/vnd.wap.wml                      wml;
     text/x-component                      htc;

     image/png                             png;
     image/tiff                            tif tiff;
     image/vnd.wap.wbmp                    wbmp;
     image/x-icon                          ico;
     image/x-jng                           jng;
     image/x-ms-bmp                        bmp;
     image/svg+xml                         svg svgz;
     image/webp                            webp;

     application/font-woff                 woff;
     application/java-archive              jar war ear;
     application/json                      json;
     application/mac-binhex40              hqx;
     application/msword                    doc;
     application/pdf                       pdf;
     application/postscript                ps eps ai;
     application/rtf                       rtf;
     application/vnd.apple.mpegurl         m3u8;
     application/vnd.ms-excel              xls;
     application/vnd.ms-fontobject         eot;
     application/vnd.ms-powerpoint         ppt;
     application/vnd.wap.wmlc              wmlc;
     application/vnd.google-earth.kml+xml  kml;
     application/vnd.google-earth.kmz      kmz;
     application/x-7z-compressed           7z;
     application/x-cocoa                   cco;
     application/x-java-archive-diff       jardiff;
     application/x-java-jnlp-file          jnlp;
     application/x-makeself                run;
     application/x-perl                    pl pm;
     application/x-pilot                   prc pdb;
     application/x-rar-compressed          rar;
     application/x-redhat-package-manager  rpm;
     application/x-sea                     sea;
     application/x-shockwave-flash         swf;
     application/x-stuffit                 sit;
     application/x-tcl                     tcl tk;
     application/x-x509-ca-cert            der pem crt;
     application/x-xpinstall               xpi;
     application/xhtml+xml                 xhtml;
     application/xspf+xml                  xspf;
     application/zip                       zip;

     application/octet-stream              bin exe dll;
     application/octet-stream              deb;
     application/octet-stream              dmg;
     application/octet-stream              iso img;
     application/octet-stream              msi msp msm;

application/vnd.openxmlformats-officedocument.wordprocessingml.document 
docx;
application/vnd.openxmlformats-officedocument.spreadsheetml.sheet xlsx;
application/vnd.openxmlformats-officedocument.presentationml.presentation 
pptx;

     audio/midi                            mid midi kar;
     audio/mpeg                            mp3;
     audio/ogg                             ogg;
     audio/x-m4a                           m4a;
     audio/x-realaudio                     ra;

     video/3gpp                            3gpp 3gp;
     video/mp2t                            ts;
     video/mp4                             mp4;
     video/mpeg                            mpeg mpg;
     video/quicktime                       mov;
     video/webm                            webm;
     video/x-flv                           flv;
     video/x-m4v                           m4v;
     video/x-mng                           mng;
     video/x-ms-asf                        asx asf;
     video/x-ms-wmv                        wmv;
     video/x-msvideo                       avi;
}

# configuration file /etc/nginx/conf.d/development.conf:
server {

     # static medias web server configuration, for development
     # and testing purposes.

     listen       80;
     server_name  localhost;
     error_log /var/log/nginx/error_log; #debug
     #access_log  /var/log/nginx/gthc.org/access.log;
     root /home/erob/www/isotopesoftware.ca;
     #autoindex on;

     location / {
     #    # host and port to fastcgi server
         fastcgi_pass 127.0.0.1:8808; # 8808=gthc.org; 8801=tm
         include fastcgi_params;
         autoindex on;
     #    # rewrite /CamelCase to /wiki/CamelCase
     #    rewrite ^/(.*[A-Z][a-z]*)$ /wiki$1 last;
         #etag on;
     #fastcgi_pass_header $http_if_none_match;
     }


     # debug url rewriting to the error log
     rewrite_log on;

     location /media {
         autoindex on;
         gzip on;
     }

     location /pub {
         autoindex on;
         gzip on;
     }

     location /webalizer {
         autoindex on;
         gzip on;
     #auth_basic "Private Property";
     #auth_basic_user_file /etc/nginx/.htpasswd;
         allow 67.68.76.70;
     deny all;
     }

     location /documentation {
         autoindex on;
         gzip on;
     }

     location /moin_static184 {
     autoindex on;
     gzip on;
     }
     location /favicon.ico {
     empty_gif;
     }
     location /robots.txt {
          root /home/www/isotopesoftware.ca;
     }
     location /sitemap.xml {
     root /home/www/isotopesoftware.ca;
     }

     #location /public_html {
     # root /home/www/;
     # autoindex on;
     #}
     # redirect server error pages to the static page /50x.html
     #error_page 404 /404.html;
     #error_page 403    /403.html;
     #error_page 500 502 503 504  /50x.html;
     #location = /50x.html {
     #    root   /var/www/nginx-default;
     #}

     include conf.d/moinmoin.conf;
     include conf.d/livestore.conf;
}


# configuration file /etc/nginx/fastcgi_params:
fastcgi_param  PATH_INFO          $fastcgi_script_name;
fastcgi_param  QUERY_STRING       $query_string;
fastcgi_param  REQUEST_METHOD     $request_method;
fastcgi_param  CONTENT_TYPE       $content_type;
fastcgi_param  CONTENT_LENGTH     $content_length;

fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
fastcgi_param  REQUEST_URI        $request_uri;
fastcgi_param  DOCUMENT_URI       $document_uri;
fastcgi_param  DOCUMENT_ROOT      $document_root;
fastcgi_param  SERVER_PROTOCOL    $server_protocol;

fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
fastcgi_param  SERVER_SOFTWARE    nginx;

fastcgi_param  REMOTE_ADDR        $remote_addr;
fastcgi_param  REMOTE_PORT        $remote_port;
#fastcgi_param  REMOTE_USER      $remote_user;
fastcgi_param  SERVER_ADDR        $server_addr;
fastcgi_param  SERVER_PORT        $server_port;
fastcgi_param  SERVER_NAME        $server_name;


#XXX
#fastcgi_param HTTP_IF_NONE_MATCH $http_if_none_match;
#fastcgi_param HTTP_IF_MODIFIED_SINCE $http_if_modified_since;


# PHP only, required if PHP was built with --enable-force-cgi-redirect
# fastcgi_param  REDIRECT_STATUS    200;

fastcgi_send_timeout 90;
fastcgi_read_timeout 90;
fastcgi_connect_timeout 40;
fastcgi_cache_valid 200 304 10m;
#fastcgi_buffer_size 128k;
#fastcgi_buffers 8 128k;
#fastcgi_busy_buffers_size 256k;
#fastcgi_temp_file_write_size 256k;


# configuration file /etc/nginx/conf.d/moinmoin.conf:


location /wiki {


       if ($uri ~ ^/wiki(.*)?){
          set $wiki_url $1;
       }
       # host and port to fastcgi server
       fastcgi_pass 127.0.0.1:8807; # 8808=gthc.org; 8801=tm; 8807=moinmoin
       #include fastcgi_params;
       fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
       fastcgi_param  SERVER_SOFTWARE    nginx;
       fastcgi_param  QUERY_STRING       $query_string;
       fastcgi_param  REQUEST_METHOD     $request_method;
       fastcgi_param  CONTENT_TYPE       $content_type;
       fastcgi_param  CONTENT_LENGTH     $content_length;
       fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
       #fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
       fastcgi_param  REQUEST_URI        $request_uri;
       fastcgi_param  DOCUMENT_URI       $document_uri;
       fastcgi_param  DOCUMENT_ROOT      $document_root;
       fastcgi_param  SERVER_PROTOCOL    $server_protocol;
       fastcgi_param  REMOTE_ADDR        $remote_addr;
       fastcgi_param  REMOTE_PORT        $remote_port;
       fastcgi_param  SERVER_ADDR        $server_addr;
       fastcgi_param  SERVER_PORT        $server_port;
       fastcgi_param  SERVER_NAME        $server_name;
       fastcgi_param PATH_INFO $wiki_url;
       fastcgi_param SCRIPT_NAME /wiki;
}
#location /moin_static184 {
#    root /home/www/isotopesoftware.ca;
#    autoindex on;
#}



# configuration file /etc/nginx/conf.d/livestore.conf:


location /store {


       if ($uri ~ ^/store(.*)?){
          set $store_url $1;
       }
       # host and port to fastcgi server
       fastcgi_pass 127.0.0.1:8809; # 8808=gthc.org; 8801=tm; 8807=moinmoin
       #include fastcgi_params;
       fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
       fastcgi_param  SERVER_SOFTWARE    nginx;
       fastcgi_param  QUERY_STRING       $query_string;
       fastcgi_param  REQUEST_METHOD     $request_method;
       fastcgi_param  CONTENT_TYPE       $content_type;
       fastcgi_param  CONTENT_LENGTH     $content_length;
       fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
       #fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
       fastcgi_param  REQUEST_URI        $request_uri;
       fastcgi_param  DOCUMENT_URI       $document_uri;
       fastcgi_param  DOCUMENT_ROOT      $document_root;
       fastcgi_param  SERVER_PROTOCOL    $server_protocol;
       fastcgi_param  REMOTE_ADDR        $remote_addr;
       fastcgi_param  REMOTE_PORT        $remote_port;
       fastcgi_param  SERVER_ADDR        $server_addr;
       fastcgi_param  SERVER_PORT        $server_port;
       fastcgi_param  SERVER_NAME        $server_name;
       fastcgi_param PATH_INFO $store_url;
       fastcgi_param SCRIPT_NAME /store;
}
location /satchmo_media {
     root /home/www/livestore;
     autoindex on;
     gzip on;
     expires 30d;
}

The problem is that nginx is duplicating the response body stream for 
the /store location. This issue does not happen in WSGI mode.

Any ideas how to fix?

Thank you in advance,

E




From nginx-forum at forum.nginx.org  Thu Jul 13 15:14:51 2017
From: nginx-forum at forum.nginx.org (aledbf)
Date: Thu, 13 Jul 2017 11:14:51 -0400
Subject: nginScript question
In-Reply-To: <98aad8c8-8885-58e5-649d-fa8796d7cf96@nginx.com>
References: <98aad8c8-8885-58e5-649d-fa8796d7cf96@nginx.com>
Message-ID: <786faa9105dbfec1a87376b081cca5e7.NginxMailingListEnglish@forum.nginx.org>

Thanks!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275459,275477#msg-275477


From nginx-forum at forum.nginx.org  Thu Jul 13 18:38:58 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Thu, 13 Jul 2017 14:38:58 -0400
Subject: how nginx deal with gzipped responses when gzip is on
Message-ID: <2dc113bfa8959b40eeaca6bf7618950a.NginxMailingListEnglish@forum.nginx.org>

i configred nginx with gzip directives:
http { 
   gzip            on;
    gzip_types      text/css application/javascript;
    gzip_comp_level 9;
    gzip_http_version 1.0;
    gzip_proxied    any;
}

some upstream servers have already gzipped the response, i see the lengh of
the respense is similar to that send from nginx to client, i'm very curious
to know if nginx just bypassed that response, zipped again or
unzipped&rezipped it.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275479,275479#msg-275479


From francis at daoine.org  Thu Jul 13 22:46:12 2017
From: francis at daoine.org (Francis Daly)
Date: Thu, 13 Jul 2017 23:46:12 +0100
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
Message-ID: <20170713224612.GE365@daoine.org>

On Thu, Jul 13, 2017 at 09:37:08AM -0400, Viaduct Lists wrote:

Hi there,

> [Wed Jul 12 06:08:41 rich at neb /var/log/nginx] nginx -t

If you were running this command as "root", would that prompt say
"root at neb" and end with a # ?

> nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
> nginx: [emerg] open() "/var/run/nginx.pid" failed (13: Permission denied)

That might relate to permissions on /, /var, or /var/run, instead of
on /var/run/nginx.pid.

But still: from what you've shown, there is no indication that user
"rich" has the necessary permissions.

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org

From lists at viaduct-productions.com  Thu Jul 13 22:58:02 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Thu, 13 Jul 2017 18:58:02 -0400
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <20170713224612.GE365@daoine.org>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
Message-ID: <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>

Hi there.  Thanks for the reply.

Persistent permissions issues are on other boxes, on OSX as well.  But I had some Passenger issues so I?ve moved into another issue.

But sudo nginx -t gets rid of the error on nginx.pid

That whole user/group issue on the user directive in nginx.conf is confusing as it ignores any attempt at using user root;.  

So it has no user.  The install has user nobody; but that isn?t used, so the user is www. 



> On Jul 13, 2017, at 6:46 PM, Francis Daly <francis at daoine.org> wrote:
> 
> Hi there,
> 
>> [Wed Jul 12 06:08:41 rich at neb /var/log/nginx] nginx -t
> 
> If you were running this command as "root", would that prompt say
> "root at neb" and end with a # ?
> 
>> nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
>> nginx: [emerg] open() "/var/run/nginx.pid" failed (13: Permission denied)
> 
> That might relate to permissions on /, /var, or /var/run, instead of
> on /var/run/nginx.pid.
> 
> But still: from what you've shown, there is no indication that user
> "rich" has the necessary permissions.
> 
> Good luck with it,


_____________
Rich in Toronto @ VP







From zchao1995 at gmail.com  Fri Jul 14 01:25:00 2017
From: zchao1995 at gmail.com (Zhang Chao)
Date: Thu, 13 Jul 2017 18:25:00 -0700
Subject: how nginx deal with gzipped responses when gzip is on
In-Reply-To: <2dc113bfa8959b40eeaca6bf7618950a.NginxMailingListEnglish@forum.nginx.org>
References: <2dc113bfa8959b40eeaca6bf7618950a.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <CAPr95Bh6dbZQm+Ff1C=iRKamH9iP5DKknDpQyWb5O+u8U=gktA@mail.gmail.com>

Hi!

NGINX will check the Content-Encoding header, so if this header exists,
gzip filter will be bypassed.


On 14 July 2017 at 02:39:07, foxgab (nginx-forum at forum.nginx.org) wrote:

i configred nginx with gzip directives:
http {
gzip on;
gzip_types text/css application/javascript;
gzip_comp_level 9;
gzip_http_version 1.0;
gzip_proxied any;
}

some upstream servers have already gzipped the response, i see the lengh of
the respense is similar to that send from nginx to client, i'm very curious
to know if nginx just bypassed that response, zipped again or
unzipped&rezipped it.

Posted at Nginx Forum:
https://forum.nginx.org/read.php?2,275479,275479#msg-275479

_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170713/539330b5/attachment.html>

From lists at lazygranch.com  Fri Jul 14 01:31:51 2017
From: lists at lazygranch.com (lists at lazygranch.com)
Date: Thu, 13 Jul 2017 18:31:51 -0700
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <20170713224612.GE365@daoine.org>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
Message-ID: <20170713183151.6040c0ba.lists@lazygranch.com>

On Thu, 13 Jul 2017 23:46:12 +0100
Francis Daly <francis at daoine.org> wrote:

> On Thu, Jul 13, 2017 at 09:37:08AM -0400, Viaduct Lists wrote:
> 
> Hi there,
> 
> > [Wed Jul 12 06:08:41 rich at neb /var/log/nginx] nginx -t  
> 
> If you were running this command as "root", would that prompt say
> "root at neb" and end with a # ?
> 
> > nginx: the configuration file /usr/local/etc/nginx/nginx.conf
> > syntax is ok nginx: [emerg] open() "/var/run/nginx.pid" failed (13:
> > Permission denied)  
> 
> That might relate to permissions on /, /var, or /var/run, instead of
> on /var/run/nginx.pid.
> 
> But still: from what you've shown, there is no indication that user
> "rich" has the necessary permissions.
> 
> Good luck with it,
> 
> 
From FreeBSD 11.0
	f
/var
drwxr-xr-x  28 root  wheel      512 Jul 12 08:00 var

/var/run
drwxr-xr-x  10 root     wheel     1024 Jul 13 03:01 run

/usr/local/www
drwxr-xr-x   5 root  wheel   512 Jun 18 04:13 www

Contents of /usr/local/www
drwxr-xr-x  3 root  wheel  512 Jun 10 00:19 .well-known
drwxr-xr-x  2 root  www    512 Jun 12 01:16 acme
lrwxr-xr-x  1 root  wheel   25 Jun 18 04:13 nginx
-> /usr/local/www/nginx-dist dr-xr-xr-x  5 root  wheel  512 Jun 18
04:13 nginx-dist

Contents of /usr/local/www/nginx-dist/
-rw-r--r--  1 root  wheel  537 Jun 18 04:12 50x.html
-rw-r--r--  1 root  wheel    1 Jun 18 04:12
EXAMPLE_DIRECTORY-DONT_ADD_OR_TOUCH_ANYTHING -rw-r--r--  1 root  wheel
612 Jun 18 04:12 index.html drwxr-xr-x  2 root  wheel  512 Jun  9 07:00
<mydomain.com>

However the nginx process is owned by www:
 823 www           1  20    0 28552K  7060K kqread   0:01   0.00% nginx





/var/run/ngin.pid
-rw-r--r--  1 root      wheel       4 Jul 12 08:00 nginx.pid

From nginx-forum at forum.nginx.org  Fri Jul 14 03:06:04 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Thu, 13 Jul 2017 23:06:04 -0400
Subject: upstream keepalive connections for all servers or each server?
In-Reply-To: <20170511005724.GO55433@mdounin.ru>
References: <20170511005724.GO55433@mdounin.ru>
Message-ID: <c37d5321846e1057eed6e7622cd8c91f.NginxMailingListEnglish@forum.nginx.org>

hi,

how nginx determines idle connections? is keepalive adaptable with
websocket?
will "proxy_ignore_client_abort off" setting exclude keepalive function?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,274098,275484#msg-275484


From nginx-forum at forum.nginx.org  Fri Jul 14 06:15:49 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Fri, 14 Jul 2017 02:15:49 -0400
Subject: Reverse Proxy with 500k connections
In-Reply-To: <CAJaC8YvnTGPRX7LzimeYrS2K4D7dgQZgv=SqpRFhP8YdkkaZ0Q@mail.gmail.com>
References: <CAJaC8YvnTGPRX7LzimeYrS2K4D7dgQZgv=SqpRFhP8YdkkaZ0Q@mail.gmail.com>
Message-ID: <92fe03862f8a747a4da0f2477f204be5.NginxMailingListEnglish@forum.nginx.org>

is upstream keepalive connetions adaptable with websocket?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,272808,275486#msg-275486


From tkadm30 at yandex.com  Fri Jul 14 12:32:01 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Fri, 14 Jul 2017 08:32:01 -0400
Subject: Duplicated response body with fastcgi
In-Reply-To: <2096f882-1f94-6b98-4de5-446b7d7753fa@yandex.com>
References: <2096f882-1f94-6b98-4de5-446b7d7753fa@yandex.com>
Message-ID: <2319d470-0be8-60f4-89dd-2e70a266e60c@yandex.com>

How can one debug the upstream FastCGI response from nginx?



Le 2017-07-13 ? 10:25, Etienne Robillard a ?crit :
> Hi,
> I'm trying to setup a Django app with nginx using fastcgi. Here's my 
> config:
>
> # configuration file /etc/nginx/nginx.conf:
> user www-data;
> worker_processes 4;
> pid /run/nginx.pid;
>
> events {
>     worker_connections 512;
>     multi_accept on;
>     use epoll;
> }
>
> http {
>
>     ##
>     # Basic Settings
>     ##
>
>     sendfile on;
>     tcp_nopush on;
>     tcp_nodelay on;
>     keepalive_timeout 80;
>     types_hash_max_size 2048;
>     # server_tokens off;
>
>     # server_names_hash_bucket_size 64;
>     # server_name_in_redirect off;
>
>     include /etc/nginx/mime.types;
>     default_type application/octet-stream;
>
>     ##
>     # SSL Settings
>     ##
>
>     ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
>     ssl_prefer_server_ciphers on;
>
>     ##
>     # Logging Settings
>     ##
>
>     access_log /var/log/nginx/access.log;
>     error_log /var/log/nginx/error.log;
>
>     ##
>     # Gzip Settings
>     ##
>
>     gzip off;
>     gzip_disable "msie6";
>
>     # gzip_vary on;
>     # gzip_proxied any;
>     # gzip_comp_level 6;
>     # gzip_buffers 16 8k;
>     # gzip_http_version 1.1;
>     # gzip_types text/plain text/css application/json 
> application/javascript text/xml application/xml application/xml+rss 
> text/javascript;
>
>     ##
>     # Virtual Host Configs
>     ##
>
>     include /etc/nginx/conf.d/development.conf;
>     #include /etc/nginx/sites-enabled/*;
> }
>
>
> #mail {
> #    # See sample authentication script at:
> #    # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
> #
> #    # auth_http localhost/auth.php;
> #    # pop3_capabilities "TOP" "USER";
> #    # imap_capabilities "IMAP4rev1" "UIDPLUS";
> #
> #    server {
> #        listen     localhost:110;
> #        protocol   pop3;
> #        proxy      on;
> #    }
> #
> #    server {
> #        listen     localhost:143;
> #        protocol   imap;
> #        proxy      on;
> #    }
> #}
>
> # configuration file /etc/nginx/mime.types:
>
> types {
>     text/html                             html htm shtml;
>     text/css                              css;
>     text/xml                              xml;
>     image/gif                             gif;
>     image/jpeg                            jpeg jpg;
>     application/javascript                js;
>     application/atom+xml                  atom;
>     application/rss+xml                   rss;
>
>     text/mathml                           mml;
>     text/plain                            txt;
>     text/vnd.sun.j2me.app-descriptor      jad;
>     text/vnd.wap.wml                      wml;
>     text/x-component                      htc;
>
>     image/png                             png;
>     image/tiff                            tif tiff;
>     image/vnd.wap.wbmp                    wbmp;
>     image/x-icon                          ico;
>     image/x-jng                           jng;
>     image/x-ms-bmp                        bmp;
>     image/svg+xml                         svg svgz;
>     image/webp                            webp;
>
>     application/font-woff                 woff;
>     application/java-archive              jar war ear;
>     application/json                      json;
>     application/mac-binhex40              hqx;
>     application/msword                    doc;
>     application/pdf                       pdf;
>     application/postscript                ps eps ai;
>     application/rtf                       rtf;
>     application/vnd.apple.mpegurl         m3u8;
>     application/vnd.ms-excel              xls;
>     application/vnd.ms-fontobject         eot;
>     application/vnd.ms-powerpoint         ppt;
>     application/vnd.wap.wmlc              wmlc;
>     application/vnd.google-earth.kml+xml  kml;
>     application/vnd.google-earth.kmz      kmz;
>     application/x-7z-compressed           7z;
>     application/x-cocoa                   cco;
>     application/x-java-archive-diff       jardiff;
>     application/x-java-jnlp-file          jnlp;
>     application/x-makeself                run;
>     application/x-perl                    pl pm;
>     application/x-pilot                   prc pdb;
>     application/x-rar-compressed          rar;
>     application/x-redhat-package-manager  rpm;
>     application/x-sea                     sea;
>     application/x-shockwave-flash         swf;
>     application/x-stuffit                 sit;
>     application/x-tcl                     tcl tk;
>     application/x-x509-ca-cert            der pem crt;
>     application/x-xpinstall               xpi;
>     application/xhtml+xml                 xhtml;
>     application/xspf+xml                  xspf;
>     application/zip                       zip;
>
>     application/octet-stream              bin exe dll;
>     application/octet-stream              deb;
>     application/octet-stream              dmg;
>     application/octet-stream              iso img;
>     application/octet-stream              msi msp msm;
>
> application/vnd.openxmlformats-officedocument.wordprocessingml.document 
> docx;
> application/vnd.openxmlformats-officedocument.spreadsheetml.sheet xlsx;
> application/vnd.openxmlformats-officedocument.presentationml.presentation 
> pptx;
>
>     audio/midi                            mid midi kar;
>     audio/mpeg                            mp3;
>     audio/ogg                             ogg;
>     audio/x-m4a                           m4a;
>     audio/x-realaudio                     ra;
>
>     video/3gpp                            3gpp 3gp;
>     video/mp2t                            ts;
>     video/mp4                             mp4;
>     video/mpeg                            mpeg mpg;
>     video/quicktime                       mov;
>     video/webm                            webm;
>     video/x-flv                           flv;
>     video/x-m4v                           m4v;
>     video/x-mng                           mng;
>     video/x-ms-asf                        asx asf;
>     video/x-ms-wmv                        wmv;
>     video/x-msvideo                       avi;
> }
>
> # configuration file /etc/nginx/conf.d/development.conf:
> server {
>
>     # static medias web server configuration, for development
>     # and testing purposes.
>
>     listen       80;
>     server_name  localhost;
>     error_log /var/log/nginx/error_log; #debug
>     #access_log  /var/log/nginx/gthc.org/access.log;
>     root /home/erob/www/isotopesoftware.ca;
>     #autoindex on;
>
>     location / {
>     #    # host and port to fastcgi server
>         fastcgi_pass 127.0.0.1:8808; # 8808=gthc.org; 8801=tm
>         include fastcgi_params;
>         autoindex on;
>     #    # rewrite /CamelCase to /wiki/CamelCase
>     #    rewrite ^/(.*[A-Z][a-z]*)$ /wiki$1 last;
>         #etag on;
>     #fastcgi_pass_header $http_if_none_match;
>     }
>
>
>     # debug url rewriting to the error log
>     rewrite_log on;
>
>     location /media {
>         autoindex on;
>         gzip on;
>     }
>
>     location /pub {
>         autoindex on;
>         gzip on;
>     }
>
>     location /webalizer {
>         autoindex on;
>         gzip on;
>     #auth_basic "Private Property";
>     #auth_basic_user_file /etc/nginx/.htpasswd;
>         allow 67.68.76.70;
>     deny all;
>     }
>
>     location /documentation {
>         autoindex on;
>         gzip on;
>     }
>
>     location /moin_static184 {
>     autoindex on;
>     gzip on;
>     }
>     location /favicon.ico {
>     empty_gif;
>     }
>     location /robots.txt {
>          root /home/www/isotopesoftware.ca;
>     }
>     location /sitemap.xml {
>     root /home/www/isotopesoftware.ca;
>     }
>
>     #location /public_html {
>     # root /home/www/;
>     # autoindex on;
>     #}
>     # redirect server error pages to the static page /50x.html
>     #error_page 404 /404.html;
>     #error_page 403    /403.html;
>     #error_page 500 502 503 504  /50x.html;
>     #location = /50x.html {
>     #    root   /var/www/nginx-default;
>     #}
>
>     include conf.d/moinmoin.conf;
>     include conf.d/livestore.conf;
> }
>
>
> # configuration file /etc/nginx/fastcgi_params:
> fastcgi_param  PATH_INFO          $fastcgi_script_name;
> fastcgi_param  QUERY_STRING       $query_string;
> fastcgi_param  REQUEST_METHOD     $request_method;
> fastcgi_param  CONTENT_TYPE       $content_type;
> fastcgi_param  CONTENT_LENGTH     $content_length;
>
> fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
> fastcgi_param  REQUEST_URI        $request_uri;
> fastcgi_param  DOCUMENT_URI       $document_uri;
> fastcgi_param  DOCUMENT_ROOT      $document_root;
> fastcgi_param  SERVER_PROTOCOL    $server_protocol;
>
> fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
> fastcgi_param  SERVER_SOFTWARE    nginx;
>
> fastcgi_param  REMOTE_ADDR        $remote_addr;
> fastcgi_param  REMOTE_PORT        $remote_port;
> #fastcgi_param  REMOTE_USER      $remote_user;
> fastcgi_param  SERVER_ADDR        $server_addr;
> fastcgi_param  SERVER_PORT        $server_port;
> fastcgi_param  SERVER_NAME        $server_name;
>
>
> #XXX
> #fastcgi_param HTTP_IF_NONE_MATCH $http_if_none_match;
> #fastcgi_param HTTP_IF_MODIFIED_SINCE $http_if_modified_since;
>
>
> # PHP only, required if PHP was built with --enable-force-cgi-redirect
> # fastcgi_param  REDIRECT_STATUS    200;
>
> fastcgi_send_timeout 90;
> fastcgi_read_timeout 90;
> fastcgi_connect_timeout 40;
> fastcgi_cache_valid 200 304 10m;
> #fastcgi_buffer_size 128k;
> #fastcgi_buffers 8 128k;
> #fastcgi_busy_buffers_size 256k;
> #fastcgi_temp_file_write_size 256k;
>
>
> # configuration file /etc/nginx/conf.d/moinmoin.conf:
>
>
> location /wiki {
>
>
>       if ($uri ~ ^/wiki(.*)?){
>          set $wiki_url $1;
>       }
>       # host and port to fastcgi server
>       fastcgi_pass 127.0.0.1:8807; # 8808=gthc.org; 8801=tm; 
> 8807=moinmoin
>       #include fastcgi_params;
>       fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
>       fastcgi_param  SERVER_SOFTWARE    nginx;
>       fastcgi_param  QUERY_STRING       $query_string;
>       fastcgi_param  REQUEST_METHOD     $request_method;
>       fastcgi_param  CONTENT_TYPE       $content_type;
>       fastcgi_param  CONTENT_LENGTH     $content_length;
>       fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
>       #fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
>       fastcgi_param  REQUEST_URI        $request_uri;
>       fastcgi_param  DOCUMENT_URI       $document_uri;
>       fastcgi_param  DOCUMENT_ROOT      $document_root;
>       fastcgi_param  SERVER_PROTOCOL    $server_protocol;
>       fastcgi_param  REMOTE_ADDR        $remote_addr;
>       fastcgi_param  REMOTE_PORT        $remote_port;
>       fastcgi_param  SERVER_ADDR        $server_addr;
>       fastcgi_param  SERVER_PORT        $server_port;
>       fastcgi_param  SERVER_NAME        $server_name;
>       fastcgi_param PATH_INFO $wiki_url;
>       fastcgi_param SCRIPT_NAME /wiki;
> }
> #location /moin_static184 {
> #    root /home/www/isotopesoftware.ca;
> #    autoindex on;
> #}
>
>
>
> # configuration file /etc/nginx/conf.d/livestore.conf:
>
>
> location /store {
>
>
>       if ($uri ~ ^/store(.*)?){
>          set $store_url $1;
>       }
>       # host and port to fastcgi server
>       fastcgi_pass 127.0.0.1:8809; # 8808=gthc.org; 8801=tm; 
> 8807=moinmoin
>       #include fastcgi_params;
>       fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
>       fastcgi_param  SERVER_SOFTWARE    nginx;
>       fastcgi_param  QUERY_STRING       $query_string;
>       fastcgi_param  REQUEST_METHOD     $request_method;
>       fastcgi_param  CONTENT_TYPE       $content_type;
>       fastcgi_param  CONTENT_LENGTH     $content_length;
>       fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
>       #fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
>       fastcgi_param  REQUEST_URI        $request_uri;
>       fastcgi_param  DOCUMENT_URI       $document_uri;
>       fastcgi_param  DOCUMENT_ROOT      $document_root;
>       fastcgi_param  SERVER_PROTOCOL    $server_protocol;
>       fastcgi_param  REMOTE_ADDR        $remote_addr;
>       fastcgi_param  REMOTE_PORT        $remote_port;
>       fastcgi_param  SERVER_ADDR        $server_addr;
>       fastcgi_param  SERVER_PORT        $server_port;
>       fastcgi_param  SERVER_NAME        $server_name;
>       fastcgi_param PATH_INFO $store_url;
>       fastcgi_param SCRIPT_NAME /store;
> }
> location /satchmo_media {
>     root /home/www/livestore;
>     autoindex on;
>     gzip on;
>     expires 30d;
> }
>
> The problem is that nginx is duplicating the response body stream for 
> the /store location. This issue does not happen in WSGI mode.
>
> Any ideas how to fix?
>
> Thank you in advance,
>
> E
>
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From francis at daoine.org  Fri Jul 14 13:29:56 2017
From: francis at daoine.org (Francis Daly)
Date: Fri, 14 Jul 2017 14:29:56 +0100
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>
Message-ID: <20170714132956.GF365@daoine.org>

On Thu, Jul 13, 2017 at 06:58:02PM -0400, Viaduct Lists wrote:

Hi there,

> But sudo nginx -t gets rid of the error on nginx.pid
> 
> That whole user/group issue on the user directive in nginx.conf is confusing as it ignores any attempt at using user root;.  

In unix land, usually, if a process starts running as root, then it is
able to "switch" to run as another user. If a process starts running as
non-root, it is not able to switch to run as another user.

And (usually) only root is able to bind() to port 80.

(Exceptions exist, but this is probably good enough for now.)

So -- if you start the nginx process as root (either by running it as
root, or by running it under "sudo"), then nginx can listen on port 80
and can switch to whatever user/group is in the config file after it is
listening on port 80.

If you start the nginx process as not-root, then it will probably fail
to listen on port 80, and fail to open any root-only files, and fail to
switch to whatever user/group is in the config file.

That seems to be what is happening here, based on the error logs shown.

	f
-- 
Francis Daly        francis at daoine.org

From lists at viaduct-productions.com  Fri Jul 14 14:39:03 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Fri, 14 Jul 2017 10:39:03 -0400
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <20170713183151.6040c0ba.lists@lazygranch.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <20170713183151.6040c0ba.lists@lazygranch.com>
Message-ID: <FF059608-F387-4BCF-9A78-6F2644D6B920@viaduct-productions.com>


> On Jul 13, 2017, at 9:31 PM, lists at lazygranch.com wrote:
> 
> However the nginx process is owned by www:
> 823 www           1  20    0 28552K  7060K kqread   0:01   0.00% nginx

Sure the process is owned, and is called upon by nginx as the www user.  The `nginx -t` report is being called by rich, with permissions at 644, so it should be able to be opened and read.  

-rw-r--r--   1 root  wheel        4 Jul 13 17:22 nginx.pid



_____________
Rich in Toronto @ VP







From lists at viaduct-productions.com  Fri Jul 14 14:40:27 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Fri, 14 Jul 2017 10:40:27 -0400
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <20170714132956.GF365@daoine.org>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>
 <20170714132956.GF365@daoine.org>
Message-ID: <6D607FBA-8D63-46DC-8B41-564BD9405670@viaduct-productions.com>

Hi there.

> On Jul 14, 2017, at 9:29 AM, Francis Daly <francis at daoine.org> wrote:
> 
> In unix land, usually, if a process starts running as root, then it is
> able to "switch" to run as another user. If a process starts running as
> non-root, it is not able to switch to run as another user.
> 
> And (usually) only root is able to bind() to port 80.
> 
> (Exceptions exist, but this is probably good enough for now.)

And in FreeBSD?s case, seeing nginx.conf to `user root;` kicks up an error saying this is unacceptable, and it ignores the directive.  

Cheers

_____________
Rich in Toronto @ VP







From jim at mailman-hosting.com  Fri Jul 14 14:58:05 2017
From: jim at mailman-hosting.com (Jim Ohlstein)
Date: Fri, 14 Jul 2017 10:58:05 -0400
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <FF059608-F387-4BCF-9A78-6F2644D6B920@viaduct-productions.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <20170713183151.6040c0ba.lists@lazygranch.com>
 <FF059608-F387-4BCF-9A78-6F2644D6B920@viaduct-productions.com>
Message-ID: <b8c0ea2b-cd87-931d-220f-c500366555ba@mailman-hosting.com>

Hello,

On 07/14/2017 10:39 AM, Viaduct Lists wrote:
> 
>> On Jul 13, 2017, at 9:31 PM, lists at lazygranch.com wrote:
>>
>> However the nginx process is owned by www:
>> 823 www           1  20    0 28552K  7060K kqread   0:01   0.00% nginx
> 
> Sure the process is owned, and is called upon by nginx as the www user.  The `nginx -t` report is being called by rich, with permissions at 644, so it should be able to be opened and read.  

Try calling it using sudo.

> 
> -rw-r--r--   1 root  wheel        4 Jul 13 17:22 nginx.pid
> 
> 

-- 
Jim Ohlstein
Professional Mailman Hosting
https://mailman-hosting.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170714/76353c0e/attachment-0001.bin>

From acg at albertocg.com  Fri Jul 14 15:04:40 2017
From: acg at albertocg.com (Alberto Castillo)
Date: Fri, 14 Jul 2017 10:04:40 -0500
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <b8c0ea2b-cd87-931d-220f-c500366555ba@mailman-hosting.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <20170713183151.6040c0ba.lists@lazygranch.com>
 <FF059608-F387-4BCF-9A78-6F2644D6B920@viaduct-productions.com>
 <b8c0ea2b-cd87-931d-220f-c500366555ba@mailman-hosting.com>
Message-ID: <20170714150440.GA13610@archsociety>

On 07/14, Jim Ohlstein wrote:
> Hello,
> 
> On 07/14/2017 10:39 AM, Viaduct Lists wrote:
> > 
> >> On Jul 13, 2017, at 9:31 PM, lists at lazygranch.com wrote:
> >>
> >> However the nginx process is owned by www:
> >> 823 www           1  20    0 28552K  7060K kqread   0:01   0.00% nginx
> > 
> > Sure the process is owned, and is called upon by nginx as the www user.  The `nginx -t` report is being called by rich, with permissions at 644, so it should be able to be opened and read.  
> 
> Try calling it using sudo.
> 

I've just set up mine on a FreeBSD box and using sudo solves the
problem, same issue with .pid.

-- 
alberto castillo gord?n
software engineering student
http://albertocg.com 
(+507) 6735-7501


From lists at lazygranch.com  Fri Jul 14 15:51:47 2017
From: lists at lazygranch.com (lists at lazygranch.com)
Date: Fri, 14 Jul 2017 08:51:47 -0700
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <20170714150440.GA13610@archsociety>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <20170713183151.6040c0ba.lists@lazygranch.com>
 <FF059608-F387-4BCF-9A78-6F2644D6B920@viaduct-productions.com>
 <b8c0ea2b-cd87-931d-220f-c500366555ba@mailman-hosting.com>
 <20170714150440.GA13610@archsociety>
Message-ID: <20170714155147.5726294.17453.33013@lazygranch.com>

I guess I'm missing something here since nginx should be invoke by "service" such as "service nginx restart".

? Original Message ?
From: Alberto Castillo
Sent: Friday, July 14, 2017 8:05 AM
To: nginx at nginx.org
Reply To: nginx at nginx.org
Subject: Re: FreeBSD Clean Install nginx.pid Permissions Errors

On 07/14, Jim Ohlstein wrote:
> Hello,
> 
> On 07/14/2017 10:39 AM, Viaduct Lists wrote:
> > 
> >> On Jul 13, 2017, at 9:31 PM, lists at lazygranch.com wrote:
> >>
> >> However the nginx process is owned by www:
> >> 823 www 1 20 0 28552K 7060K kqread 0:01 0.00% nginx
> > 
> > Sure the process is owned, and is called upon by nginx as the www user. The `nginx -t` report is being called by rich, with permissions at 644, so it should be able to be opened and read. 
> 
> Try calling it using sudo.
> 

I've just set up mine on a FreeBSD box and using sudo solves the
problem, same issue with .pid.

-- 
alberto castillo gord?n
software engineering student
http://albertocg.com 
(+507) 6735-7501

_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

From acg at albertocg.com  Fri Jul 14 16:48:15 2017
From: acg at albertocg.com (Alberto Castillo)
Date: Fri, 14 Jul 2017 11:48:15 -0500
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <20170714155147.5726294.17453.33013@lazygranch.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <20170713183151.6040c0ba.lists@lazygranch.com>
 <FF059608-F387-4BCF-9A78-6F2644D6B920@viaduct-productions.com>
 <b8c0ea2b-cd87-931d-220f-c500366555ba@mailman-hosting.com>
 <20170714150440.GA13610@archsociety>
 <20170714155147.5726294.17453.33013@lazygranch.com>
Message-ID: <20170714164815.GB13610@archsociety>

On 07/14, lists at lazygranch.com wrote:
> I guess I'm missing something here since nginx should be invoke by "service" such as "service nginx restart".
> 
> ? Original Message ?
> From: Alberto Castillo
> Sent: Friday, July 14, 2017 8:05 AM
> To: nginx at nginx.org
> Reply To: nginx at nginx.org
> Subject: Re: FreeBSD Clean Install nginx.pid Permissions Errors
> 
> I've just set up mine on a FreeBSD box and using sudo solves the
> problem, same issue with .pid.
> 

Yes, I use it as a service but since the thread seems to be trying to
execute it directly, I tried and sudoing is what worked.

-- 
alberto castillo gord?n
software engineering student
http://albertocg.com 
(+507) 6735-7501


From lists at viaduct-productions.com  Fri Jul 14 17:33:39 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Fri, 14 Jul 2017 13:33:39 -0400
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <20170714150440.GA13610@archsociety>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <20170713183151.6040c0ba.lists@lazygranch.com>
 <FF059608-F387-4BCF-9A78-6F2644D6B920@viaduct-productions.com>
 <b8c0ea2b-cd87-931d-220f-c500366555ba@mailman-hosting.com>
 <20170714150440.GA13610@archsociety>
Message-ID: <7DB65417-0635-4C45-A9FC-AE39C7A09968@viaduct-productions.com>

OK, good to know.  Thank you.  This does suggest that security isn?t really respected in this case.  

Cheers



> On Jul 14, 2017, at 11:04 AM, Alberto Castillo <acg at albertocg.com> wrote:
> 
> I've just set up mine on a FreeBSD box and using sudo solves the
> problem, same issue with .pid.


_____________
Rich in Toronto @ VP







From sdnetwork at gmail.com  Fri Jul 14 17:54:16 2017
From: sdnetwork at gmail.com (Arnaud Le-roy)
Date: Fri, 14 Jul 2017 19:54:16 +0200
Subject: Nginx Proxy seems to send twice the same request to the backend
In-Reply-To: <20170713132453.GA55433@mdounin.ru>
References: <4A8C8A80-EF80-4CD3-9C0C-240279B64E06@gmail.com>
 <20170710134935.GQ55433@mdounin.ru>
 <2BA068B8-C448-4EF9-9635-AB2497E0BE51@gmail.com>
 <20170713132453.GA55433@mdounin.ru>
Message-ID: <5B2A5DAC-3ADB-4E19-AE17-917FBEDCBD82@gmail.com>

Hello,

> 
> The error indicate that your backend server closed the connection.  
> That's more less all nginx knows, so if you want to find out why 
> your backend did this - consider looking into the backend.
> 

i found the problem, there is by default a 180s timeout on my backend.


> Could you please double-check that the behaviour you describe 
> appears when using nginx 1.13.2 and with proxy_next_upstream 
> switched off?

effectively, i tested proxy_next_stream off only on nginx/1.10.3 so i will confirm if same behavior on nginx/1.13.2

From nginx-forum at forum.nginx.org  Fri Jul 14 19:57:46 2017
From: nginx-forum at forum.nginx.org (c0nw0nk)
Date: Fri, 14 Jul 2017 15:57:46 -0400
Subject: Nginx allowed characters inside full URL / URI and ARGS
Message-ID: <82d2b762b05477f80bf7db2126be8236.NginxMailingListEnglish@forum.nginx.org>

So I have been using Lua to iron out a few dilemmas and problems lately.

Does anyone know what characters Nginx accepts inside URL's

I am achieving a higher cache HIT ratio by modifying the URL's with Lua but
it also helps in preventing unwanted forms of DoS.

Here is my code :

local function fix_url(str)
    return str:gsub("[+/=]", {["+"] = "+", ["/"] = "/", ["="] = "="})
--Needs some regex to remove multiple occurances of characters
end

ngx.var.modified_url = fix_url(ngx.var.request_uri) --Remove UN-wanted
duplicated characters that users are trying to bypass cache with. 

ngx.var.modified_url = string.lower(ngx.var.modified_url) --make all
lowercase to further increase cache HIT ratio


Now what I need to do is to introduce some regex to my gsub string in order
to remove multiple occurrences of those characters.

Example :

Good Link :
/index.php?page=about

Bad Link :
///InDex.PhP????PaGe===AboUt

You will notice that the good link is very friendly where as the Bad Link
will have the same contents served as the good link but is incredibly
dynamic and bypasses the cache each time. (What is how people are launching
their DoS attacks)

Any help on this little project would be fantastic :)

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275500,275500#msg-275500


From lists at lazygranch.com  Fri Jul 14 20:22:57 2017
From: lists at lazygranch.com (lists at lazygranch.com)
Date: Fri, 14 Jul 2017 13:22:57 -0700
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <20170714164815.GB13610@archsociety>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <20170713183151.6040c0ba.lists@lazygranch.com>
 <FF059608-F387-4BCF-9A78-6F2644D6B920@viaduct-productions.com>
 <b8c0ea2b-cd87-931d-220f-c500366555ba@mailman-hosting.com>
 <20170714150440.GA13610@archsociety>
 <20170714155147.5726294.17453.33013@lazygranch.com>
 <20170714164815.GB13610@archsociety>
Message-ID: <20170714202257.5726294.45474.33037@lazygranch.com>

But in actual use, you would just run nginx ?as a service, so I don't get the sudo initiation. In fact, unless you run a very simple website, nginx alone isn't sufficient, so you would be starting a number of services.

I make enough work for myself, but if security is an issue, I'd suggest setting up a jail. But only
after nginx itself is hardened.?


? Original Message ?
From: Alberto Castillo
Sent: Friday, July 14, 2017 9:48 AM
To: nginx at nginx.org
Reply To: nginx at nginx.org
Subject: Re: FreeBSD Clean Install nginx.pid Permissions Errors

On 07/14, lists at lazygranch.com wrote:
> I guess I'm missing something here since nginx should be invoke by "service" such as "service nginx restart".
> 
> ? Original Message ?
> From: Alberto Castillo
> Sent: Friday, July 14, 2017 8:05 AM
> To: nginx at nginx.org
> Reply To: nginx at nginx.org
> Subject: Re: FreeBSD Clean Install nginx.pid Permissions Errors
> 
> I've just set up mine on a FreeBSD box and using sudo solves the
> problem, same issue with .pid.
> 

Yes, I use it as a service but since the thread seems to be trying to
execute it directly, I tried and sudoing is what worked.

-- 
alberto castillo gord?n
software engineering student
http://albertocg.com 
(+507) 6735-7501

_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

From lists at lazygranch.com  Fri Jul 14 23:54:59 2017
From: lists at lazygranch.com (lists at lazygranch.com)
Date: Fri, 14 Jul 2017 16:54:59 -0700
Subject: Nginx allowed characters inside full URL / URI and ARGS
In-Reply-To: <82d2b762b05477f80bf7db2126be8236.NginxMailingListEnglish@forum.nginx.org>
References: <82d2b762b05477f80bf7db2126be8236.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170714235459.5726294.16261.33056@lazygranch.com>

I took the opposite approach. You put a funny character in the URL, you get a 444. ?I only allow underscore and hypen.

For a while, I was getting fuzzed. Maybe a year ago it was a thing. ?Nothing bad happened, which I would say is a tribute to Nginx. I just returned 404s, but I figured I better trap this behavior before my luck runs out.
?


? Original Message ?
From: c0nw0nk
Sent: Friday, July 14, 2017 12:58 PM
To: nginx at nginx.org
Reply To: nginx at nginx.org
Subject: Nginx allowed characters inside full URL / URI and ARGS

So I have been using Lua to iron out a few dilemmas and problems lately.

Does anyone know what characters Nginx accepts inside URL's

I am achieving a higher cache HIT ratio by modifying the URL's with Lua but
it also helps in preventing unwanted forms of DoS.

Here is my code :

local function fix_url(str)
return str:gsub("[+/=]", {["+"] = "+", ["/"] = "/", ["="] = "="})
--Needs some regex to remove multiple occurances of characters
end

ngx.var.modified_url = fix_url(ngx.var.request_uri) --Remove UN-wanted
duplicated characters that users are trying to bypass cache with. 

ngx.var.modified_url = string.lower(ngx.var.modified_url) --make all
lowercase to further increase cache HIT ratio


Now what I need to do is to introduce some regex to my gsub string in order
to remove multiple occurrences of those characters.

Example :

Good Link :
/index.php?page=about

Bad Link :
///InDex.PhP????PaGe===AboUt

You will notice that the good link is very friendly where as the Bad Link
will have the same contents served as the good link but is incredibly
dynamic and bypasses the cache each time. (What is how people are launching
their DoS attacks)

Any help on this little project would be fantastic :)

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275500,275500#msg-275500

_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

From francis at daoine.org  Sat Jul 15 02:58:00 2017
From: francis at daoine.org (Francis Daly)
Date: Sat, 15 Jul 2017 03:58:00 +0100
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <6D607FBA-8D63-46DC-8B41-564BD9405670@viaduct-productions.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>
 <20170714132956.GF365@daoine.org>
 <6D607FBA-8D63-46DC-8B41-564BD9405670@viaduct-productions.com>
Message-ID: <20170715025800.GG365@daoine.org>

On Fri, Jul 14, 2017 at 10:40:27AM -0400, Viaduct Lists wrote:
> > On Jul 14, 2017, at 9:29 AM, Francis Daly <francis at daoine.org> wrote:

Hi there,

> > In unix land, usually, if a process starts running as root, then it is
> > able to "switch" to run as another user. If a process starts running as
> > non-root, it is not able to switch to run as another user.

> And in FreeBSD?s case, seeing nginx.conf to `user root;` kicks up an error saying this is unacceptable, and it ignores the directive.  

If you are reporting that you invoke "nginx" as the super-user ("root")
and it ignores the "user" directive in nginx.conf, that sounds to me
like a bug that should be raised.

If you are reporting that you invoke "nginx" as a non-root user, and it
warns that the "user" directive in nginx.conf will be ignored, that is
the expected behaviour.

	f
-- 
Francis Daly        francis at daoine.org

From lists at viaduct-productions.com  Sat Jul 15 08:47:41 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Sat, 15 Jul 2017 04:47:41 -0400
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <20170715025800.GG365@daoine.org>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>
 <20170714132956.GF365@daoine.org>
 <6D607FBA-8D63-46DC-8B41-564BD9405670@viaduct-productions.com>
 <20170715025800.GG365@daoine.org>
Message-ID: <293E4024-2F61-4812-81B6-B7FF0611AC66@viaduct-productions.com>

The latter.  It makes little sense.  If it?s ignored then there?s no sense in having it.  

Much like how the current `nginx -t` report makes little sense as well:

nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is ok
nginx: [emerg] open() "/var/run/nginx.pid" failed (13: Permission denied)
nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed

This basically says ?config file is fine.  I can?t read the pid file, even though I?ve been given permission to.  the config file failed."


> On Jul 14, 2017, at 10:58 PM, Francis Daly <francis at daoine.org> wrote:
> 
> If you are reporting that you invoke "nginx" as the super-user ("root")
> and it ignores the "user" directive in nginx.conf, that sounds to me
> like a bug that should be raised.
> 
> If you are reporting that you invoke "nginx" as a non-root user, and it
> warns that the "user" directive in nginx.conf will be ignored, that is
> the expected behaviour.


_____________
Rich in Toronto @ VP







From tkadm30 at yandex.com  Sat Jul 15 08:55:57 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Sat, 15 Jul 2017 04:55:57 -0400
Subject: Duplicated response body with fastcgi
In-Reply-To: <2319d470-0be8-60f4-89dd-2e70a266e60c@yandex.com>
References: <2096f882-1f94-6b98-4de5-446b7d7753fa@yandex.com>
 <2319d470-0be8-60f4-89dd-2e70a266e60c@yandex.com>
Message-ID: <271c4ed0-2035-1e35-26e2-70c6c25007e2@yandex.com>

I fixed this issue. I found that uWSGI is a far better choices than 
FastCGI for serving my Django app with nginx. :)

Cheers,

E


Le 2017-07-14 ? 08:32, Etienne Robillard a ?crit :
> How can one debug the upstream FastCGI response from nginx?
>
>
>
> Le 2017-07-13 ? 10:25, Etienne Robillard a ?crit :
>> Hi,
>> I'm trying to setup a Django app with nginx using fastcgi. Here's my 
>> config:
>>
>> # configuration file /etc/nginx/nginx.conf:
>> user www-data;
>> worker_processes 4;
>> pid /run/nginx.pid;
>>
>> events {
>>     worker_connections 512;
>>     multi_accept on;
>>     use epoll;
>> }
>>
>> http {
>>
>>     ##
>>     # Basic Settings
>>     ##
>>
>>     sendfile on;
>>     tcp_nopush on;
>>     tcp_nodelay on;
>>     keepalive_timeout 80;
>>     types_hash_max_size 2048;
>>     # server_tokens off;
>>
>>     # server_names_hash_bucket_size 64;
>>     # server_name_in_redirect off;
>>
>>     include /etc/nginx/mime.types;
>>     default_type application/octet-stream;
>>
>>     ##
>>     # SSL Settings
>>     ##
>>
>>     ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
>>     ssl_prefer_server_ciphers on;
>>
>>     ##
>>     # Logging Settings
>>     ##
>>
>>     access_log /var/log/nginx/access.log;
>>     error_log /var/log/nginx/error.log;
>>
>>     ##
>>     # Gzip Settings
>>     ##
>>
>>     gzip off;
>>     gzip_disable "msie6";
>>
>>     # gzip_vary on;
>>     # gzip_proxied any;
>>     # gzip_comp_level 6;
>>     # gzip_buffers 16 8k;
>>     # gzip_http_version 1.1;
>>     # gzip_types text/plain text/css application/json 
>> application/javascript text/xml application/xml application/xml+rss 
>> text/javascript;
>>
>>     ##
>>     # Virtual Host Configs
>>     ##
>>
>>     include /etc/nginx/conf.d/development.conf;
>>     #include /etc/nginx/sites-enabled/*;
>> }
>>
>>
>> #mail {
>> #    # See sample authentication script at:
>> #    # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
>> #
>> #    # auth_http localhost/auth.php;
>> #    # pop3_capabilities "TOP" "USER";
>> #    # imap_capabilities "IMAP4rev1" "UIDPLUS";
>> #
>> #    server {
>> #        listen     localhost:110;
>> #        protocol   pop3;
>> #        proxy      on;
>> #    }
>> #
>> #    server {
>> #        listen     localhost:143;
>> #        protocol   imap;
>> #        proxy      on;
>> #    }
>> #}
>>
>> # configuration file /etc/nginx/mime.types:
>>
>> types {
>>     text/html                             html htm shtml;
>>     text/css                              css;
>>     text/xml                              xml;
>>     image/gif                             gif;
>>     image/jpeg                            jpeg jpg;
>>     application/javascript                js;
>>     application/atom+xml                  atom;
>>     application/rss+xml                   rss;
>>
>>     text/mathml                           mml;
>>     text/plain                            txt;
>>     text/vnd.sun.j2me.app-descriptor      jad;
>>     text/vnd.wap.wml                      wml;
>>     text/x-component                      htc;
>>
>>     image/png                             png;
>>     image/tiff                            tif tiff;
>>     image/vnd.wap.wbmp                    wbmp;
>>     image/x-icon                          ico;
>>     image/x-jng                           jng;
>>     image/x-ms-bmp                        bmp;
>>     image/svg+xml                         svg svgz;
>>     image/webp                            webp;
>>
>>     application/font-woff                 woff;
>>     application/java-archive              jar war ear;
>>     application/json                      json;
>>     application/mac-binhex40              hqx;
>>     application/msword                    doc;
>>     application/pdf                       pdf;
>>     application/postscript                ps eps ai;
>>     application/rtf                       rtf;
>>     application/vnd.apple.mpegurl         m3u8;
>>     application/vnd.ms-excel              xls;
>>     application/vnd.ms-fontobject         eot;
>>     application/vnd.ms-powerpoint         ppt;
>>     application/vnd.wap.wmlc              wmlc;
>>     application/vnd.google-earth.kml+xml  kml;
>>     application/vnd.google-earth.kmz      kmz;
>>     application/x-7z-compressed           7z;
>>     application/x-cocoa                   cco;
>>     application/x-java-archive-diff       jardiff;
>>     application/x-java-jnlp-file          jnlp;
>>     application/x-makeself                run;
>>     application/x-perl                    pl pm;
>>     application/x-pilot                   prc pdb;
>>     application/x-rar-compressed          rar;
>>     application/x-redhat-package-manager  rpm;
>>     application/x-sea                     sea;
>>     application/x-shockwave-flash         swf;
>>     application/x-stuffit                 sit;
>>     application/x-tcl                     tcl tk;
>>     application/x-x509-ca-cert            der pem crt;
>>     application/x-xpinstall               xpi;
>>     application/xhtml+xml                 xhtml;
>>     application/xspf+xml                  xspf;
>>     application/zip                       zip;
>>
>>     application/octet-stream              bin exe dll;
>>     application/octet-stream              deb;
>>     application/octet-stream              dmg;
>>     application/octet-stream              iso img;
>>     application/octet-stream              msi msp msm;
>>
>> application/vnd.openxmlformats-officedocument.wordprocessingml.document 
>> docx;
>> application/vnd.openxmlformats-officedocument.spreadsheetml.sheet xlsx;
>> application/vnd.openxmlformats-officedocument.presentationml.presentation 
>> pptx;
>>
>>     audio/midi                            mid midi kar;
>>     audio/mpeg                            mp3;
>>     audio/ogg                             ogg;
>>     audio/x-m4a                           m4a;
>>     audio/x-realaudio                     ra;
>>
>>     video/3gpp                            3gpp 3gp;
>>     video/mp2t                            ts;
>>     video/mp4                             mp4;
>>     video/mpeg                            mpeg mpg;
>>     video/quicktime                       mov;
>>     video/webm                            webm;
>>     video/x-flv                           flv;
>>     video/x-m4v                           m4v;
>>     video/x-mng                           mng;
>>     video/x-ms-asf                        asx asf;
>>     video/x-ms-wmv                        wmv;
>>     video/x-msvideo                       avi;
>> }
>>
>> # configuration file /etc/nginx/conf.d/development.conf:
>> server {
>>
>>     # static medias web server configuration, for development
>>     # and testing purposes.
>>
>>     listen       80;
>>     server_name  localhost;
>>     error_log /var/log/nginx/error_log; #debug
>>     #access_log  /var/log/nginx/gthc.org/access.log;
>>     root /home/erob/www/isotopesoftware.ca;
>>     #autoindex on;
>>
>>     location / {
>>     #    # host and port to fastcgi server
>>         fastcgi_pass 127.0.0.1:8808; # 8808=gthc.org; 8801=tm
>>         include fastcgi_params;
>>         autoindex on;
>>     #    # rewrite /CamelCase to /wiki/CamelCase
>>     #    rewrite ^/(.*[A-Z][a-z]*)$ /wiki$1 last;
>>         #etag on;
>>     #fastcgi_pass_header $http_if_none_match;
>>     }
>>
>>
>>     # debug url rewriting to the error log
>>     rewrite_log on;
>>
>>     location /media {
>>         autoindex on;
>>         gzip on;
>>     }
>>
>>     location /pub {
>>         autoindex on;
>>         gzip on;
>>     }
>>
>>     location /webalizer {
>>         autoindex on;
>>         gzip on;
>>     #auth_basic "Private Property";
>>     #auth_basic_user_file /etc/nginx/.htpasswd;
>>         allow 67.68.76.70;
>>     deny all;
>>     }
>>
>>     location /documentation {
>>         autoindex on;
>>         gzip on;
>>     }
>>
>>     location /moin_static184 {
>>     autoindex on;
>>     gzip on;
>>     }
>>     location /favicon.ico {
>>     empty_gif;
>>     }
>>     location /robots.txt {
>>          root /home/www/isotopesoftware.ca;
>>     }
>>     location /sitemap.xml {
>>     root /home/www/isotopesoftware.ca;
>>     }
>>
>>     #location /public_html {
>>     # root /home/www/;
>>     # autoindex on;
>>     #}
>>     # redirect server error pages to the static page /50x.html
>>     #error_page 404 /404.html;
>>     #error_page 403    /403.html;
>>     #error_page 500 502 503 504  /50x.html;
>>     #location = /50x.html {
>>     #    root   /var/www/nginx-default;
>>     #}
>>
>>     include conf.d/moinmoin.conf;
>>     include conf.d/livestore.conf;
>> }
>>
>>
>> # configuration file /etc/nginx/fastcgi_params:
>> fastcgi_param  PATH_INFO          $fastcgi_script_name;
>> fastcgi_param  QUERY_STRING       $query_string;
>> fastcgi_param  REQUEST_METHOD     $request_method;
>> fastcgi_param  CONTENT_TYPE       $content_type;
>> fastcgi_param  CONTENT_LENGTH     $content_length;
>>
>> fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
>> fastcgi_param  REQUEST_URI        $request_uri;
>> fastcgi_param  DOCUMENT_URI       $document_uri;
>> fastcgi_param  DOCUMENT_ROOT      $document_root;
>> fastcgi_param  SERVER_PROTOCOL    $server_protocol;
>>
>> fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
>> fastcgi_param  SERVER_SOFTWARE    nginx;
>>
>> fastcgi_param  REMOTE_ADDR        $remote_addr;
>> fastcgi_param  REMOTE_PORT        $remote_port;
>> #fastcgi_param  REMOTE_USER      $remote_user;
>> fastcgi_param  SERVER_ADDR        $server_addr;
>> fastcgi_param  SERVER_PORT        $server_port;
>> fastcgi_param  SERVER_NAME        $server_name;
>>
>>
>> #XXX
>> #fastcgi_param HTTP_IF_NONE_MATCH $http_if_none_match;
>> #fastcgi_param HTTP_IF_MODIFIED_SINCE $http_if_modified_since;
>>
>>
>> # PHP only, required if PHP was built with --enable-force-cgi-redirect
>> # fastcgi_param  REDIRECT_STATUS    200;
>>
>> fastcgi_send_timeout 90;
>> fastcgi_read_timeout 90;
>> fastcgi_connect_timeout 40;
>> fastcgi_cache_valid 200 304 10m;
>> #fastcgi_buffer_size 128k;
>> #fastcgi_buffers 8 128k;
>> #fastcgi_busy_buffers_size 256k;
>> #fastcgi_temp_file_write_size 256k;
>>
>>
>> # configuration file /etc/nginx/conf.d/moinmoin.conf:
>>
>>
>> location /wiki {
>>
>>
>>       if ($uri ~ ^/wiki(.*)?){
>>          set $wiki_url $1;
>>       }
>>       # host and port to fastcgi server
>>       fastcgi_pass 127.0.0.1:8807; # 8808=gthc.org; 8801=tm; 
>> 8807=moinmoin
>>       #include fastcgi_params;
>>       fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
>>       fastcgi_param  SERVER_SOFTWARE    nginx;
>>       fastcgi_param  QUERY_STRING       $query_string;
>>       fastcgi_param  REQUEST_METHOD     $request_method;
>>       fastcgi_param  CONTENT_TYPE       $content_type;
>>       fastcgi_param  CONTENT_LENGTH     $content_length;
>>       fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
>>       #fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
>>       fastcgi_param  REQUEST_URI        $request_uri;
>>       fastcgi_param  DOCUMENT_URI       $document_uri;
>>       fastcgi_param  DOCUMENT_ROOT      $document_root;
>>       fastcgi_param  SERVER_PROTOCOL    $server_protocol;
>>       fastcgi_param  REMOTE_ADDR        $remote_addr;
>>       fastcgi_param  REMOTE_PORT        $remote_port;
>>       fastcgi_param  SERVER_ADDR        $server_addr;
>>       fastcgi_param  SERVER_PORT        $server_port;
>>       fastcgi_param  SERVER_NAME        $server_name;
>>       fastcgi_param PATH_INFO $wiki_url;
>>       fastcgi_param SCRIPT_NAME /wiki;
>> }
>> #location /moin_static184 {
>> #    root /home/www/isotopesoftware.ca;
>> #    autoindex on;
>> #}
>>
>>
>>
>> # configuration file /etc/nginx/conf.d/livestore.conf:
>>
>>
>> location /store {
>>
>>
>>       if ($uri ~ ^/store(.*)?){
>>          set $store_url $1;
>>       }
>>       # host and port to fastcgi server
>>       fastcgi_pass 127.0.0.1:8809; # 8808=gthc.org; 8801=tm; 
>> 8807=moinmoin
>>       #include fastcgi_params;
>>       fastcgi_param  GATEWAY_INTERFACE  CGI/1.1;
>>       fastcgi_param  SERVER_SOFTWARE    nginx;
>>       fastcgi_param  QUERY_STRING       $query_string;
>>       fastcgi_param  REQUEST_METHOD     $request_method;
>>       fastcgi_param  CONTENT_TYPE       $content_type;
>>       fastcgi_param  CONTENT_LENGTH     $content_length;
>>       fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;
>>       #fastcgi_param  SCRIPT_NAME        $fastcgi_script_name;
>>       fastcgi_param  REQUEST_URI        $request_uri;
>>       fastcgi_param  DOCUMENT_URI       $document_uri;
>>       fastcgi_param  DOCUMENT_ROOT      $document_root;
>>       fastcgi_param  SERVER_PROTOCOL    $server_protocol;
>>       fastcgi_param  REMOTE_ADDR        $remote_addr;
>>       fastcgi_param  REMOTE_PORT        $remote_port;
>>       fastcgi_param  SERVER_ADDR        $server_addr;
>>       fastcgi_param  SERVER_PORT        $server_port;
>>       fastcgi_param  SERVER_NAME        $server_name;
>>       fastcgi_param PATH_INFO $store_url;
>>       fastcgi_param SCRIPT_NAME /store;
>> }
>> location /satchmo_media {
>>     root /home/www/livestore;
>>     autoindex on;
>>     gzip on;
>>     expires 30d;
>> }
>>
>> The problem is that nginx is duplicating the response body stream for 
>> the /store location. This issue does not happen in WSGI mode.
>>
>> Any ideas how to fix?
>>
>> Thank you in advance,
>>
>> E
>>
>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From me at nanaya.pro  Sat Jul 15 09:04:37 2017
From: me at nanaya.pro (nanaya)
Date: Sat, 15 Jul 2017 18:04:37 +0900
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <293E4024-2F61-4812-81B6-B7FF0611AC66@viaduct-productions.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>
 <20170714132956.GF365@daoine.org>
 <6D607FBA-8D63-46DC-8B41-564BD9405670@viaduct-productions.com>
 <20170715025800.GG365@daoine.org>
 <293E4024-2F61-4812-81B6-B7FF0611AC66@viaduct-productions.com>
Message-ID: <1500109477.3249973.1041673624.238C9BAD@webmail.messagingengine.com>

Hi,

On Sat, Jul 15, 2017, at 17:47, Viaduct Lists wrote:
> The latter.  It makes little sense.  If it?s ignored then there?s no
> sense in having it.  
> 

It works if you start it from user with root privilege. Otherwise you
can't switch user and thus the directive is ignored.

> Much like how the current `nginx -t` report makes little sense as well:
> 
> nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is
> ok
> nginx: [emerg] open() "/var/run/nginx.pid" failed (13: Permission denied)
> nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed
> 
> This basically says ?config file is fine.  I can?t read the pid file,
> even though I?ve been given permission to.  the config file failed."
> 

open() is more than just read. nginx needs to write to it as well and it
can't do it because your user doesn't have permission to. And thus using
the specified config will fail.

The config is syntactically okay but not actually usable.

From nginx-forum at forum.nginx.org  Sat Jul 15 09:56:21 2017
From: nginx-forum at forum.nginx.org (c0nw0nk)
Date: Sat, 15 Jul 2017 05:56:21 -0400
Subject: Nginx allowed characters inside full URL / URI and ARGS
In-Reply-To: <20170714235459.5726294.16261.33056@lazygranch.com>
References: <20170714235459.5726294.16261.33056@lazygranch.com>
Message-ID: <81f2d326b3435aadea4ee8544dd4f241.NginxMailingListEnglish@forum.nginx.org>

Yes but characters in args like = & and ? are allowed and its when they
insert more than one occurance of them nginx accepts them and they bypass
any caches that you have.

&argument=value | Cache : HIT

&&&arguement===value | Cache : MISS


And when they want to DoS you they will do something like the following.


?random=1
?random=2
?random=3
etc etc

It is easy to bypass the cache when your not suppose to.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275500,275509#msg-275509


From lists at viaduct-productions.com  Sat Jul 15 09:56:41 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Sat, 15 Jul 2017 05:56:41 -0400
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <1500109477.3249973.1041673624.238C9BAD@webmail.messagingengine.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>
 <20170714132956.GF365@daoine.org>
 <6D607FBA-8D63-46DC-8B41-564BD9405670@viaduct-productions.com>
 <20170715025800.GG365@daoine.org>
 <293E4024-2F61-4812-81B6-B7FF0611AC66@viaduct-productions.com>
 <1500109477.3249973.1041673624.238C9BAD@webmail.messagingengine.com>
Message-ID: <8C66E6A4-3ECC-484B-82DE-3A5F8D0CCE88@viaduct-productions.com>


> On Jul 15, 2017, at 5:04 AM, nanaya <me at nanaya.pro> wrote:
> 
> 
> It works if you start it from user with root privilege. Otherwise you
> can't switch user and thus the directive is ignored.

If I deliberately start up using root, why would I need a directive that indicates that?  This directive seems like a reminder after the fact.  

>> Much like how the current `nginx -t` report makes little sense as well:
>> 
>> nginx: the configuration file /usr/local/etc/nginx/nginx.conf syntax is
>> ok
>> nginx: [emerg] open() "/var/run/nginx.pid" failed (13: Permission denied)
>> nginx: configuration file /usr/local/etc/nginx/nginx.conf test failed
>> 
>> This basically says ?config file is fine.  I can?t read the pid file,
>> even though I?ve been given permission to.  the config file failed."
>> 
> 
> open() is more than just read. nginx needs to write to it as well and it
> can't do it because your user doesn't have permission to. And thus using
> the specified config will fail.

In my case, all servers reporting this are working and serving as expected.  So the failure and permissions errors are pretty much useless reporting.  

> The config is syntactically okay but not actually usable.

Aha, just what I was expecting.  

Cheers
_____________
Rich in Toronto @ VP







From nginx-forum at forum.nginx.org  Sat Jul 15 10:12:32 2017
From: nginx-forum at forum.nginx.org (itpp2012)
Date: Sat, 15 Jul 2017 06:12:32 -0400
Subject: Nginx allowed characters inside full URL / URI and ARGS
In-Reply-To: <81f2d326b3435aadea4ee8544dd4f241.NginxMailingListEnglish@forum.nginx.org>
References: <20170714235459.5726294.16261.33056@lazygranch.com>
 <81f2d326b3435aadea4ee8544dd4f241.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <982c5cd9cad45f142ddebc7ce134f088.NginxMailingListEnglish@forum.nginx.org>

Use my simple map/waf example (nginx-simple-WAF.conf) and some cleaver
regex's.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275500,275511#msg-275511


From me at nanaya.pro  Sat Jul 15 10:24:36 2017
From: me at nanaya.pro (nanaya)
Date: Sat, 15 Jul 2017 19:24:36 +0900
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <8C66E6A4-3ECC-484B-82DE-3A5F8D0CCE88@viaduct-productions.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>
 <20170714132956.GF365@daoine.org>
 <6D607FBA-8D63-46DC-8B41-564BD9405670@viaduct-productions.com>
 <20170715025800.GG365@daoine.org>
 <293E4024-2F61-4812-81B6-B7FF0611AC66@viaduct-productions.com>
 <1500109477.3249973.1041673624.238C9BAD@webmail.messagingengine.com>
 <8C66E6A4-3ECC-484B-82DE-3A5F8D0CCE88@viaduct-productions.com>
Message-ID: <1500114276.3263698.1041707544.07DDC9BF@webmail.messagingengine.com>



On Sat, Jul 15, 2017, at 18:56, Viaduct Lists wrote:
> 
> > On Jul 15, 2017, at 5:04 AM, nanaya <me at nanaya.pro> wrote:
> > 
> > 
> > It works if you start it from user with root privilege. Otherwise you
> > can't switch user and thus the directive is ignored.
> 
> If I deliberately start up using root, why would I need a directive that
> indicates that?  This directive seems like a reminder after the fact.  
> 

root is usually needed to bind port 80 and 443 so usually people want to
start it using root. But apart of the binding, having everything running
as root is dangerous especially for something that's public facing so
usually root is only used for stuff which requires it and then the
worker processes, the processes which actually handle requests, are run
as different user. The `user` directive is used to tell which user the
processes should be run as.

The directive defaults to `user nobody nobody` so `user root` means you
explicitly want the workers to run as root instead of nobody (assuming
it works).

> 
> In my case, all servers reporting this are working and serving as
> expected.  So the failure and permissions errors are pretty much useless
> reporting.  
> 

While it runs, without working pid file, you'll need alternative way of
maintaining the process (upgrade, config reload, log rotate, shutdown,
etc). Especially `-s` switch of `nginx` can't be used to control running
process because it doesn't have valid pidfile.

From kworthington at gmail.com  Sat Jul 15 13:30:32 2017
From: kworthington at gmail.com (Kevin Worthington)
Date: Sat, 15 Jul 2017 09:30:32 -0400
Subject: [nginx-announce] nginx-1.12.1
In-Reply-To: <20170711154648.GC55433@mdounin.ru>
References: <20170711154648.GC55433@mdounin.ru>
Message-ID: <CAGo79UWnWEZ+JWm6aYCkO7NO7Mma5VfjkjNa+8=MJVBwQ8WQSg@mail.gmail.com>

Hello Nginx users,

(I forgot to send this out the other day...)
Now available: Nginx 1.12.1 for Windows
https://kevinworthington.com/nginxwin1121
(32-bit and 64-bit versions)

These versions are to support legacy users who are already using Cygwin
based builds of Nginx. Officially supported native Windows binaries are at
nginx.org.

Announcements are also available here:
Twitter http://twitter.com/kworthington
Google+ https://plus.google.com/+KevinWorthington/

Thank you,
Kevin
--
Kevin Worthington
kworthington *@* (gmail]  [dot} {com)
https://kevinworthington.com/
https://twitter.com/kworthington
https://plus.google.com/+KevinWorthington/

On Tue, Jul 11, 2017 at 11:46 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Changes with nginx 1.12.1                                        11 Jul
> 2017
>
>     *) Security: a specially crafted request might result in an integer
>        overflow and incorrect processing of ranges in the range filter,
>        potentially resulting in sensitive information leak (CVE-2017-7529).
>
>
> --
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx-announce mailing list
> nginx-announce at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-announce
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170715/dd4a9f76/attachment.html>

From lists at viaduct-productions.com  Sat Jul 15 15:45:29 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Sat, 15 Jul 2017 11:45:29 -0400
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <1500114276.3263698.1041707544.07DDC9BF@webmail.messagingengine.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>
 <20170714132956.GF365@daoine.org>
 <6D607FBA-8D63-46DC-8B41-564BD9405670@viaduct-productions.com>
 <20170715025800.GG365@daoine.org>
 <293E4024-2F61-4812-81B6-B7FF0611AC66@viaduct-productions.com>
 <1500109477.3249973.1041673624.238C9BAD@webmail.messagingengine.com>
 <8C66E6A4-3ECC-484B-82DE-3A5F8D0CCE88@viaduct-productions.com>
 <1500114276.3263698.1041707544.07DDC9BF@webmail.messagingengine.com>
Message-ID: <24997719-04F2-47D3-A232-6AF6DCAE40A8@viaduct-productions.com>


> On Jul 15, 2017, at 6:24 AM, nanaya <me at nanaya.pro> wrote:
> 
>> If I deliberately start up using root, why would I need a directive that
>> indicates that?  This directive seems like a reminder after the fact.  
>> 
> 
> root is usually needed to bind port 80 and 443 so usually people want to
> start it using root. But apart of the binding, having everything running
> as root is dangerous especially for something that's public facing so
> usually root is only used for stuff which requires it and then the
> worker processes, the processes which actually handle requests, are run
> as different user. The `user` directive is used to tell which user the
> processes should be run as.
> 
> The directive defaults to `user nobody nobody` so `user root` means you
> explicitly want the workers to run as root instead of nobody (assuming
> it works).

My point was that you can start the service as root, or set the user to root in nginx.conf.  It?s confusing.  Two ways.  If I?m deliberately starting the service as root, why would I need to set the config file to indicate so?

Second, setting the nginx.conf directive to user root, whilst using the default www user as startup, only coughs up an error indicating the directive was ignored.  

A lot of this makes little sense.  

>> 
>> In my case, all servers reporting this are working and serving as
>> expected.  So the failure and permissions errors are pretty much useless
>> reporting.  
>> 
> 
> While it runs, without working pid file, you'll need alternative way of
> maintaining the process (upgrade, config reload, log rotate, shutdown,
> etc). Especially `-s` switch of `nginx` can't be used to control running
> process because it doesn't have valid pidfile.

So the only way around this as I see it is to start up as root, because I?ve tried absolutely everything, and nothing is getting rid of this nginx.pid permissions error.  

OK then.  

_____________
Rich in Toronto @ VP







From me at nanaya.pro  Sat Jul 15 16:08:09 2017
From: me at nanaya.pro (nanaya)
Date: Sun, 16 Jul 2017 01:08:09 +0900
Subject: FreeBSD Clean Install nginx.pid Permissions Errors
In-Reply-To: <24997719-04F2-47D3-A232-6AF6DCAE40A8@viaduct-productions.com>
References: <E3EEFED2-E493-4E65-A626-69EF1B825A77@viaduct-productions.com>
 <20170713224612.GE365@daoine.org>
 <011BC9F1-8C22-44D3-B2A5-D267D4BDA06B@viaduct-productions.com>
 <20170714132956.GF365@daoine.org>
 <6D607FBA-8D63-46DC-8B41-564BD9405670@viaduct-productions.com>
 <20170715025800.GG365@daoine.org>
 <293E4024-2F61-4812-81B6-B7FF0611AC66@viaduct-productions.com>
 <1500109477.3249973.1041673624.238C9BAD@webmail.messagingengine.com>
 <8C66E6A4-3ECC-484B-82DE-3A5F8D0CCE88@viaduct-productions.com>
 <1500114276.3263698.1041707544.07DDC9BF@webmail.messagingengine.com>
 <24997719-04F2-47D3-A232-6AF6DCAE40A8@viaduct-productions.com>
Message-ID: <1500134889.3327654.1041874632.34BC56AD@webmail.messagingengine.com>

Hi,

On Sun, Jul 16, 2017, at 00:45, Viaduct Lists wrote:
> 
> My point was that you can start the service as root, or set the user to
> root in nginx.conf.  It?s confusing.  Two ways.  If I?m deliberately
> starting the service as root, why would I need to set the config file to
> indicate so?
> 
> Second, setting the nginx.conf directive to user root, whilst using the
> default www user as startup, only coughs up an error indicating the
> directive was ignored.  
> 
> A lot of this makes little sense.  
> 

Only root can create process of different user id so the directive only
works for root.

It doesn't work for non-root users. nginx will ignore it if it exists
because it will not work. It's operating system level security, not
nginx. You don't want non-root user to be able to create process under
different user id, do you?

The reason the option exist is, in many operating systems, only root
user can bind to port <1024 (including port 80 and 443 used for
http/https) so nginx must be started by root to be able to listen to
those ports. 

But as previously mentioned, doing everything else as root is considered
security risk so nginx provides `user` directive to limit codes run by
root and thus reduces security risk.

1. root starts nginx
2. nginx parses config, creates master process, binds to port 80, etc as
root
3. nginx creates worker processes under different user id according to
user directive

#3 is only possible if nginx is run as root. Otherwise nginx will just
create the processes as the same user running the master process.

> 
> So the only way around this as I see it is to start up as root, because
> I?ve tried absolutely everything, and nothing is getting rid of this
> nginx.pid permissions error.  
> 
> OK then.  
> 

You can either chown the nginx.pid to your user, make it writable by
your user (chgrp + chown) or point it to other directory your user own
(there is `pid` directive [1] to set it).

[1] https://nginx.org/r/pid

From nginx-forum at forum.nginx.org  Sun Jul 16 10:42:14 2017
From: nginx-forum at forum.nginx.org (RainFlying)
Date: Sun, 16 Jul 2017 06:42:14 -0400
Subject: Got denied even the ip was allowed in the config file
Message-ID: <3d04c2727e188122d2df8fb4545a3c21.NginxMailingListEnglish@forum.nginx.org>

This is part of the virtual server config


server {
    listen 80;
    server_name .kosungames.com;

    include backend_acl.conf;

    access_log /var/log/nginx/access-admin.log main;
    error_log /var/log/nginx/error-admin.log debug;
    client_max_body_size 16m;

    root /srv/www/site;
    index admin_index.php;



The following is the content of the  backend_acl.conf:

allow 211.24.114.202/32;
allow 202.131.84.134/32;
allow 192.228.138.205/32;
allow 2001:f40:902::/48;

allow 113.10.213.250;
allow 113.10.213.252;
allow 113.10.213.253;
allow 202.131.80.74;
allow 47.52.16.173;
allow 13.112.207.63;
deny all;



However, I got 403 when I accessed the site with one of the whitelist IP.
This is part of the Nginx error log


2017/07/16 18:41:05 [error] 12579#12579: *17185 access forbidden by rule,
client: 13.112.207.63, server: kosungames.com, request: "HEAD / HTTP/1.1",
host: "cp916.kosungames.com"


What should be the cause of this issue ?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275518,275518#msg-275518


From nginx-forum at forum.nginx.org  Sun Jul 16 11:04:17 2017
From: nginx-forum at forum.nginx.org (RainFlying)
Date: Sun, 16 Jul 2017 07:04:17 -0400
Subject: Got denied even the ip was allowed in the config file
In-Reply-To: <3d04c2727e188122d2df8fb4545a3c21.NginxMailingListEnglish@forum.nginx.org>
References: <3d04c2727e188122d2df8fb4545a3c21.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <3ea18ed78b51b23710056376f8678e39.NginxMailingListEnglish@forum.nginx.org>

OK. Got it.
I'm using OpenResty, it reads different config files.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275518,275519#msg-275519


From leeon2013 at gmail.com  Mon Jul 17 04:07:22 2017
From: leeon2013 at gmail.com (David Woodstuck)
Date: Mon, 17 Jul 2017 00:07:22 -0400
Subject: How to solid three urls into one url for its users
Message-ID: <CAD6THg5KKwRqcqxztPsnYXu7ywqd4+8jvZB6EdLyRa7+7fjdqg@mail.gmail.com>

I just use Nginx for a couple of weeks. I have the following questions. I
hope I can get your help.

I have a big web application whose urls are dev01.mydomain.com,
dev02.mydomain and dev03.mydomain.com. I like to use Nginx to deliver this
application for its users. On each page, there are a lot of iframes whose
src can be one of dev01.mydomain.com, dev02.mydomain and dev03.mydomain.com.
I like to use following conf to achieve one url for its users:

server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;
        #access_log  /var/log/nginx/log/host.access.log  main;

        location / {
            root   /usr/share/nginx/html;
            index  index.html index.htm;

            proxy_pass http://dev01.mydomain.com/;
            subs_filter 'http://dev02.mydomain.com/(.*)' '
http://dev01.mydomain.com/?http://dev02.mydomain.com/$1';
            subs_filter 'http://dev03.mydomain.com/(.*)' '
http://dev01.mydomain.com/?http://dev03.mydomain.com/$1';
         }
}

My requirements are:
 (1). Request with url - http://www.mydomain.com/?http://... will proxy to
http://dev02.mydomain.com/ or http://dev03.mydomain.com/ depending on a
value after ?
 (2). Other request will proxy to http://dev01.mydomain.com/.

How do I achieve these requirements?

Thanks,
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170717/81c3f5f6/attachment.html>

From nginx-forum at forum.nginx.org  Mon Jul 17 06:06:27 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Mon, 17 Jul 2017 02:06:27 -0400
Subject: Buffering issues with nginx
Message-ID: <95440158754e80efaf521fdad433f093.NginxMailingListEnglish@forum.nginx.org>

No matter what configs I try, nginx still keeps buffering my requests.

These are the configs that I apply in my test:
```
        proxy_pass http://localhost:80;
        proxy_request_buffering off;
        proxy_buffer_size 4k;
        proxy_buffers 8 4k;
        proxy_no_cache 1;
        proxy_set_header Host $host;
        proxy_max_temp_file_size 0;```

in my case I run nodejs app directly on port 80 and I run nginx on 8080,
then I run my tests against proxied and direct version.

My nodejs app needs to know exact amount of data that was sent to remote (to
calculate speed of transfer and for billing purposes). Ideally, I'd like to
know number of ACKed bytes of a TCP connection. Node itself cannot even
provide that (as this wasn't even possible until recently on windows).

In my first test I serve 25MB test binary data. Without
`proxy_max_temp_file_size 0` nginx would read all 25MB in 5ms and then would
continue sending that data to the originator on its own without me (e.g. my
node app) ever knowing if all 25MB were delivered or transfer was aborted
after 1MB.
After I applied all these configs above this started to work better, and
since I proxy in http1.0 mode I can use connection.close event on my node
app as a rough approximation when connection was completed.

However, when I send for example 500KB of binary data, nginx still reads in
all the data in couple of milliseconds and closes connection and there is no
way for me to know on node side if that data was actually properly
delivered.
For the test I download that data using wget and I use --limit-rate=10000 to
limit download on receiver side to 10KB/s. After 5 seconds I ctrl+C and
abort transfer after loading just 50KB, while my nodejs side actually thinks
that everything went well and all 500KB were loaded.


1) how can I make nginx not to buffer more than 64KB of data?
2) can my node app know how much data nginx ended up delivering?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275526#msg-275526


From lists at lazygranch.com  Mon Jul 17 07:09:58 2017
From: lists at lazygranch.com (lists at lazygranch.com)
Date: Mon, 17 Jul 2017 00:09:58 -0700
Subject: Response code 400 rather than 404
Message-ID: <20170717000958.278c7320.lists@lazygranch.com>

I'm curious why this request got a 400 response rather than a 404. 

400 123.160.235.162 - - [16/Jul/2017:22:56:30 +0000] "GET /currentsetting.htm HTTP/1.1" 173 "-" "-" "-"

log_format  main  '$status $remote_addr - $remote_user [$time_local] "$request" '
                      '$body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

From francis at daoine.org  Mon Jul 17 09:14:33 2017
From: francis at daoine.org (Francis Daly)
Date: Mon, 17 Jul 2017 10:14:33 +0100
Subject: Buffering issues with nginx
In-Reply-To: <95440158754e80efaf521fdad433f093.NginxMailingListEnglish@forum.nginx.org>
References: <95440158754e80efaf521fdad433f093.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170717091433.GH365@daoine.org>

On Mon, Jul 17, 2017 at 02:06:27AM -0400, Dan34 wrote:

Hi there,

> No matter what configs I try, nginx still keeps buffering my requests.

proxy_request_buffering (http://nginx.org/r/proxy_request_buffering)
relates to nginx buffering the request from the client.

proxy_buffering (http://nginx.org/r/proxy_buffering) relates to nginx
buffering the response from the upstream server.

Are you concerned about the request or the response being buffered?

	f
-- 
Francis Daly        francis at daoine.org

From mdounin at mdounin.ru  Mon Jul 17 11:36:04 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Mon, 17 Jul 2017 14:36:04 +0300
Subject: Response code 400 rather than 404
In-Reply-To: <20170717000958.278c7320.lists@lazygranch.com>
References: <20170717000958.278c7320.lists@lazygranch.com>
Message-ID: <20170717113604.GC93611@mdounin.ru>

Hello!

On Mon, Jul 17, 2017 at 12:09:58AM -0700, lists at lazygranch.com wrote:

> I'm curious why this request got a 400 response rather than a 404. 
> 
> 400 123.160.235.162 - - [16/Jul/2017:22:56:30 +0000] "GET /currentsetting.htm HTTP/1.1" 173 "-" "-" "-"
> 
> log_format  main  '$status $remote_addr - $remote_user [$time_local] "$request" '
>                       '$body_bytes_sent "$http_referer" '
>                       '"$http_user_agent" "$http_x_forwarded_for"';

Your error_log should have the details once configured at the 
"info" level.

-- 
Maxim Dounin
http://nginx.org/

From tkadm30 at yandex.com  Mon Jul 17 15:29:00 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Mon, 17 Jul 2017 11:29:00 -0400
Subject: Problem setting up ssl
Message-ID: <6cbfcc7b-aa15-8798-c39e-f0b64f5c9d70@yandex.com>

Hi,

I'm trying to setup ssl on my server and "nginx -t" reports no problem 
but the server is now listening on port 256 instead of 443. Any idea 
what is wrong ?


Thanks,


Etienne


From tkadm30 at yandex.com  Mon Jul 17 15:32:59 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Mon, 17 Jul 2017 11:32:59 -0400
Subject: Problem setting up ssl
In-Reply-To: <6cbfcc7b-aa15-8798-c39e-f0b64f5c9d70@yandex.com>
References: <6cbfcc7b-aa15-8798-c39e-f0b64f5c9d70@yandex.com>
Message-ID: <dd006140-c1f0-afa5-4368-80d7673d8ea0@yandex.com>

Sorry for the noise - apparently the certicate is not yet fully working.

E


Le 2017-07-17 ? 11:29, Etienne Robillard a ?crit :
> Hi,
>
> I'm trying to setup ssl on my server and "nginx -t" reports no problem 
> but the server is now listening on port 256 instead of 443. Any idea 
> what is wrong ?
>
>
> Thanks,
>
>
> Etienne
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From nginx-forum at forum.nginx.org  Mon Jul 17 21:26:40 2017
From: nginx-forum at forum.nginx.org (DonaldCock)
Date: Mon, 17 Jul 2017 17:26:40 -0400
Subject: try_files and gzip_static
Message-ID: <bf4395d145f78bdff3a928684d61e0de.NginxMailingListEnglish@forum.nginx.org>

>From my knowledge, try_files works with gzip_static if both compressed and
uncompressed files exist, but with "gunzip on;", it doesn't seem like
try_files becomes aware of this, and so it'll only serve either the
compressed or uncompressed file.

Any idea how to get around this? I can't serve the files directly as there's
no file extension in the uri, and it's rewritten.

My config for that block:

location ~ ^/view/c1/(\w+)/(\d+)(/|\.html)?$ {
????gzip_static on;
????gunzip on;

????add_header Cache-Control "public, must-revalidate";
????try_files /static/b_$1/$2.html /view.php?b=$1&t=$2;
}

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275539,275539#msg-275539


From tkadm30 at yandex.com  Mon Jul 17 23:05:31 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Mon, 17 Jul 2017 19:05:31 -0400
Subject: Problem setting up ssl
In-Reply-To: <dd006140-c1f0-afa5-4368-80d7673d8ea0@yandex.com>
References: <6cbfcc7b-aa15-8798-c39e-f0b64f5c9d70@yandex.com>
 <dd006140-c1f0-afa5-4368-80d7673d8ea0@yandex.com>
Message-ID: <55f0b5bf-abe8-45ca-2355-a6e3e31f5dec@yandex.com>

It seems I have a problem with the SSL certificate chain, as per 
whatsmychaincert.com. However my SSL certificate provider (Gandi) says 
that the SSL certificate still needs to be attributed... I don't know 
what to do at this point but to wait for a message from Gandi. 
Whatsmychaincert.com is telling me to update my certificate (crt) file, 
but I'm not trusting this website yet. Do you guys have any advices for 
setting up a SSL certificate with nginx - should I use the generated 
certificate from whatsmychaincert.com?

Thanks in advance,

Etienne


Le 2017-07-17 ? 11:32, Etienne Robillard a ?crit :
> Sorry for the noise - apparently the certicate is not yet fully working.
>
> E
>
>
> Le 2017-07-17 ? 11:29, Etienne Robillard a ?crit :
>> Hi,
>>
>> I'm trying to setup ssl on my server and "nginx -t" reports no 
>> problem but the server is now listening on port 256 instead of 443. 
>> Any idea what is wrong ?
>>
>>
>> Thanks,
>>
>>
>> Etienne
>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From fsantiago at garbage-juice.com  Mon Jul 17 23:30:03 2017
From: fsantiago at garbage-juice.com (Fabian A. Santiago)
Date: Mon, 17 Jul 2017 23:30:03 +0000
Subject: Problem setting up ssl
In-Reply-To: <55f0b5bf-abe8-45ca-2355-a6e3e31f5dec@yandex.com>
References: <6cbfcc7b-aa15-8798-c39e-f0b64f5c9d70@yandex.com>
 <dd006140-c1f0-afa5-4368-80d7673d8ea0@yandex.com>
 <55f0b5bf-abe8-45ca-2355-a6e3e31f5dec@yandex.com>
Message-ID: <6C67E321-9E91-4A85-AFEA-923709775080@garbage-juice.com>

On July 17, 2017 7:05:31 PM EDT, Etienne Robillard <tkadm30 at yandex.com> wrote:
>It seems I have a problem with the SSL certificate chain, as per 
>whatsmychaincert.com. However my SSL certificate provider (Gandi) says 
>that the SSL certificate still needs to be attributed... I don't know 
>what to do at this point but to wait for a message from Gandi. 
>Whatsmychaincert.com is telling me to update my certificate (crt) file,
>
>but I'm not trusting this website yet. Do you guys have any advices for
>
>setting up a SSL certificate with nginx - should I use the generated 
>certificate from whatsmychaincert.com?
>
>Thanks in advance,
>
>Etienne
>
>
>Le 2017-07-17 ? 11:32, Etienne Robillard a ?crit :
>> Sorry for the noise - apparently the certicate is not yet fully
>working.
>>
>> E
>>
>>
>> Le 2017-07-17 ? 11:29, Etienne Robillard a ?crit :
>>> Hi,
>>>
>>> I'm trying to setup ssl on my server and "nginx -t" reports no 
>>> problem but the server is now listening on port 256 instead of 443. 
>>> Any idea what is wrong ?
>>>
>>>
>>> Thanks,
>>>
>>>
>>> Etienne
>>>
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>
>
>-- 
>Etienne Robillard
>tkadm30 at yandex.com
>http://www.isotopesoftware.ca/
>
>_______________________________________________
>nginx mailing list
>nginx at nginx.org
>http://mailman.nginx.org/mailman/listinfo/nginx

You likely need to concatenate your cert and your ca's intermediate root signing cert into one file and apply that as your nginx cert with your private key as such. Then reload nginx. 


-- 
Thanks.
Fabian S.

OpenPGP:

3c3fa072accb7ac5db0f723455502b0eeb9070fc

From tkadm30 at yandex.com  Mon Jul 17 23:38:28 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Mon, 17 Jul 2017 19:38:28 -0400
Subject: Problem setting up ssl
In-Reply-To: <6C67E321-9E91-4A85-AFEA-923709775080@garbage-juice.com>
References: <6cbfcc7b-aa15-8798-c39e-f0b64f5c9d70@yandex.com>
 <dd006140-c1f0-afa5-4368-80d7673d8ea0@yandex.com>
 <55f0b5bf-abe8-45ca-2355-a6e3e31f5dec@yandex.com>
 <6C67E321-9E91-4A85-AFEA-923709775080@garbage-juice.com>
Message-ID: <5856e4a3-d2bf-d44b-af7d-a45b876eb55a@yandex.com>



Le 2017-07-17 ? 19:30, Fabian A. Santiago a ?crit :
>
> You likely need to concatenate your cert and your ca's intermediate root signing cert into one file and apply that as your nginx cert with your private key as such. Then reload nginx.
>
Hi Fabian,

Is this precisely whatsmychaincert.com is doing ?

Here is the error msg from nginx when I attempt to use the generated 
cert from whatsmychaincert.com:

nginx: [emerg] 
SSL_CTX_use_PrivateKey_file("/etc/nginx/ssl/ssl-cert-livestore.key") 
failed (SSL: error:0B080074:x509 certificate 
routines:X509_check_private_key:key values mismatch)
nginx: configuration file /etc/nginx/nginx.conf test failed

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From tkadm30 at yandex.com  Tue Jul 18 00:15:18 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Mon, 17 Jul 2017 20:15:18 -0400
Subject: Problem setting up ssl
In-Reply-To: <f0fbfb78-4b7a-83fc-46fa-71c0f9cd5ec7@greengecko.co.nz>
References: <6cbfcc7b-aa15-8798-c39e-f0b64f5c9d70@yandex.com>
 <dd006140-c1f0-afa5-4368-80d7673d8ea0@yandex.com>
 <55f0b5bf-abe8-45ca-2355-a6e3e31f5dec@yandex.com>
 <6C67E321-9E91-4A85-AFEA-923709775080@garbage-juice.com>
 <5856e4a3-d2bf-d44b-af7d-a45b876eb55a@yandex.com>
 <f0fbfb78-4b7a-83fc-46fa-71c0f9cd5ec7@greengecko.co.nz>
Message-ID: <ba48a1aa-c5e4-9d00-6daa-dae5b8896b62@yandex.com>

Hi Steve,

I fixed the issue by rolling my own crt file using Fabian method. :)

Cheers,

E


Le 2017-07-17 ? 20:09, steve a ?crit :
> Hi,
>
>
>
> On 18/07/17 11:38, Etienne Robillard wrote:
>>
>>
>> Le 2017-07-17 ? 19:30, Fabian A. Santiago a ?crit :
>>>
>>> You likely need to concatenate your cert and your ca's intermediate 
>>> root signing cert into one file and apply that as your nginx cert 
>>> with your private key as such. Then reload nginx.
>>>
>> Hi Fabian,
>>
>> Is this precisely whatsmychaincert.com is doing ?
>>
>> Here is the error msg from nginx when I attempt to use the generated 
>> cert from whatsmychaincert.com:
>>
>> nginx: [emerg] 
>> SSL_CTX_use_PrivateKey_file("/etc/nginx/ssl/ssl-cert-livestore.key") 
>> failed (SSL: error:0B080074:x509 certificate 
>> routines:X509_check_private_key:key values mismatch)
>> nginx: configuration file /etc/nginx/nginx.conf test failed
>>
> this means that your key ad cert files are not a pair. You need to 
> update both ( unless you use the same signing request both times - 
> which is not a good idea ).

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From nginx-forum at forum.nginx.org  Tue Jul 18 01:47:39 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Mon, 17 Jul 2017 21:47:39 -0400
Subject: Buffering issues with nginx
In-Reply-To: <20170717091433.GH365@daoine.org>
References: <20170717091433.GH365@daoine.org>
Message-ID: <36a06de0595a6d415191be4f3e9bd249.NginxMailingListEnglish@forum.nginx.org>

Hi Francis,

> Are you concerned about the request or the response being buffered?

my problem is that response that my node app (upstream server) generates is
buffered by nginx.
My actual goal is to know speed and amount of data that my node app sent to
a client. So, when I sent 20MB binary blob from node I can wait till data is
sent out and I'll know approximate speed and size. The moment I put my node
app behind an nginx proxy things fall apart: suddenly all transfers are
instant, and I cannot even know if entire 20MB of data was delivered or if
the connection failed in the middle. When I updated some of these buffering
configs things improved, but still were failing with smaller uploads that
are still fully buffered by nginx.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275544#msg-275544


From kaushalshriyan at gmail.com  Tue Jul 18 06:31:35 2017
From: kaushalshriyan at gmail.com (Kaushal Shriyan)
Date: Tue, 18 Jul 2017 12:01:35 +0530
Subject: Naxsi Nginx security module in nginx webserver
Message-ID: <CAD7Ssm8eCnf=BO8M5v7+Qbyd45GgWhAcLmAeRwMOHOgVmS6kgA@mail.gmail.com>

Hi,

Are there any rpm binary package or yum repo for CentOS 7.x to install
Naxsi Nginx security module in nginx webserver?

Regards,

Kaushal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170718/242110a5/attachment.html>

From nginx-forum at forum.nginx.org  Tue Jul 18 10:16:34 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Tue, 18 Jul 2017 06:16:34 -0400
Subject: number of connections which nginx is waiting the resopnse from
 upstream
Message-ID: <75cbcc5fd787a5661337fce2f9b81160.NginxMailingListEnglish@forum.nginx.org>

according to the documentation of http_stub_status_module, there are 3 kinds
of connections show in the status page, but I'm confused that which kind the
connections, which are waiting or reading responses from upstream servers,
should be classified into ?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275549,275549#msg-275549


From mdounin at mdounin.ru  Tue Jul 18 12:26:45 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Tue, 18 Jul 2017 15:26:45 +0300
Subject: number of connections which nginx is waiting the resopnse from
 upstream
In-Reply-To: <75cbcc5fd787a5661337fce2f9b81160.NginxMailingListEnglish@forum.nginx.org>
References: <75cbcc5fd787a5661337fce2f9b81160.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170718122645.GJ93611@mdounin.ru>

Hello!

On Tue, Jul 18, 2017 at 06:16:34AM -0400, foxgab wrote:

> according to the documentation of http_stub_status_module, there are 3 kinds
> of connections show in the status page, but I'm confused that which kind the
> connections, which are waiting or reading responses from upstream servers,
> should be classified into ?

All connections where there is an active request, and nginx 
already finished reading the request header are classified as 
"Writing".

-- 
Maxim Dounin
http://nginx.org/

From nginx-forum at forum.nginx.org  Wed Jul 19 13:47:46 2017
From: nginx-forum at forum.nginx.org (efcunha)
Date: Wed, 19 Jul 2017 09:47:46 -0400
Subject: Error login submit
Message-ID: <617074d94fc99bf2312861592a3de478.NginxMailingListEnglish@forum.nginx.org>

Does not connect with login and password

[root at pugbalanc01 nginx]# cat /etc/nginx/conf.d/pug01.conf
upstream pug01_servers {
        server 172.16.1.23:80;
        server 172.16.1.24:80;
}

server {
  listen 80;
  server_name xxx.xxx.xx.xxx.xx;
  return  307 https://$server_name$request_uri;
}

server {
  listen 443 ssl;
  server_name xxx.xxx.xx.xxx.xx;

  # SSL code
  ssl on;
  ssl_certificate /etc/nginx/conf.d/certificado.crt;
  ssl_certificate_key /etc/nginx/conf.d/certificado.key;
  ssl_prefer_server_ciphers       on;

  location / {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto https;
    proxy_set_header Host $http_host;
    proxy_redirect off;

    proxy_pass http://pug01_servers;
  }

  client_max_body_size 4G;
  keepalive_timeout 10;
}


10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET / HTTP/1.1" 307 187 "-"
"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0"
"-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET / HTTP/1.1" 200 5530 "-"
"Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0"
"-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /css/print.css HTTP/1.1"
200 586 "https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT 10.0; WOW64;
rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /js/pug.js HTTP/1.1" 200
7942 "https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT 10.0; WOW64;
rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET
/css/jquery-ui-1.8.9.custom.css HTTP/1.1" 200 32681
"https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0)
Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /js/aba.js HTTP/1.1" 200
627 "https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT 10.0; WOW64;
rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /css/pug.css HTTP/1.1" 200
12160 "https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT 10.0; WOW64;
rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /js/jquery-1.4.4.min.js
HTTP/1.1" 200 78601 "https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT
10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET
/js/jquery-ui-1.8.9.custom.min.js HTTP/1.1" 200 166384
"https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0)
Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /images/logomarca.png
HTTP/1.1" 200 27616 "https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT
10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /images/ic_lock.png
HTTP/1.1" 200 12296 "https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT
10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET
/images/tit_jurisdicionados.png HTTP/1.1" 200 56297
"https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0)
Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /favicon.ico HTTP/1.1" 404
2788 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101
Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /images/ajax-loader3.gif
HTTP/1.1" 200 1737 "https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT
10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /css/main.css HTTP/1.1" 200
12108 "https://xxx.xxx.xx.xxx.xx/css/print.css" "Mozilla/5.0 (Windows NT
10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /images/fundo_geral.png
HTTP/1.1" 200 377 "https://xxx.xxx.xx.xxx.xx/css/main.css" "Mozilla/5.0
(Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /images/topo.jpg HTTP/1.1"
200 4124 "https://xxx.xxx.xx.xxx.xx/css/main.css" "Mozilla/5.0 (Windows NT
10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /images/background.jpg
HTTP/1.1" 200 326 "https://xxx.xxx.xx.xxx.xx/css/main.css" "Mozilla/5.0
(Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /images/ilus_bandeira.jpg
HTTP/1.1" 200 1845 "https://xxx.xxx.xx.xxx.xx/css/main.css" "Mozilla/5.0
(Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET
/images/bg_botao_sistemas.jpg HTTP/1.1" 200 330
"https://xxx.xxx.xx.xxx.xx/css/main.css" "Mozilla/5.0 (Windows NT 10.0;
WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /images/bg_sombra_form.gif
HTTP/1.1" 200 59 "https://xxx.xxx.xx.xxx.xx/css/main.css" "Mozilla/5.0
(Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:35 -0400] "GET /favicon.ico HTTP/1.1" 404
2788 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101
Firefox/46.0" "-"
10.69.39.9 - - [19/Jul/2017:05:36:36 -0400] "GET /images/backgroud_menu.jpg
HTTP/1.1" 200 308 "https://xxx.xxx.xx.xxx.xx/css/main.css" "Mozilla/5.0
(Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"
172.16.0.19 - - [19/Jul/2017:05:36:37 -0400] "GET / HTTP/1.1" 307 187 "-"
"curl/7.29.0" "-"
172.16.0.19 - - [19/Jul/2017:05:36:39 -0400] "GET / HTTP/1.1" 307 187 "-"
"curl/7.29.0" "-"

===>>>> Error submit password not connect <=======

10.69.39.9 - - [19/Jul/2017:05:36:40 -0400] "POST /home/loginSubmit
HTTP/1.1" 302 99 "https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT 10.0;
WOW64; rv:46.0) Gecko/20100101 Firefox/46.0" "-"

10.69.39.9 - - [19/Jul/2017:05:36:40 -0400] "GET / HTTP/1.1" 200 5530
"https://xxx.xxx.xx.xxx.xx/" "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0)
Gecko/20100101 Firefox/46.0" "-"

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275563,275563#msg-275563


From nginx-forum at forum.nginx.org  Wed Jul 19 14:33:01 2017
From: nginx-forum at forum.nginx.org (garyc)
Date: Wed, 19 Jul 2017 10:33:01 -0400
Subject: disable request body buffering for file upload
Message-ID: <b31df8a96dd568b9e0e83de137fc94dc.NginxMailingListEnglish@forum.nginx.org>

Hello, hopefully someone can tell me if i am attempting the impossible. 

We use nginx and php5-fpm to handle the upload of test files used by our web
app that may be several gigabytes in size. 

To achieve this we use a location block for the upload url and define the
fastcgi_pass directive to provide the location of the fastcgi socket i.e.

location /api/filter/analysis/upload {
   fastcgi_pass unix:/tmp/php5-fpm.sock;
   include fastcgi_params;
   fastcgi_params SCRIPTFILENAME $document_root/PHP/uploadFile.php;
}

The above configuration functions however we have observed that nginx will
cache the entire request body before calling the php script which
effectively copies the uploaded temporary file to a different location. 

Under certain circumstances this can be a major issue for us as our host
environment is within an instrument that is designed to capture data
continuously and will fill the hard drive close to 100% capacity before
stopping. If a user instigates the upload of a 1gb file that is cached first
by nginx and then copied elsewhere by the php script we can find ourselves
out of disk space at which point nginx (and other processes) understandably
stops functioning leaving the instrument in a state that is difficult to
recover from.

If we can prevent nginx from caching the entire request body and have it
pass it straight to our php script we should be able to take preventative
steps in the script and reject the upload attempt if disk space is too low.

I have looked at the directives 'proxy_request_buffering off;' and
'fastcgi_request_buffering off;' however i have not been successful in
stopping the initial nginx upload before our php script is called.

If this is the wrong approach can anyone suggest a different one? basically
we need to intercept the file upload request and reject it if we are too low
on disk space the current setup effectively uploads the file before we can
perform such a check.

Many thanks

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275567,275567#msg-275567


From mdounin at mdounin.ru  Wed Jul 19 14:59:51 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Wed, 19 Jul 2017 17:59:51 +0300
Subject: disable request body buffering for file upload
In-Reply-To: <b31df8a96dd568b9e0e83de137fc94dc.NginxMailingListEnglish@forum.nginx.org>
References: <b31df8a96dd568b9e0e83de137fc94dc.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170719145951.GP93611@mdounin.ru>

Hello!

On Wed, Jul 19, 2017 at 10:33:01AM -0400, garyc wrote:

> Hello, hopefully someone can tell me if i am attempting the impossible. 
> 
> We use nginx and php5-fpm to handle the upload of test files used by our web
> app that may be several gigabytes in size. 
> 
> To achieve this we use a location block for the upload url and define the
> fastcgi_pass directive to provide the location of the fastcgi socket i.e.
> 
> location /api/filter/analysis/upload {
>    fastcgi_pass unix:/tmp/php5-fpm.sock;
>    include fastcgi_params;
>    fastcgi_params SCRIPTFILENAME $document_root/PHP/uploadFile.php;
> }
> 
> The above configuration functions however we have observed that nginx will
> cache the entire request body before calling the php script which
> effectively copies the uploaded temporary file to a different location. 
> 
> Under certain circumstances this can be a major issue for us as our host
> environment is within an instrument that is designed to capture data
> continuously and will fill the hard drive close to 100% capacity before
> stopping. If a user instigates the upload of a 1gb file that is cached first
> by nginx and then copied elsewhere by the php script we can find ourselves
> out of disk space at which point nginx (and other processes) understandably
> stops functioning leaving the instrument in a state that is difficult to
> recover from.
> 
> If we can prevent nginx from caching the entire request body and have it
> pass it straight to our php script we should be able to take preventative
> steps in the script and reject the upload attempt if disk space is too low.
> 
> I have looked at the directives 'proxy_request_buffering off;' and
> 'fastcgi_request_buffering off;' however i have not been successful in
> stopping the initial nginx upload before our php script is called.

With "fastcgi_request_buffering off;" nginx will send the request 
body to the FastCGI application immediately, without trying to 
buffer it anywhere.

An example configuration:

    location / {
        fastcgi_pass 127.0.0.1:10002;
        fastcgi_request_buffering off;
    }

It is up to your FastCGI application to handle this though, 
and PHP as well as PHP-FPM may impose additional limitations 
and/or require additional configuration for this to work.

> If this is the wrong approach can anyone suggest a different one? basically
> we need to intercept the file upload request and reject it if we are too low
> on disk space the current setup effectively uploads the file before we can
> perform such a check.

Doing such checks in nginx (for example, you can use embedded perl 
to do such checks) and/or keeping client_max_body_size low enough might 
be a better solution.  Nevertheless, fastcgi_request_buffering is 
expected to work as well.

-- 
Maxim Dounin
http://nginx.org/

From nginx-forum at forum.nginx.org  Wed Jul 19 15:22:04 2017
From: nginx-forum at forum.nginx.org (garyc)
Date: Wed, 19 Jul 2017 11:22:04 -0400
Subject: disable request body buffering for file upload
In-Reply-To: <20170719145951.GP93611@mdounin.ru>
References: <20170719145951.GP93611@mdounin.ru>
Message-ID: <6961dc7a56fed6bdceabfe015980d696.NginxMailingListEnglish@forum.nginx.org>

Thank you, it sounds like this approach may yet work.

> It is up to your FastCGI application to handle this though, 
> and PHP as well as PHP-FPM may impose additional limitations 
> and/or require additional configuration for this to work.

I will dig into the php configuration details and see if i can progress
this, it is reassuring to hear that the fastcgi_request_buffering directive
is intended to facilitate this.

Thanks for the quick response.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275567,275570#msg-275570


From kaushalshriyan at gmail.com  Wed Jul 19 16:58:36 2017
From: kaushalshriyan at gmail.com (Kaushal Shriyan)
Date: Wed, 19 Jul 2017 22:28:36 +0530
Subject: Naxsi Nginx security module in nginx webserver
In-Reply-To: <CAD7Ssm8eCnf=BO8M5v7+Qbyd45GgWhAcLmAeRwMOHOgVmS6kgA@mail.gmail.com>
References: <CAD7Ssm8eCnf=BO8M5v7+Qbyd45GgWhAcLmAeRwMOHOgVmS6kgA@mail.gmail.com>
Message-ID: <CAD7Ssm-7zBEzW17TLMnzZBaR9qXT=zKgkSai-zADh7yMEpSN=w@mail.gmail.com>

On Tue, Jul 18, 2017 at 12:01 PM, Kaushal Shriyan <kaushalshriyan at gmail.com>
wrote:

> Hi,
>
> Are there any rpm binary package or yum repo for CentOS 7.x to install
> Naxsi Nginx security module in nginx webserver?
>
> Regards,
>
> Kaushal
>
>
Hi,

I will appreciate if somebody can pitch in for help for earlier post to
this mailing list.

Thanks in Advance.

Regards,

Kaushal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170719/d1a7765e/attachment.html>

From francis at daoine.org  Wed Jul 19 20:15:51 2017
From: francis at daoine.org (Francis Daly)
Date: Wed, 19 Jul 2017 21:15:51 +0100
Subject: Buffering issues with nginx
In-Reply-To: <36a06de0595a6d415191be4f3e9bd249.NginxMailingListEnglish@forum.nginx.org>
References: <20170717091433.GH365@daoine.org>
 <36a06de0595a6d415191be4f3e9bd249.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170719201551.GK365@daoine.org>

On Mon, Jul 17, 2017 at 09:47:39PM -0400, Dan34 wrote:

Hi there,

> > Are you concerned about the request or the response being buffered?
> 
> my problem is that response that my node app (upstream server) generates is
> buffered by nginx.

So you want proxy_buffering (http://nginx.org/r/proxy_buffering) set to
"off" in the location that does the proxy_pass.

> My actual goal is to know speed and amount of data that my node app sent to
> a client.

It may be the case that that is not knowable, in general, in a single
http request.

Depending on the compromises you are willing to make, to accuracy or
convenience, you may be able to come up with something good enough.

> So, when I sent 20MB binary blob from node I can wait till data is
> sent out and I'll know approximate speed and size. The moment I put my node
> app behind an nginx proxy things fall apart: suddenly all transfers are
> instant, and I cannot even know if entire 20MB of data was delivered or if
> the connection failed in the middle.

Yes. That is (part of) what a proxy does. Even without nginx as a
reverse-proxy, your client might be talking through one or more proxy
servers. You will never know whether your response got to the actual
end client, without some extra verification step that only the end
client does.

> When I updated some of these buffering
> configs things improved, but still were failing with smaller uploads that
> are still fully buffered by nginx.

proxy_buffers and proxy_buffer_size can be tuned (lowered, in this case,
probably) to slow down nginx's receive-rate from your upstream.

If you can show one working configuration with a description of how
it does not do what you want it to do, possibly someone can offer some
advice on what to change.

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org

From igal at lucee.org  Wed Jul 19 20:20:05 2017
From: igal at lucee.org (Igal @ Lucee.org)
Date: Wed, 19 Jul 2017 13:20:05 -0700
Subject: Buffering issues with nginx
In-Reply-To: <36a06de0595a6d415191be4f3e9bd249.NginxMailingListEnglish@forum.nginx.org>
References: <20170717091433.GH365@daoine.org>
 <36a06de0595a6d415191be4f3e9bd249.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <cb007dcd-2c6b-e6c9-c310-35452699884b@lucee.org>

> my problem is that response that my node app (upstream server) generates is
> buffered by nginx.
Generate the following header from node:

     X-Accel-Buffering: no

That will disable nginx's buffering for the request.


Igal Sapir
Lucee Core Developer
Lucee.org <http://lucee.org/>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170719/7c545195/attachment-0001.html>

From kayla.manchette at devplateau.com  Wed Jul 19 22:02:16 2017
From: kayla.manchette at devplateau.com (Kayla Manchette)
Date: Wed, 19 Jul 2017 18:02:16 -0400
Subject: Pages under main page not loading with Nginx
Message-ID: <8acc232b-77dd-dd85-efa6-d234a10e77cb@devplateau.com>

Hello,

I just got Nginx working with my website today, looking at transitioning 
from Apache. I have pointed Nginx at the folder where all the files for 
my website are located, and Nginx will show the main page just fine. 
When I try to go to another page on my website, it cannot tell where 
those files are. I just receive a 404 not found error.

Can anyone help me figure out why I cannot get Nginx to recognize the 
other pages of my site?

The main page is an html file and the rest are php files. Does that have 
something to do with it?

I have attached the Nginx file for my site.

Any help would be greatly appreciated.


-------------- next part --------------
#  You may add here your
# statements for each of your virtual hosts to this file

##
# You should look at the following URL's in order to grasp a solid understanding
# of Nginx configuration files in order to fully unleash the power of Nginx.
# http://wiki.nginx.org/Pitfalls
# http://wiki.nginx.org/QuickStart
# http://wiki.nginx.org/Configuration
#
# Generally, you will want to move this file somewhere, and start with a clean
# file but keep this around for reference. Or just disable in sites-enabled.
#
# Please see /usr/share/doc/nginx-doc/examples/ for more detailed examples.
##

server {
	listen 8888 default_server;
	listen [::]:8888 default_server ipv6only=on;

	root /var/www/html/;
	index index.php index.html index.htm;

	# Make site accessible from http://localhost/
	server_name devplateau.com www.devplateau.com;

	location / {
		# First attempt to serve request as file, then
		# as directory, then fall back to displaying a 404.
		try_files $uri $uri/ =404;
		# Uncomment to enable naxsi on this location
#		include /etc/nginx/naxsi.rules
	}

	# Only for nginx-naxsi used with nginx-naxsi-ui : process denied requests
#	location /RequestDenied {
#		proxy_pass http://127.0.0.1:8080;    
#	}

	error_page 404 /404.html;

	# redirect server error pages to the static page /50x.html
	#
	error_page 500 502 503 504 /50x.html;
	location = /50x.html {
		root /usr/share/nginx/html;
	}

	# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
	#
	location ~ \.php$ {
		fastcgi_split_path_info ^(.+\.php)(/.+)$;
	#	# NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
	#
	#	# With php5-cgi alone:
		fastcgi_pass 127.0.0.1:9000;
	#	# With php5-fpm:
	#	fastcgi_pass unix:/var/run/php5-fpm.sock;
		fastcgi_index index.php;
		include fastcgi_params;
	}

	# deny access to .htaccess files, if Apache's document root
	# concurs with nginx's one
	#
#	location ~ /\.ht {
#		deny all;
#	}
}


# another virtual host using mix of IP-, name-, and port-based configuration

#server {
#	listen 8000;
#	listen devplateau.com:8080;
#	server_name devplateau.com alias www.devplateau.com;
#	root /var/www/html;
#	index index.php;

#	location / {
#		try_files $uri $uri/ =404;
#	}
        
#        location /doc/ {
#                alias /usr/share/doc/;
#                autoindex on;
#                allow 127.0.0.1;
#                deny all;
#        }

#        location ~/\.ht {
#                deny all;
#        }
#}


# HTTPS server
#
#server {
#	listen 443;
#	server_name localhost;

#	root html;
#	index index.html index.htm;

#	ssl on;
#	ssl_certificate cert.pem;
#	ssl_certificate_key cert.key;

#	ssl_session_timeout 5m;

#	ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
#	ssl_ciphers "HIGH:!aNULL:!MD5 or HIGH:!aNULL:!MD5:!3DES";
#	ssl_prefer_server_ciphers on;

#	location / {
#		try_files $uri $uri/ =404;
#	}
#}

#server {
#        listen 443 ssl default_server;
#        listen [::]:443 ssl default_server;

#        root /var/www/html;

#        index index.html index.htm index.nginx-debian.html;

#        server_name devplateau.com;

#        location / {
#                try_files $uri $uri/ =404;
#        }

#        ssl_certificate /etc/nginx/ssl/devplateau.com.crt;
#        ssl_certificate_key /etc/nginx/ssl/devplateau.com.key;
#        ssl_dhparam /etc/nginx/ssl/dhparam.pem;
#}


#server {
#       listen         80;
#       listen    [::]:80;
#       server_name    devplateau.com;
#       return         301 https://$server_name$request_uri;
#}

From phil at pricom.com.au  Thu Jul 20 03:44:10 2017
From: phil at pricom.com.au (Philip Rhoades)
Date: Thu, 20 Jul 2017 13:44:10 +1000
Subject: Specify a Vary: Accept-Encoding header
Message-ID: <dd61248537d0b68aa2a79cb6e82ffc7b@abic.net.au>

People,

I have moved my (very low hit) web sites from a Digital Ocean server to 
my own Fedora 25 x86-64 server with 8GB RAM and an ADSL2+ upload speed 
of only about 1Mbit/sec.

The plain HTML and Jekyll sites response times are not too bad but the 
Rails sites are very slow.

Using:

   https://tools.pingdom.com

takes more than a minute to test one of my Rails sites:

   http://cryonics.org.au

so it appears that I need to do something for all the Rails sites at 
least but ALL sites report a value of 50 or less for the condition in 
the Subject of this mail - I tried putting:

   gzip on;
   gzip_min_length  1100;
   gzip_buffers  4 32k;
   gzip_types    text/plain application/x-javascript text/xml text/css;
   gzip_vary on;

in the nginx server conf for cryonics.org.au and restarted nginx but it 
did not make any difference.

Suggestions about how to improve the situation?

Thanks,

Phil.
-- 
Philip Rhoades

PO Box 896
Cowra  NSW  2794
Australia
E-mail:  phil at pricom.com.au

From kgorlo at gmail.com  Thu Jul 20 09:18:54 2017
From: kgorlo at gmail.com (Kamil Gorlo)
Date: Thu, 20 Jul 2017 09:18:54 +0000
Subject: Total timeout for proxy_pass
Message-ID: <CAHzX=KAFdZA7eZO04QXOp+xvBVNTczFoCyoryd31hHXTYp6Tuw@mail.gmail.com>

Hi,

is it possible to somehow configure proxy_pass (or with help with some
additional module) to have total timeout for whole request to upstream
(including all send/read calls + connect for non-keepalive connections)?

I have use case when I need to respond with default content when my backend
do not respond in some strict time. Any help would be appreciated!

Cheers,
Kamil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170720/728a085a/attachment.html>

From director_de at yahoo.de  Thu Jul 20 09:17:02 2017
From: director_de at yahoo.de (tom)
Date: Thu, 20 Jul 2017 09:17:02 +0000 (UTC)
Subject: nginx_mail_proxy authenticate to imap_ssl upstream ssl
References: <593578730.1541649.1500542222120.ref@mail.yahoo.com>
Message-ID: <593578730.1541649.1500542222120@mail.yahoo.com>

Hello list,
I configured sucessfully the mail_proxy for nginx 1.10.2 von RHEL7, but authentication only succeeds if upstream server which is provided by the auth_http Server is cleartext, e.g. if the auth-server responds
2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Status: OK"
2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Server: 192.168.0.200"
2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Port: 143"
then everything works fine, but having
2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Status: OK"
2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Server: 192.168.0.200"
2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Port: 993"
2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-SSL: on"

I get 
2017/07/20 11:03:47 [info] 9535#0: *49 upstream timed out (110: Connection timed out) while connecting to upstream, client: 192.168.0.200, server: 0.0.0.0:10993, login: "user at domain.com", upstream: 192.168.0.200:993
When I directly do a 

openssl s_client -connect 192.168.0.200:993 -crlf
I am able to login with
. login user at domain.com password

Any help is appreciated.
Thomas

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170720/a396245f/attachment.html>

From nginx-forum at forum.nginx.org  Thu Jul 20 10:36:21 2017
From: nginx-forum at forum.nginx.org (garyc)
Date: Thu, 20 Jul 2017 06:36:21 -0400
Subject: disable request body buffering for file upload
In-Reply-To: <20170719145951.GP93611@mdounin.ru>
References: <20170719145951.GP93611@mdounin.ru>
Message-ID: <32e60dfc1f6f6d1975da3192cb6509fc.NginxMailingListEnglish@forum.nginx.org>

Hi Maxim,

> With "fastcgi_request_buffering off;" nginx will send the request 
> body to the FastCGI application immediately, without trying to 
> buffer it anywhere.

I have been monitoring the disk space while uploading a test file of 1.1 GB
in size and have confirmed that with the fastcgi_request_buffering directive
set to 'on' the disk space reduces by approximately 2.2 GB. 

With the fastcgi_request_buffering directive set to 'off' the disk space
reduces by approximately 1.1GB so it does look like the
fastcgi_request_buffering directive is doing what it should however nginx
still appears to cache the entire request body before it is proxied to
fastcgi.

> It is up to your FastCGI application to handle this though, 
> and PHP as well as PHP-FPM may impose additional limitations 
> and/or require additional configuration for this to work.

I am still looking into PHP & php5-fpm configuration parameters in case this
having an effect but it all feels like this cache is happening before the
proxy.

>From what i have researched so far it looks like nginx may always initially
cache the entire request body in some form before the request is proxied.

Another approach we could take would be to add a disk space check api to our
web app so we can confirm we have sufficient space before the upload POST
call is made.

I appreciate that running a web server in a low disk space environment is
not a common scenario however, in your opinion, should it be possible to get
details of the upload POST request to a script before the entire request
body is written to disk by nginx?

Many thanks
Gary

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275567,275584#msg-275584


From r1ch+nginx at teamliquid.net  Thu Jul 20 12:45:03 2017
From: r1ch+nginx at teamliquid.net (Richard Stanway)
Date: Thu, 20 Jul 2017 14:45:03 +0200
Subject: Specify a Vary: Accept-Encoding header
In-Reply-To: <dd61248537d0b68aa2a79cb6e82ffc7b@abic.net.au>
References: <dd61248537d0b68aa2a79cb6e82ffc7b@abic.net.au>
Message-ID: <CAKWQeyicxxY2tmsCVFt2sHzewu_JAO0i3YAK+THzpGfsZ35k8g@mail.gmail.com>

The issue is not with your page size or gzip (or anything nginx
related actually). Your Rails backend is generating the content far
too slow. You should investigate why your backend is so slow.

            time_namelookup:  0.004209
               time_connect:  0.241082
            time_appconnect:  0.000000
           time_pretransfer:  0.241121
              time_redirect:  0.000000
         time_starttransfer:  20.519778
                            ----------
                 time_total:  20.568794


On Thu, Jul 20, 2017 at 5:44 AM, Philip Rhoades <phil at pricom.com.au> wrote:
> People,
>
> I have moved my (very low hit) web sites from a Digital Ocean server to my
> own Fedora 25 x86-64 server with 8GB RAM and an ADSL2+ upload speed of only
> about 1Mbit/sec.
>
> The plain HTML and Jekyll sites response times are not too bad but the Rails
> sites are very slow.
>
> Using:
>
>   https://tools.pingdom.com
>
> takes more than a minute to test one of my Rails sites:
>
>   http://cryonics.org.au
>
> so it appears that I need to do something for all the Rails sites at least
> but ALL sites report a value of 50 or less for the condition in the Subject
> of this mail - I tried putting:
>
>   gzip on;
>   gzip_min_length  1100;
>   gzip_buffers  4 32k;
>   gzip_types    text/plain application/x-javascript text/xml text/css;
>   gzip_vary on;
>
> in the nginx server conf for cryonics.org.au and restarted nginx but it did
> not make any difference.
>
> Suggestions about how to improve the situation?
>
> Thanks,
>
> Phil.
> --
> Philip Rhoades
>
> PO Box 896
> Cowra  NSW  2794
> Australia
> E-mail:  phil at pricom.com.au
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

From mdounin at mdounin.ru  Thu Jul 20 13:59:48 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 20 Jul 2017 16:59:48 +0300
Subject: nginx_mail_proxy authenticate to imap_ssl upstream ssl
In-Reply-To: <593578730.1541649.1500542222120@mail.yahoo.com>
References: <593578730.1541649.1500542222120.ref@mail.yahoo.com>
 <593578730.1541649.1500542222120@mail.yahoo.com>
Message-ID: <20170720135947.GZ93611@mdounin.ru>

Hello!

On Thu, Jul 20, 2017 at 09:17:02AM +0000, tom via nginx wrote:

> Hello list,
> I configured sucessfully the mail_proxy for nginx 1.10.2 von RHEL7, but authentication only succeeds if upstream server which is provided by the auth_http Server is cleartext, e.g. if the auth-server responds
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Status: OK"
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Server: 192.168.0.200"
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Port: 143"
> then everything works fine, but having
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Status: OK"
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Server: 192.168.0.200"
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Port: 993"
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-SSL: on"
> 
> I get 
> 2017/07/20 11:03:47 [info] 9535#0: *49 upstream timed out (110: Connection timed out) while connecting to upstream, client: 192.168.0.200, server: 0.0.0.0:10993, login: "user at domain.com", upstream: 192.168.0.200:993
> When I directly do a 
> 
> openssl s_client -connect 192.168.0.200:993 -crlf
> I am able to login with
> . login user at domain.com password
> 
> Any help is appreciated.

The "Auth-SSL" header is meaningful in auth_http requests, and 
means that client used SSL.  It doesn't mean anything in 
auth_http responses.

Moreover, connecting to SSL mail backends is not supported.  If you 
really need it, consider connecting to a tunnel wich will do SSL 
for you - for example, the stream module can be configured to do 
this.

-- 
Maxim Dounin
http://nginx.org/

From mdounin at mdounin.ru  Thu Jul 20 14:10:11 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 20 Jul 2017 17:10:11 +0300
Subject: disable request body buffering for file upload
In-Reply-To: <32e60dfc1f6f6d1975da3192cb6509fc.NginxMailingListEnglish@forum.nginx.org>
References: <20170719145951.GP93611@mdounin.ru>
 <32e60dfc1f6f6d1975da3192cb6509fc.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170720141011.GA93611@mdounin.ru>

Hello!

On Thu, Jul 20, 2017 at 06:36:21AM -0400, garyc wrote:

[...]

> Another approach we could take would be to add a disk space check api to our
> web app so we can confirm we have sufficient space before the upload POST
> call is made.
> 
> I appreciate that running a web server in a low disk space environment is
> not a common scenario however, in your opinion, should it be possible to get
> details of the upload POST request to a script before the entire request
> body is written to disk by nginx?

Consider the auth request module, 
http://nginx.org/en/docs/http/ngx_http_auth_request_module.html.

-- 
Maxim Dounin
http://nginx.org/

From director_de at yahoo.de  Thu Jul 20 14:41:10 2017
From: director_de at yahoo.de (tom)
Date: Thu, 20 Jul 2017 14:41:10 +0000 (UTC)
Subject: nginx_mail_proxy authenticate to imap_ssl upstream ssl
In-Reply-To: <20170720135947.GZ93611@mdounin.ru>
References: <593578730.1541649.1500542222120.ref@mail.yahoo.com>
 <593578730.1541649.1500542222120@mail.yahoo.com>
 <20170720135947.GZ93611@mdounin.ru>
Message-ID: <968734194.1971566.1500561670471@mail.yahoo.com>

Hello Maxim,is there any way with the stream module to get the authentication failures reported? As I did not get the authentication failure in my nginx log when using the stream module, I switched to the mail module. But my Backend speaks only IMAP-SSL, therefore this does not work.Any help is appreciated,Best regards,Thomas
 

    Maxim Dounin <mdounin at mdounin.ru> schrieb am 15:59 Donnerstag, 20.Juli 2017:
 

 Hello!

On Thu, Jul 20, 2017 at 09:17:02AM +0000, tom via nginx wrote:

> Hello list,
> I configured sucessfully the mail_proxy for nginx 1.10.2 von RHEL7, but authentication only succeeds if upstream server which is provided by the auth_http Server is cleartext, e.g. if the auth-server responds
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Status: OK"
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Server: 192.168.0.200"
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Port: 143"
> then everything works fine, but having
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Status: OK"
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Server: 192.168.0.200"
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-Port: 993"
> 2017/07/20 11:02:47 [debug] 9535#0: *49 mail auth http header: "Auth-SSL: on"
> 
> I get 
> 2017/07/20 11:03:47 [info] 9535#0: *49 upstream timed out (110: Connection timed out) while connecting to upstream, client: 192.168.0.200, server: 0.0.0.0:10993, login: "user at domain.com", upstream: 192.168.0.200:993
> When I directly do a 
> 
> openssl s_client -connect 192.168.0.200:993 -crlf
> I am able to login with
> . login user at domain.com password
> 
> Any help is appreciated.

The "Auth-SSL" header is meaningful in auth_http requests, and 
means that client used SSL.? It doesn't mean anything in 
auth_http responses.

Moreover, connecting to SSL mail backends is not supported.? If you 
really need it, consider connecting to a tunnel wich will do SSL 
for you - for example, the stream module can be configured to do 
this.

-- 
Maxim Dounin
http://nginx.org/


   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170720/1a503215/attachment.html>

From leeon2013 at gmail.com  Thu Jul 20 15:24:44 2017
From: leeon2013 at gmail.com (David Woodstuck)
Date: Thu, 20 Jul 2017 11:24:44 -0400
Subject: conditional proxy_pass
Message-ID: <CAD6THg64C5Vvt2Gq-nU2JKE4d1b8dE+49inXHFrWgQuN8zK1uA@mail.gmail.com>

      I just use Nginx for a couple of weeks. I have the following
questions. I hope I can get your help.

I have a big web application whose urls are dev01.mydomain.com,
dev02.mydomain and dev03.mydomain.com. I like to use Nginx to deliver this
application for its users. On each page, there are a lot of iframes whose
src can be one of dev01.mydomain.com, dev02.mydomain and dev03.mydomain.com.
I like to use following conf to achieve one url for its users:

server {
        listen       80;
        server_name  localhost;

        #charset koi8-r;
        #access_log  /var/log/nginx/log/host.access.log  main;

        location / {
            root   /usr/share/nginx/html;
            index  index.html index.htm;

            proxy_pass http://dev01.mydomain.com/;
            subs_filter 'http://dev02.mydomain.com/(.*)' '
http://dev01.mydomain.com/?http://dev02.mydomain.com/$1';
            subs_filter 'http://dev03.mydomain.com/(.*)' '
http://dev01.mydomain.com/?http://dev03.mydomain.com/$1';
         }
}

My requirements are:
 (1). Request with url - http://www.mydomain.com/?http://... will proxy to
http://dev02.mydomain.com/ or http://dev03.mydomain.com/ depending on a
value after ?
 (2). Other request will proxy to http://dev01.mydomain.com/.

How do I achieve these requirements?

Thanks,

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170720/c1f0ad8a/attachment.html>

From mdounin at mdounin.ru  Thu Jul 20 15:43:56 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 20 Jul 2017 18:43:56 +0300
Subject: nginx_mail_proxy authenticate to imap_ssl upstream ssl
In-Reply-To: <968734194.1971566.1500561670471@mail.yahoo.com>
References: <593578730.1541649.1500542222120.ref@mail.yahoo.com>
 <593578730.1541649.1500542222120@mail.yahoo.com>
 <20170720135947.GZ93611@mdounin.ru>
 <968734194.1971566.1500561670471@mail.yahoo.com>
Message-ID: <20170720154356.GB93611@mdounin.ru>

Hello!

On Thu, Jul 20, 2017 at 02:41:10PM +0000, tom via nginx wrote:

> Hello Maxim,is there any way with the stream module to get the 
> authentication failures reported? As I did not get the 
> authentication failure in my nginx log when using the stream 
> module, I switched to the mail module. But my Backend speaks 
> only IMAP-SSL, therefore this does not work.Any help is 
> appreciated,Best regards,Thomas

The stream module is not expected to report anything.  My 
suggestion was to use the mail module _and_ the stream module to 
do SSL.

-- 
Maxim Dounin
http://nginx.org/

From francis at daoine.org  Thu Jul 20 21:11:39 2017
From: francis at daoine.org (Francis Daly)
Date: Thu, 20 Jul 2017 22:11:39 +0100
Subject: disable request body buffering for file upload
In-Reply-To: <32e60dfc1f6f6d1975da3192cb6509fc.NginxMailingListEnglish@forum.nginx.org>
References: <20170719145951.GP93611@mdounin.ru>
 <32e60dfc1f6f6d1975da3192cb6509fc.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170720211139.GL365@daoine.org>

On Thu, Jul 20, 2017 at 06:36:21AM -0400, garyc wrote:

Hi there,

> With the fastcgi_request_buffering directive set to 'off' the disk space
> reduces by approximately 1.1GB so it does look like the
> fastcgi_request_buffering directive is doing what it should however nginx
> still appears to cache the entire request body before it is proxied to
> fastcgi.

Why do you think that?

If the problem is nginx buffering when it should not, then the fix should
be on the nginx side. If the problem is something else -- the fastcgi
server or something else doing the buffering -- then time spent changing
nginx is wasted.

Can you find out what is really happening?

For example: change the fastcgi server to listen on a network port;
change nginx to fastcgi_pass to that; and use tcpdump to watch the
traffic involving that port.

Then do a file upload that takes 30 seconds or more.

Does the tcpdump trace show the right amount of traffic going from
nginx to the fastcgi server during that 30 seconds while the upload is
happening? Or does it show no traffic until the upload has completed,
and then lots?

With that information, it may become clear whether the problem can be
fixed in nginx, or whether it must be handled elsewhere.

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org

From francis at daoine.org  Thu Jul 20 21:20:05 2017
From: francis at daoine.org (Francis Daly)
Date: Thu, 20 Jul 2017 22:20:05 +0100
Subject: Pages under main page not loading with Nginx
In-Reply-To: <8acc232b-77dd-dd85-efa6-d234a10e77cb@devplateau.com>
References: <8acc232b-77dd-dd85-efa6-d234a10e77cb@devplateau.com>
Message-ID: <20170720212005.GM365@daoine.org>

On Wed, Jul 19, 2017 at 06:02:16PM -0400, Kayla Manchette wrote:

Hi there,

> I have pointed Nginx at the folder where
> all the files for my website are located, and Nginx will show the
> main page just fine. When I try to go to another page on my website,
> it cannot tell where those files are. I just receive a 404 not found
> error.

What one request do you make that gives a 404?

(It should be in the nginx logs.)

What file on the filesystem do you want nginx to return, instead of
that 404?

> The main page is an html file and the rest are php files. Does that
> have something to do with it?

> 	location ~ \.php$ {
> 		fastcgi_pass 127.0.0.1:9000;
> 		include fastcgi_params;
> 	}

Does that fastcgi_params file include SCRIPT_FILENAME? If not, you may
need to include some appropriate variant of "fastcgi_param SCRIPT_FILENAME
$request_filename;" within that location{}.

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org

From shuxinyang.oss at gmail.com  Fri Jul 21 05:32:15 2017
From: shuxinyang.oss at gmail.com (Shuxin Yang)
Date: Thu, 20 Jul 2017 22:32:15 -0700
Subject: nginx security advisory (CVE-2017-7529)
In-Reply-To: <20170711154745.GF55433@mdounin.ru>
References: <20170711154745.GF55433@mdounin.ru>
Message-ID: <55405e97-8238-de6e-1d0f-571f3e7bb37a@gmail.com>

HI, There:

    I try to exploit this bug in an attempt to do something nasty :-). 
However, the more I dig into it, the more I get confused.

    As far as I know, one necessary conditional to trigger the problem 
is that range-filter kicks in, and range-filter is called if and *ONLY* 
if (FIXME):

   1) Nginx serves static file.

   2) Nginx serves as a reverse-proxy *with* cache capability, and the 
resource being accessed is *cache-able*. In this case, the proxy will 
fetch the entire file from origin, cache it, then send the ranges down 
to downstream via range-filter. (If content is not cache-able, range 
range in reverse proxy will not be enabled, instead, the range request 
is directly forwarded to upstream. Also, in proxy without cache, rang 
filter will not be invoked)

Questions:

======

   a). Does this bug have any negative impact to 1)?

   b). Where exactly does buffer overflow take place? How does this 
patch resolve the problem. As far as I can understand, the patch only 
check if total size of the ranges exceeds 4G (assuming 32-bit system for 
simplicity). The start/end pointer of each range is guaranteed in the 
range of "[0, content-length]" regardless the patched condition.

   c) Could following statement have overflow problem as well when the 
start/end point is very close 4G?

      850         if (ngx_buf_in_memory(buf)) {
      851             b->pos = buf->pos + (size_t) range[i].start;
      852             b->last = buf->pos + (size_t) range[i].end;
      853         }

    d) the patch guarantees the total size of ranges is smaller than 4G 
(again, assume 32bit system). But what if it ends up very close to 4G, 
making the "len" variable in function variable 
ngx_http_range_multipart_header() overflow. The "len" is to calculate 
the content-length the resulting response, it is the total size of 
multi-part overhead plus ranges.

     Is there any simple example to reproduce the problem?

Thanks a lot!

Shuxin

On 07/11/2017 08:47 AM, Maxim Dounin wrote:
> Hello!
>
> A security issue was identified in nginx range filter.  A specially
> crafted request might result in an integer overflow and incorrect
> processing of ranges, potentially resulting in sensitive information
> leak (CVE-2017-7529).
>
> When using nginx with standard modules this allows an attacker to
> obtain a cache file header if a response was returned from cache.
> In some configurations a cache file header may contain IP address
> of the backend server or other sensitive information.
>
> Besides, with 3rd party modules it is potentially possible that
> the issue may lead to a denial of service or a disclosure of
> a worker process memory.  No such modules are currently known though.
>
> The issue affects nginx 0.5.6 - 1.13.2.
> The issue is fixed in nginx 1.13.3, 1.12.1.
>
> For older versions, the following configuration can be used
> as a temporary workaround:
>
>      max_ranges 1;
>
> Patch for the issue can be found here:
>
> http://nginx.org/download/patch.2017.ranges.txt
>
>


From nginx-forum at forum.nginx.org  Fri Jul 21 08:22:03 2017
From: nginx-forum at forum.nginx.org (S.A.N)
Date: Fri, 21 Jul 2017 04:22:03 -0400
Subject: disable request body buffering for file upload
In-Reply-To: <20170720211139.GL365@daoine.org>
References: <20170720211139.GL365@daoine.org>
Message-ID: <89b5c022e58d98426e7309ee6024688b.NginxMailingListEnglish@forum.nginx.org>

I advise you implementing - direct file upload.
Read more:
https://stackoverflow.com/questions/44371643/nginx-php-failing-with-large-file-uploads-over-6-gb/44751210#44751210

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275567,275602#msg-275602


From nginx-forum at forum.nginx.org  Fri Jul 21 09:12:40 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Fri, 21 Jul 2017 05:12:40 -0400
Subject: Buffering issues with nginx
In-Reply-To: <cb007dcd-2c6b-e6c9-c310-35452699884b@lucee.org>
References: <cb007dcd-2c6b-e6c9-c310-35452699884b@lucee.org>
Message-ID: <411cecaf1f3ae2e1c3037dc04864738a.NginxMailingListEnglish@forum.nginx.org>

> X-Accel-Buffering: no
> That will disable nginx's buffering for the request.

At first it looked like exactly what I was looking for (after reading nginx
docs), but after trying I observed that there were no effects from that.
In code that writes headers I added res.setHeader('X-Accel-Buffering',
'no');

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275603#msg-275603


From peter_booth at me.com  Fri Jul 21 10:58:27 2017
From: peter_booth at me.com (Peter Booth)
Date: Fri, 21 Jul 2017 06:58:27 -0400
Subject: Specify a Vary: Accept-Encoding header
In-Reply-To: <CAKWQeyicxxY2tmsCVFt2sHzewu_JAO0i3YAK+THzpGfsZ35k8g@mail.gmail.com>
References: <dd61248537d0b68aa2a79cb6e82ffc7b@abic.net.au>
 <CAKWQeyicxxY2tmsCVFt2sHzewu_JAO0i3YAK+THzpGfsZ35k8g@mail.gmail.com>
Message-ID: <AAD91198-92EA-4B43-B5F4-8ECB36A08A77@me.com>

It looks as if the static content is being served by the Rails asset pipeline rather than directly by nginx 
and the impact is enormous. It took 25s for the base page - but it also took 
another 25s for the http://cryonics.org.au/assets/application.js <http://cryonics.org.au/assets/application.js> resource
and another 20s for http://cryonics.org.au/assets/bg.gif <http://cryonics.org.au/assets/bg.gif>
and another 20s for http://cryonics.org.au/assets/front.jpg

See https://www.webpagetest.org/result/170720_M2_1T8G/1/details/#waterfall_view_step1 <https://www.webpagetest.org/result/170720_M2_1T8G/1/details/#waterfall_view_step1> for a detailed breakdown.


In some ways Rails is the antithesis of nginx. Whilst nginx is fast out of the box with default 
configuration, rails is stunningly slow without a lot of tweaking. There are many tools that 
can help with Ruby / Rails tuning.

The one?s that I find most useful are New Relic - for its visualization of how slow vs fast 
requests vary, rack-mini-profiler and the bullet gem.  I have heard good things about 
Skylight and about Scout but haven?t tried either myself. 

Good luck,

Peter


> On Jul 20, 2017, at 8:45 AM, Richard Stanway <r1ch+nginx at teamliquid.net> wrote:
> 
> The issue is not with your page size or gzip (or anything nginx
> related actually). Your Rails backend is generating the content far
> too slow. You should investigate why your backend is so slow.
> 
>            time_namelookup:  0.004209
>               time_connect:  0.241082
>            time_appconnect:  0.000000
>           time_pretransfer:  0.241121
>              time_redirect:  0.000000
>         time_starttransfer:  20.519778
>                            ----------
>                 time_total:  20.568794
> 
> 
> On Thu, Jul 20, 2017 at 5:44 AM, Philip Rhoades <phil at pricom.com.au> wrote:
>> People,
>> 
>> I have moved my (very low hit) web sites from a Digital Ocean server to my
>> own Fedora 25 x86-64 server with 8GB RAM and an ADSL2+ upload speed of only
>> about 1Mbit/sec.
>> 
>> The plain HTML and Jekyll sites response times are not too bad but the Rails
>> sites are very slow.
>> 
>> Using:
>> 
>>  https://tools.pingdom.com
>> 
>> takes more than a minute to test one of my Rails sites:
>> 
>>  http://cryonics.org.au
>> 
>> so it appears that I need to do something for all the Rails sites at least
>> but ALL sites report a value of 50 or less for the condition in the Subject
>> of this mail - I tried putting:
>> 
>>  gzip on;
>>  gzip_min_length  1100;
>>  gzip_buffers  4 32k;
>>  gzip_types    text/plain application/x-javascript text/xml text/css;
>>  gzip_vary on;
>> 
>> in the nginx server conf for cryonics.org.au and restarted nginx but it did
>> not make any difference.
>> 
>> Suggestions about how to improve the situation?
>> 
>> Thanks,
>> 
>> Phil.
>> --
>> Philip Rhoades
>> 
>> PO Box 896
>> Cowra  NSW  2794
>> Australia
>> E-mail:  phil at pricom.com.au
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170721/e28f6da0/attachment-0001.html>

From nginx-forum at forum.nginx.org  Fri Jul 21 11:02:07 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Fri, 21 Jul 2017 07:02:07 -0400
Subject: Buffering issues with nginx
In-Reply-To: <20170719201551.GK365@daoine.org>
References: <20170719201551.GK365@daoine.org>
Message-ID: <3285187829c36da8fd9b2fe87e3dbaa0.NginxMailingListEnglish@forum.nginx.org>

> Depending on the compromises you are willing to make, to accuracy or
> convenience, you may be able to come up with something good enough.

I have a more or less working solution. nginx breaks it and I'm trying to
figure out how to fix it.


> Yes. That is (part of) what a proxy does. Even without nginx as a
> reverse-proxy, your client might be talking through one or more proxy
> servers. You will never know whether your response got to the actual
> end client, without some extra verification step that only the end
> client does.

I don't care for the case if there are any other proxies, I care for bytes
that left my server. Specifically, bytes that left my server and were ACKed
by next point (either final user or some proxy in between). Verification
isn't an option.

> > When I updated some of these buffering
> > configs things improved, but still were failing with smaller uploads
that
> > are still fully buffered by nginx.

> proxy_buffers and proxy_buffer_size can be tuned (lowered, in this case,
> probably) to slow down nginx's receive-rate from your upstream.
> 
> If you can show one working configuration with a description of how
> it does not do what you want it to do, possibly someone can offer some
> advice on what to change.

I tried proxy_buffers off; and it didn't make a difference. I'm fairly
confident that it's a bug in nginx, or some "feature" that doesn't get
disabled with any configs.

Here's full config that I use:

    location / {
        proxy_pass http://localhost:80;
        #proxy_http_version 1.1;
        #proxy_http_version 1.0;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header Host $http_host;
        proxy_set_header X-NginX-Proxy true;
        #proxy_set_header Upgrade $http_upgrade;
        #proxy_set_header Connection 'upgrade';
        #proxy_set_header Connection 'close';
        proxy_buffering off;
        proxy_request_buffering off;
        #proxy_buffer_size 4k;
        #proxy_buffers 3 4k;
        proxy_no_cache 1;
        proxy_set_header Host $host;
        #proxy_cache_bypass $http_upgrade;
        proxy_hide_header X-Powered-By;
        proxy_max_temp_file_size 0;
    }

I run nginx on 8080, for testing, since it's not suitable for live use on 80
in my case and I'm trying to figure out how to fix it.
And here's why I believe that there is a bug.

In my case, I wrote test code on node side that serves some binary content.
I can control speed at what node serves this content. On receiving end (on
the other side of the planet) I use wget with --limite-rate. In the test
that I'm trying to fix I send 5MB from nodejs at 20KB/s speed, client that
requests that binary data reads it at 10KB/s. Obviously overall speed has to
be 10KB/s as it's limited by the client that requests the data.

What happens is that entire connection from nginx to node is closed after
node sends all data to nginx. Basically in my test 5MB will take
approximately 500s to deliver, but node gets tcp connection closed 255 s
from start (when there is still 250 more seconds to go and 2.5MB is still
stuck on nginx side). So, no matter what I do nginx totally breaks my
scenario, it does not obey any configs and still buffers 2.5MB

Just in case if nginx devs ever read here, I have 1.12.1 version on ubuntu.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275605#msg-275605


From vbart at nginx.com  Fri Jul 21 13:19:27 2017
From: vbart at nginx.com (Valentin V. Bartenev)
Date: Fri, 21 Jul 2017 16:19:27 +0300
Subject: Buffering issues with nginx
In-Reply-To: <3285187829c36da8fd9b2fe87e3dbaa0.NginxMailingListEnglish@forum.nginx.org>
References: <20170719201551.GK365@daoine.org>
 <3285187829c36da8fd9b2fe87e3dbaa0.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <2934599.InOR0em21l@vbart-workstation>

On Friday 21 July 2017 07:02:07 Dan34 wrote:
[..]
> I run nginx on 8080, for testing, since it's not suitable for live use on 80
> in my case and I'm trying to figure out how to fix it.
> And here's why I believe that there is a bug.
> 
> In my case, I wrote test code on node side that serves some binary content.
> I can control speed at what node serves this content. On receiving end (on
> the other side of the planet) I use wget with --limite-rate. In the test
> that I'm trying to fix I send 5MB from nodejs at 20KB/s speed, client that
> requests that binary data reads it at 10KB/s. Obviously overall speed has to
> be 10KB/s as it's limited by the client that requests the data.
> 
> What happens is that entire connection from nginx to node is closed after
> node sends all data to nginx. Basically in my test 5MB will take
> approximately 500s to deliver, but node gets tcp connection closed 255 s
> from start (when there is still 250 more seconds to go and 2.5MB is still
> stuck on nginx side). So, no matter what I do nginx totally breaks my
> scenario, it does not obey any configs and still buffers 2.5MB
> 
[..]

No buffering doesn't mean no buffers used at all.

In your scenario there are at least 5 buffers involved, and 4 of them
are in OS kernel.

Here's the list:

1. Write socket buffer in kernel on node.js side where node.js
   writes data.

2. Read socket buffer in kernel for node.js connection from what nginx
   reads data.

3. Heap memory buffer to that nginx reads data from kernel socket buffer
   (controlled by proxy_buffers and proxy_buffer_size directives).

   No buffering here means that nginx doesn't keep that data in buffers
   for some time, but writes it immediately to write socket buffer in kernel
   for client connection.

4. Write socket buffer in kernel for client connection where nginx
   writes data.

5. Read socket buffer in kernel for client connection from what wget
   reads data.


   wbr, Valentin V. Bartenev


From nginx-forum at forum.nginx.org  Fri Jul 21 17:45:51 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Fri, 21 Jul 2017 13:45:51 -0400
Subject: Buffering issues with nginx
In-Reply-To: <2934599.InOR0em21l@vbart-workstation>
References: <2934599.InOR0em21l@vbart-workstation>
Message-ID: <e6a708142833c9d6369ffb67cfe82d3d.NginxMailingListEnglish@forum.nginx.org>

Hello Valentin,

> 1. Write socket buffer in kernel on node.js side where node.js writes
data.

we can throw this out from equation, as I measure my end time by the event
when socket is closed on nodejs side, (I use http1.0 from nginx to node to
make it simple for this case).

> 2. Read socket buffer in kernel for node.js connection from what nginx
reads data.

SO_RCVBUF shouldn't be over 64KB by default. What does nginx use, is there a
config that controls it?.. still this shouldn't be a big issue, I'm fine if
there is such a constant buf.


> 3. Heap memory buffer to that nginx reads data from kernel socket buffer
(controlled by proxy_buffers
> and proxy_buffer_size directives).
> 
> No buffering here means that nginx doesn't keep that data in buffers
> for some time, but writes it immediately to write socket buffer in kernel
> for client connection.

I'm trying to configure these to be skipped or used to minimum. E.g. I don't
wan any data to be held in these.

> 4. Write socket buffer in kernel for client connection where nginx writes
data.

SO_SNDBUF shouldn't be over 64KB by default, perhaps nginx changes it as
well. What's the value that nginx uses and is there a config that controls
it?

> 5. Read socket buffer in kernel for client connection from what wget reads
data.

We can throw this out from equation, we may assume these aren't used, as for
my test I use final time when wget finishes and prints stats. There is
obviously highly unlikely chance that wget actually reads data twice faster
from network, but shows slower speed in it's cli results and "waits" for
data even though it's already received and is in local buffers. This is
totally dumb and I don't think this might happen, but I could check with
wireshark just in case.


In short, these could affect my case: SO_RCVBUF, SO_SNDBUF on nginx side and
whatever buffering nginx uses for handling data. I run that same test with
25MB data and I got totally identical result: 12.5MB was buffered on nginx
side. That stuff that could affect my case cannot really add up to 12.5MB
and 10 minute of time.
There is a wild possibility that tcp window scaling resulted in some huge
window on node->nginx side and ended up storing that 12MB in tcp window
itself but i'm not sure if TCP window should be accounted into these
SO_RCVBUF or that RCVBUF is extra data on top of internals of TCP.

So,.. any ideas how come nginx ends up buffering 12.5MB data?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275608#msg-275608


From vbart at nginx.com  Fri Jul 21 21:05:08 2017
From: vbart at nginx.com (Valentin V. Bartenev)
Date: Sat, 22 Jul 2017 00:05:08 +0300
Subject: Buffering issues with nginx
In-Reply-To: <e6a708142833c9d6369ffb67cfe82d3d.NginxMailingListEnglish@forum.nginx.org>
References: <2934599.InOR0em21l@vbart-workstation>
 <e6a708142833c9d6369ffb67cfe82d3d.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <2000492.Bcq8h6jK24@vbart-laptop>

On Friday 21 July 2017 13:45:51 Dan34 wrote:
[..]
> 
> In short, these could affect my case: SO_RCVBUF, SO_SNDBUF on nginx side and
> whatever buffering nginx uses for handling data. I run that same test with
> 25MB data and I got totally identical result: 12.5MB was buffered on nginx
> side. That stuff that could affect my case cannot really add up to 12.5MB
> and 10 minute of time.
> There is a wild possibility that tcp window scaling resulted in some huge
> window on node->nginx side and ended up storing that 12MB in tcp window
> itself but i'm not sure if TCP window should be accounted into these
> SO_RCVBUF or that RCVBUF is extra data on top of internals of TCP.
> 

nginx doesn't set SO_RCVBUF/SO_SNDBUF by default, which usually means that 
kernel will use system defaults and auto-scalling. 


> So,.. any ideas how come nginx ends up buffering 12.5MB data?
> 

You should check tcpdump (or wireshark) to see where actually 12.5MB
of data have been stuck.

  wbr, Valentin V. Bartenev


From nginx-forum at forum.nginx.org  Sat Jul 22 06:00:37 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Sat, 22 Jul 2017 02:00:37 -0400
Subject: Buffering issues with nginx
In-Reply-To: <2000492.Bcq8h6jK24@vbart-laptop>
References: <2000492.Bcq8h6jK24@vbart-laptop>
Message-ID: <fba1f2d47425467a6fdc36f0484f12d1.NginxMailingListEnglish@forum.nginx.org>

> You should check tcpdump (or wireshark) to see where actually 12.5MB
> of data have been stuck.

Wireshark confirms my assumption. All the data is buffered by nginx. More
over, I see some buggy behavior, and I've seen that happen quite often.

This is localhost tcp screenshot: http://i.imgur.com/9Rz6Acs.png

You can see that after 1327 seconds nginx ACKed 18.5MB (which is 13.9MB/s).
node actually writes at 20MB/s to the socket, node will internally buffer
all unset data. At this point node stops sending any data and in 30 seconds
nginx closes socket (at 1399s).
Then nginx goes on to deliver all the data that it got buffered and when it
finishes sending 18.5M that it got from node before closing TCP connection
it also closes connection to wget. Wget simply restarts file transfer with a
new HTTP range request to download starting from 18.5MB, at this point you
can see on this screenshot that around 1820sec nginx sends new GET request
to node (that's the range get).


Here you can see outgoing packets from node around the same time when nginx
closed socket to node at 1399sec: http://i.imgur.com/pdnDIFS.png
 You can see that by this time remote (wget) ACKed exactly 14MB (as I run
wget with 10KB/s rate limit).

So, without any tcp buffers involved nginx does buffer like 5MB of data.
Moreover, when I review node->nginx packet capture, nginx clearly was
reading full speed (20KB/s speed limit on node side) and at some point
perhaps something triggered nginx to stop reading fullspeed. This happened
at 391sec, at which point nginx ACKed 7.8MB (which is exactly 20KB/s). At
the same time wget ACKed only 4MB, at this point nginx was buffering around
4MB and started to slow down read speed from node.

So, configs do not have any effect. What else should I check? Effectively,
in this scenario nginx should also read from node at 10KB/s (plus some fixed
buffer) and this doesn't seem to work properly in nginx.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275611#msg-275611


From nginx-forum at forum.nginx.org  Sat Jul 22 15:10:02 2017
From: nginx-forum at forum.nginx.org (mohini)
Date: Sat, 22 Jul 2017 11:10:02 -0400
Subject: Filter installation for WebSocket protocol in Nginx.
Message-ID: <8bca4889102081899abccc6d3a9129c6.NginxMailingListEnglish@forum.nginx.org>

I have an working nginx module (http_content_Modify_module) that acts as a
proxy and manipulates some specific http data coming to the proxy before
sending it to the upstream server.Used nginx1.6.2 for this and was
implemented by installing filters in the "post-configuration" setup.
Now I need to move from http to websocket protocol support. 
By changing the configuration file to support "Upgrade" header,
http_content_Modify_module  was able to talk to upstream webSocket server. 
However I am not sure if nginx supports filter installation for websockets
like it does for http so that I can manipulate/change data coming over the
websocket
before sending it to upstream.
Pls advice.
Thanks,
Mohini.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275612,275612#msg-275612


From nginx-forum at forum.nginx.org  Sat Jul 22 18:23:01 2017
From: nginx-forum at forum.nginx.org (DonaldCock)
Date: Sat, 22 Jul 2017 14:23:01 -0400
Subject: try_files and gzip_static
In-Reply-To: <bf4395d145f78bdff3a928684d61e0de.NginxMailingListEnglish@forum.nginx.org>
References: <bf4395d145f78bdff3a928684d61e0de.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <67e6bcb73281949ce667df584ff5a87a.NginxMailingListEnglish@forum.nginx.org>

Still have been working on it, and I can tell gunzip works just fine even
without uncompressed files.
But try_files and rewrite just don't acknowledge it at all. Is this a bug or
is it intended?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275539,275613#msg-275613


From geraldjunkmail at gmail.com  Sun Jul 23 01:45:21 2017
From: geraldjunkmail at gmail.com (Gee Bunny)
Date: Sat, 22 Jul 2017 21:45:21 -0400
Subject: nginx 1.13 binaries
Message-ID: <CAFE-bHsDMQhoEAiuhMGftbrtiN-KUN8u4QD4OWxg2wQedJ_6ig@mail.gmail.com>

Are there plans to add nginx 1.13 to any of the yum repositories?

Like:

http://nginx.org/packages/centos/6/x86_64/RPMS/

Some of us would like to run the latest version of NGINX with TLS1.3
support, without having to compile from source.

Are there any plans or ETA to have nginx 1.13 aded to the repos within a
timely fashion? Or is the only option to get TLS1.3 support to compile from
source for the forseeable future?

Thanks

P.S. Awesome work on NGINX- best disruptive web-server tech to hit the
scene in 20+ years.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170722/6476d99d/attachment.html>

From kaushalshriyan at gmail.com  Sun Jul 23 17:32:06 2017
From: kaushalshriyan at gmail.com (Kaushal Shriyan)
Date: Sun, 23 Jul 2017 23:02:06 +0530
Subject: Naxsi Nginx security module in nginx webserver
Message-ID: <CAD7Ssm8Q8RO-MZ8PN3ZrRfGpDjg-Gx82HLhumKpdZd3C6Qdanw@mail.gmail.com>

Hi,

I will appreciate if somebody can pitch in for help for earlier post to
this mailing list.

Thanks in Advance.

Regards,

Kaushal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170723/3630261e/attachment.html>

From francis at daoine.org  Sun Jul 23 18:18:43 2017
From: francis at daoine.org (Francis Daly)
Date: Sun, 23 Jul 2017 19:18:43 +0100
Subject: nginx 1.13 binaries
In-Reply-To: <CAFE-bHsDMQhoEAiuhMGftbrtiN-KUN8u4QD4OWxg2wQedJ_6ig@mail.gmail.com>
References: <CAFE-bHsDMQhoEAiuhMGftbrtiN-KUN8u4QD4OWxg2wQedJ_6ig@mail.gmail.com>
Message-ID: <20170723181843.GN365@daoine.org>

On Sat, Jul 22, 2017 at 09:45:21PM -0400, Gee Bunny wrote:

Hi there,

> Are there plans to add nginx 1.13 to any of the yum repositories?

My guess: they are already there.

See http://nginx.org/en/linux_packages.html

> Like:
> 
> http://nginx.org/packages/centos/6/x86_64/RPMS/

"Pre-Built Packages for Stable version".

If you want the non-stable version, see the section called

"Pre-Built Packages for Mainline version"

and accept the different meaning of the version name.

> Are there any plans or ETA to have nginx 1.13 aded to the repos within a
> timely fashion?

Even numbers are "Stable"; odd numbers are "Mainline". So 1.13 is not
going to appear in the "Stable" repository.

	f
-- 
Francis Daly        francis at daoine.org

From johnhedge at gmail.com  Mon Jul 24 03:32:02 2017
From: johnhedge at gmail.com (John)
Date: Mon, 24 Jul 2017 13:32:02 +1000
Subject: Matt Wilcox's Setting up a (reasonably) secure home web-server with
 Raspberry Pi 'howto'
Message-ID: <CAHaZY7itg5=D_puo=fV+xVQEo6yGbVwdfHSFc1H4HfH0KWhp9w@mail.gmail.com>

I've been implementing (my 4th iteration) Matt Wilcox's Setting up a
(reasonably) secure home web-server with Raspberry Pi
<https://mattwilcox.net/web-development/setting-up-a-secure-home-web-server-with-raspberry-pi/>
'howto' but I keep getting a 404 on my local network (both html and php)
when trying to connect to my RPi nginx web server.

My hosts file is:

127.0.0.1 localhost
10.0.0.46 hedge.local

My hedge config file is:

server {

        listen 80 default_server;

        root /websites/hedge/www;

        index index.html index.htm index.php;

        server_name hedge.local hedge.com.au www.hedge.com.au;

        location / {

                # First attempt to serve request as file, then

                # as directory, then fall back to displaying a 404.

                try_files $uri $uri/ =404;

                # try_files $url $url/ /index.php?q=$url&$args;

        }

        error_log /websites/hedge/logs/error.log error;

        access_log /websites/hedge/logs/access.log;

        location ~ [^/].php(/|$) {

        fastcgi_split_path_info ^(.+?.php)(/.*)$;

        if (!-f $document_root$fastcgi_script_name) {

        return 404;

        }

        fastcgi_pass unix:/var/run/php5-fpm.sock;

        fastcgi_index index.php;

        include fastcgi_params;

        }

}

The nginx config file is:

user www-data;

worker_processes 2;

pid /run/nginx.pid;

events {

        worker_connections 768;

        # multi_accept on;

}

http {

        ##

        # Basic Settings

        ##

        sendfile on;

        tcp_nopush on;

        tcp_nodelay on;

        # keepalive_timeout 65;

        types_hash_max_size 2048;

        server_tokens off;

        server_names_hash_bucket_size 64;

        # server_name_in_redirect off;

        include /etc/nginx/mime.types;

        default_type application/octet-stream;

        ##

        # SSL Settings

        ##

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE

        ssl_prefer_server_ciphers on;

        ##

        # Logging Settings

        ##

        access_log /var/log/nginx/access.log;

        error_log /var/log/nginx/error.log;

        ##

        # Gzip Settings

        ##

        gzip on;

        gzip_disable "msie6";

        gzip_min_length 1100;

        gzip_vary on;

        gzip_proxied any;

        gzip_comp_level 6;

        gzip_buffers 16 8k;

        gzip_http_version 1.1;

        gzip_types text/plain text/css applciation/json
application/x-javascript text/xml application/xml application/rss+xml
text/javascript images/svg+xml application/x-font-ttf font/opentype
application/vnd.ms-fontobject;

        ##

        # Virtual Host Configs

        ##

        include /etc/nginx/conf.d/*.conf;

        include /etc/nginx/sites-enabled/*;

        include /etc/nginx/perfect-forward-secrecy.conf;

        client_header_timeout 10;

        client_body_timeout   10;

        keepalive_timeout     10 10;

        send_timeout          10;

}

The 'hedge' error.log is empty. The 'hedge' access.log has:

10.0.0.206 - - [24/Jul/2017:06:29:42 +1000] "GET /index.html HTTP/1.1" 404
564 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"

10.0.0.206 - - [24/Jul/2017:06:31:04 +1000] "GET /index.php HTTP/1.1" 404
564 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
(KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"

The /var/log/nginx access.log is empty. The error.log has:

2017/07/23 16:27:29 [notice] 17471#0: signal process started

2017/07/23 16:36:20 [notice] 17519#0: signal process started

2017/07/24 12:36:34 [notice] 20858#0: signal process started

I hope I've thought of everything but if not please let me know what else
you require to get me up and going, please.

TIA

John
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170724/3705b131/attachment-0001.html>

From johnhedge at gmail.com  Mon Jul 24 06:01:02 2017
From: johnhedge at gmail.com (John)
Date: Mon, 24 Jul 2017 16:01:02 +1000
Subject: Matt Wilcox's Setting up a (reasonably) secure home web-server
 with Raspberry Pi 'howto'
In-Reply-To: <CAHaZY7itg5=D_puo=fV+xVQEo6yGbVwdfHSFc1H4HfH0KWhp9w@mail.gmail.com>
References: <CAHaZY7itg5=D_puo=fV+xVQEo6yGbVwdfHSFc1H4HfH0KWhp9w@mail.gmail.com>
Message-ID: <CAHaZY7iaxbytAVhPU4_AHFDWDLymupLZMEGYLs+M+Z1tpBEiuw@mail.gmail.com>

It seems I've solved it myself. I wish I knew how!

John

On 24 July 2017 at 13:32, John <johnhedge at gmail.com> wrote:

> I've been implementing (my 4th iteration) Matt Wilcox's Setting up a
> (reasonably) secure home web-server with Raspberry Pi
> <https://mattwilcox.net/web-development/setting-up-a-secure-home-web-server-with-raspberry-pi/>
> 'howto' but I keep getting a 404 on my local network (both html and php)
> when trying to connect to my RPi nginx web server.
>
> My hosts file is:
>
> 127.0.0.1 localhost
> 10.0.0.46 hedge.local
>
> My hedge config file is:
>
> server {
>
>         listen 80 default_server;
>
>         root /websites/hedge/www;
>
>         index index.html index.htm index.php;
>
>         server_name hedge.local hedge.com.au www.hedge.com.au;
>
>         location / {
>
>                 # First attempt to serve request as file, then
>
>                 # as directory, then fall back to displaying a 404.
>
>                 try_files $uri $uri/ =404;
>
>                 # try_files $url $url/ /index.php?q=$url&$args;
>
>         }
>
>         error_log /websites/hedge/logs/error.log error;
>
>         access_log /websites/hedge/logs/access.log;
>
>         location ~ [^/].php(/|$) {
>
>         fastcgi_split_path_info ^(.+?.php)(/.*)$;
>
>         if (!-f $document_root$fastcgi_script_name) {
>
>         return 404;
>
>         }
>
>         fastcgi_pass unix:/var/run/php5-fpm.sock;
>
>         fastcgi_index index.php;
>
>         include fastcgi_params;
>
>         }
>
> }
>
> The nginx config file is:
>
> user www-data;
>
> worker_processes 2;
>
> pid /run/nginx.pid;
>
> events {
>
>         worker_connections 768;
>
>         # multi_accept on;
>
> }
>
> http {
>
>         ##
>
>         # Basic Settings
>
>         ##
>
>         sendfile on;
>
>         tcp_nopush on;
>
>         tcp_nodelay on;
>
>         # keepalive_timeout 65;
>
>         types_hash_max_size 2048;
>
>         server_tokens off;
>
>         server_names_hash_bucket_size 64;
>
>         # server_name_in_redirect off;
>
>         include /etc/nginx/mime.types;
>
>         default_type application/octet-stream;
>
>         ##
>
>         # SSL Settings
>
>         ##
>
>         ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
>
>         ssl_prefer_server_ciphers on;
>
>         ##
>
>         # Logging Settings
>
>         ##
>
>         access_log /var/log/nginx/access.log;
>
>         error_log /var/log/nginx/error.log;
>
>         ##
>
>         # Gzip Settings
>
>         ##
>
>         gzip on;
>
>         gzip_disable "msie6";
>
>         gzip_min_length 1100;
>
>         gzip_vary on;
>
>         gzip_proxied any;
>
>         gzip_comp_level 6;
>
>         gzip_buffers 16 8k;
>
>         gzip_http_version 1.1;
>
>         gzip_types text/plain text/css applciation/json
> application/x-javascript text/xml application/xml application/rss+xml
> text/javascript images/svg+xml application/x-font-ttf font/opentype
> application/vnd.ms-fontobject;
>
>         ##
>
>         # Virtual Host Configs
>
>         ##
>
>         include /etc/nginx/conf.d/*.conf;
>
>         include /etc/nginx/sites-enabled/*;
>
>         include /etc/nginx/perfect-forward-secrecy.conf;
>
>         client_header_timeout 10;
>
>         client_body_timeout   10;
>
>         keepalive_timeout     10 10;
>
>         send_timeout          10;
>
> }
>
> The 'hedge' error.log is empty. The 'hedge' access.log has:
>
> 10.0.0.206 - - [24/Jul/2017:06:29:42 +1000] "GET /index.html HTTP/1.1" 404
> 564 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
> (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
>
> 10.0.0.206 - - [24/Jul/2017:06:31:04 +1000] "GET /index.php HTTP/1.1" 404
> 564 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36
> (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"
>
> The /var/log/nginx access.log is empty. The error.log has:
>
> 2017/07/23 16:27:29 [notice] 17471#0: signal process started
>
> 2017/07/23 16:36:20 [notice] 17519#0: signal process started
>
> 2017/07/24 12:36:34 [notice] 20858#0: signal process started
>
> I hope I've thought of everything but if not please let me know what else
> you require to get me up and going, please.
>
> TIA
>
> John
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170724/c46f40c2/attachment-0001.html>

From peter_booth at me.com  Mon Jul 24 07:23:38 2017
From: peter_booth at me.com (Peter Booth)
Date: Mon, 24 Jul 2017 03:23:38 -0400
Subject: Specify a Vary: Accept-Encoding header
In-Reply-To: <AAD91198-92EA-4B43-B5F4-8ECB36A08A77@me.com>
References: <dd61248537d0b68aa2a79cb6e82ffc7b@abic.net.au>
 <CAKWQeyicxxY2tmsCVFt2sHzewu_JAO0i3YAK+THzpGfsZ35k8g@mail.gmail.com>
 <AAD91198-92EA-4B43-B5F4-8ECB36A08A77@me.com>
Message-ID: <4EE380D2-E51E-418A-AEC6-410ECC30E40A@me.com>

Phillip,

Right now this Rails website is almost too slow to tune, and so
 you will need to make some radical changes that you might later choose to undo.

You should run the rails app in production mode, which, by default will cache everything. 
That should give you th breathing room needs dto run other tools.

Peter


> On Jul 21, 2017, at 6:58 AM, Peter Booth <peter_booth at me.com> wrote:
> 
> It looks as if the static content is being served by the Rails asset pipeline rather than directly by nginx 
> and the impact is enormous. It took 25s for the base page - but it also took 
> another 25s for the http://cryonics.org.au/assets/application.js <http://cryonics.org.au/assets/application.js> resource
> and another 20s for http://cryonics.org.au/assets/bg.gif <http://cryonics.org.au/assets/bg.gif>
> and another 20s for http://cryonics.org.au/assets/front.jpg <http://cryonics.org.au/assets/front.jpg>
> 
> See https://www.webpagetest.org/result/170720_M2_1T8G/1/details/#waterfall_view_step1 <https://www.webpagetest.org/result/170720_M2_1T8G/1/details/#waterfall_view_step1> for a detailed breakdown.
> 
> 
> In some ways Rails is the antithesis of nginx. Whilst nginx is fast out of the box with default 
> configuration, rails is stunningly slow without a lot of tweaking. There are many tools that 
> can help with Ruby / Rails tuning.
> 
> The one?s that I find most useful are New Relic - for its visualization of how slow vs fast 
> requests vary, rack-mini-profiler and the bullet gem.  I have heard good things about 
> Skylight and about Scout but haven?t tried either myself. 
> 
> Good luck,
> 
> Peter
> 
> 
>> On Jul 20, 2017, at 8:45 AM, Richard Stanway <r1ch+nginx at teamliquid.net <mailto:r1ch+nginx at teamliquid.net>> wrote:
>> 
>> The issue is not with your page size or gzip (or anything nginx
>> related actually). Your Rails backend is generating the content far
>> too slow. You should investigate why your backend is so slow.
>> 
>>            time_namelookup:  0.004209
>>               time_connect:  0.241082
>>            time_appconnect:  0.000000
>>           time_pretransfer:  0.241121
>>              time_redirect:  0.000000
>>         time_starttransfer:  20.519778
>>                            ----------
>>                 time_total:  20.568794
>> 
>> 
>> On Thu, Jul 20, 2017 at 5:44 AM, Philip Rhoades <phil at pricom.com.au <mailto:phil at pricom.com.au>> wrote:
>>> People,
>>> 
>>> I have moved my (very low hit) web sites from a Digital Ocean server to my
>>> own Fedora 25 x86-64 server with 8GB RAM and an ADSL2+ upload speed of only
>>> about 1Mbit/sec.
>>> 
>>> The plain HTML and Jekyll sites response times are not too bad but the Rails
>>> sites are very slow.
>>> 
>>> Using:
>>> 
>>>  https://tools.pingdom.com <https://tools.pingdom.com/>
>>> 
>>> takes more than a minute to test one of my Rails sites:
>>> 
>>>  http://cryonics.org.au <http://cryonics.org.au/>
>>> 
>>> so it appears that I need to do something for all the Rails sites at least
>>> but ALL sites report a value of 50 or less for the condition in the Subject
>>> of this mail - I tried putting:
>>> 
>>>  gzip on;
>>>  gzip_min_length  1100;
>>>  gzip_buffers  4 32k;
>>>  gzip_types    text/plain application/x-javascript text/xml text/css;
>>>  gzip_vary on;
>>> 
>>> in the nginx server conf for cryonics.org.au <http://cryonics.org.au/> and restarted nginx but it did
>>> not make any difference.
>>> 
>>> Suggestions about how to improve the situation?
>>> 
>>> Thanks,
>>> 
>>> Phil.
>>> --
>>> Philip Rhoades
>>> 
>>> PO Box 896
>>> Cowra  NSW  2794
>>> Australia
>>> E-mail:  phil at pricom.com.au <mailto:phil at pricom.com.au>
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170724/d3355e2c/attachment.html>

From tkadm30 at yandex.com  Mon Jul 24 10:13:28 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Mon, 24 Jul 2017 06:13:28 -0400
Subject: How to rate-limit jorgee malware scanner?
Message-ID: <581b79a9-4634-c20e-6539-d66375591d2b@yandex.com>

Hi,

The Jorgee malware scanner is creating a lot of activity on my site. I 
would like to rate-limit its connections to nginx based on the 
User-Agent, since blocking all IP addresses with iptables seems 
impossible. Is their a quick way of doing this ?

Thank you in advance ,

E

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From jonathan at dabnis.com  Mon Jul 24 11:52:26 2017
From: jonathan at dabnis.com (Jonathan Parker)
Date: Mon, 24 Jul 2017 12:52:26 +0100
Subject: post_action
Message-ID: <023c01d30473$523a9f00$f6afdd00$@dabnis.com>

Hi All,

As post_action is no longer documented could someone tell me if post_action
will be dropped from future releases, should I use it?

 

Thanks.

Jonathan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170724/57ca3a45/attachment.html>

From lists at lazygranch.com  Mon Jul 24 12:09:39 2017
From: lists at lazygranch.com (Gary Sellani)
Date: Mon, 24 Jul 2017 05:09:39 -0700
Subject: How to rate-limit jorgee malware scanner?
In-Reply-To: <581b79a9-4634-c20e-6539-d66375591d2b@yandex.com>
Message-ID: <mailman.53.1500898201.94785.nginx@nginx.org>

I just detect the use agent and return 444, but every attempt to get a file will show up in your access.log. 

https://www.buildersociety.com/threads/block-unwanted-bots-on-apache-nginx-constantly-updated.1898/

I get two or three jorgee "sessions" a day. They tend not to use the domain name but reference your server by IP, so there might be some better blocking scheme. 

? Original Message ?
From: tkadm30 at yandex.com
Sent: July 24, 2017 3:14 AM
To: nginx at nginx.org
Reply-to: nginx at nginx.org
Subject: How to rate-limit jorgee malware scanner?

Hi,

The Jorgee malware scanner is creating a lot of activity on my site. I 
would like to rate-limit its connections to nginx based on the 
User-Agent, since blocking all IP addresses with iptables seems 
impossible. Is their a quick way of doing this ?

Thank you in advance ,

E

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/

_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx

From zchao1995 at gmail.com  Mon Jul 24 12:12:06 2017
From: zchao1995 at gmail.com (Zhang Chao)
Date: Mon, 24 Jul 2017 05:12:06 -0700
Subject: How to rate-limit jorgee malware scanner?
In-Reply-To: <20170724121001.A9E902C56AAB@mail.nginx.com>
References: <20170724121001.A9E902C56AAB@mail.nginx.com>
Message-ID: <CAPr95Bh8tXhmNqTeoMV7oGaVO9k7Rw9xSjzec-sy7tiFZ1SK1w@mail.gmail.com>

Hi!

Nginx carries with the limit_req_module
<http://nginx.org/en/docs/http/ngx_http_limit_req_module.html>. I think it
is a good helper.



On 24 July 2017 at 20:10:05, Gary Sellani (lists at lazygranch.com) wrote:

I just detect the use agent and return 444, but every attempt to get a file
will show up in your access.log.

https://www.buildersociety.com/threads/block-unwanted-bots-on-apache-nginx-constantly-updated.1898/

I get two or three jorgee "sessions" a day. They tend not to use the domain
name but reference your server by IP, so there might be some better
blocking scheme.

  Original Message
From: tkadm30 at yandex.com
Sent: July 24, 2017 3:14 AM
To: nginx at nginx.org
Reply-to: nginx at nginx.org
Subject: How to rate-limit jorgee malware scanner?

Hi,

The Jorgee malware scanner is creating a lot of activity on my site. I
would like to rate-limit its connections to nginx based on the
User-Agent, since blocking all IP addresses with iptables seems
impossible. Is their a quick way of doing this ?

Thank you in advance ,

E

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/

_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170724/60fbe15a/attachment.html>

From lists at viaduct-productions.com  Mon Jul 24 12:20:48 2017
From: lists at viaduct-productions.com (Viaduct Lists)
Date: Mon, 24 Jul 2017 08:20:48 -0400
Subject: Matt Wilcox's Setting up a (reasonably) secure home web-server
 with Raspberry Pi 'howto'
In-Reply-To: <CAHaZY7itg5=D_puo=fV+xVQEo6yGbVwdfHSFc1H4HfH0KWhp9w@mail.gmail.com>
References: <CAHaZY7itg5=D_puo=fV+xVQEo6yGbVwdfHSFc1H4HfH0KWhp9w@mail.gmail.com>
Message-ID: <7DA6D80F-2295-4C6F-A9DD-28376022230D@viaduct-productions.com>

I?ve done the same.

Try listen port 8080, as anything < port 1024 needs to run as root.  Then in your url, enter hedge.local:8080.  Shove hedge.local into your /etc/hosts file and point to the proper IP.  But you need to enter the port number in that url to fetch it on the LAN.  


> On Jul 23, 2017, at 11:32 PM, John <johnhedge at gmail.com> wrote:
> 
> I hope I've thought of everything but if not please let me know what else you require to get me up and going, please.


_____________
Rich in Toronto @ VP







From tkadm30 at yandex.com  Mon Jul 24 14:06:24 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Mon, 24 Jul 2017 10:06:24 -0400
Subject: How to rate-limit jorgee malware scanner?
In-Reply-To: <CAPr95Bh8tXhmNqTeoMV7oGaVO9k7Rw9xSjzec-sy7tiFZ1SK1w@mail.gmail.com>
References: <20170724121001.A9E902C56AAB@mail.nginx.com>
 <CAPr95Bh8tXhmNqTeoMV7oGaVO9k7Rw9xSjzec-sy7tiFZ1SK1w@mail.gmail.com>
Message-ID: <59486ede-e034-805d-07a9-ad4a188adf6e@yandex.com>

Hi all,

Unfortunately, its impossible to use limit_req within the http location 
using a "if" statement like so:

http {

limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;

if ($http_user_agent ~* (Jorgee)) {

limit_req zone=one burst=5;

return 403;

}

}


As a workaround I use limit_req within a location to prevent my uwsgi 
app for being abused.

Cheers,
E

Le 2017-07-24 ? 08:12, Zhang Chao a ?crit :
>
> Hi!
>
> Nginx carries with the limit_req_module 
> <http://nginx.org/en/docs/http/ngx_http_limit_req_module.html>. I 
> think it is a good helper.
>
>
>
> On 24 July 2017 at 20:10:05, Gary Sellani (lists at lazygranch.com 
> <mailto:lists at lazygranch.com>) wrote:
>
>> I just detect the use agent and return 444, but every attempt to get 
>> a file will show up in your access.log.
>>
>> https://www.buildersociety.com/threads/block-unwanted-bots-on-apache-nginx-constantly-updated.1898/
>>
>> I get two or three jorgee "sessions" a day. They tend not to use the 
>> domain name but reference your server by IP, so there might be some 
>> better blocking scheme.
>>
>>   Original Message
>> From: tkadm30 at yandex.com <mailto:tkadm30 at yandex.com>
>> Sent: July 24, 2017 3:14 AM
>> To: nginx at nginx.org <mailto:nginx at nginx.org>
>> Reply-to: nginx at nginx.org <mailto:nginx at nginx.org>
>> Subject: How to rate-limit jorgee malware scanner?
>>
>> Hi,
>>
>> The Jorgee malware scanner is creating a lot of activity on my site. I
>> would like to rate-limit its connections to nginx based on the
>> User-Agent, since blocking all IP addresses with iptables seems
>> impossible. Is their a quick way of doing this ?
>>
>> Thank you in advance ,
>>
>> E
>>
>> -- 
>> Etienne Robillard
>> tkadm30 at yandex.com <mailto:tkadm30 at yandex.com>
>> http://www.isotopesoftware.ca/
>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170724/6b034b73/attachment-0001.html>

From lists at lazygranch.com  Mon Jul 24 14:13:42 2017
From: lists at lazygranch.com (Gary Sellani)
Date: Mon, 24 Jul 2017 07:13:42 -0700
Subject: How to rate-limit jorgee malware scanner?
In-Reply-To: <CAPr95Bh8tXhmNqTeoMV7oGaVO9k7Rw9xSjzec-sy7tiFZ1SK1w@mail.gmail.com>
Message-ID: <mailman.54.1500905638.94785.nginx@nginx.org>

An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170724/7f122948/attachment.html>

From nginx-forum at forum.nginx.org  Mon Jul 24 14:28:35 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Mon, 24 Jul 2017 10:28:35 -0400
Subject: Buffering issues with nginx
In-Reply-To: <fba1f2d47425467a6fdc36f0484f12d1.NginxMailingListEnglish@forum.nginx.org>
References: <2000492.Bcq8h6jK24@vbart-laptop>
 <fba1f2d47425467a6fdc36f0484f12d1.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <76e928ff1110d701859a0ffbb5df3df6.NginxMailingListEnglish@forum.nginx.org>

I wrote my own proxy and it appears that the data is all stuck in socket
buffers. If SNDBUF isn't set, then OS will resize it if you try to write
more data than remote can accept. Overall, in my tests I see that this
buffer grows to 2.5MB and in wireshark I see that difference grows up to
5MB. As docs from SO_SNDBUF state, actual internal value is usually twice
larger than what SO_SNDBUF ports then it kind of starts to make sense where
this max 5MB difference is wasted.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275635#msg-275635


From ryan at rbftpnetworks.com  Mon Jul 24 15:20:26 2017
From: ryan at rbftpnetworks.com (Ryan Barclay)
Date: Mon, 24 Jul 2017 16:20:26 +0100
Subject: Disable NGINX caching 304 Responses from Origin Server
Message-ID: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>

We have a pretty simple setup with NGINX sitting on the front and a 
backend server (on a separate physical server) that provides the content.

Nginx then caches content based on the EXPIRES and Cache-Control headers 
set by the origin server.

We noticed that NGINX was not issuing 304 headers to images that were 
not in the local NGINX cache when the If-Modified-Since header was sent 
by the client. Instead, it would issue a 200 with the full data file.

To fix this, we applied:
proxy_set_header If-Modified-Since $http_if_modified_since

So then the If-Modified-Since header was passed to the backend and of 
course, it returned correctly with the 304 header - great.

But what we noticed was that NGINX would cache this 304 response and 
deliver future responses as 304 to clients even without the 
If-Modified-Since header.

How can we disable caching of 304 responses and fix this issue?

Thank you for your help, suggestions, and tips in advance.

From nginx-forum at forum.nginx.org  Mon Jul 24 16:24:43 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Mon, 24 Jul 2017 12:24:43 -0400
Subject: Buffering issues with nginx
In-Reply-To: <76e928ff1110d701859a0ffbb5df3df6.NginxMailingListEnglish@forum.nginx.org>
References: <2000492.Bcq8h6jK24@vbart-laptop>
 <fba1f2d47425467a6fdc36f0484f12d1.NginxMailingListEnglish@forum.nginx.org>
 <76e928ff1110d701859a0ffbb5df3df6.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <be3e2c901181dbe91e0214d8a1dec93c.NginxMailingListEnglish@forum.nginx.org>

I did some logs on my proxy test and compared results with wireshark trace
at some random point in time (t=511sec)
And numbers match exactly between logs and wireshark. 

This is a log line from my test proxy:
time: 511s, bytesSent:5571760, down:{ SND:478720 OUTQ:280480 } up:{
RCV:5109117 INQ:3837047 }
SND is SNDBUF, RCV is RCVBUF, OUTQ is SIOCOUTQ, INQ is SIOCINQ/FIONREAD.
down is the link between wget and proxy, and up is the link from proxy to
node on localhost.

At the same time in wireshark the up link has 9409056 bytes ACKed, down link
has 5291529 bytes ACKed, or 9409056-5291529=4117527 bytes got accumulated
inside proxy process. This is exactly the same number of bytes that's stuck
in socket queues on up+down links (280480 + 3837047 =4117527).

>From logs it also looks like when INQ and OUTQ reach values close to SND/RCV
then in wireshark I see TCP packets with window size = 0 to stop any
transmission.
Perhaps, if I set SND+RCV to 64K then total buffering should not exceed
128KB inside proxy buffers.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275640#msg-275640


From francis at daoine.org  Mon Jul 24 19:05:55 2017
From: francis at daoine.org (Francis Daly)
Date: Mon, 24 Jul 2017 20:05:55 +0100
Subject: Buffering issues with nginx
In-Reply-To: <be3e2c901181dbe91e0214d8a1dec93c.NginxMailingListEnglish@forum.nginx.org>
References: <2000492.Bcq8h6jK24@vbart-laptop>
 <fba1f2d47425467a6fdc36f0484f12d1.NginxMailingListEnglish@forum.nginx.org>
 <76e928ff1110d701859a0ffbb5df3df6.NginxMailingListEnglish@forum.nginx.org>
 <be3e2c901181dbe91e0214d8a1dec93c.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170724190555.GO365@daoine.org>

On Mon, Jul 24, 2017 at 12:24:43PM -0400, Dan34 wrote:

Hi there,

> I did some logs on my proxy test and compared results with wireshark trace
> at some random point in time (t=511sec)
> And numbers match exactly between logs and wireshark. 

Out of interest -- are these buffers especially big because "localhost"
is involved?

As in: if you do a direct wget-to-node connection on 127.0.0.1, do you
see the same problem as with nginx-to-node?

Or: if you make node and nginx be on different machines and do
wget-to-nginx, does that remove (or at least shrink) the problem?


It's not ideal, but if it's a pure-config way to get "total buffers"
to more closely match the desired buffers, it might be adequate.

Cheers,

	f
-- 
Francis Daly        francis at daoine.org

From johnhedge at gmail.com  Mon Jul 24 20:26:11 2017
From: johnhedge at gmail.com (John)
Date: Tue, 25 Jul 2017 06:26:11 +1000
Subject: Matt Wilcox's Setting up a (reasonably) secure home web-server
 with Raspberry Pi 'howto'
In-Reply-To: <7DA6D80F-2295-4C6F-A9DD-28376022230D@viaduct-productions.com>
References: <CAHaZY7itg5=D_puo=fV+xVQEo6yGbVwdfHSFc1H4HfH0KWhp9w@mail.gmail.com>
 <7DA6D80F-2295-4C6F-A9DD-28376022230D@viaduct-productions.com>
Message-ID: <CAHaZY7gLTvSYorM+BfZKQKuWsSnZEvoe0Bmzbj2Hk0tkB91nHw@mail.gmail.com>

Thanks. Sorted :-)

John

On 24 July 2017 at 22:20, Viaduct Lists <lists at viaduct-productions.com>
wrote:

> I?ve done the same.
>
> Try listen port 8080, as anything < port 1024 needs to run as root.  Then
> in your url, enter hedge.local:8080.  Shove hedge.local into your
> /etc/hosts file and point to the proper IP.  But you need to enter the port
> number in that url to fetch it on the LAN.
>
>
> > On Jul 23, 2017, at 11:32 PM, John <johnhedge at gmail.com> wrote:
> >
> > I hope I've thought of everything but if not please let me know what
> else you require to get me up and going, please.
>
>
> _____________
> Rich in Toronto @ VP
>
>
>
>
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170725/26dcc45b/attachment.html>

From zchao1995 at gmail.com  Tue Jul 25 01:35:59 2017
From: zchao1995 at gmail.com (Zhang Chao)
Date: Mon, 24 Jul 2017 21:35:59 -0400
Subject: Disable NGINX caching 304 Responses from Origin Server
In-Reply-To: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>
References: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>
Message-ID: <CAPr95Bivg4oqev5NHk6JwkU8wx6hXwx2ZPPru_uYtTH+GiYnyg@mail.gmail.com>

Hi!

Have you used the proxy_cache_valid
<http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid>
?

Maybe you can add this directive to disable the 304 cache explicitly.

proxy_cache_valid 304 0;


On 24 July 2017 at 23:20:35, Ryan Barclay (ryan at rbftpnetworks.com) wrote:

We have a pretty simple setup with NGINX sitting on the front and a
backend server (on a separate physical server) that provides the content.

Nginx then caches content based on the EXPIRES and Cache-Control headers
set by the origin server.

We noticed that NGINX was not issuing 304 headers to images that were
not in the local NGINX cache when the If-Modified-Since header was sent
by the client. Instead, it would issue a 200 with the full data file.

To fix this, we applied:
proxy_set_header If-Modified-Since $http_if_modified_since

So then the If-Modified-Since header was passed to the backend and of
course, it returned correctly with the 304 header - great.

But what we noticed was that NGINX would cache this 304 response and
deliver future responses as 304 to clients even without the
If-Modified-Since header.

How can we disable caching of 304 responses and fix this issue?

Thank you for your help, suggestions, and tips in advance.
_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170724/22ea7691/attachment.html>

From ryan at rbftpnetworks.com  Tue Jul 25 07:24:30 2017
From: ryan at rbftpnetworks.com (Ryan Barclay)
Date: Tue, 25 Jul 2017 08:24:30 +0100
Subject: Disable NGINX caching 304 Responses from Origin Server
In-Reply-To: <CAPr95Bivg4oqev5NHk6JwkU8wx6hXwx2ZPPru_uYtTH+GiYnyg@mail.gmail.com>
References: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>
 <CAPr95Bivg4oqev5NHk6JwkU8wx6hXwx2ZPPru_uYtTH+GiYnyg@mail.gmail.com>
Message-ID: <ef32d76d-8aa2-f95d-0f3c-83aab04d9390@rbftpnetworks.com>

That's an interesting solution.  Would nginx still cache our other items 
as normal using this config?  We specify cache settings via 
Cache-Control and Expires headers sent from the origin server.  Also, we 
are using X-Accel-Expires for some pages which are sent from the origin 
server too.  This wouldn't affect any of our caching?  It would simply 
not cache any resources with 304 sent from the origin?

Another thought... would proxy_cache_valid override the Expires / 
Cache-Control headers sent along with the 304 response from the origin 
server?  Because with these 304 responses, the origin server sets a 3 
month expires.

Thank you for your help, it's much appreciated.



On 25/07/2017 02:35, Zhang Chao wrote:
>
> Hi!
>
> Have you used the proxy_cache_valid 
> <http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_cache_valid>?
>
> Maybe you can add this directive to disable the 304 cache explicitly.
>
> |proxy_cache_valid 304 0;|
>
> On 24 July 2017 at 23:20:35, Ryan Barclay (ryan at rbftpnetworks.com 
> <mailto:ryan at rbftpnetworks.com>) wrote:
>
>> We have a pretty simple setup with NGINX sitting on the front and a
>> backend server (on a separate physical server) that provides the 
>> content.
>>
>> Nginx then caches content based on the EXPIRES and Cache-Control headers
>> set by the origin server.
>>
>> We noticed that NGINX was not issuing 304 headers to images that were
>> not in the local NGINX cache when the If-Modified-Since header was sent
>> by the client. Instead, it would issue a 200 with the full data file.
>>
>> To fix this, we applied:
>> proxy_set_header If-Modified-Since $http_if_modified_since
>>
>> So then the If-Modified-Since header was passed to the backend and of
>> course, it returned correctly with the 304 header - great.
>>
>> But what we noticed was that NGINX would cache this 304 response and
>> deliver future responses as 304 to clients even without the
>> If-Modified-Since header.
>>
>> How can we disable caching of 304 responses and fix this issue?
>>
>> Thank you for your help, suggestions, and tips in advance.
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170725/c836ad3d/attachment-0001.html>

From ryan at rbftpnetworks.com  Wed Jul 26 07:29:12 2017
From: ryan at rbftpnetworks.com (Ryan Barclay)
Date: Wed, 26 Jul 2017 08:29:12 +0100
Subject: Disable NGINX caching 304 Responses from Origin Server
In-Reply-To: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>
References: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>
Message-ID: <2c91dc9e-879f-d8d6-b0b6-df10b7d51c75@rbftpnetworks.com>

The following config seems to work for the situation I discussed:

proxy_cache_valid 200 3M;
proxy_cache_valid 304 0;
proxy_cache_revalidate on;
proxy_set_header If-Modified-Since $http_if_modified_since;
proxy_ignore_headers Cache-Control Expires;


... can anybody see any problems with this config or future problems 
that may arise?

  

On 24/07/2017 16:20, Ryan Barclay wrote:
> We have a pretty simple setup with NGINX sitting on the front and a 
> backend server (on a separate physical server) that provides the content.
>
> Nginx then caches content based on the EXPIRES and Cache-Control 
> headers set by the origin server.
>
> We noticed that NGINX was not issuing 304 headers to images that were 
> not in the local NGINX cache when the If-Modified-Since header was 
> sent by the client. Instead, it would issue a 200 with the full data 
> file.
>
> To fix this, we applied:
> proxy_set_header If-Modified-Since $http_if_modified_since
>
> So then the If-Modified-Since header was passed to the backend and of 
> course, it returned correctly with the 304 header - great.
>
> But what we noticed was that NGINX would cache this 304 response and 
> deliver future responses as 304 to clients even without the 
> If-Modified-Since header.
>
> How can we disable caching of 304 responses and fix this issue?
>
> Thank you for your help, suggestions, and tips in advance.


From peter_booth at me.com  Wed Jul 26 08:11:10 2017
From: peter_booth at me.com (Peter Booth)
Date: Wed, 26 Jul 2017 04:11:10 -0400
Subject: Disable NGINX caching 304 Responses from Origin Server
In-Reply-To: <2c91dc9e-879f-d8d6-b0b6-df10b7d51c75@rbftpnetworks.com>
References: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>
 <2c91dc9e-879f-d8d6-b0b6-df10b7d51c75@rbftpnetworks.com>
Message-ID: <63438CEF-8E36-4496-A069-1A51F33F0C24@me.com>

I can?t see an obvious issue, but I can say that there is no such thing as a simple web server setup where caching is involved.
I have gray hairs that appeared after working with a high traffic retail website that had seven levels of caching
(browser cache, CDN, hardware load balancer, nginx reverse proxy, servlets that write content, tangosol /oracle coherence, endeca caching)

I?m hoping that you are living in  a saner world than that one but I?m sure that you will have some craziness. 
I would encourage you to add $upstream_cache_status  to your log format
and/or add the directive  add_header X-Cache-Status $upstream_cache_status;
Instrumenting the cache can be a real life-saver when things go awry.

I?d also strongly encourage you to use redbot.org to check for aberrant behavior and webpagetest.org
to see how different browsers handle your site. 

Peter



> On Jul 26, 2017, at 3:29 AM, Ryan Barclay <ryan at rbftpnetworks.com> wrote:
> 
> The following config seems to work for the situation I discussed:
> 
> proxy_cache_valid 200 3M;
> proxy_cache_valid 304 0;
> proxy_cache_revalidate on;
> proxy_set_header If-Modified-Since $http_if_modified_since;
> proxy_ignore_headers Cache-Control Expires;
> 
> 
> ... can anybody see any problems with this config or future problems that may arise?
> 
> 
> On 24/07/2017 16:20, Ryan Barclay wrote:
>> We have a pretty simple setup with NGINX sitting on the front and a backend server (on a separate physical server) that provides the content.
>> 
>> Nginx then caches content based on the EXPIRES and Cache-Control headers set by the origin server.
>> 
>> We noticed that NGINX was not issuing 304 headers to images that were not in the local NGINX cache when the If-Modified-Since header was sent by the client. Instead, it would issue a 200 with the full data file.
>> 
>> To fix this, we applied:
>> proxy_set_header If-Modified-Since $http_if_modified_since
>> 
>> So then the If-Modified-Since header was passed to the backend and of course, it returned correctly with the 304 header - great.
>> 
>> But what we noticed was that NGINX would cache this 304 response and deliver future responses as 304 to clients even without the If-Modified-Since header.
>> 
>> How can we disable caching of 304 responses and fix this issue?
>> 
>> Thank you for your help, suggestions, and tips in advance.
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170726/5b4ea16a/attachment.html>

From ryan at rbftpnetworks.com  Wed Jul 26 08:57:16 2017
From: ryan at rbftpnetworks.com (Ryan Barclay)
Date: Wed, 26 Jul 2017 09:57:16 +0100
Subject: Disable NGINX caching 304 Responses from Origin Server
In-Reply-To: <63438CEF-8E36-4496-A069-1A51F33F0C24@me.com>
References: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>
 <2c91dc9e-879f-d8d6-b0b6-df10b7d51c75@rbftpnetworks.com>
 <63438CEF-8E36-4496-A069-1A51F33F0C24@me.com>
Message-ID: <6913bf6b-aaee-885a-41b1-a386ea788e34@rbftpnetworks.com>

Thanks for the reply Peter.

I've noticed something interesting and wondered if you could shed some 
light on it.

Simply adding:

proxy_ignore_headers Cache-Control Expires;

Enables 304 responses from the origin server without setting:

proxy_set_header If-Modified-Since $http_if_modified_since;

I'm confused.

On 26/07/2017 09:11, Peter Booth wrote:
> I can?t see an obvious issue, but I can say that there is no such 
> thing as a simple web server setup where caching is involved.
> I have gray hairs that appeared after working with a high traffic 
> retail website that had seven levels of caching
> (browser cache, CDN, hardware load balancer, nginx reverse proxy, 
> servlets that write content, tangosol /oracle coherence, endeca caching)
>
> I?m hoping that you are living in  a saner world than that one but I?m 
> sure that you will have some craziness.
> I would encourage you to add $upstream_cache_status to your log format
> and/or add the directive add_headerX-Cache-Status $upstream_cache_status;
> Instrumenting the cache can be a real life-saver when things go awry.
>
> I?d also strongly encourage you to use redbot.org <http://redbot.org> 
> to check for aberrant behavior and webpagetest.org 
> <http://webpagetest.org>
> to see how different browsers handle your site.
>
> Peter
>
>
>
>> On Jul 26, 2017, at 3:29 AM, Ryan Barclay <ryan at rbftpnetworks.com 
>> <mailto:ryan at rbftpnetworks.com>> wrote:
>>
>> The following config seems to work for the situation I discussed:
>>
>> proxy_cache_valid 200 3M;
>> proxy_cache_valid 304 0;
>> proxy_cache_revalidate on;
>> proxy_set_header If-Modified-Since $http_if_modified_since;
>> proxy_ignore_headers Cache-Control Expires;
>>
>>
>> ... can anybody see any problems with this config or future problems 
>> that may arise?
>>
>>
>> On 24/07/2017 16:20, Ryan Barclay wrote:
>>> We have a pretty simple setup with NGINX sitting on the front and a 
>>> backend server (on a separate physical server) that provides the 
>>> content.
>>>
>>> Nginx then caches content based on the EXPIRES and Cache-Control 
>>> headers set by the origin server.
>>>
>>> We noticed that NGINX was not issuing 304 headers to images that 
>>> were not in the local NGINX cache when the If-Modified-Since header 
>>> was sent by the client. Instead, it would issue a 200 with the full 
>>> data file.
>>>
>>> To fix this, we applied:
>>> proxy_set_header If-Modified-Since $http_if_modified_since
>>>
>>> So then the If-Modified-Since header was passed to the backend and 
>>> of course, it returned correctly with the 304 header - great.
>>>
>>> But what we noticed was that NGINX would cache this 304 response and 
>>> deliver future responses as 304 to clients even without the 
>>> If-Modified-Since header.
>>>
>>> How can we disable caching of 304 responses and fix this issue?
>>>
>>> Thank you for your help, suggestions, and tips in advance.
>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170726/b39df2b9/attachment.html>

From ryan at rbftpnetworks.com  Wed Jul 26 09:10:09 2017
From: ryan at rbftpnetworks.com (Ryan Barclay)
Date: Wed, 26 Jul 2017 10:10:09 +0100
Subject: Disable NGINX caching 304 Responses from Origin Server
In-Reply-To: <6913bf6b-aaee-885a-41b1-a386ea788e34@rbftpnetworks.com>
References: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>
 <2c91dc9e-879f-d8d6-b0b6-df10b7d51c75@rbftpnetworks.com>
 <63438CEF-8E36-4496-A069-1A51F33F0C24@me.com>
 <6913bf6b-aaee-885a-41b1-a386ea788e34@rbftpnetworks.com>
Message-ID: <e73a793d-ee09-7c9f-d161-7c6faf636ace@rbftpnetworks.com>

So this config seems to work:

proxy_cache_valid 200 3M;
proxy_cache_valid 304 0;
proxy_cache_valid 404 0;
proxy_cache_revalidate on;
proxy_ignore_headers Cache-Control Expires;


There is no need for:

proxy_set_header If-Modified-Since $http_if_modified_since;



On 26/07/2017 09:57, Ryan Barclay wrote:
>
> Thanks for the reply Peter.
>
> I've noticed something interesting and wondered if you could shed some 
> light on it.
>
> Simply adding:
>
> proxy_ignore_headers Cache-Control Expires;
>
> Enables 304 responses from the origin server without setting:
>
> proxy_set_header If-Modified-Since $http_if_modified_since;
>
> I'm confused.
>
> On 26/07/2017 09:11, Peter Booth wrote:
>> I can?t see an obvious issue, but I can say that there is no such 
>> thing as a simple web server setup where caching is involved.
>> I have gray hairs that appeared after working with a high traffic 
>> retail website that had seven levels of caching
>> (browser cache, CDN, hardware load balancer, nginx reverse proxy, 
>> servlets that write content, tangosol /oracle coherence, endeca caching)
>>
>> I?m hoping that you are living in  a saner world than that one but 
>> I?m sure that you will have some craziness.
>> I would encourage you to add $upstream_cache_status to your log format
>> and/or add the directive add_headerX-Cache-Status $upstream_cache_status;
>> Instrumenting the cache can be a real life-saver when things go awry.
>>
>> I?d also strongly encourage you to use redbot.org <http://redbot.org> 
>> to check for aberrant behavior and webpagetest.org 
>> <http://webpagetest.org>
>> to see how different browsers handle your site.
>>
>> Peter
>>
>>
>>
>>> On Jul 26, 2017, at 3:29 AM, Ryan Barclay <ryan at rbftpnetworks.com 
>>> <mailto:ryan at rbftpnetworks.com>> wrote:
>>>
>>> The following config seems to work for the situation I discussed:
>>>
>>> proxy_cache_valid 200 3M;
>>> proxy_cache_valid 304 0;
>>> proxy_cache_revalidate on;
>>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>> proxy_ignore_headers Cache-Control Expires;
>>>
>>>
>>> ... can anybody see any problems with this config or future problems 
>>> that may arise?
>>>
>>>
>>> On 24/07/2017 16:20, Ryan Barclay wrote:
>>>> We have a pretty simple setup with NGINX sitting on the front and a 
>>>> backend server (on a separate physical server) that provides the 
>>>> content.
>>>>
>>>> Nginx then caches content based on the EXPIRES and Cache-Control 
>>>> headers set by the origin server.
>>>>
>>>> We noticed that NGINX was not issuing 304 headers to images that 
>>>> were not in the local NGINX cache when the If-Modified-Since header 
>>>> was sent by the client. Instead, it would issue a 200 with the full 
>>>> data file.
>>>>
>>>> To fix this, we applied:
>>>> proxy_set_header If-Modified-Since $http_if_modified_since
>>>>
>>>> So then the If-Modified-Since header was passed to the backend and 
>>>> of course, it returned correctly with the 304 header - great.
>>>>
>>>> But what we noticed was that NGINX would cache this 304 response 
>>>> and deliver future responses as 304 to clients even without the 
>>>> If-Modified-Since header.
>>>>
>>>> How can we disable caching of 304 responses and fix this issue?
>>>>
>>>> Thank you for your help, suggestions, and tips in advance.
>>>
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>
>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org
>> http://mailman.nginx.org/mailman/listinfo/nginx
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170726/1fa9b4cf/attachment-0001.html>

From peter_booth at me.com  Wed Jul 26 09:43:31 2017
From: peter_booth at me.com (Peter Booth)
Date: Wed, 26 Jul 2017 05:43:31 -0400
Subject: Disable NGINX caching 304 Responses from Origin Server
In-Reply-To: <e73a793d-ee09-7c9f-d161-7c6faf636ace@rbftpnetworks.com>
References: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>
 <2c91dc9e-879f-d8d6-b0b6-df10b7d51c75@rbftpnetworks.com>
 <63438CEF-8E36-4496-A069-1A51F33F0C24@me.com>
 <6913bf6b-aaee-885a-41b1-a386ea788e34@rbftpnetworks.com>
 <e73a793d-ee09-7c9f-d161-7c6faf636ace@rbftpnetworks.com>
Message-ID: <8729C230-278E-46AE-8E71-E7E05C351732@me.com>

Ryan,

Just to be pedantic, can you spell out  exactly what you meant when you said "and deliver future responses as 304 to clients even without the If-Modified-Since header?"
What requests were triggering the 304 response?
Were you observing what a browser was seeing or were you using curl or wget to trigger the response?

Peter

Sent from my iPhone

> On Jul 26, 2017, at 5:10 AM, Ryan Barclay <ryan at rbftpnetworks.com> wrote:
> 
> So this config seems to work:
> 
> proxy_cache_valid 200 3M;
> proxy_cache_valid 304 0;
> proxy_cache_valid 404 0;
> proxy_cache_revalidate on;
> proxy_ignore_headers Cache-Control Expires;
> 
> 
> There is no need for: 
> proxy_set_header If-Modified-Since $http_if_modified_since;
> 
> 
> 
>> On 26/07/2017 09:57, Ryan Barclay wrote:
>> Thanks for the reply Peter.
>> 
>> I've noticed something interesting and wondered if you could shed some light on it.
>> 
>> Simply adding:
>> proxy_ignore_headers Cache-Control Expires;
>> 
>> Enables 304 responses from the origin server without setting:
>> 
>> proxy_set_header If-Modified-Since $http_if_modified_since;
>> 
>> I'm confused.
>>> On 26/07/2017 09:11, Peter Booth wrote:
>>> I can?t see an obvious issue, but I can say that there is no such thing as a simple web server setup where caching is involved.
>>> I have gray hairs that appeared after working with a high traffic retail website that had seven levels of caching
>>> (browser cache, CDN, hardware load balancer, nginx reverse proxy, servlets that write content, tangosol /oracle coherence, endeca caching)
>>> 
>>> I?m hoping that you are living in  a saner world than that one but I?m sure that you will have some craziness. 
>>> I would encourage you to add $upstream_cache_status  to your log format
>>> and/or add the directive  add_header X-Cache-Status $upstream_cache_status;
>>> Instrumenting the cache can be a real life-saver when things go awry.
>>> 
>>> I?d also strongly encourage you to use redbot.org to check for aberrant behavior and webpagetest.org
>>> to see how different browsers handle your site. 
>>> 
>>> Peter
>>> 
>>> 
>>> 
>>>> On Jul 26, 2017, at 3:29 AM, Ryan Barclay <ryan at rbftpnetworks.com> wrote:
>>>> 
>>>> The following config seems to work for the situation I discussed:
>>>> 
>>>> proxy_cache_valid 200 3M;
>>>> proxy_cache_valid 304 0;
>>>> proxy_cache_revalidate on;
>>>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>>> proxy_ignore_headers Cache-Control Expires;
>>>> 
>>>> 
>>>> ... can anybody see any problems with this config or future problems that may arise?
>>>> 
>>>> 
>>>>> On 24/07/2017 16:20, Ryan Barclay wrote:
>>>>> We have a pretty simple setup with NGINX sitting on the front and a backend server (on a separate physical server) that provides the content.
>>>>> 
>>>>> Nginx then caches content based on the EXPIRES and Cache-Control headers set by the origin server.
>>>>> 
>>>>> We noticed that NGINX was not issuing 304 headers to images that were not in the local NGINX cache when the If-Modified-Since header was sent by the client. Instead, it would issue a 200 with the full data file.
>>>>> 
>>>>> To fix this, we applied:
>>>>> proxy_set_header If-Modified-Since $http_if_modified_since
>>>>> 
>>>>> So then the If-Modified-Since header was passed to the backend and of course, it returned correctly with the 304 header - great.
>>>>> 
>>>>> But what we noticed was that NGINX would cache this 304 response and deliver future responses as 304 to clients even without the If-Modified-Since header.
>>>>> 
>>>>> How can we disable caching of 304 responses and fix this issue?
>>>>> 
>>>>> Thank you for your help, suggestions, and tips in advance.
>>>> 
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx at nginx.org
>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>> 
>>> 
>>> 
>>> _______________________________________________
>>> nginx mailing list
>>> nginx at nginx.org
>>> http://mailman.nginx.org/mailman/listinfo/nginx
>> 
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170726/64d13168/attachment.html>

From nginx-ml at acheronmedia.hr  Wed Jul 26 10:15:10 2017
From: nginx-ml at acheronmedia.hr (Vlad K.)
Date: Wed, 26 Jul 2017 12:15:10 +0200
Subject: Identifying "Writing" connections in status stub
Message-ID: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>


Hello list,

I'm graphing information from the nginx status page, and have noticed 
something odd. The "Writing" connections are flat over time, not 
correlated to the Active/Reading/Waiting connections and are steadily 
increasing over time. Example for the past week:

https://pasteboard.co/GCHKB3B.png

Where it drops, is where I've restarted (not reloaded) the service, and 
starts growing up after a short while. This server FreeBSD but I've 
noticed it also in Debian.

Is there a way to find out which connections are these, which remote IPs 
they are so I can track them with netstat or sockstat? This looks to me 
like connection FD or something has been leaking. If this is a bug, I'm 
not sure what to report.

I've also noticed, from time to time, connections lingering for long 
time in "CLOSED" state (as reported by netstat), googling for which 
seems to suggest a bug in application, where it doesn't release the FD 
after the remote has closed.


Thanks.


-- 
Vlad K.

From ryan at rbftpnetworks.com  Wed Jul 26 10:18:33 2017
From: ryan at rbftpnetworks.com (Ryan Barclay)
Date: Wed, 26 Jul 2017 11:18:33 +0100
Subject: Disable NGINX caching 304 Responses from Origin Server
In-Reply-To: <8729C230-278E-46AE-8E71-E7E05C351732@me.com>
References: <d55b34d0-e500-060d-453a-e68c93ea4473@rbftpnetworks.com>
 <2c91dc9e-879f-d8d6-b0b6-df10b7d51c75@rbftpnetworks.com>
 <63438CEF-8E36-4496-A069-1A51F33F0C24@me.com>
 <6913bf6b-aaee-885a-41b1-a386ea788e34@rbftpnetworks.com>
 <e73a793d-ee09-7c9f-d161-7c6faf636ace@rbftpnetworks.com>
 <8729C230-278E-46AE-8E71-E7E05C351732@me.com>
Message-ID: <d23f395a-a60e-2aa8-12d4-3c434059ee18@rbftpnetworks.com>

Hi Peter,

When we used this:

proxy_set_header If-Modified-Since $http_if_modified_since

...Nginx was caching 304 responses from the upstream when the client 
requested If-Modified-Since (if it was the first request for that 
object).  Future requests by clients that didn't set the 
If-Modified-Since header were also being served the 304 (with no body) 
as that was what was in the cache.

I hope that makes it clearer?

Thanks Peter.

On 26/07/2017 10:43, Peter Booth wrote:
> Ryan,
>
> Just to be pedantic, can you spell out  exactly what you meant when 
> you said "and deliver future responses as 304 to clients even without 
> the If-Modified-Since header?"
> What requests were triggering the 304 response?
> Were you observing what a browser was seeing or were you using curl or 
> wget to trigger the response?
>
> Peter
>
> Sent from my iPhone
>
> On Jul 26, 2017, at 5:10 AM, Ryan Barclay <ryan at rbftpnetworks.com 
> <mailto:ryan at rbftpnetworks.com>> wrote:
>
>> So this config seems to work:
>>
>> proxy_cache_valid 200 3M;
>> proxy_cache_valid 304 0;
>> proxy_cache_valid 404 0;
>> proxy_cache_revalidate on;
>> proxy_ignore_headers Cache-Control Expires;
>>
>>
>> There is no need for:
>>
>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>
>>
>>
>> On 26/07/2017 09:57, Ryan Barclay wrote:
>>>
>>> Thanks for the reply Peter.
>>>
>>> I've noticed something interesting and wondered if you could shed 
>>> some light on it.
>>>
>>> Simply adding:
>>>
>>> proxy_ignore_headers Cache-Control Expires;
>>>
>>> Enables 304 responses from the origin server without setting:
>>>
>>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>>
>>> I'm confused.
>>>
>>> On 26/07/2017 09:11, Peter Booth wrote:
>>>> I can?t see an obvious issue, but I can say that there is no such 
>>>> thing as a simple web server setup where caching is involved.
>>>> I have gray hairs that appeared after working with a high traffic 
>>>> retail website that had seven levels of caching
>>>> (browser cache, CDN, hardware load balancer, nginx reverse proxy, 
>>>> servlets that write content, tangosol /oracle coherence, endeca 
>>>> caching)
>>>>
>>>> I?m hoping that you are living in  a saner world than that one but 
>>>> I?m sure that you will have some craziness.
>>>> I would encourage you to add $upstream_cache_status to your log format
>>>> and/or add the directive add_headerX-Cache-Status 
>>>> $upstream_cache_status;
>>>> Instrumenting the cache can be a real life-saver when things go awry.
>>>>
>>>> I?d also strongly encourage you to use redbot.org 
>>>> <http://redbot.org> to check for aberrant behavior and 
>>>> webpagetest.org <http://webpagetest.org>
>>>> to see how different browsers handle your site.
>>>>
>>>> Peter
>>>>
>>>>
>>>>
>>>>> On Jul 26, 2017, at 3:29 AM, Ryan Barclay <ryan at rbftpnetworks.com 
>>>>> <mailto:ryan at rbftpnetworks.com>> wrote:
>>>>>
>>>>> The following config seems to work for the situation I discussed:
>>>>>
>>>>> proxy_cache_valid 200 3M;
>>>>> proxy_cache_valid 304 0;
>>>>> proxy_cache_revalidate on;
>>>>> proxy_set_header If-Modified-Since $http_if_modified_since;
>>>>> proxy_ignore_headers Cache-Control Expires;
>>>>>
>>>>>
>>>>> ... can anybody see any problems with this config or future 
>>>>> problems that may arise?
>>>>>
>>>>>
>>>>> On 24/07/2017 16:20, Ryan Barclay wrote:
>>>>>> We have a pretty simple setup with NGINX sitting on the front and 
>>>>>> a backend server (on a separate physical server) that provides 
>>>>>> the content.
>>>>>>
>>>>>> Nginx then caches content based on the EXPIRES and Cache-Control 
>>>>>> headers set by the origin server.
>>>>>>
>>>>>> We noticed that NGINX was not issuing 304 headers to images that 
>>>>>> were not in the local NGINX cache when the If-Modified-Since 
>>>>>> header was sent by the client. Instead, it would issue a 200 with 
>>>>>> the full data file.
>>>>>>
>>>>>> To fix this, we applied:
>>>>>> proxy_set_header If-Modified-Since $http_if_modified_since
>>>>>>
>>>>>> So then the If-Modified-Since header was passed to the backend 
>>>>>> and of course, it returned correctly with the 304 header - great.
>>>>>>
>>>>>> But what we noticed was that NGINX would cache this 304 response 
>>>>>> and deliver future responses as 304 to clients even without the 
>>>>>> If-Modified-Since header.
>>>>>>
>>>>>> How can we disable caching of 304 responses and fix this issue?
>>>>>>
>>>>>> Thank you for your help, suggestions, and tips in advance.
>>>>>
>>>>> _______________________________________________
>>>>> nginx mailing list
>>>>> nginx at nginx.org <mailto:nginx at nginx.org>
>>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> nginx mailing list
>>>> nginx at nginx.org
>>>> http://mailman.nginx.org/mailman/listinfo/nginx
>>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170726/602820e5/attachment-0001.html>

From peter_booth at me.com  Wed Jul 26 11:36:53 2017
From: peter_booth at me.com (Peter Booth)
Date: Wed, 26 Jul 2017 07:36:53 -0400
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
Message-ID: <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>

Vlad,

I'd suggest beginning by seeing whether or not this is real. If you create a cron job that invokes netstat -ant every hour, then summarize the connections and either view them manually or write them into an influxdb and graph with grafana you will see whether or not the #tcp connections really is growing and, if so, which connections are growing.

That would seem like a useful first step.

Peter 

Sent from my iPhone

> On Jul 26, 2017, at 6:15 AM, Vlad K. <nginx-ml at acheronmedia.hr> wrote:
> 
> 
> Hello list,
> 
> I'm graphing information from the nginx status page, and have noticed something odd. The "Writing" connections are flat over time, not correlated to the Active/Reading/Waiting connections and are steadily increasing over time. Example for the past week:
> 
> https://pasteboard.co/GCHKB3B.png
> 
> Where it drops, is where I've restarted (not reloaded) the service, and starts growing up after a short while. This server FreeBSD but I've noticed it also in Debian.
> 
> Is there a way to find out which connections are these, which remote IPs they are so I can track them with netstat or sockstat? This looks to me like connection FD or something has been leaking. If this is a bug, I'm not sure what to report.
> 
> I've also noticed, from time to time, connections lingering for long time in "CLOSED" state (as reported by netstat), googling for which seems to suggest a bug in application, where it doesn't release the FD after the remote has closed.
> 
> 
> Thanks.
> 
> 
> -- 
> Vlad K.
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

From nginx-ml at acheronmedia.hr  Wed Jul 26 16:05:28 2017
From: nginx-ml at acheronmedia.hr (Vlad K.)
Date: Wed, 26 Jul 2017 18:05:28 +0200
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
 <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
Message-ID: <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>

On 2017-07-26 13:36, Peter Booth wrote:
> Vlad,
> 
> I'd suggest beginning by seeing whether or not this is real. If you
> create a cron job that invokes netstat -ant every hour, then summarize
> the connections and either view them manually or write them into an
> influxdb and graph with grafana you will see whether or not the #tcp
> connections really is growing and, if so, which connections are
> growing.

Thanks for the suggestion, but with it slow progression and low 
signal-to-noise ration in comparison with the daily and weekly 
connection cycle, I'm not sure it would be practically possible to 
measure it like that. But your suggestion gave me another idea, to 
record IPs of established tcp conns every 5 minutes and then see which 
ones remain constant.

But are you suggesting that nginx status is reporting wrong/fake 
numbers? What do you mean by "real"?

And what about connections that are staying in "CLOSED" state until I 
restart or reload nginx?




-- 
Vlad K.

From nginx-forum at forum.nginx.org  Wed Jul 26 16:12:07 2017
From: nginx-forum at forum.nginx.org (anish10dec)
Date: Wed, 26 Jul 2017 12:12:07 -0400
Subject: Secure Link Md5 Implementation
Message-ID: <e2bce17db0df9646fd81f8f192ff0d33.NginxMailingListEnglish@forum.nginx.org>

Trying  to implement the secure link md5 token check .

Is there a way to verify  secure link i.e. to generate token using secret
key and verify the token. If it matches it should allow the request .
And also to allow the request for token which doesn't matches so that while
rolling out the update it may happen that some of the client request will
come without token . 
Those request should also get allowed meanwhile till all the client are
updated with new update of enabling token based authentication.

Secondly, Is there a way to implement the token authentication with two
secret key i.e primary and secondary
So that If the first one did not work, then try the second one.
This would be helpful while changing the Secret Key in production so that
some user will be allowed with old secret and some with new secret whose
client has been updated.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275668,275668#msg-275668


From nginx-forum at forum.nginx.org  Wed Jul 26 18:06:42 2017
From: nginx-forum at forum.nginx.org (c0nw0nk)
Date: Wed, 26 Jul 2017 14:06:42 -0400
Subject: Secure Link Md5 Implementation
In-Reply-To: <e2bce17db0df9646fd81f8f192ff0d33.NginxMailingListEnglish@forum.nginx.org>
References: <e2bce17db0df9646fd81f8f192ff0d33.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <0d03e8b125cd6965a96efb1db57a00bd.NginxMailingListEnglish@forum.nginx.org>

Update your web application for example (PHP) first then how ever many hours
later when all caches for your web application have cleared restart your
Nginx so it only accepts secure links.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275668,275669#msg-275669


From nginx-forum at forum.nginx.org  Thu Jul 27 05:55:08 2017
From: nginx-forum at forum.nginx.org (c0nw0nk)
Date: Thu, 27 Jul 2017 01:55:08 -0400
Subject: A Nginx Upstream DDoS Blackhole how does it work ?
Message-ID: <31f5aee60520ba2d94a63b4fe2e2f455.NginxMailingListEnglish@forum.nginx.org>

So I was looking at a upstream that has been flooded from multiple locations
and read that you can create what is called a blackhole within the upstream
what helps with the DDoS scenario.

Here Is My upstream config :

upstream web_rack {
server 127.0.0.1:9000 weight=1 fail_timeout=4;
server 127.0.0.1:9001 weight=1 fail_timeout=4;
server 127.0.0.1:9002 weight=1 fail_timeout=4;
server 127.0.0.1:9003 weight=1 fail_timeout=4;
server 127.0.0.1:9004 weight=1 fail_timeout=4;
server 127.0.0.1:9005 weight=1 fail_timeout=4;
server 127.0.0.1:9006 weight=1 fail_timeout=4;
server 127.0.0.1:9007 weight=1 fail_timeout=4;
server 127.0.0.1:9008 weight=1 fail_timeout=4;
server 127.0.0.1:9009 weight=1 fail_timeout=4;
server 127.0.0.1:9010 weight=1 fail_timeout=4;
least_conn;
}



My question is how does a blackhole in the upstream help and work... is
setting the "DOWN" state a permament 503 for all timed out upstream requests
?

And here is the upstream i read can act as a blackhole ( itpp2012 gets
credit for this )

A simple configuration like:
upstream myLoadBalancer {
server 192.168.169.22:80 weight=1 fail_timeout=5;
server 192.168.169.17:80 weight=1 fail_timeout=5;
server 192.168.169.26:80 weight=1 fail_timeout=5;
server 192.168.169.23:80 weight=1 fail_timeout=5;
server 192.168.169.27:80 weight=1 fail_timeout=5 down;
server 192.168.169.28:80 weight=1 fail_timeout=5 down;
least_conn;
}
upstream myLoadBalancerDDOS {
server 127.0.0.1:8081 weight=1 fail_timeout=5;
server 127.0.0.1:8082 weight=1 fail_timeout=5;
server 127.0.0.1:8083 weight=1 fail_timeout=5 down;
server 127.0.0.1:8084 weight=1 fail_timeout=5 down;
server 127.0.0.1:8085 weight=1 fail_timeout=5 down;
server 192.168.169.254:80 weight=1 fail_timeout=5 down;
least_conn;
}

In myLoadBalancer you have set 2 extra backends ready for expanding
capacity.
In myLoadBalancerDDOS you have set 2 backends (or internal redirects to 503
location blocks) to deal with attacks, a backend (...169.254:80) to serve as
a blackhole and 3 more for expanding capacity.

Of course you can set as many backends and their destinations as you like,
other webservers, faster blackholes, offloading addresses, swap between
cloud providers, etc...

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275671,275671#msg-275671


From nginx-forum at forum.nginx.org  Thu Jul 27 06:34:09 2017
From: nginx-forum at forum.nginx.org (anish10dec)
Date: Thu, 27 Jul 2017 02:34:09 -0400
Subject: Secure Link Md5 Implementation
In-Reply-To: <0d03e8b125cd6965a96efb1db57a00bd.NginxMailingListEnglish@forum.nginx.org>
References: <e2bce17db0df9646fd81f8f192ff0d33.NginxMailingListEnglish@forum.nginx.org>
 <0d03e8b125cd6965a96efb1db57a00bd.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <4a4bf9e611af4f5fcd05079ffdff8500.NginxMailingListEnglish@forum.nginx.org>

For validating all the m3u8 , below is the configuration 

location  ~.*.m3u8 {

secure_link $arg_token,$arg_expires;

secure_link_md5 "appsecret$uri$arg_expires";

if ($secure_link = "") {return 403;}
if ($secure_link = "0"){return 410;}

proxy_pass http://appserver:80;

}

What I need is the way to temporarily allow users who are not coming with
token as well.
 As when we release the update of the app, it will not get updated to
everyone at once. So temporarily want to allow users coming without token as
well.
It should be accessible to both below URL
 
http://testapp.com/video/master.m3u8?token=xyz&exp=123

http://testapp.com/video/master.m3u8

Regards,
Anish

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275668,275672#msg-275672


From nginx-forum at forum.nginx.org  Thu Jul 27 07:55:40 2017
From: nginx-forum at forum.nginx.org (foxgab)
Date: Thu, 27 Jul 2017 03:55:40 -0400
Subject: can variables be used in regular expressions?
Message-ID: <7bc824ad51871f3b88e7c40f0450377b.NginxMailingListEnglish@forum.nginx.org>

i want to change the redirect location URI protocol to https if that is
using, so i tried to configure like this:

  proxy_redirect      ~*^http://$host(/.*)?$ $scheme://$host$1;

but that didn't work, so i change it to:

  proxy_redirect      http://$http_host $scheme://$http_host;

then, it works. I suspect that the variable in regular expressions is treat
as a string, hence my former directive failed, am I right?
Is there any way to make variables effect in regular expressions?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275675,275675#msg-275675


From nginx-forum at forum.nginx.org  Thu Jul 27 08:28:41 2017
From: nginx-forum at forum.nginx.org (emios)
Date: Thu, 27 Jul 2017 04:28:41 -0400
Subject: Transfer Zend App to new VPS show blank page
Message-ID: <8c76c668e4b076004535f666707f337c.NginxMailingListEnglish@forum.nginx.org>

I bought a new VPS and I try to transfer my working ZendApp (work in
previous hosting with DirectAdmin). Now i install nginx with php5 and mysql
i this is my php configuration - works well

http://178.216.200.85/info.php

I turn on display errors in php.ini and it shows me error in writable
/_cache/ folder soo i set it to 777 permission. Error resolved but i still
have error and don't see anything in my site
http://178.216.200.85/index.php

I don't have any .htaccess file

Here is my /etc/nginx/sites-avaible/default content

server {
    listen 80 default_server;
    listen [::]:80 default_server;

    # SSL configuration
    #
    # listen 443 ssl default_server;
    # listen [::]:443 ssl default_server;
    #
    # Self signed certs generated by the ssl-cert package
    # Don't use them in a production server!
    #
    # include snippets/snakeoil.conf;

    root /var/www/html;

    # Add index.php to the list if you are using PHP
    index index.php index.html index.htm index.nginx-debian.html;

    server_name 178.216.200.85;

    location / {
        # First attempt to serve request as file, then
        # as directory, then fall back to displaying a 404.
        try_files $uri $uri/ =404;
    }


    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_pass unix:/var/run/php5-fpm.sock;
    }

    location ~ /\.ht {
        deny all;
    }
}

What i should to add or someting to run my app in new server?

Any advice will be very valuable

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275676,275676#msg-275676


From mdounin at mdounin.ru  Thu Jul 27 08:56:40 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 27 Jul 2017 11:56:40 +0300
Subject: Filter installation for WebSocket protocol in Nginx.
In-Reply-To: <8bca4889102081899abccc6d3a9129c6.NginxMailingListEnglish@forum.nginx.org>
References: <8bca4889102081899abccc6d3a9129c6.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170727085640.GK93611@mdounin.ru>

Hello!

On Sat, Jul 22, 2017 at 11:10:02AM -0400, mohini wrote:

> I have an working nginx module (http_content_Modify_module) that acts as a
> proxy and manipulates some specific http data coming to the proxy before
> sending it to the upstream server.Used nginx1.6.2 for this and was
> implemented by installing filters in the "post-configuration" setup.
> Now I need to move from http to websocket protocol support. 
> By changing the configuration file to support "Upgrade" header,
> http_content_Modify_module  was able to talk to upstream webSocket server. 
> However I am not sure if nginx supports filter installation for websockets
> like it does for http so that I can manipulate/change data coming over the
> websocket
> before sending it to upstream.

No, it is not supported.

-- 
Maxim Dounin
http://nginx.org/

From nginx-forum at forum.nginx.org  Thu Jul 27 09:39:24 2017
From: nginx-forum at forum.nginx.org (c0nw0nk)
Date: Thu, 27 Jul 2017 05:39:24 -0400
Subject: Secure Link Md5 Implementation
In-Reply-To: <4a4bf9e611af4f5fcd05079ffdff8500.NginxMailingListEnglish@forum.nginx.org>
References: <e2bce17db0df9646fd81f8f192ff0d33.NginxMailingListEnglish@forum.nginx.org>
 <0d03e8b125cd6965a96efb1db57a00bd.NginxMailingListEnglish@forum.nginx.org>
 <4a4bf9e611af4f5fcd05079ffdff8500.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <a3896df2685d3092a547eb3cdfbe8903.NginxMailingListEnglish@forum.nginx.org>

Like i said before

c0nw0nk Wrote:
-------------------------------------------------------
> Update your web application for example (PHP) first then how ever many
> hours later when all caches for your web application have cleared
> restart your Nginx so it only accepts secure links.


Update your app first so your app outputs secured links.

Then when all caches and users are ready update Nginx config / restart Nginx
to force secured links only.

Both of your links

http://testapp.com/video/master.m3u8?token=xyz&exp=123

http://testapp.com/video/master.m3u8 

Will work without Nginx being updated thats why you update your app outputs
first common sense that way everyone gets using the secured link without it
being secured but your logs will show you accepting ?token &args.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275668,275679#msg-275679


From mdounin at mdounin.ru  Thu Jul 27 10:31:29 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Thu, 27 Jul 2017 13:31:29 +0300
Subject: can variables be used in regular expressions?
In-Reply-To: <7bc824ad51871f3b88e7c40f0450377b.NginxMailingListEnglish@forum.nginx.org>
References: <7bc824ad51871f3b88e7c40f0450377b.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170727103129.GN93611@mdounin.ru>

Hello!

On Thu, Jul 27, 2017 at 03:55:40AM -0400, foxgab wrote:

> i want to change the redirect location URI protocol to https if that is
> using, so i tried to configure like this:
> 
>   proxy_redirect      ~*^http://$host(/.*)?$ $scheme://$host$1;
> 
> but that didn't work, so i change it to:
> 
>   proxy_redirect      http://$http_host $scheme://$http_host;
> 
> then, it works. I suspect that the variable in regular expressions is treat
> as a string, hence my former directive failed, am I right?
> Is there any way to make variables effect in regular expressions?

No.  Regular expressions are compiled during configuration 
parsing, and using variables in them is not possible.

-- 
Maxim Dounin
http://nginx.org/

From r1ch+nginx at teamliquid.net  Thu Jul 27 11:10:14 2017
From: r1ch+nginx at teamliquid.net (Richard Stanway)
Date: Thu, 27 Jul 2017 13:10:14 +0200
Subject: Transfer Zend App to new VPS show blank page
In-Reply-To: <8c76c668e4b076004535f666707f337c.NginxMailingListEnglish@forum.nginx.org>
References: <8c76c668e4b076004535f666707f337c.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <CAKWQeyhnAFu33UCTZQgV9EYmakmWehYuhSR_N-hPj3bZdum3rg@mail.gmail.com>

Your backend is returning a HTTP 500, which likely indicates a PHP Fatal
Error. You should check the error log for the site.

On Thu, Jul 27, 2017 at 10:28 AM, emios <nginx-forum at forum.nginx.org> wrote:

> I bought a new VPS and I try to transfer my working ZendApp (work in
> previous hosting with DirectAdmin). Now i install nginx with php5 and mysql
> i this is my php configuration - works well
>
> http://178.216.200.85/info.php
>
> I turn on display errors in php.ini and it shows me error in writable
> /_cache/ folder soo i set it to 777 permission. Error resolved but i still
> have error and don't see anything in my site
> http://178.216.200.85/index.php
>
> I don't have any .htaccess file
>
> Here is my /etc/nginx/sites-avaible/default content
>
> server {
>     listen 80 default_server;
>     listen [::]:80 default_server;
>
>     # SSL configuration
>     #
>     # listen 443 ssl default_server;
>     # listen [::]:443 ssl default_server;
>     #
>     # Self signed certs generated by the ssl-cert package
>     # Don't use them in a production server!
>     #
>     # include snippets/snakeoil.conf;
>
>     root /var/www/html;
>
>     # Add index.php to the list if you are using PHP
>     index index.php index.html index.htm index.nginx-debian.html;
>
>     server_name 178.216.200.85;
>
>     location / {
>         # First attempt to serve request as file, then
>         # as directory, then fall back to displaying a 404.
>         try_files $uri $uri/ =404;
>     }
>
>
>     location ~ \.php$ {
>         include snippets/fastcgi-php.conf;
>         fastcgi_pass unix:/var/run/php5-fpm.sock;
>     }
>
>     location ~ /\.ht {
>         deny all;
>     }
> }
>
> What i should to add or someting to run my app in new server?
>
> Any advice will be very valuable
>
> Posted at Nginx Forum: https://forum.nginx.org/read.
> php?2,275676,275676#msg-275676
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170727/1fef0009/attachment.html>

From tkadm30 at yandex.com  Thu Jul 27 11:45:28 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Thu, 27 Jul 2017 07:45:28 -0400
Subject: Problem with uWSGI and PATH_INFO
Message-ID: <3f77e732-0400-9859-5048-8651147da727@yandex.com>

Hi, uWSGI apparently does some weird voodoo tricks to manage SCRIPT_NAME 
and PATH_INFO when using nginx: "The WSGI standard dictates that 
|SCRIPT_NAME| is the variable used to select a specific application. 
Unfortunately u nginx is not able to rewrite PATH_INFO accordingly to 
SCRIPT_NAME. For such reason you need to instruct WSGI to map specific 
apps in the so called ?mountpoint? and rewrite SCRIPT_NAME and PATH_INFO 
automatically"

I'm not sure I understand the logic of this. Can someone please explain 
why the variable PATH_INFO is set to $document_uri in uwsgi_params?

Thank you in advance,

E


-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170727/44ee5d4c/attachment.html>

From nginx-forum at forum.nginx.org  Thu Jul 27 11:50:03 2017
From: nginx-forum at forum.nginx.org (anish10dec)
Date: Thu, 27 Jul 2017 07:50:03 -0400
Subject: Secure Link Md5 Implementation
In-Reply-To: <e2bce17db0df9646fd81f8f192ff0d33.NginxMailingListEnglish@forum.nginx.org>
References: <e2bce17db0df9646fd81f8f192ff0d33.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <97baf30234e3cfdc3d5a524ca183947b.NginxMailingListEnglish@forum.nginx.org>

Thanks 

But what about the next part when actually we are in production and if there
is need for change of secret Key on Nginx.

" Is there a way to implement the token authentication with two secret key
i.e primary and secondary 
So that If the first one did not work, then try the second one. 
This would be helpful while changing the Secret Key in production so that
some user will be allowed with old secret and some with new secret whose
client has been updated."

Regards,
Anish

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275668,275685#msg-275685


From tkadm30 at yandex.com  Thu Jul 27 12:52:36 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Thu, 27 Jul 2017 08:52:36 -0400
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <7c415135f32036ca0bed7c1d534e93ae.squirrel@squirrelmail.unbit.it>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <9ed9825f-828e-d672-2381-9d933cac3ace@yandex.com>
 <7c415135f32036ca0bed7c1d534e93ae.squirrel@squirrelmail.unbit.it>
Message-ID: <2f1e6037-a142-7ad7-f455-7e1b7314276e@yandex.com>

Hi Roberto,

My Django app runs perfectly ok under FastCGI and nginx but is not 
capable of resolving the proper PATH_INFO under uWSGI. In 
fastcgi_params, the value of PATH_INFO and SCRIPT_NAME are set to 
"$fastcgi_script_name". My nginx config looks like this:

location / {

  uwsgi_pass django;

include uwsgi_params;

}

My wsgi app (django-hotsauce) really relay on a HTTP/1.1 compatible 
PATH_INFO value, just like in wsgiref. Why is it not possible to 
implement $uwsgi_script_name variable for nginx?

Best regards,
Etienne


Le 2017-07-27 ? 08:36, Roberto De Ioris a ?crit :
>
>> Hi, this is an example on how to configure multiple apps:
>>
>> http://uwsgi-docs.readthedocs.io/en/latest/Snippets.html#multiple-flask-apps-in-different-mountpoints
>>
>> Best thing to do is completely avoid nginx manipulating the SCRIPT_NAME
>> variable.
>>
>>
>> Even when under fastcgi, the WSGI standard expects a different meaning for
>> SCRIPT_NAME
>>

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From roberto at unbit.it  Thu Jul 27 13:01:22 2017
From: roberto at unbit.it (Roberto De Ioris)
Date: Thu, 27 Jul 2017 15:01:22 +0200
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <2f1e6037-a142-7ad7-f455-7e1b7314276e@yandex.com>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <9ed9825f-828e-d672-2381-9d933cac3ace@yandex.com>
 <7c415135f32036ca0bed7c1d534e93ae.squirrel@squirrelmail.unbit.it>
 <2f1e6037-a142-7ad7-f455-7e1b7314276e@yandex.com>
Message-ID: <c7d16ce5f9152a2e03653ef638e4ae31.squirrel@squirrelmail.unbit.it>


> Hi Roberto,
>
> My Django app runs perfectly ok under FastCGI and nginx but is not
> capable of resolving the proper PATH_INFO under uWSGI. In
> fastcgi_params, the value of PATH_INFO and SCRIPT_NAME are set to
> "$fastcgi_script_name". My nginx config looks like this:
>
> location / {
>
>   uwsgi_pass django;
>
> include uwsgi_params;
>
> }
>
> My wsgi app (django-hotsauce) really relay on a HTTP/1.1 compatible
> PATH_INFO value, just like in wsgiref. Why is it not possible to
> implement $uwsgi_script_name variable for nginx?
>
> Best regards,
> Etienne
>
>

Hi,

if the url is something like /foo/bar and you have a location like

location /foo {
    ...
}

the WSGI standard expects SCRIPT_NAME to be /foo and PATH_INFO to be /bar

if you manually set SCRIPT_NAME to /foo in nginx, PATH_INFO will continue
to be /foo/bar.

Obviously having nginx managing it could be useful, but nowadays it is way
more versatile to manage this part in uWSGI itself (using the
manage-script-name or internal routing to manually rewrite apps), or
directly in your WSGI middleware.

your fastcgi adapter (i suppose flup) hardcodes this magic using the
classic apache fastcgi patterns (something we cannot rely on as we need to
support multiple environments)

-- 
Roberto De Ioris
http://unbit.com

From nginx-forum at forum.nginx.org  Thu Jul 27 13:13:04 2017
From: nginx-forum at forum.nginx.org (itpp2012)
Date: Thu, 27 Jul 2017 09:13:04 -0400
Subject: A Nginx Upstream DDoS Blackhole how does it work ?
In-Reply-To: <31f5aee60520ba2d94a63b4fe2e2f455.NginxMailingListEnglish@forum.nginx.org>
References: <31f5aee60520ba2d94a63b4fe2e2f455.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <4f08c942804319ee1526a15088374dc1.NginxMailingListEnglish@forum.nginx.org>

See the 'upstream_EBLB_with_IWCP.txt' document and the pdf documentation,
the primary control (ie. signaling) should be (initiated) triggered by your
edge controllers or router.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275671,275688#msg-275688


From tkadm30 at yandex.com  Thu Jul 27 13:18:48 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Thu, 27 Jul 2017 09:18:48 -0400
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <c7d16ce5f9152a2e03653ef638e4ae31.squirrel@squirrelmail.unbit.it>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <9ed9825f-828e-d672-2381-9d933cac3ace@yandex.com>
 <7c415135f32036ca0bed7c1d534e93ae.squirrel@squirrelmail.unbit.it>
 <2f1e6037-a142-7ad7-f455-7e1b7314276e@yandex.com>
 <c7d16ce5f9152a2e03653ef638e4ae31.squirrel@squirrelmail.unbit.it>
Message-ID: <62e042e9-0617-4ffe-0abd-3176ac226ceb@yandex.com>

Hi Roberto,

I use gevent-fastcgi for production. I wish to have the time to dive in 
the internals of uWSGI to understand how PATH_INFO and SCRIPT_NAME gets 
rewritten by uWSGI. I believe the implementation of --manage-script-name 
is incorrect and should be fixed to support having a empty or not set 
SCRIPT_NAME value.

Etienne


Le 2017-07-27 ? 09:01, Roberto De Ioris a ?crit :
>> Hi,
>>
>> if the url is something like /foo/bar and you have a location like
>>
>> location /foo {
>>      ...
>> }
>>
>> the WSGI standard expects SCRIPT_NAME to be /foo and PATH_INFO to be /bar
>>
>> if you manually set SCRIPT_NAME to /foo in nginx, PATH_INFO will continue
>> to be /foo/bar.
>>
>> Obviously having nginx managing it could be useful, but nowadays it is way
>> more versatile to manage this part in uWSGI itself (using the
>> manage-script-name or internal routing to manually rewrite apps), or
>> directly in your WSGI middleware.
>>
>> your fastcgi adapter (i suppose flup) hardcodes this magic using the
>> classic apache fastcgi patterns (something we cannot rely on as we need to
>> support multiple environments)
>>

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From roberto at unbit.it  Thu Jul 27 13:52:09 2017
From: roberto at unbit.it (Roberto De Ioris)
Date: Thu, 27 Jul 2017 15:52:09 +0200
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <62e042e9-0617-4ffe-0abd-3176ac226ceb@yandex.com>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <9ed9825f-828e-d672-2381-9d933cac3ace@yandex.com>
 <7c415135f32036ca0bed7c1d534e93ae.squirrel@squirrelmail.unbit.it>
 <2f1e6037-a142-7ad7-f455-7e1b7314276e@yandex.com>
 <c7d16ce5f9152a2e03653ef638e4ae31.squirrel@squirrelmail.unbit.it>
 <62e042e9-0617-4ffe-0abd-3176ac226ceb@yandex.com>
Message-ID: <82a02b1e4dc8124c32adca3c222cce36.squirrel@squirrelmail.unbit.it>


> Hi Roberto,
>
> I use gevent-fastcgi for production. I wish to have the time to dive in
> the internals of uWSGI to understand how PATH_INFO and SCRIPT_NAME gets
> rewritten by uWSGI. I believe the implementation of --manage-script-name
> is incorrect and should be fixed to support having a empty or not set
> SCRIPT_NAME value.
>
> Etienne


Hi again, this is not what manage-script-name is for. It is a
uWSGI-specific option for when you only PATH_INFO and you want to generate
SCRIPT_NAME accordingly. This requires to "mount" apps under specific
paths.

uWSGI by itself does not rewrite the vars passed by the webserver, if you
want to do it you have to use internal routing accordingly (or use a WSGI
middleware that is generally more handy and portable, albeit annoying if
you already have an entry point)

-- 
Roberto De Ioris
http://unbit.com

From tkadm30 at yandex.com  Thu Jul 27 14:20:09 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Thu, 27 Jul 2017 10:20:09 -0400
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <82a02b1e4dc8124c32adca3c222cce36.squirrel@squirrelmail.unbit.it>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <9ed9825f-828e-d672-2381-9d933cac3ace@yandex.com>
 <7c415135f32036ca0bed7c1d534e93ae.squirrel@squirrelmail.unbit.it>
 <2f1e6037-a142-7ad7-f455-7e1b7314276e@yandex.com>
 <c7d16ce5f9152a2e03653ef638e4ae31.squirrel@squirrelmail.unbit.it>
 <62e042e9-0617-4ffe-0abd-3176ac226ceb@yandex.com>
 <82a02b1e4dc8124c32adca3c222cce36.squirrel@squirrelmail.unbit.it>
Message-ID: <fdce4591-c61d-cc5f-0698-5131c7564eda@yandex.com>

Hi Roberto,


Le 2017-07-27 ? 09:52, Roberto De Ioris a ?crit :
>
>> Hi again, this is not what manage-script-name is for. It is a
>> uWSGI-specific option for when you only PATH_INFO and you want to generate
>> SCRIPT_NAME accordingly. This requires to "mount" apps under specific
>> paths.
Ok. I really don't want to "mount" my app under a specific script name.
>> uWSGI by itself does not rewrite the vars passed by the webserver, if you
>> want to do it you have to use internal routing accordingly (or use a WSGI
>> middleware that is generally more handy and portable, albeit annoying if
>> you already have an entry point)
Nginx internal routing seems a reasonable choice here. My app is looking 
like this:

dispatch-django.uwsgi:

def application(environ, start_response):
       wsgi_app = make_app()
       return wsgi_app(environ, start_response)

Then I invoke uwsgi like so:
% uwsgi --socket localhost:8000 --wsgi-file dispatch-django.uwsgi 
--daemonize /var/log/uwsgi.log --enable-threads --workers 2
I think there's really something wrong with uWSGI internal routing 
system. The so-called "mountpoints" seems a little obscure to me. 
Perhaps the documentation should define a method to mount a dynamic 
django app without the need of --manage-script-name ?

Etienne

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From roberto at unbit.it  Thu Jul 27 15:25:05 2017
From: roberto at unbit.it (Roberto De Ioris)
Date: Thu, 27 Jul 2017 17:25:05 +0200
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <fdce4591-c61d-cc5f-0698-5131c7564eda@yandex.com>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <9ed9825f-828e-d672-2381-9d933cac3ace@yandex.com>
 <7c415135f32036ca0bed7c1d534e93ae.squirrel@squirrelmail.unbit.it>
 <2f1e6037-a142-7ad7-f455-7e1b7314276e@yandex.com>
 <c7d16ce5f9152a2e03653ef638e4ae31.squirrel@squirrelmail.unbit.it>
 <62e042e9-0617-4ffe-0abd-3176ac226ceb@yandex.com>
 <82a02b1e4dc8124c32adca3c222cce36.squirrel@squirrelmail.unbit.it>
 <fdce4591-c61d-cc5f-0698-5131c7564eda@yandex.com>
Message-ID: <8fbba656ee7506681c290a8339f9428c.squirrel@squirrelmail.unbit.it>


> Hi Roberto,
>
>
> Le 2017-07-27 ? 09:52, Roberto De Ioris a ?crit :
>>
>>> Hi again, this is not what manage-script-name is for. It is a
>>> uWSGI-specific option for when you only PATH_INFO and you want to
>>> generate
>>> SCRIPT_NAME accordingly. This requires to "mount" apps under specific
>>> paths.
> Ok. I really don't want to "mount" my app under a specific script name.
>>> uWSGI by itself does not rewrite the vars passed by the webserver, if
>>> you
>>> want to do it you have to use internal routing accordingly (or use a
>>> WSGI
>>> middleware that is generally more handy and portable, albeit annoying
>>> if
>>> you already have an entry point)
> Nginx internal routing seems a reasonable choice here. My app is looking
> like this:
>
> dispatch-django.uwsgi:
>
> def application(environ, start_response):
>        wsgi_app = make_app()
>        return wsgi_app(environ, start_response)
>
> Then I invoke uwsgi like so:
>

I was meaning uWSGI internal routing :)

http://uwsgi-docs.readthedocs.io/en/latest/InternalRouting.html

and yes, for manage-script-name to have effect you need to specify the
mountpoint of your app like described here:

http://uwsgi-docs.readthedocs.io/en/latest/Snippets.html#multiple-flask-apps-in-different-mountpoints

(it is flask app but it is the same concept)

-- 
Roberto De Ioris
http://unbit.com

From francis at daoine.org  Thu Jul 27 15:25:16 2017
From: francis at daoine.org (Francis Daly)
Date: Thu, 27 Jul 2017 16:25:16 +0100
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
Message-ID: <20170727152516.GQ365@daoine.org>

On Thu, Jul 27, 2017 at 07:45:28AM -0400, Etienne Robillard wrote:

Hi there,

> I'm not sure I understand the logic of this. Can someone please
> explain why the variable PATH_INFO is set to $document_uri in
> uwsgi_params?

My guess (without knowing the history):

The uwsgi_params values are an example of what can be done. The
combination of SCRIPT_NAME and PATH_INFO in that file is consistent, and
is valid in some cases (that is, cases where the application corresponds
to the "root" of the server).

No-one has reported that there is a problem and provided a fix.

In general, only the administrator knows what SCRIPT_NAME and PATH_INFO
values are appropriate in any one case, so that's the person who should
configure the two to match their particular case.


In the case of fastcgi, someone saw a possibly-similar issue and went
to the bother of adding a fastcgi_split_path_info directives which splits
the request uri string into two variables.

Perhaps it would be useful for something similar to exist for uwsgi? The
fact that it does not exist already suggests that no-one has enough of
a need, to arrange that it be added.

I guess that a server-level "if" could "set" two variables that could be
used to populate SCRIPT_NAME and PATH_INFO, until a uwsgi_split_path_info
directive is introduced. And that might be enough to avoid the need for
a dedicated directive.

(Actually: perhaps fastcgi_split_path_info can be used directly, even
in a location{} which does not do fastcgi_pass? It should be easy enough
to test whether that can work.)


The hard part is usually deciding what exactly is wanted. After that,
the implementation can be considered.

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org

From tkadm30 at yandex.com  Thu Jul 27 16:06:42 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Thu, 27 Jul 2017 12:06:42 -0400
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <20170727152516.GQ365@daoine.org>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <20170727152516.GQ365@daoine.org>
Message-ID: <f498f7a5-0b5b-69e5-06c2-5d78003c4594@yandex.com>

Hi all,

Le 2017-07-27 ? 11:25, Francis Daly a ?crit :
> On Thu, Jul 27, 2017 at 07:45:28AM -0400, Etienne Robillard wrote:
>
> Hi there,
>
>> I'm not sure I understand the logic of this. Can someone please
>> explain why the variable PATH_INFO is set to $document_uri in
>> uwsgi_params?
> My guess (without knowing the history):
>
> The uwsgi_params values are an example of what can be done. The
> combination of SCRIPT_NAME and PATH_INFO in that file is consistent, and
> is valid in some cases (that is, cases where the application corresponds
> to the "root" of the server).
Actually my problem is that SCRIPT_NAME is not defined in uwsgi_params. 
SCRIPT_NAME is a mandatory environment variable as per HTTP/1.1 spec.
>
> No-one has reported that there is a problem and provided a fix.
That's a major issue. A standard WSGI application should be able to use 
a uWSGI upstream server within the "/" location block without the need 
to define any "mountpoints".

> In general, only the administrator knows what SCRIPT_NAME and PATH_INFO
> values are appropriate in any one case, so that's the person who should
> configure the two to match their particular case.
>
Correct. Attempting to rewrite PATH_INFO as DOCUMENT_URI seems not a 
reliable solution for my simple use-case.

> Perhaps it would be useful for something similar to exist for uwsgi? The
> fact that it does not exist already suggests that no-one has enough of
> a need, to arrange that it be added.
A $uwsgi_script_name variable would be a worthy addition.
> I guess that a server-level "if" could "set" two variables that could be
> used to populate SCRIPT_NAME and PATH_INFO, until a uwsgi_split_path_info
> directive is introduced. And that might be enough to avoid the need for
> a dedicated directive.
>
> (Actually: perhaps fastcgi_split_path_info can be used directly, even
> in a location{} which does not do fastcgi_pass? It should be easy enough
> to test whether that can work.)
>
>
> The hard part is usually deciding what exactly is wanted. After that,
> the implementation can be considered.
>
> Good luck with it,
>
> 	f
Note that I don't use fastcgi_split_path_info. The only modification I 
did to fastcgi_params is:
fastcgi_param SCRIPT_NAME $fastcgi_script_name
fastcgi_param PATH_INFO $fastcgi_script_name

Etienne

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From roberto at unbit.it  Thu Jul 27 16:47:58 2017
From: roberto at unbit.it (Roberto De Ioris)
Date: Thu, 27 Jul 2017 18:47:58 +0200
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <f498f7a5-0b5b-69e5-06c2-5d78003c4594@yandex.com>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <20170727152516.GQ365@daoine.org>
 <f498f7a5-0b5b-69e5-06c2-5d78003c4594@yandex.com>
Message-ID: <880a4847029511357c5421e81a20882a.squirrel@squirrelmail.unbit.it>


> Hi all,
>
> Le 2017-07-27 ? 11:25, Francis Daly a ?crit :
>> On Thu, Jul 27, 2017 at 07:45:28AM -0400, Etienne Robillard wrote:
>>
>> Hi there,
>>
>>> I'm not sure I understand the logic of this. Can someone please
>>> explain why the variable PATH_INFO is set to $document_uri in
>>> uwsgi_params?
>> My guess (without knowing the history):
>>
>> The uwsgi_params values are an example of what can be done. The
>> combination of SCRIPT_NAME and PATH_INFO in that file is consistent, and
>> is valid in some cases (that is, cases where the application corresponds
>> to the "root" of the server).
> Actually my problem is that SCRIPT_NAME is not defined in uwsgi_params.
> SCRIPT_NAME is a mandatory environment variable as per HTTP/1.1 spec.
>>
>> No-one has reported that there is a problem and provided a fix.
> That's a major issue. A standard WSGI application should be able to use
> a uWSGI upstream server within the "/" location block without the need
> to define any "mountpoints".
>

Then just add

uwsgi_param SCRIPT_NAME "";

in uwsgi_params :)


-- 
Roberto De Ioris
http://unbit.com

From tkadm30 at yandex.com  Thu Jul 27 17:04:24 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Thu, 27 Jul 2017 13:04:24 -0400
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <880a4847029511357c5421e81a20882a.squirrel@squirrelmail.unbit.it>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <20170727152516.GQ365@daoine.org>
 <f498f7a5-0b5b-69e5-06c2-5d78003c4594@yandex.com>
 <880a4847029511357c5421e81a20882a.squirrel@squirrelmail.unbit.it>
Message-ID: <b19f2cf2-b0a3-fa58-1399-48a9b4194e6f@yandex.com>

Hi Roberto,

That is not effective. My app uses PATH_INFO to resolve a URL (ie: 
/blog/create/) to a callback function.

Could it be possible to just use :

uwsgi_param PATH_INFO $path_info

Assuming $path_info is the request_uri minus the location...

E

Le 2017-07-27 ? 12:47, Roberto De Ioris a ?crit :
> Then just add
>
> uwsgi_param SCRIPT_NAME "";
>
> in uwsgi_params :)
>

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From roberto at unbit.it  Thu Jul 27 17:45:06 2017
From: roberto at unbit.it (Roberto De Ioris)
Date: Thu, 27 Jul 2017 19:45:06 +0200
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <b19f2cf2-b0a3-fa58-1399-48a9b4194e6f@yandex.com>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <20170727152516.GQ365@daoine.org>
 <f498f7a5-0b5b-69e5-06c2-5d78003c4594@yandex.com>
 <880a4847029511357c5421e81a20882a.squirrel@squirrelmail.unbit.it>
 <b19f2cf2-b0a3-fa58-1399-48a9b4194e6f@yandex.com>
Message-ID: <8f04b508edc30eece7912cb537f14398.squirrel@squirrelmail.unbit.it>


> Hi Roberto,
>
> That is not effective. My app uses PATH_INFO to resolve a URL (ie:
> /blog/create/) to a callback function.
>
> Could it be possible to just use :
>
> uwsgi_param PATH_INFO $path_info
>
> Assuming $path_info is the request_uri minus the location...
>
> E
>

Nope, as already said there is no way to manage this directly in nginx
(and technically any 'automatic' management could not be so easy to be
'fair'). You have to rewrite PATH_INFO and SCRIPT_NAME in uWSGI itself
following the links i pasted before.

To be more clear:

nginx passes to uWSGI:

PATH_INFO = /foo/bar

uWSGI receives it and rewrite it as

SCRIPT_NAME = /foo
PATH_INFO = /bar

by using something like this (in the config, tune the regexps as required):

[uwsgi]
route = ^/(.+?)/(.+)$ setscriptname:/$1
route = ^/(.+?)/(.+)$ setpathinfo:/$2

-- 
Roberto De Ioris
http://unbit.com

From geraldjunkmail at gmail.com  Thu Jul 27 18:47:01 2017
From: geraldjunkmail at gmail.com (Gee Bunny)
Date: Thu, 27 Jul 2017 14:47:01 -0400
Subject: nginx 1.13 binaries
In-Reply-To: <CAFE-bHsDMQhoEAiuhMGftbrtiN-KUN8u4QD4OWxg2wQedJ_6ig@mail.gmail.com>
References: <CAFE-bHsDMQhoEAiuhMGftbrtiN-KUN8u4QD4OWxg2wQedJ_6ig@mail.gmail.com>
Message-ID: <CAFE-bHvK849Qo=BK6gAWtbOmD8uTYDhd5GjNMTVCBwSadeTJbA@mail.gmail.com>

There might be talk of binaries being available on the page you
linked, but if you check the repo link I provided (to the NGINX hosted
YUM Repository for CentOS 6 x86_64) there's no version 1.13 it goes as
high as nginx-1.12.1 only.

Don't take my word for it, see for yourself:

http://nginx.org/packages/centos/6/x86_64/RPMS/


>>> My guess: they are already there.

>>> See http://nginx.org/en/linux_packages.html



On Sat, Jul 22, 2017 at 9:45 PM, Gee Bunny <geraldjunkmail at gmail.com> wrote:

> Are there plans to add nginx 1.13 to any of the yum repositories?
>
> Like:
>
> http://nginx.org/packages/centos/6/x86_64/RPMS/
>
> Some of us would like to run the latest version of NGINX with TLS1.3
> support, without having to compile from source.
>
> Are there any plans or ETA to have nginx 1.13 aded to the repos within a
> timely fashion? Or is the only option to get TLS1.3 support to compile from
> source for the forseeable future?
>
> Thanks
>
> P.S. Awesome work on NGINX- best disruptive web-server tech to hit the
> scene in 20+ years.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170727/41dc10bc/attachment.html>

From tkadm30 at yandex.com  Thu Jul 27 21:40:54 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Thu, 27 Jul 2017 17:40:54 -0400
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <8f04b508edc30eece7912cb537f14398.squirrel@squirrelmail.unbit.it>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <20170727152516.GQ365@daoine.org>
 <f498f7a5-0b5b-69e5-06c2-5d78003c4594@yandex.com>
 <880a4847029511357c5421e81a20882a.squirrel@squirrelmail.unbit.it>
 <b19f2cf2-b0a3-fa58-1399-48a9b4194e6f@yandex.com>
 <8f04b508edc30eece7912cb537f14398.squirrel@squirrelmail.unbit.it>
Message-ID: <7cdae960-9322-d8c2-4dc1-b062723fe2cd@yandex.com>

Hi Roberto,

Le 2017-07-27 ? 13:45, Roberto De Ioris a ?crit :
>
> by using something like this (in the config, tune the regexps as required):
>
> [uwsgi]
> route = ^/(.+?)/(.+)$ setscriptname:/$1
> route = ^/(.+?)/(.+)$ setpathinfo:/$2
>
I use a dynamic regex URL resolver to resolve a request URI to a 
callback function. Also I prefer avoiding to modify uWSGI internal 
routing system for my basic use-case. I have no choice to fallback on 
using fastcgi until this is fixed either in nginx or uWSGI.

Etienne

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From paul.j.smith0 at gmail.com  Fri Jul 28 00:55:42 2017
From: paul.j.smith0 at gmail.com (Paul Smith)
Date: Thu, 27 Jul 2017 18:55:42 -0600
Subject: nginx 1.13 binaries
In-Reply-To: <CAFE-bHvK849Qo=BK6gAWtbOmD8uTYDhd5GjNMTVCBwSadeTJbA@mail.gmail.com>
References: <CAFE-bHsDMQhoEAiuhMGftbrtiN-KUN8u4QD4OWxg2wQedJ_6ig@mail.gmail.com>
 <CAFE-bHvK849Qo=BK6gAWtbOmD8uTYDhd5GjNMTVCBwSadeTJbA@mail.gmail.com>
Message-ID: <CACoaAtPhuFGNX6zbAb2kgDsQVcdhh11fEbDZUM-VYkTOPWPbsw@mail.gmail.com>

As Francis stated, you need to look for the mainline version, not the
stable version. They have different repositories.

http://nginx.org/packages/mainline/centos/6/x86_64/RPMS/


On Thu, Jul 27, 2017 at 12:47 PM, Gee Bunny <geraldjunkmail at gmail.com> wrote:
> There might be talk of binaries being available on the page you linked, but
> if you check the repo link I provided (to the NGINX hosted YUM Repository
> for CentOS 6 x86_64) there's no version 1.13 it goes as high as nginx-1.12.1
> only.
>
> Don't take my word for it, see for yourself:
>
> http://nginx.org/packages/centos/6/x86_64/RPMS/
>
>>>> My guess: they are already there.
>
>>>> See http://nginx.org/en/linux_packages.html
>
>
>
> On Sat, Jul 22, 2017 at 9:45 PM, Gee Bunny <geraldjunkmail at gmail.com> wrote:
>>
>> Are there plans to add nginx 1.13 to any of the yum repositories?
>>
>> Like:
>>
>> http://nginx.org/packages/centos/6/x86_64/RPMS/
>>
>> Some of us would like to run the latest version of NGINX with TLS1.3
>> support, without having to compile from source.
>>
>> Are there any plans or ETA to have nginx 1.13 aded to the repos within a
>> timely fashion? Or is the only option to get TLS1.3 support to compile from
>> source for the forseeable future?
>>
>> Thanks
>>
>> P.S. Awesome work on NGINX- best disruptive web-server tech to hit the
>> scene in 20+ years.
>
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

From nginx-forum at forum.nginx.org  Fri Jul 28 03:44:29 2017
From: nginx-forum at forum.nginx.org (m.i)
Date: Thu, 27 Jul 2017 23:44:29 -0400
Subject: Regarding URL Encode/Decode of the Parameter
Message-ID: <d6de1c1e6642bb21bed6dce8992e4394.NginxMailingListEnglish@forum.nginx.org>

Hello, I have a following problem with Nginx. Here is the scenario.

There is a client app that sends request to Nginx Proxy with following
parameter. The parameter is already url encoded.
GET /X/Y/Z.aspx?id=abc%3D%3D

Now, I noticed that Nginx applies url encode to % and make it as %25 when it
redirect the message. The parameter now become below.
GET /X/Y/Z.aspx?id=abc%253D%253D

I want Nginx to redirect the parameter as is. Below is the one that I want
Nginx to redirect.
GET /X/Y/Z.aspx?id=abc%3D%3D

Could you please, help me out to do what I want?

Any suggestions are very much welcome!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275704,275704#msg-275704


From gryzli.the.bugbear at gmail.com  Fri Jul 28 06:59:53 2017
From: gryzli.the.bugbear at gmail.com (Gryzli Bugbear)
Date: Fri, 28 Jul 2017 09:59:53 +0300
Subject: Accessing Nginx Cache mappings
Message-ID: <12b1f175-8ed8-cd0b-9be4-69bea263a8a7@gmail.com>

Hi guys,

I'm trying to implement cache-purger, which is using lua and separate 
VirtualHosts, to clean the cache for a certain site/location based on 
URI regex (similar to Varnish).

Is there some easy way (module/function or whatever), by which I could 
access the internal Nginx cache mapping from where I can get the file 
location for a certain cache key ?

Thanks in advance !



From francis at daoine.org  Fri Jul 28 07:17:04 2017
From: francis at daoine.org (Francis Daly)
Date: Fri, 28 Jul 2017 08:17:04 +0100
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <f498f7a5-0b5b-69e5-06c2-5d78003c4594@yandex.com>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <20170727152516.GQ365@daoine.org>
 <f498f7a5-0b5b-69e5-06c2-5d78003c4594@yandex.com>
Message-ID: <20170728071704.GR365@daoine.org>

On Thu, Jul 27, 2017 at 12:06:42PM -0400, Etienne Robillard wrote:
> Le 2017-07-27 ? 11:25, Francis Daly a ?crit :
> >On Thu, Jul 27, 2017 at 07:45:28AM -0400, Etienne Robillard wrote:

Hi there,

> >The uwsgi_params values are an example of what can be done. The
> >combination of SCRIPT_NAME and PATH_INFO in that file is consistent, and
> >is valid in some cases (that is, cases where the application corresponds
> >to the "root" of the server).

> Actually my problem is that SCRIPT_NAME is not defined in
> uwsgi_params. SCRIPT_NAME is a mandatory environment variable as per
> HTTP/1.1 spec.

?

HTTP/1.1 does not care about uwsgi, no?

I can see what appears to be the WSGI spec at
https://www.python.org/dev/peps/pep-0333/

My initial reading of that says that SCRIPT_NAME is optional.

Is there a uWSGI spec which says something different?

If you want SCRIPT_NAME set to a particular value, you can set SCRIPT_NAME
to a particular value in your config.

> >No-one has reported that there is a problem and provided a fix.

> That's a major issue. A standard WSGI application should be able to
> use a uWSGI upstream server within the "/" location block without
> the need to define any "mountpoints".

So configure your nginx to send whatever parameters your uWSGI upstream
expects for your WSGI application to work the way you want it to.

> >In general, only the administrator knows what SCRIPT_NAME and PATH_INFO
> >values are appropriate in any one case, so that's the person who should
> >configure the two to match their particular case.
> 
> Correct. Attempting to rewrite PATH_INFO as DOCUMENT_URI seems not a
> reliable solution for my simple use-case.

The defaults do not work for your use case.

What settings do you want for your use case?

> >(Actually: perhaps fastcgi_split_path_info can be used directly, even
> >in a location{} which does not do fastcgi_pass? It should be easy enough
> >to test whether that can work.)

> Note that I don't use fastcgi_split_path_info. The only modification
> I did to fastcgi_params is:
> fastcgi_param SCRIPT_NAME $fastcgi_script_name
> fastcgi_param PATH_INFO $fastcgi_script_name

What I meant there was that you could possibly use fastcgi_split_path_info
to define how you want your $request_uri to be split into parts for your
SCRIPT_NAME and PATH_INFO as uwsgi_param values.

So your eventual config could include

  uwsgi_param SCRIPT_NAME $fastcgi_script_name;
  uwsgi_param PATH_INFO $fastcgi_path_info;

after you have defined the first directive appropriately.

It all comes down to: for one specific http request, what values do
you want SCRIPT_NAME and PATH_INFO to have when they are sent to the
uwsgi upstream?

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org

From francis at daoine.org  Fri Jul 28 07:22:39 2017
From: francis at daoine.org (Francis Daly)
Date: Fri, 28 Jul 2017 08:22:39 +0100
Subject: nginx 1.13 binaries
In-Reply-To: <CAFE-bHvK849Qo=BK6gAWtbOmD8uTYDhd5GjNMTVCBwSadeTJbA@mail.gmail.com>
References: <CAFE-bHsDMQhoEAiuhMGftbrtiN-KUN8u4QD4OWxg2wQedJ_6ig@mail.gmail.com>
 <CAFE-bHvK849Qo=BK6gAWtbOmD8uTYDhd5GjNMTVCBwSadeTJbA@mail.gmail.com>
Message-ID: <20170728072239.GS365@daoine.org>

On Thu, Jul 27, 2017 at 02:47:01PM -0400, Gee Bunny wrote:

Hi there,

> There might be talk of binaries being available on the page you
> linked, but if you check the repo link I provided (to the NGINX hosted
> YUM Repository for CentOS 6 x86_64) there's no version 1.13 it goes as
> high as nginx-1.12.1 only.
> 
> Don't take my word for it, see for yourself:
> 
> http://nginx.org/packages/centos/6/x86_64/RPMS/

If you read http://nginx.org/en/linux_packages.html, you will see that
there are two sets of repositories listed.

One is at http://nginx.org/packages/centos/6/x86_64/RPMS/

The other is at http://nginx.org/packages/mainline/centos/6/x86_64/RPMS/

	f
-- 
Francis Daly        francis at daoine.org

From roberto at unbit.it  Fri Jul 28 07:25:34 2017
From: roberto at unbit.it (Roberto De Ioris)
Date: Fri, 28 Jul 2017 09:25:34 +0200
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <7cdae960-9322-d8c2-4dc1-b062723fe2cd@yandex.com>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <20170727152516.GQ365@daoine.org>
 <f498f7a5-0b5b-69e5-06c2-5d78003c4594@yandex.com>
 <880a4847029511357c5421e81a20882a.squirrel@squirrelmail.unbit.it>
 <b19f2cf2-b0a3-fa58-1399-48a9b4194e6f@yandex.com>
 <8f04b508edc30eece7912cb537f14398.squirrel@squirrelmail.unbit.it>
 <7cdae960-9322-d8c2-4dc1-b062723fe2cd@yandex.com>
Message-ID: <78f3077429412174cd7c9bd6f69af63b.squirrel@squirrelmail.unbit.it>


> Hi Roberto,
>
> Le 2017-07-27 ? 13:45, Roberto De Ioris a ?crit :
>>
>> by using something like this (in the config, tune the regexps as
>> required):
>>
>> [uwsgi]
>> route = ^/(.+?)/(.+)$ setscriptname:/$1
>> route = ^/(.+?)/(.+)$ setpathinfo:/$2
>>
> I use a dynamic regex URL resolver to resolve a request URI to a
> callback function. Also I prefer avoiding to modify uWSGI internal
> routing system for my basic use-case.


Sorry, but someone (nginx or uWSGI) must be informed of your needs (like
Francis explained yesterday). What you consider a 'basic use-case', it is
absolutely not in this specific scenario where you want to map some form
of namespace (like SCRIPT_NAME). The 'basic use case' here is the app
mounted under the root. It is only a lucky coincidence that the fastcgi
hack exposed by nginx (that came from a php need) has been handy for you.
You could even use uWSGI in fastcgi mode. Maybe you have been confused by
the 'uwsgi' name of the protocol (yes, unfortunate choice made 8 years ago
:( ). It has nothing to do with the python WSGI standard, it is only a
transport protocol. It is the backend app that gives meaning to data.

I have no choice to fallback on
> using fastcgi until this is fixed either in nginx or uWSGI.


frankly speaking, there is nothing to fix in uWSGI because it already
honours what the webserver passes to it and eventually (via routing or
code) it is able to rewrite it if the webserver cannot make assumptions. I
have shown at least 3 ways to deal with this, i have literally no idea of
other solutions :)

and honestly, i do not think nginx should add ad-hoc use cases to the
uwsgi protocol (that is a simple dictionary serializer protocol and should
never interpret its content). But obviously this second part is up to the
nginx developers ;)


-- 
Roberto De Ioris
http://unbit.com

From francis at daoine.org  Fri Jul 28 08:02:33 2017
From: francis at daoine.org (Francis Daly)
Date: Fri, 28 Jul 2017 09:02:33 +0100
Subject: Regarding URL Encode/Decode of the Parameter
In-Reply-To: <d6de1c1e6642bb21bed6dce8992e4394.NginxMailingListEnglish@forum.nginx.org>
References: <d6de1c1e6642bb21bed6dce8992e4394.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170728080233.GT365@daoine.org>

On Thu, Jul 27, 2017 at 11:44:29PM -0400, m.i wrote:

Hi there,

> Now, I noticed that Nginx applies url encode to % and make it as %25 when it
> redirect the message. The parameter now become below.
> GET /X/Y/Z.aspx?id=abc%253D%253D
> 
> I want Nginx to redirect the parameter as is. Below is the one that I want
> Nginx to redirect.
> GET /X/Y/Z.aspx?id=abc%3D%3D

What configuration do you have that shows the behaviour that you report?

("redirect" might mean one of many things. "return 301 $uri$is_args$args;"
does not appear to url-encode anything special when I test it.)

	f
-- 
Francis Daly        francis at daoine.org

From nginx-forum at forum.nginx.org  Fri Jul 28 08:30:33 2017
From: nginx-forum at forum.nginx.org (m.i)
Date: Fri, 28 Jul 2017 04:30:33 -0400
Subject: Regarding URL Encode/Decode of the Parameter
In-Reply-To: <20170728080233.GT365@daoine.org>
References: <20170728080233.GT365@daoine.org>
Message-ID: <1999abd53d3b98a3888af347c709367a.NginxMailingListEnglish@forum.nginx.org>

Hello, thank you so much for your post.
Below is the configuration I'm using. It's pretty much straight forward.
I just want to transfer the massage using "proxy_pass."

--------------------
server {
  listen       8881;
  server_name localhost;

    location / {
        proxy_pass http://111.111.111.111;
    }
 }
--------------------

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275704,275710#msg-275710


From francis at daoine.org  Fri Jul 28 08:56:17 2017
From: francis at daoine.org (Francis Daly)
Date: Fri, 28 Jul 2017 09:56:17 +0100
Subject: Regarding URL Encode/Decode of the Parameter
In-Reply-To: <1999abd53d3b98a3888af347c709367a.NginxMailingListEnglish@forum.nginx.org>
References: <20170728080233.GT365@daoine.org>
 <1999abd53d3b98a3888af347c709367a.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170728085617.GU365@daoine.org>

On Fri, Jul 28, 2017 at 04:30:33AM -0400, m.i wrote:

Hi there,

> --------------------
> server {
>   listen       8881;
>   server_name localhost;
> 
>     location / {
>         proxy_pass http://111.111.111.111;
>     }
>  }
> --------------------

When I use this (proxy_pass:ing to an address:port I control), I can do

  curl -i http://127.0.0.1:8881/X/Y/Z.aspx?id=abc%3D%3D

and I see the proxy_pass traffic is "GET /X/Y/Z.aspx?id=abc%3D%3D".

What, specifically, do you do to see something else?

Make so that I can repeat your test case and see the same result.

Perhaps I am using a different nginx version from you.

Perhaps the problem you see does not come from nginx.

	f
-- 
Francis Daly        francis at daoine.org

From smntov at gmail.com  Fri Jul 28 11:13:23 2017
From: smntov at gmail.com (ST)
Date: Fri, 28 Jul 2017 14:13:23 +0300
Subject: redirect related questions...
Message-ID: <1501240403.1638.111.camel@gmail.com>

Hello,

I have several questions related to redirects:

Here is my setup:

server {
 server_name www.example.org example.com; # and some more domains
 return 301 $scheme://example.org$request_uri;
}

server {
 listen 80;
 server_name example.org;
 ...
 if ($http_user_agent !~ facebookexternalhit/1.1) {
  return 301 https://$host$request_uri;
 }
}

server {
 listen 443 ssl;
 server_name example.org;
 ...
}

1. http://example.com redirects correctly to https://example.org (via
http://example.org), but not https://example.com - why?

2. neither http://www.example.org nor https://www.example.org redirect
to https://example.org (not even to http://example.org) - why?

How can I achieve that?

Thank you in advance!


From nginx-forum at forum.nginx.org  Fri Jul 28 22:55:25 2017
From: nginx-forum at forum.nginx.org (m.i)
Date: Fri, 28 Jul 2017 18:55:25 -0400
Subject: Regarding URL Encode/Decode of the Parameter
In-Reply-To: <20170728085617.GU365@daoine.org>
References: <20170728085617.GU365@daoine.org>
Message-ID: <b47d73804bb98f26785499d1443dad16.NginxMailingListEnglish@forum.nginx.org>

As I invetsgate more, as you pointed out, I found Nginx is working as
expected. 
It does not apply url encoding to what already has been url encoded. So,
Nginx is good. I mean, great!

We had another proxy before Nginx and it was making the url encoding issue.
Thank you for pointing out for me to see other causes.

So, again, Nginx is great!

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275704,275720#msg-275720


From nginx-forum at forum.nginx.org  Sat Jul 29 09:35:08 2017
From: nginx-forum at forum.nginx.org (lamsung)
Date: Sat, 29 Jul 2017 05:35:08 -0400
Subject: Workers CPU leak [epoll_wait,epoll_ctl]
Message-ID: <9f143d02ca74ab59ceb626751177ac9c.NginxMailingListEnglish@forum.nginx.org>

Hello, I have strange issuses with nginx workers. For some time after start
Nginx I notice that some process of workers cause high load to CPU (
principally sys CPU). 

At first I've got syscall traces from one of such process: 

futex(0x157d914, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x157d910, {FUTEX_OP_SET, 0,
FUTEX_OP_CMP_GT, 1}) = 1 
epoll_wait(38, {{EPOLLIN, {u32=7156096, u64=7156096}}}, 512, -1) = 1 
epoll_ctl(38, EPOLL_CTL_ADD, 178, {EPOLLOUT|EPOLLET, {u32=3888102096,
u64=140028411886288}}) = 0 
epoll_wait(38, {{EPOLLOUT, {u32=3888102096, u64=140028411886288}}}, 512, -1)
= 1 
epoll_ctl(38, EPOLL_CTL_DEL, 178, 7ffda2bc7f30) = 0 
futex(0x157d914, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x157d910, {FUTEX_OP_SET, 0,
FUTEX_OP_CMP_GT, 1}) = 1 
epoll_wait(38, {{EPOLLIN, {u32=7156096, u64=7156096}}}, 512, -1) = 1 
epoll_ctl(38, EPOLL_CTL_ADD, 178, {EPOLLOUT|EPOLLET, {u32=3888102096,
u64=140028411886288}}) = 0 
epoll_wait(38, {{EPOLLOUT, {u32=3888102096, u64=140028411886288}}}, 512, -1)
= 1 
epoll_ctl(38, EPOLL_CTL_DEL, 178, 7ffda2bc7f30) = 0 
futex(0x157d914, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x157d910, {FUTEX_OP_SET, 0,
FUTEX_OP_CMP_GT, 1}) = 1 
futex(0x157d8d0, FUTEX_WAKE_PRIVATE, 1) = 1 

epoll_wait, epoll_ctl, futex are repeated circularly. 

Then I've got lsof of process and see who owns of 38 file descriptor: 

nginx 18862 www 38u a_inode 0,9 0 6 [eventpoll] 

also I see several CLOSE_WAIT sockets 

nginx 18862 www 101u IPv4 85643376 0t0 TCP
154.59.82.194:http->105.107.179.210:24519 (CLOSE_WAIT) 
nginx 18862 www 133r REG 8,3 0 4743 /mnt/ssd1/wwwroot/71/7/27394667.mp4
(deleted) 
nginx 18862 www 178u IPv4 86054929 0t0 TCP
154.59.82.194:http->5adc98ed.bb.sky.com:45665 (CLOSE_WAIT) 
nginx 18862 www 179r REG 8,3 0 5098 /mnt/ssd1/wwwroot/21/9/29603499.mp4
(deleted) 


Nginx has such version and modules: 

nginx version: nginx/1.9.11 
built with OpenSSL 1.0.2f 28 Jan 2016 
TLS SNI support enabled 
configure arguments: --prefix=/usr --conf-path=/etc/nginx/nginx.conf
--error-log-path=/var/log/nginx/error_log --pid-path=/run/nginx.pid
--lock-path=/run/lock/nginx.lock --with-cc-opt=-I/usr/include
--with-ld-opt=-L/usr/lib64 --http-log-path=/var/log/nginx/access_log
--http-client-body-temp-path=/var/lib/nginx/tmp/client
--http-proxy-temp-path=/var/lib/nginx/tmp/proxy
--http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi
--http-scgi-temp-path=/var/lib/nginx/tmp/scgi
--http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --with-file-aio --with-ipv6
--with-pcre --with-threads --without-http_autoindex_module
--without-http_fastcgi_module --without-http_geo_module
--without-http_limit_req_module --without-http_limit_conn_module
--without-http_memcached_module --without-http_uwsgi_module
--with-http_flv_module --with-http_gzip_static_module --with-http_mp4_module
--with-http_perl_module
--add-module=external_module/headers-more-nginx-module-0.261
--add-module=external_module/ngx_estreaming_module-0.01
--add-module=external_module/ngx_slice_module-0.01 --with-http_ssl_module
--without-mail_imap_module --without-mail_pop3_module
--without-mail_smtp_module --user='www --group=www' 

and using for video streaming. 

Has anyone encountered such behavior ? Help please.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275721,275721#msg-275721


From tkadm30 at yandex.com  Sat Jul 29 10:09:01 2017
From: tkadm30 at yandex.com (Etienne Robillard)
Date: Sat, 29 Jul 2017 06:09:01 -0400
Subject: Problem with uWSGI and PATH_INFO
In-Reply-To: <20170728071704.GR365@daoine.org>
References: <3f77e732-0400-9859-5048-8651147da727@yandex.com>
 <20170727152516.GQ365@daoine.org>
 <f498f7a5-0b5b-69e5-06c2-5d78003c4594@yandex.com>
 <20170728071704.GR365@daoine.org>
Message-ID: <e28a0a0d-ce3b-9a40-5f61-1c5ca2ab6436@yandex.com>

Hi Francis,


Le 2017-07-28 ? 03:17, Francis Daly a ?crit :
>
> What I meant there was that you could possibly use fastcgi_split_path_info
> to define how you want your $request_uri to be split into parts for your
> SCRIPT_NAME and PATH_INFO as uwsgi_param values.
>
> So your eventual config could include
>
>    uwsgi_param SCRIPT_NAME $fastcgi_script_name;
>    uwsgi_param PATH_INFO $fastcgi_path_info;
>
> after you have defined the first directive appropriately.
>
> It all comes down to: for one specific http request, what values do
> you want SCRIPT_NAME and PATH_INFO to have when they are sent to the
> uwsgi upstream?
>
>
I'll try your workaround. I want nginx to manage SCRIPT_NAME and 
PATH_INFO in FastCGI mode, and uWSGI to act as a FastCGI handler.

Etienne

-- 
Etienne Robillard
tkadm30 at yandex.com
http://www.isotopesoftware.ca/


From francis at daoine.org  Sat Jul 29 19:25:03 2017
From: francis at daoine.org (Francis Daly)
Date: Sat, 29 Jul 2017 20:25:03 +0100
Subject: redirect related questions...
In-Reply-To: <1501240403.1638.111.camel@gmail.com>
References: <1501240403.1638.111.camel@gmail.com>
Message-ID: <20170729192503.GW365@daoine.org>

On Fri, Jul 28, 2017 at 02:13:23PM +0300, ST wrote:

Hi there,

> server {
>  server_name www.example.org example.com; # and some more domains
>  return 301 $scheme://example.org$request_uri;
> }
> 
> server {
>  listen 80;
>  server_name example.org;
>  ...
>  if ($http_user_agent !~ facebookexternalhit/1.1) {
>   return 301 https://$host$request_uri;
>  }
> }
> 
> server {
>  listen 443 ssl;
>  server_name example.org;
>  ...
> }

If that is your config, then the first server{} is used for http
connections for everything except example.org; the second server is used
for http connections for only example.org; and the third server is used
for all https connections.

> 1. http://example.com redirects correctly to https://example.org (via
> http://example.org), but not https://example.com - why?

https goes to server{} three; you have no redirection there.

> 2. neither http://www.example.org nor https://www.example.org redirect
> to https://example.org (not even to http://example.org) - why?

https won't anyway, as per question 1.

http would, but only if the request actually gets to nginx. What do the
nginx logs say? Does www.example.org resolve to an address on the nginx
server, as far as this client is concerned?

> How can I achieve that?

See why it fails right now.

If the request does not get to nginx, change things outside nginx so
that the request does get to nginx.

If the request does get to nginx, change things inside nginx so that it
does what you want.

That probably involves no change for http, but might involve a new server
for https which is the default server, and which does the redirect that
you want. Note that the client may choose not to accept the (redirect)
response if the certificate does not match whatever name they used to
connect to the server.

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org

From mdounin at mdounin.ru  Sat Jul 29 21:13:25 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sun, 30 Jul 2017 00:13:25 +0300
Subject: Accessing Nginx Cache mappings
In-Reply-To: <12b1f175-8ed8-cd0b-9be4-69bea263a8a7@gmail.com>
References: <12b1f175-8ed8-cd0b-9be4-69bea263a8a7@gmail.com>
Message-ID: <20170729211324.GP93611@mdounin.ru>

Hello!

On Fri, Jul 28, 2017 at 09:59:53AM +0300, Gryzli Bugbear wrote:

> Hi guys,
> 
> I'm trying to implement cache-purger, which is using lua and separate 
> VirtualHosts, to clean the cache for a certain site/location based on 
> URI regex (similar to Varnish).
> 
> Is there some easy way (module/function or whatever), by which I could 
> access the internal Nginx cache mapping from where I can get the file 
> location for a certain cache key ?

No.

-- 
Maxim Dounin
http://nginx.org/

From mdounin at mdounin.ru  Sat Jul 29 22:59:10 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sun, 30 Jul 2017 01:59:10 +0300
Subject: Workers CPU leak [epoll_wait,epoll_ctl]
In-Reply-To: <9f143d02ca74ab59ceb626751177ac9c.NginxMailingListEnglish@forum.nginx.org>
References: <9f143d02ca74ab59ceb626751177ac9c.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <20170729225910.GR93611@mdounin.ru>

Hello!

On Sat, Jul 29, 2017 at 05:35:08AM -0400, lamsung wrote:

> Hello, I have strange issuses with nginx workers. For some time after start
> Nginx I notice that some process of workers cause high load to CPU (
> principally sys CPU). 

[...]

> Nginx has such version and modules: 
> 
> nginx version: nginx/1.9.11 
> built with OpenSSL 1.0.2f 28 Jan 2016 
> TLS SNI support enabled 
> configure arguments: --prefix=/usr --conf-path=/etc/nginx/nginx.conf
> --error-log-path=/var/log/nginx/error_log --pid-path=/run/nginx.pid
> --lock-path=/run/lock/nginx.lock --with-cc-opt=-I/usr/include
> --with-ld-opt=-L/usr/lib64 --http-log-path=/var/log/nginx/access_log
> --http-client-body-temp-path=/var/lib/nginx/tmp/client
> --http-proxy-temp-path=/var/lib/nginx/tmp/proxy
> --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi
> --http-scgi-temp-path=/var/lib/nginx/tmp/scgi
> --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi --with-file-aio --with-ipv6
> --with-pcre --with-threads --without-http_autoindex_module
> --without-http_fastcgi_module --without-http_geo_module
> --without-http_limit_req_module --without-http_limit_conn_module
> --without-http_memcached_module --without-http_uwsgi_module
> --with-http_flv_module --with-http_gzip_static_module --with-http_mp4_module
> --with-http_perl_module
> --add-module=external_module/headers-more-nginx-module-0.261
> --add-module=external_module/ngx_estreaming_module-0.01
> --add-module=external_module/ngx_slice_module-0.01 --with-http_ssl_module
> --without-mail_imap_module --without-mail_pop3_module
> --without-mail_smtp_module --user='www --group=www' 
> 
> and using for video streaming. 
> 
> Has anyone encountered such behavior ? Help please.

An obvious first step would be to try to reproduce the problem 
without 3rd party modules.

Trying something newer than 1.9.11 is also recommended - it is not 
supported since release of nginx 1.9.12 on 24 Feb 2016.  Current 
versions are 1.13.3 (mainline) and 1.12.1 (stable).

-- 
Maxim Dounin
http://nginx.org/

From mdounin at mdounin.ru  Sat Jul 29 23:47:58 2017
From: mdounin at mdounin.ru (Maxim Dounin)
Date: Sun, 30 Jul 2017 02:47:58 +0300
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
 <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
 <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>
Message-ID: <20170729234758.GU93611@mdounin.ru>

Hello!

On Wed, Jul 26, 2017 at 06:05:28PM +0200, Vlad K. wrote:

> On 2017-07-26 13:36, Peter Booth wrote:
> > Vlad,
> > 
> > I'd suggest beginning by seeing whether or not this is real. If you
> > create a cron job that invokes netstat -ant every hour, then summarize
> > the connections and either view them manually or write them into an
> > influxdb and graph with grafana you will see whether or not the #tcp
> > connections really is growing and, if so, which connections are
> > growing.
> 
> Thanks for the suggestion, but with it slow progression and low 
> signal-to-noise ration in comparison with the daily and weekly 
> connection cycle, I'm not sure it would be practically possible to 
> measure it like that. But your suggestion gave me another idea, to 
> record IPs of established tcp conns every 5 minutes and then see which 
> ones remain constant.
> 
> But are you suggesting that nginx status is reporting wrong/fake 
> numbers? What do you mean by "real"?
> 
> And what about connections that are staying in "CLOSED" state until I 
> restart or reload nginx?

Connections in "CLOSED" state are likely leaked sockets.  On a 
reload nginx should write appropriate alerts to error log, saying 
something like "open socket #<fd> left in connection <n>".  These 
alerts are logged in appropriate connection numbers, and details 
of a particular connection can be traced through debug log.

It might not be trivial to debug such socket leaks though, and 
before doing anything else it is in general a good idea to:

- make sure you are using latest nginx version, and

- the problem is not in a 3rd party module (that is, you can 
  reproduce it without 3rd party modules).

-- 
Maxim Dounin
http://nginx.org/

From nginx-forum at forum.nginx.org  Sun Jul 30 02:16:36 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Sat, 29 Jul 2017 22:16:36 -0400
Subject: Buffering issues with nginx
In-Reply-To: <20170724190555.GO365@daoine.org>
References: <20170724190555.GO365@daoine.org>
Message-ID: <4ab091925575d137451cae3f06cc6911.NginxMailingListEnglish@forum.nginx.org>

It looks like for localhost buffers are bigger, but even if it's not local
host I do get 5MB stuck in socket buffers. I was only able to get perfect
results by writing my own proxy in c and doing some obscure nodejs code to
avoid buffering.

In any case, if nginx does not provide a way to control sockets buffer I
cannot use it. For example, 'X-Accel-Buffering: no' supposedly disables
caching (I didn't see any effect of that anyways), so I wanted to add some
kind of headers to be able to tell nginx what buffers to set per connection.
In my case I do regular reverse proxy stuff with nginx, but on certain
connections I need exact control and nginx doesn't provide any of that.
haproxy for example worked much better for me, but it's sndbuf/rcvbuf
settings are global, which is equally unacceptable.

Would that be easy to add headers like these X-Accel-Up-Rcvbuf: 12345,
X-Accel-Down-Sndbuf: 4567 so that on getting them from upstream nginx would
configure sockets that are used by that connection?

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275729#msg-275729


From nginx-forum at forum.nginx.org  Sun Jul 30 02:41:25 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Sat, 29 Jul 2017 22:41:25 -0400
Subject: Buffering issues with nginx
In-Reply-To: <4ab091925575d137451cae3f06cc6911.NginxMailingListEnglish@forum.nginx.org>
References: <20170724190555.GO365@daoine.org>
 <4ab091925575d137451cae3f06cc6911.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <e51131797c0476f5a3f5f7af9aa49b99.NginxMailingListEnglish@forum.nginx.org>

In nginx docs I see sndbuf option in listen directive.
Is there something that I don't understand about it, or developers of nginx
don't understand meaning of sndbuf... but I do not see a point to set sndbuf
on a listening socket. It just does not make any sense!
sndbuf/rcvbuf is needed perhaps for an upstream proxy connection, or for a
connected socket (from outside), but it just have no meaning to set that on
a listening socket. Total nonsense.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275730#msg-275730


From nginx-ml at acheronmedia.hr  Sun Jul 30 08:09:28 2017
From: nginx-ml at acheronmedia.hr (Vlad K.)
Date: Sun, 30 Jul 2017 10:09:28 +0200
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <20170729234758.GU93611@mdounin.ru>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
 <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
 <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>
 <20170729234758.GU93611@mdounin.ru>
Message-ID: <fd44ba612aec5fc8bf4c9ba5512dc485@acheronmedia.hr>

On 2017-07-30 01:47, Maxim Dounin wrote:
> 
> It might not be trivial to debug such socket leaks though, and
> before doing anything else it is in general a good idea to:
> 
> - make sure you are using latest nginx version, and
> 
> - the problem is not in a 3rd party module (that is, you can
>   reproduce it without 3rd party modules).

It's latest stable, 1.12.1 on FreeBSD.

Unfortunately I can't remove 3rd party modules as this is production. I 
have no idea what to do to try replicate that in testing.

But thanks for your reply.



-- 
Vlad K.

From smntov at gmail.com  Sun Jul 30 09:12:03 2017
From: smntov at gmail.com (ST)
Date: Sun, 30 Jul 2017 12:12:03 +0300
Subject: redirect related questions...
In-Reply-To: <20170729192503.GW365@daoine.org>
References: <1501240403.1638.111.camel@gmail.com>
 <20170729192503.GW365@daoine.org>
Message-ID: <1501405923.1638.117.camel@gmail.com>

Hi Francis,

thank you for the detailed answer...
I tried to take care of the first problem by doing this:


server {
 listen 80;
 listen 443 ssl;
 server_name www.example.org example.com; # and some more domains
 return 301 https://example.org$request_uri;
}

But the site stopped working all together, both http and https once
checked with curl say:
curl: (35) Unknown SSL protocol error in connection to
www.example.org:443

Why? Is it wrong to have two listen directives in one server?

Thank you!

On Sat, 2017-07-29 at 20:25 +0100, Francis Daly wrote:
> On Fri, Jul 28, 2017 at 02:13:23PM +0300, ST wrote:
> 
> Hi there,
> 
> > server {
> >  server_name www.example.org example.com; # and some more domains
> >  return 301 $scheme://example.org$request_uri;
> > }
> > 
> > server {
> >  listen 80;
> >  server_name example.org;
> >  ...
> >  if ($http_user_agent !~ facebookexternalhit/1.1) {
> >   return 301 https://$host$request_uri;
> >  }
> > }
> > 
> > server {
> >  listen 443 ssl;
> >  server_name example.org;
> >  ...
> > }
> 
> If that is your config, then the first server{} is used for http
> connections for everything except example.org; the second server is used
> for http connections for only example.org; and the third server is used
> for all https connections.
> 
> > 1. http://example.com redirects correctly to https://example.org (via
> > http://example.org), but not https://example.com - why?
> 
> https goes to server{} three; you have no redirection there.
> 
> > 2. neither http://www.example.org nor https://www.example.org redirect
> > to https://example.org (not even to http://example.org) - why?
> 
> https won't anyway, as per question 1.
> 
> http would, but only if the request actually gets to nginx. What do the
> nginx logs say? Does www.example.org resolve to an address on the nginx
> server, as far as this client is concerned?
> 
> > How can I achieve that?
> 
> See why it fails right now.
> 
> If the request does not get to nginx, change things outside nginx so
> that the request does get to nginx.
> 
> If the request does get to nginx, change things inside nginx so that it
> does what you want.
> 
> That probably involves no change for http, but might involve a new server
> for https which is the default server, and which does the redirect that
> you want. Note that the client may choose not to accept the (redirect)
> response if the certificate does not match whatever name they used to
> connect to the server.
> 
> Good luck with it,
> 
> 	f


From peter_booth at me.com  Sun Jul 30 09:15:25 2017
From: peter_booth at me.com (Peter Booth)
Date: Sun, 30 Jul 2017 05:15:25 -0400
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <fd44ba612aec5fc8bf4c9ba5512dc485@acheronmedia.hr>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
 <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
 <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>
 <20170729234758.GU93611@mdounin.ru>
 <fd44ba612aec5fc8bf4c9ba5512dc485@acheronmedia.hr>
Message-ID: <28B07F43-04D5-48B6-9A67-74B1E7B79481@me.com>

Vlad,

You might not need to replicate it- you have it happening in production in front of you.
Some questions:

1. When is the last time that your production nginx was restarted?
2. Do you have regular restarts?
3. Is there an obstacle to restarting at some point?
4. Is this a single instance or do you have multiple nginx hosts?
5. What 3rd party models are you using?
6. Is the website in question an enterprise app or something that is internet visible? 

Maxim?s hypothesis of leaking sockets from third party plugin is the simplest, most likely explanation for what you report.

I start from a position of trusting nothing. If you can you capture the output of lsof -i :80 or net stat -ant | grep TCP or a 
similar ss command you can know for certain that your visualization is ?telling the truth?
Certainly the line labeled ?Writing? looks unusual. Do you know of any site events that might have caused the minimum on
23 July, the spike on 24th, and the step up on 25th July?

Peter 




> On Jul 30, 2017, at 4:09 AM, Vlad K. <nginx-ml at acheronmedia.hr> wrote:
> 
> On 2017-07-30 01:47, Maxim Dounin wrote:
>> It might not be trivial to debug such socket leaks though, and
>> before doing anything else it is in general a good idea to:
>> - make sure you are using latest nginx version, and
>> - the problem is not in a 3rd party module (that is, you can
>>  reproduce it without 3rd party modules).
> 
> It's latest stable, 1.12.1 on FreeBSD.
> 
> Unfortunately I can't remove 3rd party modules as this is production. I have no idea what to do to try replicate that in testing.
> 
> But thanks for your reply.
> 
> 
> 
> -- 
> Vlad K.
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/f94e14a7/attachment-0001.html>

From peter_booth at me.com  Sun Jul 30 09:26:52 2017
From: peter_booth at me.com (Peter Booth)
Date: Sun, 30 Jul 2017 05:26:52 -0400
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <28B07F43-04D5-48B6-9A67-74B1E7B79481@me.com>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
 <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
 <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>
 <20170729234758.GU93611@mdounin.ru>
 <fd44ba612aec5fc8bf4c9ba5512dc485@acheronmedia.hr>
 <28B07F43-04D5-48B6-9A67-74B1E7B79481@me.com>
Message-ID: <51A15DBE-7FB8-4860-9330-3148CF059911@me.com>



I just reread the thread and realize that you answered q2, and that makes the graph even more
surprising. You say that it son FreeBSD - does this mean that you don?t have /proc available to you?
Is there a procstat or other way to see the equivalent of /proc/<pid>/fd - a list of all open file descriptions for a specific pid?



> On Jul 30, 2017, at 5:15 AM, Peter Booth <peter_booth at me.com> wrote:
> 
> Vlad,
> 
> You might not need to replicate it- you have it happening in production in front of you.
> Some questions:
> 
> 1. When is the last time that your production nginx was restarted?
> 2. Do you have regular restarts?
> 3. Is there an obstacle to restarting at some point?
> 4. Is this a single instance or do you have multiple nginx hosts?
> 5. What 3rd party models are you using?
> 6. Is the website in question an enterprise app or something that is internet visible? 
> 
> Maxim?s hypothesis of leaking sockets from third party plugin is the simplest, most likely explanation for what you report.
> 
> I start from a position of trusting nothing. If you can you capture the output of lsof -i :80 or net stat -ant | grep TCP or a 
> similar ss command you can know for certain that your visualization is ?telling the truth?
> Certainly the line labeled ?Writing? looks unusual. Do you know of any site events that might have caused the minimum on
> 23 July, the spike on 24th, and the step up on 25th July?
> 
> Peter 
> 
> 
> 
> 
>> On Jul 30, 2017, at 4:09 AM, Vlad K. <nginx-ml at acheronmedia.hr <mailto:nginx-ml at acheronmedia.hr>> wrote:
>> 
>> On 2017-07-30 01:47, Maxim Dounin wrote:
>>> It might not be trivial to debug such socket leaks though, and
>>> before doing anything else it is in general a good idea to:
>>> - make sure you are using latest nginx version, and
>>> - the problem is not in a 3rd party module (that is, you can
>>>  reproduce it without 3rd party modules).
>> 
>> It's latest stable, 1.12.1 on FreeBSD.
>> 
>> Unfortunately I can't remove 3rd party modules as this is production. I have no idea what to do to try replicate that in testing.
>> 
>> But thanks for your reply.
>> 
>> 
>> 
>> -- 
>> Vlad K.
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/b312bcd2/attachment.html>

From smntov at gmail.com  Sun Jul 30 09:34:45 2017
From: smntov at gmail.com (ST)
Date: Sun, 30 Jul 2017 12:34:45 +0300
Subject: redirect related questions...
In-Reply-To: <20170729192503.GW365@daoine.org>
References: <1501240403.1638.111.camel@gmail.com>
 <20170729192503.GW365@daoine.org>
Message-ID: <1501407285.1638.119.camel@gmail.com>

PS:

actually merely adding "listen 443 ssl;" to the first server causes the
same error (curl: (35) Unknown SSL protocol error in connection to
www.example.org:443)

server {
 listen 443 ssl;
 server_name www.example.org example.com; # and some more domains
 return 301 https://example.org$request_uri;
}

Why? nginx restarts normally... is there any conflicts in such a setup
with other 2 servers?

Thank you!

---------------------------------------

Hi Francis,

thank you for the detailed answer...
I tried to take care of the first problem by doing this:


server {
 listen 80;
 listen 443 ssl;
 server_name www.example.org example.com; # and some more domains
 return 301 https://example.org$request_uri;
}

But the site stopped working all together, both http and https once
checked with curl say:
curl: (35) Unknown SSL protocol error in connection to
www.example.org:443

Why? Is it wrong to have two listen directives in one server?

Thank you!

On Sat, 2017-07-29 at 20:25 +0100, Francis Daly wrote:
> On Fri, Jul 28, 2017 at 02:13:23PM +0300, ST wrote:
> 
> Hi there,
> 
> > server {
> >  server_name www.example.org example.com; # and some more domains
> >  return 301 $scheme://example.org$request_uri;
> > }
> > 
> > server {
> >  listen 80;
> >  server_name example.org;
> >  ...
> >  if ($http_user_agent !~ facebookexternalhit/1.1) {
> >   return 301 https://$host$request_uri;
> >  }
> > }
> > 
> > server {
> >  listen 443 ssl;
> >  server_name example.org;
> >  ...
> > }
> 
> If that is your config, then the first server{} is used for http
> connections for everything except example.org; the second server is used
> for http connections for only example.org; and the third server is used
> for all https connections.
> 
> > 1. http://example.com redirects correctly to https://example.org (via
> > http://example.org), but not https://example.com - why?
> 
> https goes to server{} three; you have no redirection there.
> 
> > 2. neither http://www.example.org nor https://www.example.org redirect
> > to https://example.org (not even to http://example.org) - why?
> 
> https won't anyway, as per question 1.
> 
> http would, but only if the request actually gets to nginx. What do the
> nginx logs say? Does www.example.org resolve to an address on the nginx
> server, as far as this client is concerned?
> 
> > How can I achieve that?
> 
> See why it fails right now.
> 
> If the request does not get to nginx, change things outside nginx so
> that the request does get to nginx.
> 
> If the request does get to nginx, change things inside nginx so that it
> does what you want.
> 
> That probably involves no change for http, but might involve a new server
> for https which is the default server, and which does the redirect that
> you want. Note that the client may choose not to accept the (redirect)
> response if the certificate does not match whatever name they used to
> connect to the server.
> 
> Good luck with it,
> 
> 	f



From smntov at gmail.com  Sun Jul 30 09:56:55 2017
From: smntov at gmail.com (ST)
Date: Sun, 30 Jul 2017 12:56:55 +0300
Subject: redirect related questions...
In-Reply-To: <20170729192503.GW365@daoine.org>
References: <1501240403.1638.111.camel@gmail.com>
 <20170729192503.GW365@daoine.org>
Message-ID: <1501408615.1638.124.camel@gmail.com>

PPS:

my fault: there is no ssl key info so obviously it should not work. At
least for those server name listed inside first server{} (strange is
that https://example.org - server{} three also stops working...)

Is it a good idea to use DNS forwarding in order not to obtain/install
ssl keys for example.com as we don't plan to use it? This should make
redirection faster and requires no setup on nginx... Are there any down
sides of such a solution?

Thank you!

---------------------------------------

PS:

actually merely adding "listen 443 ssl;" to the first server causes the
same error (curl: (35) Unknown SSL protocol error in connection to
www.example.org:443)

server {
 listen 443 ssl;
 server_name www.example.org example.com; # and some more domains
 return 301 https://example.org$request_uri;
}

Why? nginx restarts normally... is there any conflicts in such a setup
with other 2 servers?

Thank you!

---------------------------------------

Hi Francis,

thank you for the detailed answer...
I tried to take care of the first problem by doing this:


server {
 listen 80;
 listen 443 ssl;
 server_name www.example.org example.com; # and some more domains
 return 301 https://example.org$request_uri;
}

But the site stopped working all together, both http and https once
checked with curl say:
curl: (35) Unknown SSL protocol error in connection to
www.example.org:443

Why? Is it wrong to have two listen directives in one server?

Thank you!

On Sat, 2017-07-29 at 20:25 +0100, Francis Daly wrote:
> On Fri, Jul 28, 2017 at 02:13:23PM +0300, ST wrote:
> 
> Hi there,
> 
> > server {
> >  server_name www.example.org example.com; # and some more domains
> >  return 301 $scheme://example.org$request_uri;
> > }
> > 
> > server {
> >  listen 80;
> >  server_name example.org;
> >  ...
> >  if ($http_user_agent !~ facebookexternalhit/1.1) {
> >   return 301 https://$host$request_uri;
> >  }
> > }
> > 
> > server {
> >  listen 443 ssl;
> >  server_name example.org;
> >  ...
> > }
> 
> If that is your config, then the first server{} is used for http
> connections for everything except example.org; the second server is used
> for http connections for only example.org; and the third server is used
> for all https connections.
> 
> > 1. http://example.com redirects correctly to https://example.org (via
> > http://example.org), but not https://example.com - why?
> 
> https goes to server{} three; you have no redirection there.
> 
> > 2. neither http://www.example.org nor https://www.example.org redirect
> > to https://example.org (not even to http://example.org) - why?
> 
> https won't anyway, as per question 1.
> 
> http would, but only if the request actually gets to nginx. What do the
> nginx logs say? Does www.example.org resolve to an address on the nginx
> server, as far as this client is concerned?
> 
> > How can I achieve that?
> 
> See why it fails right now.
> 
> If the request does not get to nginx, change things outside nginx so
> that the request does get to nginx.
> 
> If the request does get to nginx, change things inside nginx so that it
> does what you want.
> 
> That probably involves no change for http, but might involve a new server
> for https which is the default server, and which does the redirect that
> you want. Note that the client may choose not to accept the (redirect)
> response if the certificate does not match whatever name they used to
> connect to the server.
> 
> Good luck with it,
> 
> 	f




From nginx-ml at acheronmedia.hr  Sun Jul 30 10:12:53 2017
From: nginx-ml at acheronmedia.hr (Vlad K.)
Date: Sun, 30 Jul 2017 12:12:53 +0200
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <51A15DBE-7FB8-4860-9330-3148CF059911@me.com>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
 <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
 <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>
 <20170729234758.GU93611@mdounin.ru>
 <fd44ba612aec5fc8bf4c9ba5512dc485@acheronmedia.hr>
 <28B07F43-04D5-48B6-9A67-74B1E7B79481@me.com>
 <51A15DBE-7FB8-4860-9330-3148CF059911@me.com>
Message-ID: <bf86c6a8ca3b75097207cf2080bf502a@acheronmedia.hr>

On 2017-07-30 11:26, Peter Booth wrote:
> I just reread the thread and realize that you answered q2, and that
> makes the graph even more
> surprising. You say that it son FreeBSD - does this mean that you
> don?t have /proc available to you?
> Is there a procstat or other way to see the equivalent of
> /proc/<pid>/fd - a list of all open file descriptions for a specific
> pid?

procfs is technically available but I'm not enabling it. We do have 
sockstat and fstat though that basically can show the same. I have 
checked and I don't see any corresponding open connections, that's why I 
was wondering about how to list what nginx thinks are connections being 
Written to, so I could find what those connections did last in the 
access log.

I ran a periodic netstat, sorted the IPs (not removing src ports), and 
ran a diff from previous run, keeping only those that aren't +/-. Over 
some period of time, the only connections that stayed open for long were 
actively using the roundcube webmail which (especially with our 
keepalive and http2 enabled) runs a request every minute. But the number 
of those did not correspond to the number reported as Writing. Plus 
these connections have daily highs and downs. "Writing" in the nginx 
status does not correlate with the rest.


> 1. When is the last time that your production nginx was restarted?

In my first post in this thread is a link to the graph (by Munin) where 
it shows a dip in Writing, that's where I restarted. Reloading doesn't 
change the number of Writing reported by nginx.


> 2. Do you have regular restarts?

No, just regular reloads.


> 3. Is there an obstacle to restarting at some point?

If you're asking me if I can restart nginx to check something, I can do 
that.


> 4. Is this a single instance or do you have multiple nginx hosts?

Single instance, one master and one worker thread, in a jail, and 
there's no other jail running nginx on the server.


> 5. What 3rd party models are you using?

These are the options/modules ENABLED for the port:

      DSO=on: Enable dynamic modules support
      FILE_AIO=on: Enable file aio
      HTTP=on: Enable HTTP module
      HTTPV2=on: Enable HTTP/2 protocol support (SSL req.)
      HTTP_ADDITION=on: Enable http_addition module
      HTTP_AUTH_REQ=on: Enable http_auth_request module
      HTTP_CACHE=on: Enable http_cache module
      HTTP_DAV=on: Enable http_webdav module
      HTTP_FLV=on: Enable http_flv module
      HTTP_GUNZIP_FILTER=on: Enable http_gunzip_filter module
      HTTP_GZIP_STATIC=on: Enable http_gzip_static module
      HTTP_MP4=on: Enable http_mp4 module
      HTTP_RANDOM_INDEX=on: Enable http_random_index module
      HTTP_REALIP=on: Enable http_realip module
      HTTP_REWRITE=on: Enable http_rewrite module
      HTTP_SECURE_LINK=on: Enable http_secure_link module
      HTTP_SLICE=on: Enable http_slice module
      HTTP_SSL=on: Enable http_ssl module
      HTTP_STATUS=on: Enable http_stub_status module
      HTTP_SUB=on: Enable http_sub module
      IPV6=on: Enable IPv6 support
      MAIL=on: Enable IMAP4/POP3/SMTP proxy module
      MAIL_IMAP=on: Enable IMAP4 proxy module
      MAIL_POP3=on: Enable POP3 proxy module
      MAIL_SMTP=on: Enable SMTP proxy module
      MAIL_SSL=on: Enable mail_ssl module
      STREAM=on: Enable stream module
      STREAM_SSL=on: Enable stream_ssl module (SSL req.)
      STREAM_SSL_PREREAD=on: Enable stream_ssl_preread module (SSL req.)
      THREADS=on: Enable threads support
      WWW=on: Enable html sample files

Which is all default for the port, except I also enabled MAIL_* ones as 
I'll be needing some mail proxying. But at the moment I have no mail {} 
blocks defined. Looking at these I guess I could trim down defaults. Who 
needs FLV nowadays :)

I'm not sure which are or aren't 3rd party, but if the descriptions are 
fully correct, then it looks like I'm not using any "3rd party" ones, 
because we have options that explicitly state when a module id "3rd 
party" (and the part I'm not sure is if all are listed as such).

However, it's also compiled with DSO=on, and with the above options, it 
produces these:

      /usr/local/libexec/nginx/ngx_mail_module.so
      /usr/local/libexec/nginx/ngx_stream_module.so

None of which I've loaded at the moment.


> 6. Is the website in question an enterprise app or something that is 
> internet visible?

The nginx jail is serving numerous PHP sites and a Python web app, each 
in their own jails. Using fastcgi for php-fpm and uwsgi for python, all 
over tcp connections between jails.



-- 
Vlad K.

From smntov at gmail.com  Sun Jul 30 11:09:11 2017
From: smntov at gmail.com (ST)
Date: Sun, 30 Jul 2017 14:09:11 +0300
Subject: serving certain file for all but one server{}
Message-ID: <1501412951.1638.128.camel@gmail.com>

Hello,

I have a lot of server{}s with different server_names all over my setup.
I want to serve robots_closed.txt instead of robot.txt for all but one
domain. And for that one domain, let's say example.org, I want robot.txt
to served normally. What is the right way to achieve that without
writing an explicit redirect in all the server{}s? 

Thank you!


From peter_booth at me.com  Sun Jul 30 11:30:25 2017
From: peter_booth at me.com (Peter Booth)
Date: Sun, 30 Jul 2017 07:30:25 -0400
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <bf86c6a8ca3b75097207cf2080bf502a@acheronmedia.hr>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
 <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
 <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>
 <20170729234758.GU93611@mdounin.ru>
 <fd44ba612aec5fc8bf4c9ba5512dc485@acheronmedia.hr>
 <28B07F43-04D5-48B6-9A67-74B1E7B79481@me.com>
 <51A15DBE-7FB8-4860-9330-3148CF059911@me.com>
 <bf86c6a8ca3b75097207cf2080bf502a@acheronmedia.hr>
Message-ID: <2BA185F9-7C0F-4624-B6EA-AF311A012210@me.com>

See below


> On Jul 30, 2017, at 6:12 AM, Vlad K. <nginx-ml at acheronmedia.hr> wrote:
> 
> On 2017-07-30 11:26, Peter Booth wrote:
>> I just reread the thread and realize that you answered q2, and that
>> makes the graph even more
>> surprising. You say that it son FreeBSD - does this mean that you
>> don?t have /proc available to you?
>> Is there a procstat or other way to see the equivalent of
>> /proc/<pid>/fd - a list of all open file descriptions for a specific
>> pid?
> 
> procfs is technically available but I'm not enabling it. We do have sockstat and fstat though that basically can show the same. I have checked and I don't see any corresponding open connections, that's why I was wondering about how to list what nginx thinks are connections being Written to, so I could find what those connections did last in the access log.

It appears that you have  a lot of data that could help in this analysis.
How frequently is the status page being queried? Does every status datapoint get recorded
or is munin showing some rolled up rrd  data?



> 
> I ran a periodic netstat, sorted the IPs (not removing src ports), and ran a diff from previous run, keeping only those that aren't +/-. Over some period of time, the only connections that stayed open for long were actively using the roundcube webmail which (especially with our keepalive and http2 enabled) runs a request every minute. But the number of those did not correspond to the number reported as Writing. Plus these connections have daily highs and downs. "Writing" in the nginx status does not correlate with the rest.

If you open the status page in a browser do the numbers report match what you see with netstat?


> 
> 
>> 1. When is the last time that your production nginx was restarted?
> 
> In my first post in this thread is a link to the graph (by Munin) where it shows a dip in Writing, that's where I restarted. Reloading doesn't change the number of Writing reported by nginx.

Thats what I thought. I think that the graph looks weird. Over the time interval [18 July, 23 July] the time series
 labelled  ?writing? connections increases almost monotonically from approx 5 to 15. Then the nginx is restarted
and the graph jumps back unto about 12/13 and continues writing again. Do you have a hypothesis that explains 
why the graph could jump back to 12/13, rather than spend a few days increasing linearly in the way it did from 
the 18th to the 23rd?

How long was nginx down for? If you graph only the ?writing? variable for just 23rd July does the length of 
time that the # of writing connections is thoughtto be 0 make sense?

I wonder whether what you are seeing could be a side-effect of the server being in a FreeBSD jail?
I haven?t used FreeBSD. My understanding is that FreeBSD jails are more than just chroots, similar to Solaris Zones
or OpenVZ on Linux. Does each jail have separate, independent sysctl settings/ulimits?
Do any of the other nginx sites in other jails exhibit the same behavior?
In FreeBSD jails is there an equivalent of Dom) in a XEN hypervisor? A parent or root OS?
If so, do you see all connections on al jails the you log into it? If wondering if you are hitting some ulimit or 
resource shortage on the host as a whole?

> 
> 
>> 2. Do you have regular restarts?
> 
> No, just regular reloads.
> 
> 
>> 3. Is there an obstacle to restarting at some point?
> 
> If you're asking me if I can restart nginx to check something, I can do that.
> 
> 
>> 4. Is this a single instance or do you have multiple nginx hosts?
> 
> Single instance, one master and one worker thread, in a jail, and there's no other jail running nginx on the server.
> 
> 
>> 5. What 3rd party models are you using?
> 
> These are the options/modules ENABLED for the port:
> 
>     DSO=on: Enable dynamic modules support
>     FILE_AIO=on: Enable file aio
>     HTTP=on: Enable HTTP module
>     HTTPV2=on: Enable HTTP/2 protocol support (SSL req.)
>     HTTP_ADDITION=on: Enable http_addition module
>     HTTP_AUTH_REQ=on: Enable http_auth_request module
>     HTTP_CACHE=on: Enable http_cache module
>     HTTP_DAV=on: Enable http_webdav module
>     HTTP_FLV=on: Enable http_flv module
>     HTTP_GUNZIP_FILTER=on: Enable http_gunzip_filter module
>     HTTP_GZIP_STATIC=on: Enable http_gzip_static module
>     HTTP_MP4=on: Enable http_mp4 module
>     HTTP_RANDOM_INDEX=on: Enable http_random_index module
>     HTTP_REALIP=on: Enable http_realip module
>     HTTP_REWRITE=on: Enable http_rewrite module
>     HTTP_SECURE_LINK=on: Enable http_secure_link module
>     HTTP_SLICE=on: Enable http_slice module
>     HTTP_SSL=on: Enable http_ssl module
>     HTTP_STATUS=on: Enable http_stub_status module
>     HTTP_SUB=on: Enable http_sub module
>     IPV6=on: Enable IPv6 support
>     MAIL=on: Enable IMAP4/POP3/SMTP proxy module
>     MAIL_IMAP=on: Enable IMAP4 proxy module
>     MAIL_POP3=on: Enable POP3 proxy module
>     MAIL_SMTP=on: Enable SMTP proxy module
>     MAIL_SSL=on: Enable mail_ssl module
>     STREAM=on: Enable stream module
>     STREAM_SSL=on: Enable stream_ssl module (SSL req.)
>     STREAM_SSL_PREREAD=on: Enable stream_ssl_preread module (SSL req.)
>     THREADS=on: Enable threads support
>     WWW=on: Enable html sample files
> 
> Which is all default for the port, except I also enabled MAIL_* ones as I'll be needing some mail proxying. But at the moment I have no mail {} blocks defined. Looking at these I guess I could trim down defaults. Who needs FLV nowadays :)
> 
> I'm not sure which are or aren't 3rd party, but if the descriptions are fully correct, then it looks like I'm not using any "3rd party" ones, because we have options that explicitly state when a module id "3rd party" (and the part I'm not sure is if all are listed as such).
> 
> However, it's also compiled with DSO=on, and with the above options, it produces these:
> 
>     /usr/local/libexec/nginx/ngx_mail_module.so
>     /usr/local/libexec/nginx/ngx_stream_module.so
> 
> None of which I've loaded at the moment.
> 
> 
>> 6. Is the website in question an enterprise app or something that is internet visible?
> 
> The nginx jail is serving numerous PHP sites and a Python web app, each in their own jails. Using fastcgi for php-fpm and uwsgi for python, all over tcp connections between jails.
> 
> 
> 
> -- 
> Vlad K.
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/e8d483d5/attachment.html>

From zchao1995 at gmail.com  Sun Jul 30 11:34:49 2017
From: zchao1995 at gmail.com (Zhang Chao)
Date: Sun, 30 Jul 2017 04:34:49 -0700
Subject: serving certain file for all but one server{}
In-Reply-To: <1501412951.1638.128.camel@gmail.com>
References: <1501412951.1638.128.camel@gmail.com>
Message-ID: <CAPr95Bhjw2FgLSLZdLEFOVTg4zzRt30h08xsLrDzuWUxGymhAQ@mail.gmail.com>

Hi!

You can rewrite the uri in the special server {} by the ?rewrite? directive.


On 30 July 2017 at 19:09:27, ST (smntov at gmail.com) wrote:

Hello,

I have a lot of server{}s with different server_names all over my setup.
I want to serve robots_closed.txt instead of robot.txt for all but one
domain. And for that one domain, let's say example.org, I want robot.txt
to served normally. What is the right way to achieve that without
writing an explicit redirect in all the server{}s?

Thank you!

_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/94988347/attachment-0001.html>

From smntov at gmail.com  Sun Jul 30 11:48:05 2017
From: smntov at gmail.com (ST)
Date: Sun, 30 Jul 2017 14:48:05 +0300
Subject: serving certain file for all but one server{}
In-Reply-To: <CAPr95Bhjw2FgLSLZdLEFOVTg4zzRt30h08xsLrDzuWUxGymhAQ@mail.gmail.com>
References: <1501412951.1638.128.camel@gmail.com>
 <CAPr95Bhjw2FgLSLZdLEFOVTg4zzRt30h08xsLrDzuWUxGymhAQ@mail.gmail.com>
Message-ID: <1501415285.1638.130.camel@gmail.com>

That is the problem - the special server{} (example.org) should serve
robot.txt as is. Those are all the other server{}s that need a redirect
from /robot.txt to /robot_closed.txt. Are there other ways to do that
except for explicit redirect inside of all those server{}s?


On Sun, 2017-07-30 at 04:34 -0700, Zhang Chao wrote:
> Hi!
> 
> 
> You can rewrite the uri in the special server {} by the ?rewrite?
> directive.
> 
> 
> 
> On 30 July 2017 at 19:09:27, ST (smntov at gmail.com) wrote:
> 
> > 
> > Hello, 
> > 
> > I have a lot of server{}s with different server_names all over my
> > setup. 
> > I want to serve robots_closed.txt instead of robot.txt for all but
> > one 
> > domain. And for that one domain, let's say example.org, I want
> > robot.txt 
> > to served normally. What is the right way to achieve that without 
> > writing an explicit redirect in all the server{}s? 
> > 
> > Thank you! 
> > 
> > _______________________________________________ 
> > nginx mailing list 
> > nginx at nginx.org 
> > http://mailman.nginx.org/mailman/listinfo/nginx 
> > 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx


From nginx-ml at acheronmedia.hr  Sun Jul 30 13:31:51 2017
From: nginx-ml at acheronmedia.hr (Vlad K.)
Date: Sun, 30 Jul 2017 15:31:51 +0200
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <2BA185F9-7C0F-4624-B6EA-AF311A012210@me.com>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
 <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
 <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>
 <20170729234758.GU93611@mdounin.ru>
 <fd44ba612aec5fc8bf4c9ba5512dc485@acheronmedia.hr>
 <28B07F43-04D5-48B6-9A67-74B1E7B79481@me.com>
 <51A15DBE-7FB8-4860-9330-3148CF059911@me.com>
 <bf86c6a8ca3b75097207cf2080bf502a@acheronmedia.hr>
 <2BA185F9-7C0F-4624-B6EA-AF311A012210@me.com>
Message-ID: <bc68510d4be54ec23db774f09762d958@acheronmedia.hr>



On 2017-07-30 13:30, Peter Booth wrote:
> 
> It appears that you have  a lot of data that could help in this
> analysis.
> How frequently is the status page being queried? Does every status
> datapoint get recorded
> or is munin showing some rolled up rrd  data?

The nginx status page is queried every 5 minutes (default Munin polling 
time), and it stores raw metrics into rrd database. But munin is not 
imporant in this issue. I get the same values if I query the status page 
directly.


> If you open the status page in a browser do the numbers report match
> what you see with netstat?

Waiting does:

# netstat -n | grep -E "tcp4|tcp6" | grep ESTABLISHED | wc -l \
   && echo "----------------------------" \
   && fetch -qo - http://10.0.0.4/nginx_status

       82
----------------------------
Active connections: 89
server accepts handled requests
  669843 669843 3158515
Reading: 0 Writing: 22 Waiting: 82

And I ran it a few times with several minutes in between, the above is 
just an example from the last run. This is inside the nginx jail, so 
grepping tcp4|tcp6 shows only connections to the nginx server.

Now, the part I don't quite understand is whether Active = Reading + 
Writing + Waiting. The above certainly doesn't seem to suggest so.



> Do you have a hypothesis that explains
> why the graph could jump back to 12/13, rather than spend a few days
> increasing linearly in the way it did from
> the 18th to the 23rd?

Bots crawling the sites, pacing themselves over a longer time frame so 
there's no correlation to daily sinusoid caused by live visitors. We do 
have a lot of resources on all those sites to crawl through. They're all 
real estate agency sites, and there are tens of thousands of pages with 
hundreds of thousands of images. And looking at the logs, quite a number 
of requests from bots (that are decent enough to say they're bots).

We've deviated a bit into assuming this is a bug or some unexpected 
behavior (my fault for suggesting it in the beginning). That's why all I 
wanted to do was to check which IPs are those that nginx considers 
"Writing" to. The only reason this caught my attention was apparently 
"flat" appearance of Writing, but now thinking about bots, this could be 
quite normal.


> How long was nginx down for? If you graph only the ?writing?
> variable for just 23rd July does the length of
> time that the # of writing connections is thoughtto be 0 make sense?

It was only restarted. It appears the "offending" connections started 
showing up less than an hour later.


> 
> I wonder whether what you are seeing could be a side-effect of the
> server being in a FreeBSD jail?

I doubt it. I used to see this when the server was on Debian Jessie, but 
it was much less noticeable. Then again, back then we had much less 
traffic and much less content.



> Do any of the other nginx sites in other jails exhibit the same
> behavior?

There is only one instance of nginx running on the server. Individual 
sites are only runing php-fpm or uwsgi in their jails.


> In FreeBSD jails is there an equivalent of Dom) in a XEN hypervisor? A
> parent or root OS?

FreeBSD jails are OS-level virtualization. It's basically similar to 
containers on Linux but with more isolation (it's not just namespacing).


> If so, do you see all connections on al jails the you log into it? If
> wondering if you are hitting some ulimit or
> resource shortage on the host as a whole?

I don't think it's that, as limits are far above the current demands for 
traffic, and there's nothing logged about potential resource exhaustion.



Thanks for helping me figure this out.


-- 
Vlad K.

From vbart at nginx.com  Sun Jul 30 14:15:05 2017
From: vbart at nginx.com (Valentin V. Bartenev)
Date: Sun, 30 Jul 2017 17:15:05 +0300
Subject: Buffering issues with nginx
In-Reply-To: <e51131797c0476f5a3f5f7af9aa49b99.NginxMailingListEnglish@forum.nginx.org>
References: <20170724190555.GO365@daoine.org>
 <4ab091925575d137451cae3f06cc6911.NginxMailingListEnglish@forum.nginx.org>
 <e51131797c0476f5a3f5f7af9aa49b99.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <13669415.3dim1n3r2K@vbart-laptop>

On Saturday 29 July 2017 22:41:25 Dan34 wrote:
> In nginx docs I see sndbuf option in listen directive.
> Is there something that I don't understand about it, or developers of nginx
> don't understand meaning of sndbuf... but I do not see a point to set sndbuf
> on a listening socket. It just does not make any sense!
> sndbuf/rcvbuf is needed perhaps for an upstream proxy connection, or for a
> connected socket (from outside), but it just have no meaning to set that on
> a listening socket. Total nonsense.
> 

When nginx calls accept(), SO_RCVBUF/SO_SNDBUF options are inherited from the 
listening socket.

  wbr, Valentin V. Bartenev


From nginx-forum at forum.nginx.org  Sun Jul 30 18:03:08 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Sun, 30 Jul 2017 14:03:08 -0400
Subject: Buffering issues with nginx
In-Reply-To: <13669415.3dim1n3r2K@vbart-laptop>
References: <13669415.3dim1n3r2K@vbart-laptop>
Message-ID: <e22ee200c21884e18213d75a92876f37.NginxMailingListEnglish@forum.nginx.org>

Yes, I tested that and it appears to be the case. However, I don't see where
nginx sets rcvbuf on the upstream socket as this one cannot be inherited.
Somehow even with SND/RCV buffers set to low values and buffering disabled I
get around 2.5BM stuck on nginx side. With my own simple proxy I get perfect
results: when socket buffers are low there is no data accumulated on the
proxy side.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275744#msg-275744


From peter_booth at me.com  Sun Jul 30 18:03:20 2017
From: peter_booth at me.com (Peter Booth)
Date: Sun, 30 Jul 2017 14:03:20 -0400
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <bc68510d4be54ec23db774f09762d958@acheronmedia.hr>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
 <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
 <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>
 <20170729234758.GU93611@mdounin.ru>
 <fd44ba612aec5fc8bf4c9ba5512dc485@acheronmedia.hr>
 <28B07F43-04D5-48B6-9A67-74B1E7B79481@me.com>
 <51A15DBE-7FB8-4860-9330-3148CF059911@me.com>
 <bf86c6a8ca3b75097207cf2080bf502a@acheronmedia.hr>
 <2BA185F9-7C0F-4624-B6EA-AF311A012210@me.com>
 <bc68510d4be54ec23db774f09762d958@acheronmedia.hr>
Message-ID: <68305643-F820-4FC0-A14F-0864DBFCCF66@me.com>

During a busier part of the day,  what is your minimum, median,99%, max requests per sec?

> On Jul 30, 2017, at 9:31 AM, Vlad K. <nginx-ml at acheronmedia.hr> wrote:
> 
> 
>> If you open the status page in a browser do the numbers report match
>> what you see with netstat?
> 
> Waiting does:
> 
> # netstat -n | grep -E "tcp4|tcp6" | grep ESTABLISHED | wc -l \
>  && echo "----------------------------" \
>  && fetch -qo - http://10.0.0.4/nginx_status
> 
>      82
> ----------------------------
> Active connections: 89
> server accepts handled requests
> 669843 669843 3158515
> Reading: 0 Writing: 22 Waiting: 82
> 
> And I ran it a few times with several minutes in between, the above is just an example from the last run. This is inside the nginx jail, so grepping tcp4|tcp6 shows only connections to the nginx server.
> 
> Now, the part I don't quite understand is whether Active = Reading + Writing + Waiting. The above certainly doesn't seem to suggest so.


So when you look at two different documentation pages explaining the status page,
 they both show that Active = Reading + Writing + Waiting

https://www.cyberciti.biz/faq/nginx-enable-and-see-current-status-page/ <https://www.cyberciti.biz/faq/nginx-enable-and-see-current-status-page/>
https://www.keycdn.com/support/nginx-status/ <https://www.keycdn.com/support/nginx-status/>


I think that suggests that, in your environment, Writing = ?really writing? + ?leaked sockets that nginx thinks are writing"

I?m pretty confident that this is a bug, because of the shape of the graph. There's no obvious healthy explanation for the 
number of writing connections to increase over days and return to its current value after a restart.


> 
> 
> 
>> Do you have a hypothesis that explains
>> why the graph could jump back to 12/13, rather than spend a few days
>> increasing linearly in the way it did from
>> the 18th to the 23rd?
> 
> Bots crawling the sites, pacing themselves over a longer time frame so there's no correlation to daily sinusoid caused by live visitors. We do have a lot of resources on all those sites to crawl through. They're all real estate agency sites, and there are tens of thousands of pages with hundreds of thousands of images. And looking at the logs, quite a number of requests from bots (that are decent enough to say they're bots).


Last public site I worked on had approx 40% of requests from bots or spiders (including our own active testing) and only 1/2 of the 400 user agents that weren?t interactive browsers actually identified themselves. many pretended to be browsers, and might have been scripted browsers, but were easy to identify because of the pattern software the URLS they requested.


> We've deviated a bit into assuming this is a bug or some unexpected behavior (my fault for suggesting it in the beginning). That's why all I wanted to do was to check which IPs are those that nginx considers "Writing" to. The only reason this caught my attention was apparently "flat" appearance of Writing, but now thinking about bots, this could be quite normal.







> 
> 
>> How long was nginx down for? If you graph only the ?writing?
>> variable for just 23rd July does the length of
>> time that the # of writing connections is thoughtto be 0 make sense?
> 
> It was only restarted. It appears the "offending" connections started showing up less than an hour later.
> 
> 
>> I wonder whether what you are seeing could be a side-effect of the
>> server being in a FreeBSD jail?
> 
> I doubt it. I used to see this when the server was on Debian Jessie, but it was much less noticeable. Then again, back then we had much less traffic and much less content.
> 
> 
> 
>> Do any of the other nginx sites in other jails exhibit the same
>> behavior?
> 
> There is only one instance of nginx running on the server. Individual sites are only runing php-fpm or uwsgi in their jails.
> 
> 
>> In FreeBSD jails is there an equivalent of Dom) in a XEN hypervisor? A
>> parent or root OS?
> 
> FreeBSD jails are OS-level virtualization. It's basically similar to containers on Linux but with more isolation (it's not just namespacing).
> 
> 
>> If so, do you see all connections on al jails the you log into it? If
>> wondering if you are hitting some ulimit or
>> resource shortage on the host as a whole?
> 
> I don't think it's that, as limits are far above the current demands for traffic, and there's nothing logged about potential resource exhaustion.
> 
> 
> 
> Thanks for helping me figure this out.
> 
> 
> -- 
> Vlad K.
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/c5f954d5/attachment.html>

From francis at daoine.org  Sun Jul 30 18:26:15 2017
From: francis at daoine.org (Francis Daly)
Date: Sun, 30 Jul 2017 19:26:15 +0100
Subject: serving certain file for all but one server{}
In-Reply-To: <1501415285.1638.130.camel@gmail.com>
References: <1501412951.1638.128.camel@gmail.com>
 <CAPr95Bhjw2FgLSLZdLEFOVTg4zzRt30h08xsLrDzuWUxGymhAQ@mail.gmail.com>
 <1501415285.1638.130.camel@gmail.com>
Message-ID: <20170730182615.GX365@daoine.org>

On Sun, Jul 30, 2017 at 02:48:05PM +0300, ST wrote:

Hi there,

> That is the problem - the special server{} (example.org) should serve
> robot.txt as is. Those are all the other server{}s that need a redirect
> from /robot.txt to /robot_closed.txt. Are there other ways to do that
> except for explicit redirect inside of all those server{}s?

Just for clarity of what you expect:

A request for http://normal1/robots.txt should return what?

* a http redirect to http://normal1/robot_closed.txt
* a http redirect to http://common/robot_closed.txt
* the content of the file /usr/local/nginx/normal1/html/robot_closed.txt
* the content of the file /usr/local/nginx/html/robot_closed.txt
* something else?

If it is "a" or "c", then presumably a request for
http://normal2/robots.txt should return the equivalent thing; if it is
"b" or "d", then a a request for http://normal2/robots.txt should return
the same thing.

And, a request for http://special/robots.txt should return what?

* the content of the file /usr/local/nginx/special/html/robots.txt
* the content of the file /usr/local/nginx/html/robots.txt

Depending on your answers, maybe the easiest thing would be to rename
"robots.txt" to "robots_special.txt", and rename "robot_closed.txt" to
"robots.txt", and just put a config in the special server{} of the form

 location = /robots.txt {alias /usr/local/nginx/html/robots_special.txt;}

Good luck with it,

	f
-- 
Francis Daly        francis at daoine.org

From francis at daoine.org  Sun Jul 30 18:32:15 2017
From: francis at daoine.org (Francis Daly)
Date: Sun, 30 Jul 2017 19:32:15 +0100
Subject: redirect related questions...
In-Reply-To: <1501408615.1638.124.camel@gmail.com>
References: <1501240403.1638.111.camel@gmail.com>
 <20170729192503.GW365@daoine.org>
 <1501408615.1638.124.camel@gmail.com>
Message-ID: <20170730183215.GY365@daoine.org>

On Sun, Jul 30, 2017 at 12:56:55PM +0300, ST wrote:

Hi there,

> Is it a good idea to use DNS forwarding in order not to obtain/install
> ssl keys for example.com as we don't plan to use it? This should make
> redirection faster and requires no setup on nginx... Are there any down
> sides of such a solution?

I'm not sure what exactly you mean by that.

nginx can listen on one or more address:port combinations, for http
or https.

Each hostname that your client will try to connect to will resolve to
one address (at a time) that the client will try to use.

If your client connects to an address:port that nginx is listening on,
nginx will have to do some work to process the request. If you do not,
nginx will not.

	f
-- 
Francis Daly        francis at daoine.org

From rramirezm at hatandslash.com  Mon Jul 31 00:57:35 2017
From: rramirezm at hatandslash.com (Raymundo Ramirez Mata)
Date: Sun, 30 Jul 2017 19:57:35 -0500
Subject: Nginx Address is already in use
Message-ID: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>



Hi, 



My nginx has stopped working, I upgraded and updated today, also purged mysql and installed mariadb, so I don't know what might broke it. when i run



sudo nginx I get:



nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)



Reading some answers I looked for who is using those ports but only nginx appers using it.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/300e644a/attachment.html>

From sales.skylinehosting at gmail.com  Mon Jul 31 01:28:39 2017
From: sales.skylinehosting at gmail.com (spectre inc)
Date: Mon, 31 Jul 2017 01:28:39 +0000
Subject: Nginx Address is already in use
In-Reply-To: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
References: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
Message-ID: <CAJ6dsWcLpvK=OfuPrRfbHx34TAs0biO1idMMHUG=GjspxDfahQ@mail.gmail.com>

 Fine the process and kill it then restart nginx

On Sun, Jul 30, 2017 at 8:57 PM Raymundo Ramirez Mata <
rramirezm at hatandslash.com> wrote:

>
> Hi,
>
> My nginx has stopped working, I upgraded and updated today, also purged
> mysql and installed mariadb, so I don't know what might broke it. when i run
>
> sudo nginx I get:
>
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
>
> Reading some answers I looked for who is using those ports but only nginx
> appers using it.
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170731/a9ea60a5/attachment.html>

From rramirezm at hatandslash.com  Mon Jul 31 01:32:07 2017
From: rramirezm at hatandslash.com (Raymundo Ramirez Mata)
Date: Sun, 30 Jul 2017 20:32:07 -0500
Subject: Nginx Address is already in use
In-Reply-To: <CAJ6dsWcLpvK=OfuPrRfbHx34TAs0biO1idMMHUG=GjspxDfahQ@mail.gmail.com>
References: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
 <CAJ6dsWcLpvK=OfuPrRfbHx34TAs0biO1idMMHUG=GjspxDfahQ@mail.gmail.com>
Message-ID: <15d964591b3.d5d603b3210267.296015088257575493@hatandslash.com>

I did, but the problem persists




---- On Sun, 30 Jul 2017 20:28:39 -0500 spectre inc &lt;sales.skylinehosting at gmail.com&gt; wrote ----




 Fine the process and kill it then restart nginx 



On Sun, Jul 30, 2017 at 8:57 PM Raymundo Ramirez Mata &lt;rramirezm at hatandslash.com&gt; wrote:





_______________________________________________

nginx mailing list 

nginx at nginx.org 

http://mailman.nginx.org/mailman/listinfo/nginx






Hi, 



My nginx has stopped working, I upgraded and updated today, also purged mysql and installed mariadb, so I don't know what might broke it. when i run



sudo nginx I get:



nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)



Reading some answers I looked for who is using those ports but only nginx appers using it.



_______________________________________________

 nginx mailing list

 nginx at nginx.org

 http://mailman.nginx.org/mailman/listinfo/nginx





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/682fbbcd/attachment.html>

From sales.skylinehosting at gmail.com  Mon Jul 31 01:33:36 2017
From: sales.skylinehosting at gmail.com (spectre inc)
Date: Mon, 31 Jul 2017 01:33:36 +0000
Subject: Nginx Address is already in use
In-Reply-To: <15d964591b3.d5d603b3210267.296015088257575493@hatandslash.com>
References: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
 <CAJ6dsWcLpvK=OfuPrRfbHx34TAs0biO1idMMHUG=GjspxDfahQ@mail.gmail.com>
 <15d964591b3.d5d603b3210267.296015088257575493@hatandslash.com>
Message-ID: <CAJ6dsWcV0Z1GJCxzb9OkCeWn91vUtjnN4fSV+LOH1NTkCxDk=w@mail.gmail.com>

 I asking my tech to see if he can help

On Sun, Jul 30, 2017 at 9:32 PM Raymundo Ramirez Mata <
rramirezm at hatandslash.com> wrote:

> I did, but the problem persists
>
>
> ---- On Sun, 30 Jul 2017 20:28:39 -0500 *spectre inc
> <sales.skylinehosting at gmail.com <sales.skylinehosting at gmail.com>>* wrote
> ----
>
>  Fine the process and kill it then restart nginx
>
> On Sun, Jul 30, 2017 at 8:57 PM Raymundo Ramirez Mata <
> rramirezm at hatandslash.com> wrote:
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
>
> Hi,
>
> My nginx has stopped working, I upgraded and updated today, also purged
> mysql and installed mariadb, so I don't know what might broke it. when i run
>
> sudo nginx I get:
>
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
>
> Reading some answers I looked for who is using those ports but only nginx
> appers using it.
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170731/690ea96b/attachment-0001.html>

From sales.skylinehosting at gmail.com  Mon Jul 31 01:40:14 2017
From: sales.skylinehosting at gmail.com (spectre inc)
Date: Sun, 30 Jul 2017 21:40:14 -0400
Subject: Nginx Address is already in use
In-Reply-To: <15d964591b3.d5d603b3210267.296015088257575493@hatandslash.com>
References: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
 <CAJ6dsWcLpvK=OfuPrRfbHx34TAs0biO1idMMHUG=GjspxDfahQ@mail.gmail.com>
 <15d964591b3.d5d603b3210267.296015088257575493@hatandslash.com>
Message-ID: <03DC25AC-056A-4D93-B2E7-7553DF236D95@gmail.com>

Service nginx stop
Service httpd stop
Then 
Service nginx start
Service httpd start 



Sent from my iPhone

> On Jul 30, 2017, at 9:32 PM, Raymundo Ramirez Mata <rramirezm at hatandslash.com> wrote:
> 
> I did, but the problem persists
> 
> 
> ---- On Sun, 30 Jul 2017 20:28:39 -0500 spectre inc <sales.skylinehosting at gmail.com> wrote ----
> 
>  Fine the process and kill it then restart nginx 
> 
> On Sun, Jul 30, 2017 at 8:57 PM Raymundo Ramirez Mata <rramirezm at hatandslash.com> wrote:
> 
> _______________________________________________
> nginx mailing list 
> nginx at nginx.org 
> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> 
> Hi, 
> 
> My nginx has stopped working, I upgraded and updated today, also purged mysql and installed mariadb, so I don't know what might broke it. when i run
> 
> sudo nginx I get:
> 
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> 
> Reading some answers I looked for who is using those ports but only nginx appers using it.
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/b6ca3070/attachment.html>

From rramirezm at hatandslash.com  Mon Jul 31 01:45:02 2017
From: rramirezm at hatandslash.com (Raymundo Ramirez Mata)
Date: Sun, 30 Jul 2017 20:45:02 -0500
Subject: Nginx Address is already in use
In-Reply-To: <03DC25AC-056A-4D93-B2E7-7553DF236D95@gmail.com>
References: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
 <CAJ6dsWcLpvK=OfuPrRfbHx34TAs0biO1idMMHUG=GjspxDfahQ@mail.gmail.com>
 <15d964591b3.d5d603b3210267.296015088257575493@hatandslash.com>
 <03DC25AC-056A-4D93-B2E7-7553DF236D95@gmail.com>
Message-ID: <15d965163f3.da68ee05210861.2169026720404192624@hatandslash.com>

I do not have httpd:



sudo service httpd stop 

Failed to stop httpd.service: Unit httpd.service not loaded.





---- On Sun, 30 Jul 2017 20:40:14 -0500 spectre inc &lt;sales.skylinehosting at gmail.com&gt; wrote ----




Service nginx stop

Service httpd stop

Then 

Service nginx start

Service httpd start 







Sent from my iPhone




On Jul 30, 2017, at 9:32 PM, Raymundo Ramirez Mata &lt;rramirezm at hatandslash.com&gt; wrote:




_______________________________________________

nginx mailing list

nginx at nginx.org

http://mailman.nginx.org/mailman/listinfo/nginx


_______________________________________________ 

nginx mailing list 

nginx at nginx.org 

http://mailman.nginx.org/mailman/listinfo/nginx


I did, but the problem persists





---- On Sun, 30 Jul 2017 20:28:39 -0500 spectre inc &lt;sales.skylinehosting at gmail.com&gt; wrote ----




 Fine the process and kill it then restart nginx 



On Sun, Jul 30, 2017 at 8:57 PM Raymundo Ramirez Mata &lt;rramirezm at hatandslash.com&gt; wrote:





_______________________________________________

nginx mailing list 

nginx at nginx.org 

http://mailman.nginx.org/mailman/listinfo/nginx






Hi, 



My nginx has stopped working, I upgraded and updated today, also purged mysql and installed mariadb, so I don't know what might broke it. when i run



sudo nginx I get:



nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)

nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)



Reading some answers I looked for who is using those ports but only nginx appers using it.



_______________________________________________

nginx mailing list

nginx at nginx.org

http://mailman.nginx.org/mailman/listinfo/nginx










-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/6e54d9aa/attachment.html>

From sales.skylinehosting at gmail.com  Mon Jul 31 01:48:40 2017
From: sales.skylinehosting at gmail.com (spectre inc)
Date: Sun, 30 Jul 2017 21:48:40 -0400
Subject: Nginx Address is already in use
In-Reply-To: <15d965163f3.da68ee05210861.2169026720404192624@hatandslash.com>
References: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
 <CAJ6dsWcLpvK=OfuPrRfbHx34TAs0biO1idMMHUG=GjspxDfahQ@mail.gmail.com>
 <15d964591b3.d5d603b3210267.296015088257575493@hatandslash.com>
 <03DC25AC-056A-4D93-B2E7-7553DF236D95@gmail.com>
 <15d965163f3.da68ee05210861.2169026720404192624@hatandslash.com>
Message-ID: <62ED8F85-1313-48AC-B42A-CCA5D1309220@gmail.com>

Is there contact email so I can give it my head tech 

Sent from my iPhone

> On Jul 30, 2017, at 9:45 PM, Raymundo Ramirez Mata <rramirezm at hatandslash.com> wrote:
> 
> I do not have httpd:
> 
> sudo service httpd stop 
> Failed to stop httpd.service: Unit httpd.service not loaded.
> 
> 
> ---- On Sun, 30 Jul 2017 20:40:14 -0500 spectre inc <sales.skylinehosting at gmail.com> wrote ----
> 
> Service nginx stop
> Service httpd stop
> Then 
> Service nginx start
> Service httpd start 
> 
> 
> 
> Sent from my iPhone
> 
> On Jul 30, 2017, at 9:32 PM, Raymundo Ramirez Mata <rramirezm at hatandslash.com> wrote:
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
> _______________________________________________ 
> nginx mailing list 
> nginx at nginx.org 
> http://mailman.nginx.org/mailman/listinfo/nginx
> I did, but the problem persists
> 
> 
> ---- On Sun, 30 Jul 2017 20:28:39 -0500 spectre inc <sales.skylinehosting at gmail.com> wrote ----
> 
>  Fine the process and kill it then restart nginx 
> 
> On Sun, Jul 30, 2017 at 8:57 PM Raymundo Ramirez Mata <rramirezm at hatandslash.com> wrote:
> 
> _______________________________________________
> nginx mailing list 
> nginx at nginx.org 
> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> 
> Hi, 
> 
> My nginx has stopped working, I upgraded and updated today, also purged mysql and installed mariadb, so I don't know what might broke it. when i run
> 
> sudo nginx I get:
> 
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> 
> Reading some answers I looked for who is using those ports but only nginx appers using it.
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
> 
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/7a007979/attachment-0001.html>

From igal at lucee.org  Mon Jul 31 03:39:30 2017
From: igal at lucee.org (Igal @ Lucee.org)
Date: Sun, 30 Jul 2017 20:39:30 -0700
Subject: Nginx Address is already in use
In-Reply-To: <62ED8F85-1313-48AC-B42A-CCA5D1309220@gmail.com>
References: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
 <CAJ6dsWcLpvK=OfuPrRfbHx34TAs0biO1idMMHUG=GjspxDfahQ@mail.gmail.com>
 <15d964591b3.d5d603b3210267.296015088257575493@hatandslash.com>
 <03DC25AC-056A-4D93-B2E7-7553DF236D95@gmail.com>
 <15d965163f3.da68ee05210861.2169026720404192624@hatandslash.com>
 <62ED8F85-1313-48AC-B42A-CCA5D1309220@gmail.com>
Message-ID: <9517a276-802a-654c-1df1-7398cf33ab5b@lucee.org>

Try the following to find which process listens on port 80:

   sudo netstat -tulpn | grep ':80 '


Igal Sapir
Lucee Core Developer
Lucee.org <http://lucee.org/>

On 7/30/2017 6:48 PM, spectre inc wrote:
> Is there contact email so I can give it my head tech
>
> Sent from my iPhone
>
> On Jul 30, 2017, at 9:45 PM, Raymundo Ramirez Mata 
> <rramirezm at hatandslash.com <mailto:rramirezm at hatandslash.com>> wrote:
>
>> I do not have httpd:
>>
>> sudo service httpd stop
>> Failed to stop httpd.service: Unit httpd.service not loaded.
>>
>>
>> ---- On Sun, 30 Jul 2017 20:40:14 -0500 *spectre inc 
>> <sales.skylinehosting at gmail.com 
>> <mailto:sales.skylinehosting at gmail.com>>* wrote ----
>>
>>     Service nginx stop
>>     Service httpd stop
>>     Then
>>     Service nginx start
>>     Service httpd start
>>
>>
>>
>>     Sent from my iPhone
>>
>>     On Jul 30, 2017, at 9:32 PM, Raymundo Ramirez Mata
>>     <rramirezm at hatandslash.com <mailto:rramirezm at hatandslash.com>> wrote:
>>
>>         _______________________________________________
>>         nginx mailing list
>>         nginx at nginx.org <mailto:nginx at nginx.org>
>>         http://mailman.nginx.org/mailman/listinfo/nginx
>>
>>     _______________________________________________
>>     nginx mailing list
>>     nginx at nginx.org <mailto:nginx at nginx.org>
>>     http://mailman.nginx.org/mailman/listinfo/nginx
>>
>>         I did, but the problem persists
>>
>>
>>         ---- On Sun, 30 Jul 2017 20:28:39 -0500 *spectre inc
>>         <sales.skylinehosting at gmail.com
>>         <mailto:sales.skylinehosting at gmail.com>>* wrote ----
>>
>>              Fine the process and kill it then restart nginx
>>
>>             On Sun, Jul 30, 2017 at 8:57 PM Raymundo Ramirez Mata
>>             <rramirezm at hatandslash.com
>>             <mailto:rramirezm at hatandslash.com>> wrote:
>>
>>             _______________________________________________
>>             nginx mailing list
>>             nginx at nginx.org <mailto:nginx at nginx.org>
>>             http://mailman.nginx.org/mailman/listinfo/nginx
>>
>>
>>
>>                 Hi,
>>
>>                 My nginx has stopped working, I upgraded and updated
>>                 today, also purged mysql and installed mariadb, so I
>>                 don't know what might broke it. when i run
>>
>>                 sudo nginx I get:
>>
>>                 nginx: [emerg] bind() to [::]:80 failed (98: Address
>>                 already in use)
>>                 nginx: [emerg] bind() to [::]:443 failed (98: Address
>>                 already in use)
>>                 nginx: [emerg] bind() to [::]:80 failed (98: Address
>>                 already in use)
>>                 nginx: [emerg] bind() to [::]:443 failed (98: Address
>>                 already in use)
>>                 nginx: [emerg] bind() to [::]:80 failed (98: Address
>>                 already in use)
>>                 nginx: [emerg] bind() to [::]:443 failed (98: Address
>>                 already in use)
>>                 nginx: [emerg] bind() to [::]:80 failed (98: Address
>>                 already in use)
>>                 nginx: [emerg] bind() to [::]:443 failed (98: Address
>>                 already in use)
>>                 nginx: [emerg] bind() to [::]:80 failed (98: Address
>>                 already in use)
>>                 nginx: [emerg] bind() to [::]:443 failed (98: Address
>>                 already in use)
>>
>>                 Reading some answers I looked for who is using those
>>                 ports but only nginx appers using it.
>>                 _______________________________________________
>>                 nginx mailing list
>>                 nginx at nginx.org <mailto:nginx at nginx.org>
>>                 http://mailman.nginx.org/mailman/listinfo/nginx
>>
>>
>>
>> _______________________________________________
>> nginx mailing list
>> nginx at nginx.org <mailto:nginx at nginx.org>
>> http://mailman.nginx.org/mailman/listinfo/nginx
>
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170730/5c9e4f00/attachment.html>

From fsantiago at garbage-juice.com  Mon Jul 31 03:44:23 2017
From: fsantiago at garbage-juice.com (Fabian A. Santiago)
Date: Mon, 31 Jul 2017 03:44:23 +0000
Subject: Nginx Address is already in use
In-Reply-To: <15d965163f3.da68ee05210861.2169026720404192624@hatandslash.com>
References: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
 <CAJ6dsWcLpvK=OfuPrRfbHx34TAs0biO1idMMHUG=GjspxDfahQ@mail.gmail.com>
 <15d964591b3.d5d603b3210267.296015088257575493@hatandslash.com>
 <03DC25AC-056A-4D93-B2E7-7553DF236D95@gmail.com>
 <15d965163f3.da68ee05210861.2169026720404192624@hatandslash.com>
Message-ID: <E44AC224-E638-452D-A680-971414B747AB@garbage-juice.com>

On July 30, 2017 9:45:02 PM EDT, Raymundo Ramirez Mata <rramirezm at hatandslash.com> wrote:
>I do not have httpd:
>
>
>
>sudo service httpd stop 
>
>Failed to stop httpd.service: Unit httpd.service not loaded.
>
>
>
>
>
>---- On Sun, 30 Jul 2017 20:40:14 -0500 spectre inc
>&lt;sales.skylinehosting at gmail.com&gt; wrote ----
>
>
>
>
>Service nginx stop
>
>Service httpd stop
>
>Then 
>
>Service nginx start
>
>Service httpd start 
>
>
>
>
>
>
>
>Sent from my iPhone
>
>
>
>
>On Jul 30, 2017, at 9:32 PM, Raymundo Ramirez Mata
>&lt;rramirezm at hatandslash.com&gt; wrote:
>
>
>
>
>_______________________________________________
>
>nginx mailing list
>
>nginx at nginx.org
>
>http://mailman.nginx.org/mailman/listinfo/nginx
>
>
>_______________________________________________ 
>
>nginx mailing list 
>
>nginx at nginx.org 
>
>http://mailman.nginx.org/mailman/listinfo/nginx
>
>
>I did, but the problem persists
>
>
>
>
>
>---- On Sun, 30 Jul 2017 20:28:39 -0500 spectre inc
>&lt;sales.skylinehosting at gmail.com&gt; wrote ----
>
>
>
>
> Fine the process and kill it then restart nginx 
>
>
>
>On Sun, Jul 30, 2017 at 8:57 PM Raymundo Ramirez Mata
>&lt;rramirezm at hatandslash.com&gt; wrote:
>
>
>
>
>
>_______________________________________________
>
>nginx mailing list 
>
>nginx at nginx.org 
>
>http://mailman.nginx.org/mailman/listinfo/nginx
>
>
>
>
>
>
>Hi, 
>
>
>
>My nginx has stopped working, I upgraded and updated today, also purged
>mysql and installed mariadb, so I don't know what might broke it. when
>i run
>
>
>
>sudo nginx I get:
>
>
>
>nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
>
>nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
>
>nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
>
>nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
>
>nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
>
>nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
>
>nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
>
>nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
>
>nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
>
>nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
>
>
>
>Reading some answers I looked for who is using those ports but only
>nginx appers using it.
>
>
>
>_______________________________________________
>
>nginx mailing list
>
>nginx at nginx.org
>
>http://mailman.nginx.org/mailman/listinfo/nginx

It looks as though you MAY not have it installed but why are trying to restart httpd? That's Apache. You only mentioned nginx. If both are running, the ports by default would conflict. Am I missing something? Sorry if I am...

Maybe look for apache2?

-- 
Thanks.
Fabian S.

OpenPGP:

3c3fa072accb7ac5db0f723455502b0eeb9070fc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 870 bytes
Desc: not available
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170731/52747f89/attachment.bin>

From nginx-forum at forum.nginx.org  Mon Jul 31 06:37:14 2017
From: nginx-forum at forum.nginx.org (Dan34)
Date: Mon, 31 Jul 2017 02:37:14 -0400
Subject: Buffering issues with nginx
In-Reply-To: <e22ee200c21884e18213d75a92876f37.NginxMailingListEnglish@forum.nginx.org>
References: <13669415.3dim1n3r2K@vbart-laptop>
 <e22ee200c21884e18213d75a92876f37.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <844c30a08735a802bcedd462edb40e2a.NginxMailingListEnglish@forum.nginx.org>

I've run some tests and I'm pretty sure the reason I was getting 5MB stuck
on nginx side was because of RCVBUF on upstream socket uses default socket
buffers and by default it ends up with 5MB RCV buffer.

I added logs to check that value and even after I configured sndbuf and
rcvbuf inside listen directive I was getting 5MB buffer on upstream socket.
After I configured these buffers on upstream (by fiddling with nginx code) I
got immediate results and right away that 5MB delay disappeared.
Similar to 'X-Accel-Buffering' I added X-Accel-Up-RCVBUF and
X-Accel-Down-SNDBUF headers and they seem to be working as expected.

I'm testing this scenario: downstream has limited bandwidth and upstream
(node) can generate data much faster. My goal is to ensure that overall read
speed from upsteram gets limited by downstream, so that nginx doesn't try to
read faster than downstream is capable of. Basically I'm ok if nginx buffers
some constant amount of data (e.g. not more than 1 sec of data at downstream
speed.

Even after fixing it, nginx doesn't work as well as simple single-threaded
vanilla test proxy that I wrote for testing.
That vanilla proxy delivers perfect results for this simple reason: I set
upstream and downstream buffers to some low value (e.g. 128KB) and then I
use blocking recv and blocking send in the same thread. This way whatever it
reads from upstream it sends right away downstream in the same loop

Any reason why would this not work with nginx?.. I don't see why it wouldn't
work with async sockets the same way as with blocking read/send loop in
vanilla proxy.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275526,275758#msg-275758


From nginx-ml at acheronmedia.hr  Mon Jul 31 07:59:49 2017
From: nginx-ml at acheronmedia.hr (Vlad K.)
Date: Mon, 31 Jul 2017 09:59:49 +0200
Subject: Identifying "Writing" connections in status stub
In-Reply-To: <68305643-F820-4FC0-A14F-0864DBFCCF66@me.com>
References: <1ee22cf90e9e86416310b14e6eb40ed8@acheronmedia.hr>
 <B5F0D780-EF90-484A-AEC8-3154885BC13C@me.com>
 <9e57aa6ce577d90bfc9a5d56f56ea111@acheronmedia.hr>
 <20170729234758.GU93611@mdounin.ru>
 <fd44ba612aec5fc8bf4c9ba5512dc485@acheronmedia.hr>
 <28B07F43-04D5-48B6-9A67-74B1E7B79481@me.com>
 <51A15DBE-7FB8-4860-9330-3148CF059911@me.com>
 <bf86c6a8ca3b75097207cf2080bf502a@acheronmedia.hr>
 <2BA185F9-7C0F-4624-B6EA-AF311A012210@me.com>
 <bc68510d4be54ec23db774f09762d958@acheronmedia.hr>
 <68305643-F820-4FC0-A14F-0864DBFCCF66@me.com>
Message-ID: <3e12d361127b82e08a0dbef085e875ca@acheronmedia.hr>

On 2017-07-30 20:03, Peter Booth wrote:
> During a busier part of the day,  what is your minimum, median,99%,
> max requests per sec?


Image's worth a thousand words :)

https://pasteboard.co/GDs6JSz.png

The peaks between days are API clients syncing data, they usually do it 
in the night (clients are all in teh same timezone).


> I?m pretty confident that this is a bug, because of the shape of the
> graph. There's no obvious healthy explanation for the
> number of writing connections to increase over days and return to its
> current value after a restart.


I've restarted nginx yesterday, and until now, the Writing has stayed 
down at 1-3 for the same level of traffic as usual. I suppose that's not 
bots then...



-- 
Vlad K.

From nginx-forum at forum.nginx.org  Mon Jul 31 09:35:47 2017
From: nginx-forum at forum.nginx.org (lifeisjustabout)
Date: Mon, 31 Jul 2017 05:35:47 -0400
Subject: implementation of access.log with if condition
Message-ID: <71e2c3bd706625212f90d3f06353d544.NginxMailingListEnglish@forum.nginx.org>

Hi I would like to implement if condition on access and error log. Prior to
nginx restart I want nginx to look for first log path location eg.
/externalhdd/log/nginx/example.com_access.log, if log doesn't exist use
second path which is  /var/log/nginx/example.com_access.log I have given
like my example.  I guess I need some if conditions or something.

server {
    listen       80;
    server_name  example.com;
    root         /var/www/example.com/public_html;
    index        index.php;
    IF ACCESS LOG EXIST USE THIS PATH
    access_log  
/externalhdd/log/nginx/example.com/logs/example.com_access.log;
    IF ACCESS LOG DOESN'T EXIST USE THIS PATH
    access_log   /var/log/nginx/example.com/logs/example.com_access.log;   


    error_log    /var/www/example.com/logs/example.com_error.log error;

    location / {
        index index.php index.html;
    }

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275761,275761#msg-275761


From iippolitov at nginx.com  Mon Jul 31 10:16:16 2017
From: iippolitov at nginx.com (Igor A. Ippolitov)
Date: Mon, 31 Jul 2017 13:16:16 +0300
Subject: implementation of access.log with if condition
In-Reply-To: <71e2c3bd706625212f90d3f06353d544.NginxMailingListEnglish@forum.nginx.org>
References: <71e2c3bd706625212f90d3f06353d544.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <d5347110-b1c5-de3d-d142-2f730a4b4a50@nginx.com>

Hello,

As far as I know, it's impossible using Nginx.
But you can archive the same result using only 'access_log' statement 
into 'permanent' location.
Just mount your external drive 'nginx' directory over /var/log/nginx like:
     mount -o bind /externalhdd/log/nginx/ /var/log/nginx/

If it is unsuitable for you to 'hide' whole directory, you can create
a subdirectory for your logs and bind external hdd dir there:
     mount -o bind /externalhdd/log/nginx/ /var/log/nginx/mysitelogs/

Hope this helps.

On 31.07.2017 12:35, lifeisjustabout wrote:
> Hi I would like to implement if condition on access and error log. Prior to
> nginx restart I want nginx to look for first log path location eg.
> /externalhdd/log/nginx/example.com_access.log, if log doesn't exist use
> second path which is  /var/log/nginx/example.com_access.log I have given
> like my example.  I guess I need some if conditions or something.
>
> server {
>      listen       80;
>      server_name  example.com;
>      root         /var/www/example.com/public_html;
>      index        index.php;
>      IF ACCESS LOG EXIST USE THIS PATH
>      access_log
> /externalhdd/log/nginx/example.com/logs/example.com_access.log;
>      IF ACCESS LOG DOESN'T EXIST USE THIS PATH
>      access_log   /var/log/nginx/example.com/logs/example.com_access.log;
>
>
>      error_log    /var/www/example.com/logs/example.com_error.log error;
>
>      location / {
>          index index.php index.html;
>      }
>
> Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275761,275761#msg-275761
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx



From nginx-forum at forum.nginx.org  Mon Jul 31 11:08:20 2017
From: nginx-forum at forum.nginx.org (lifeisjustabout)
Date: Mon, 31 Jul 2017 07:08:20 -0400
Subject: implementation of access.log with if condition
In-Reply-To: <71e2c3bd706625212f90d3f06353d544.NginxMailingListEnglish@forum.nginx.org>
References: <71e2c3bd706625212f90d3f06353d544.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <44d46aaf5541f91396121d5cb2885c2c.NginxMailingListEnglish@forum.nginx.org>

Thanks for the response. The reason why I want to implement this is because
I have dedicated server and thanks to nginx everything is work perfectly so
I don't need to check server everyday. I don't want to locate logs on ssd
main (hardrive /var/log/nginx/) which is obviously smaller hard disk than my
external sata hdd (/externalhdd/log/nginx/). I am afraid if something
happens to external hard drive time to time, I still want my website to
operate after restart because every week server restart by itself due to
some limitation of some programs. I guess it will give me error message
nginx couldn't found access.log if external hard disk dies prior to restart
nginx. I want to avoid this problem.

thanks.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275761,275763#msg-275763


From lucas at lucasrolff.com  Mon Jul 31 12:26:04 2017
From: lucas at lucasrolff.com (Lucas Rolff)
Date: Mon, 31 Jul 2017 12:26:04 +0000
Subject: implementation of access.log with if condition
In-Reply-To: <44d46aaf5541f91396121d5cb2885c2c.NginxMailingListEnglish@forum.nginx.org>
References: <71e2c3bd706625212f90d3f06353d544.NginxMailingListEnglish@forum.nginx.org>
 <44d46aaf5541f91396121d5cb2885c2c.NginxMailingListEnglish@forum.nginx.org>
Message-ID: <9823FBE1-7E21-4721-BF50-F093E263E3D7@lucasrolff.com>

> I don't want to locate logs on ssd main (hardrive /var/log/nginx/) which is obviously smaller hard disk than my external sata hdd (/externalhdd/log/nginx/

Setup log rotation depending on size of logs or daily rotation.

> I still want my website to operate after restart because every week server restart by itself due to some limitation of some programs

Instead of randomly restarting ? do a config check and only restart if the config is valid (should also validate the log location)

You write in your initial post that if the access log exists, then you should use the external harddrive, however ? have you considered that the file might exist but the device is in read-only?

You could modify your init script or systemd start script to actually check for these things and generate a new nginx config depending on the result (hint: use place holders).

You could also symlink /externalhdd/log/nginx to /var/log/nginx ? in this case if the drive is mounted it would use the external drive, if it?s not mounted the file would get written to your SSD ? but still won?t make up for a read only device in case that happens.

To be honest, log rotation or symlinks would IMO be the best options.

On 31/07/2017, 13.08, "nginx on behalf of lifeisjustabout" <nginx-bounces at nginx.org on behalf of nginx-forum at forum.nginx.org> wrote:

    Thanks for the response. The reason why I want to implement this is because
    I have dedicated server and thanks to nginx everything is work perfectly so
    I don't need to check server everyday. I don't want to locate logs on ssd
    main (hardrive /var/log/nginx/) which is obviously smaller hard disk than my
    external sata hdd (/externalhdd/log/nginx/). I am afraid if something
    happens to external hard drive time to time, I still want my website to
    operate after restart because every week server restart by itself due to
    some limitation of some programs. I guess it will give me error message
    nginx couldn't found access.log if external hard disk dies prior to restart
    nginx. I want to avoid this problem.
    
    thanks.
    
    Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275761,275763#msg-275763
    
    _______________________________________________
    nginx mailing list
    nginx at nginx.org
    http://mailman.nginx.org/mailman/listinfo/nginx
    


From nginx-forum at forum.nginx.org  Mon Jul 31 14:33:08 2017
From: nginx-forum at forum.nginx.org (lifeisjustabout)
Date: Mon, 31 Jul 2017 10:33:08 -0400
Subject: implementation of access.log with if condition
In-Reply-To: <9823FBE1-7E21-4721-BF50-F093E263E3D7@lucasrolff.com>
References: <9823FBE1-7E21-4721-BF50-F093E263E3D7@lucasrolff.com>
Message-ID: <03bdcb78156966cd9031e0780c48c4ef.NginxMailingListEnglish@forum.nginx.org>

Thanks for response,

> Setup log rotation depending on size of logs or daily rotation. 
If I use log rotation would I still have access to all access data logs
where I moved? 

>Instead of randomly restarting ? do a config check and only restart if the
config is valid (should also validate the log location) 
There is a scheduled script restarts server to close some program sockets
once every week. Sorry I didn't explain correctly. 

>You write in your initial post that if the access log exists, then you
should use the external hard drive, however ? have you considered that the
file might exist but the device is in read-only? 
I never had problem before on this servers external hdd In terms of device
never become read only for last 3 years.

>You could also symlink /externalhdd/log/nginx to /var/log/nginx ? in this
case if the drive is mounted it would use the external drive, if it?s not
mounted the file would get written to your SSD ? but still won?t make up for
a read only device in case that happens. 
My friend is also suggested symblink, I will look on this suggestion 

thank you.

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275761,275770#msg-275770


From nginx-forum at forum.nginx.org  Mon Jul 31 14:46:07 2017
From: nginx-forum at forum.nginx.org (Ortal)
Date: Mon, 31 Jul 2017 10:46:07 -0400
Subject: connection error bit set to 1
Message-ID: <0fa4ba37298ea9d6ee4210b8136b36af.NginxMailingListEnglish@forum.nginx.org>

Hello,
I am writing a nginx module.
I would like to know it which flow is the error filed of the connection set
to 1, I am running a test which the nginx epoll call
ngx_http_finalize_request  with: r->connection->error = 1, this will
terminate my request before I finished with my job.

Thanks

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,275771,275771#msg-275771


From ek at nginx.com  Mon Jul 31 22:49:24 2017
From: ek at nginx.com (Ekaterina Kukushkina)
Date: Tue, 1 Aug 2017 01:49:24 +0300
Subject: Nginx Address is already in use
In-Reply-To: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
References: <15d9625f15a.c47319fe208882.6514015783780099097@hatandslash.com>
Message-ID: <045F9F2F-9CB9-4B92-B907-D903C49C8BA7@nginx.com>

Hello,

> On 31 Jul 2017, at 03:57, Raymundo Ramirez Mata <rramirezm at hatandslash.com> wrote:
> 
> Hi, 
> 
> My nginx has stopped working, I upgraded and updated today, also purged mysql and installed mariadb, so I don't know what might broke it. when i run
> 
> sudo nginx I get:
> 
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
> nginx: [emerg] bind() to [::]:443 failed (98: Address already in use)
> 
> Reading some answers I looked for who is using those ports but only nginx appers using it.

Please check /etc/nginx/conf.d/default.conf
If this file is missing before upgrade, it will be installed
And it can cause mentioned problem.

Try to comment out its content and restart nginx again.


--
Ekaterina Kukushkina