From christian at staudte.it Sun Apr 4 20:13:46 2021 From: christian at staudte.it (Christian Staudte) Date: Sun, 4 Apr 2021 22:13:46 +0200 Subject: limit_req_zone for IPv6 subnets Message-ID: <86bc3beb-0d00-908e-5c6f-2cfc5377aaf7@staudte.it> Hello, regarding rate limiting in IPv6 configurations I see the following problem: As normally a subnet between a /56 and a /64 is assigned to a client by an ISP, and both $binary_remote_addr and $remote_addr always contain the whole IPv6 address, a single client can always spoof the rate limiter by simply choosing another IPv6 address from his own subnet. Currently I have two options to avoid this: a) Disabling IPv6 (well, not really considering that) b) Using application-level rate limiting in PHP which is awkwardly slow Did I miss some configuration options or some dirty hack to do the rate limit matching for example on /64 subnets, or is this simply not possible in nginx? Regards, Chris From teward at thomas-ward.net Sun Apr 4 22:57:58 2021 From: teward at thomas-ward.net (Thomas Ward) Date: Sun, 04 Apr 2021 18:57:58 -0400 Subject: limit_req_zone for IPv6 subnets In-Reply-To: <86bc3beb-0d00-908e-5c6f-2cfc5377aaf7@staudte.it> Message-ID: <4FD8MV70Hwz2GcT@mail.syn-ack.link> I dont think limit_req works on CIDR rather individual IPs.? At least per the description of the module for limiting requests, it works on a single IP level not on a CIDR range level and I don't immediately see a way to make that happen - whether IPv4 or IPv6.Sent from my T-Mobile 4G LTE Device -------- Original message --------From: Christian Staudte Date: 4/4/21 16:14 (GMT-05:00) To: nginx at nginx.org Subject: limit_req_zone for IPv6 subnets Hello,regarding rate limiting in IPv6 configurations I see the followingproblem: As normally a subnet between a /56 and a /64 is assigned to aclient by an ISP, and both $binary_remote_addr and $remote_addr alwayscontain the whole IPv6 address, a single client can always spoof therate limiter by simply choosing another IPv6 address from his own subnet.Currently I have two options to avoid this:a) Disabling IPv6 (well, not really considering that)b) Using application-level rate limiting in PHP which is awkwardly slowDid I miss some configuration options or some dirty hack to do the ratelimit matching for example on /64 subnets, or is this simply notpossible in nginx?Regards, Chris_______________________________________________nginx mailing listnginx at nginx.orghttp://mailman.nginx.org/mailman/listinfo/nginx -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Sun Apr 4 23:13:36 2021 From: mdounin at mdounin.ru (Maxim Dounin) Date: Mon, 5 Apr 2021 02:13:36 +0300 Subject: limit_req_zone for IPv6 subnets In-Reply-To: <86bc3beb-0d00-908e-5c6f-2cfc5377aaf7@staudte.it> References: <86bc3beb-0d00-908e-5c6f-2cfc5377aaf7@staudte.it> Message-ID: Hello! On Sun, Apr 04, 2021 at 10:13:46PM +0200, Christian Staudte wrote: > regarding rate limiting in IPv6 configurations I see the following > problem: As normally a subnet between a /56 and a /64 is assigned to a > client by an ISP, and both $binary_remote_addr and $remote_addr always > contain the whole IPv6 address, a single client can always spoof the > rate limiter by simply choosing another IPv6 address from his own subnet. > > Currently I have two options to avoid this: > a) Disabling IPv6 (well, not really considering that) > b) Using application-level rate limiting in PHP which is awkwardly slow > > Did I miss some configuration options or some dirty hack to do the rate > limit matching for example on /64 subnets, or is this simply not > possible in nginx? You can use anything as a key in limit_req_zone, including your own variables. If you want to limit IPv6 addresses per /64 subnets, something like this should work: map $binary_remote_addr $subnet { "~^(\C{8})" $1; default $binary_remote_addr; } limit_req_zone $subnet zone=one:10m rate=1r/s; -- Maxim Dounin http://mdounin.ru/ From christian at staudte.it Mon Apr 5 11:46:17 2021 From: christian at staudte.it (Christian Staudte) Date: Mon, 5 Apr 2021 13:46:17 +0200 Subject: limit_req_zone for IPv6 subnets In-Reply-To: References: <86bc3beb-0d00-908e-5c6f-2cfc5377aaf7@staudte.it> Message-ID: <35e4111a-539b-881e-1e4c-1b22f14d2cba@staudte.it> Hi, On 05.04.21 01:13, Maxim Dounin wrote: > You can use anything as a key in limit_req_zone, including your > own variables. If you want to limit IPv6 addresses per /64 > subnets, something like this should work: > > map $binary_remote_addr $subnet { > "~^(\C{8})" $1; > default $binary_remote_addr; > } > > limit_req_zone $subnet zone=one:10m rate=1r/s; This seems to work, very nice! That should definitely be added to the docs and maybe also to the blog post (https://www.nginx.com/blog/rate-limiting-nginx/). Regards, Chris From osa at freebsd.org.ru Mon Apr 5 16:47:25 2021 From: osa at freebsd.org.ru (Sergey A. Osokin) Date: Mon, 5 Apr 2021 19:47:25 +0300 Subject: nginx-1.19.9 In-Reply-To: References: Message-ID: Hi there, FreeBSD ports tree is under Subversion to Git transition, https://wiki.freebsd.org/git. For those who interested to test this release with FreeBSD ports/packages system, please visit http://github.com/osokin/nginx-devel. Thanks. -- Sergey On Tue, Mar 30, 2021 at 06:00:30PM +0300, Maxim Dounin wrote: > Changes with nginx 1.19.9 30 Mar 2021 > > *) Bugfix: nginx could not be built with the mail proxy module, but > without the ngx_mail_ssl_module; the bug had appeared in 1.19.8. > > *) Bugfix: "upstream sent response body larger than indicated content > length" errors might occur when working with gRPC backends; the bug > had appeared in 1.19.1. > > *) Bugfix: nginx might not close a connection till keepalive timeout > expiration if the connection was closed by the client while > discarding the request body. > > *) Bugfix: nginx might not detect that a connection was already closed > by the client when waiting for auth_delay or limit_req delay, or when > working with backends. > > *) Bugfix: in the eventport method. > > > -- > Maxim Dounin > http://nginx.org/ > _______________________________________________ > nginx mailing list > nginx at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx From demitrious at automattic.com Wed Apr 7 22:07:34 2021 From: demitrious at automattic.com (Demitrious Kelly) Date: Wed, 7 Apr 2021 15:07:34 -0700 Subject: Capturing Encoded Location Variable Data Message-ID: Hello, Given the following two location configurations: location ~ ^/test/named/(?.+)$ { return 200 $foo; } location ~ ^/test/numbered/(.+)$ { set $foo $1; return 200 $foo; } Are the following two results expected? # curl http://127.0.0.1:8088/test/named/%D7%94%D7%98%D7%99%D7%95%D7%9C ????? # curl http://127.0.0.1:8088/test/numbered/%D7%94%D7%98%D7%99%D7%95%D7%9C %D7%94%D7%98%D7%99%D7%95%D7%9C I have not been able do find any documentation, yet, explaining why these two methods of capturing the same data yield different results. Thanks in advance! DK From mdounin at mdounin.ru Wed Apr 7 23:13:58 2021 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 8 Apr 2021 02:13:58 +0300 Subject: Capturing Encoded Location Variable Data In-Reply-To: References: Message-ID: Hello! On Wed, Apr 07, 2021 at 03:07:34PM -0700, Demitrious Kelly wrote: > Given the following two location configurations: > > location ~ ^/test/named/(?.+)$ { > return 200 $foo; > } > > location ~ ^/test/numbered/(.+)$ { > set $foo $1; > return 200 $foo; > } > Are the following two results expected? > > # curl http://127.0.0.1:8088/test/named/%D7%94%D7%98%D7%99%D7%95%D7%9C > ????? > > # curl http://127.0.0.1:8088/test/numbered/%D7%94%D7%98%D7%99%D7%95%D7%9C > %D7%94%D7%98%D7%99%D7%95%D7%9C > > I have not been able do find any documentation, yet, explaining why these two methods of capturing the same data yield different results. It's a bug, see https://trac.nginx.org/nginx/ticket/348. -- Maxim Dounin http://mdounin.ru/ From demitrious at automattic.com Thu Apr 8 00:16:20 2021 From: demitrious at automattic.com (Demitrious Kelly) Date: Wed, 7 Apr 2021 17:16:20 -0700 Subject: Fwd: Capturing Encoded Location Variable Data References: Message-ID: <975222A7-0E57-43B2-844D-C8581BFB1E16@automattic.com> Thanks very much. It was not an easy thing to google to get from symptom to bug report :) From the text in the ticket it sounds like the named capture functions as intended and if this bug gets fixed the numeric capture example will be made to work the same as named does? Cheers! DK > Begin forwarded message: > > From: Maxim Dounin > Subject: Re: Capturing Encoded Location Variable Data > Date: April 7, 2021 at 4:13:58 PM PDT > To: nginx at nginx.org > Reply-To: nginx at nginx.org > > Hello! > > On Wed, Apr 07, 2021 at 03:07:34PM -0700, Demitrious Kelly wrote: > >> Given the following two location configurations: >> >> location ~ ^/test/named/(?.+)$ { >> return 200 $foo; >> } >> >> location ~ ^/test/numbered/(.+)$ { >> set $foo $1; >> return 200 $foo; >> } >> Are the following two results expected? >> >> # curl http://127.0.0.1:8088/test/named/%D7%94%D7%98%D7%99%D7%95%D7%9C >> ????? >> >> # curl http://127.0.0.1:8088/test/numbered/%D7%94%D7%98%D7%99%D7%95%D7%9C >> %D7%94%D7%98%D7%99%D7%95%D7%9C >> >> I have not been able do find any documentation, yet, explaining why these two methods of capturing the same data yield different results. > > It's a bug, see https://trac.nginx.org/nginx/ticket/348 . > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx mailing list > nginx at nginx.org > http://mailman.nginx.org/mailman/listinfo/nginx -------------- next part -------------- An HTML attachment was scrubbed... URL: From nginx-forum at forum.nginx.org Thu Apr 8 09:54:42 2021 From: nginx-forum at forum.nginx.org (xdrew) Date: Thu, 08 Apr 2021 05:54:42 -0400 Subject: init module callback called twice Message-ID: <9af40dbeefd2c143ae6ad24877ba16e8.NginxMailingListEnglish@forum.nginx.org> Hello, I'm developing a little custom module for nginx, and I need to execute some user code once my module is loaded. I do this by attaching to the hook in ngx_module_t structure: ngx_module_t ngx_http_hello_world_module = { ... NULL, /* init master */ init_module, /* init module */ NULL, /* init process */ ... } static ngx_int_t init_module(ngx_cycle_t *cycle) { ngx_log_stderr (0, "Initializing module") ; } Surprisingly my callback is called twice. First time it follows log message nginx: the configuration file /etc/nginx/nginx.conf syntax is ok nginx: Initializing module and then nginx: configuration file /etc/nginx/nginx.conf test is successful nginx: Initializing module Is there a way to recognize that I'm called in some different context (e.g. some value from ngx_cycle_t structure)? Or may be I'm doing something completely wrong? Thanks Andrew Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291171,291171#msg-291171 From mdounin at mdounin.ru Thu Apr 8 13:04:34 2021 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 8 Apr 2021 16:04:34 +0300 Subject: Fwd: Capturing Encoded Location Variable Data In-Reply-To: <975222A7-0E57-43B2-844D-C8581BFB1E16@automattic.com> References: <975222A7-0E57-43B2-844D-C8581BFB1E16@automattic.com> Message-ID: Hello! On Wed, Apr 07, 2021 at 05:16:20PM -0700, Demitrious Kelly wrote: > Thanks very much. It was not an easy thing to google to get > from symptom to bug report :) From the text in the ticket it > sounds like the named capture functions as intended and if this > bug gets fixed the numeric capture example will be made to work > the same as named does? Yes. -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Thu Apr 8 13:10:37 2021 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 8 Apr 2021 16:10:37 +0300 Subject: init module callback called twice In-Reply-To: <9af40dbeefd2c143ae6ad24877ba16e8.NginxMailingListEnglish@forum.nginx.org> References: <9af40dbeefd2c143ae6ad24877ba16e8.NginxMailingListEnglish@forum.nginx.org> Message-ID: Hello! On Thu, Apr 08, 2021 at 05:54:42AM -0400, xdrew wrote: > Hello, > > I'm developing a little custom module for nginx, and I need to execute some > user code once my module is loaded. I do this by attaching to the hook in > ngx_module_t structure: > > ngx_module_t ngx_http_hello_world_module = { > ... > NULL, /* init master */ > init_module, /* init module */ > NULL, /* init process */ > ... > } > > static ngx_int_t init_module(ngx_cycle_t *cycle) { > ngx_log_stderr (0, "Initializing module") ; } > > Surprisingly my callback is called twice. First time it follows log message > > nginx: the configuration file /etc/nginx/nginx.conf syntax is ok > nginx: Initializing module > > and then > > nginx: configuration file /etc/nginx/nginx.conf test is successful > nginx: Initializing module > > Is there a way to recognize that I'm called in some different context (e.g. > some value from ngx_cycle_t structure)? > Or may be I'm doing something completely wrong? What you observe is perfectly expected: the module initialization callback is called once per configuration parsing, and your output seems to be from running a startup script which does something like "nginx -t; nginx", which starts nginx twice: once to test the configuration, and again to actually start nginx. -- Maxim Dounin http://mdounin.ru/ From nginx-forum at forum.nginx.org Thu Apr 8 13:48:41 2021 From: nginx-forum at forum.nginx.org (xdrew) Date: Thu, 08 Apr 2021 09:48:41 -0400 Subject: init module callback called twice In-Reply-To: References: Message-ID: <60e911c086bc32974d943a7407940106.NginxMailingListEnglish@forum.nginx.org> Thanks Maxim, this makes perfect sense! However the part of the question still stands: is there a way from ngx_cycle_t structure or from some global structure to figure out in which mode nginx is running - testing the configuration or actually starting? Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291171,291194#msg-291194 From mdounin at mdounin.ru Thu Apr 8 15:20:26 2021 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 8 Apr 2021 18:20:26 +0300 Subject: init module callback called twice In-Reply-To: <60e911c086bc32974d943a7407940106.NginxMailingListEnglish@forum.nginx.org> References: <60e911c086bc32974d943a7407940106.NginxMailingListEnglish@forum.nginx.org> Message-ID: Hello! On Thu, Apr 08, 2021 at 09:48:41AM -0400, xdrew wrote: > Thanks Maxim, this makes perfect sense! However the part of the question > still stands: is there a way from ngx_cycle_t structure or from some global > structure to figure out in which mode nginx is running - testing the > configuration or actually starting? In most casses a properly written module shouldn't care. If for some reason your module should, there is the ngx_test_config global variable which makes it possible to check if nginx is testing a configuration rather than parsing a new configuration when starting or reloading. -- Maxim Dounin http://mdounin.ru/ From nginx-forum at forum.nginx.org Thu Apr 8 15:37:19 2021 From: nginx-forum at forum.nginx.org (anish10dec) Date: Thu, 08 Apr 2021 11:37:19 -0400 Subject: Alert: ignore long locked inactive cache entry Message-ID: <906f6c37747f19cb5984ab3dd957ba4d.NginxMailingListEnglish@forum.nginx.org> Hi Team, Intermittently there are multiple below errors reported in error.log file. [alert] 41456#41456: ignore long locked inactive cache entry efcd5613750302a2657fca63c07fc777, count:1 This comes momentarily with a spike of 50-90 K such errors in a minute time span. During this period server load and cpu utilization increases to Maximum dropping all the traffic with 0% Idle CPU and Load rising to more than 100. This happens for 5 min after which server comes back into normal state. Please help What causes this alert and how to avoid this scenario Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291199,291199#msg-291199 From osa at freebsd.org.ru Thu Apr 8 16:09:01 2021 From: osa at freebsd.org.ru (Sergey A. Osokin) Date: Thu, 8 Apr 2021 19:09:01 +0300 Subject: Alert: ignore long locked inactive cache entry In-Reply-To: <906f6c37747f19cb5984ab3dd957ba4d.NginxMailingListEnglish@forum.nginx.org> References: <906f6c37747f19cb5984ab3dd957ba4d.NginxMailingListEnglish@forum.nginx.org> Message-ID: Hi, thanks for the report. On Thu, Apr 08, 2021 at 11:37:19AM -0400, anish10dec wrote: > Hi Team, > > Intermittently there are multiple below errors reported in error.log file. > > [alert] 41456#41456: ignore long locked inactive cache entry > efcd5613750302a2657fca63c07fc777, count:1 > > This comes momentarily with a spike of 50-90 K such errors in a minute time > span. > > During this period server load and cpu utilization increases to Maximum > dropping all the traffic with 0% Idle CPU and Load rising to more than 100. > > This happens for 5 min after which server comes back into normal state. > > Please help What causes this alert and how to avoid this scenario Could you please share `nginx -V' output. There was a fix long time ago, with 1.1.16 for a similar issue. Thank you. -- Sergey Osokin From nginx-forum at forum.nginx.org Thu Apr 8 18:58:01 2021 From: nginx-forum at forum.nginx.org (bouvierh) Date: Thu, 08 Apr 2021 14:58:01 -0400 Subject: Plain text token in config! Message-ID: <6d8032c55ae4dc13151359eb607fb758.NginxMailingListEnglish@forum.nginx.org> Hello! I currently use Nginx as a reverse proxy for my backend services. Nginx authenticates itself to the backend services using a Token that is generated by a process every 10 minutes and that process is writing the token in the config file and reloading nginx regularly: location / { proxy_set_header Authorization "PLAIN TEXT TOKEN WRITTEN BY PROCESS"; proxy_pass https://backend; } I would like to avoid having a token in plain text. Is there a way to avoid that? I though of the following options: - Use env var: But that is impossible nginx doesn't support it - Query the token by having the process establish a local server. Could work but how can the process return the result as a variable to nginx? - Pass the config in memory instead of writing it to a file. Could be a simple option but I didn't find a way to do that. Do you have any idea how I can achieve that? Thank you! Hugues Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291202,291202#msg-291202 From osa at freebsd.org.ru Thu Apr 8 19:43:30 2021 From: osa at freebsd.org.ru (Sergey A. Osokin) Date: Thu, 8 Apr 2021 22:43:30 +0300 Subject: Plain text token in config! In-Reply-To: <6d8032c55ae4dc13151359eb607fb758.NginxMailingListEnglish@forum.nginx.org> References: <6d8032c55ae4dc13151359eb607fb758.NginxMailingListEnglish@forum.nginx.org> Message-ID: Hi Hugues, hope you're doing well. On Thu, Apr 08, 2021 at 02:58:01PM -0400, bouvierh wrote: > Hello! > > I currently use Nginx as a reverse proxy for my backend services. > > Nginx authenticates itself to the backend services using a Token that is > generated by a process every 10 minutes and that process is writing the > token in the config file and reloading nginx regularly: > > location / { > proxy_set_header Authorization "PLAIN TEXT TOKEN WRITTEN BY PROCESS"; > > proxy_pass https://backend; > } > > I would like to avoid having a token in plain text. Is there a way to avoid > that? > I though of the following options: > - Use env var: But that is impossible nginx doesn't support it NGINX does support environment variables, please see details http://nginx.org/en/docs/ngx_core_module.html#env > - Query the token by having the process establish a local server. Could work > but how can the process return the result as a variable to nginx? That probably depends on how a response looks like. It's possible to use NGINX JavaScript module to parse or modify a response. > - Pass the config in memory instead of writing it to a file. Could be a > simple option but I didn't find a way to do that. Some tricks are available with NGINX Plus distribution because of the key-value module, http://nginx.org/en/docs/http/ngx_http_keyval_module.html. -- Sergey Osokin From nginx-forum at forum.nginx.org Thu Apr 8 20:36:47 2021 From: nginx-forum at forum.nginx.org (uragnorson) Date: Thu, 08 Apr 2021 16:36:47 -0400 Subject: location not working Message-ID: <7466011fb64d1f491ef0c6f9cd5b99ae.NginxMailingListEnglish@forum.nginx.org> On RHEL I have, location / { root /usr/share/nginx.html; } location /dist/ { alias /usr/share/nginx/html/dist/; } I am able to navigate to http://server/dist but in dist the index.html is looking for http://server/js but it should be http:/server/dist/js is there a way to add the extra "dist" ? Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291204,291204#msg-291204 From nginx-forum at forum.nginx.org Fri Apr 9 00:21:21 2021 From: nginx-forum at forum.nginx.org (bouvierh) Date: Thu, 08 Apr 2021 20:21:21 -0400 Subject: Plain text token in config! In-Reply-To: References: Message-ID: Thanks for your help!! Are there any other ways that I might have missed? Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291202,291205#msg-291205 From nginx-forum at forum.nginx.org Fri Apr 9 06:14:59 2021 From: nginx-forum at forum.nginx.org (bouvierh) Date: Fri, 09 Apr 2021 02:14:59 -0400 Subject: Plain text token in config! In-Reply-To: References: Message-ID: Thanks for your help!! Are there any other ways that I might have missed? Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291202,291206#msg-291206 From francis at daoine.org Fri Apr 9 06:55:09 2021 From: francis at daoine.org (Francis Daly) Date: Fri, 9 Apr 2021 07:55:09 +0100 Subject: location not working In-Reply-To: <7466011fb64d1f491ef0c6f9cd5b99ae.NginxMailingListEnglish@forum.nginx.org> References: <7466011fb64d1f491ef0c6f9cd5b99ae.NginxMailingListEnglish@forum.nginx.org> Message-ID: <20210409065509.GP16474@daoine.org> On Thu, Apr 08, 2021 at 04:36:47PM -0400, uragnorson wrote: Hi there, > location / { > root /usr/share/nginx.html; > } > > location /dist/ { > alias /usr/share/nginx/html/dist/; > } Because, in this case, the "location" part matches the end of the "alias" part, you could instead choose to use "root" here as well. http://nginx.org/r/alias > I am able to navigate to http://server/dist but in dist the index.html is > looking for http://server/js but it should be http:/server/dist/js is there > a way to add the extra "dist" ? The easiest way would be to change the index.html. If it refers to http://server/js or to /js, make it instead refer to js. Failing that, you could try setting up extra location{}s or redirects to handle this one exception; but I imagine that will turn in to "just one more exception as well...", multiple times. Cheers, f -- Francis Daly francis at daoine.org From nginx at netdirect.fr Mon Apr 12 09:30:30 2021 From: nginx at netdirect.fr (Artur) Date: Mon, 12 Apr 2021 11:30:30 +0200 Subject: Rewrite with regex Message-ID: <474176bd-f01c-9901-db7b-b05c9e0ca817@netdirect.fr> Hello ! I have the following setup : location / { ?try_files $uri $uri/ @shortnames; } location @shortnames { ?rewrite "^/([a-zA-Z0-9]{1,32})$" /index.php?short_name=$1 last; ?return 404; } I filter the 'shortnames' URI to have a format similar to /dfg6df4g64 with minimum and maximum size, only digits and letters allowed. Working well. I want to add hyphen in the 'shortnames' so /ad5-ff56 or /5f9-dfdf4-55f-ddfg are allowed ( but not /-ff4fg or /dgfgdf- ). The regex that matches these conditions is something like (without the size match) : ^/([a-zA-Z0-9]+([-][a-zA-Z0-9])*)$ However there is no match and rewrite fails. My current setup is a Debain 10 server with nginx-full (14.2.2) installed. Any idea on how I can solve this problem, please ? -- Best regards, Artur From nginx at netdirect.fr Mon Apr 12 10:02:32 2021 From: nginx at netdirect.fr (Artur) Date: Mon, 12 Apr 2021 12:02:32 +0200 Subject: Rewrite with regex In-Reply-To: <474176bd-f01c-9901-db7b-b05c9e0ca817@netdirect.fr> References: <474176bd-f01c-9901-db7b-b05c9e0ca817@netdirect.fr> Message-ID: This seems to work : rewrite "^/((?:[a-zA-Z0-9]+(?:[-][a-zA-Z0-9])*)+)$" /index.php?short_name=$1 last; However, I suppose there is no way to check the size of $1 here. Le 12/04/2021 ? 11:30, Artur a ?crit?: > Hello ! > > I have the following setup : > > location / { > ?try_files $uri $uri/ @shortnames; > } > location @shortnames { > ?rewrite "^/([a-zA-Z0-9]{1,32})$" /index.php?short_name=$1 last; > ?return 404; > } > > I filter the 'shortnames' URI to have a format similar to /dfg6df4g64 > with minimum and maximum size, only digits and letters allowed. Working > well. > > I want to add hyphen in the 'shortnames' so /ad5-ff56 or > /5f9-dfdf4-55f-ddfg are allowed ( but not /-ff4fg or /dgfgdf- ). > The regex that matches these conditions is something like (without the > size match) : ^/([a-zA-Z0-9]+([-][a-zA-Z0-9])*)$ > However there is no match and rewrite fails. > > My current setup is a Debain 10 server with nginx-full (14.2.2) installed. > > Any idea on how I can solve this problem, please ? -- Best regards, Artur From francis at daoine.org Mon Apr 12 10:19:42 2021 From: francis at daoine.org (Francis Daly) Date: Mon, 12 Apr 2021 11:19:42 +0100 Subject: Rewrite with regex In-Reply-To: References: <474176bd-f01c-9901-db7b-b05c9e0ca817@netdirect.fr> Message-ID: <20210412101942.GA27756@daoine.org> On Mon, Apr 12, 2021 at 12:02:32PM +0200, Artur wrote: Hi there, > This seems to work : > > rewrite "^/((?:[a-zA-Z0-9]+(?:[-][a-zA-Z0-9])*)+)$" > /index.php?short_name=$1 last; > > However, I suppose there is no way to check the size of $1 here. So far, it looks like you want to allow either: * 1 letter-or-number or * 1 letter-or-number, then 0-to-30 letter-or-number-or-hyphen, then 1 letter-or-number and it's probably simplest to do that as two separate "rewrite" rules. Good luck with it, f -- Francis Daly francis at daoine.org From nginx at netdirect.fr Mon Apr 12 12:21:15 2021 From: nginx at netdirect.fr (Artur) Date: Mon, 12 Apr 2021 14:21:15 +0200 Subject: Rewrite with regex In-Reply-To: <20210412101942.GA27756@daoine.org> References: <474176bd-f01c-9901-db7b-b05c9e0ca817@netdirect.fr> <20210412101942.GA27756@daoine.org> Message-ID: Le 12/04/2021 ? 12:19, Francis Daly a ?crit?: > On Mon, Apr 12, 2021 at 12:02:32PM +0200, Artur wrote: > > Hi there, > >> This seems to work : >> >> rewrite "^/((?:[a-zA-Z0-9]+(?:[-][a-zA-Z0-9])*)+)$" >> /index.php?short_name=$1 last; >> >> However, I suppose there is no way to check the size of $1 here. > So far, it looks like you want to allow either: > > * 1 letter-or-number > > or > > * 1 letter-or-number, then 0-to-30 letter-or-number-or-hyphen, then > 1 letter-or-number No, + means 1 or more. > and it's probably simplest to do that as two separate "rewrite" rules. In fact it works as is. The only problem is that there is no simple way to check the total string length. But having several rewrite rules may be also a good idea. Is it possible to check the size of $uri ? -- Best regards, Artur From francis at daoine.org Mon Apr 12 12:46:47 2021 From: francis at daoine.org (Francis Daly) Date: Mon, 12 Apr 2021 13:46:47 +0100 Subject: Rewrite with regex In-Reply-To: References: <474176bd-f01c-9901-db7b-b05c9e0ca817@netdirect.fr> <20210412101942.GA27756@daoine.org> Message-ID: <20210412124647.GB27756@daoine.org> On Mon, Apr 12, 2021 at 02:21:15PM +0200, Artur wrote: > Le 12/04/2021 ? 12:19, Francis Daly a ?crit?: > > On Mon, Apr 12, 2021 at 12:02:32PM +0200, Artur wrote: Hi there, > In fact it works as is. The only problem is that there is no simple way > to check the total string length. > But having several rewrite rules may be also a good idea. You had rewrite "^/([a-zA-Z0-9]{1,32})$" /index.php?short_name=$1 last; For brevity here, let's just use "lower case letter" instead of "letter or number", so effectively you had rewrite "^/([a-z]{1,32})$" /index.php?short_name=$1 last; and now you want to also allow "-" anywhere except the start or end. Does rewrite "^/([a-z][-a-z]{0,30}[a-z])$" ... do what you want in most cases? And does rewrite "^/([a-z])$" ... do what you want in the other cases? f -- Francis Daly francis at daoine.org From nginx at netdirect.fr Mon Apr 12 13:01:12 2021 From: nginx at netdirect.fr (Artur) Date: Mon, 12 Apr 2021 15:01:12 +0200 Subject: Rewrite with regex In-Reply-To: <20210412124647.GB27756@daoine.org> References: <474176bd-f01c-9901-db7b-b05c9e0ca817@netdirect.fr> <20210412101942.GA27756@daoine.org> <20210412124647.GB27756@daoine.org> Message-ID: Le 12/04/2021 ? 14:46, Francis Daly a ?crit?: > > Does > > rewrite "^/([a-z][-a-z]{0,30}[a-z])$" ... > > do what you want in most cases? No, because it allows these cases I don't want : - a-----b - a---b---c---d By the way, the regex I sent before don't work correctly. It should be (still testing for errors...) : rewrite "^/([a-zA-Z0-9]+(?:-[a-zA-Z0-9]+)*)$" /index.php?short_name=$1 last; -- Best regards, Artur From francis at daoine.org Mon Apr 12 21:12:13 2021 From: francis at daoine.org (Francis Daly) Date: Mon, 12 Apr 2021 22:12:13 +0100 Subject: Rewrite with regex In-Reply-To: References: <474176bd-f01c-9901-db7b-b05c9e0ca817@netdirect.fr> <20210412101942.GA27756@daoine.org> <20210412124647.GB27756@daoine.org> Message-ID: <20210412211213.GD27756@daoine.org> On Mon, Apr 12, 2021 at 03:01:12PM +0200, Artur wrote: > Le 12/04/2021 ? 14:46, Francis Daly a ?crit?: Hi there, > > Does > > > > rewrite "^/([a-z][-a-z]{0,30}[a-z])$" ... > > > > do what you want in most cases? > > No, because it allows these cases I don't want : > > - a-----b > - a---b---c---d That's reasonable. You'll probably find it easier to come up with the regex or regexes that you want, if you can list all of the requirements. That will make it easier to demonstrate that they are or are not met by a suggested solution. Good luck with it, f -- Francis Daly francis at daoine.org From nginx-forum at forum.nginx.org Tue Apr 13 00:17:14 2021 From: nginx-forum at forum.nginx.org (bouvierh) Date: Mon, 12 Apr 2021 20:17:14 -0400 Subject: How to create a share variable between 2 server blocks Message-ID: <2befe45cb1c59c13dba24dfe7d443e01.NginxMailingListEnglish@forum.nginx.org> Hello! On a previous post I asked how I could get give a token to my Nginx server with having to put it in plain text in the config. One suggestion was to send to token in a request and use the javascript module to extract it. This is what I have been trying: load_module modules/ngx_http_js_module.so; events { } http { js_include /iotedge/edge-modules/api-proxy-module/templates/hello_world.js; proxy_buffers 32 160k; proxy_buffer_size 160k; proxy_read_timeout 3600; error_log /dev/stdout info; access_log /dev/stdout; js_var $test; server { listen 80 default_server; location / { js_content hello; } } server { listen 8000 default_server; location / { js_content change; } } } function hello(r) { r.return(200, r.variables['test']); } function change(r) { r.variables['test'] = 2; r.return(200, r.variables['test']); } curl http://127.0.0.1:8000 returns correctly 2. curl http://127.0.0.1 returns 1, even after curl http://127.0.0.1:8000 was called. It seems that the test variable is reset to 1 everytime. If there a way to make it persistent. Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291224,291224#msg-291224 From nginx-forum at forum.nginx.org Tue Apr 13 09:47:07 2021 From: nginx-forum at forum.nginx.org (vybralk) Date: Tue, 13 Apr 2021 05:47:07 -0400 Subject: Replace location paramter Message-ID: i have ngnix in front of java web server. My request is correctly send by ngnix to java web, which redirect user request to another url as user is not authenticated. It set location header to following value location = https://login.server.com/oidc/2/auth?response_type=code&client_id=123&scope=openid+profile+email+params&redirect_uri=https://internal.server.com:8443/ords/apex_authentication.callback&state=123123123123asdfsadfasdf but the server name isn't correct as it is accessible under different name from internet. I need to change it. I have tried to replace the response header with sub_filter, rewrite directive but without success. I have tried to search solution usign google but all the cases which i have found are replacing/modifing incoming request but not the response. Is there a way how to replace text in response location header? Is nginx capable to do that ? Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291230,291230#msg-291230 From francis at daoine.org Tue Apr 13 12:29:12 2021 From: francis at daoine.org (Francis Daly) Date: Tue, 13 Apr 2021 13:29:12 +0100 Subject: Replace location paramter In-Reply-To: References: Message-ID: <20210413122912.GF27756@daoine.org> On Tue, Apr 13, 2021 at 05:47:07AM -0400, vybralk wrote: Hi there, > i have ngnix in front of java web server. My request is correctly send by > ngnix to java web, which redirect user request to another url as user is not > authenticated. It set location header to following value > > location = > https://login.server.com/oidc/2/auth?response_type=code&client_id=123&scope=openid+profile+email+params&redirect_uri=https://internal.server.com:8443/ords/apex_authentication.callback&state=123123123123asdfsadfasdf > > but the server name isn't correct as it is accessible under different name > from internet. I need to change it. Does "proxy_redirect" do what you want? http://nginx.org/r/proxy_redirect f -- Francis Daly francis at daoine.org From nginx-forum at forum.nginx.org Tue Apr 13 14:13:11 2021 From: nginx-forum at forum.nginx.org (xdrew) Date: Tue, 13 Apr 2021 10:13:11 -0400 Subject: launching a user thread from a module Message-ID: Hello, I'd like to spawn a single custom thread as a part of my nginx module. The idea is that it can be used to perform some periodic tasks, asynchronously and independently from HTTP requests. (In other words, I use nginx/module as a framework.) And here I faced with a problem: when I launch my thread in different module hooks, it got killed as soon as parent process exits. So naturally I should be doing this from the master process context, i.e. in init_master hook - which is not implemented, according to the documentation. Is there another place which I can use to launch a user thread? Thanks Andrew P.S. I also tried to offload my thread function as a task to a nginx-managed thread_pool, but this also did not work, as it seems in the stages where I try to use thread_pool (init_module, postconfig), it is not yet fully initialized and a call to ngx_thread_task_post crashes when accessing some queue structure which is zero at that moment. Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291244,291244#msg-291244 From mdounin at mdounin.ru Tue Apr 13 15:41:47 2021 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 13 Apr 2021 18:41:47 +0300 Subject: nginx-1.19.10 Message-ID: Changes with nginx 1.19.10 13 Apr 2021 *) Change: the default value of the "keepalive_requests" directive was changed to 1000. *) Feature: the "keepalive_time" directive. *) Feature: the $connection_time variable. *) Workaround: "gzip filter failed to use preallocated memory" alerts appeared in logs when using zlib-ng. -- Maxim Dounin http://nginx.org/ From nginx-forum at forum.nginx.org Wed Apr 14 16:48:52 2021 From: nginx-forum at forum.nginx.org (kay) Date: Wed, 14 Apr 2021 12:48:52 -0400 Subject: nginx DNS doesn't update the upstream IP Message-ID: <51b3d73b9d14b0dcf282a86117c6cb15.NginxMailingListEnglish@forum.nginx.org> We tested with a default "resolver 8.8.8.8;" and "resolver 8.8.8.8 30s;" options. Both keep using the outdated IP address until you run "nginx -s reload". The upstream is AWS LoadBalancer, which changes IPs quite frequently. I'm using nginx 1.19.3. Any clue what could be wrong? Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291252,291252#msg-291252 From osa at freebsd.org.ru Wed Apr 14 16:55:15 2021 From: osa at freebsd.org.ru (Sergey A. Osokin) Date: Wed, 14 Apr 2021 19:55:15 +0300 Subject: nginx DNS doesn't update the upstream IP In-Reply-To: <51b3d73b9d14b0dcf282a86117c6cb15.NginxMailingListEnglish@forum.nginx.org> References: <51b3d73b9d14b0dcf282a86117c6cb15.NginxMailingListEnglish@forum.nginx.org> Message-ID: Hi, On Wed, Apr 14, 2021 at 12:48:52PM -0400, kay wrote: > We tested with a default "resolver 8.8.8.8;" and "resolver 8.8.8.8 30s;" > options. Both keep using the outdated IP address until you run "nginx -s > reload". The upstream is AWS LoadBalancer, which changes IPs quite > frequently. > > I'm using nginx 1.19.3. Any clue what could be wrong? Could you provide some light how exactly it's been tested. Thanks. -- Sergey Osokin From gk at leniwiec.biz Wed Apr 14 16:55:38 2021 From: gk at leniwiec.biz (Grzegorz Kulewski) Date: Wed, 14 Apr 2021 18:55:38 +0200 Subject: nginx DNS doesn't update the upstream IP In-Reply-To: <51b3d73b9d14b0dcf282a86117c6cb15.NginxMailingListEnglish@forum.nginx.org> References: <51b3d73b9d14b0dcf282a86117c6cb15.NginxMailingListEnglish@forum.nginx.org> Message-ID: <6facc4a0-f0ce-0a10-86df-c59e23cf0f4b@leniwiec.biz> W dniu 14.04.2021 o?18:48, kay pisze: > We tested with a default "resolver 8.8.8.8;" and "resolver 8.8.8.8 30s;" > options. Both keep using the outdated IP address until you run "nginx -s > reload". The upstream is AWS LoadBalancer, which changes IPs quite > frequently. > > I'm using nginx 1.19.3. Any clue what could be wrong? You should probably paste at least proxy_pass and upstream directives from your configuration. Please note that nginx open source does not dynamically resolve hosts from server directives in upstream - only those from proxy_pass. From nginx-forum at forum.nginx.org Wed Apr 14 17:19:08 2021 From: nginx-forum at forum.nginx.org (kay) Date: Wed, 14 Apr 2021 13:19:08 -0400 Subject: nginx DNS doesn't update the upstream IP In-Reply-To: <51b3d73b9d14b0dcf282a86117c6cb15.NginxMailingListEnglish@forum.nginx.org> References: <51b3d73b9d14b0dcf282a86117c6cb15.NginxMailingListEnglish@forum.nginx.org> Message-ID: Here is an example of the proxy_pass config, we don't use IP's proxy_pass https://example.com; When "example.com" changes the IP address, it is not reflected in nginx. We're using k8s pods. Sometime one or two pods have stale DNS cache. Other pods work fine... Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291252,291255#msg-291255 From nginx-forum at forum.nginx.org Wed Apr 14 17:26:35 2021 From: nginx-forum at forum.nginx.org (kay) Date: Wed, 14 Apr 2021 13:26:35 -0400 Subject: nginx DNS doesn't update the upstream IP In-Reply-To: References: <51b3d73b9d14b0dcf282a86117c6cb15.NginxMailingListEnglish@forum.nginx.org> Message-ID: <834ac394e20517552959d595e9c07bfc.NginxMailingListEnglish@forum.nginx.org> Probably I also need to mention that "example.com" has more than one IP address (in our case 2) Posted at Nginx Forum: https://forum.nginx.org/read.php?2,291252,291256#msg-291256 From mdounin at mdounin.ru Wed Apr 14 20:18:02 2021 From: mdounin at mdounin.ru (Maxim Dounin) Date: Wed, 14 Apr 2021 23:18:02 +0300 Subject: nginx DNS doesn't update the upstream IP In-Reply-To: References: <51b3d73b9d14b0dcf282a86117c6cb15.NginxMailingListEnglish@forum.nginx.org> Message-ID: Hello! On Wed, Apr 14, 2021 at 01:19:08PM -0400, kay wrote: > Here is an example of the proxy_pass config, we don't use IP's > > proxy_pass https://example.com; > > When "example.com" changes the IP address, it is not reflected in nginx. > We're using k8s pods. Sometime one or two pods have stale DNS cache. Other > pods work fine... That's exactly how it is expected to work: names in the configuration are resolved during configuration parsing. If you want to re-resolve names, you have to reload the configuration. Alternatively, you can configure proxy_pass using variables. As documented in http://nginx.org/r/proxy_pass: : Parameter value can contain variables. In this case, if an : address is specified as a domain name, the name is searched among : the described server groups, and, if not found, is determined : using a resolver. Hope this helps. -- Maxim Dounin http://mdounin.ru/