From bill0119 at gmail.com Thu Dec 7 01:34:37 2023 From: bill0119 at gmail.com (BILL) Date: Thu, 7 Dec 2023 09:34:37 +0800 Subject: nginx 1.20.0 coverity errors Message-ID: Hi, We have a coverity testing on nginx 1.20.0 and we got some errors. Have any plan to resolve these errors? Checker Number ARRAY_VS_SINGLETON 3 BAD_FREE 3 BUFFER_SIZE 1 CHECKED_RETURN 10 COPY_PASTE_ERROR 1 DC.WEAK_CRYPTO 18 DEADCODE 8 FORWARD_NULL 49 MISSING_RESTORE 1 NO_EFFECT 8 NULL_RETURNS 8 OVERRUN 12 PW.INCLUDE_RECURSION 8 RESOURCE_LEAK 5 REVERSE_INULL 5 SIGN_EXTENSION 1 SIZEOF_MISMATCH 8 STACK_USE 1 STRING_NULL 1 TAINTED_SCALAR 1 TOCTOU 12 UNINIT 10 UNREACHABLE 63 UNUSED_VALUE 4 USE_AFTER_FREE 1 Total 242 -------------- next part -------------- An HTML attachment was scrubbed... URL: From kiwi at oav.net Thu Dec 7 07:29:51 2023 From: kiwi at oav.net (Xavier Beaudouin) Date: Thu, 7 Dec 2023 07:29:51 +0000 (UTC) Subject: nginx 1.20.0 coverity errors In-Reply-To: References: Message-ID: <512018249.7867.1701934191030.JavaMail.zimbra@oav.net> Hello Bill, > We have a coverity testing on nginx 1.20.0 and we got some errors. > Have any plan to resolve these errors? Maybe you should try same thing on 1.24.0 ? Because AFAIK 1.20.0 has been released on 20 Apr 2021 and there is more than 2 years of development in between. So you tested old code deprecated code. Regards, Xavier From kesavardhana.gandla at medtronic.com Thu Dec 7 09:17:37 2023 From: kesavardhana.gandla at medtronic.com (Gandla, Kesavardhana) Date: Thu, 7 Dec 2023 09:17:37 +0000 Subject: Error: nginx: [emerg] unknown directive "stream" in /etc/nginx/nginx.conf:11 Message-ID: Dear NGINX community, I am Kesavardhana Gandla from Medtronic R&D, Bangalore. I am trying to evaluate the nginx on YOCTO Linux based embedded product. My Linux version: 5.15.71 aarch64 GNU/Linux. While using the stream option on target the below error is coming. Error: nginx: [emerg] unknown directive "stream" in /etc/nginx/nginx.conf:11 Attached the nginx.conf file for reference. nginx -V nginx version: nginx/1.20.1 built with OpenSSL 3.0.7 1 Nov 2022 TLS SNI support enabled configure arguments: --crossbuild=Linux:aarch64 --with-endian=little --with-int=4 --with-long=8 --with-long-long=8 --with-ptr-size=8 --with-sig-atomic-t=8 --with-size-t=8 --with-off-t=8 --with-time-t=8 --with-sys-nerr=132 --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/run/nginx/client_body_temp --http-proxy-temp-path=/run/nginx/proxy_temp --http-fastcgi-temp-path=/run/nginx/fastcgi_temp --http-uwsgi-temp-path=/run/nginx/uwsgi_temp --http-scgi-temp-path=/run/nginx/scgi_temp --pid-path=/run/nginx/nginx.pid --prefix=/usr --with-threads --with-http_gzip_static_module --with-http_ssl_module Could you please provide your feedback to resolve the above error "unknown directive stream"? Does open source version of NGNX support stream directive? If yes, how to include it in the yocto build? Thanks, Kesavardhana Gandla [CONFIDENTIALITY AND PRIVACY NOTICE] Information transmitted by this email is proprietary to Medtronic and is intended for use only by the individual or entity to which it is addressed, and may contain information that is private, privileged, confidential or exempt from disclosure under applicable law. If you are not the intended recipient or it appears that this mail has been forwarded to you without proper authority, you are notified that any use or dissemination of this information in any manner is strictly prohibited. In such cases, please delete this mail from your records. To view this notice in other languages you can either select the following link or manually copy and paste the link into the address bar of a web browser: http://emaildisclaimer.medtronic.com -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- user www; worker_processes 1; pid /run/nginx/nginx.pid; include /etc/nginx/modules-enabled/*.conf; events { worker_connections 768; # multi_accept on; } stream { server { listen 8081; # proxy_pass IP_ADDRESS:PORT_NUMBER; } } http { # Basic Settings sendfile on; tcp_nopush on; tcp_nodelay on; keepalive_timeout 65; types_hash_max_size 2048; # server_tokens off; # server_names_hash_bucket_size 64; # server_name_in_redirect off; include /etc/nginx/mime.types; default_type application/octet-stream; # SSL Settings ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE ssl_prefer_server_ciphers on; ## Logging access_log /var/log/nginx/access.log; error_log /var/log/nginx/error.log; ## Gzip settings gzip on; gzip_vary on; gzip_proxied any; gzip_comp_level 6; gzip_buffers 16 8k; gzip_http_version 1.1; gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript; ## Virtual Host Configs ## Virtual Host Configs include /etc/nginx/conf.d/*.conf; include /etc/nginx/sites-enabled/*; } From arut at nginx.com Thu Dec 7 12:43:26 2023 From: arut at nginx.com (Roman Arutyunyan) Date: Thu, 7 Dec 2023 16:43:26 +0400 Subject: Error: nginx: [emerg] unknown directive "stream" in /etc/nginx/nginx.conf:11 In-Reply-To: References: Message-ID: <1EE74BB7-6DF1-427A-B6FC-9BB3F80C70C0@nginx.com> Hi, > On 7 Dec 2023, at 13:17, Gandla, Kesavardhana via nginx wrote: > > Dear NGINX community, > > I am Kesavardhana Gandla from Medtronic R&D, Bangalore. I am trying to evaluate the nginx on YOCTO Linux based embedded product. > My Linux version: 5.15.71 aarch64 GNU/Linux. > While using the stream option on target the below error is coming. > > Error: nginx: [emerg] unknown directive "stream" in /etc/nginx/nginx.conf:11 > Attached the nginx.conf file for reference. > nginx -V > nginx version: nginx/1.20.1 > built with OpenSSL 3.0.7 1 Nov 2022 > TLS SNI support enabled > configure arguments: --crossbuild=Linux:aarch64 --with-endian=little --with-int=4 --with-long=8 --with-long-long=8 --with-ptr-size=8 --with-sig-atomic-t=8 --with-size-t=8 --with-off-t=8 --with-time-t=8 --with-sys-nerr=132 --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/run/nginx/client_body_temp --http-proxy-temp-path=/run/nginx/proxy_temp --http-fastcgi-temp-path=/run/nginx/fastcgi_temp --http-uwsgi-temp-path=/run/nginx/uwsgi_temp --http-scgi-temp-path=/run/nginx/scgi_temp --pid-path=/run/nginx/nginx.pid --prefix=/usr --with-threads --with-http_gzip_static_module --with-http_ssl_module > > Could you please provide your feedback to resolve the above error “unknown directive stream”? > Does open source version of NGNX support stream directive? If yes, how to include it in the yocto build? Stream support in nginx is enabled by "--with-stream" configuration option. Apparently your nginx is now built without Stream support. ---- Roman Arutyunyan arut at nginx.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From r1ch+nginx at teamliquid.net Thu Dec 7 14:57:25 2023 From: r1ch+nginx at teamliquid.net (Richard Stanway) Date: Thu, 7 Dec 2023 15:57:25 +0100 Subject: nginx 1.20.0 coverity errors In-Reply-To: References: Message-ID: This is like reading a book, not understanding some words and then complaining to the author to fix their spelling. Please don't rely on SAST analysis without understanding the code. I would expect the vast majority of these are false positives - provide evidence that these are real bugs if you want them to be taken seriously. On Thu, 7 Dec 2023 at 02:35, BILL wrote: > Hi, > > We have a coverity testing on nginx 1.20.0 and we got some errors. > Have any plan to resolve these errors? > > > Checker Number > ARRAY_VS_SINGLETON 3 > BAD_FREE 3 > BUFFER_SIZE 1 > CHECKED_RETURN 10 > COPY_PASTE_ERROR 1 > DC.WEAK_CRYPTO 18 > DEADCODE 8 > FORWARD_NULL 49 > MISSING_RESTORE 1 > NO_EFFECT 8 > NULL_RETURNS 8 > OVERRUN 12 > PW.INCLUDE_RECURSION 8 > RESOURCE_LEAK 5 > REVERSE_INULL 5 > SIGN_EXTENSION 1 > SIZEOF_MISMATCH 8 > STACK_USE 1 > STRING_NULL 1 > TAINTED_SCALAR 1 > TOCTOU 12 > UNINIT 10 > UNREACHABLE 63 > UNUSED_VALUE 4 > USE_AFTER_FREE 1 > Total 242 > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx > -------------- next part -------------- An HTML attachment was scrubbed... URL: From maxim at nginx.com Thu Dec 7 15:28:34 2023 From: maxim at nginx.com (Maxim Konovalov) Date: Thu, 7 Dec 2023 07:28:34 -0800 Subject: nginx 1.20.0 coverity errors In-Reply-To: References: Message-ID: <67767917-26e7-4319-b313-40c3c89bffed@nginx.com> FWIW, you can find daily nginx Coverity scan results (and even be subscribed to updates) here https://scan.coverity.com/projects/nginx Thanks to Synopsys and Coverity team for the great service. Maxim On 07.12.2023 06:57, Richard Stanway via nginx wrote: > This is like reading a book, not understanding some words and then > complaining to the author to fix their spelling. Please don't rely on > SAST analysis without understanding the code. I would expect the vast > majority of these are false positives - provide evidence that these are > real bugs if you want them to be taken seriously. > > On Thu, 7 Dec 2023 at 02:35, BILL > wrote: > > Hi, > > We have a coverity testing on nginx 1.20.0 and we got some errors. > Have any plan to resolve these errors? > > > Checker Number > ARRAY_VS_SINGLETON 3 > BAD_FREE 3 > BUFFER_SIZE 1 > CHECKED_RETURN 10 > COPY_PASTE_ERROR 1 > DC.WEAK_CRYPTO 18 > DEADCODE 8 > FORWARD_NULL 49 > MISSING_RESTORE 1 > NO_EFFECT 8 > NULL_RETURNS 8 > OVERRUN 12 > PW.INCLUDE_RECURSION 8 > RESOURCE_LEAK 5 > REVERSE_INULL 5 > SIGN_EXTENSION 1 > SIZEOF_MISMATCH 8 > STACK_USE 1 > STRING_NULL 1 > TAINTED_SCALAR 1 > TOCTOU 12 > UNINIT 10 > UNREACHABLE 63 > UNUSED_VALUE 4 > USE_AFTER_FREE 1 > Total 242 > > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx > > > > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx -- Maxim Konovalov From b.bishop at stp.co.nz Fri Dec 8 04:55:12 2023 From: b.bishop at stp.co.nz (Brad Bishop) Date: Fri, 8 Dec 2023 04:55:12 +0000 Subject: NIC deletes all listeners when rejecting new listener on reserved port Message-ID: Hi Folks, We're using NGINX Ingress Controller 3.0.2 (NGINX 1.23.3) in AKS on a couple AKSUbuntu-2204gen2containerd-202309.06.0 nodes. We do regular helm release installs of a single-tenanted TCP & HTTP service for law firms. Today we had a P1 issue when we added a listener for a new law firm to GlobalConfiguration and set the port number to 9113. NGINX rejected the change because 9113 is reserved for prometheus - fair enough. But it also immediately deleted all other existing listeners, which broke 100 TransportServers and blocked access to 100 law firms. We reproduced this on a second AKS cluster. Is this the intended behaviour? I expected in this case that NGINX would reject the bad config and revert to last-good config, and the docs suggest this is what should happen: https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/globalconfiguration-resource/#:~:text=the%20Ingress%20Controller%20will%20ignore%20the%20new%20version Thanks, Brad Bishop From jordanc.carter at outlook.com Fri Dec 8 09:56:57 2023 From: jordanc.carter at outlook.com (J Carter) Date: Fri, 8 Dec 2023 09:56:57 +0000 Subject: NIC deletes all listeners when rejecting new listener on reserved port In-Reply-To: References: Message-ID: Hi Brad, I'd recommend raising your concern the NIC Github repo's issue tracker. https://github.com/nginxinc/kubernetes-ingress/issues On Fri, 8 Dec 2023 04:55:12 +0000 Brad Bishop via nginx wrote: > Hi Folks, > > We're using NGINX Ingress Controller 3.0.2 (NGINX 1.23.3) in AKS on a couple AKSUbuntu-2204gen2containerd-202309.06.0 nodes. We do regular helm release installs of a single-tenanted TCP & HTTP service for law firms. Today we had a P1 issue when we added a listener for a new law firm to GlobalConfiguration and set the port number to 9113. NGINX rejected the change because 9113 is reserved for prometheus - fair enough. But it also immediately deleted all other existing listeners, which broke 100 TransportServers and blocked access to 100 law firms. We reproduced this on a second AKS cluster. Is this the intended behaviour? > > I expected in this case that NGINX would reject the bad config and revert to last-good config, and the docs suggest this is what should happen: > https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/globalconfiguration-resource/#:~:text=the%20Ingress%20Controller%20will%20ignore%20the%20new%20version > > Thanks, > Brad Bishop From b.bishop at stp.co.nz Fri Dec 8 11:48:50 2023 From: b.bishop at stp.co.nz (Brad Bishop) Date: Fri, 8 Dec 2023 11:48:50 +0000 Subject: NIC deletes all listeners when rejecting new listener on reserved port In-Reply-To: References: Message-ID: Will do, thanks. ________________________________ From: nginx on behalf of J Carter Sent: Friday, December 8, 2023 10:56:57 PM To: nginx at nginx.org Subject: Re: NIC deletes all listeners when rejecting new listener on reserved port [You don't often get email from jordanc.carter at outlook.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hi Brad, I'd recommend raising your concern the NIC Github repo's issue tracker. https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnginxinc%2Fkubernetes-ingress%2Fissues&data=05%7C02%7Cb.bishop%40stp.co.nz%7C472235cf4ee94fc960aa08dbf7d40f98%7C5265a077601f4313b5e9946c239ec817%7C1%7C0%7C638376328252310922%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ztY5XRbYO2%2FnYPCuGNNWtorCYVupeqYgZxw0OEkCjTg%3D&reserved=0 On Fri, 8 Dec 2023 04:55:12 +0000 Brad Bishop via nginx wrote: > Hi Folks, > > We're using NGINX Ingress Controller 3.0.2 (NGINX 1.23.3) in AKS on a couple AKSUbuntu-2204gen2containerd-202309.06.0 nodes. We do regular helm release installs of a single-tenanted TCP & HTTP service for law firms. Today we had a P1 issue when we added a listener for a new law firm to GlobalConfiguration and set the port number to 9113. NGINX rejected the change because 9113 is reserved for prometheus - fair enough. But it also immediately deleted all other existing listeners, which broke 100 TransportServers and blocked access to 100 law firms. We reproduced this on a second AKS cluster. Is this the intended behaviour? > > I expected in this case that NGINX would reject the bad config and revert to last-good config, and the docs suggest this is what should happen: > https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.nginx.com%2Fnginx-ingress-controller%2Fconfiguration%2Fglobal-configuration%2Fglobalconfiguration-resource%2F%23%3A~%3Atext%3Dthe%2520Ingress%2520Controller%2520will%2520ignore%2520the%2520new%2520version&data=05%7C02%7Cb.bishop%40stp.co.nz%7C472235cf4ee94fc960aa08dbf7d40f98%7C5265a077601f4313b5e9946c239ec817%7C1%7C0%7C638376328252310922%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=JZqVSxsr3JyFAtUEa%2ByLnpaW5%2FdabvQJc8qUkBmfkgg%3D&reserved=0 > > Thanks, > Brad Bishop _______________________________________________ nginx mailing list nginx at nginx.org https://apc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailman.nginx.org%2Fmailman%2Flistinfo%2Fnginx&data=05%7C02%7Cb.bishop%40stp.co.nz%7C472235cf4ee94fc960aa08dbf7d40f98%7C5265a077601f4313b5e9946c239ec817%7C1%7C0%7C638376328252310922%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=RD18v3ghjPUXTfm3azMx6ZZcKab4scBm%2FUchvRsrwEU%3D&reserved=0 Brad Bishop Software Developer Stratos Technology Partners T +64 3 376 4525 E b.bishop at stp.co.nz -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image001.jpg Type: image/jpeg Size: 464870 bytes Desc: not available URL: From jordanc.carter at outlook.com Fri Dec 8 20:38:36 2023 From: jordanc.carter at outlook.com (J Carter) Date: Fri, 8 Dec 2023 20:38:36 +0000 Subject: Limiting number of client TLS connections In-Reply-To: References: <489b61d6-d20e-429f-be71-cdc704d983a9@aosc.io> Message-ID: Hello again, By coincidence, and since my previous email, someone has kindly submitted a fixed window rate limiting example to the NJS examples Github repo. https://github.com/nginx/njs-examples/pull/31/files/ba33771cefefdc019ba76bd1f176e25e18adbc67 https://github.com/nginx/njs-examples/tree/master/conf/http/rate-limit The example is for rate limiting in http context, however I believe you'd be able to adapt this for stream (and your use case) with minor modifications (use js_access rather than 'if' as mentioned previously, setting key to a fixed value). Just forwarding it on in case you need it. On Sat, 25 Nov 2023 16:03:37 +0800 Zero King wrote: > Hi Jordan, > > Thanks for your suggestion. I will give it a try and also try to push > our K8s team to implement a firewall if possible. > > On 20/11/23 10:33, J Carter wrote: > > Hello, > > > > A self contained solution would be to double proxy, first through nginx stream server and then locally back to nginx http server (with proxy_pass via unix socket, or to localhost on a different port). > > > > You can implement your own custom rate limiting logic in the stream server with NJS (js_access) and use the new js_shared_dict_zone (which is shared between workers) for persistently storing rate calculations. > > > > You'd have additional overhead from the stream tcp proxy and the njs, but it shouldn't be too great (at least compared to overhead of TLS handshakes). > > > > Regards, > > Jordan Carter. > > > > ________________________________________ > > From: nginx on behalf of Zero King > > Sent: Saturday, November 18, 2023 6:44 AM > > To: nginx at nginx.org > > Subject: Limiting number of client TLS connections > > > > Hi all, > > > > I want Nginx to limit the rate of new TLS connections and the total (or > > per-worker) number of all client-facing connections, so that under a > > sudden surge of requests, existing connections can get enough share of > > CPU to be served properly, while excessive connections are rejected and > > retried against other servers in the cluster. > > > > I am running Nginx on a managed Kubernetes cluster, so tuning kernel > > parameters or configuring layer 4 firewall is not an option. > > > > To serve existing connections well, worker_connections can not be used, > > because it also affects connections with proxied servers. > > > > Is there a way to implement these measures in Nginx configuration? > > _______________________________________________ From ml at fatbsd.com Tue Dec 12 15:17:11 2023 From: ml at fatbsd.com (=?UTF-8?B?SsOpcsO0bWUgTG95ZXQ=?=) Date: Tue, 12 Dec 2023 16:17:11 +0100 Subject: serving files from /proc Message-ID: Hello, I'm trying to serve some files from /proc but nginx return a 0 bytes content because the file size of many files in /proc/ tree is simply 0 by design. here is my sample conf file: ... location = /route { root /proc/net; } and the result of the corresponding curl: > GET /route HTTP/1.1 > Host: 172.16.0.3:1513 > User-Agent: curl/7.68.0 > Accept: */* > * Mark bundle as not supporting multiuse < HTTP/1.1 200 OK < Server: nginx/1.23.1 < Date: Tue, 12 Dec 2023 15:08:00 GMT < Content-Type: text/plain < Content-Length: 0 < Last-Modified: Tue, 12 Dec 2023 15:08:00 GMT < Connection: keep-alive < ETag: "65787750-0" < Accept-Ranges: bytes is there a simple way to configure nginx to return the cotent of /proc/net/route or any other file in /proc ? thanks regards ++ Jerome -------------- next part -------------- An HTML attachment was scrubbed... URL: From francis at daoine.org Tue Dec 12 17:42:15 2023 From: francis at daoine.org (Francis Daly) Date: Tue, 12 Dec 2023 17:42:15 +0000 Subject: serving files from /proc In-Reply-To: References: Message-ID: <20231212174215.GF6038@daoine.org> On Tue, Dec 12, 2023 at 04:17:11PM +0100, Jérôme Loyet wrote: Hi there, > I'm trying to serve some files from /proc but nginx return a 0 bytes > content because the file size of many files in /proc/ tree is simply 0 by > design. I suspect that you are going to have to write something to read the file and tell nginx what the "real" size and content is. Stock nginx knows that for static files, when the filesystem says that st_size is 0, the file has a size of 0. That happens to not be true for some things within /proc; so either you get to change the code behind /proc to return "real" values, or you get to write something to return real values. I guess it will be simpler to run something like a fastcgi process that will "cat" the file, and tell nginx to fastcgi_pass to that process for these requests; than to rewrite the /proc code or to rewrite the nginx static file handler to do extra things when it is told that the size is 0. > is there a simple way to configure nginx to return the cotent of > /proc/net/route or any other file in /proc ? Untested, but I suspect "directly: no; indirectly, maybe (as above)". Cheers, f -- Francis Daly francis at daoine.org From jordanc.carter at outlook.com Wed Dec 13 02:45:54 2023 From: jordanc.carter at outlook.com (J Carter) Date: Wed, 13 Dec 2023 02:45:54 +0000 Subject: serving files from /proc In-Reply-To: References: Message-ID: Hello, On Tue, 12 Dec 2023 16:17:11 +0100 Jérôme Loyet wrote: > Hello, > > I'm trying to serve some files from /proc but nginx return a 0 bytes > content because the file size of many files in /proc/ tree is simply 0 by > design. That is correct, reading Virtual File System files would require special handling compared to regular files. Nginx doesn't appear to have this. > > here is my sample conf file: > ... > location = /route { > root /proc/net; > } > > and the result of the corresponding curl: > > GET /route HTTP/1.1 > > Host: 172.16.0.3:1513 > > User-Agent: curl/7.68.0 > > Accept: */* > > > * Mark bundle as not supporting multiuse > < HTTP/1.1 200 OK > < Server: nginx/1.23.1 > < Date: Tue, 12 Dec 2023 15:08:00 GMT > < Content-Type: text/plain > < Content-Length: 0 > < Last-Modified: Tue, 12 Dec 2023 15:08:00 GMT > < Connection: keep-alive > < ETag: "65787750-0" > < Accept-Ranges: bytes > > is there a simple way to configure nginx to return the cotent of > /proc/net/route or any other file in /proc ? For a solution entirely within nginx, you can use njs to serve /proc/net/route. https://github.com/nginx/njs-examples?tab=readme-ov-file#file-io-misc-file-io Njs does handle VFS/zero sized files in a special way. If the file is zero sized, it will read up to 4096 bytes from the file. https://github.com/nginx/njs/blob/2c937050a4589bbc196db334fef22e6de772dd49/external/njs_fs_module.c#L2732 > thanks > regards > ++ Jerome From jordanc.carter at outlook.com Wed Dec 13 03:19:24 2023 From: jordanc.carter at outlook.com (J Carter) Date: Wed, 13 Dec 2023 03:19:24 +0000 Subject: serving files from /proc In-Reply-To: References: Message-ID: On Wed, 13 Dec 2023 02:45:54 +0000 J Carter wrote: > Hello, > > On Tue, 12 Dec 2023 16:17:11 +0100 > Jérôme Loyet wrote: > > > Hello, > > > > I'm trying to serve some files from /proc but nginx return a 0 bytes > > content because the file size of many files in /proc/ tree is simply 0 by > > design. > > That is correct, reading Virtual File System files would require special > handling compared to regular files. Nginx doesn't appear to have this. > > > > > here is my sample conf file: > > ... > > location = /route { > > root /proc/net; > > } > > > > and the result of the corresponding curl: > > > GET /route HTTP/1.1 > > > Host: 172.16.0.3:1513 > > > User-Agent: curl/7.68.0 > > > Accept: */* > > > > > * Mark bundle as not supporting multiuse > > < HTTP/1.1 200 OK > > < Server: nginx/1.23.1 > > < Date: Tue, 12 Dec 2023 15:08:00 GMT > > < Content-Type: text/plain > > < Content-Length: 0 > > < Last-Modified: Tue, 12 Dec 2023 15:08:00 GMT > > < Connection: keep-alive > > < ETag: "65787750-0" > > < Accept-Ranges: bytes > > > > is there a simple way to configure nginx to return the cotent of > > /proc/net/route or any other file in /proc ? > > For a solution entirely within nginx, you can use njs to serve /proc/net/route. > > > https://github.com/nginx/njs-examples?tab=readme-ov-file#file-io-misc-file-io > > Njs does handle VFS/zero sized files in a special way. If the file is zero > sized, it will read up to 4096 bytes from the file. > *Correction - it read up until EOF. So any sized VFS file will work actually even above 4096 bytes. > > https://github.com/nginx/njs/blob/2c937050a4589bbc196db334fef22e6de772dd49/external/njs_fs_module.c#L2732 > > > > thanks > > regards > > ++ Jerome > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx From kaushalshriyan at gmail.com Sat Dec 16 13:02:44 2023 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Sat, 16 Dec 2023 18:32:44 +0530 Subject: Uploading files to remote server running nginx 1.24 returns http status code 405 Not Allowed. Message-ID: Hi, I am running nginx version: nginx/1.24.0 on CentOS Linux release 7.9.2009 (Core). I have the below config file /etc/nginx/conf.d/upload.conf. ############################################################################################################################################ #cat upload.conf server { listen 443 ssl; server_name software.example.com; ssl_protocols TLSv1.3 TLSv1.2; ssl_stapling off; ssl_stapling_verify on; ssl_certificate /etc/letsencrypt/live/ software.example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/ software.example.com/privkey.pem; ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384; root /var/www/html/cbsoftwarebinaries; auth_basic "Restricted Access"; auth_basic_user_file /etc/nginx/.htpasswd; autoindex on; ssl_prefer_server_ciphers on; ssl_dhparam /etc/ssl/certs/software.example.com/dhparam.pem; add_header Strict-Transport-Security: max-age=63072000; #index index.html index.htm; location /var/www/html/cbapibankingsoftwarebinaries { try_files $uri $uri/ =404; } } server { listen 443 ssl; server_name software.example.com; ssl_protocols TLSv1.3 TLSv1.2; ssl_stapling off; ssl_stapling_verify on; ssl_certificate /etc/letsencrypt/live/ software.example.com/fullchain.pem; ssl_certificate_key /etc/letsencrypt/live/ software.example.com/privkey.pem; ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384; root /var/www/html/softwareuploads; ssl_prefer_server_ciphers on; ssl_dhparam /etc/ssl/certs/software.example.com/dhparam.pem; add_header Strict-Transport-Security: max-age=63072000; index index.html index.php; location = /upload.php { fastcgi_pass 127.0.0.1:9000; fastcgi_index index.php; include fastcgi_params; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; } client_max_body_size 10M; # Adjust the maximum allowed file size } ############################################################################################################################################ When I hit https://software.example.com I get all the files and folders as per directory listing which is working as expected as per the below screenshot [image: image.png] When I am trying to upload files by hitting https://software.example.com/upload_form.html I am encountering http status code 405 Not Allowed. Please find inline a screenshot for your reference. [image: image.png] [image: image.png] Please guide me. Thanks in advance. Best Regards, Kaushal -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 21924 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 19056 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: image.png Type: image/png Size: 112131 bytes Desc: not available URL: From venefax at gmail.com Sat Dec 16 19:16:45 2023 From: venefax at gmail.com (Saint Michael) Date: Sat, 16 Dec 2023 14:16:45 -0500 Subject: Forcing incognito mode on a reverse proxy Message-ID: I have a reverse proxy but for security reasons, I need to force the client to work the closest to an Incognito session as possible. I tried adding the following: proxy_set_header Cookie ""; add_header Set-Cookie "cookie_name=; Expires=Thu, 01 Jan 1970 00:00:01 GMT;"; } but it still does not work correctly. Is there a way to do this? From r at roze.lv Mon Dec 18 14:48:58 2023 From: r at roze.lv (Reinis Rozitis) Date: Mon, 18 Dec 2023 16:48:58 +0200 Subject: Uploading files to remote server running nginx 1.24 returns http status code 405 Not Allowed. In-Reply-To: References: Message-ID: <000201da31c1$54f08930$fed19b90$@roze.lv> > When I am trying to upload files by hitting https://software.example.com/upload_form.html I am encountering http status code 405 Not Allowed. Please find inline a screenshot for your reference. Does your upload form point exactly to '/upload.php' ? as by by default Nginx doesn't allow POST on static content which would indicate that the form submits to a static document. rr From kaushalshriyan at gmail.com Mon Dec 18 16:54:14 2023 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Mon, 18 Dec 2023 22:24:14 +0530 Subject: Capture full request/response body logging in nginx configured in reverse proxy mode reports nginx: [emerg] unknown directive "log_by_lua_block" Message-ID: Hi, I am running nginx version: nginx/1.24.0 as reverse proxy on CentOS Linux release 7.9.2009 (Core). I have the below config file /etc/nginx/conf.d/microservice.conf I want to capture full request/response body logging in nginx. # cat /etc/nginx/conf.d/microservice.conf server { listen 80; server_name 192.168.0.129; location / { # Capture request headers and body log_by_lua_block { ngx.var.request_headers = ngx.req.raw_header() ngx.var.request_body = ngx.req.get_body_data() } # Your regular location configuration here # Capture response headers and body body_filter_by_lua_block { local resp_body = ngx.arg[1] ngx.ctx.buffered = (ngx.ctx.buffered or "") .. resp_body if ngx.arg[2] then ngx.var.resp_body = ngx.ctx.buffered end } } #nginx -t *nginx: [emerg] unknown directive "log_by_lua_block" in* /etc/nginx/conf.d/microservice.conf:8 nginx: configuration file /etc/nginx/nginx.conf test failed Please guide me. Thanks in advance. Best Regards, Kaushal -------------- next part -------------- An HTML attachment was scrubbed... URL: From osa at freebsd.org.ru Mon Dec 18 17:49:15 2023 From: osa at freebsd.org.ru (Sergey A. Osokin) Date: Mon, 18 Dec 2023 20:49:15 +0300 Subject: Capture full request/response body logging in nginx configured in reverse proxy mode reports nginx: [emerg] unknown directive "log_by_lua_block" In-Reply-To: References: Message-ID: Hi Kaushal, On Mon, Dec 18, 2023 at 10:24:14PM +0530, Kaushal Shriyan wrote: > > I am running nginx version: nginx/1.24.0 as reverse proxy on CentOS Linux > release 7.9.2009 (Core). I have the below config > file /etc/nginx/conf.d/microservice.conf > I want to capture full request/response body logging in nginx. Since you question is related to a third-party module, I'd recommend to get support in a appropriate mailing list. Thank you. -- Sergey A. Osokin From kaushalshriyan at gmail.com Mon Dec 18 18:21:13 2023 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Mon, 18 Dec 2023 23:51:13 +0530 Subject: Capture full request/response body logging in nginx configured in reverse proxy mode reports nginx: [emerg] unknown directive "log_by_lua_block" In-Reply-To: References: Message-ID: On Mon, Dec 18, 2023 at 11:19 PM Sergey A. Osokin wrote: > Hi Kaushal, > > On Mon, Dec 18, 2023 at 10:24:14PM +0530, Kaushal Shriyan wrote: > > > > I am running nginx version: nginx/1.24.0 as reverse proxy on CentOS Linux > > release 7.9.2009 (Core). I have the below config > > file /etc/nginx/conf.d/microservice.conf > > I want to capture full request/response body logging in nginx. > > Since you question is related to a third-party module, I'd recommend > to get support in a appropriate mailing list. > > Thanks Sergey for the quick response. Is there a way to capture full request/response body logging in nginx without Lua module support? Please ignore the earlier OS version, CentOS Linux release 7.9.2009 (Core). #cat /etc/redhat-release Red Hat Enterprise Linux release 8.8 (Ootpa) # nginx -V nginx version: nginx/1.24.0 built by gcc 8.5.0 20210514 (Red Hat 8.5.0-4) (GCC) built with OpenSSL 1.1.1k FIPS 25 Mar 2021 TLS SNI support enabled configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie' # rpm -qa | grep nginx nginx-1.24.0-1.el8.ngx.x86_64 #nginx -v nginx version: nginx/1.24.0 # # yum search nginx Updating Subscription Management repositories. Last metadata expiration check: 3:07:49 ago on Mon 18 Dec 2023 08:40:51 PM IST. ================================================================ Name Exactly Matched: nginx ================================================================= nginx.x86_64 : High performance web server =============================================================== Name & Summary Matched: nginx ================================================================ collectd-nginx.x86_64 : Nginx plugin for collectd lemonldap-ng-nginx.noarch : LemonLDAP-NG Nginx support munin-nginx.noarch : NGINX support for Munin resource monitoring nginx-all-modules.noarch : A meta package that installs all available Nginx modules nginx-debuginfo.x86_64 : Debug information for package nginx nginx-filesystem.noarch : The basic directory layout for the Nginx server nginx-mod-http-image-filter.x86_64 : Nginx HTTP image filter module nginx-mod-http-perl.x86_64 : Nginx HTTP perl module nginx-mod-http-xslt-filter.x86_64 : Nginx XSLT module nginx-mod-mail.x86_64 : Nginx mail modules nginx-mod-stream.x86_64 : Nginx stream modules nginx-module-image-filter.x86_64 : nginx image filter dynamic module nginx-module-image-filter-debuginfo.x86_64 : Debug information for package nginx-module-image-filter nginx-module-njs.x86_64 : nginx njs dynamic modules nginx-module-njs-debuginfo.x86_64 : Debug information for package nginx-module-njs nginx-module-perl.x86_64 : nginx Perl dynamic module nginx-module-perl-debuginfo.x86_64 : Debug information for package nginx-module-perl nginx-module-xslt.x86_64 : nginx xslt dynamic module nginx-module-xslt-debuginfo.x86_64 : Debug information for package nginx-module-xslt pagure-web-nginx.noarch : Nginx configuration for Pagure pcp-pmda-nginx.x86_64 : Performance Co-Pilot (PCP) metrics for the Nginx Webserver python3-certbot-nginx.noarch : The nginx plugin for certbot sympa-nginx.x86_64 : Sympa with nginx # yum -y install nginx-all-modules.noarch Updating Subscription Management repositories. Last metadata expiration check: 3:08:26 ago on Mon 18 Dec 2023 08:40:51 PM IST. Dependencies resolved. ============================================================================================================================================================== Package Architecture Version Repository Size ============================================================================================================================================================== Installing: nginx-all-modules noarch 1:1.14.1-9.module+el8.0.0+4108+af250afe rhel-8-for-x86_64-appstream-rpms 24 k Installing dependencies: nginx-mod-http-image-filter x86_64 1:1.14.1-9.module+el8.0.0+4108+af250afe rhel-8-for-x86_64-appstream-rpms 35 k nginx-mod-http-perl x86_64 1:1.14.1-9.module+el8.0.0+4108+af250afe rhel-8-for-x86_64-appstream-rpms 46 k nginx-mod-http-xslt-filter x86_64 1:1.14.1-9.module+el8.0.0+4108+af250afe rhel-8-for-x86_64-appstream-rpms 34 k nginx-mod-mail x86_64 1:1.14.1-9.module+el8.0.0+4108+af250afe rhel-8-for-x86_64-appstream-rpms 64 k nginx-mod-stream x86_64 1:1.14.1-9.module+el8.0.0+4108+af250afe rhel-8-for-x86_64-appstream-rpms 85 k Enabling module streams: nginx 1.14 Transaction Summary ============================================================================================================================================================== Install 6 Packages Total download size: 287 k Installed size: 399 k Downloading Packages: (1/6): nginx-all-modules-1.14.1-9.module+el8.0.0+4108+af250afe.noarch.rpm 10 kB/s | 24 kB 00:02 (2/6): nginx-mod-http-image-filter-1.14.1-9.module+el8.0.0+4108+af250afe.x86_64.rpm 15 kB/s | 35 kB 00:02 (3/6): nginx-mod-http-perl-1.14.1-9.module+el8.0.0+4108+af250afe.x86_64.rpm 20 kB/s | 46 kB 00:02 (4/6): nginx-mod-http-xslt-filter-1.14.1-9.module+el8.0.0+4108+af250afe.x86_64.rpm 17 kB/s | 34 kB 00:02 (5/6): nginx-mod-mail-1.14.1-9.module+el8.0.0+4108+af250afe.x86_64.rpm 32 kB/s | 64 kB 00:02 (6/6): nginx-mod-stream-1.14.1-9.module+el8.0.0+4108+af250afe.x86_64.rpm 43 kB/s | 85 kB 00:02 -------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 67 kB/s | 287 kB 00:04 Running transaction check Transaction check succeeded. Running transaction test Transaction test succeeded. Running transaction Preparing : 1/1 Installing : nginx-mod-stream-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 1/6 Running scriptlet: nginx-mod-stream-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 1/6 Installing : nginx-mod-mail-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 2/6 Running scriptlet: nginx-mod-mail-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 2/6 Installing : nginx-mod-http-xslt-filter-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 3/6 Running scriptlet: nginx-mod-http-xslt-filter-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 3/6 Installing : nginx-mod-http-perl-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 4/6 Running scriptlet: nginx-mod-http-perl-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 4/6 Installing : nginx-mod-http-image-filter-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 5/6 Running scriptlet: nginx-mod-http-image-filter-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 5/6 Installing : nginx-all-modules-1:1.14.1-9.module+el8.0.0+4108+af250afe.noarch 6/6 Running scriptlet: nginx-all-modules-1:1.14.1-9.module+el8.0.0+4108+af250afe.noarch 6/6 Verifying : nginx-all-modules-1:1.14.1-9.module+el8.0.0+4108+af250afe.noarch 1/6 Verifying : nginx-mod-http-image-filter-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 2/6 Verifying : nginx-mod-http-perl-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 3/6 Verifying : nginx-mod-http-xslt-filter-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 4/6 Verifying : nginx-mod-mail-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 5/6 Verifying : nginx-mod-stream-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 6/6 Installed products updated. Installed: nginx-all-modules-1:1.14.1-9.module+el8.0.0+4108+af250afe.noarch nginx-mod-http-image-filter-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 nginx-mod-http-perl-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 nginx-mod-http-xslt-filter-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 nginx-mod-mail-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 nginx-mod-stream-1:1.14.1-9.module+el8.0.0+4108+af250afe.x86_64 Complete! # nginx -t nginx: [emerg] unknown directive "log_by_lua_block" in /etc/nginx/conf.d/microservice.conf:8 nginx: configuration file /etc/nginx/nginx.conf test failed # Please guide me. Thanks in advance. Best Regards, Kaushal -------------- next part -------------- An HTML attachment was scrubbed... URL: From r at roze.lv Tue Dec 19 13:58:12 2023 From: r at roze.lv (Reinis Rozitis) Date: Tue, 19 Dec 2023 15:58:12 +0200 Subject: Capture full request/response body logging in nginx configured in reverse proxy mode reports nginx: [emerg] unknown directive "log_by_lua_block" In-Reply-To: References: Message-ID: <000101da3283$67de08c0$379a1a40$@roze.lv> > nginx: [emerg] unknown directive "log_by_lua_block" in /etc/nginx/conf.d/microservice.conf:8 > nginx: configuration file /etc/nginx/nginx.conf test failed You need nginx lua/openresty module, but I'm not sure if Centos (community version) repository has it. You might need to build it yourself https://github.com/openresty/lua-nginx-module#installation Or use the nginx plus commercial version: https://docs.nginx.com/nginx/admin-guide/dynamic-modules/lua/ rr From francis at daoine.org Wed Dec 20 10:59:18 2023 From: francis at daoine.org (Francis Daly) Date: Wed, 20 Dec 2023 10:59:18 +0000 Subject: Forcing incognito mode on a reverse proxy In-Reply-To: References: Message-ID: <20231220105918.GG6038@daoine.org> On Sat, Dec 16, 2023 at 02:16:45PM -0500, Saint Michael wrote: Hi there, > I have a reverse proxy but for security reasons, I need to force the > client to work the closest to an Incognito session as possible. I suspect that that can only reliably be done by telling the client to use an Incognito session. nginx-in-the-middle will not be able to do it, without lots of extra state being stored across requests. (Which may well be doable by you writing the code to do it; but I suspect that it can't be done purely in stock nginx configuration.) > I tried adding the following: > > proxy_set_header Cookie ""; > add_header Set-Cookie "cookie_name=; Expires=Thu, 01 Jan 1970 00:00:01 GMT;"; } > > but it still does not work correctly. I suspect that it will be useful to learn what exactly you consider an Incognito session to be. My understanding is that, among other things, the client will choose not to send any cookies that had been set outside of this session, but will choose to send cookies that were set within this session. If that is correct, then "never sending cookies" is not the correct design. The client can know when the cookies that it has were set; for nginx to know that, it would need to keep track of the Set-Cookie responses for each client, and only allow through matching Cookie requests from the matching client. And by default, nginx does not know or care about that information. > Is there a way to do this? Probably not trivially. Good luck with it! f -- Francis Daly francis at daoine.org From mureninc at gmail.com Wed Dec 20 17:37:16 2023 From: mureninc at gmail.com (Constantine A. Murenin) Date: Wed, 20 Dec 2023 11:37:16 -0600 Subject: Forcing incognito mode on a reverse proxy In-Reply-To: References: Message-ID: On 16/12/2023, Saint Michael wrote: > I have a reverse proxy but for security reasons, I need to force the > client to work the closest to an Incognito session as possible. > I tried adding the following: > > proxy_set_header Cookie ""; > add_header Set-Cookie "cookie_name=; Expires=Thu, 01 Jan 1970 00:00:01 > GMT;"; } > > but it still does not work correctly. > > Is there a way to do this? Copied from my 2013 answer at https://serverfault.com/a/467774: This can be addressed through nginx with the following directives placed within the server context: proxy_hide_header Set-Cookie; proxy_ignore_headers Set-Cookie; # important! Remember the special inheritance rules for proxy_set_header: # http://nginx.org/ru/docs/http/ngx_http_proxy_module.html#proxy_set_header proxy_set_header Cookie ""; All three directives above are very important: * proxy_hide_header ensures the header will not be passed back to the client, * proxy_ignore_headers ensures that the header will not automatically disable caching within nginx and, finally, * proxy_set_header ensures that a client cannot pass any prior cookies to the webapp and spoil your cache. Note my comment regarding proxy_set_header inheritance — you cannot nest this directive (have to define all or none at a given level). C. From clima.gabrielphoto at gmail.com Thu Dec 21 12:34:35 2023 From: clima.gabrielphoto at gmail.com (Clima Gabriel) Date: Thu, 21 Dec 2023 14:34:35 +0200 Subject: ngx_quic_create_stream segfault Message-ID: Hello everyone, My Nginx worker process has frequent segfaults on this codepath. (ngx_quic_create_stream) Here are some observations I have made so far. 1. The faults happen with tcmalloc and malloc so this is not the issue. 2. master_process is on 3. 1 worker is enough 4. HTTP3 requests need to come in fairly frequently, at least 2 per second 5. At least one http1 or http2 request needs to be received as well, regardless of what port I would really appreciate any suggestions on where do I continue investigating this. #0 0x00007ffb0b01876a in (anonymous namespace)::do_memalign(unsigned long, unsigned long) () from /lib64/libtcmalloc.so.4 #1 0x00007ffb0b037010 in tc_posix_memalign () from /lib64/libtcmalloc.so.4 #2 0x00000000005a7041 in ngx_memalign (alignment=alignment at entry=16, size=size at entry=16384, log=log at entry=0x60b44d8) at src/os/unix/ngx_alloc.c:57 #3 0x000000000058067c in ngx_create_pool (size=size at entry=16384, log=0x60b44d8) at src/core/ngx_palloc.c:23 #4 0x00000000005ca7f0 in ngx_quic_create_stream () at src/event/quic/ngx_event_quic_streams.c:685 #5 0x00000000005cb1f6 in ngx_quic_get_stream () at src/event/quic/ngx_event_quic_streams.c:458 #6 0x00000000005cc745 in ngx_quic_handle_stream_frame (c=c at entry=0x8417068, pkt=pkt at entry=0x7ffef1048250, frame=frame at entry=0x7ffef1048140) at src/event/quic/ngx_event_quic_streams.c:1265 #7 0x00000000005bd5f3 in ngx_quic_handle_frames (c=0x8417068, pkt=0x7ffef1048250) at src/event/quic/ngx_event_quic.c:1254 #8 0x00000000005bf022 in ngx_quic_handle_packet (pkt=0x7ffef1048250, conf=0x0, c=0x8417068) at src/event/quic/ngx_event_quic.c:850 #9 ngx_quic_handle_datagram (c=c at entry=0x8417068, b=0x7ffef1048480, conf=conf at entry=0x0) at src/event/quic/ngx_event_quic.c:700 #10 0x00000000005bff6b in ngx_quic_input_handler (rev=0x95965a0) at src/event/quic/ngx_event_quic.c:443 #11 0x00000000005c0884 in ngx_quic_recvmsg (ev=0x95963c0) at src/event/quic/ngx_event_quic_udp.c:195 #12 0x00000000005af0c8 in ngx_epoll_process_events (cycle=0x57aa050, timer=, flags=) at src/event/modules/ngx_epoll_module.c:901 #13 0x00000000005a2941 in ngx_process_events_and_timers (cycle=cycle at entry=0x57aa050) at src/event/ngx_event.c:251 #14 0x00000000005abd19 in ngx_worker_process_cycle (cycle=0x57aa050, data=) at src/os/unix/ngx_process_cycle.c:1135 #15 0x00000000005aa323 in ngx_spawn_process (cycle=cycle at entry=0x57aa050, proc=proc at entry=0x5abc00 , data=data at entry=0x15, name=name at entry=0xbc6066 "worker process", respawn=respawn at entry=-4) at src/os/unix/ngx_process.c:209 #16 0x00000000005ac798 in ngx_start_worker_processes (cycle=cycle at entry=0x57aa050, n=54, type=type at entry=-4, worker_spawn_start_index=worker_spawn_start_index at entry=0x0, workers_to_exclude=workers_to_exclude at entry=0x0) at src/os/unix/ngx_process_cycle.c:600 #17 0x00000000005ae069 in ngx_master_process_cycle (cycle=0x57aa050) at src/os/unix/ngx_process_cycle.c:424 #18 0x000000000057d19c in main (argc=1, argv=) at src/core/nginx.c:523 -------------- next part -------------- An HTML attachment was scrubbed... URL: From noloader at gmail.com Thu Dec 21 15:02:24 2023 From: noloader at gmail.com (Jeffrey Walton) Date: Thu, 21 Dec 2023 10:02:24 -0500 Subject: ngx_quic_create_stream segfault In-Reply-To: References: Message-ID: On Thu, Dec 21, 2023 at 7:35 AM Clima Gabriel wrote: > > Hello everyone, > > My Nginx worker process has frequent segfaults on this codepath. (ngx_quic_create_stream) > Here are some observations I have made so far. > 1. The faults happen with tcmalloc and malloc so this is not the issue. > 2. master_process is on > 3. 1 worker is enough > 4. HTTP3 requests need to come in fairly frequently, at least 2 per second > 5. At least one http1 or http2 request needs to be received as well, regardless of what port > > I would really appreciate any suggestions on where do I continue investigating this. > > #0 0x00007ffb0b01876a in (anonymous namespace)::do_memalign(unsigned long, unsigned long) () from /lib64/libtcmalloc.so.4 > #1 0x00007ffb0b037010 in tc_posix_memalign () from /lib64/libtcmalloc.so.4 > [...] It's good you have debug symbols available. A 'bt full' may be more helpful so parameter values are available in the stack trace. Jeff From akashshrivastava.mu.99 at gmail.com Fri Dec 22 06:35:51 2023 From: akashshrivastava.mu.99 at gmail.com (Akash Shrivastava) Date: Fri, 22 Dec 2023 12:05:51 +0530 Subject: Support on Nginx-ingress V3.4.0 Message-ID: Hi there, Urgent support needed on Nginx-ingress 3.4.0 https://github.com/nginxinc/kubernetes-ingress/pull/4428 What does this mean? What consideration needs to be taken before uploading this version? -------------- next part -------------- An HTML attachment was scrubbed... URL: From clima.gabrielphoto at gmail.com Fri Dec 22 09:59:22 2023 From: clima.gabrielphoto at gmail.com (Clima Gabriel) Date: Fri, 22 Dec 2023 11:59:22 +0200 Subject: ngx_quic_create_stream segfault In-Reply-To: References: Message-ID: Thanks. I ended up using valgrind and got much closer to the answer. Arguments I used: valgrind --leak-check=full --show-leak-kinds=all --trace-children=yes --track-origins=yes --verbose --log-file=valgrind-output.txt /root/nginx/objs/nginx -c /etc/nginx/nginx.conf That led me to find that the HTTP and QUIC requests somehow end upsing the same pointer to http_connection_t, which seems obviously wrong. #0 ngx_SSL_early_cb_fn (s=0x55ae3ad5cae0, al=0x7fff1cf4c8f4, arg=0x7f3417bf4b00) at src/event/ngx_event_openssl.c:1949 #1 0x00007f3578bc2eba in ?? () from /lib/x86_64-linux-gnu/libssl.so.3 #2 0x00007f3578bb4b8c in ?? () from /lib/x86_64-linux-gnu/libssl.so.3 #3 0x00007f3578bb6608 in ?? () from /lib/x86_64-linux-gnu/libssl.so.3 #4 0x00007f3578b89d42 in SSL_read_early_data () from /lib/x86_64-linux-gnu/libssl.so.3 *#5 0x000055ae379f85cb in ngx_ssl_try_early_data (c=0x7f3417bf4b00) at src/event/ngx_event_openssl.c:2229*#6 0x000055ae379f7f6f in ngx_ssl_handshake (c=0x7f3417bf4b00) at src/event/ngx_event_openssl.c:1998 #7 0x000055ae37a3f01d in ngx_http_ssl_handshake (rev=0x7f34175f33d0) at src/http/ngx_http_request.c:785 #8 0x000055ae379f0cca in ngx_epoll_process_events (cycle=0x55ae39d80390, timer=3101, flags=1) at src/event/modules/ngx_epoll_module.c:901 #9 0x000055ae379daacb in ngx_process_events_and_timers (cycle=0x55ae39d80390) at src/event/ngx_event.c:251 #10 0x000055ae379ed424 in ngx_worker_process_cycle (cycle=0x55ae39d80390, data=0x0) at src/os/unix/ngx_process_cycle.c:936 #11 0x000055ae379e8c49 in ngx_spawn_process (cycle=0x55ae39d80390, proc=0x55ae379ed2c3 , data=0x0, name=0x55ae37fa1cc5 "worker process", respawn=-3) at src/os/unix/ngx_process.c:209 #12 0x000055ae379ebf69 in ngx_start_worker_processes (cycle=0x55ae39d80390, n=1, type=-3) at src/os/unix/ngx_process_cycle.c:525 #13 0x000055ae379eb480 in ngx_master_process_cycle (cycle=0x55ae39d80390) at src/os/unix/ngx_process_cycle.c:279 #14 0x000055ae3799d8d1 in main (argc=3, argv=0x7fff1cf4e498) at src/core/nginx.c:489 (gdb) continue Continuing. Breakpoint 1, ngx_SSL_early_cb_fn (s=0x55ae3ad5cae0, al=0x7fff1cf4d214, arg=0x7f3417bf4b00) at src/event/ngx_event_openssl.c:1949 1949 ngx_SSL_early_cb_fn(SSL *s, int *al, void *arg) { #0 ngx_SSL_early_cb_fn (s=0x55ae3ad5cae0, al=0x7fff1cf4d214, arg=0x7f3417bf4b00) at src/event/ngx_event_openssl.c:1949 #1 0x00007f3578bc2eba in ?? () from /lib/x86_64-linux-gnu/libssl.so.3 #2 0x00007f3578bb4b8c in ?? () from /lib/x86_64-linux-gnu/libssl.so.3 #3 0x00007f3578bb6608 in ?? () from /lib/x86_64-linux-gnu/libssl.so.3 *#4 0x000055ae37a1e1c4 in ngx_quic_crypto_input (c=0x7f3417bf4b00, data=0x7fff1cf4d680, level=ssl_encryption_initial) at src/event/quic/ngx_event_quic_ssl.c:412*#5 0x000055ae37a1dffc in ngx_quic_handle_crypto_frame (c=0x7f3417bf4b00, pkt=0x7fff1cf4d880, frame=0x7fff1cf4d6e0) at src/event/quic/ngx_event_quic_ssl.c:358 #6 0x000055ae37a0af0f in ngx_quic_handle_frames (c=0x7f3417bf4b00, pkt=0x7fff1cf4d880) at src/event/quic/ngx_event_quic.c:1242 #7 0x000055ae37a0a744 in ngx_quic_handle_payload (c=0x7f3417bf4b00, pkt=0x7fff1cf4d880) at src/event/quic/ngx_event_quic.c:1054 #8 0x000055ae37a0a12b in ngx_quic_handle_packet (c=0x7f3417bf4b00, conf=0x55ae39e42ae8, pkt=0x7fff1cf4d880) at src/event/quic/ngx_event_quic.c:946 #9 0x000055ae37a09630 in ngx_quic_handle_datagram (c=0x7f3417bf4b00, b=0x55ae3ac65f30, conf=0x55ae39e42ae8) at src/event/quic/ngx_event_quic.c:700 #10 0x000055ae37a07eb1 in ngx_quic_run (c=0x7f3417bf4b00, conf=0x55ae39e42ae8) at src/event/quic/ngx_event_quic.c:204 #11 0x000055ae37aace43 in ngx_http_v3_init_stream (c=0x7f3417bf4b00) at src/http/v3/ngx_http_v3_request.c:75 #12 0x000055ae37a3db4a in ngx_http_init_connection (c=0x7f3417bf4b00) at src/http/ngx_http_request.c:329 #13 0x000055ae37a0c3cd in ngx_quic_recvmsg (ev=0x7f34175f30d0) at src/event/quic/ngx_event_quic_udp.c:339 #14 0x000055ae379f0cca in ngx_epoll_process_events (cycle=0x55ae39d80390, timer=5653, flags=1) at src/event/modules/ngx_epoll_module.c:901 #15 0x000055ae379daacb in ngx_process_events_and_timers (cycle=0x55ae39d80390) at src/event/ngx_event.c:251 #16 0x000055ae379ed424 in ngx_worker_process_cycle (cycle=0x55ae39d80390, data=0x0) at src/os/unix/ngx_process_cycle.c:936 #17 0x000055ae379e8c49 in ngx_spawn_process (cycle=0x55ae39d80390, proc=0x55ae379ed2c3 , data=0x0, name=0x55ae37fa1cc5 "worker process", respawn=-3) at src/os/unix/ngx_process.c:209 #18 0x000055ae379ebf69 in ngx_start_worker_processes (cycle=0x55ae39d80390, n=1, type=-3) at src/os/unix/ngx_process_cycle.c:525 #19 0x000055ae379eb480 in ngx_master_process_cycle (cycle=0x55ae39d80390) at src/os/unix/ngx_process_cycle.c:279 #20 0x000055ae3799d8d1 in main (argc=3, argv=0x7fff1cf4e498) at src/core/nginx.c:489 On Thu, Dec 21, 2023 at 5:03 PM Jeffrey Walton wrote: > On Thu, Dec 21, 2023 at 7:35 AM Clima Gabriel > wrote: > > > > Hello everyone, > > > > My Nginx worker process has frequent segfaults on this codepath. > (ngx_quic_create_stream) > > Here are some observations I have made so far. > > 1. The faults happen with tcmalloc and malloc so this is not the issue. > > 2. master_process is on > > 3. 1 worker is enough > > 4. HTTP3 requests need to come in fairly frequently, at least 2 per > second > > 5. At least one http1 or http2 request needs to be received as well, > regardless of what port > > > > I would really appreciate any suggestions on where do I continue > investigating this. > > > > #0 0x00007ffb0b01876a in (anonymous namespace)::do_memalign(unsigned > long, unsigned long) () from /lib64/libtcmalloc.so.4 > > #1 0x00007ffb0b037010 in tc_posix_memalign () from > /lib64/libtcmalloc.so.4 > > [...] > > It's good you have debug symbols available. > > A 'bt full' may be more helpful so parameter values are available in > the stack trace. > > Jeff > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx > -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri Dec 22 14:13:34 2023 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 22 Dec 2023 17:13:34 +0300 Subject: ngx_quic_create_stream segfault In-Reply-To: References: Message-ID: Hello! On Fri, Dec 22, 2023 at 11:59:22AM +0200, Clima Gabriel wrote: > Thanks. > I ended up using valgrind and got much closer to the answer. > Arguments I used: > valgrind --leak-check=full --show-leak-kinds=all --trace-children=yes > --track-origins=yes --verbose --log-file=valgrind-output.txt > /root/nginx/objs/nginx -c /etc/nginx/nginx.conf > That led me to find that the HTTP and QUIC requests somehow end upsing the > same pointer to http_connection_t, which seems obviously wrong. This might be perfectly correct as long as the original connection was closed. > #0 ngx_SSL_early_cb_fn (s=0x55ae3ad5cae0, al=0x7fff1cf4c8f4, > arg=0x7f3417bf4b00) at src/event/ngx_event_openssl.c:1949 There is no such function in nginx, so it looks like you are using some 3rd party modifications. You may want to start with compiling vanilla nginx as available from nginx.org without any 3rd party modules and/or patches and testing if you are able to reproduce the problem. [...] -- Maxim Dounin http://mdounin.ru/ From jordanc.carter at outlook.com Fri Dec 22 15:00:56 2023 From: jordanc.carter at outlook.com (J Carter) Date: Fri, 22 Dec 2023 15:00:56 +0000 Subject: Support on Nginx-ingress V3.4.0 In-Reply-To: References: Message-ID: Hello, On Fri, 22 Dec 2023 12:05:51 +0530 Akash Shrivastava wrote: > Hi there, > Urgent support needed on Nginx-ingress 3.4.0 I'd recommend posting ingress controller related questions on the discussions section of it's Github repo. https://github.com/nginxinc/kubernetes-ingress/discussions > > https://github.com/nginxinc/kubernetes-ingress/pull/4428 > > What does this mean? > >From an end user's perspective all they've done is merge all CRDs that used to be in /deployments/crds/ into one file /deploy/crds.yaml, for convenience. There is also now kustomize templates to re-generate CRDs (I can't think of a reason for a user to use those unless you want to apply patches, just use the single CRD). You can see docs on that here, and how to use the single CRD and kustomize templates here. https://docs.nginx.com/nginx-ingress-controller/installation/installing-nic/installation-with-manifests/#create-custom-resources > > > What consideration needs to be taken before uploading this version? Generally the release notes will list from version to version if there is a breaking change, or some special consideration needs to be made. https://docs.nginx.com/nginx-ingress-controller/releases/ At least from last version, there doesn't appear to be any. From clima.gabrielphoto at gmail.com Fri Dec 22 15:02:02 2023 From: clima.gabrielphoto at gmail.com (Clima Gabriel) Date: Fri, 22 Dec 2023 17:02:02 +0200 Subject: ngx_quic_create_stream segfault In-Reply-To: References: Message-ID: Hello Maxim, You're right. Disabling the ssl-ja3 module was sufficient to stop the segfaults. Thanks! On Fri, Dec 22, 2023 at 4:14 PM Maxim Dounin wrote: > Hello! > > On Fri, Dec 22, 2023 at 11:59:22AM +0200, Clima Gabriel wrote: > > > Thanks. > > I ended up using valgrind and got much closer to the answer. > > Arguments I used: > > valgrind --leak-check=full --show-leak-kinds=all --trace-children=yes > > --track-origins=yes --verbose --log-file=valgrind-output.txt > > /root/nginx/objs/nginx -c /etc/nginx/nginx.conf > > That led me to find that the HTTP and QUIC requests somehow end upsing > the > > same pointer to http_connection_t, which seems obviously wrong. > > This might be perfectly correct as long as the original connection > was closed. > > > #0 ngx_SSL_early_cb_fn (s=0x55ae3ad5cae0, al=0x7fff1cf4c8f4, > > arg=0x7f3417bf4b00) at src/event/ngx_event_openssl.c:1949 > > There is no such function in nginx, so it looks like you are using > some 3rd party modifications. > > You may want to start with compiling vanilla nginx as available > from nginx.org without any 3rd party modules and/or patches and > testing if you are able to reproduce the problem. > > [...] > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx > -------------- next part -------------- An HTML attachment was scrubbed... URL: From drodriguez at unau.edu.ar Tue Dec 26 22:57:41 2023 From: drodriguez at unau.edu.ar (Daniel A. Rodriguez) Date: Tue, 26 Dec 2023 19:57:41 -0300 Subject: Wrong content served Message-ID: Hi, This behavior is driving me crazy. Currently have more than 30 sites behind this reverse proxy, but the latest is refusing to work. Config is simple and pretty similar between them all. server {     listen 80;     server_name material.av.domain;     include /etc/nginx/snippets/location-letsencrypt.conf; #    return 301 https://$server_name$request_uri; } If I point the browser to material.av.domain got redirected to another sub-domain, among the 30 mentioned before. However, everything else works just fine. The configuration of the domain it redirects to is the same as the site I am trying to deploy. Even tried changing URL, but result is the same. DNS is resolving to correct IP. Got no error logged neither. Since I'm stuck, I welcome any ideas and/or suggestions. --
________________________________________________
Daniel A. Rodriguez
Informática, Conectividad y Sistemas
Universidad Nacional del Alto Uruguay
San Vicente - Misiones - Argentina
informatica.unau.edu.ar
From francis at daoine.org Tue Dec 26 23:59:22 2023 From: francis at daoine.org (Francis Daly) Date: Tue, 26 Dec 2023 23:59:22 +0000 Subject: Wrong content served In-Reply-To: References: Message-ID: <20231226235922.GH6038@daoine.org> On Tue, Dec 26, 2023 at 07:57:41PM -0300, Daniel A. Rodriguez wrote: Hi there, > This behavior is driving me crazy. Currently have more than 30 sites behind > this reverse proxy, but the latest is refusing to work. Can you provide more details? > Config is simple and pretty similar between them all. "include" means "anything in that file is effectively in this config". Nobody but you knows what is in that file. > server { >     listen 80; >     server_name material.av.domain; > >     include /etc/nginx/snippets/location-letsencrypt.conf; > > #    return 301 https://$server_name$request_uri; > > } Your test request is: $ curl -i http://material.av.domain/ What response do you get? What response do you want to get instead? The "return" is commented out, so unless there is something surprising in the location-letsencrypt.conf file, I would expect a http 200 response with the content of "the default" index.html file. > If I point the browser to material.av.domain got redirected to another > sub-domain, among the 30 mentioned before. However, everything else works > just fine. Can you show the response to the "curl" request, to see whether "redirect" is a http 301 from the web server, or is something like a http 200 from the web server with maybe some javascript content that redirects to "the wrong" place? Cheers, f -- Francis Daly francis at daoine.org From jeff.dyke at gmail.com Wed Dec 27 00:15:23 2023 From: jeff.dyke at gmail.com (Jeff Dyke) Date: Tue, 26 Dec 2023 19:15:23 -0500 Subject: Wrong content served In-Reply-To: <20231226235922.GH6038@daoine.org> References: <20231226235922.GH6038@daoine.org> Message-ID: In addition to Francis' always helpful ask. You have a domain problem with material.av.domain and it may be from /etc/hosts all the way to public DNS. Or, incorrectly supplied *location-letsencrypt.conf.* If you provide that file contents, you'll likely see your own error as you send it (i've done it dozens of times, its not an insult) On Tue, Dec 26, 2023 at 6:59 PM Francis Daly wrote: > On Tue, Dec 26, 2023 at 07:57:41PM -0300, Daniel A. Rodriguez wrote: > > Hi there, > > > This behavior is driving me crazy. Currently have more than 30 sites > behind > > this reverse proxy, but the latest is refusing to work. > > Can you provide more details? > > > Config is simple and pretty similar between them all. > > "include" means "anything in that file is effectively in this > config". Nobody but you knows what is in that file. > > > server { > > listen 80; > > server_name material.av.domain; > > > > include /etc/nginx/snippets/location-letsencrypt.conf; > > > > # return 301 https://$server_name$request_uri; > > > > } > > Your test request is: > > $ curl -i http://material.av.domain/ > > What response do you get? What response do you want to get instead? > > The "return" is commented out, so unless there is something surprising > in the location-letsencrypt.conf file, I would expect a http 200 response > with the content of "the default" index.html file. > > > If I point the browser to material.av.domain got redirected to another > > sub-domain, among the 30 mentioned before. However, everything else works > > just fine. > > Can you show the response to the "curl" request, to see whether "redirect" > is a http 301 from the web server, or is something like a http 200 from > the web server with maybe some javascript content that redirects to > "the wrong" place? > > Cheers, > > f > -- > Francis Daly francis at daoine.org > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx > -------------- next part -------------- An HTML attachment was scrubbed... URL: From rejaine at bhz.jamef.com.br Fri Dec 29 12:54:30 2023 From: rejaine at bhz.jamef.com.br (Rejaine Monteiro) Date: Fri, 29 Dec 2023 09:54:30 -0300 Subject: Calculating requests per second, per IP address Message-ID: Hi all, I´m running Nginx community edition and need to implement rate limiting There's plenty of guides out there on how to do this, but no guides on how to get real values/stats from the access logs What I need to get from the NGINX access logs is: - Requests per minute, per IP address - Average requests per minute, derived from all IP addresses - Max requests per minute, per IP address We have a few endpoints with different functionalities and we simply cannot have a common rule that works for everyone. Any tips on a tool or script that could help generate this type of information (if possible in real time or collect this data for future analysis)? I appreciate any tips. -- *Esta mensagem pode conter informações confidenciais ou privilegiadas, sendo seu sigilo protegido por lei. Se você não for o destinatário ou a pessoa autorizada a receber esta mensagem, não pode usar, copiar ou divulgar as informações nela contidas ou tomar qualquer ação baseada nessas informações. Se você recebeu esta mensagem por engano, por favor avise imediatamente ao remetente, respondendo o e-mail e em seguida apague-o. Agradecemos sua cooperação.* -------------- next part -------------- An HTML attachment was scrubbed... URL: From mdounin at mdounin.ru Fri Dec 29 14:07:04 2023 From: mdounin at mdounin.ru (Maxim Dounin) Date: Fri, 29 Dec 2023 17:07:04 +0300 Subject: Calculating requests per second, per IP address In-Reply-To: References: Message-ID: Hello! On Fri, Dec 29, 2023 at 09:54:30AM -0300, Rejaine Monteiro wrote: > Hi all, > > I´m running Nginx community edition and need to implement rate limiting > > There's plenty of guides out there on how to do this, but no guides on how > to get real values/stats from the access logs > > > What I need to get from the NGINX access logs is: > > - Requests per minute, per IP address > > - Average requests per minute, derived from all IP addresses > > - Max requests per minute, per IP address > > We have a few endpoints with different functionalities and we simply cannot > have a common rule that works for everyone. > > Any tips on a tool or script that could help generate this type of > information (if possible in real time or collect this data for future > analysis)? Note that nginx's limit_req module implements limiting based on the "leaky bucket" algorithm, which permits traffic bursts when properly configured, and therefore there is no need to calculate exact rates of requests per minute and so on. Rather, I would recommend to focus on the number of requests a typical legitimate user can generate to the protected resources within a short period of time, and set large enough burst= value to ensure legitimate users are not limited. Then use some reasonable long-term rate - this does not need to be exact, but rather somewhat reasonable, for example, to cover typical number of requests per day from a legitimate user. Note well that using "nodelay" (or "delay=N") is recommended with such approach, see http://nginx.org/r/limit_req for details. -- Maxim Dounin http://mdounin.ru/ From rejaine at bhz.jamef.com.br Fri Dec 29 15:33:30 2023 From: rejaine at bhz.jamef.com.br (Rejaine Monteiro) Date: Fri, 29 Dec 2023 12:33:30 -0300 Subject: Calculating requests per second, per IP address In-Reply-To: References: Message-ID: Thanks for the explanation I understand how Nginx works with limit_req, and what I'm looking for is precisely to find this "reasonable value" for each endpoint (testing the application side behavior is also important, but since there are multiple endpoints with hundreds of different functions, it might be a bit challenging to analyze in a short amount of time). We already use rate_limit in one of these applications, but I had to conduct a series of tests to arrive at a reasonable value. As we have several other services, I would like an analysis and monitoring tool to track requests over time and be able to assess requests per minute/second for each endpoint. This way, I can initially get a better idea and later make a more customized adjustment. For this specific endpoint, for example, I'm using: limit_req_zone $binary_remote_addr zone=api:10m rate=1r/s limit_req zone=api burst=10 nodelay On Fri, Dec 29, 2023 at 11:07 AM Maxim Dounin wrote: > Hello! > > On Fri, Dec 29, 2023 at 09:54:30AM -0300, Rejaine Monteiro wrote: > > > Hi all, > > > > I´m running Nginx community edition and need to implement rate limiting > > > > There's plenty of guides out there on how to do this, but no guides on > how > > to get real values/stats from the access logs > > > > > > What I need to get from the NGINX access logs is: > > > > - Requests per minute, per IP address > > > > - Average requests per minute, derived from all IP addresses > > > > - Max requests per minute, per IP address > > > > We have a few endpoints with different functionalities and we simply > cannot > > have a common rule that works for everyone. > > > > Any tips on a tool or script that could help generate this type of > > information (if possible in real time or collect this data for future > > analysis)? > > Note that nginx's limit_req module implements limiting based on > the "leaky bucket" algorithm, which permits traffic bursts when > properly configured, and therefore there is no need to calculate > exact rates of requests per minute and so on. > > Rather, I would recommend to focus on the number of requests a > typical legitimate user can generate to the protected resources > within a short period of time, and set large enough burst= value > to ensure legitimate users are not limited. Then use some > reasonable long-term rate - this does not need to be exact, but > rather somewhat reasonable, for example, to cover typical number > of requests per day from a legitimate user. > > Note well that using "nodelay" (or "delay=N") is recommended with > such approach, see http://nginx.org/r/limit_req for details. > > -- > Maxim Dounin > http://mdounin.ru/ > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx > -- *Esta mensagem pode conter informações confidenciais ou privilegiadas, sendo seu sigilo protegido por lei. Se você não for o destinatário ou a pessoa autorizada a receber esta mensagem, não pode usar, copiar ou divulgar as informações nela contidas ou tomar qualquer ação baseada nessas informações. Se você recebeu esta mensagem por engano, por favor avise imediatamente ao remetente, respondendo o e-mail e em seguida apague-o. Agradecemos sua cooperação.* -------------- next part -------------- An HTML attachment was scrubbed... URL: