From openedgarage at gmail.com Wed Aug 2 10:42:46 2023 From: openedgarage at gmail.com (O G) Date: Wed, 2 Aug 2023 13:42:46 +0300 Subject: Reverse proxying with URL Rewrite Message-ID: Dear Community, I would like to reverse proxy the following: https://test.us/citizensolutions/AB* to *https://*AB*.test.us/ and all the subdirectories so that all files like https://test.us/citizensolutions/AB/css/*.css reverse proxy to https://*AB*.test.us/css/*.css AB in the above example can be any dynamic input that nginx needs to treat as a variable. Can you please advise how to achieve this? Thanks Alex -------------- next part -------------- An HTML attachment was scrubbed... URL: From peljasz at yahoo.co.uk Thu Aug 3 12:00:11 2023 From: peljasz at yahoo.co.uk (lejeczek) Date: Thu, 3 Aug 2023 14:00:11 +0200 Subject: wordpress - Primary script unknown References: Message-ID: Hi guys. This must be trivial - I certainly thought - but it confused the hell out of me: I'm doing - what many must have done before me - this: ...   root         /var/www/ale.xyx_wordpress;   location / {     try_files $uri $uri/ /index.php?$args;   }   location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {     expires max;     log_not_found off;   }   location = /favicon.ico {     log_not_found off;     access_log off;   }   location = /robots.txt {     allow all;     log_not_found off;     access_log off;   }   index index.php index.html index.htm;   location ~ \.(php|phar)(/.*)?$ {     fastcgi_split_path_info ^(.+\.(?:php|phar))(/.*)$;     fastcgi_intercept_errors on;     fastcgi_index  index.php;     include        fastcgi_params;     fastcgi_param  SCRIPT_FILENAME $document_root$fastcgi_script_name;     #fastcgi_param  PATH_INFO $fastcgi_path_info;     fastcgi_pass   php-fpm;   } # upstream upstream php-fpm {         server unix:/run/php-fpm/www.sock; } when I go to web root, Nginx logs: ... 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 http script copy: "SERVER_PORT" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 http script var: "443" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "SERVER_PORT: 443" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 http script copy: "SERVER_NAME" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 http script var: "ale.xyz" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "SERVER_NAME: ale.xyz" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 http script copy: "REDIRECT_STATUS" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 http script copy: "200" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "REDIRECT_STATUS: 200" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 http script copy: "SCRIPT_FILENAME" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 http script var: "/var/www/ale.xyx_wordpress" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 http script var: "/index.php" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "SCRIPT_FILENAME: /var/www/ale.xyx_wordpress/index.php" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_HOST: ale.xyz" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_CACHE_CONTROL: max-age=0" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_UPGRADE_INSECURE_REQUESTS: 1" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_USER_AGENT: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/115.0.0.0 Safari/537.36" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_ACCEPT: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_SEC_GPC: 1" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_ACCEPT_LANGUAGE: en-GB,en;q=0.9" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_SEC_FETCH_SITE: none" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_SEC_FETCH_MODE: navigate" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_SEC_FETCH_USER: ?1" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_SEC_FETCH_DEST: document" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_ACCEPT_ENCODING: gzip, deflate, br" 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: "HTTP_COOKIE: wordpress_test_cookie=WP%20Cookie%20check; wp-settings-time-1=1686766849; PHPSESSID=caa771c30b66d7bf6c86ab15b132479f" ... and eventually: ... 2023/08/03 13:50:24 [error] 1112963#1112963: *27 FastCGI sent in stderr: "Primary script unknown" while reading response header from upstream, client: 10.3.9.144, server: ale.xyz, request: "GET / HTTP/2.0", upstream: "fastcgi://unix:/run/php-fpm/www.sock:", host: "ale.xyz" ... This is pretty much vanilla-default on Centos 9, those configs are - what am I missing? I hope it's be trivial for some people here and I could get some pointers. many thanks, L. -------------- next part -------------- An HTML attachment was scrubbed... URL: From y.abadi at f5.com Fri Aug 4 15:29:00 2023 From: y.abadi at f5.com (Yuval Abadi) Date: Fri, 4 Aug 2023 15:29:00 +0000 Subject: master process Message-ID: Hi I am writing plugin code that read the configuration, nginx.conf, and do some periodic continues processing. In the configuration CBs, the process PID is not the MASTER, after NGINX start run he MASTER pid is other process. It seems that temporary process read the nginx.conf. and then spawn the MASTER. I don’t want to use the workers processes, I need only one instance. and I need the option to stop NGINX run, same as happened when nginx.conf is wrong, which is what my processing does at init. How could I add code that run under the MASTER? Is any CB , event? Or, could use the initiating process, the one that read the conf, to do continues processing ? Thanks Yuval Abadi -------------- next part -------------- An HTML attachment was scrubbed... URL: From francis at daoine.org Sat Aug 5 06:23:21 2023 From: francis at daoine.org (Francis Daly) Date: Sat, 5 Aug 2023 07:23:21 +0100 Subject: wordpress - Primary script unknown In-Reply-To: References: Message-ID: <20230805062321.GL6038@daoine.org> On Thu, Aug 03, 2023 at 02:00:11PM +0200, lejeczek via nginx wrote: Hi there, > 2023/08/03 13:50:24 [debug] 1112963#1112963: *27 fastcgi param: > "SCRIPT_FILENAME: /var/www/ale.xyx_wordpress/index.php" > 2023/08/03 13:50:24 [error] 1112963#1112963: *27 FastCGI sent in stderr: > "Primary script unknown" while reading response header from upstream, > client: 10.3.9.144, server: ale.xyz, request: "GET / HTTP/2.0", upstream: > "fastcgi://unix:/run/php-fpm/www.sock:", host: "ale.xyz" > This is pretty much vanilla-default on Centos 9, those configs are - what am > I missing? "Primary script unknown" is a message from the fastcgi server saying that it is unable to access the file that it has been asked to use. Usually, that filename comes from a specific one of the SCRIPT_FILENAME values it receives (if it receives more than one). Your fastcgi server might log more somewhere about what it thinks it was doing. But can you check: can the user that the fastcgi service is running as, read the file /var/www/ale.xyx_wordpress/index.php from the perspective of the fastcgi service? That is -- do directory permissions towards that file and file permissions on that file allow that user/group to read? Does the file exist at that path name, if the fastcgi service is running in a chroot or other confined context? Do selinux or other access control mechanisms allow the fastcgi service to read the file? Some other (less likely?) possibilities include -- does your nginx send more than one value for SCRIPT_FILENAME; and if so, is your fastcgi server trying to use a different one? Does your fastcgi server actually use SCRIPT_FILENAME, or does it use some other param or combination of params? Cheers, f -- Francis Daly francis at daoine.org From francis at daoine.org Sun Aug 6 08:21:03 2023 From: francis at daoine.org (Francis Daly) Date: Sun, 6 Aug 2023 09:21:03 +0100 Subject: Reverse proxying with URL Rewrite In-Reply-To: References: Message-ID: <20230806082103.GM6038@daoine.org> On Wed, Aug 02, 2023 at 01:42:46PM +0300, O G wrote: Hi there, > I would like to reverse proxy the following: > > https://test.us/citizensolutions/AB* to *https://*AB*.test.us/ and all the > subdirectories so that > all files like https://test.us/citizensolutions/AB/css/*.css reverse proxy > to https://*AB*.test.us/css/*.css > > AB in the above example can be any dynamic input that nginx needs to treat > as a variable. Use "map" to create the variables that you want, and then later use the variables. You say "any dynamic input"; I'm going to use "up to the next slash, which must be present"; and you will want to decide how you want to handle the cases of "AB is empty" or "AB is index.html" and the like. Strictly, I'd use "map" to create one variable, and a (pcre) regex named capture group to create the other, like so: === map $request_uri $the_bit_after_the_prefix { ~/citizensolutions/([^/]*)(?/.*) $1; default ""; } === (http://ninx.org/r/map) And then for testing, something like === location = /citizensolutions/ { return 200 "Nope; try again\n"; } location ^~ /citizensolutions/ { return 200 "Would use $the_bit_after_the_prefix, $the_rest_of_the_request\n"; } === and then when you are happy that the "Would use" is showing you what you expect in all cases you care about, replace that location{} content with === if ($the_bit_after_the_prefix = "") { return 301 $request_uri/; } proxy_pass https://$the_bit_after_the_prefix.test.us$the_rest_of_the_request; === You are effectively telling your nginx to resolve-and-access a dns name provided by the user, so you'll want to be happy that your nginx can do that, or the users will see 502 errors. Good luck with it, f -- Francis Daly francis at daoine.org From kaushalshriyan at gmail.com Tue Aug 8 16:20:25 2023 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Tue, 8 Aug 2023 21:50:25 +0530 Subject: Upload files and folders to nginx web server from the browser Message-ID: Hi, Is there a way to upload files to nginx web server https://software.mydomain.com from the browser ? I have the below nginx config file. I am running nginx version: nginx/1.24.0 on CentOS Linux release 7.9.2009 (Core) # nginx -v nginx version: nginx/1.24.0 # cat /etc/redhat-release CentOS Linux release 7.9.2009 (Core) # *cat /etc/nginx/conf.d/default.conf* > server { > listen 80 default_server; > server_name software.mydomain.com; > return 301 https://$server_name$request_uri; > } > > server { > listen 443 ssl; > server_name software.mydomain.com; > ssl_protocols TLSv1.3 TLSv1.2; > ssl_stapling off; > ssl_stapling_verify on; > ssl_certificate /etc/letsencrypt/live/ > software.mydomain.com/fullchain.pem; > ssl_certificate_key /etc/letsencrypt/live/ > software.mydomain.com/privkey.pem; > ssl_ciphers > ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384; > root /var/www/html/nbapibankingsoftwarebinaries; > auth_basic "Restricted Access"; > auth_basic_user_file /etc/nginx/.htpasswd; > autoindex on; > ssl_prefer_server_ciphers on; > ssl_dhparam /etc/ssl/certs/software.mydomain.com/dhparam.pem; > add_header Strict-Transport-Security: max-age=63072000; > > location /var/www/html/cbapibankingsoftwarebinaries { > try_files $uri $uri/ =404; > } > } > Please guide me. Thanks in Advance. Best Regards, Kaushal -------------- next part -------------- An HTML attachment was scrubbed... URL: From al-nginx at none.at Wed Aug 9 22:24:28 2023 From: al-nginx at none.at (Aleksandar Lazic) Date: Thu, 10 Aug 2023 00:24:28 +0200 Subject: Upload files and folders to nginx web server from the browser In-Reply-To: References: Message-ID: Hi. On 2023-08-08 (Di.) 18:20, Kaushal Shriyan wrote: > Hi, > > Is there a way to upload files to nginx web server > https://software.mydomain.com from the > browser ? I have the below nginx config file. I am running nginx > version: nginx/1.24.0 > on CentOS Linux release 7.9.2009 (Core) According to the page is there a Upload module for nginx. https://www.nginx.com/resources/wiki/modules/upload/ https://github.com/fdintino/nginx-upload-module If you don't want to rebuild nginx can you try to use any other upload mechanism from the many solution out there based on several programming languages. https://html.duckduckgo.com/html?q=http%20upload%20server One possible solution is this go solution. https://github.com/git001/caddyv2-upload Hope that helps Regards Alex > # nginx -v > nginx version: nginx/1.24.0 > # cat /etc/redhat-release > CentOS Linux release 7.9.2009 (Core) > # > > _cat /etc/nginx/conf.d/default.conf_ > > server { >       listen       80 default_server; >       server_name software.mydomain.com; >       return 301 https://$server_name$request_uri; > } > > server { >         listen 443 ssl; >         server_name software.mydomain.com; >         ssl_protocols          TLSv1.3 TLSv1.2; >         ssl_stapling off; >         ssl_stapling_verify on; >         ssl_certificate > /etc/letsencrypt/live/software.mydomain.com/fullchain.pem; >         ssl_certificate_key > /etc/letsencrypt/live/software.mydomain.com/privkey.pem; >         ssl_ciphers > ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384; >         root /var/www/html/nbapibankingsoftwarebinaries; >         auth_basic "Restricted Access"; >         auth_basic_user_file /etc/nginx/.htpasswd; >         autoindex on; >         ssl_prefer_server_ciphers on; >         ssl_dhparam > /etc/ssl/certs/software.mydomain.com/dhparam.pem; >         add_header Strict-Transport-Security: max-age=63072000; > > location /var/www/html/cbapibankingsoftwarebinaries { >     try_files $uri $uri/ =404; > } > } > > > Please guide me. Thanks in Advance. > > Best Regards, > > Kaushal > > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx From baalchina at gmail.com Mon Aug 14 13:56:19 2023 From: baalchina at gmail.com (baalchina) Date: Mon, 14 Aug 2023 21:56:19 +0800 Subject: Can I rewrite https to http? Message-ID: Hi all, I had an nginx as a reserve proxy, with many web sites behind. And the nginx listen in 443(https) and 80(http). But not all my sites behind support https, so in some sites ,I only listen 80, like this: * server { listen 80; listen [::]:80; server_name nossl.abc.com ; location / { proxy_pass_header Server; proxy_set_header Host $http_host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Scheme $scheme; proxy_pass http://nossl ; } }* And I got a problem here, when I using http://nossl.abc.com, it works fine. But some browser( like edge in a newly installed Windows 11), it will auto redirect to the https site(https://nossl.abc.com), then we got a default site of my nginx instead of nossl.abc.com So, can I redirect the request, when user open https://nossl.abc.com, the will be redirect to http://nossl.abc.com? Thanks. -- from:baalchina -------------- next part -------------- An HTML attachment was scrubbed... URL: From community at thoughtmaybe.com Tue Aug 15 05:18:04 2023 From: community at thoughtmaybe.com (Jore) Date: Tue, 15 Aug 2023 15:18:04 +1000 Subject: Can I rewrite https to http? In-Reply-To: References: Message-ID: <46091399-2178-00a1-6925-90c78ecf61bc@thoughtmaybe.com> Hi there, Hmm. I imagine this will be tricky to not run into infinite loops, if it's the behaviour of the browser to redirect a HTTP page to a HTTPS one. Wouldn't it just be easier to serve your site over HTTPS? Because in order to handle a redirect you'll need a valid TLS certificate anyway... Jore On 14/8/23 11:56 pm, baalchina wrote: > Hi all, > I had an nginx as a reserve proxy, with many web sites behind. And the > nginx listen in 443(https) and 80(http). > > But not all my sites behind support https, so in some sites ,I only > listen 80, like this: > /    server { >       listen 80; >       listen [::]:80; >       server_name nossl.abc.com ; >       location / { >         proxy_pass_header Server; >         proxy_set_header Host $http_host; >         proxy_set_header X-Real-IP $remote_addr; >         proxy_set_header X-Scheme $scheme; >         proxy_pass http://nossl; >        } >     }/ > > And I got a problem here, when I using http://nossl.abc.com, it works > fine. But some browser( like edge in a newly installed Windows 11), it > will auto redirect to the https site(https://nossl.abc.com), then we > got a default site of my nginx instead of nossl.abc.com > > > So, can I redirect the request, when user open https://nossl.abc.com, > the will be redirect to http://nossl.abc.com? > > Thanks. > > -- > from:baalchina > > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralph at ml.seichter.de Tue Aug 15 13:51:55 2023 From: ralph at ml.seichter.de (Ralph Seichter) Date: Tue, 15 Aug 2023 15:51:55 +0200 Subject: Can I rewrite https to http? In-Reply-To: References: Message-ID: <87a5us9zwk.fsf@ra.horus-it.com> * baalchina at gmail.com: > So, can I redirect the request, when user open https://nossl.abc.com, the > will be redirect to http://nossl.abc.com? While technically possible, you might find that connecting clients may interpret this as a "downgrade attack", because the client asked for an encrypted connection. Obtaining a certificate from Let's Encryt or similar seems a more suitable approach to me. -Ralph From mdounin at mdounin.ru Tue Aug 15 18:37:16 2023 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 15 Aug 2023 21:37:16 +0300 Subject: nginx-1.25.2 Message-ID: Changes with nginx 1.25.2 15 Aug 2023 *) Feature: path MTU discovery when using HTTP/3. *) Feature: TLS_AES_128_CCM_SHA256 cipher suite support when using HTTP/3. *) Change: now nginx uses appname "nginx" when loading OpenSSL configuration. *) Change: now nginx does not try to load OpenSSL configuration if the --with-openssl option was used to built OpenSSL and the OPENSSL_CONF environment variable is not set. *) Bugfix: in the $body_bytes_sent variable when using HTTP/3. *) Bugfix: in HTTP/3. -- Maxim Dounin http://nginx.org/ From kaushalshriyan at gmail.com Fri Aug 18 01:22:16 2023 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Fri, 18 Aug 2023 06:52:16 +0530 Subject: Enable two way SSL on Nginx web server Message-ID: Hi, I am running nginx version: nginx/1.24.0 on CentOS Linux release 7.9.2009 (Core) # nginx -v nginx version: nginx/1.24.0 # cat /etc/redhat-release CentOS Linux release 7.9.2009 (Core) # Is there a way to enable two way SSL on Nginx web server? Please guide me. Thanks in Advance. Best Regards, Kaushal -------------- next part -------------- An HTML attachment was scrubbed... URL: From al-nginx at none.at Fri Aug 18 08:09:22 2023 From: al-nginx at none.at (Aleksandar Lazic) Date: Fri, 18 Aug 2023 10:09:22 +0200 Subject: Enable two way SSL on Nginx web server In-Reply-To: References: Message-ID: <966c42ae-5213-8979-12a3-119b24f19d24@none.at> Hi Kaushal. On 2023-08-18 (Fr.) 03:22, Kaushal Shriyan wrote: > Hi, > > I am running nginx version: nginx/1.24.0  on CentOS Linux > release 7.9.2009 (Core) > > # nginx -v > nginx version: nginx/1.24.0 > # cat /etc/redhat-release > CentOS Linux release 7.9.2009 (Core) > # > > Is there a way to enable two way SSL on Nginx web server? > > Please guide me. Thanks in Advance. Haven't you asked a similar question on the tomcat-user list? https://marc.info/?t=169146150800001&r=1&w=3 There was some questions send to you which haven't received any answers, the questions here are similar. * could you be specific about what you have actually done? * what's your configuration There are several blog entries in the internet which you can try to follow to setup the mutual authentication. https://html.duckduckgo.com/html?q=nginx%20mutual%20authentication > Best Regards, > > Kaushal Regards Alex From kaushalshriyan at gmail.com Fri Aug 18 13:47:05 2023 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Fri, 18 Aug 2023 19:17:05 +0530 Subject: How to run a shell script on every request? Message-ID: Hi, I am running nginx version: nginx/1.24.0 on CentOS Linux release 7.9.2009 (Core) # nginx -v nginx version: nginx/1.24.0 # cat /etc/redhat-release CentOS Linux release 7.9.2009 (Core) # I want to run a shell script every time my nginx server receives any HTTP request. Any simple ways to do this? Please guide me. Thanks in Advance. Best Regards, Kaushal -------------- next part -------------- An HTML attachment was scrubbed... URL: From kaushalshriyan at gmail.com Fri Aug 18 13:48:53 2023 From: kaushalshriyan at gmail.com (Kaushal Shriyan) Date: Fri, 18 Aug 2023 19:18:53 +0530 Subject: Enable two way SSL on Nginx web server In-Reply-To: <966c42ae-5213-8979-12a3-119b24f19d24@none.at> References: <966c42ae-5213-8979-12a3-119b24f19d24@none.at> Message-ID: On Fri, Aug 18, 2023 at 1:39 PM Aleksandar Lazic wrote: > Hi Kaushal. > > On 2023-08-18 (Fr.) 03:22, Kaushal Shriyan wrote: > > Hi, > > > > I am running nginx version: nginx/1.24.0 on CentOS Linux > > release 7.9.2009 (Core) > > > > # nginx -v > > nginx version: nginx/1.24.0 > > # cat /etc/redhat-release > > CentOS Linux release 7.9.2009 (Core) > > # > > > > Is there a way to enable two way SSL on Nginx web server? > > > > Please guide me. Thanks in Advance. > > Haven't you asked a similar question on the tomcat-user list? > https://marc.info/?t=169146150800001&r=1&w=3 > > There was some questions send to you which haven't received any answers, > the questions here are similar. > > * could you be specific about what you have actually done? > * what's your configuration > > There are several blog entries in the internet which you can try to > follow to setup the mutual authentication. > https://html.duckduckgo.com/html?q=nginx%20mutual%20authentication > > > Best Regards, > > > > Kaushal > > Regards > Alex > Thanks Alex for the quick response. I could do it using the below configuration in nginx.conf ssl_verify_client on; > ssl_client_certificate > /etc/ssl/certs/cbclientcerts/clientfullchaincert.pem; > ssl_verify_depth 1; Best Regards, Kaushal -------------- next part -------------- An HTML attachment was scrubbed... URL: From jeff.dyke at gmail.com Sat Aug 19 01:39:13 2023 From: jeff.dyke at gmail.com (Jeff Dyke) Date: Fri, 18 Aug 2023 21:39:13 -0400 Subject: How to run a shell script on every request? In-Reply-To: References: Message-ID: Can you explain why? I would never tie a script to a request. I post process logs all of the time. If it needs to be in the application, don't force it into Nginx. Strong statement, but would love to hear why? On Fri, Aug 18, 2023 at 9:47 AM Kaushal Shriyan wrote: > Hi, > > I am running nginx version: nginx/1.24.0 on CentOS Linux release 7.9.2009 > (Core) > > # nginx -v > nginx version: nginx/1.24.0 > # cat /etc/redhat-release > CentOS Linux release 7.9.2009 (Core) > # > > I want to run a shell script every time my nginx server receives any HTTP > request. Any simple ways to do this? > > Please guide me. Thanks in Advance. > > Best Regards, > > Kaushal > > _______________________________________________ > nginx mailing list > nginx at nginx.org > https://mailman.nginx.org/mailman/listinfo/nginx > -------------- next part -------------- An HTML attachment was scrubbed... URL: From me at gend.moe Mon Aug 21 02:36:19 2023 From: me at gend.moe (Gentry Deng) Date: Mon, 21 Aug 2023 10:36:19 +0800 Subject: About X25519Kyber768Draft00 Message-ID: <2412d0ad-a6c6-44cc-b6cc-042ac6f8953d@gend.moe> Hello there, Chrome will begin supporting X25519Kyber768 for establishing symmetric secrets in TLS, starting in Chrome 116, and available behind a flag in Chrome 115. Cloudflare , the world's leading CDN provider, has been supporting Hybrid Kyber KEM since last year. I would like to know if nginx with BoringSSL can support X25519Kyber768Draft00? Best regards, Gentry -------------- next part -------------- An HTML attachment was scrubbed... URL: From pluknet at nginx.com Mon Aug 21 15:03:13 2023 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 21 Aug 2023 19:03:13 +0400 Subject: About X25519Kyber768Draft00 In-Reply-To: <2412d0ad-a6c6-44cc-b6cc-042ac6f8953d@gend.moe> References: <2412d0ad-a6c6-44cc-b6cc-042ac6f8953d@gend.moe> Message-ID: <5E2710DE-647E-4AC7-9FC5-C792314F3CA3@nginx.com> > On 21 Aug 2023, at 06:36, Gentry Deng via nginx wrote: > > Hello there, > > > > Chrome will begin supporting X25519Kyber768 for establishing symmetric secrets in TLS, starting in Chrome 116, and available behind a flag in Chrome 115. Cloudflare, the world's leading CDN provider, has been supporting Hybrid Kyber KEM since last year. > > I would like to know if nginx with BoringSSL can support X25519Kyber768Draft00? > It does, you can test it with BoringSSL itself. ssl_ecdh_curve prime256v1:X25519Kyber768Draft00; server { listen 8443 ssl; server_name localhost; return 200 "$ssl_curve\n"; } $ printf "GET / HTTP/1.0\n\n" | ./install/bin/bssl client -connect 127.1:8443 -curves X25519Kyber768Draft00 Connecting to 127.0.0.1:8443 Connected. Version: TLSv1.3 Resumed session: no Cipher: TLS_AES_128_GCM_SHA256 ECDHE group: X25519Kyber768Draft00 Signature algorithm: rsa_pss_rsae_sha256 Secure renegotiation: yes Extended master secret: yes Next protocol negotiated: ALPN protocol: OCSP staple: no SCT list: no Early data: no Encrypted ClientHello: no Cert subject: CN = localhost Cert issuer: CN = localhost HTTP/1.1 200 OK Server: nginx/1.25.2 Date: Mon, 21 Aug 2023 14:58:40 GMT Content-Type: text/plain Content-Length: 23 Connection: close X25519Kyber768Draft00 -- Sergey Kandaurov From kapouer at melix.org Sat Aug 26 14:21:07 2023 From: kapouer at melix.org (=?UTF-8?B?SsOpcsOpbXkgTGFs?=) Date: Sat, 26 Aug 2023 16:21:07 +0200 Subject: status-line trailing SP is missing ? Message-ID: Hi, https://bugs.debian.org/1050571 reports that the Status line doesn't always end with space, which seems contradictory to RFC9112 which states: "A server MUST send the space that separates the status-code from the reason-phrase even when the reason-phrase is absent (i.e., the status-line would end with the space)." Is it a documented nginx 1.24.0 behavior ? Thanks, Jérémy -------------- next part -------------- An HTML attachment was scrubbed... URL: From jordanc.carter at outlook.com Sun Aug 27 07:01:30 2023 From: jordanc.carter at outlook.com (J Carter) Date: Sun, 27 Aug 2023 08:01:30 +0100 Subject: How to run a shell script on every request? In-Reply-To: References: Message-ID: +1 on "why are you doing this?". However, to answer the question - rather than spawning a new shell for every request, use a loop in your bash script that is driven by access log output. For example. tail -n0 -f /var/log/nginx/access.log | \ while read; do echo "one request"; done; You'll need to handle what happens when log rotation takes place. On Fri, 18 Aug 2023 21:39:13 -0400 Jeff Dyke wrote: > Can you explain why? I would never tie a script to a request. I post > process logs all of the time. If it needs to be in the application, don't > force it into Nginx. > > Strong statement, but would love to hear why? > > On Fri, Aug 18, 2023 at 9:47 AM Kaushal Shriyan > wrote: > > > Hi, > > > > I am running nginx version: nginx/1.24.0 on CentOS Linux release 7.9.2009 > > (Core) > > > > # nginx -v > > nginx version: nginx/1.24.0 > > # cat /etc/redhat-release > > CentOS Linux release 7.9.2009 (Core) > > # > > > > I want to run a shell script every time my nginx server receives any HTTP > > request. Any simple ways to do this? > > > > Please guide me. Thanks in Advance. > > > > Best Regards, > > > > Kaushal > > > > _______________________________________________ > > nginx mailing list > > nginx at nginx.org > > https://mailman.nginx.org/mailman/listinfo/nginx > > From pluknet at nginx.com Mon Aug 28 16:59:28 2023 From: pluknet at nginx.com (Sergey Kandaurov) Date: Mon, 28 Aug 2023 20:59:28 +0400 Subject: status-line trailing SP is missing ? In-Reply-To: References: Message-ID: <51548C1A-DB6D-4BD0-8F12-6415D122A84F@nginx.com> > On 26 Aug 2023, at 18:21, Jérémy Lal wrote: > > Hi, > > https://bugs.debian.org/1050571 > reports that the Status line doesn't always end with space, which seems contradictory to RFC9112 which states: > "A server MUST send the space that separates the status-code from the reason-phrase even when the reason-phrase is absent (i.e., the status-line would end with the space)." > > Is it a documented nginx 1.24.0 behavior ? > Note that the response line with empty reason phrase is properly generated since nginx 1.5.6. The exception remains is proxying FastCGI responses as there is no distinguished response line in CGI syntax. The reason is that Status is a CGI header field, and hence it is parsed by a generic routine that cuts trailing spaces. But it can have a trailing space per RFC 3875, section 6.3.3. So it needs a special treatment to preserve SP before empty reason phrase. The below patch should help; although it doesn't look efficient and can be polished, I think this is quite enough for valid use cases. # HG changeset patch # User Sergey Kandaurov # Date 1693238094 -14400 # Mon Aug 28 19:54:54 2023 +0400 # Node ID f621c60dfa277774ab5fadb3c8b805957ca3f281 # Parent 2880f60a80c3e2706151dc7b48dc1267e39c47a9 FastCGI: preserved SP for empty Status header reason phrase. diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c --- a/src/http/modules/ngx_http_fastcgi_module.c +++ b/src/http/modules/ngx_http_fastcgi_module.c @@ -2048,7 +2048,25 @@ ngx_http_fastcgi_process_header(ngx_http } u->headers_in.status_n = status; - u->headers_in.status_line = *status_line; + + if (status_line->len == 3) { + /* preserve SP for empty reason phrase */ + + u->headers_in.status_line.data = ngx_pnalloc(r->pool, + 5); + if (u->headers_in.status_line.data == NULL) { + return NGX_ERROR; + } + + ngx_memcpy(u->headers_in.status_line.data, + status_line->data, 3); + u->headers_in.status_line.data[3] = ' '; + u->headers_in.status_line.data[4] = '\0'; + u->headers_in.status_line.len = 4; + + } else { + u->headers_in.status_line = *status_line; + } } else if (u->headers_in.location) { u->headers_in.status_n = 302; Alternatively, the reason-phrase value from FastCGI response can be ignored, nginx will translate it with its own value: # HG changeset patch # User Sergey Kandaurov # Date 1693241054 -14400 # Mon Aug 28 20:44:14 2023 +0400 # Node ID f1a452cbd57ff4fba1caf3f36cb3624bd45411ea # Parent 2880f60a80c3e2706151dc7b48dc1267e39c47a9 FastCGI: preserved SP for empty Status header reason phrase. As per RFC 3875 sec 6.3.3, the status code is always followed by SP. The header is parsed by the common routine, and it can be challenging to properly translate empty reason phrase to HTTP/1.1 status line, as that would require an additional memory allocation. For simplicity, the reason phrase is now ignored; the header filter will take care of it by inserting its own reason phrase for well known status codes. diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c --- a/src/http/modules/ngx_http_fastcgi_module.c +++ b/src/http/modules/ngx_http_fastcgi_module.c @@ -2048,7 +2048,6 @@ ngx_http_fastcgi_process_header(ngx_http } u->headers_in.status_n = status; - u->headers_in.status_line = *status_line; } else if (u->headers_in.location) { u->headers_in.status_n = 302; -- Sergey Kandaurov From randa.dan at gmail.com Tue Aug 29 02:07:13 2023 From: randa.dan at gmail.com (Dan Randa) Date: Mon, 28 Aug 2023 21:07:13 -0500 Subject: Nginx Amplify permissions errors with letsencrypt certificates and access log Message-ID: <8ECD37AF-9CFA-4841-BB95-CF6EC5D74EEC@gmail.com> I am running Ubuntu 22.04 server. I have installed Nginx Amplify. In my /var/log/amplify-agent/agent.log I am getting permissions errors with letsencrypt certificates. cat /var/log/amplify-agent/agent.log 2023-08-28 20:34:50,140 [8851] supervisor ssl certificate found /etc/letsencrypt/live/example.com/fullchain.pem 2023-08-28 20:34:50,140 [8851] supervisor could not read /etc/letsencrypt/live/example.com/fullchain.pem (maybe permissions?) 2023-08-28 20:34:50,943 [8851] supervisor failed to initialize pipeline for "/var/log/nginx/access.log" due to PermissionError (maybe has no rights?) 2023-08-28 20:34:50,953 [8851] supervisor failed to initialize pipeline for "/var/log/nginx/mail.example.com.access.log" due to PermissionError (maybe has no rights?) 2023-08-28 20:34:50,954 [8851] supervisor failed to initialize pipeline for "/var/log/nginx/mail.example.com.error.log" due to PermissionError (maybe has no rights?) How can I fix all of these permissions errors so nginx can view my certificates from letsencrypt? Also, how can I fix the pipeline errors? Thanks in advance for any help. Dan From mdounin at mdounin.ru Tue Aug 29 04:14:58 2023 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 29 Aug 2023 07:14:58 +0300 Subject: status-line trailing SP is missing ? In-Reply-To: References: Message-ID: Hello! On Sat, Aug 26, 2023 at 04:21:07PM +0200, Jérémy Lal wrote: > Hi, > > https://bugs.debian.org/1050571 > reports that the Status line doesn't always end with space, which seems > contradictory to RFC9112 which states: > "A server MUST send the space that separates the status-code from the > reason-phrase even when the reason-phrase is absent (i.e., the status-line > would end with the space)." > > Is it a documented nginx 1.24.0 behavior ? Interesting. As you can see from the report referenced, nginx returns in the HTTP status what is sent by the FastCGI application in the "Status" response header. The tricky part is an attempt to use "Status" header to sent status with a trailing space, "404 ". HTTP header values cannot contain trailing spaces, see here: https://www.rfc-editor.org/rfc/rfc9112.html#name-field-syntax https://www.rfc-editor.org/rfc/rfc9112.html#section-5.1-3 : A field line value might be preceded and/or followed by optional : whitespace (OWS); a single SP preceding the field line value is : preferred for consistent readability by humans. The field line : value does not include that leading or trailing whitespace: OWS : occurring before the first non-whitespace octet of the field line : value, or after the last non-whitespace octet of the field line : value, is excluded by parsers when extracting the field line value : from a field line. As such, both "Status: 404" and "Status: 404 " are equivalent and both contain value "404", without any trailing spaces. And this is what nginx uses in the response, and this is what is seen in the original report. Following the original CGI specification, which uses HTTP headers syntax in the response[1], this basically means that an application is not allowed to sent Status with an empty reason phrase. [1] https://web.archive.org/web/20100212084036/http://hoohoo.ncsa.illinois.edu/cgi/out.html : The output of scripts begins with a small header. This header : consists of text lines, in the same format as an HTTP header, : terminated by a blank line (a line with only a linefeed or CR/LF). Similarly, original CGI specification RFC draft, draft-robinson-www-interface-00 explicitly defines a generic syntax for CGI response headers, which does not use implied LWS rule (as in HTTP), but explicitly allows whitespaces between tokens in the headers: https://datatracker.ietf.org/doc/html/draft-robinson-www-interface-00#section-9.2 The CGI headers have the generic syntax: generic-header = field-name ":" [ field-value ] NL field-name = 1* field-value = *( field-content | LWSP ) field-content = *( token | tspecial | quoted-string) While it does not explicitly state, like HTTP specification, that leading and trailing LWSPs are not part of the field value, this is the only approach which makes the resulting specification usable. Similarly, RFC 3875 uses the same generic syntax for headers, and also tries to clarify whitespace usage: https://datatracker.ietf.org/doc/html/rfc3875#section-6.3 ... Whitespace is permitted between the ":" and the field-value (but not between the field-name and the ":"), and also between tokens in the field-value. Though in contrast to draft-robinson-www-interface-00, it only defines generic syntax for non-CGI headers. CGI headers, that is, "Content-Type", "Location", and "Status", use their own syntax defined separately. In particular, the Status header is defined as follows: https://datatracker.ietf.org/doc/html/rfc3875#section-6.3.3 Status = "Status:" status-code SP reason-phrase NL status-code = "200" | "302" | "400" | "501" | extension-code extension-code = 3digit reason-phrase = *TEXT Note that whitespaces are _not_ allowed by the syntax. This, however, rules out responses like "Status: 200 OK" due to the space between "Status:" and "200 OK". As such, it looks reasonable to assume this is a specification bug, and at least some whitespaces are expected to be allowed. Overall, it does not look like RFC 3875 provides a consistent approach to whitespace handling, and using the same approach as for HTTP seem to be the best available option. Summing the above, I tend to think that it is generally a bad idea to use Status header without a reason-phrase, as it is going to result in missing SP sooner or later. At least if you do care about missing SP in the status line (AFAIK, it causes no practical issues, though I'm not really tested). As for the nginx behaviour, I don't think we want to try to implement custom parsing for the Status header to preserve trailing SP if it's present. We can, however, consider using only the status code from such Status headers, so nginx will provide reason phrase by itself. Something like this should do the trick: # HG changeset patch # User Maxim Dounin # Date 1693282407 -10800 # Tue Aug 29 07:13:27 2023 +0300 # Node ID 10aec7047ed8c8e429e8e9b9d676a83751899bc6 # Parent 44536076405cf79ebdd82a6a0ab27bb3aed86b04 Upstream: fixed handling of Status headers without reason-phrase. Status header with an empty reason-phrase, such as "Status: 404 ", is valid per CGI specification, but looses the trailing space during parsing. Currently, this results in "HTTP/1.1 404" HTTP status line in the response, which violates HTTP specification due to missing trailing space. With this change, only the status code is used from such short Status header lines, so nginx will generate status line itself, with the space and appropriate reason phrase if available. Reported at: https://mailman.nginx.org/pipermail/nginx/2023-August/EX7G4JUUHJWJE5UOAZMO5UD6OJILCYGX.html diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c --- a/src/http/modules/ngx_http_fastcgi_module.c +++ b/src/http/modules/ngx_http_fastcgi_module.c @@ -2048,7 +2048,10 @@ ngx_http_fastcgi_process_header(ngx_http } u->headers_in.status_n = status; - u->headers_in.status_line = *status_line; + + if (status_line->len > 3) { + u->headers_in.status_line = *status_line; + } } else if (u->headers_in.location) { u->headers_in.status_n = 302; diff --git a/src/http/modules/ngx_http_scgi_module.c b/src/http/modules/ngx_http_scgi_module.c --- a/src/http/modules/ngx_http_scgi_module.c +++ b/src/http/modules/ngx_http_scgi_module.c @@ -1153,7 +1153,10 @@ ngx_http_scgi_process_header(ngx_http_re } u->headers_in.status_n = status; - u->headers_in.status_line = *status_line; + + if (status_line->len > 3) { + u->headers_in.status_line = *status_line; + } } else if (u->headers_in.location) { u->headers_in.status_n = 302; diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c --- a/src/http/modules/ngx_http_uwsgi_module.c +++ b/src/http/modules/ngx_http_uwsgi_module.c @@ -1381,7 +1381,10 @@ ngx_http_uwsgi_process_header(ngx_http_r } u->headers_in.status_n = status; - u->headers_in.status_line = *status_line; + + if (status_line->len > 3) { + u->headers_in.status_line = *status_line; + } } else if (u->headers_in.location) { u->headers_in.status_n = 302; -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Tue Aug 29 04:33:23 2023 From: mdounin at mdounin.ru (Maxim Dounin) Date: Tue, 29 Aug 2023 07:33:23 +0300 Subject: status-line trailing SP is missing ? In-Reply-To: <51548C1A-DB6D-4BD0-8F12-6415D122A84F@nginx.com> References: <51548C1A-DB6D-4BD0-8F12-6415D122A84F@nginx.com> Message-ID: Hello! On Mon, Aug 28, 2023 at 08:59:28PM +0400, Sergey Kandaurov wrote: > > > On 26 Aug 2023, at 18:21, Jérémy Lal wrote: > > > > Hi, > > > > https://bugs.debian.org/1050571 > > reports that the Status line doesn't always end with space, which seems contradictory to RFC9112 which states: > > "A server MUST send the space that separates the status-code from the reason-phrase even when the reason-phrase is absent (i.e., the status-line would end with the space)." > > > > Is it a documented nginx 1.24.0 behavior ? > > > > Note that the response line with empty reason phrase > is properly generated since nginx 1.5.6. > The exception remains is proxying FastCGI responses > as there is no distinguished response line in CGI syntax. > > The reason is that Status is a CGI header field, and hence > it is parsed by a generic routine that cuts trailing spaces. > But it can have a trailing space per RFC 3875, section 6.3.3. > So it needs a special treatment to preserve SP before empty reason > phrase. The below patch should help; although it doesn't look > efficient and can be polished, I think this is quite enough for > valid use cases. I very much doubt that RFC 3875 properly defines whitespace handling, see my response to the original report. In this particular case, it seems to define a header which cannot be correctly parsed if reason-phrase is empty. > > # HG changeset patch > # User Sergey Kandaurov > # Date 1693238094 -14400 > # Mon Aug 28 19:54:54 2023 +0400 > # Node ID f621c60dfa277774ab5fadb3c8b805957ca3f281 > # Parent 2880f60a80c3e2706151dc7b48dc1267e39c47a9 > FastCGI: preserved SP for empty Status header reason phrase. > > diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c > --- a/src/http/modules/ngx_http_fastcgi_module.c > +++ b/src/http/modules/ngx_http_fastcgi_module.c > @@ -2048,7 +2048,25 @@ ngx_http_fastcgi_process_header(ngx_http > } > > u->headers_in.status_n = status; > - u->headers_in.status_line = *status_line; > + > + if (status_line->len == 3) { > + /* preserve SP for empty reason phrase */ > + > + u->headers_in.status_line.data = ngx_pnalloc(r->pool, > + 5); > + if (u->headers_in.status_line.data == NULL) { > + return NGX_ERROR; > + } > + > + ngx_memcpy(u->headers_in.status_line.data, > + status_line->data, 3); > + u->headers_in.status_line.data[3] = ' '; > + u->headers_in.status_line.data[4] = '\0'; > + u->headers_in.status_line.len = 4; > + > + } else { > + u->headers_in.status_line = *status_line; > + } > > } else if (u->headers_in.location) { > u->headers_in.status_n = 302; Do you think we really need this complexity here? I tend to think that if (status_line->len > 3) { u->headers_in.status_line = *status_line; } would be enough, so nginx will generate status line by itself for such headers. The only downside I can see is that it will provide no way to actually generate a response with well-known status code, yet with an empty reason phrase. (Also note that uwsgi and scgi modules needs to be patched similarly.) > Alternatively, the reason-phrase value from FastCGI response > can be ignored, nginx will translate it with its own value: > > # HG changeset patch > # User Sergey Kandaurov > # Date 1693241054 -14400 > # Mon Aug 28 20:44:14 2023 +0400 > # Node ID f1a452cbd57ff4fba1caf3f36cb3624bd45411ea > # Parent 2880f60a80c3e2706151dc7b48dc1267e39c47a9 > FastCGI: preserved SP for empty Status header reason phrase. > > As per RFC 3875 sec 6.3.3, the status code is always followed by SP. > The header is parsed by the common routine, and it can be challenging > to properly translate empty reason phrase to HTTP/1.1 status line, as > that would require an additional memory allocation. For simplicity, > the reason phrase is now ignored; the header filter will take care of > it by inserting its own reason phrase for well known status codes. > > diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c > --- a/src/http/modules/ngx_http_fastcgi_module.c > +++ b/src/http/modules/ngx_http_fastcgi_module.c > @@ -2048,7 +2048,6 @@ ngx_http_fastcgi_process_header(ngx_http > } > > u->headers_in.status_n = status; > - u->headers_in.status_line = *status_line; > > } else if (u->headers_in.location) { > u->headers_in.status_n = 302; I don't think it's a good idea, since this always drops the reason phrase provided by the upstream server. It can contain some meaningful information which will be lost as a result, most notably for non-standard error codes. -- Maxim Dounin http://mdounin.ru/ From pluknet at nginx.com Wed Aug 30 12:20:15 2023 From: pluknet at nginx.com (Sergey Kandaurov) Date: Wed, 30 Aug 2023 16:20:15 +0400 Subject: status-line trailing SP is missing ? In-Reply-To: References: <51548C1A-DB6D-4BD0-8F12-6415D122A84F@nginx.com> Message-ID: > On 29 Aug 2023, at 08:33, Maxim Dounin wrote: > > Hello! > > On Mon, Aug 28, 2023 at 08:59:28PM +0400, Sergey Kandaurov wrote: > >> >>> On 26 Aug 2023, at 18:21, Jérémy Lal wrote: >>> >>> Hi, >>> >>> https://bugs.debian.org/1050571 >>> reports that the Status line doesn't always end with space, which seems contradictory to RFC9112 which states: >>> "A server MUST send the space that separates the status-code from the reason-phrase even when the reason-phrase is absent (i.e., the status-line would end with the space)." >>> >>> Is it a documented nginx 1.24.0 behavior ? >>> >> >> Note that the response line with empty reason phrase >> is properly generated since nginx 1.5.6. >> The exception remains is proxying FastCGI responses >> as there is no distinguished response line in CGI syntax. >> >> The reason is that Status is a CGI header field, and hence >> it is parsed by a generic routine that cuts trailing spaces. >> But it can have a trailing space per RFC 3875, section 6.3.3. >> So it needs a special treatment to preserve SP before empty reason >> phrase. The below patch should help; although it doesn't look >> efficient and can be polished, I think this is quite enough for >> valid use cases. > > I very much doubt that RFC 3875 properly defines whitespace > handling, see my response to the original report. In this > particular case, it seems to define a header which cannot be > correctly parsed if reason-phrase is empty. > Yes, it is quite dubious how this can be parsed correctly. Although it is valid to have a trailing space in Status, this contradicts to header field value syntax per RFC 3875: field-content = *( token | separator | quoted-string ) >> >> # HG changeset patch >> # User Sergey Kandaurov >> # Date 1693238094 -14400 >> # Mon Aug 28 19:54:54 2023 +0400 >> # Node ID f621c60dfa277774ab5fadb3c8b805957ca3f281 >> # Parent 2880f60a80c3e2706151dc7b48dc1267e39c47a9 >> FastCGI: preserved SP for empty Status header reason phrase. >> >> diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c >> --- a/src/http/modules/ngx_http_fastcgi_module.c >> +++ b/src/http/modules/ngx_http_fastcgi_module.c >> @@ -2048,7 +2048,25 @@ ngx_http_fastcgi_process_header(ngx_http >> } >> >> u->headers_in.status_n = status; >> - u->headers_in.status_line = *status_line; >> + >> + if (status_line->len == 3) { >> + /* preserve SP for empty reason phrase */ >> + >> + u->headers_in.status_line.data = ngx_pnalloc(r->pool, >> + 5); >> + if (u->headers_in.status_line.data == NULL) { >> + return NGX_ERROR; >> + } >> + >> + ngx_memcpy(u->headers_in.status_line.data, >> + status_line->data, 3); >> + u->headers_in.status_line.data[3] = ' '; >> + u->headers_in.status_line.data[4] = '\0'; >> + u->headers_in.status_line.len = 4; >> + >> + } else { >> + u->headers_in.status_line = *status_line; >> + } >> >> } else if (u->headers_in.location) { >> u->headers_in.status_n = 302; > > Do you think we really need this complexity here? I tend to think > that See above comment about efficiency. I doubt empty reason phrase is seen often in the wild, so this is a reasonable cost for kinda abusing protocol. But this complexity also reflects the ugliness, that's why I preferred the second version, also for its brevity. > > if (status_line->len > 3) { > u->headers_in.status_line = *status_line; > } > > would be enough, so nginx will generate status line by itself for > such headers. This is a nice trade-off between handling empty reason phrase with extra allocation and dropping it altogether, I like it. > > The only downside I can see is that it will provide no way to > actually generate a response with well-known status code, yet with > an empty reason phrase. > > (Also note that uwsgi and scgi modules needs to be patched > similarly.) These modules use a distinct routine to parse status line, that is ngx_http_parse_status_line, which respects spaces, and they expect the first line to look like a status line (similar to the proxy module). So I don't see immediate necessity to patch them as well. > >> Alternatively, the reason-phrase value from FastCGI response >> can be ignored, nginx will translate it with its own value: >> >> # HG changeset patch >> # User Sergey Kandaurov >> # Date 1693241054 -14400 >> # Mon Aug 28 20:44:14 2023 +0400 >> # Node ID f1a452cbd57ff4fba1caf3f36cb3624bd45411ea >> # Parent 2880f60a80c3e2706151dc7b48dc1267e39c47a9 >> FastCGI: preserved SP for empty Status header reason phrase. >> >> As per RFC 3875 sec 6.3.3, the status code is always followed by SP. >> The header is parsed by the common routine, and it can be challenging >> to properly translate empty reason phrase to HTTP/1.1 status line, as >> that would require an additional memory allocation. For simplicity, >> the reason phrase is now ignored; the header filter will take care of >> it by inserting its own reason phrase for well known status codes. >> >> diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c >> --- a/src/http/modules/ngx_http_fastcgi_module.c >> +++ b/src/http/modules/ngx_http_fastcgi_module.c >> @@ -2048,7 +2048,6 @@ ngx_http_fastcgi_process_header(ngx_http >> } >> >> u->headers_in.status_n = status; >> - u->headers_in.status_line = *status_line; >> >> } else if (u->headers_in.location) { >> u->headers_in.status_n = 302; > > I don't think it's a good idea, since this always drops the > reason phrase provided by the upstream server. It can contain > some meaningful information which will be lost as a result, most > notably for non-standard error codes. It is little to no benefit to carry reason phrase, also this is consistent with HTTP/1.1: : A client SHOULD ignore the reason-phrase content because : it is not a reliable channel for information Note also that HTTP/2 and then HTTP/3 do not define a way to carry the reason phrase that is included in HTTP/1.1. Personally I'd proceed with your approach for fastcgi module only, though I won't insist to patch other modules as well. Although this would introduce an extra condition in these other modules without a reason (as per my above analysis), this would keep sources in sync between them, which is fine, and will allow to stop using ngx_http_parse_status_line(). For the latter, uwsgi seems to use HTTP syntax in response, so it is ok to call ngx_http_parse_status_line() here, but scgi seems to follows CGI syntax where status line is carried in the Status header field, so the call would always return NGX_ERROR. Luckily this is already handled by falling back to ngx_http_scgi_process_header(). -- Sergey Kandaurov From mdounin at mdounin.ru Thu Aug 31 10:28:39 2023 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 31 Aug 2023 13:28:39 +0300 Subject: status-line trailing SP is missing ? In-Reply-To: References: <51548C1A-DB6D-4BD0-8F12-6415D122A84F@nginx.com> Message-ID: Hello! On Wed, Aug 30, 2023 at 04:20:15PM +0400, Sergey Kandaurov wrote: > > On 29 Aug 2023, at 08:33, Maxim Dounin wrote: > > > > On Mon, Aug 28, 2023 at 08:59:28PM +0400, Sergey Kandaurov wrote: > > > >>> On 26 Aug 2023, at 18:21, Jérémy Lal wrote: > >>> > >>> Hi, > >>> > >>> https://bugs.debian.org/1050571 > >>> reports that the Status line doesn't always end with space, which seems contradictory to RFC9112 which states: > >>> "A server MUST send the space that separates the status-code from the reason-phrase even when the reason-phrase is absent (i.e., the status-line would end with the space)." > >>> > >>> Is it a documented nginx 1.24.0 behavior ? > >>> > >> > >> Note that the response line with empty reason phrase > >> is properly generated since nginx 1.5.6. > >> The exception remains is proxying FastCGI responses > >> as there is no distinguished response line in CGI syntax. > >> > >> The reason is that Status is a CGI header field, and hence > >> it is parsed by a generic routine that cuts trailing spaces. > >> But it can have a trailing space per RFC 3875, section 6.3.3. > >> So it needs a special treatment to preserve SP before empty reason > >> phrase. The below patch should help; although it doesn't look > >> efficient and can be polished, I think this is quite enough for > >> valid use cases. > > > > I very much doubt that RFC 3875 properly defines whitespace > > handling, see my response to the original report. In this > > particular case, it seems to define a header which cannot be > > correctly parsed if reason-phrase is empty. > > > > Yes, it is quite dubious how this can be parsed correctly. > Although it is valid to have a trailing space in Status, > this contradicts to header field value syntax per RFC 3875: > field-content = *( token | separator | quoted-string ) Note that field-value syntax does no apply to the "Status" header, its syntax is defined separately. On the other hand, the "Status" header syntax does not allow any spaces after the colon, which rules out headers like "Status: 200 OK", and makes the "Status" header syntax highly questionable. As already suggested in my response to the original report, I tend to think that the best available option is to ignore RFC 3875 idea about headers syntax, and use HTTP instead. > > >> > >> # HG changeset patch > >> # User Sergey Kandaurov > >> # Date 1693238094 -14400 > >> # Mon Aug 28 19:54:54 2023 +0400 > >> # Node ID f621c60dfa277774ab5fadb3c8b805957ca3f281 > >> # Parent 2880f60a80c3e2706151dc7b48dc1267e39c47a9 > >> FastCGI: preserved SP for empty Status header reason phrase. > >> > >> diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c > >> --- a/src/http/modules/ngx_http_fastcgi_module.c > >> +++ b/src/http/modules/ngx_http_fastcgi_module.c > >> @@ -2048,7 +2048,25 @@ ngx_http_fastcgi_process_header(ngx_http > >> } > >> > >> u->headers_in.status_n = status; > >> - u->headers_in.status_line = *status_line; > >> + > >> + if (status_line->len == 3) { > >> + /* preserve SP for empty reason phrase */ > >> + > >> + u->headers_in.status_line.data = ngx_pnalloc(r->pool, > >> + 5); > >> + if (u->headers_in.status_line.data == NULL) { > >> + return NGX_ERROR; > >> + } > >> + > >> + ngx_memcpy(u->headers_in.status_line.data, > >> + status_line->data, 3); > >> + u->headers_in.status_line.data[3] = ' '; > >> + u->headers_in.status_line.data[4] = '\0'; > >> + u->headers_in.status_line.len = 4; > >> + > >> + } else { > >> + u->headers_in.status_line = *status_line; > >> + } > >> > >> } else if (u->headers_in.location) { > >> u->headers_in.status_n = 302; > > > > Do you think we really need this complexity here? I tend to think > > that > > See above comment about efficiency. > I doubt empty reason phrase is seen often in the wild, > so this is a reasonable cost for kinda abusing protocol. > But this complexity also reflects the ugliness, that's > why I preferred the second version, also for its brevity. > > > > > if (status_line->len > 3) { > > u->headers_in.status_line = *status_line; > > } > > > > would be enough, so nginx will generate status line by itself for > > such headers. > > This is a nice trade-off between handling empty reason phrase > with extra allocation and dropping it altogether, I like it. Please see my response to the original report for the full patch. > > > > The only downside I can see is that it will provide no way to > > actually generate a response with well-known status code, yet with > > an empty reason phrase. > > > > (Also note that uwsgi and scgi modules needs to be patched > > similarly.) > > These modules use a distinct routine to parse status line, > that is ngx_http_parse_status_line, which respects spaces, > and they expect the first line to look like a status line > (similar to the proxy module). > So I don't see immediate necessity to patch them as well. Sure, but these are distinct parsing modes: parsing status line as in raw HTTP responses (which, in particular, are used by NPH scripts), or parsing the Status header, as in CGI responses. Both should work correctly. In particular, current SCGI specification (https://python.ca/scgi/protocol.txt) uses "Status: ..." in the example (whereas original version used "HTTP/1.0 ...", https://web.archive.org/web/20020403050958/http://python.ca/nas/scgi/protocol.txt). I see no reasons why "Status: 404 " should work correctly in FastCGI, but shouldn't in SCGI. > >> Alternatively, the reason-phrase value from FastCGI response > >> can be ignored, nginx will translate it with its own value: > >> > >> # HG changeset patch > >> # User Sergey Kandaurov > >> # Date 1693241054 -14400 > >> # Mon Aug 28 20:44:14 2023 +0400 > >> # Node ID f1a452cbd57ff4fba1caf3f36cb3624bd45411ea > >> # Parent 2880f60a80c3e2706151dc7b48dc1267e39c47a9 > >> FastCGI: preserved SP for empty Status header reason phrase. > >> > >> As per RFC 3875 sec 6.3.3, the status code is always followed by SP. > >> The header is parsed by the common routine, and it can be challenging > >> to properly translate empty reason phrase to HTTP/1.1 status line, as > >> that would require an additional memory allocation. For simplicity, > >> the reason phrase is now ignored; the header filter will take care of > >> it by inserting its own reason phrase for well known status codes. > >> > >> diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c > >> --- a/src/http/modules/ngx_http_fastcgi_module.c > >> +++ b/src/http/modules/ngx_http_fastcgi_module.c > >> @@ -2048,7 +2048,6 @@ ngx_http_fastcgi_process_header(ngx_http > >> } > >> > >> u->headers_in.status_n = status; > >> - u->headers_in.status_line = *status_line; > >> > >> } else if (u->headers_in.location) { > >> u->headers_in.status_n = 302; > > > > I don't think it's a good idea, since this always drops the > > reason phrase provided by the upstream server. It can contain > > some meaningful information which will be lost as a result, most > > notably for non-standard error codes. > > It is little to no benefit to carry reason phrase, > also this is consistent with HTTP/1.1: > : A client SHOULD ignore the reason-phrase content because > : it is not a reliable channel for information > > Note also that HTTP/2 and then HTTP/3 do not define a way > to carry the reason phrase that is included in HTTP/1.1. In my practice, it is quite usable when debugging and analyzing the protocol flow. HTTP/2 decision to drop it looks highly questionable to me, and generally reflects the fact that HTTP/2 is not designed for debugging. > Personally I'd proceed with your approach for fastcgi module > only, though I won't insist to patch other modules as well. > Although this would introduce an extra condition in these > other modules without a reason (as per my above analysis), > this would keep sources in sync between them, which is fine, > and will allow to stop using ngx_http_parse_status_line(). > > For the latter, uwsgi seems to use HTTP syntax in response, > so it is ok to call ngx_http_parse_status_line() here, but > scgi seems to follows CGI syntax where status line is > carried in the Status header field, so the call would > always return NGX_ERROR. Luckily this is already handled > by falling back to ngx_http_scgi_process_header(). SCGI does not define a response syntax at all, and at least "Status: ..." and "HTTP/1.0 ..." were seen in the different versions of the specification, see above. As such, I tend to think that both variants needs to be (correctly) supported. (On the other hand, for uwsgi I'm not sure if "Status: ..." is allowed, it seems to contradict the protocol (see https://uwsgi-docs.readthedocs.io/en/latest/Protocol.html, "Every uwsgi request generates a response in the uwsgi format"). We might consider removing Status header parsing in uwsgi, but this is probably out of the scope of this issue.) -- Maxim Dounin http://mdounin.ru/ From pluknet at nginx.com Thu Aug 31 11:45:06 2023 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 31 Aug 2023 15:45:06 +0400 Subject: status-line trailing SP is missing ? In-Reply-To: References: <51548C1A-DB6D-4BD0-8F12-6415D122A84F@nginx.com> Message-ID: > On 31 Aug 2023, at 14:28, Maxim Dounin wrote: > > Hello! > > On Wed, Aug 30, 2023 at 04:20:15PM +0400, Sergey Kandaurov wrote: > >>> On 29 Aug 2023, at 08:33, Maxim Dounin wrote: >>> >>> On Mon, Aug 28, 2023 at 08:59:28PM +0400, Sergey Kandaurov wrote: >>> >>>>> On 26 Aug 2023, at 18:21, Jérémy Lal wrote: >>>>> >>>>> Hi, >>>>> >>>>> https://bugs.debian.org/1050571 >>>>> reports that the Status line doesn't always end with space, which seems contradictory to RFC9112 which states: >>>>> "A server MUST send the space that separates the status-code from the reason-phrase even when the reason-phrase is absent (i.e., the status-line would end with the space)." >>>>> >>>>> Is it a documented nginx 1.24.0 behavior ? >>>>> >>>> >>>> Note that the response line with empty reason phrase >>>> is properly generated since nginx 1.5.6. >>>> The exception remains is proxying FastCGI responses >>>> as there is no distinguished response line in CGI syntax. >>>> >>>> The reason is that Status is a CGI header field, and hence >>>> it is parsed by a generic routine that cuts trailing spaces. >>>> But it can have a trailing space per RFC 3875, section 6.3.3. >>>> So it needs a special treatment to preserve SP before empty reason >>>> phrase. The below patch should help; although it doesn't look >>>> efficient and can be polished, I think this is quite enough for >>>> valid use cases. >>> >>> I very much doubt that RFC 3875 properly defines whitespace >>> handling, see my response to the original report. In this >>> particular case, it seems to define a header which cannot be >>> correctly parsed if reason-phrase is empty. >>> >> >> Yes, it is quite dubious how this can be parsed correctly. >> Although it is valid to have a trailing space in Status, >> this contradicts to header field value syntax per RFC 3875: >> field-content = *( token | separator | quoted-string ) > > Note that field-value syntax does no apply to the "Status" header, > its syntax is defined separately. Well, per RFC 3875 BNF, Status is CGI-field, which if generalized to other-field, consists of field-content. > > On the other hand, the "Status" header syntax does not allow any > spaces after the colon, which rules out headers like "Status: 200 OK", > and makes the "Status" header syntax highly questionable. Again, if generalize the "Status" header to header-field, then there's a note about whitespace between the ":" and the field-value. > > As already suggested in my response to the original report, I tend > to think that the best available option is to ignore RFC 3875 idea > about headers syntax, and use HTTP instead. I don't mind. [..] >>> >>> (Also note that uwsgi and scgi modules needs to be patched >>> similarly.) >> >> These modules use a distinct routine to parse status line, >> that is ngx_http_parse_status_line, which respects spaces, >> and they expect the first line to look like a status line >> (similar to the proxy module). >> So I don't see immediate necessity to patch them as well. > > Sure, but these are distinct parsing modes: parsing status line as > in raw HTTP responses (which, in particular, are used by NPH scripts), > or parsing the Status header, as in CGI responses. Both should > work correctly. > > In particular, current SCGI specification > (https://python.ca/scgi/protocol.txt) uses "Status: ..." in the > example (whereas original version used "HTTP/1.0 ...", > https://web.archive.org/web/20020403050958/http://python.ca/nas/scgi/protocol.txt). > Ahh ok, didn't know that. That explains why all three modules should be patches. > I see no reasons why "Status: 404 " should work correctly in > FastCGI, but shouldn't in SCGI. > >>>> Alternatively, the reason-phrase value from FastCGI response >>>> can be ignored, nginx will translate it with its own value: >>>> >>>> # HG changeset patch >>>> # User Sergey Kandaurov >>>> # Date 1693241054 -14400 >>>> # Mon Aug 28 20:44:14 2023 +0400 >>>> # Node ID f1a452cbd57ff4fba1caf3f36cb3624bd45411ea >>>> # Parent 2880f60a80c3e2706151dc7b48dc1267e39c47a9 >>>> FastCGI: preserved SP for empty Status header reason phrase. >>>> >>>> As per RFC 3875 sec 6.3.3, the status code is always followed by SP. >>>> The header is parsed by the common routine, and it can be challenging >>>> to properly translate empty reason phrase to HTTP/1.1 status line, as >>>> that would require an additional memory allocation. For simplicity, >>>> the reason phrase is now ignored; the header filter will take care of >>>> it by inserting its own reason phrase for well known status codes. >>>> >>>> diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c >>>> --- a/src/http/modules/ngx_http_fastcgi_module.c >>>> +++ b/src/http/modules/ngx_http_fastcgi_module.c >>>> @@ -2048,7 +2048,6 @@ ngx_http_fastcgi_process_header(ngx_http >>>> } >>>> >>>> u->headers_in.status_n = status; >>>> - u->headers_in.status_line = *status_line; >>>> >>>> } else if (u->headers_in.location) { >>>> u->headers_in.status_n = 302; >>> >>> I don't think it's a good idea, since this always drops the >>> reason phrase provided by the upstream server. It can contain >>> some meaningful information which will be lost as a result, most >>> notably for non-standard error codes. >> >> It is little to no benefit to carry reason phrase, >> also this is consistent with HTTP/1.1: >> : A client SHOULD ignore the reason-phrase content because >> : it is not a reliable channel for information >> >> Note also that HTTP/2 and then HTTP/3 do not define a way >> to carry the reason phrase that is included in HTTP/1.1. > > In my practice, it is quite usable when debugging and analyzing > the protocol flow. HTTP/2 decision to drop it looks highly > questionable to me, and generally reflects the fact that HTTP/2 is > not designed for debugging. > >> Personally I'd proceed with your approach for fastcgi module >> only, though I won't insist to patch other modules as well. >> Although this would introduce an extra condition in these >> other modules without a reason (as per my above analysis), >> this would keep sources in sync between them, which is fine, >> and will allow to stop using ngx_http_parse_status_line(). >> >> For the latter, uwsgi seems to use HTTP syntax in response, >> so it is ok to call ngx_http_parse_status_line() here, but >> scgi seems to follows CGI syntax where status line is >> carried in the Status header field, so the call would >> always return NGX_ERROR. Luckily this is already handled >> by falling back to ngx_http_scgi_process_header(). > > SCGI does not define a response syntax at all, and at least > "Status: ..." and "HTTP/1.0 ..." were seen in the different > versions of the specification, see above. As such, I tend to > think that both variants needs to be (correctly) supported. > Sure, it makes sense then to keep both variants. > (On the other hand, for uwsgi I'm not sure if "Status: ..." is > allowed, it seems to contradict the protocol (see > https://uwsgi-docs.readthedocs.io/en/latest/Protocol.html, "Every > uwsgi request generates a response in the uwsgi format"). We > might consider removing Status header parsing in uwsgi, but this > is probably out of the scope of this issue.) -- Sergey Kandaurov From pluknet at nginx.com Thu Aug 31 11:45:18 2023 From: pluknet at nginx.com (Sergey Kandaurov) Date: Thu, 31 Aug 2023 15:45:18 +0400 Subject: status-line trailing SP is missing ? In-Reply-To: References: Message-ID: <554287A4-82A0-4268-A07F-329F536C06A3@nginx.com> > On 29 Aug 2023, at 08:14, Maxim Dounin wrote: > > Hello! > > On Sat, Aug 26, 2023 at 04:21:07PM +0200, Jérémy Lal wrote: > >> Hi, >> >> https://bugs.debian.org/1050571 >> reports that the Status line doesn't always end with space, which seems >> contradictory to RFC9112 which states: >> "A server MUST send the space that separates the status-code from the >> reason-phrase even when the reason-phrase is absent (i.e., the status-line >> would end with the space)." >> >> Is it a documented nginx 1.24.0 behavior ? > > Interesting. > > As you can see from the report referenced, nginx returns in the > HTTP status what is sent by the FastCGI application in the > "Status" response header. > > [..] > > Summing the above, I tend to think that it is generally a bad idea > to use Status header without a reason-phrase, as it is going to > result in missing SP sooner or later. At least if you do care > about missing SP in the status line (AFAIK, it causes no practical > issues, though I'm not really tested). Agree. > > As for the nginx behaviour, I don't think we want to try to > implement custom parsing for the Status header to preserve > trailing SP if it's present. We can, however, consider using > only the status code from such Status headers, so nginx will > provide reason phrase by itself. > > Something like this should do the trick: > > # HG changeset patch > # User Maxim Dounin > # Date 1693282407 -10800 > # Tue Aug 29 07:13:27 2023 +0300 > # Node ID 10aec7047ed8c8e429e8e9b9d676a83751899bc6 > # Parent 44536076405cf79ebdd82a6a0ab27bb3aed86b04 > Upstream: fixed handling of Status headers without reason-phrase. > > Status header with an empty reason-phrase, such as "Status: 404 ", is > valid per CGI specification, but looses the trailing space during parsing. > Currently, this results in "HTTP/1.1 404" HTTP status line in the response, > which violates HTTP specification due to missing trailing space. > > With this change, only the status code is used from such short Status > header lines, so nginx will generate status line itself, with the space > and appropriate reason phrase if available. > > Reported at: > https://mailman.nginx.org/pipermail/nginx/2023-August/EX7G4JUUHJWJE5UOAZMO5UD6OJILCYGX.html > > diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c > --- a/src/http/modules/ngx_http_fastcgi_module.c > +++ b/src/http/modules/ngx_http_fastcgi_module.c > @@ -2048,7 +2048,10 @@ ngx_http_fastcgi_process_header(ngx_http > } > > u->headers_in.status_n = status; > - u->headers_in.status_line = *status_line; > + > + if (status_line->len > 3) { > + u->headers_in.status_line = *status_line; > + } > > } else if (u->headers_in.location) { > u->headers_in.status_n = 302; > diff --git a/src/http/modules/ngx_http_scgi_module.c b/src/http/modules/ngx_http_scgi_module.c > --- a/src/http/modules/ngx_http_scgi_module.c > +++ b/src/http/modules/ngx_http_scgi_module.c > @@ -1153,7 +1153,10 @@ ngx_http_scgi_process_header(ngx_http_re > } > > u->headers_in.status_n = status; > - u->headers_in.status_line = *status_line; > + > + if (status_line->len > 3) { > + u->headers_in.status_line = *status_line; > + } > > } else if (u->headers_in.location) { > u->headers_in.status_n = 302; > diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c > --- a/src/http/modules/ngx_http_uwsgi_module.c > +++ b/src/http/modules/ngx_http_uwsgi_module.c > @@ -1381,7 +1381,10 @@ ngx_http_uwsgi_process_header(ngx_http_r > } > > u->headers_in.status_n = status; > - u->headers_in.status_line = *status_line; > + > + if (status_line->len > 3) { > + u->headers_in.status_line = *status_line; > + } > > } else if (u->headers_in.location) { > u->headers_in.status_n = 302; > > After discussion in the adjacent thread, I think the change is fine. -- Sergey Kandaurov From mdounin at mdounin.ru Thu Aug 31 19:24:51 2023 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 31 Aug 2023 22:24:51 +0300 Subject: status-line trailing SP is missing ? In-Reply-To: References: <51548C1A-DB6D-4BD0-8F12-6415D122A84F@nginx.com> Message-ID: Hello! On Thu, Aug 31, 2023 at 03:45:06PM +0400, Sergey Kandaurov wrote: > > On 31 Aug 2023, at 14:28, Maxim Dounin wrote: > > > > Hello! > > > > On Wed, Aug 30, 2023 at 04:20:15PM +0400, Sergey Kandaurov wrote: > > > >>> On 29 Aug 2023, at 08:33, Maxim Dounin wrote: > >>> > >>> On Mon, Aug 28, 2023 at 08:59:28PM +0400, Sergey Kandaurov wrote: > >>> > >>>>> On 26 Aug 2023, at 18:21, Jérémy Lal wrote: > >>>>> > >>>>> Hi, > >>>>> > >>>>> https://bugs.debian.org/1050571 > >>>>> reports that the Status line doesn't always end with space, which seems contradictory to RFC9112 which states: > >>>>> "A server MUST send the space that separates the status-code from the reason-phrase even when the reason-phrase is absent (i.e., the status-line would end with the space)." > >>>>> > >>>>> Is it a documented nginx 1.24.0 behavior ? > >>>>> > >>>> > >>>> Note that the response line with empty reason phrase > >>>> is properly generated since nginx 1.5.6. > >>>> The exception remains is proxying FastCGI responses > >>>> as there is no distinguished response line in CGI syntax. > >>>> > >>>> The reason is that Status is a CGI header field, and hence > >>>> it is parsed by a generic routine that cuts trailing spaces. > >>>> But it can have a trailing space per RFC 3875, section 6.3.3. > >>>> So it needs a special treatment to preserve SP before empty reason > >>>> phrase. The below patch should help; although it doesn't look > >>>> efficient and can be polished, I think this is quite enough for > >>>> valid use cases. > >>> > >>> I very much doubt that RFC 3875 properly defines whitespace > >>> handling, see my response to the original report. In this > >>> particular case, it seems to define a header which cannot be > >>> correctly parsed if reason-phrase is empty. > >>> > >> > >> Yes, it is quite dubious how this can be parsed correctly. > >> Although it is valid to have a trailing space in Status, > >> this contradicts to header field value syntax per RFC 3875: > >> field-content = *( token | separator | quoted-string ) > > > > Note that field-value syntax does no apply to the "Status" header, > > its syntax is defined separately. > > Well, per RFC 3875 BNF, Status is CGI-field, which if generalized > to other-field, consists of field-content. Syntax is as follows (https://datatracker.ietf.org/doc/html/rfc3875#section-6.3): header-field = CGI-field | other-field CGI-field = Content-Type | Location | Status other-field = protocol-field | extension-field protocol-field = generic-field extension-field = generic-field generic-field = field-name ":" [ field-value ] NL CGI-field and other-field are mutually exclusive variants of header-field, and there is no generalization. Generalization was present in the original RFC draft (https://datatracker.ietf.org/doc/html/draft-robinson-www-interface-00#section-9.2), but was removed during work on RFC 3875 (in https://datatracker.ietf.org/doc/html/draft-coar-cgi-v11-03). Not sure about the reasons, but might be because Status indeed does not conform to the generic-field syntax, and this was an attempt to fix it. Either way, the fact is: RFC 3875 does not correctly specify whitespace handling, and shouldn't be relied upon. [...] -- Maxim Dounin http://mdounin.ru/ From mdounin at mdounin.ru Thu Aug 31 20:13:21 2023 From: mdounin at mdounin.ru (Maxim Dounin) Date: Thu, 31 Aug 2023 23:13:21 +0300 Subject: status-line trailing SP is missing ? In-Reply-To: <554287A4-82A0-4268-A07F-329F536C06A3@nginx.com> References: <554287A4-82A0-4268-A07F-329F536C06A3@nginx.com> Message-ID: Hello! On Thu, Aug 31, 2023 at 03:45:18PM +0400, Sergey Kandaurov wrote: > > On 29 Aug 2023, at 08:14, Maxim Dounin wrote: > > > > Hello! > > > > On Sat, Aug 26, 2023 at 04:21:07PM +0200, Jérémy Lal wrote: > > > >> Hi, > >> > >> https://bugs.debian.org/1050571 > >> reports that the Status line doesn't always end with space, which seems > >> contradictory to RFC9112 which states: > >> "A server MUST send the space that separates the status-code from the > >> reason-phrase even when the reason-phrase is absent (i.e., the status-line > >> would end with the space)." > >> > >> Is it a documented nginx 1.24.0 behavior ? > > > > Interesting. > > > > As you can see from the report referenced, nginx returns in the > > HTTP status what is sent by the FastCGI application in the > > "Status" response header. > > > > [..] > > > > Summing the above, I tend to think that it is generally a bad idea > > to use Status header without a reason-phrase, as it is going to > > result in missing SP sooner or later. At least if you do care > > about missing SP in the status line (AFAIK, it causes no practical > > issues, though I'm not really tested). > > Agree. > > > > > As for the nginx behaviour, I don't think we want to try to > > implement custom parsing for the Status header to preserve > > trailing SP if it's present. We can, however, consider using > > only the status code from such Status headers, so nginx will > > provide reason phrase by itself. > > > > Something like this should do the trick: > > > > # HG changeset patch > > # User Maxim Dounin > > # Date 1693282407 -10800 > > # Tue Aug 29 07:13:27 2023 +0300 > > # Node ID 10aec7047ed8c8e429e8e9b9d676a83751899bc6 > > # Parent 44536076405cf79ebdd82a6a0ab27bb3aed86b04 > > Upstream: fixed handling of Status headers without reason-phrase. > > > > Status header with an empty reason-phrase, such as "Status: 404 ", is > > valid per CGI specification, but looses the trailing space during parsing. > > Currently, this results in "HTTP/1.1 404" HTTP status line in the response, > > which violates HTTP specification due to missing trailing space. > > > > With this change, only the status code is used from such short Status > > header lines, so nginx will generate status line itself, with the space > > and appropriate reason phrase if available. > > > > Reported at: > > https://mailman.nginx.org/pipermail/nginx/2023-August/EX7G4JUUHJWJE5UOAZMO5UD6OJILCYGX.html > > > > diff --git a/src/http/modules/ngx_http_fastcgi_module.c b/src/http/modules/ngx_http_fastcgi_module.c > > --- a/src/http/modules/ngx_http_fastcgi_module.c > > +++ b/src/http/modules/ngx_http_fastcgi_module.c > > @@ -2048,7 +2048,10 @@ ngx_http_fastcgi_process_header(ngx_http > > } > > > > u->headers_in.status_n = status; > > - u->headers_in.status_line = *status_line; > > + > > + if (status_line->len > 3) { > > + u->headers_in.status_line = *status_line; > > + } > > > > } else if (u->headers_in.location) { > > u->headers_in.status_n = 302; > > diff --git a/src/http/modules/ngx_http_scgi_module.c b/src/http/modules/ngx_http_scgi_module.c > > --- a/src/http/modules/ngx_http_scgi_module.c > > +++ b/src/http/modules/ngx_http_scgi_module.c > > @@ -1153,7 +1153,10 @@ ngx_http_scgi_process_header(ngx_http_re > > } > > > > u->headers_in.status_n = status; > > - u->headers_in.status_line = *status_line; > > + > > + if (status_line->len > 3) { > > + u->headers_in.status_line = *status_line; > > + } > > > > } else if (u->headers_in.location) { > > u->headers_in.status_n = 302; > > diff --git a/src/http/modules/ngx_http_uwsgi_module.c b/src/http/modules/ngx_http_uwsgi_module.c > > --- a/src/http/modules/ngx_http_uwsgi_module.c > > +++ b/src/http/modules/ngx_http_uwsgi_module.c > > @@ -1381,7 +1381,10 @@ ngx_http_uwsgi_process_header(ngx_http_r > > } > > > > u->headers_in.status_n = status; > > - u->headers_in.status_line = *status_line; > > + > > + if (status_line->len > 3) { > > + u->headers_in.status_line = *status_line; > > + } > > > > } else if (u->headers_in.location) { > > u->headers_in.status_n = 302; > > > > > > After discussion in the adjacent thread, > I think the change is fine. Pushed to http://mdounin.ru/hg/nginx, thanks for the review. -- Maxim Dounin http://mdounin.ru/