From ano at bestmx.net Thu Feb 3 10:40:42 2022 From: ano at bestmx.net (Andrey Oktyabrskiy) Date: Thu, 3 Feb 2022 13:40:42 +0300 Subject: assignment from incompatible pointer type Message-ID: Strange error during "go build": # unit.nginx.org/go nxt_cgo_lib.c: In function ‘nxt_cgo_run’: nxt_cgo_lib.c:29:36: warning: assignment to ‘void (*)(nxt_unit_t *, nxt_unit_ctx_t *, nxt_unit_port_t *)’ {aka ‘void (*)(struct nxt_unit_s *, struct nxt_unit_ctx_s *, struct nxt_unit_port_s *)’} from incompatible pointer type ‘void (*)(nxt_unit_t *, nxt_unit_port_t *)’ {aka ‘void (*)(struct nxt_unit_s *, struct nxt_unit_port_s *)’} [-Wincompatible-pointer-types] 29 | init.callbacks.remove_port = nxt_go_remove_port; | ^ nxt_cgo_lib.c:31:36: warning: assignment to ‘ssize_t (*)(nxt_unit_ctx_t *, nxt_unit_port_t *, void *, size_t, void *, size_t *)’ {aka ‘long int (*)(struct nxt_unit_ctx_s *, struct nxt_unit_port_s *, void *, long unsigned int, void *, long unsigned int *)’} from incompatible pointer type ‘ssize_t (*)(nxt_unit_ctx_t *, nxt_unit_port_t *, void *, size_t, void *, size_t)’ {aka ‘long int (*)(struct nxt_unit_ctx_s *, struct nxt_unit_port_s *, void *, long unsigned int, void *, long unsigned int)’} [-Wincompatible-pointer-types] 31 | init.callbacks.port_recv = nxt_cgo_port_recv; | ^ What's wrong and how to fix? From max.romanov at nginx.com Thu Feb 3 13:39:05 2022 From: max.romanov at nginx.com (Max Romanov) Date: Thu, 3 Feb 2022 16:39:05 +0300 Subject: assignment from incompatible pointer type In-Reply-To: References: Message-ID: <16851CC6-34CD-47D5-8070-ACFF31D5C64C@nginx.com> Hello. It seems go module sources is outdated. Please fetch the version tagged same version as Unit. — Max > On 3 Feb 2022, at 13:40, Andrey Oktyabrskiy wrote: > > Strange error during "go build": > > # unit.nginx.org/go > nxt_cgo_lib.c: In function ‘nxt_cgo_run’: > nxt_cgo_lib.c:29:36: warning: assignment to ‘void (*)(nxt_unit_t *, nxt_unit_ctx_t *, nxt_unit_port_t *)’ {aka ‘void (*)(struct nxt_unit_s *, struct nxt_unit_ctx_s *, struct nxt_unit_port_s *)’} from incompatible pointer type ‘void (*)(nxt_unit_t *, nxt_unit_port_t *)’ {aka ‘void (*)(struct nxt_unit_s *, struct nxt_unit_port_s *)’} [-Wincompatible-pointer-types] > 29 | init.callbacks.remove_port = nxt_go_remove_port; > | ^ > nxt_cgo_lib.c:31:36: warning: assignment to ‘ssize_t (*)(nxt_unit_ctx_t *, nxt_unit_port_t *, void *, size_t, void *, size_t *)’ {aka ‘long int (*)(struct nxt_unit_ctx_s *, struct nxt_unit_port_s *, void *, long unsigned int, void *, long unsigned int *)’} from incompatible pointer type ‘ssize_t (*)(nxt_unit_ctx_t *, nxt_unit_port_t *, void *, size_t, void *, size_t)’ {aka ‘long int (*)(struct nxt_unit_ctx_s *, struct nxt_unit_port_s *, void *, long unsigned int, void *, long unsigned int)’} [-Wincompatible-pointer-types] > 31 | init.callbacks.port_recv = nxt_cgo_port_recv; > | ^ > > > What's wrong and how to fix? > _______________________________________________ > unit mailing list -- unit at nginx.org > To unsubscribe send an email to unit-leave at nginx.org From alx.manpages at gmail.com Mon Feb 14 11:33:21 2022 From: alx.manpages at gmail.com (Alejandro Colomar (man-pages)) Date: Mon, 14 Feb 2022 12:33:21 +0100 Subject: [PATCH v4 07/12] Static: add "index" option In-Reply-To: <36737EAD-4644-409F-B6E2-022E48AFDD47@nginx.com> References: <20211219013030.51372-1-alx.manpages@gmail.com> <20211223192508.12146-8-alx.manpages@gmail.com> <36737EAD-4644-409F-B6E2-022E48AFDD47@nginx.com> Message-ID: <59461183-7804-bbc6-c245-ed0c3b235da0@gmail.com> Hi, Andrei! On 1/28/22 13:27, Andrei Zeliankou wrote: >> On 23 Dec 2021, at 19:25, Alejandro Colomar wrote: >> >> This supports a new option "index", that configures a custom index >> file name that will be served when a directory is requested. This >> initial support only allows a single fixed string. An example: >> >> { >> "share": "/www/data/static/$uri", >> "index": "lookatthis.htm" >> } >> [...] > Also noticed problems: > > 1) can't compile patch set with specified `--debug` option: > > src/nxt_http_static.c:264:23: error: no member named 'index' in 'nxt_http_static_conf_t' > nxt_var_raw(conf->index, &idx); > ~~~~ ^ Oops, fixed. > > 2) router crash when index is the empty array I'm still investigating this one. > > 3) router crash when variable in index value is empty The same behavior happens when you set up index to the name of a dir: $ cat config.json { "listeners": { "*:80": { "pass": "routes" } }, "routes": [ { "action": { "share": "/home/alx/srv/www/static/$uri", "index": "dir" } } ] } $ ls -l /home/alx/srv/www/static/ total 16 -rw-r--r-- 1 alx alx 4 Dec 14 17:51 GET.html drwxr-xr-x 2 alx alx 4096 Feb 14 12:18 dir -rw-r--r-- 1 alx alx 6 Dec 14 17:52 index.html -rw-r--r-- 1 alx alx 10 Dec 15 12:55 localhost.html $ curl --dump-header - http://localhost HTTP/1.1 301 Moved Permanently Location: // Server: Unit/1.27.0 Date: Mon, 14 Feb 2022 11:19:24 GMT Content-Length: 0 Setting "dir/" instead of "dir" behaves identically too. What is the expected behavior when trying to serve a directory? With old nginx (open-source), with autoindex off, and try_files ${uri}/, I'm getting 403 Forbidden. Where does Unit decide about that? Thanks, Alex -- Alejandro Colomar Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/ http://www.alejandro-colomar.es/ From vbart at nginx.com Mon Feb 14 12:12:51 2022 From: vbart at nginx.com (Valentin V. Bartenev) Date: Mon, 14 Feb 2022 15:12:51 +0300 Subject: [PATCH v4 07/12] Static: add "index" option In-Reply-To: <59461183-7804-bbc6-c245-ed0c3b235da0@gmail.com> References: <20211219013030.51372-1-alx.manpages@gmail.com> <36737EAD-4644-409F-B6E2-022E48AFDD47@nginx.com> <59461183-7804-bbc6-c245-ed0c3b235da0@gmail.com> Message-ID: <2813833.e9J7NaK4W3@vbart-laptop> On Monday, 14 February 2022 14:33:21 MSK Alejandro Colomar (man-pages) wrote: [..] > > 3) router crash when variable in index value is empty > > The same behavior happens when you set up index to the name of a dir: > > $ cat config.json > { > "listeners": { > "*:80": { > "pass": "routes" > } > }, > > "routes": [ > { > "action": { > "share": "/home/alx/srv/www/static/$uri", > "index": "dir" > } > } > ] > } > $ ls -l /home/alx/srv/www/static/ > total 16 > -rw-r--r-- 1 alx alx 4 Dec 14 17:51 GET.html > drwxr-xr-x 2 alx alx 4096 Feb 14 12:18 dir > -rw-r--r-- 1 alx alx 6 Dec 14 17:52 index.html > -rw-r--r-- 1 alx alx 10 Dec 15 12:55 localhost.html > > > $ curl --dump-header - http://localhost > HTTP/1.1 301 Moved Permanently > Location: // > Server: Unit/1.27.0 > Date: Mon, 14 Feb 2022 11:19:24 GMT > Content-Length: 0 > > > Setting "dir/" instead of "dir" behaves identically too. > > What is the expected behavior when trying to serve a directory? With > old nginx (open-source), with autoindex off, and try_files ${uri}/, I'm > getting 403 Forbidden. Where does Unit decide about that? [..] I think there shouldn't be any differences between different situations when "index" doesn't point to an accessible file. That can be a directory or can be a socket, for example. Both Unit and nginx return in similar cases 404: if (!of.is_file) { ngx_log_error(NGX_LOG_CRIT, log, 0, "\"%s\" is not a regular file", path.data); return NGX_HTTP_NOT_FOUND; } -- if (nxt_slow_path(!nxt_is_dir(&fi))) { nxt_log(task, NXT_LOG_ERR, "\"%FN\" is not a regular file", f->name); nxt_http_static_next(task, r, ctx, NXT_HTTP_NOT_FOUND); return; } Because it's like the file was requested (or indirectly requested via "index"), but no file was found by that name. -- Valentin From alx.manpages at gmail.com Mon Feb 14 14:14:41 2022 From: alx.manpages at gmail.com (Alejandro Colomar (man-pages)) Date: Mon, 14 Feb 2022 15:14:41 +0100 Subject: [PATCH v4 07/12] Static: add "index" option In-Reply-To: <2813833.e9J7NaK4W3@vbart-laptop> References: <20211219013030.51372-1-alx.manpages@gmail.com> <36737EAD-4644-409F-B6E2-022E48AFDD47@nginx.com> <59461183-7804-bbc6-c245-ed0c3b235da0@gmail.com> <2813833.e9J7NaK4W3@vbart-laptop> Message-ID: <024b479b-3d45-c689-c1b9-8274601388e7@gmail.com> On 2/14/22 13:12, Valentin V. Bartenev wrote: > On Monday, 14 February 2022 14:33:21 MSK Alejandro Colomar (man-pages) wrote: > [..] >>> 3) router crash when variable in index value is empty >> >> The same behavior happens when you set up index to the name of a dir: >> >> $ cat config.json >> { >> "listeners": { >> "*:80": { >> "pass": "routes" >> } >> }, >> >> "routes": [ >> { >> "action": { >> "share": "/home/alx/srv/www/static/$uri", >> "index": "dir" >> } >> } >> ] >> } >> $ ls -l /home/alx/srv/www/static/ >> total 16 >> -rw-r--r-- 1 alx alx 4 Dec 14 17:51 GET.html >> drwxr-xr-x 2 alx alx 4096 Feb 14 12:18 dir >> -rw-r--r-- 1 alx alx 6 Dec 14 17:52 index.html >> -rw-r--r-- 1 alx alx 10 Dec 15 12:55 localhost.html >> >> >> $ curl --dump-header - http://localhost >> HTTP/1.1 301 Moved Permanently >> Location: // >> Server: Unit/1.27.0 >> Date: Mon, 14 Feb 2022 11:19:24 GMT >> Content-Length: 0 >> >> >> Setting "dir/" instead of "dir" behaves identically too. >> >> What is the expected behavior when trying to serve a directory? With >> old nginx (open-source), with autoindex off, and try_files ${uri}/, I'm >> getting 403 Forbidden. Where does Unit decide about that? > [..] > > I think there shouldn't be any differences between different situations > when "index" doesn't point to an accessible file. That can be a directory > or can be a socket, for example. > > Both Unit and nginx return in similar cases 404: > > if (!of.is_file) { > ngx_log_error(NGX_LOG_CRIT, log, 0, > "\"%s\" is not a regular file", path.data); > > return NGX_HTTP_NOT_FOUND; > } > > -- > if (nxt_slow_path(!nxt_is_dir(&fi))) { > nxt_log(task, NXT_LOG_ERR, "\"%FN\" is not a regular file", > f->name); > > nxt_http_static_next(task, r, ctx, NXT_HTTP_NOT_FOUND); > return; > } > > Because it's like the file was requested (or indirectly requested via "index"), > but no file was found by that name. I could reproduce the 301 behavior in nginx with a clean debian nginx install, by changing the index line in : ... # index index.html index.htm index.nginx-debian.html; index dir; ... And then create a directory 'dir' in (which is the 'root'): $ sudo mkdir /var/www/html/dir $ sudo systemctl reload nginx After that, a curl on localhost gives 301 Moved Permanently, exactly as in Unit after the proposed patch set. Regards, Alex -- Alejandro Colomar Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/ http://www.alejandro-colomar.es/ From alx.manpages at gmail.com Mon Feb 14 16:50:31 2022 From: alx.manpages at gmail.com (Alejandro Colomar (man-pages)) Date: Mon, 14 Feb 2022 17:50:31 +0100 Subject: [PATCH v4 07/12] Static: add "index" option In-Reply-To: <59461183-7804-bbc6-c245-ed0c3b235da0@gmail.com> References: <20211219013030.51372-1-alx.manpages@gmail.com> <20211223192508.12146-8-alx.manpages@gmail.com> <36737EAD-4644-409F-B6E2-022E48AFDD47@nginx.com> <59461183-7804-bbc6-c245-ed0c3b235da0@gmail.com> Message-ID: <3722268a-564c-f632-3a73-6b88d42e72a4@gmail.com> On 2/14/22 12:33, Alejandro Colomar (man-pages) wrote: > Hi, Andrei! > > On 1/28/22 13:27, Andrei Zeliankou wrote: >>> On 23 Dec 2021, at 19:25, Alejandro Colomar >>> wrote: >>> >>> This supports a new option "index", that configures a custom index >>> file name that will be served when a directory is requested.  This >>> initial support only allows a single fixed string.  An example: >>> >>> { >>>     "share": "/www/data/static/$uri", >>>     "index": "lookatthis.htm" >>> } >>> > [...] > >> Also noticed problems: >> >> 1) can't compile patch set with specified `--debug` option: >> >> src/nxt_http_static.c:264:23: error: no member named 'index' in >> 'nxt_http_static_conf_t' >>      nxt_var_raw(conf->index, &idx); >>                  ~~~~  ^ > > Oops, fixed. > >> >> 2) router crash when index is the empty array > > I'm still investigating this one. > Fixed too. So the tests are still pending, and the question regarding the 301 for non-regular files too. Cheers, Alex -- Alejandro Colomar Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/ http://www.alejandro-colomar.es/ From alx.manpages at gmail.com Mon Feb 14 17:22:28 2022 From: alx.manpages at gmail.com (Alejandro Colomar (man-pages)) Date: Mon, 14 Feb 2022 18:22:28 +0100 Subject: [PATCH v4 07/12] Static: add "index" option In-Reply-To: <024b479b-3d45-c689-c1b9-8274601388e7@gmail.com> References: <20211219013030.51372-1-alx.manpages@gmail.com> <36737EAD-4644-409F-B6E2-022E48AFDD47@nginx.com> <59461183-7804-bbc6-c245-ed0c3b235da0@gmail.com> <2813833.e9J7NaK4W3@vbart-laptop> <024b479b-3d45-c689-c1b9-8274601388e7@gmail.com> Message-ID: On 2/14/22 15:14, Alejandro Colomar (man-pages) wrote: > I could reproduce the 301 behavior in nginx with a clean debian nginx > install, by changing the index line in > : > > ... > #    index index.html index.htm index.nginx-debian.html; >     index dir; > ... > > And then create a directory 'dir' in (which is the 'root'): > > $ sudo mkdir /var/www/html/dir > $ sudo systemctl reload nginx > > After that, a curl on localhost gives 301 Moved Permanently, exactly as > in Unit after the proposed patch set. > > > Regards, > > Alex > Full config file: $ cat /etc/nginx/sites-available/default | grep -v '\s*#' server { listen 80 default_server; listen [::]:80 default_server; root /var/www/html; index dir; server_name _; location / { try_files $uri $uri/ =404; } } $ ls -l /var/www/html/ total 8 drwxr-xr-x 2 root root 4096 Feb 14 14:56 dir -rw-r--r-- 1 root root 612 Feb 14 14:40 index.nginx-debian.html $ curl localhost 301 Moved Permanently

301 Moved Permanently


nginx/1.18.0
$ sudo mv /var/www/html/dir /var/www/directory user at sqli:/etc/nginx/sites-available$ curl localhost 403 Forbidden

403 Forbidden


nginx/1.18.0
So if nginx tries to serve a nonexisting file, I get 403, but if it tries to serve a directory (and autoindex is off), I get 301. -- Alejandro Colomar Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/ http://www.alejandro-colomar.es/ From alx.manpages at gmail.com Tue Feb 15 13:28:45 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 14:28:45 +0100 Subject: [RFC] Static: return 404 when "index" is a non-regular file Message-ID: <20220215132845.32651-1-alx.manpages@gmail.com> When index is a file, but not a regular file nor a directory, so it may be for example a FIFO, Unit returns 404. However, for directories, it was returning 301. Nginx also returns 301 (and is documented, as Valentin pointed out)[1], but we prefer to be consistent (and intuitive) with other invalid files in Unit and so return 404 for dirs too. [1]: I got the following pytest errors after this patch, but I'm not sure of what those tests mean. ===================================== FAILURES ===================================== ____________________ TestStaticChroot.test_static_chroot_empty _____________________ self = temp_dir = '/tmp/unit-test-19ucrwpo' def test_static_chroot_empty(self, temp_dir): assert 'success' in self.conf( {"share": temp_dir + "/assets$uri", "chroot": ""}, 'routes/0/action', ), 'configure chroot empty absolute' assert ( self.get(url='/dir/file')['status'] == 200 ), 'chroot empty absolute' assert 'success' in self.conf( {"share": ".$uri", "chroot": ""}, 'routes/0/action', ), 'configure chroot empty relative' > assert ( self.get(url=self.test_path)['status'] == 200 ), 'chroot empty relative' E AssertionError: chroot empty relative E assert 404 == 200 E +404 E -200 test_static_chroot.py:107: AssertionError ___________________ TestStaticChroot.test_static_chroot_relative ___________________ self = , is_su = False temp_dir = '/tmp/unit-test-19ucrwpo' def test_static_chroot_relative(self, is_su, temp_dir): if is_su: pytest.skip('does\'t work under root') assert 'success' in self.conf( {"share": temp_dir + "/assets$uri", "chroot": "."}, 'routes/0/action', ), 'configure relative chroot' assert self.get(url='/dir/file')['status'] == 403, 'relative chroot' assert 'success' in self.conf( {"share": ".$uri"}, 'routes/0/action', ), 'configure relative share' > assert self.get(url=self.test_path)['status'] == 200, 'relative share' E AssertionError: relative share E assert 404 == 200 E +404 E -200 test_static_chroot.py:126: AssertionError ============================= short test summary info ============================== FAILED test_static_chroot.py::TestStaticChroot::test_static_chroot_empty - Assert... FAILED test_static_chroot.py::TestStaticChroot::test_static_chroot_relative - Ass... ==================== 2 failed, 62 passed, 706 skipped in 2.77s ===================== Reported-by: Andrei Zeliankou Suggested-by: "Valentin V. Bartenev" Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: Igor Sysoev Cc: Zhidao HONG Cc: Oisin Canty Cc: Maxim Romanov --- Hi Valentin, I'm not entirely convinced about this patch. I used the same condition that triggers the use of the index, to consider that a directory is invalid. That is, we know that the path is a dir _after_ index has been applied. But since I don't know all the other conditions that can lead to that code, I'm not entirely sure this is exactly what we want. src/nxt_http_static.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index ea97147..74245cd 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -623,7 +623,9 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) /* Not a file. */ nxt_file_close(task, f); - if (nxt_slow_path(!nxt_is_dir(&fi))) { + if (nxt_slow_path(!nxt_is_dir(&fi) + || ctx->share.start[ctx->share.length - 1] == '/')) + { nxt_log(task, NXT_LOG_ERR, "\"%FN\" is not a regular file", f->name); -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:41 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:41 +0100 Subject: [PATCH v6 01/15] .gitignore: ignore new files In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-2-alx.manpages@gmail.com> When working with git, one has to be very careful of not adding temporary files resulting from compilation, testing, or text editors. With this .gitignore, one can add all of the changes with `git add .` without having to specify the files manually, avoiding accidents. When a new file is created and should really be added, one can add it still with the '-f' option to git-add(1). Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- .gitignore | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 .gitignore diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..ed833f0 --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +# Ignore everything new by default +/* -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:43 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:43 +0100 Subject: [PATCH v6 03/15] Remove special cases for non-NXT_CONF_VALUE_ARRAY In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-4-alx.manpages@gmail.com> The previous commit made the functions accepting NXT_CONF_VALUE_ARRAY more generic, so that now they can handle non-arrays correctly. Modify calling code to remove special cases for arrays and non-arrays, taking special care that the path for non arrays is logically equivalent to the previous special cased code. Use the now-generic array code only. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_http_route.c | 45 +++-------------------------------------- src/nxt_http_static.c | 28 ++++++------------------- src/nxt_openssl.c | 24 ++++++---------------- src/nxt_router.c | 23 ++++++--------------- src/python/nxt_python.c | 9 --------- 5 files changed, 21 insertions(+), 108 deletions(-) diff --git a/src/nxt_http_route.c b/src/nxt_http_route.c index 606bf26..71ebf3f 100644 --- a/src/nxt_http_route.c +++ b/src/nxt_http_route.c @@ -718,13 +718,11 @@ nxt_http_route_table_create(nxt_task_t *task, nxt_mp_t *mp, { size_t size; uint32_t i, n; - nxt_bool_t array; nxt_conf_value_t *ruleset_cv; nxt_http_route_table_t *table; nxt_http_route_ruleset_t *ruleset; - array = (nxt_conf_type(table_cv) == NXT_CONF_ARRAY); - n = array ? nxt_conf_array_elements_count(table_cv) : 1; + n = nxt_conf_array_elements_count(table_cv); size = sizeof(nxt_http_route_table_t) + n * sizeof(nxt_http_route_ruleset_t *); @@ -736,18 +734,6 @@ nxt_http_route_table_create(nxt_task_t *task, nxt_mp_t *mp, table->items = n; table->object = NXT_HTTP_ROUTE_TABLE; - if (!array) { - ruleset = nxt_http_route_ruleset_create(task, mp, table_cv, object, - case_sensitive, encoding); - if (nxt_slow_path(ruleset == NULL)) { - return NULL; - } - - table->ruleset[0] = ruleset; - - return table; - } - for (i = 0; i < n; i++) { ruleset_cv = nxt_conf_get_array_element(table_cv, i); @@ -911,13 +897,11 @@ nxt_http_route_rule_create(nxt_task_t *task, nxt_mp_t *mp, size_t size; uint32_t i, n; nxt_int_t ret; - nxt_bool_t string; nxt_conf_value_t *value; nxt_http_route_rule_t *rule; nxt_http_route_pattern_t *pattern; - string = (nxt_conf_type(cv) != NXT_CONF_ARRAY); - n = string ? 1 : nxt_conf_array_elements_count(cv); + n = nxt_conf_array_elements_count(cv); size = sizeof(nxt_http_route_rule_t) + n * sizeof(nxt_http_route_pattern_t); rule = nxt_mp_alloc(mp, size); @@ -929,17 +913,6 @@ nxt_http_route_rule_create(nxt_task_t *task, nxt_mp_t *mp, pattern = &rule->pattern[0]; - if (string) { - pattern[0].case_sensitive = case_sensitive; - ret = nxt_http_route_pattern_create(task, mp, cv, &pattern[0], - pattern_case, encoding); - if (nxt_slow_path(ret != NXT_OK)) { - return NULL; - } - - return rule; - } - nxt_conf_array_qsort(cv, nxt_http_pattern_compare); for (i = 0; i < n; i++) { @@ -963,13 +936,11 @@ nxt_http_route_addr_rule_create(nxt_task_t *task, nxt_mp_t *mp, { size_t size; uint32_t i, n; - nxt_bool_t array; nxt_conf_value_t *value; nxt_http_route_addr_rule_t *addr_rule; nxt_http_route_addr_pattern_t *pattern; - array = (nxt_conf_type(cv) == NXT_CONF_ARRAY); - n = array ? nxt_conf_array_elements_count(cv) : 1; + n = nxt_conf_array_elements_count(cv); size = sizeof(nxt_http_route_addr_rule_t) + n * sizeof(nxt_http_route_addr_pattern_t); @@ -981,16 +952,6 @@ nxt_http_route_addr_rule_create(nxt_task_t *task, nxt_mp_t *mp, addr_rule->items = n; - if (!array) { - pattern = &addr_rule->addr_pattern[0]; - - if (nxt_http_route_addr_pattern_parse(mp, pattern, cv) != NXT_OK) { - return NULL; - } - - return addr_rule; - } - for (i = 0; i < n; i++) { pattern = &addr_rule->addr_pattern[i]; value = nxt_conf_get_array_element(cv, i); diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 36c1ebc..6b7cee7 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -77,7 +77,6 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, nxt_mp_t *mp; nxt_str_t str; nxt_var_t *var; - nxt_bool_t array; nxt_conf_value_t *cv; nxt_http_static_conf_t *conf; @@ -91,39 +90,24 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, action->handler = nxt_http_static; action->u.conf = conf; - array = (nxt_conf_type(acf->share) == NXT_CONF_ARRAY); - conf->nshares = array ? nxt_conf_array_elements_count(acf->share) : 1; - + conf->nshares = nxt_conf_array_elements_count(acf->share); conf->shares = nxt_mp_zget(mp, sizeof(nxt_http_static_share_t) * conf->nshares); if (nxt_slow_path(conf->shares == NULL)) { return NXT_ERROR; } - if (array) { - for (i = 0; i < conf->nshares; i++) { - cv = nxt_conf_get_array_element(acf->share, i); - nxt_conf_get_string(cv, &str); - - var = nxt_var_compile(&str, mp, 1); - if (nxt_slow_path(var == NULL)) { - return NXT_ERROR; - } - - conf->shares[i].var = var; - conf->shares[i].is_const = nxt_var_is_const(var); - } - - } else { - nxt_conf_get_string(acf->share, &str); + for (i = 0; i < conf->nshares; i++) { + cv = nxt_conf_get_array_element(acf->share, i); + nxt_conf_get_string(cv, &str); var = nxt_var_compile(&str, mp, 1); if (nxt_slow_path(var == NULL)) { return NXT_ERROR; } - conf->shares[0].var = var; - conf->shares[0].is_const = nxt_var_is_const(var); + conf->shares[i].var = var; + conf->shares[i].is_const = nxt_var_is_const(var); } #if (NXT_HAVE_OPENAT2) diff --git a/src/nxt_openssl.c b/src/nxt_openssl.c index 1e08015..a4564f1 100644 --- a/src/nxt_openssl.c +++ b/src/nxt_openssl.c @@ -644,16 +644,10 @@ nxt_tls_ticket_keys(nxt_task_t *task, SSL_CTX *ctx, nxt_tls_init_t *tls_init, return NXT_OK; } - if (nxt_conf_type(tickets_conf) == NXT_CONF_ARRAY) { - count = nxt_conf_array_elements_count(tickets_conf); + count = nxt_conf_array_elements_count(tickets_conf); - if (count == 0) { - goto no_ticket; - } - - } else { - /* nxt_conf_type(tickets_conf) == NXT_CONF_STRING */ - count = 1; + if (count == 0) { + goto no_ticket; } #ifdef SSL_CTRL_SET_TLSEXT_TICKET_KEY_CB @@ -673,15 +667,9 @@ nxt_tls_ticket_keys(nxt_task_t *task, SSL_CTX *ctx, nxt_tls_init_t *tls_init, i++; - if (nxt_conf_type(tickets_conf) == NXT_CONF_ARRAY) { - member = nxt_conf_get_array_element(tickets_conf, count - i); - if (member == NULL) { - break; - } - - } else { - /* nxt_conf_type(tickets_conf) == NXT_CONF_STRING */ - member = tickets_conf; + member = nxt_conf_get_array_element(tickets_conf, count - i); + if (member == NULL) { + break; } nxt_conf_get_string(member, &value); diff --git a/src/nxt_router.c b/src/nxt_router.c index 52ea0f3..b8ee0c7 100644 --- a/src/nxt_router.c +++ b/src/nxt_router.c @@ -1924,25 +1924,14 @@ nxt_router_conf_create(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, tls_init->tickets_conf = nxt_conf_get_path(listener, &conf_tickets); - if (nxt_conf_type(certificate) == NXT_CONF_ARRAY) { - n = nxt_conf_array_elements_count(certificate); - - for (i = 0; i < n; i++) { - value = nxt_conf_get_array_element(certificate, i); - - nxt_assert(value != NULL); + n = nxt_conf_array_elements_count(certificate); + for (i = 0; i < n; i++) { + value = nxt_conf_get_array_element(certificate, i); - ret = nxt_router_conf_tls_insert(tmcf, value, skcf, - tls_init, i == 0); - if (nxt_slow_path(ret != NXT_OK)) { - goto fail; - } - } + nxt_assert(value != NULL); - } else { - /* NXT_CONF_STRING */ - ret = nxt_router_conf_tls_insert(tmcf, certificate, skcf, - tls_init, 1); + ret = nxt_router_conf_tls_insert(tmcf, value, skcf, + tls_init, i == 0); if (nxt_slow_path(ret != NXT_OK)) { goto fail; } diff --git a/src/python/nxt_python.c b/src/python/nxt_python.c index 8687c86..3edaa08 100644 --- a/src/python/nxt_python.c +++ b/src/python/nxt_python.c @@ -411,14 +411,7 @@ nxt_python_set_path(nxt_task_t *task, nxt_conf_value_t *value) /* sys is a Borrowed reference. */ - if (nxt_conf_type(value) == NXT_CONF_STRING) { - n = 0; - goto value_is_string; - } - - /* NXT_CONF_ARRAY */ array = value; - n = nxt_conf_array_elements_count(array); while (n != 0) { @@ -432,8 +425,6 @@ nxt_python_set_path(nxt_task_t *task, nxt_conf_value_t *value) value = nxt_conf_get_array_element(array, n); - value_is_string: - nxt_conf_get_string(value, &str); path = PyString_FromStringAndSize((char *) str.start, str.length); -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:42 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:42 +0100 Subject: [PATCH v6 02/15] nxt_conf.c: Treat non-arrays as arrays of size 1 In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-3-alx.manpages@gmail.com> Similar to how C pointers to variables can always be considered as pointers to the first element of an array of size 1 (see the following code for an example of how they are equivalent), treating non-NXT_CONF_VALUE_ARRAY as if they were NXT_CONF_VALUE_ARRAYs of size 1 allows for simpler and more generic code. void foo(ptrdiff_t sz, int arr[sz]) { for (ptrdiff_t i = 0; i < sz; i++) arr[i] = 0; } void bar(void) { int x; int y[1]; foo(1, &x); foo(1, y); } nxt_conf_array_elements_count(): Return a size of 1 when input is non-array, instead of causing undefined behavior. That value (1) makes sense because it will be used as the limiter of a loop that loops over the array and calls nxt_conf_get_array_element(), which will return a correct element for such loops. nxt_conf_get_array_element(): Return the input pointer unmodified (i.e., a pointer to the unique element of a hypothetical array), instead of returning NULL, which wasn't very useful. nxt_conf_array_qsort(): Since it's a no-op for non-arrays, this doesn't need to be modified. Example of (correct) usage: void baz(nxt_conf_value_t *x) { /* 'x' may or may not be a NXT_CONF_VALUE_ARRAY */ ptrdiff_t n; nxt_conf_value_t *elem; n = nxt_conf_array_elements_count(x); for (ptrdiff_t i = 0; i < n; i++) { elem = nxt_conf_get_array_element(x, i); /* do some work with 'elem' */ } } Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_conf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/nxt_conf.c b/src/nxt_conf.c index 1aca0a7..3f0cfa8 100644 --- a/src/nxt_conf.c +++ b/src/nxt_conf.c @@ -387,7 +387,7 @@ nxt_conf_set_element_string_dup(nxt_conf_value_t *array, nxt_mp_t *mp, nxt_uint_t nxt_conf_array_elements_count(nxt_conf_value_t *value) { - return value->u.array->count; + return (value->type == NXT_CONF_VALUE_ARRAY) ? value->u.array->count : 1; } @@ -736,7 +736,7 @@ nxt_conf_get_array_element(nxt_conf_value_t *value, uint32_t index) nxt_conf_array_t *array; if (value->type != NXT_CONF_VALUE_ARRAY) { - return NULL; + return (index == 0) ? value : NULL; } array = value->u.array; -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:40 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:40 +0100 Subject: [PATCH v6 00/15] Static: Implement new "index" option Message-ID: <20220215145755.40239-1-alx.manpages@gmail.com> Changes: v2: - Add support for variables in "index" - Don't move share_idx, since that caused unnecessary padding. - Use v3 : - Don't use (as previously discussed). - Add a .gitignore: While doing this work, I had quite some accidents. This prevented them from happening again. I based it on the .gitignore of the Linux man-pages . - Treat non-arrays as arrays of size 1 (as discussed earlier today: ). - Simplify all of the code that can be simplified thanks to the change mentioned above (arrays of size 1). - Fix the indentation of some random lines that I found to have an incorrect indentation. - Reduce branching (especially 'else's), to improve readability. I didn't merge this change into the patch that adds the feature because there it adds too much complexity for nothing, and the patches read better separately. However, I patched it previous to the third step to simplify that patch too, so this change sits on a patch of its own better, I think. - Use const for a read-only parameter of a function. I didn't change other functions (not even in the same file), since that would mean a huuuge diff, and for consistency we would need to do it everywhere. So I only did it where I *needed* it to compile, for now. v4 : - Actually iterate. v5 : - Remove spurious 'else' [Valentin] v6: - Add Andrei's tests (and remove my changes to the existing tests). - Fix bugs reported by Andrei: - Reject []. - Don't return 301 for a directory (maybe I should add a check that the string can't be empty; do we want that?) - Fix non-compiling debug code. Alejandro Colomar (14): .gitignore: ignore new files nxt_conf.c: Treat non-arrays as arrays of size 1 Remove special cases for non-NXT_CONF_VALUE_ARRAY Fix indentation nxt_http_static_ctx_t: rename field 'index' to 'share_idx' test_static.py: formatting fix Static: return 404 when "index" is a non-regular file Static: add "index" option nxt_http_static_iterate(): generalize code Static: variables in the "index" option nxt_conf_set_string_dup(): Use 'const' for read-only parameter nxt_http_static_init(): Simplify branching Static: multiple paths in the "index" option Static: optimize "index" iteration Andrei Zeliankou (1): Tests: added tests for "index" option. .gitignore | 2 + docs/changes.xml | 18 +++++ src/nxt_conf.c | 7 +- src/nxt_conf.h | 2 +- src/nxt_conf_validation.c | 55 +++++++++++++++- src/nxt_gnutls.c | 4 +- src/nxt_http.h | 15 +++-- src/nxt_http_route.c | 54 +++------------ src/nxt_http_static.c | 134 +++++++++++++++++++++++++------------- src/nxt_main_process.c | 2 +- src/nxt_openssl.c | 24 ++----- src/nxt_router.c | 23 ++----- src/nxt_time_parse.c | 6 +- src/nxt_unit.c | 4 +- src/nxt_work_queue.h | 4 +- src/perl/nxt_perl_psgi.c | 2 +- src/python/nxt_python.c | 9 --- test/test_static.py | 63 ++++++++++++++++-- 18 files changed, 266 insertions(+), 162 deletions(-) create mode 100644 .gitignore -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:46 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:46 +0100 Subject: [PATCH v6 06/15] test_static.py: formatting fix In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-7-alx.manpages@gmail.com> Expand indentation, to allow for the addition of an "index" configuration. This only changes whitespace/indentation. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- test/test_static.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/test/test_static.py b/test/test_static.py index 1d9c754..656bb7d 100644 --- a/test/test_static.py +++ b/test/test_static.py @@ -31,7 +31,11 @@ class TestStatic(TestApplicationProto): { "listeners": {"*:7080": {"pass": "routes"}}, "routes": [ - {"action": {"share": option.temp_dir + "/assets$uri"}} + { + "action": { + "share": option.temp_dir + "/assets$uri" + } + } ], "settings": { "http": { -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:48 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:48 +0100 Subject: [PATCH v6 08/15] Static: return 404 when "index" is a non-regular file In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-9-alx.manpages@gmail.com> When index is a file, but not a regular file nor a directory, so it may be for example a FIFO, Unit returns 404. However, for directories, it was returning 301. Nginx also returns 301 (and is documented, as Valentin pointed out)[1], but we prefer to be consistent (and intuitive) with other invalid files in Unit and so return 404 for dirs too. [1]: I got the following pytest errors after this patch, but I'm not sure of what those tests mean. ===================================== FAILURES ===================================== ____________________ TestStaticChroot.test_static_chroot_empty _____________________ self = temp_dir = '/tmp/unit-test-19ucrwpo' def test_static_chroot_empty(self, temp_dir): assert 'success' in self.conf( {"share": temp_dir + "/assets$uri", "chroot": ""}, 'routes/0/action', ), 'configure chroot empty absolute' assert ( self.get(url='/dir/file')['status'] == 200 ), 'chroot empty absolute' assert 'success' in self.conf( {"share": ".$uri", "chroot": ""}, 'routes/0/action', ), 'configure chroot empty relative' > assert ( self.get(url=self.test_path)['status'] == 200 ), 'chroot empty relative' E AssertionError: chroot empty relative E assert 404 == 200 E +404 E -200 test_static_chroot.py:107: AssertionError ___________________ TestStaticChroot.test_static_chroot_relative ___________________ self = , is_su = False temp_dir = '/tmp/unit-test-19ucrwpo' def test_static_chroot_relative(self, is_su, temp_dir): if is_su: pytest.skip('does\'t work under root') assert 'success' in self.conf( {"share": temp_dir + "/assets$uri", "chroot": "."}, 'routes/0/action', ), 'configure relative chroot' assert self.get(url='/dir/file')['status'] == 403, 'relative chroot' assert 'success' in self.conf( {"share": ".$uri"}, 'routes/0/action', ), 'configure relative share' > assert self.get(url=self.test_path)['status'] == 200, 'relative share' E AssertionError: relative share E assert 404 == 200 E +404 E -200 test_static_chroot.py:126: AssertionError ============================= short test summary info ============================== FAILED test_static_chroot.py::TestStaticChroot::test_static_chroot_empty - Assert... FAILED test_static_chroot.py::TestStaticChroot::test_static_chroot_relative - Ass... ==================== 2 failed, 62 passed, 706 skipped in 2.77s ===================== Reported-by: Andrei Zeliankou Suggested-by: "Valentin V. Bartenev" Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: Igor Sysoev Cc: Zhidao HONG Cc: Oisin Canty Cc: Maxim Romanov --- src/nxt_http_static.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 663f896..7f1254c 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -571,7 +571,9 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) /* Not a file. */ nxt_file_close(task, f); - if (nxt_slow_path(!nxt_is_dir(&fi))) { + if (nxt_slow_path(!nxt_is_dir(&fi) + || ctx->share.start[ctx->share.length - 1] == '/')) + { nxt_log(task, NXT_LOG_ERR, "\"%FN\" is not a regular file", f->name); -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:49 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:49 +0100 Subject: [PATCH v6 09/15] Static: add "index" option In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-10-alx.manpages@gmail.com> This supports a new option "index", that configures a custom index file name that will be served when a directory is requested. This initial support only allows a single fixed string. An example: { "share": "/www/data/static/$uri", "index": "lookatthis.htm" } When is requested, will be served. If the option is missing, default to "index.html". === nxt_conf_validator.c: Accept "index" as a member of "share", and make sure it is a string. === I tried this feature in my own computer, where I tried the following: - Setting "index" to "lookatthis.htm", and check that the correct file is being served (check both a different name and a different extension). - Not setting "index", and check that is being served. - Settind "index" to an array of strings, and check that the configuration fails: { "error": "Invalid configuration.", "detail": "The \"index\" value must be a string, but not an array." } Tested-by: Alejandro Colomar Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Maxim Romanov --- docs/changes.xml | 6 ++++++ src/nxt_conf_validation.c | 3 +++ src/nxt_http.h | 1 + src/nxt_http_route.c | 5 +++++ src/nxt_http_static.c | 32 +++++++++++++++++++++++--------- test/test_static.py | 11 ++++++----- 6 files changed, 44 insertions(+), 14 deletions(-) diff --git a/docs/changes.xml b/docs/changes.xml index a6b1178..81708c3 100644 --- a/docs/changes.xml +++ b/docs/changes.xml @@ -18,6 +18,12 @@ date="" time="" packager="Andrei Belov <defan at nginx.com>"> + + +new "index" option to specify a custom index file name. + + + NGINX Unit updated to 1.27.0. diff --git a/src/nxt_conf_validation.c b/src/nxt_conf_validation.c index 7edd638..bdc38b0 100644 --- a/src/nxt_conf_validation.c +++ b/src/nxt_conf_validation.c @@ -645,6 +645,9 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_share_action_members[] = { .name = nxt_string("share"), .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, .validator = nxt_conf_vldt_share, + }, { + .name = nxt_string("index"), + .type = NXT_CONF_VLDT_STRING, }, { .name = nxt_string("types"), .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, diff --git a/src/nxt_http.h b/src/nxt_http.h index 7cef134..9b819fb 100644 --- a/src/nxt_http.h +++ b/src/nxt_http.h @@ -209,6 +209,7 @@ typedef struct { nxt_str_t location; nxt_conf_value_t *proxy; nxt_conf_value_t *share; + nxt_conf_value_t *index; nxt_str_t chroot; nxt_conf_value_t *follow_symlinks; nxt_conf_value_t *traverse_mounts; diff --git a/src/nxt_http_route.c b/src/nxt_http_route.c index 26f0662..c92f2cd 100644 --- a/src/nxt_http_route.c +++ b/src/nxt_http_route.c @@ -639,6 +639,11 @@ static nxt_conf_map_t nxt_http_route_action_conf[] = { NXT_CONF_MAP_PTR, offsetof(nxt_http_action_conf_t, share) }, + { + nxt_string("index"), + NXT_CONF_MAP_PTR, + offsetof(nxt_http_action_conf_t, index) + }, { nxt_string("chroot"), NXT_CONF_MAP_STR, diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 7f1254c..0327861 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -19,6 +19,7 @@ typedef struct { typedef struct { nxt_uint_t nshares; nxt_http_static_share_t *shares; + nxt_str_t index; #if (NXT_HAVE_OPENAT2) nxt_var_t *chroot; nxt_uint_t resolve; @@ -30,6 +31,7 @@ typedef struct { typedef struct { nxt_http_action_t *action; nxt_str_t share; + nxt_str_t index; #if (NXT_HAVE_OPENAT2) nxt_str_t chroot; #endif @@ -80,6 +82,8 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, nxt_conf_value_t *cv; nxt_http_static_conf_t *conf; + static const nxt_str_t default_index = nxt_string("index.html"); + mp = tmcf->router_conf->mem_pool; conf = nxt_mp_zget(mp, sizeof(nxt_http_static_conf_t)); @@ -110,6 +114,13 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, conf->shares[i].is_const = nxt_var_is_const(var); } + if (acf->index == NULL) { + conf->index = default_index; + + } else { + nxt_conf_get_string(acf->index, &conf->index); + } + #if (NXT_HAVE_OPENAT2) if (acf->chroot.length > 0) { nxt_str_t chr, shr; @@ -222,8 +233,10 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, #if (NXT_DEBUG) nxt_str_t shr; + nxt_str_t idx; nxt_var_raw(share->var, &shr); + idx = conf->index; #if (NXT_HAVE_OPENAT2) nxt_str_t chr; @@ -235,12 +248,15 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_str_set(&chr, ""); } - nxt_debug(task, "http static: \"%V\" (chroot: \"%V\")", &shr, &chr); + nxt_debug(task, "http static: \"%V\", index: \"%V\" (chroot: \"%V\")", + &shr, &idx, &chr); #else - nxt_debug(task, "http static: \"%V\"", &shr); + nxt_debug(task, "http static: \"%V\", index: \"%V\"", &shr, &idx); #endif #endif /* NXT_DEBUG */ + ctx->index = conf->index; + if (share->is_const) { nxt_var_raw(share->var, &ctx->share); @@ -282,7 +298,7 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) struct tm tm; nxt_buf_t *fb; nxt_int_t ret; - nxt_str_t *shr, exten, *mtype; + nxt_str_t *shr, *index, exten, *mtype; nxt_uint_t level; nxt_file_t *f, file; nxt_file_info_t fi; @@ -295,8 +311,6 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) nxt_http_static_ctx_t *ctx; nxt_http_static_conf_t *conf; - static const nxt_str_t index = nxt_string("index.html"); - r = obj; ctx = data; action = ctx->action; @@ -307,12 +321,12 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) mtype = NULL; shr = &ctx->share; + index = &ctx->index; if (shr->start[shr->length - 1] == '/') { - /* TODO: dynamic index setting. */ - nxt_str_set(&exten, ".html"); + nxt_http_static_extract_extension(index, &exten); - length = shr->length + index.length; + length = shr->length + index->length; fname = nxt_mp_nget(r->mem_pool, length + 1); if (nxt_slow_path(fname == NULL)) { @@ -321,7 +335,7 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) p = fname; p = nxt_cpymem(p, shr->start, shr->length); - p = nxt_cpymem(p, index.start, index.length); + p = nxt_cpymem(p, index->start, index->length); *p = '\0'; } else { diff --git a/test/test_static.py b/test/test_static.py index bb96f09..9eea445 100644 --- a/test/test_static.py +++ b/test/test_static.py @@ -15,7 +15,7 @@ class TestStatic(TestApplicationProto): def setup_method(self): os.makedirs(option.temp_dir + '/assets/dir') - with open(option.temp_dir + '/assets/index.html', 'w') as index, open( + with open(option.temp_dir + '/assets/index_.html', 'w') as index, open( option.temp_dir + '/assets/README', 'w' ) as readme, open( option.temp_dir + '/assets/log.log', 'w' @@ -33,7 +33,8 @@ class TestStatic(TestApplicationProto): "routes": [ { "action": { - "share": option.temp_dir + "/assets$uri" + "share": option.temp_dir + "/assets$uri", + "index": "index_.html" } } ], @@ -147,7 +148,7 @@ class TestStatic(TestApplicationProto): assert self.get(url='/#blah')['body'] == '0123456789', 'index anchor' assert self.get(url='/dir/')['status'] == 404, 'index not found' - resp = self.get(url='/index.html/') + resp = self.get(url='/index_.html/') assert resp['status'] == 404, 'index not found 2 status' assert ( resp['headers']['Content-Type'] == 'text/html' @@ -171,7 +172,7 @@ class TestStatic(TestApplicationProto): assert etag != etag_2, 'different ETag' assert etag == self.get(url='/')['headers']['ETag'], 'same ETag' - with open(temp_dir + '/assets/index.html', 'w') as f: + with open(temp_dir + '/assets/index_.html', 'w') as f: f.write('blah') assert etag != self.get(url='/')['headers']['ETag'], 'new ETag' @@ -310,7 +311,7 @@ class TestStatic(TestApplicationProto): == 'text/x-code/x-blah/x-blah' ), 'mime_types string case insensitive' assert ( - self.get(url='/index.html')['headers']['Content-Type'] + self.get(url='/index_.html')['headers']['Content-Type'] == 'text/plain' ), 'mime_types html' assert ( -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:44 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:44 +0100 Subject: [PATCH v6 04/15] Fix indentation In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-5-alx.manpages@gmail.com> Some lines (incorrectly) had an indenteation of 5 or 9 spaces instead of 4 or 8. Fix them. Found with: $ ## 5 spaces instead of 4: $ $ find src/ -type f \ |grep -n '^ [^ *]'; $ ## 9 spaces instead of 8: $ $ find src/ -type f \ |grep -l '^ [^ *]' \ |xargs grep -L '^ [^ *].*)$' \ |xargs grep -n '^ [^ *]' Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_conf_validation.c | 4 ++-- src/nxt_gnutls.c | 4 ++-- src/nxt_http.h | 14 +++++++------- src/nxt_http_route.c | 4 ++-- src/nxt_main_process.c | 2 +- src/nxt_time_parse.c | 6 +++--- src/nxt_unit.c | 4 ++-- src/nxt_work_queue.h | 4 ++-- src/perl/nxt_perl_psgi.c | 2 +- 9 files changed, 22 insertions(+), 22 deletions(-) diff --git a/src/nxt_conf_validation.c b/src/nxt_conf_validation.c index 3f068bb..7edd638 100644 --- a/src/nxt_conf_validation.c +++ b/src/nxt_conf_validation.c @@ -114,7 +114,7 @@ static nxt_int_t nxt_conf_vldt_pass(nxt_conf_validation_t *vldt, static nxt_int_t nxt_conf_vldt_return(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, void *data); static nxt_int_t nxt_conf_vldt_share(nxt_conf_validation_t *vldt, - nxt_conf_value_t *value, void *data); + nxt_conf_value_t *value, void *data); static nxt_int_t nxt_conf_vldt_share_element(nxt_conf_validation_t *vldt, nxt_conf_value_t *value); static nxt_int_t nxt_conf_vldt_proxy(nxt_conf_validation_t *vldt, @@ -194,7 +194,7 @@ static nxt_int_t nxt_conf_vldt_java_classpath(nxt_conf_validation_t *vldt, static nxt_int_t nxt_conf_vldt_java_option(nxt_conf_validation_t *vldt, nxt_conf_value_t *value); static nxt_int_t nxt_conf_vldt_upstream(nxt_conf_validation_t *vldt, - nxt_str_t *name, nxt_conf_value_t *value); + nxt_str_t *name, nxt_conf_value_t *value); static nxt_int_t nxt_conf_vldt_server(nxt_conf_validation_t *vldt, nxt_str_t *name, nxt_conf_value_t *value); static nxt_int_t nxt_conf_vldt_server_weight(nxt_conf_validation_t *vldt, diff --git a/src/nxt_gnutls.c b/src/nxt_gnutls.c index 4618dce..aab4699 100644 --- a/src/nxt_gnutls.c +++ b/src/nxt_gnutls.c @@ -708,11 +708,11 @@ nxt_gnutls_log_error_level(nxt_event_conn_t *c, ssize_t err) case GNUTLS_E_UNKNOWN_CIPHER_SUITE: /* -21 */ - /* Disable gnutls_bye(), because it returns GNUTLS_E_INTERNAL_ERROR. */ + /* Disable gnutls_bye(), because it returns GNUTLS_E_INTERNAL_ERROR. */ ssltls = c->u.ssltls; ssltls->no_shutdown = 1; - /* Fall through. */ + /* Fall through. */ case GNUTLS_E_UNEXPECTED_PACKET_LENGTH: /* -9 */ c->socket.error = 1000; /* Nonexistent errno code. */ diff --git a/src/nxt_http.h b/src/nxt_http.h index 02d66f5..7cef134 100644 --- a/src/nxt_http.h +++ b/src/nxt_http.h @@ -90,17 +90,17 @@ typedef union { #define nxt_http_field_name_set(_field, _name) \ do { \ - (_field)->name_length = nxt_length(_name); \ - (_field)->name = (u_char *) _name; \ + (_field)->name_length = nxt_length(_name); \ + (_field)->name = (u_char *) _name; \ } while (0) #define nxt_http_field_set(_field, _name, _value) \ do { \ - (_field)->name_length = nxt_length(_name); \ - (_field)->value_length = nxt_length(_value); \ - (_field)->name = (u_char *) _name; \ - (_field)->value = (u_char *) _value; \ + (_field)->name_length = nxt_length(_name); \ + (_field)->value_length = nxt_length(_value); \ + (_field)->name = (u_char *) _name; \ + (_field)->value = (u_char *) _value; \ } while (0) @@ -238,7 +238,7 @@ typedef struct { void (*body_read)(nxt_task_t *task, nxt_http_request_t *r); void (*local_addr)(nxt_task_t *task, nxt_http_request_t *r); void (*header_send)(nxt_task_t *task, nxt_http_request_t *r, - nxt_work_handler_t body_handler, void *data); + nxt_work_handler_t body_handler, void *data); void (*send)(nxt_task_t *task, nxt_http_request_t *r, nxt_buf_t *out); nxt_off_t (*body_bytes_sent)(nxt_task_t *task, nxt_http_proto_t proto); void (*discard)(nxt_task_t *task, nxt_http_request_t *r, nxt_buf_t *last); diff --git a/src/nxt_http_route.c b/src/nxt_http_route.c index 71ebf3f..26f0662 100644 --- a/src/nxt_http_route.c +++ b/src/nxt_http_route.c @@ -932,7 +932,7 @@ nxt_http_route_rule_create(nxt_task_t *task, nxt_mp_t *mp, nxt_http_route_addr_rule_t * nxt_http_route_addr_rule_create(nxt_task_t *task, nxt_mp_t *mp, - nxt_conf_value_t *cv) + nxt_conf_value_t *cv) { size_t size; uint32_t i, n; @@ -2294,7 +2294,7 @@ nxt_http_route_cookie_parse(nxt_array_t *cookies, u_char *start, u_char *end) name = NULL; start = p + 1; - } + } } if (name != NULL) { diff --git a/src/nxt_main_process.c b/src/nxt_main_process.c index 9883f04..03761a1 100644 --- a/src/nxt_main_process.c +++ b/src/nxt_main_process.c @@ -579,7 +579,7 @@ nxt_main_process_created_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg) -1, msg->port_msg.stream, 0, NULL); return; } - } + } #endif diff --git a/src/nxt_time_parse.c b/src/nxt_time_parse.c index 5a32b21..94c4328 100644 --- a/src/nxt_time_parse.c +++ b/src/nxt_time_parse.c @@ -282,9 +282,9 @@ nxt_time_parse(const u_char *p, size_t len) days = days - 719527 + 31 + 28; s = (uint64_t) days * 86400 - + (nxt_uint_t) hour * 3600 - + (nxt_uint_t) min * 60 - + (nxt_uint_t) sec; + + (nxt_uint_t) hour * 3600 + + (nxt_uint_t) min * 60 + + (nxt_uint_t) sec; #if (NXT_TIME_T_SIZE <= 4) diff --git a/src/nxt_unit.c b/src/nxt_unit.c index dac612b..32bb07a 100644 --- a/src/nxt_unit.c +++ b/src/nxt_unit.c @@ -5114,9 +5114,9 @@ nxt_unit_ctx_alloc(nxt_unit_ctx_t *ctx, void *data) rc = nxt_unit_ctx_init(lib, new_ctx, data); if (nxt_slow_path(rc != NXT_UNIT_OK)) { - nxt_unit_free(ctx, new_ctx); + nxt_unit_free(ctx, new_ctx); - return NULL; + return NULL; } queue_fd = -1; diff --git a/src/nxt_work_queue.h b/src/nxt_work_queue.h index 6c2d6c2..ffa21d2 100644 --- a/src/nxt_work_queue.h +++ b/src/nxt_work_queue.h @@ -16,12 +16,12 @@ struct nxt_task_s { uint32_t ident; nxt_work_t *next_work; - /* TODO: exception_handler, prev/next task, subtasks. */ + /* TODO: exception_handler, prev/next task, subtasks. */ }; #define nxt_task_next_ident() \ - ((uint32_t) nxt_atomic_fetch_add(&nxt_task_ident, 1) & 0x3FFFFFFF) + ((uint32_t) nxt_atomic_fetch_add(&nxt_task_ident, 1) & 0x3FFFFFFF) /* diff --git a/src/perl/nxt_perl_psgi.c b/src/perl/nxt_perl_psgi.c index 749ebd8..08a6f29 100644 --- a/src/perl/nxt_perl_psgi.c +++ b/src/perl/nxt_perl_psgi.c @@ -612,7 +612,7 @@ nxt_perl_psgi_env_create(PerlInterpreter *my_perl, do { \ if (nxt_slow_path((FNS) != NXT_UNIT_OK)) \ goto fail; \ - } while (0) + } while (0) #define NL(S) (S), sizeof(S)-1 -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:45 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:45 +0100 Subject: [PATCH v6 05/15] nxt_http_static_ctx_t: rename field 'index' to 'share_idx' In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-6-alx.manpages@gmail.com> Having a configurable index filename will require adding an index field to this structure. The most natural name for that field is 'index', so the current index field should be renamed to allow for that. A sensible name is 'share_idx', since it's the index of the shares array in 'nxt_http_static_conf_t'. Instead of 'share_index' I opted for the shorter 'share_idx'. Also, when 'index' allows an array of filenames in a following commit, another similar variable 'index_idx' should be created, and having a different prefix and suffix seems more readable than for example 'index_index'. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_http_static.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 6b7cee7..663f896 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -33,7 +33,7 @@ typedef struct { #if (NXT_HAVE_OPENAT2) nxt_str_t chroot; #endif - uint32_t index; + uint32_t share_idx; uint8_t need_body; /* 1 bit */ } nxt_http_static_ctx_t; @@ -218,7 +218,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, conf = ctx->action->u.conf; - share = &conf->shares[ctx->index]; + share = &conf->shares[ctx->share_idx]; #if (NXT_DEBUG) nxt_str_t shr; @@ -245,7 +245,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_var_raw(share->var, &ctx->share); #if (NXT_HAVE_OPENAT2) - if (conf->chroot != NULL && ctx->index == 0) { + if (conf->chroot != NULL && ctx->share_idx == 0) { nxt_var_raw(conf->chroot, &ctx->chroot); } #endif @@ -262,7 +262,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_var_query(task, r->var_query, share->var, &ctx->share); #if (NXT_HAVE_OPENAT2) - if (conf->chroot != NULL && ctx->index == 0) { + if (conf->chroot != NULL && ctx->share_idx == 0) { nxt_var_query(task, r->var_query, conf->chroot, &ctx->chroot); } #endif @@ -357,7 +357,7 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) nxt_uint_t resolve; nxt_http_static_share_t *share; - share = &conf->shares[ctx->index]; + share = &conf->shares[ctx->share_idx]; resolve = conf->resolve; chr = &ctx->chroot; @@ -659,9 +659,9 @@ nxt_http_static_next(nxt_task_t *task, nxt_http_request_t *r, action = ctx->action; conf = action->u.conf; - ctx->index++; + ctx->share_idx++; - if (ctx->index < conf->nshares) { + if (ctx->share_idx < conf->nshares) { nxt_http_static_iterate(task, r, ctx); return; } -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:47 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:47 +0100 Subject: [PATCH v6 07/15] Tests: added tests for "index" option. In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-8-alx.manpages@gmail.com> From: Andrei Zeliankou Authored-by: Andrei Zeliankou Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Maxim Romanov --- test/test_static.py | 47 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/test/test_static.py b/test/test_static.py index 656bb7d..bb96f09 100644 --- a/test/test_static.py +++ b/test/test_static.py @@ -91,6 +91,53 @@ class TestStatic(TestApplicationProto): assert self.get(url='/')['body'] == '0123456789', 'before 1.26.0 2' def test_static_index(self): + def set_index(index): + assert 'success' in self.conf( + {"share": option.temp_dir + "/assets$uri", "index": index}, + 'routes/0/action', + ), 'configure index' + + set_index('README') + assert self.get()['body'] == 'readme', 'index' + + self.conf_delete('routes/0/action/index') + assert self.get()['body'] == '0123456789', 'delete index' + + set_index('$host') + assert ( + self.get(headers={"Host": "README", "Connection": "close"})['body'] + == 'readme' + ), 'index var' + self.get(headers={"Connection": "close"})[ + 'status' + ] == 404, 'index var empty' + + set_index([]) + self.get()['status'] == 404, 'index array empty' + + set_index(['blah']) + self.get()['status'] == 404, 'index array not found' + + set_index(['$host', 'blah', 'index.html']) + assert self.get()['body'] == '0123456789', 'index array' + assert ( + self.get(headers={"Host": "README", "Connection": "close"})['body'] + == 'readme' + ), 'index array 2' + + def test_static_index_invalid(self, skip_alert): + skip_alert(r'failed to apply new conf') + + def check_index(index): + assert 'error' in self.conf( + {"share": option.temp_dir + "/assets$uri", "index": index}, + 'routes/0/action', + ) + + check_index({}) + check_index(['index.html', '$blah']) + + def test_static_index_default(self): assert self.get(url='/index.html')['body'] == '0123456789', 'index' assert self.get(url='/')['body'] == '0123456789', 'index 2' assert self.get(url='//')['body'] == '0123456789', 'index 3' -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:54 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:54 +0100 Subject: [PATCH v6 14/15] Static: multiple paths in the "index" option In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-15-alx.manpages@gmail.com> This adds support for multiple paths in the "index" option. An example: { "share": "/srv/www/stat/static/$uri", "index": [ "$method.html", "alt.html" ] } When is requested, will be served if it exists; otherwise, will be served. === I also tried this feature in my own computer, where I tried the following: - Setting "index" to ["idontexist", "$host.html"], and check that is being served. - Not setting "index", and check that is being served. - Setting "index" to [], and check that it reports an error when loading the configuration. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- docs/changes.xml | 6 +++++ src/nxt_conf_validation.c | 51 +++++++++++++++++++++++++++++++++++++-- src/nxt_http_static.c | 38 ++++++++++++++++++++--------- test/test_static.py | 2 +- 4 files changed, 83 insertions(+), 14 deletions(-) diff --git a/docs/changes.xml b/docs/changes.xml index 7709fc8..18e180d 100644 --- a/docs/changes.xml +++ b/docs/changes.xml @@ -18,6 +18,12 @@ date="" time="" packager="Andrei Belov <defan at nginx.com>"> + + +multiple paths in the "index" option. + + + variables support in the "index" option. diff --git a/src/nxt_conf_validation.c b/src/nxt_conf_validation.c index 6236720..5b94116 100644 --- a/src/nxt_conf_validation.c +++ b/src/nxt_conf_validation.c @@ -117,6 +117,10 @@ static nxt_int_t nxt_conf_vldt_share(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, void *data); static nxt_int_t nxt_conf_vldt_share_element(nxt_conf_validation_t *vldt, nxt_conf_value_t *value); +static nxt_int_t nxt_conf_vldt_index(nxt_conf_validation_t *vldt, + nxt_conf_value_t *value, void *data); +static nxt_int_t nxt_conf_vldt_index_element(nxt_conf_validation_t *vldt, + nxt_conf_value_t *value); static nxt_int_t nxt_conf_vldt_proxy(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, void *data); static nxt_int_t nxt_conf_vldt_python(nxt_conf_validation_t *vldt, @@ -647,8 +651,8 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_share_action_members[] = { .validator = nxt_conf_vldt_share, }, { .name = nxt_string("index"), - .type = NXT_CONF_VLDT_STRING, - .flags = NXT_CONF_VLDT_VAR, + .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, + .validator = nxt_conf_vldt_index, }, { .name = nxt_string("types"), .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, @@ -1694,6 +1698,49 @@ nxt_conf_vldt_share_element(nxt_conf_validation_t *vldt, } +static nxt_int_t +nxt_conf_vldt_index(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, + void *data) +{ + if (nxt_conf_type(value) == NXT_CONF_ARRAY) { + if (nxt_conf_array_elements_count(value) == 0) { + return nxt_conf_vldt_error(vldt, "The \"index\" array " + "must contain at least one element."); + } + + return nxt_conf_vldt_array_iterator(vldt, value, + &nxt_conf_vldt_index_element); + } + + /* NXT_CONF_STRING */ + + return nxt_conf_vldt_index_element(vldt, value); +} + + +static nxt_int_t +nxt_conf_vldt_index_element(nxt_conf_validation_t *vldt, + nxt_conf_value_t *value) +{ + nxt_str_t str; + + static nxt_str_t index = nxt_string("index"); + + if (nxt_conf_type(value) != NXT_CONF_STRING) { + return nxt_conf_vldt_error(vldt, "The \"index\" array must " + "contain only string values."); + } + + nxt_conf_get_string(value, &str); + + if (nxt_is_var(&str)) { + return nxt_conf_vldt_var(vldt, &index, &str); + } + + return NXT_OK; +} + + static nxt_int_t nxt_conf_vldt_proxy(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, void *data) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index cbc8cbf..d6bb88d 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -18,8 +18,9 @@ typedef struct { typedef struct { nxt_uint_t nshares; + nxt_uint_t nindices; nxt_http_static_share_t *shares; - nxt_var_t *index; + nxt_var_t **indices; #if (NXT_HAVE_OPENAT2) nxt_var_t *chroot; nxt_uint_t resolve; @@ -36,6 +37,7 @@ typedef struct { nxt_str_t chroot; #endif uint32_t share_idx; + uint32_t index_idx; uint8_t need_body; /* 1 bit */ } nxt_http_static_ctx_t; @@ -123,14 +125,23 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, nxt_conf_set_string_dup(acf->index, mp, &default_index); } - nxt_conf_get_string(acf->index, &str); - - var = nxt_var_compile(&str, mp, 1); - if (nxt_slow_path(var == NULL)) { + conf->nindices = nxt_conf_array_elements_count(acf->index); + conf->indices = nxt_mp_zget(mp, sizeof(*conf->indices) * conf->nindices); + if (nxt_slow_path(conf->indices == NULL)) { return NXT_ERROR; } - conf->index = var; + for (i = 0; i < conf->nindices; i++) { + cv = nxt_conf_get_array_element(acf->index, i); + nxt_conf_get_string(cv, &str); + + var = nxt_var_compile(&str, mp, 1); + if (nxt_slow_path(var == NULL)) { + return NXT_ERROR; + } + + conf->indices[i] = var; + } #if (NXT_HAVE_OPENAT2) if (acf->chroot.length > 0) { @@ -235,6 +246,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_http_static_ctx_t *ctx) { nxt_int_t ret; + nxt_var_t *index; nxt_http_static_conf_t *conf; nxt_http_static_share_t *share; nxt_bool_t shr_is_const, idx_is_const; @@ -242,13 +254,14 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, conf = ctx->action->u.conf; share = &conf->shares[ctx->share_idx]; + index = conf->indices[ctx->index_idx]; #if (NXT_DEBUG) nxt_str_t shr; nxt_str_t idx; nxt_var_raw(share->var, &shr); - nxt_var_raw(conf->index, &idx); + nxt_var_raw(index, &idx); #if (NXT_HAVE_OPENAT2) nxt_str_t chr; @@ -268,7 +281,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, #endif /* NXT_DEBUG */ shr_is_const = share->is_const; - idx_is_const = nxt_var_is_const(conf->index); + idx_is_const = nxt_var_is_const(index); if (!shr_is_const || !idx_is_const) { ret = nxt_var_query_init(&r->var_query, r, r->mem_pool); @@ -279,10 +292,10 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, } if (idx_is_const) { - nxt_var_raw(conf->index, &ctx->index); + nxt_var_raw(index, &ctx->index); } else { - nxt_var_query(task, r->var_query, conf->index, &ctx->index); + nxt_var_query(task, r->var_query, index, &ctx->index); } if (shr_is_const) { @@ -700,7 +713,10 @@ nxt_http_static_next(nxt_task_t *task, nxt_http_request_t *r, action = ctx->action; conf = action->u.conf; - ctx->share_idx++; + ctx->index_idx = (ctx->index_idx + 1) % conf->nindices; + if (ctx->index_idx == 0) { + ctx->share_idx++; + } if (ctx->share_idx < conf->nshares) { nxt_http_static_iterate(task, r, ctx); diff --git a/test/test_static.py b/test/test_static.py index 0368af6..e1909cd 100644 --- a/test/test_static.py +++ b/test/test_static.py @@ -35,7 +35,7 @@ class TestStatic(TestApplicationProto): { "action": { "share": option.temp_dir + "/assets$uri", - "index": "$host.html" + "index": ["idontexist", "$host.html"] } } ], -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:50 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:50 +0100 Subject: [PATCH v6 10/15] nxt_http_static_iterate(): generalize code In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-11-alx.manpages@gmail.com> Separate into separate blocks of code the initialization of the var query, and the final action, to allow for a simpler patch that allows using variables for the 'index'. Add a nxt_bool_t shr_is_const variable to allow for a similar idx_is_const, and improve readability. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_http_static.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 0327861..2528b95 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -226,6 +226,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_int_t ret; nxt_http_static_conf_t *conf; nxt_http_static_share_t *share; + nxt_bool_t shr_is_const; conf = ctx->action->u.conf; @@ -255,9 +256,19 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, #endif #endif /* NXT_DEBUG */ + shr_is_const = share->is_const; + + if (!shr_is_const) { + ret = nxt_var_query_init(&r->var_query, r, r->mem_pool); + if (nxt_slow_path(ret != NXT_OK)) { + nxt_http_request_error(task, r, NXT_HTTP_INTERNAL_SERVER_ERROR); + return; + } + } + ctx->index = conf->index; - if (share->is_const) { + if (shr_is_const) { nxt_var_raw(share->var, &ctx->share); #if (NXT_HAVE_OPENAT2) @@ -266,15 +277,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, } #endif - nxt_http_static_send_ready(task, r, ctx); - } else { - ret = nxt_var_query_init(&r->var_query, r, r->mem_pool); - if (nxt_slow_path(ret != NXT_OK)) { - nxt_http_request_error(task, r, NXT_HTTP_INTERNAL_SERVER_ERROR); - return; - } - nxt_var_query(task, r->var_query, share->var, &ctx->share); #if (NXT_HAVE_OPENAT2) @@ -282,11 +285,16 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_var_query(task, r->var_query, conf->chroot, &ctx->chroot); } #endif + } + + if (shr_is_const) { + nxt_http_static_send_ready(task, r, ctx); + } else { nxt_var_query_resolve(task, r->var_query, ctx, nxt_http_static_send_ready, nxt_http_static_var_error); - } + } } -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:51 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:51 +0100 Subject: [PATCH v6 11/15] Static: variables in the "index" option In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-12-alx.manpages@gmail.com> This adds support for variables in the "index" option. An example: { "share": "/srv/www/data/static/$uri", "index": "$method.html" } When is requested, will be served. === I also tried this feature in my own computer, where I tried the following: - Setting "index" to "$method.html", and check that is being served. - Not setting index, and check that is being served. - Setting "index" to "$asdf", and check that the configuration fails: { "error": "Invalid configuration.", "detail": "Unknown variable \"asdf\" in the \"index\" value." } Tested-by: Alejandro Colomar Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- docs/changes.xml | 6 ++++++ src/nxt_conf_validation.c | 1 + src/nxt_http_static.c | 29 +++++++++++++++++++++-------- test/test_static.py | 11 ++++++----- 4 files changed, 34 insertions(+), 13 deletions(-) diff --git a/docs/changes.xml b/docs/changes.xml index 81708c3..7709fc8 100644 --- a/docs/changes.xml +++ b/docs/changes.xml @@ -18,6 +18,12 @@ date="" time="" packager="Andrei Belov <defan at nginx.com>"> + + +variables support in the "index" option. + + + new "index" option to specify a custom index file name. diff --git a/src/nxt_conf_validation.c b/src/nxt_conf_validation.c index bdc38b0..6236720 100644 --- a/src/nxt_conf_validation.c +++ b/src/nxt_conf_validation.c @@ -648,6 +648,7 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_share_action_members[] = { }, { .name = nxt_string("index"), .type = NXT_CONF_VLDT_STRING, + .flags = NXT_CONF_VLDT_VAR, }, { .name = nxt_string("types"), .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 2528b95..bee85b2 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -19,7 +19,7 @@ typedef struct { typedef struct { nxt_uint_t nshares; nxt_http_static_share_t *shares; - nxt_str_t index; + nxt_var_t *index; #if (NXT_HAVE_OPENAT2) nxt_var_t *chroot; nxt_uint_t resolve; @@ -115,12 +115,19 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, } if (acf->index == NULL) { - conf->index = default_index; + str = default_index; } else { - nxt_conf_get_string(acf->index, &conf->index); + nxt_conf_get_string(acf->index, &str); } + var = nxt_var_compile(&str, mp, 1); + if (nxt_slow_path(var == NULL)) { + return NXT_ERROR; + } + + conf->index = var; + #if (NXT_HAVE_OPENAT2) if (acf->chroot.length > 0) { nxt_str_t chr, shr; @@ -226,7 +233,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_int_t ret; nxt_http_static_conf_t *conf; nxt_http_static_share_t *share; - nxt_bool_t shr_is_const; + nxt_bool_t shr_is_const, idx_is_const; conf = ctx->action->u.conf; @@ -237,7 +244,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_str_t idx; nxt_var_raw(share->var, &shr); - idx = conf->index; + nxt_var_raw(conf->index, &idx); #if (NXT_HAVE_OPENAT2) nxt_str_t chr; @@ -257,8 +264,9 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, #endif /* NXT_DEBUG */ shr_is_const = share->is_const; + idx_is_const = nxt_var_is_const(conf->index); - if (!shr_is_const) { + if (!shr_is_const || !idx_is_const) { ret = nxt_var_query_init(&r->var_query, r, r->mem_pool); if (nxt_slow_path(ret != NXT_OK)) { nxt_http_request_error(task, r, NXT_HTTP_INTERNAL_SERVER_ERROR); @@ -266,7 +274,12 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, } } - ctx->index = conf->index; + if (idx_is_const) { + nxt_var_raw(conf->index, &ctx->index); + + } else { + nxt_var_query(task, r->var_query, conf->index, &ctx->index); + } if (shr_is_const) { nxt_var_raw(share->var, &ctx->share); @@ -287,7 +300,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, #endif } - if (shr_is_const) { + if (shr_is_const && idx_is_const) { nxt_http_static_send_ready(task, r, ctx); } else { diff --git a/test/test_static.py b/test/test_static.py index 9eea445..0368af6 100644 --- a/test/test_static.py +++ b/test/test_static.py @@ -15,7 +15,8 @@ class TestStatic(TestApplicationProto): def setup_method(self): os.makedirs(option.temp_dir + '/assets/dir') - with open(option.temp_dir + '/assets/index_.html', 'w') as index, open( + with open(option.temp_dir + '/assets/localhost.html', 'w' + ) as index, open( option.temp_dir + '/assets/README', 'w' ) as readme, open( option.temp_dir + '/assets/log.log', 'w' @@ -34,7 +35,7 @@ class TestStatic(TestApplicationProto): { "action": { "share": option.temp_dir + "/assets$uri", - "index": "index_.html" + "index": "$host.html" } } ], @@ -148,7 +149,7 @@ class TestStatic(TestApplicationProto): assert self.get(url='/#blah')['body'] == '0123456789', 'index anchor' assert self.get(url='/dir/')['status'] == 404, 'index not found' - resp = self.get(url='/index_.html/') + resp = self.get(url='/localhost.html/') assert resp['status'] == 404, 'index not found 2 status' assert ( resp['headers']['Content-Type'] == 'text/html' @@ -172,7 +173,7 @@ class TestStatic(TestApplicationProto): assert etag != etag_2, 'different ETag' assert etag == self.get(url='/')['headers']['ETag'], 'same ETag' - with open(temp_dir + '/assets/index_.html', 'w') as f: + with open(temp_dir + '/assets/localhost.html', 'w') as f: f.write('blah') assert etag != self.get(url='/')['headers']['ETag'], 'new ETag' @@ -311,7 +312,7 @@ class TestStatic(TestApplicationProto): == 'text/x-code/x-blah/x-blah' ), 'mime_types string case insensitive' assert ( - self.get(url='/index_.html')['headers']['Content-Type'] + self.get(url='/localhost.html')['headers']['Content-Type'] == 'text/plain' ), 'mime_types html' assert ( -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:52 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:52 +0100 Subject: [PATCH v6 12/15] nxt_conf_set_string_dup(): Use 'const' for read-only parameter In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-13-alx.manpages@gmail.com> That parameter is not being modified in the function. Make it 'const' to allow passing 'static const' variables. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_conf.c | 3 ++- src/nxt_conf.h | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/src/nxt_conf.c b/src/nxt_conf.c index 3f0cfa8..967c455 100644 --- a/src/nxt_conf.c +++ b/src/nxt_conf.c @@ -192,7 +192,8 @@ nxt_conf_set_string(nxt_conf_value_t *value, nxt_str_t *str) nxt_int_t -nxt_conf_set_string_dup(nxt_conf_value_t *value, nxt_mp_t *mp, nxt_str_t *str) +nxt_conf_set_string_dup(nxt_conf_value_t *value, nxt_mp_t *mp, + const nxt_str_t *str) { nxt_str_t tmp, *ptr; diff --git a/src/nxt_conf.h b/src/nxt_conf.h index 8b3565f..09f2175 100644 --- a/src/nxt_conf.h +++ b/src/nxt_conf.h @@ -115,7 +115,7 @@ nxt_int_t nxt_conf_validate(nxt_conf_validation_t *vldt); NXT_EXPORT void nxt_conf_get_string(nxt_conf_value_t *value, nxt_str_t *str); NXT_EXPORT void nxt_conf_set_string(nxt_conf_value_t *value, nxt_str_t *str); NXT_EXPORT nxt_int_t nxt_conf_set_string_dup(nxt_conf_value_t *value, - nxt_mp_t *mp, nxt_str_t *str); + nxt_mp_t *mp, const nxt_str_t *str); NXT_EXPORT double nxt_conf_get_number(nxt_conf_value_t *value); NXT_EXPORT uint8_t nxt_conf_get_boolean(nxt_conf_value_t *value); -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:55 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:55 +0100 Subject: [PATCH v6 15/15] Static: optimize "index" iteration In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-16-alx.manpages@gmail.com> Iterate over "index" only if "share" is a directory. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_http_static.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index d6bb88d..74245cd 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -713,7 +713,10 @@ nxt_http_static_next(nxt_task_t *task, nxt_http_request_t *r, action = ctx->action; conf = action->u.conf; - ctx->index_idx = (ctx->index_idx + 1) % conf->nindices; + if (ctx->share.start[ctx->share.length - 1] == '/') { + ctx->index_idx = (ctx->index_idx + 1) % conf->nindices; + } + if (ctx->index_idx == 0) { ctx->share_idx++; } -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 14:57:53 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 15:57:53 +0100 Subject: [PATCH v6 13/15] nxt_http_static_init(): Simplify branching In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <20220215145755.40239-14-alx.manpages@gmail.com> Add a bit of code to the special case of no "index", so that the normal case is not in a branch. This makes the normal flow more readable, by hiding details of special cases in branches, even if that means making the special case a bit more complex than it needs to be. 'else' distracts the mental parser of the reader of the code: it means it needs to branch in both cases. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_http_static.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index bee85b2..cbc8cbf 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -115,12 +115,16 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, } if (acf->index == NULL) { - str = default_index; + acf->index = nxt_conf_create_object(mp, 1); + if (nxt_slow_path(acf->index == NULL)) { + return NXT_ERROR; + } - } else { - nxt_conf_get_string(acf->index, &str); + nxt_conf_set_string_dup(acf->index, mp, &default_index); } + nxt_conf_get_string(acf->index, &str); + var = nxt_var_compile(&str, mp, 1); if (nxt_slow_path(var == NULL)) { return NXT_ERROR; -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:01:51 2022 From: alx.manpages at gmail.com (Alejandro Colomar (man-pages)) Date: Tue, 15 Feb 2022 16:01:51 +0100 Subject: Failing tests (Re: [PATCH v6 00/15] Static: Implement new "index" option) In-Reply-To: <20220215145755.40239-1-alx.manpages@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> Message-ID: <8aaa3889-7c4a-2515-0647-b0720287c680@gmail.com> On 2/15/22 15:57, Alejandro Colomar wrote: > Changes: > > v2: > - Add support for variables in "index" > - Don't move share_idx, since that caused unnecessary padding. > - Use > > v3 : > - Don't use (as previously discussed). > - Add a .gitignore: > While doing this work, I had quite some accidents. This > prevented them from happening again. I based it on the > .gitignore of the Linux man-pages > > . > - Treat non-arrays as arrays of size 1 > (as discussed earlier today: > ). > - Simplify all of the code that can be simplified thanks to the > change mentioned above (arrays of size 1). > - Fix the indentation of some random lines that I found to have an > incorrect indentation. > - Reduce branching (especially 'else's), to improve readability. > I didn't merge this change into the patch that adds the feature > because there it adds too much complexity for nothing, and the > patches read better separately. However, I patched it previous > to the third step to simplify that patch too, so this change > sits on a patch of its own better, I think. > - Use const for a read-only parameter of a function. I didn't > change other functions (not even in the same file), since that > would mean a huuuge diff, and for consistency we would need to > do it everywhere. So I only did it where I *needed* it to > compile, for now. > > v4 : > - Actually iterate. > > v5 : > - Remove spurious 'else' [Valentin] > > v6: > - Add Andrei's tests (and remove my changes to the existing tests). > - Fix bugs reported by Andrei: > - Reject []. > - Don't return 301 for a directory (maybe I should add a check > that the string can't be empty; do we want that?) > - Fix non-compiling debug code. > > > Alejandro Colomar (14): > .gitignore: ignore new files > nxt_conf.c: Treat non-arrays as arrays of size 1 > Remove special cases for non-NXT_CONF_VALUE_ARRAY > Fix indentation > nxt_http_static_ctx_t: rename field 'index' to 'share_idx' > test_static.py: formatting fix > Static: return 404 when "index" is a non-regular file > Static: add "index" option > nxt_http_static_iterate(): generalize code > Static: variables in the "index" option > nxt_conf_set_string_dup(): Use 'const' for read-only parameter > nxt_http_static_init(): Simplify branching > Static: multiple paths in the "index" option > Static: optimize "index" iteration > > Andrei Zeliankou (1): > Tests: added tests for "index" option. > I forgot to write in this cover letter: Some of Andrei's tests fail. I'm not sure why, since I'm not used to them yet. Please, Andrei, help me with that. Also, as I pointed out in the RFC I sent a few minutes ago, the fix for the 301 error also makes some existing tests fail, and I also don't know why. Thanks, Alex -- Alejandro Colomar Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/ http://www.alejandro-colomar.es/ From alx.manpages at gmail.com Tue Feb 15 15:10:53 2022 From: alx.manpages at gmail.com (Alejandro Colomar (man-pages)) Date: Tue, 15 Feb 2022 16:10:53 +0100 Subject: Failing tests (Re: [PATCH v6 00/15] Static: Implement new "index" option) In-Reply-To: <8aaa3889-7c4a-2515-0647-b0720287c680@gmail.com> References: <20220215145755.40239-1-alx.manpages@gmail.com> <8aaa3889-7c4a-2515-0647-b0720287c680@gmail.com> Message-ID: <6f27e380-0f74-8d88-42a0-b32e1815a513@gmail.com> On 2/15/22 16:01, Alejandro Colomar (man-pages) wrote: >> v6: >> - Add Andrei's tests (and remove my changes to the existing tests). Oops, I didn't remove all of my changes. I'll fix that. -- Alejandro Colomar Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/ http://www.alejandro-colomar.es/ From alx.manpages at gmail.com Tue Feb 15 15:25:10 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:10 +0100 Subject: [PATCH v7 00/14] Static: Implement new "index" option Message-ID: <20220215152524.41119-1-alx.manpages@gmail.com> Some pytests are failing (both existing ones and new ones), but the existing ones I don't understand them very well, and the new ones may be incorrect, since the manual tests I performed work correctly, AFAICT. See patch 07/14 to see which existing tests are failing and why. Changes: v2: - Add support for variables in "index" - Don't move share_idx, since that caused unnecessary padding. - Use v3 : - Don't use (as previously discussed). - Add a .gitignore: While doing this work, I had quite some accidents. This prevented them from happening again. I based it on the .gitignore of the Linux man-pages . - Treat non-arrays as arrays of size 1 (as discussed earlier today: ). - Simplify all of the code that can be simplified thanks to the change mentioned above (arrays of size 1). - Fix the indentation of some random lines that I found to have an incorrect indentation. - Reduce branching (especially 'else's), to improve readability. I didn't merge this change into the patch that adds the feature because there it adds too much complexity for nothing, and the patches read better separately. However, I patched it previous to the third step to simplify that patch too, so this change sits on a patch of its own better, I think. - Use const for a read-only parameter of a function. I didn't change other functions (not even in the same file), since that would mean a huuuge diff, and for consistency we would need to do it everywhere. So I only did it where I *needed* it to compile, for now. v4 : - Actually iterate. v5 : - Remove spurious 'else' [Valentin] v6: - Add Andrei's tests (and remove my changes to the existing tests). - Fix bugs reported by Andrei: - Reject []. - Don't return 301 for a directory (maybe I should add a check that the string can't be empty; do we want that?) - Fix non-compiling debug code. v7: - Remove all of my changes to . Alejandro Colomar (13): .gitignore: ignore new files nxt_conf.c: Treat non-arrays as arrays of size 1 Remove special cases for non-NXT_CONF_VALUE_ARRAY Fix indentation nxt_http_static_ctx_t: rename field 'index' to 'share_idx' Static: return 404 when "index" is a non-regular file Static: add "index" option nxt_http_static_iterate(): generalize code Static: variables in the "index" option nxt_conf_set_string_dup(): Use 'const' for read-only parameter nxt_http_static_init(): Simplify branching Static: multiple paths in the "index" option Static: optimize "index" iteration Andrei Zeliankou (1): Tests: added tests for "index" option. .gitignore | 2 + docs/changes.xml | 18 +++++ src/nxt_conf.c | 7 +- src/nxt_conf.h | 2 +- src/nxt_conf_validation.c | 55 +++++++++++++++- src/nxt_gnutls.c | 4 +- src/nxt_http.h | 15 +++-- src/nxt_http_route.c | 54 +++------------ src/nxt_http_static.c | 134 +++++++++++++++++++++++++------------- src/nxt_main_process.c | 2 +- src/nxt_openssl.c | 24 ++----- src/nxt_router.c | 23 ++----- src/nxt_time_parse.c | 6 +- src/nxt_unit.c | 4 +- src/nxt_work_queue.h | 4 +- src/perl/nxt_perl_psgi.c | 2 +- src/python/nxt_python.c | 9 --- test/test_static.py | 47 +++++++++++++ 18 files changed, 255 insertions(+), 157 deletions(-) create mode 100644 .gitignore -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:11 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:11 +0100 Subject: [PATCH v7 01/14] .gitignore: ignore new files In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-2-alx.manpages@gmail.com> When working with git, one has to be very careful of not adding temporary files resulting from compilation, testing, or text editors. With this .gitignore, one can add all of the changes with `git add .` without having to specify the files manually, avoiding accidents. When a new file is created and should really be added, one can add it still with the '-f' option to git-add(1). Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- .gitignore | 2 ++ 1 file changed, 2 insertions(+) create mode 100644 .gitignore diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..ed833f0 --- /dev/null +++ b/.gitignore @@ -0,0 +1,2 @@ +# Ignore everything new by default +/* -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:12 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:12 +0100 Subject: [PATCH v7 02/14] nxt_conf.c: Treat non-arrays as arrays of size 1 In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-3-alx.manpages@gmail.com> Similar to how C pointers to variables can always be considered as pointers to the first element of an array of size 1 (see the following code for an example of how they are equivalent), treating non-NXT_CONF_VALUE_ARRAY as if they were NXT_CONF_VALUE_ARRAYs of size 1 allows for simpler and more generic code. void foo(ptrdiff_t sz, int arr[sz]) { for (ptrdiff_t i = 0; i < sz; i++) arr[i] = 0; } void bar(void) { int x; int y[1]; foo(1, &x); foo(1, y); } nxt_conf_array_elements_count(): Return a size of 1 when input is non-array, instead of causing undefined behavior. That value (1) makes sense because it will be used as the limiter of a loop that loops over the array and calls nxt_conf_get_array_element(), which will return a correct element for such loops. nxt_conf_get_array_element(): Return the input pointer unmodified (i.e., a pointer to the unique element of a hypothetical array), instead of returning NULL, which wasn't very useful. nxt_conf_array_qsort(): Since it's a no-op for non-arrays, this doesn't need to be modified. Example of (correct) usage: void baz(nxt_conf_value_t *x) { /* 'x' may or may not be a NXT_CONF_VALUE_ARRAY */ ptrdiff_t n; nxt_conf_value_t *elem; n = nxt_conf_array_elements_count(x); for (ptrdiff_t i = 0; i < n; i++) { elem = nxt_conf_get_array_element(x, i); /* do some work with 'elem' */ } } Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_conf.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/nxt_conf.c b/src/nxt_conf.c index 1aca0a7..3f0cfa8 100644 --- a/src/nxt_conf.c +++ b/src/nxt_conf.c @@ -387,7 +387,7 @@ nxt_conf_set_element_string_dup(nxt_conf_value_t *array, nxt_mp_t *mp, nxt_uint_t nxt_conf_array_elements_count(nxt_conf_value_t *value) { - return value->u.array->count; + return (value->type == NXT_CONF_VALUE_ARRAY) ? value->u.array->count : 1; } @@ -736,7 +736,7 @@ nxt_conf_get_array_element(nxt_conf_value_t *value, uint32_t index) nxt_conf_array_t *array; if (value->type != NXT_CONF_VALUE_ARRAY) { - return NULL; + return (index == 0) ? value : NULL; } array = value->u.array; -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:14 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:14 +0100 Subject: [PATCH v7 04/14] Fix indentation In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-5-alx.manpages@gmail.com> Some lines (incorrectly) had an indenteation of 5 or 9 spaces instead of 4 or 8. Fix them. Found with: $ ## 5 spaces instead of 4: $ $ find src/ -type f \ |grep -n '^ [^ *]'; $ ## 9 spaces instead of 8: $ $ find src/ -type f \ |grep -l '^ [^ *]' \ |xargs grep -L '^ [^ *].*)$' \ |xargs grep -n '^ [^ *]' Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_conf_validation.c | 4 ++-- src/nxt_gnutls.c | 4 ++-- src/nxt_http.h | 14 +++++++------- src/nxt_http_route.c | 4 ++-- src/nxt_main_process.c | 2 +- src/nxt_time_parse.c | 6 +++--- src/nxt_unit.c | 4 ++-- src/nxt_work_queue.h | 4 ++-- src/perl/nxt_perl_psgi.c | 2 +- 9 files changed, 22 insertions(+), 22 deletions(-) diff --git a/src/nxt_conf_validation.c b/src/nxt_conf_validation.c index 3f068bb..7edd638 100644 --- a/src/nxt_conf_validation.c +++ b/src/nxt_conf_validation.c @@ -114,7 +114,7 @@ static nxt_int_t nxt_conf_vldt_pass(nxt_conf_validation_t *vldt, static nxt_int_t nxt_conf_vldt_return(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, void *data); static nxt_int_t nxt_conf_vldt_share(nxt_conf_validation_t *vldt, - nxt_conf_value_t *value, void *data); + nxt_conf_value_t *value, void *data); static nxt_int_t nxt_conf_vldt_share_element(nxt_conf_validation_t *vldt, nxt_conf_value_t *value); static nxt_int_t nxt_conf_vldt_proxy(nxt_conf_validation_t *vldt, @@ -194,7 +194,7 @@ static nxt_int_t nxt_conf_vldt_java_classpath(nxt_conf_validation_t *vldt, static nxt_int_t nxt_conf_vldt_java_option(nxt_conf_validation_t *vldt, nxt_conf_value_t *value); static nxt_int_t nxt_conf_vldt_upstream(nxt_conf_validation_t *vldt, - nxt_str_t *name, nxt_conf_value_t *value); + nxt_str_t *name, nxt_conf_value_t *value); static nxt_int_t nxt_conf_vldt_server(nxt_conf_validation_t *vldt, nxt_str_t *name, nxt_conf_value_t *value); static nxt_int_t nxt_conf_vldt_server_weight(nxt_conf_validation_t *vldt, diff --git a/src/nxt_gnutls.c b/src/nxt_gnutls.c index 4618dce..aab4699 100644 --- a/src/nxt_gnutls.c +++ b/src/nxt_gnutls.c @@ -708,11 +708,11 @@ nxt_gnutls_log_error_level(nxt_event_conn_t *c, ssize_t err) case GNUTLS_E_UNKNOWN_CIPHER_SUITE: /* -21 */ - /* Disable gnutls_bye(), because it returns GNUTLS_E_INTERNAL_ERROR. */ + /* Disable gnutls_bye(), because it returns GNUTLS_E_INTERNAL_ERROR. */ ssltls = c->u.ssltls; ssltls->no_shutdown = 1; - /* Fall through. */ + /* Fall through. */ case GNUTLS_E_UNEXPECTED_PACKET_LENGTH: /* -9 */ c->socket.error = 1000; /* Nonexistent errno code. */ diff --git a/src/nxt_http.h b/src/nxt_http.h index 02d66f5..7cef134 100644 --- a/src/nxt_http.h +++ b/src/nxt_http.h @@ -90,17 +90,17 @@ typedef union { #define nxt_http_field_name_set(_field, _name) \ do { \ - (_field)->name_length = nxt_length(_name); \ - (_field)->name = (u_char *) _name; \ + (_field)->name_length = nxt_length(_name); \ + (_field)->name = (u_char *) _name; \ } while (0) #define nxt_http_field_set(_field, _name, _value) \ do { \ - (_field)->name_length = nxt_length(_name); \ - (_field)->value_length = nxt_length(_value); \ - (_field)->name = (u_char *) _name; \ - (_field)->value = (u_char *) _value; \ + (_field)->name_length = nxt_length(_name); \ + (_field)->value_length = nxt_length(_value); \ + (_field)->name = (u_char *) _name; \ + (_field)->value = (u_char *) _value; \ } while (0) @@ -238,7 +238,7 @@ typedef struct { void (*body_read)(nxt_task_t *task, nxt_http_request_t *r); void (*local_addr)(nxt_task_t *task, nxt_http_request_t *r); void (*header_send)(nxt_task_t *task, nxt_http_request_t *r, - nxt_work_handler_t body_handler, void *data); + nxt_work_handler_t body_handler, void *data); void (*send)(nxt_task_t *task, nxt_http_request_t *r, nxt_buf_t *out); nxt_off_t (*body_bytes_sent)(nxt_task_t *task, nxt_http_proto_t proto); void (*discard)(nxt_task_t *task, nxt_http_request_t *r, nxt_buf_t *last); diff --git a/src/nxt_http_route.c b/src/nxt_http_route.c index 71ebf3f..26f0662 100644 --- a/src/nxt_http_route.c +++ b/src/nxt_http_route.c @@ -932,7 +932,7 @@ nxt_http_route_rule_create(nxt_task_t *task, nxt_mp_t *mp, nxt_http_route_addr_rule_t * nxt_http_route_addr_rule_create(nxt_task_t *task, nxt_mp_t *mp, - nxt_conf_value_t *cv) + nxt_conf_value_t *cv) { size_t size; uint32_t i, n; @@ -2294,7 +2294,7 @@ nxt_http_route_cookie_parse(nxt_array_t *cookies, u_char *start, u_char *end) name = NULL; start = p + 1; - } + } } if (name != NULL) { diff --git a/src/nxt_main_process.c b/src/nxt_main_process.c index 9883f04..03761a1 100644 --- a/src/nxt_main_process.c +++ b/src/nxt_main_process.c @@ -579,7 +579,7 @@ nxt_main_process_created_handler(nxt_task_t *task, nxt_port_recv_msg_t *msg) -1, msg->port_msg.stream, 0, NULL); return; } - } + } #endif diff --git a/src/nxt_time_parse.c b/src/nxt_time_parse.c index 5a32b21..94c4328 100644 --- a/src/nxt_time_parse.c +++ b/src/nxt_time_parse.c @@ -282,9 +282,9 @@ nxt_time_parse(const u_char *p, size_t len) days = days - 719527 + 31 + 28; s = (uint64_t) days * 86400 - + (nxt_uint_t) hour * 3600 - + (nxt_uint_t) min * 60 - + (nxt_uint_t) sec; + + (nxt_uint_t) hour * 3600 + + (nxt_uint_t) min * 60 + + (nxt_uint_t) sec; #if (NXT_TIME_T_SIZE <= 4) diff --git a/src/nxt_unit.c b/src/nxt_unit.c index dac612b..32bb07a 100644 --- a/src/nxt_unit.c +++ b/src/nxt_unit.c @@ -5114,9 +5114,9 @@ nxt_unit_ctx_alloc(nxt_unit_ctx_t *ctx, void *data) rc = nxt_unit_ctx_init(lib, new_ctx, data); if (nxt_slow_path(rc != NXT_UNIT_OK)) { - nxt_unit_free(ctx, new_ctx); + nxt_unit_free(ctx, new_ctx); - return NULL; + return NULL; } queue_fd = -1; diff --git a/src/nxt_work_queue.h b/src/nxt_work_queue.h index 6c2d6c2..ffa21d2 100644 --- a/src/nxt_work_queue.h +++ b/src/nxt_work_queue.h @@ -16,12 +16,12 @@ struct nxt_task_s { uint32_t ident; nxt_work_t *next_work; - /* TODO: exception_handler, prev/next task, subtasks. */ + /* TODO: exception_handler, prev/next task, subtasks. */ }; #define nxt_task_next_ident() \ - ((uint32_t) nxt_atomic_fetch_add(&nxt_task_ident, 1) & 0x3FFFFFFF) + ((uint32_t) nxt_atomic_fetch_add(&nxt_task_ident, 1) & 0x3FFFFFFF) /* diff --git a/src/perl/nxt_perl_psgi.c b/src/perl/nxt_perl_psgi.c index 749ebd8..08a6f29 100644 --- a/src/perl/nxt_perl_psgi.c +++ b/src/perl/nxt_perl_psgi.c @@ -612,7 +612,7 @@ nxt_perl_psgi_env_create(PerlInterpreter *my_perl, do { \ if (nxt_slow_path((FNS) != NXT_UNIT_OK)) \ goto fail; \ - } while (0) + } while (0) #define NL(S) (S), sizeof(S)-1 -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:15 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:15 +0100 Subject: [PATCH v7 05/14] nxt_http_static_ctx_t: rename field 'index' to 'share_idx' In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-6-alx.manpages@gmail.com> Having a configurable index filename will require adding an index field to this structure. The most natural name for that field is 'index', so the current index field should be renamed to allow for that. A sensible name is 'share_idx', since it's the index of the shares array in 'nxt_http_static_conf_t'. Instead of 'share_index' I opted for the shorter 'share_idx'. Also, when 'index' allows an array of filenames in a following commit, another similar variable 'index_idx' should be created, and having a different prefix and suffix seems more readable than for example 'index_index'. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_http_static.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 6b7cee7..663f896 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -33,7 +33,7 @@ typedef struct { #if (NXT_HAVE_OPENAT2) nxt_str_t chroot; #endif - uint32_t index; + uint32_t share_idx; uint8_t need_body; /* 1 bit */ } nxt_http_static_ctx_t; @@ -218,7 +218,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, conf = ctx->action->u.conf; - share = &conf->shares[ctx->index]; + share = &conf->shares[ctx->share_idx]; #if (NXT_DEBUG) nxt_str_t shr; @@ -245,7 +245,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_var_raw(share->var, &ctx->share); #if (NXT_HAVE_OPENAT2) - if (conf->chroot != NULL && ctx->index == 0) { + if (conf->chroot != NULL && ctx->share_idx == 0) { nxt_var_raw(conf->chroot, &ctx->chroot); } #endif @@ -262,7 +262,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_var_query(task, r->var_query, share->var, &ctx->share); #if (NXT_HAVE_OPENAT2) - if (conf->chroot != NULL && ctx->index == 0) { + if (conf->chroot != NULL && ctx->share_idx == 0) { nxt_var_query(task, r->var_query, conf->chroot, &ctx->chroot); } #endif @@ -357,7 +357,7 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) nxt_uint_t resolve; nxt_http_static_share_t *share; - share = &conf->shares[ctx->index]; + share = &conf->shares[ctx->share_idx]; resolve = conf->resolve; chr = &ctx->chroot; @@ -659,9 +659,9 @@ nxt_http_static_next(nxt_task_t *task, nxt_http_request_t *r, action = ctx->action; conf = action->u.conf; - ctx->index++; + ctx->share_idx++; - if (ctx->index < conf->nshares) { + if (ctx->share_idx < conf->nshares) { nxt_http_static_iterate(task, r, ctx); return; } -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:16 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:16 +0100 Subject: [PATCH v7 06/14] Tests: added tests for "index" option. In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-7-alx.manpages@gmail.com> From: Andrei Zeliankou Authored-by: Andrei Zeliankou Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Maxim Romanov --- test/test_static.py | 47 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) diff --git a/test/test_static.py b/test/test_static.py index 1d9c754..7d77752 100644 --- a/test/test_static.py +++ b/test/test_static.py @@ -87,6 +87,53 @@ class TestStatic(TestApplicationProto): assert self.get(url='/')['body'] == '0123456789', 'before 1.26.0 2' def test_static_index(self): + def set_index(index): + assert 'success' in self.conf( + {"share": option.temp_dir + "/assets$uri", "index": index}, + 'routes/0/action', + ), 'configure index' + + set_index('README') + assert self.get()['body'] == 'readme', 'index' + + self.conf_delete('routes/0/action/index') + assert self.get()['body'] == '0123456789', 'delete index' + + set_index('$host') + assert ( + self.get(headers={"Host": "README", "Connection": "close"})['body'] + == 'readme' + ), 'index var' + self.get(headers={"Connection": "close"})[ + 'status' + ] == 404, 'index var empty' + + set_index([]) + self.get()['status'] == 404, 'index array empty' + + set_index(['blah']) + self.get()['status'] == 404, 'index array not found' + + set_index(['$host', 'blah', 'index.html']) + assert self.get()['body'] == '0123456789', 'index array' + assert ( + self.get(headers={"Host": "README", "Connection": "close"})['body'] + == 'readme' + ), 'index array 2' + + def test_static_index_invalid(self, skip_alert): + skip_alert(r'failed to apply new conf') + + def check_index(index): + assert 'error' in self.conf( + {"share": option.temp_dir + "/assets$uri", "index": index}, + 'routes/0/action', + ) + + check_index({}) + check_index(['index.html', '$blah']) + + def test_static_index_default(self): assert self.get(url='/index.html')['body'] == '0123456789', 'index' assert self.get(url='/')['body'] == '0123456789', 'index 2' assert self.get(url='//')['body'] == '0123456789', 'index 3' -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:18 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:18 +0100 Subject: [PATCH v7 08/14] Static: add "index" option In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-9-alx.manpages@gmail.com> This supports a new option "index", that configures a custom index file name that will be served when a directory is requested. This initial support only allows a single fixed string. An example: { "share": "/www/data/static/$uri", "index": "lookatthis.htm" } When is requested, will be served. If the option is missing, default to "index.html". === nxt_conf_validator.c: Accept "index" as a member of "share", and make sure it is a string. === I tried this feature in my own computer, where I tried the following: - Setting "index" to "lookatthis.htm", and check that the correct file is being served (check both a different name and a different extension). - Not setting "index", and check that is being served. - Settind "index" to an array of strings, and check that the configuration fails: { "error": "Invalid configuration.", "detail": "The \"index\" value must be a string, but not an array." } Tested-by: Alejandro Colomar Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Maxim Romanov --- docs/changes.xml | 6 ++++++ src/nxt_conf_validation.c | 3 +++ src/nxt_http.h | 1 + src/nxt_http_route.c | 5 +++++ src/nxt_http_static.c | 32 +++++++++++++++++++++++--------- 5 files changed, 38 insertions(+), 9 deletions(-) diff --git a/docs/changes.xml b/docs/changes.xml index a6b1178..81708c3 100644 --- a/docs/changes.xml +++ b/docs/changes.xml @@ -18,6 +18,12 @@ date="" time="" packager="Andrei Belov <defan at nginx.com>"> + + +new "index" option to specify a custom index file name. + + + NGINX Unit updated to 1.27.0. diff --git a/src/nxt_conf_validation.c b/src/nxt_conf_validation.c index 7edd638..bdc38b0 100644 --- a/src/nxt_conf_validation.c +++ b/src/nxt_conf_validation.c @@ -645,6 +645,9 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_share_action_members[] = { .name = nxt_string("share"), .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, .validator = nxt_conf_vldt_share, + }, { + .name = nxt_string("index"), + .type = NXT_CONF_VLDT_STRING, }, { .name = nxt_string("types"), .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, diff --git a/src/nxt_http.h b/src/nxt_http.h index 7cef134..9b819fb 100644 --- a/src/nxt_http.h +++ b/src/nxt_http.h @@ -209,6 +209,7 @@ typedef struct { nxt_str_t location; nxt_conf_value_t *proxy; nxt_conf_value_t *share; + nxt_conf_value_t *index; nxt_str_t chroot; nxt_conf_value_t *follow_symlinks; nxt_conf_value_t *traverse_mounts; diff --git a/src/nxt_http_route.c b/src/nxt_http_route.c index 26f0662..c92f2cd 100644 --- a/src/nxt_http_route.c +++ b/src/nxt_http_route.c @@ -639,6 +639,11 @@ static nxt_conf_map_t nxt_http_route_action_conf[] = { NXT_CONF_MAP_PTR, offsetof(nxt_http_action_conf_t, share) }, + { + nxt_string("index"), + NXT_CONF_MAP_PTR, + offsetof(nxt_http_action_conf_t, index) + }, { nxt_string("chroot"), NXT_CONF_MAP_STR, diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 7f1254c..0327861 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -19,6 +19,7 @@ typedef struct { typedef struct { nxt_uint_t nshares; nxt_http_static_share_t *shares; + nxt_str_t index; #if (NXT_HAVE_OPENAT2) nxt_var_t *chroot; nxt_uint_t resolve; @@ -30,6 +31,7 @@ typedef struct { typedef struct { nxt_http_action_t *action; nxt_str_t share; + nxt_str_t index; #if (NXT_HAVE_OPENAT2) nxt_str_t chroot; #endif @@ -80,6 +82,8 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, nxt_conf_value_t *cv; nxt_http_static_conf_t *conf; + static const nxt_str_t default_index = nxt_string("index.html"); + mp = tmcf->router_conf->mem_pool; conf = nxt_mp_zget(mp, sizeof(nxt_http_static_conf_t)); @@ -110,6 +114,13 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, conf->shares[i].is_const = nxt_var_is_const(var); } + if (acf->index == NULL) { + conf->index = default_index; + + } else { + nxt_conf_get_string(acf->index, &conf->index); + } + #if (NXT_HAVE_OPENAT2) if (acf->chroot.length > 0) { nxt_str_t chr, shr; @@ -222,8 +233,10 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, #if (NXT_DEBUG) nxt_str_t shr; + nxt_str_t idx; nxt_var_raw(share->var, &shr); + idx = conf->index; #if (NXT_HAVE_OPENAT2) nxt_str_t chr; @@ -235,12 +248,15 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_str_set(&chr, ""); } - nxt_debug(task, "http static: \"%V\" (chroot: \"%V\")", &shr, &chr); + nxt_debug(task, "http static: \"%V\", index: \"%V\" (chroot: \"%V\")", + &shr, &idx, &chr); #else - nxt_debug(task, "http static: \"%V\"", &shr); + nxt_debug(task, "http static: \"%V\", index: \"%V\"", &shr, &idx); #endif #endif /* NXT_DEBUG */ + ctx->index = conf->index; + if (share->is_const) { nxt_var_raw(share->var, &ctx->share); @@ -282,7 +298,7 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) struct tm tm; nxt_buf_t *fb; nxt_int_t ret; - nxt_str_t *shr, exten, *mtype; + nxt_str_t *shr, *index, exten, *mtype; nxt_uint_t level; nxt_file_t *f, file; nxt_file_info_t fi; @@ -295,8 +311,6 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) nxt_http_static_ctx_t *ctx; nxt_http_static_conf_t *conf; - static const nxt_str_t index = nxt_string("index.html"); - r = obj; ctx = data; action = ctx->action; @@ -307,12 +321,12 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) mtype = NULL; shr = &ctx->share; + index = &ctx->index; if (shr->start[shr->length - 1] == '/') { - /* TODO: dynamic index setting. */ - nxt_str_set(&exten, ".html"); + nxt_http_static_extract_extension(index, &exten); - length = shr->length + index.length; + length = shr->length + index->length; fname = nxt_mp_nget(r->mem_pool, length + 1); if (nxt_slow_path(fname == NULL)) { @@ -321,7 +335,7 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) p = fname; p = nxt_cpymem(p, shr->start, shr->length); - p = nxt_cpymem(p, index.start, index.length); + p = nxt_cpymem(p, index->start, index->length); *p = '\0'; } else { -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:13 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:13 +0100 Subject: [PATCH v7 03/14] Remove special cases for non-NXT_CONF_VALUE_ARRAY In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-4-alx.manpages@gmail.com> The previous commit made the functions accepting NXT_CONF_VALUE_ARRAY more generic, so that now they can handle non-arrays correctly. Modify calling code to remove special cases for arrays and non-arrays, taking special care that the path for non arrays is logically equivalent to the previous special cased code. Use the now-generic array code only. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_http_route.c | 45 +++-------------------------------------- src/nxt_http_static.c | 28 ++++++------------------- src/nxt_openssl.c | 24 ++++++---------------- src/nxt_router.c | 23 ++++++--------------- src/python/nxt_python.c | 9 --------- 5 files changed, 21 insertions(+), 108 deletions(-) diff --git a/src/nxt_http_route.c b/src/nxt_http_route.c index 606bf26..71ebf3f 100644 --- a/src/nxt_http_route.c +++ b/src/nxt_http_route.c @@ -718,13 +718,11 @@ nxt_http_route_table_create(nxt_task_t *task, nxt_mp_t *mp, { size_t size; uint32_t i, n; - nxt_bool_t array; nxt_conf_value_t *ruleset_cv; nxt_http_route_table_t *table; nxt_http_route_ruleset_t *ruleset; - array = (nxt_conf_type(table_cv) == NXT_CONF_ARRAY); - n = array ? nxt_conf_array_elements_count(table_cv) : 1; + n = nxt_conf_array_elements_count(table_cv); size = sizeof(nxt_http_route_table_t) + n * sizeof(nxt_http_route_ruleset_t *); @@ -736,18 +734,6 @@ nxt_http_route_table_create(nxt_task_t *task, nxt_mp_t *mp, table->items = n; table->object = NXT_HTTP_ROUTE_TABLE; - if (!array) { - ruleset = nxt_http_route_ruleset_create(task, mp, table_cv, object, - case_sensitive, encoding); - if (nxt_slow_path(ruleset == NULL)) { - return NULL; - } - - table->ruleset[0] = ruleset; - - return table; - } - for (i = 0; i < n; i++) { ruleset_cv = nxt_conf_get_array_element(table_cv, i); @@ -911,13 +897,11 @@ nxt_http_route_rule_create(nxt_task_t *task, nxt_mp_t *mp, size_t size; uint32_t i, n; nxt_int_t ret; - nxt_bool_t string; nxt_conf_value_t *value; nxt_http_route_rule_t *rule; nxt_http_route_pattern_t *pattern; - string = (nxt_conf_type(cv) != NXT_CONF_ARRAY); - n = string ? 1 : nxt_conf_array_elements_count(cv); + n = nxt_conf_array_elements_count(cv); size = sizeof(nxt_http_route_rule_t) + n * sizeof(nxt_http_route_pattern_t); rule = nxt_mp_alloc(mp, size); @@ -929,17 +913,6 @@ nxt_http_route_rule_create(nxt_task_t *task, nxt_mp_t *mp, pattern = &rule->pattern[0]; - if (string) { - pattern[0].case_sensitive = case_sensitive; - ret = nxt_http_route_pattern_create(task, mp, cv, &pattern[0], - pattern_case, encoding); - if (nxt_slow_path(ret != NXT_OK)) { - return NULL; - } - - return rule; - } - nxt_conf_array_qsort(cv, nxt_http_pattern_compare); for (i = 0; i < n; i++) { @@ -963,13 +936,11 @@ nxt_http_route_addr_rule_create(nxt_task_t *task, nxt_mp_t *mp, { size_t size; uint32_t i, n; - nxt_bool_t array; nxt_conf_value_t *value; nxt_http_route_addr_rule_t *addr_rule; nxt_http_route_addr_pattern_t *pattern; - array = (nxt_conf_type(cv) == NXT_CONF_ARRAY); - n = array ? nxt_conf_array_elements_count(cv) : 1; + n = nxt_conf_array_elements_count(cv); size = sizeof(nxt_http_route_addr_rule_t) + n * sizeof(nxt_http_route_addr_pattern_t); @@ -981,16 +952,6 @@ nxt_http_route_addr_rule_create(nxt_task_t *task, nxt_mp_t *mp, addr_rule->items = n; - if (!array) { - pattern = &addr_rule->addr_pattern[0]; - - if (nxt_http_route_addr_pattern_parse(mp, pattern, cv) != NXT_OK) { - return NULL; - } - - return addr_rule; - } - for (i = 0; i < n; i++) { pattern = &addr_rule->addr_pattern[i]; value = nxt_conf_get_array_element(cv, i); diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 36c1ebc..6b7cee7 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -77,7 +77,6 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, nxt_mp_t *mp; nxt_str_t str; nxt_var_t *var; - nxt_bool_t array; nxt_conf_value_t *cv; nxt_http_static_conf_t *conf; @@ -91,39 +90,24 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, action->handler = nxt_http_static; action->u.conf = conf; - array = (nxt_conf_type(acf->share) == NXT_CONF_ARRAY); - conf->nshares = array ? nxt_conf_array_elements_count(acf->share) : 1; - + conf->nshares = nxt_conf_array_elements_count(acf->share); conf->shares = nxt_mp_zget(mp, sizeof(nxt_http_static_share_t) * conf->nshares); if (nxt_slow_path(conf->shares == NULL)) { return NXT_ERROR; } - if (array) { - for (i = 0; i < conf->nshares; i++) { - cv = nxt_conf_get_array_element(acf->share, i); - nxt_conf_get_string(cv, &str); - - var = nxt_var_compile(&str, mp, 1); - if (nxt_slow_path(var == NULL)) { - return NXT_ERROR; - } - - conf->shares[i].var = var; - conf->shares[i].is_const = nxt_var_is_const(var); - } - - } else { - nxt_conf_get_string(acf->share, &str); + for (i = 0; i < conf->nshares; i++) { + cv = nxt_conf_get_array_element(acf->share, i); + nxt_conf_get_string(cv, &str); var = nxt_var_compile(&str, mp, 1); if (nxt_slow_path(var == NULL)) { return NXT_ERROR; } - conf->shares[0].var = var; - conf->shares[0].is_const = nxt_var_is_const(var); + conf->shares[i].var = var; + conf->shares[i].is_const = nxt_var_is_const(var); } #if (NXT_HAVE_OPENAT2) diff --git a/src/nxt_openssl.c b/src/nxt_openssl.c index 1e08015..a4564f1 100644 --- a/src/nxt_openssl.c +++ b/src/nxt_openssl.c @@ -644,16 +644,10 @@ nxt_tls_ticket_keys(nxt_task_t *task, SSL_CTX *ctx, nxt_tls_init_t *tls_init, return NXT_OK; } - if (nxt_conf_type(tickets_conf) == NXT_CONF_ARRAY) { - count = nxt_conf_array_elements_count(tickets_conf); + count = nxt_conf_array_elements_count(tickets_conf); - if (count == 0) { - goto no_ticket; - } - - } else { - /* nxt_conf_type(tickets_conf) == NXT_CONF_STRING */ - count = 1; + if (count == 0) { + goto no_ticket; } #ifdef SSL_CTRL_SET_TLSEXT_TICKET_KEY_CB @@ -673,15 +667,9 @@ nxt_tls_ticket_keys(nxt_task_t *task, SSL_CTX *ctx, nxt_tls_init_t *tls_init, i++; - if (nxt_conf_type(tickets_conf) == NXT_CONF_ARRAY) { - member = nxt_conf_get_array_element(tickets_conf, count - i); - if (member == NULL) { - break; - } - - } else { - /* nxt_conf_type(tickets_conf) == NXT_CONF_STRING */ - member = tickets_conf; + member = nxt_conf_get_array_element(tickets_conf, count - i); + if (member == NULL) { + break; } nxt_conf_get_string(member, &value); diff --git a/src/nxt_router.c b/src/nxt_router.c index 52ea0f3..b8ee0c7 100644 --- a/src/nxt_router.c +++ b/src/nxt_router.c @@ -1924,25 +1924,14 @@ nxt_router_conf_create(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, tls_init->tickets_conf = nxt_conf_get_path(listener, &conf_tickets); - if (nxt_conf_type(certificate) == NXT_CONF_ARRAY) { - n = nxt_conf_array_elements_count(certificate); - - for (i = 0; i < n; i++) { - value = nxt_conf_get_array_element(certificate, i); - - nxt_assert(value != NULL); + n = nxt_conf_array_elements_count(certificate); + for (i = 0; i < n; i++) { + value = nxt_conf_get_array_element(certificate, i); - ret = nxt_router_conf_tls_insert(tmcf, value, skcf, - tls_init, i == 0); - if (nxt_slow_path(ret != NXT_OK)) { - goto fail; - } - } + nxt_assert(value != NULL); - } else { - /* NXT_CONF_STRING */ - ret = nxt_router_conf_tls_insert(tmcf, certificate, skcf, - tls_init, 1); + ret = nxt_router_conf_tls_insert(tmcf, value, skcf, + tls_init, i == 0); if (nxt_slow_path(ret != NXT_OK)) { goto fail; } diff --git a/src/python/nxt_python.c b/src/python/nxt_python.c index 8687c86..3edaa08 100644 --- a/src/python/nxt_python.c +++ b/src/python/nxt_python.c @@ -411,14 +411,7 @@ nxt_python_set_path(nxt_task_t *task, nxt_conf_value_t *value) /* sys is a Borrowed reference. */ - if (nxt_conf_type(value) == NXT_CONF_STRING) { - n = 0; - goto value_is_string; - } - - /* NXT_CONF_ARRAY */ array = value; - n = nxt_conf_array_elements_count(array); while (n != 0) { @@ -432,8 +425,6 @@ nxt_python_set_path(nxt_task_t *task, nxt_conf_value_t *value) value = nxt_conf_get_array_element(array, n); - value_is_string: - nxt_conf_get_string(value, &str); path = PyString_FromStringAndSize((char *) str.start, str.length); -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:20 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:20 +0100 Subject: [PATCH v7 10/14] Static: variables in the "index" option In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-11-alx.manpages@gmail.com> This adds support for variables in the "index" option. An example: { "share": "/srv/www/data/static/$uri", "index": "$method.html" } When is requested, will be served. === I also tried this feature in my own computer, where I tried the following: - Setting "index" to "$method.html", and check that is being served. - Not setting index, and check that is being served. - Setting "index" to "$asdf", and check that the configuration fails: { "error": "Invalid configuration.", "detail": "Unknown variable \"asdf\" in the \"index\" value." } Tested-by: Alejandro Colomar Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- docs/changes.xml | 6 ++++++ src/nxt_conf_validation.c | 1 + src/nxt_http_static.c | 29 +++++++++++++++++++++-------- 3 files changed, 28 insertions(+), 8 deletions(-) diff --git a/docs/changes.xml b/docs/changes.xml index 81708c3..7709fc8 100644 --- a/docs/changes.xml +++ b/docs/changes.xml @@ -18,6 +18,12 @@ date="" time="" packager="Andrei Belov <defan at nginx.com>"> + + +variables support in the "index" option. + + + new "index" option to specify a custom index file name. diff --git a/src/nxt_conf_validation.c b/src/nxt_conf_validation.c index bdc38b0..6236720 100644 --- a/src/nxt_conf_validation.c +++ b/src/nxt_conf_validation.c @@ -648,6 +648,7 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_share_action_members[] = { }, { .name = nxt_string("index"), .type = NXT_CONF_VLDT_STRING, + .flags = NXT_CONF_VLDT_VAR, }, { .name = nxt_string("types"), .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 2528b95..bee85b2 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -19,7 +19,7 @@ typedef struct { typedef struct { nxt_uint_t nshares; nxt_http_static_share_t *shares; - nxt_str_t index; + nxt_var_t *index; #if (NXT_HAVE_OPENAT2) nxt_var_t *chroot; nxt_uint_t resolve; @@ -115,12 +115,19 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, } if (acf->index == NULL) { - conf->index = default_index; + str = default_index; } else { - nxt_conf_get_string(acf->index, &conf->index); + nxt_conf_get_string(acf->index, &str); } + var = nxt_var_compile(&str, mp, 1); + if (nxt_slow_path(var == NULL)) { + return NXT_ERROR; + } + + conf->index = var; + #if (NXT_HAVE_OPENAT2) if (acf->chroot.length > 0) { nxt_str_t chr, shr; @@ -226,7 +233,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_int_t ret; nxt_http_static_conf_t *conf; nxt_http_static_share_t *share; - nxt_bool_t shr_is_const; + nxt_bool_t shr_is_const, idx_is_const; conf = ctx->action->u.conf; @@ -237,7 +244,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_str_t idx; nxt_var_raw(share->var, &shr); - idx = conf->index; + nxt_var_raw(conf->index, &idx); #if (NXT_HAVE_OPENAT2) nxt_str_t chr; @@ -257,8 +264,9 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, #endif /* NXT_DEBUG */ shr_is_const = share->is_const; + idx_is_const = nxt_var_is_const(conf->index); - if (!shr_is_const) { + if (!shr_is_const || !idx_is_const) { ret = nxt_var_query_init(&r->var_query, r, r->mem_pool); if (nxt_slow_path(ret != NXT_OK)) { nxt_http_request_error(task, r, NXT_HTTP_INTERNAL_SERVER_ERROR); @@ -266,7 +274,12 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, } } - ctx->index = conf->index; + if (idx_is_const) { + nxt_var_raw(conf->index, &ctx->index); + + } else { + nxt_var_query(task, r->var_query, conf->index, &ctx->index); + } if (shr_is_const) { nxt_var_raw(share->var, &ctx->share); @@ -287,7 +300,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, #endif } - if (shr_is_const) { + if (shr_is_const && idx_is_const) { nxt_http_static_send_ready(task, r, ctx); } else { -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:21 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:21 +0100 Subject: [PATCH v7 11/14] nxt_conf_set_string_dup(): Use 'const' for read-only parameter In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-12-alx.manpages@gmail.com> That parameter is not being modified in the function. Make it 'const' to allow passing 'static const' variables. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_conf.c | 3 ++- src/nxt_conf.h | 2 +- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/src/nxt_conf.c b/src/nxt_conf.c index 3f0cfa8..967c455 100644 --- a/src/nxt_conf.c +++ b/src/nxt_conf.c @@ -192,7 +192,8 @@ nxt_conf_set_string(nxt_conf_value_t *value, nxt_str_t *str) nxt_int_t -nxt_conf_set_string_dup(nxt_conf_value_t *value, nxt_mp_t *mp, nxt_str_t *str) +nxt_conf_set_string_dup(nxt_conf_value_t *value, nxt_mp_t *mp, + const nxt_str_t *str) { nxt_str_t tmp, *ptr; diff --git a/src/nxt_conf.h b/src/nxt_conf.h index 8b3565f..09f2175 100644 --- a/src/nxt_conf.h +++ b/src/nxt_conf.h @@ -115,7 +115,7 @@ nxt_int_t nxt_conf_validate(nxt_conf_validation_t *vldt); NXT_EXPORT void nxt_conf_get_string(nxt_conf_value_t *value, nxt_str_t *str); NXT_EXPORT void nxt_conf_set_string(nxt_conf_value_t *value, nxt_str_t *str); NXT_EXPORT nxt_int_t nxt_conf_set_string_dup(nxt_conf_value_t *value, - nxt_mp_t *mp, nxt_str_t *str); + nxt_mp_t *mp, const nxt_str_t *str); NXT_EXPORT double nxt_conf_get_number(nxt_conf_value_t *value); NXT_EXPORT uint8_t nxt_conf_get_boolean(nxt_conf_value_t *value); -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:22 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:22 +0100 Subject: [PATCH v7 12/14] nxt_http_static_init(): Simplify branching In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-13-alx.manpages@gmail.com> Add a bit of code to the special case of no "index", so that the normal case is not in a branch. This makes the normal flow more readable, by hiding details of special cases in branches, even if that means making the special case a bit more complex than it needs to be. 'else' distracts the mental parser of the reader of the code: it means it needs to branch in both cases. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_http_static.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index bee85b2..cbc8cbf 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -115,12 +115,16 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, } if (acf->index == NULL) { - str = default_index; + acf->index = nxt_conf_create_object(mp, 1); + if (nxt_slow_path(acf->index == NULL)) { + return NXT_ERROR; + } - } else { - nxt_conf_get_string(acf->index, &str); + nxt_conf_set_string_dup(acf->index, mp, &default_index); } + nxt_conf_get_string(acf->index, &str); + var = nxt_var_compile(&str, mp, 1); if (nxt_slow_path(var == NULL)) { return NXT_ERROR; -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:23 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:23 +0100 Subject: [PATCH v7 13/14] Static: multiple paths in the "index" option In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-14-alx.manpages@gmail.com> This adds support for multiple paths in the "index" option. An example: { "share": "/srv/www/stat/static/$uri", "index": [ "$method.html", "alt.html" ] } When is requested, will be served if it exists; otherwise, will be served. === I also tried this feature in my own computer, where I tried the following: - Setting "index" to ["idontexist", "$host.html"], and check that is being served. - Not setting "index", and check that is being served. - Setting "index" to [], and check that it reports an error when loading the configuration. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- docs/changes.xml | 6 +++++ src/nxt_conf_validation.c | 51 +++++++++++++++++++++++++++++++++++++-- src/nxt_http_static.c | 38 ++++++++++++++++++++--------- 3 files changed, 82 insertions(+), 13 deletions(-) diff --git a/docs/changes.xml b/docs/changes.xml index 7709fc8..18e180d 100644 --- a/docs/changes.xml +++ b/docs/changes.xml @@ -18,6 +18,12 @@ date="" time="" packager="Andrei Belov <defan at nginx.com>"> + + +multiple paths in the "index" option. + + + variables support in the "index" option. diff --git a/src/nxt_conf_validation.c b/src/nxt_conf_validation.c index 6236720..5b94116 100644 --- a/src/nxt_conf_validation.c +++ b/src/nxt_conf_validation.c @@ -117,6 +117,10 @@ static nxt_int_t nxt_conf_vldt_share(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, void *data); static nxt_int_t nxt_conf_vldt_share_element(nxt_conf_validation_t *vldt, nxt_conf_value_t *value); +static nxt_int_t nxt_conf_vldt_index(nxt_conf_validation_t *vldt, + nxt_conf_value_t *value, void *data); +static nxt_int_t nxt_conf_vldt_index_element(nxt_conf_validation_t *vldt, + nxt_conf_value_t *value); static nxt_int_t nxt_conf_vldt_proxy(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, void *data); static nxt_int_t nxt_conf_vldt_python(nxt_conf_validation_t *vldt, @@ -647,8 +651,8 @@ static nxt_conf_vldt_object_t nxt_conf_vldt_share_action_members[] = { .validator = nxt_conf_vldt_share, }, { .name = nxt_string("index"), - .type = NXT_CONF_VLDT_STRING, - .flags = NXT_CONF_VLDT_VAR, + .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, + .validator = nxt_conf_vldt_index, }, { .name = nxt_string("types"), .type = NXT_CONF_VLDT_STRING | NXT_CONF_VLDT_ARRAY, @@ -1694,6 +1698,49 @@ nxt_conf_vldt_share_element(nxt_conf_validation_t *vldt, } +static nxt_int_t +nxt_conf_vldt_index(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, + void *data) +{ + if (nxt_conf_type(value) == NXT_CONF_ARRAY) { + if (nxt_conf_array_elements_count(value) == 0) { + return nxt_conf_vldt_error(vldt, "The \"index\" array " + "must contain at least one element."); + } + + return nxt_conf_vldt_array_iterator(vldt, value, + &nxt_conf_vldt_index_element); + } + + /* NXT_CONF_STRING */ + + return nxt_conf_vldt_index_element(vldt, value); +} + + +static nxt_int_t +nxt_conf_vldt_index_element(nxt_conf_validation_t *vldt, + nxt_conf_value_t *value) +{ + nxt_str_t str; + + static nxt_str_t index = nxt_string("index"); + + if (nxt_conf_type(value) != NXT_CONF_STRING) { + return nxt_conf_vldt_error(vldt, "The \"index\" array must " + "contain only string values."); + } + + nxt_conf_get_string(value, &str); + + if (nxt_is_var(&str)) { + return nxt_conf_vldt_var(vldt, &index, &str); + } + + return NXT_OK; +} + + static nxt_int_t nxt_conf_vldt_proxy(nxt_conf_validation_t *vldt, nxt_conf_value_t *value, void *data) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index cbc8cbf..d6bb88d 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -18,8 +18,9 @@ typedef struct { typedef struct { nxt_uint_t nshares; + nxt_uint_t nindices; nxt_http_static_share_t *shares; - nxt_var_t *index; + nxt_var_t **indices; #if (NXT_HAVE_OPENAT2) nxt_var_t *chroot; nxt_uint_t resolve; @@ -36,6 +37,7 @@ typedef struct { nxt_str_t chroot; #endif uint32_t share_idx; + uint32_t index_idx; uint8_t need_body; /* 1 bit */ } nxt_http_static_ctx_t; @@ -123,14 +125,23 @@ nxt_http_static_init(nxt_task_t *task, nxt_router_temp_conf_t *tmcf, nxt_conf_set_string_dup(acf->index, mp, &default_index); } - nxt_conf_get_string(acf->index, &str); - - var = nxt_var_compile(&str, mp, 1); - if (nxt_slow_path(var == NULL)) { + conf->nindices = nxt_conf_array_elements_count(acf->index); + conf->indices = nxt_mp_zget(mp, sizeof(*conf->indices) * conf->nindices); + if (nxt_slow_path(conf->indices == NULL)) { return NXT_ERROR; } - conf->index = var; + for (i = 0; i < conf->nindices; i++) { + cv = nxt_conf_get_array_element(acf->index, i); + nxt_conf_get_string(cv, &str); + + var = nxt_var_compile(&str, mp, 1); + if (nxt_slow_path(var == NULL)) { + return NXT_ERROR; + } + + conf->indices[i] = var; + } #if (NXT_HAVE_OPENAT2) if (acf->chroot.length > 0) { @@ -235,6 +246,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_http_static_ctx_t *ctx) { nxt_int_t ret; + nxt_var_t *index; nxt_http_static_conf_t *conf; nxt_http_static_share_t *share; nxt_bool_t shr_is_const, idx_is_const; @@ -242,13 +254,14 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, conf = ctx->action->u.conf; share = &conf->shares[ctx->share_idx]; + index = conf->indices[ctx->index_idx]; #if (NXT_DEBUG) nxt_str_t shr; nxt_str_t idx; nxt_var_raw(share->var, &shr); - nxt_var_raw(conf->index, &idx); + nxt_var_raw(index, &idx); #if (NXT_HAVE_OPENAT2) nxt_str_t chr; @@ -268,7 +281,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, #endif /* NXT_DEBUG */ shr_is_const = share->is_const; - idx_is_const = nxt_var_is_const(conf->index); + idx_is_const = nxt_var_is_const(index); if (!shr_is_const || !idx_is_const) { ret = nxt_var_query_init(&r->var_query, r, r->mem_pool); @@ -279,10 +292,10 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, } if (idx_is_const) { - nxt_var_raw(conf->index, &ctx->index); + nxt_var_raw(index, &ctx->index); } else { - nxt_var_query(task, r->var_query, conf->index, &ctx->index); + nxt_var_query(task, r->var_query, index, &ctx->index); } if (shr_is_const) { @@ -700,7 +713,10 @@ nxt_http_static_next(nxt_task_t *task, nxt_http_request_t *r, action = ctx->action; conf = action->u.conf; - ctx->share_idx++; + ctx->index_idx = (ctx->index_idx + 1) % conf->nindices; + if (ctx->index_idx == 0) { + ctx->share_idx++; + } if (ctx->share_idx < conf->nshares) { nxt_http_static_iterate(task, r, ctx); -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:17 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:17 +0100 Subject: [PATCH v7 07/14] Static: return 404 when "index" is a non-regular file In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-8-alx.manpages@gmail.com> When index is a file, but not a regular file nor a directory, so it may be for example a FIFO, Unit returns 404. However, for directories, it was returning 301. Nginx also returns 301 (and is documented, as Valentin pointed out)[1], but we prefer to be consistent (and intuitive) with other invalid files in Unit and so return 404 for dirs too. [1]: I got the following pytest errors after this patch, but I'm not sure of what those tests mean. ===================================== FAILURES ===================================== ____________________ TestStaticChroot.test_static_chroot_empty _____________________ self = temp_dir = '/tmp/unit-test-19ucrwpo' def test_static_chroot_empty(self, temp_dir): assert 'success' in self.conf( {"share": temp_dir + "/assets$uri", "chroot": ""}, 'routes/0/action', ), 'configure chroot empty absolute' assert ( self.get(url='/dir/file')['status'] == 200 ), 'chroot empty absolute' assert 'success' in self.conf( {"share": ".$uri", "chroot": ""}, 'routes/0/action', ), 'configure chroot empty relative' > assert ( self.get(url=self.test_path)['status'] == 200 ), 'chroot empty relative' E AssertionError: chroot empty relative E assert 404 == 200 E +404 E -200 test_static_chroot.py:107: AssertionError ___________________ TestStaticChroot.test_static_chroot_relative ___________________ self = , is_su = False temp_dir = '/tmp/unit-test-19ucrwpo' def test_static_chroot_relative(self, is_su, temp_dir): if is_su: pytest.skip('does\'t work under root') assert 'success' in self.conf( {"share": temp_dir + "/assets$uri", "chroot": "."}, 'routes/0/action', ), 'configure relative chroot' assert self.get(url='/dir/file')['status'] == 403, 'relative chroot' assert 'success' in self.conf( {"share": ".$uri"}, 'routes/0/action', ), 'configure relative share' > assert self.get(url=self.test_path)['status'] == 200, 'relative share' E AssertionError: relative share E assert 404 == 200 E +404 E -200 test_static_chroot.py:126: AssertionError ============================= short test summary info ============================== FAILED test_static_chroot.py::TestStaticChroot::test_static_chroot_empty - Assert... FAILED test_static_chroot.py::TestStaticChroot::test_static_chroot_relative - Ass... ==================== 2 failed, 62 passed, 706 skipped in 2.77s ===================== Reported-by: Andrei Zeliankou Suggested-by: "Valentin V. Bartenev" Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: Igor Sysoev Cc: Zhidao HONG Cc: Oisin Canty Cc: Maxim Romanov --- src/nxt_http_static.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 663f896..7f1254c 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -571,7 +571,9 @@ nxt_http_static_send_ready(nxt_task_t *task, void *obj, void *data) /* Not a file. */ nxt_file_close(task, f); - if (nxt_slow_path(!nxt_is_dir(&fi))) { + if (nxt_slow_path(!nxt_is_dir(&fi) + || ctx->share.start[ctx->share.length - 1] == '/')) + { nxt_log(task, NXT_LOG_ERR, "\"%FN\" is not a regular file", f->name); -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:24 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:24 +0100 Subject: [PATCH v7 14/14] Static: optimize "index" iteration In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-15-alx.manpages@gmail.com> Iterate over "index" only if "share" is a directory. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_http_static.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index d6bb88d..74245cd 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -713,7 +713,10 @@ nxt_http_static_next(nxt_task_t *task, nxt_http_request_t *r, action = ctx->action; conf = action->u.conf; - ctx->index_idx = (ctx->index_idx + 1) % conf->nindices; + if (ctx->share.start[ctx->share.length - 1] == '/') { + ctx->index_idx = (ctx->index_idx + 1) % conf->nindices; + } + if (ctx->index_idx == 0) { ctx->share_idx++; } -- 2.34.1 From alx.manpages at gmail.com Tue Feb 15 15:25:19 2022 From: alx.manpages at gmail.com (Alejandro Colomar) Date: Tue, 15 Feb 2022 16:25:19 +0100 Subject: [PATCH v7 09/14] nxt_http_static_iterate(): generalize code In-Reply-To: <20220215152524.41119-1-alx.manpages@gmail.com> References: <20220215152524.41119-1-alx.manpages@gmail.com> Message-ID: <20220215152524.41119-10-alx.manpages@gmail.com> Separate into separate blocks of code the initialization of the var query, and the final action, to allow for a simpler patch that allows using variables for the 'index'. Add a nxt_bool_t shr_is_const variable to allow for a similar idx_is_const, and improve readability. Signed-off-by: Alejandro Colomar Cc: Nginx Unit Cc: "Valentin V. Bartenev" Cc: Zhidao HONG Cc: Igor Sysoev Cc: Oisin Canty Cc: Andrei Zeliankou Cc: Maxim Romanov --- src/nxt_http_static.c | 28 ++++++++++++++++++---------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/src/nxt_http_static.c b/src/nxt_http_static.c index 0327861..2528b95 100644 --- a/src/nxt_http_static.c +++ b/src/nxt_http_static.c @@ -226,6 +226,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_int_t ret; nxt_http_static_conf_t *conf; nxt_http_static_share_t *share; + nxt_bool_t shr_is_const; conf = ctx->action->u.conf; @@ -255,9 +256,19 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, #endif #endif /* NXT_DEBUG */ + shr_is_const = share->is_const; + + if (!shr_is_const) { + ret = nxt_var_query_init(&r->var_query, r, r->mem_pool); + if (nxt_slow_path(ret != NXT_OK)) { + nxt_http_request_error(task, r, NXT_HTTP_INTERNAL_SERVER_ERROR); + return; + } + } + ctx->index = conf->index; - if (share->is_const) { + if (shr_is_const) { nxt_var_raw(share->var, &ctx->share); #if (NXT_HAVE_OPENAT2) @@ -266,15 +277,7 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, } #endif - nxt_http_static_send_ready(task, r, ctx); - } else { - ret = nxt_var_query_init(&r->var_query, r, r->mem_pool); - if (nxt_slow_path(ret != NXT_OK)) { - nxt_http_request_error(task, r, NXT_HTTP_INTERNAL_SERVER_ERROR); - return; - } - nxt_var_query(task, r->var_query, share->var, &ctx->share); #if (NXT_HAVE_OPENAT2) @@ -282,11 +285,16 @@ nxt_http_static_iterate(nxt_task_t *task, nxt_http_request_t *r, nxt_var_query(task, r->var_query, conf->chroot, &ctx->chroot); } #endif + } + + if (shr_is_const) { + nxt_http_static_send_ready(task, r, ctx); + } else { nxt_var_query_resolve(task, r->var_query, ctx, nxt_http_static_send_ready, nxt_http_static_var_error); - } + } } -- 2.34.1 From alx.manpages at gmail.com Tue Feb 22 09:24:53 2022 From: alx.manpages at gmail.com (Alejandro Colomar (man-pages)) Date: Tue, 22 Feb 2022 10:24:53 +0100 Subject: Tests for "index" Message-ID: Hi Andrei, Now I've got some pattern for the tests that are failing. It seems that it's not interpreting $host as README in the tests, when we're setting Host to README. With the following changes, all tests pass. Do you have any idea why? --- $ git diff -U6 diff --git a/test/test_static.py b/test/test_static.py index 52c5270..5ddc368 100644 --- a/test/test_static.py +++ b/test/test_static.py @@ -97,28 +97,28 @@ class TestStatic(TestApplicationProto): assert self.get()['body'] == 'readme', 'index' self.conf_delete('routes/0/action/index') assert self.get()['body'] == '0123456789', 'delete index' set_index('$host') - assert ( - self.get(headers={"Host": "README", "Connection": "close"})['body'] - == 'readme' - ), 'index var' + #assert ( + #self.get(headers={"Host": "README", "Connection": "close"})['body'] + #== 'readme' + #), 'index var' self.get(headers={"Connection": "close"})[ 'status' ] == 404, 'index var empty' set_index(['blah']) self.get()['status'] == 404, 'index array not found' set_index(['$host', 'blah', 'index.html']) assert self.get()['body'] == '0123456789', 'index array' assert ( self.get(headers={"Host": "README", "Connection": "close"})['body'] - == 'readme' + == '0123456789' ), 'index array 2' def test_static_index_invalid(self, skip_alert): skip_alert(r'failed to apply new conf') def check_index(index): -- Alejandro Colomar Linux man-pages comaintainer; http://www.kernel.org/doc/man-pages/ http://www.alejandro-colomar.es/ From ranier.vf at gmail.com Thu Feb 24 12:50:29 2022 From: ranier.vf at gmail.com (Ranier Vilela) Date: Thu, 24 Feb 2022 09:50:29 -0300 Subject: Unsubscribe from Unit Project Message-ID: Hi, Please unsubscribe me from the Coverity Unit project and Unit mail list. Ranier Vilela --Stop the War. -------------- next part -------------- An HTML attachment was scrubbed... URL: From maxim at nginx.com Thu Feb 24 13:37:02 2022 From: maxim at nginx.com (Maxim Konovalov) Date: Thu, 24 Feb 2022 16:37:02 +0300 Subject: Unsubscribe from Unit Project In-Reply-To: References: Message-ID: Hi Ranier, I removed ranier_gyn at hotmail.com email from the nginx-unit coverity notification list. Thanks, Maxim On 24.02.2022 15:50, Ranier Vilela wrote: > Hi, > > Please unsubscribe me from the Coverity Unit project > and Unit mail list. > > Ranier Vilela > --Stop the War. > > _______________________________________________ > unit mailing list -- unit at nginx.org > To unsubscribe send an email to unit-leave at nginx.org -- Maxim Konovalov From ranier.vf at gmail.com Thu Feb 24 13:38:58 2022 From: ranier.vf at gmail.com (Ranier Vilela) Date: Thu, 24 Feb 2022 10:38:58 -0300 Subject: Unsubscribe from Unit Project In-Reply-To: References: Message-ID: Em qui., 24 de fev. de 2022 às 10:37, Maxim Konovalov escreveu: > Hi Ranier, > > I removed ranier_gyn at hotmail.com email from the nginx-unit coverity > notification list. > Thank you. regards, Ranier Vilela --Stop the War. -------------- next part -------------- An HTML attachment was scrubbed... URL: From kaystein92 at gmail.com Thu Feb 24 14:09:55 2022 From: kaystein92 at gmail.com (kaystein92) Date: Thu, 24 Feb 2022 08:09:55 -0600 Subject: Unsubscribe from Unit Project In-Reply-To: Message-ID: <621791b5.1c69fb81.7e641.9af6@mx.google.com> Hi,Please unsubscribe me from the Coverity Unit projectand Unit mail list. -------- Original message --------From: Ranier Vilela Date: 2/24/22 6:50 AM (GMT-06:00) To: unit at nginx.org Subject: Unsubscribe from Unit Project Hi,Please unsubscribe me from the Coverity Unit projectand Unit mail list.Ranier Vilela--Stop the War. -------------- next part -------------- An HTML attachment was scrubbed... URL: