range_filter_module get duplicated Accept-Ranges response headers

[Roman Arutyunyan]

Hello,

On Mon, Jul 06, 2020 at 06:49:15AM -0400, webber wrote:
> Hello, 
> 
> Recently,  we found if we use nginx slice module , and upstream server is
> such as a static file server,  nginx will response duplicated
> `Accept-Ranges` headers if client request is not included range header.
> 
> the minimal config example as follow:
> 
> ```
> server {
>         listen                      80;
>         server_name          _;
>         default_type           text/html;
>         
>         location /get_file {
>             slice 256k;
>             proxy_set_header Range $slice_range;
>             proxy_pass       http://127.0.0.1:8899;
>         } 
> }
> ```
> 
> use curl to get 1mb file:
> ```
> curl -s http://localhost/get_file/1mb.test -D- -o /dev/null
> 
> HTTP/1.1 200 OK
> Date: Mon, 06 Jul 2020 10:32:58 GMT
> Content-Type: application/octet-stream
> Content-Length: 1048576
> Connection: keep-alive
> Last-Modified: Mon, 06 Jul 2020 07:34:23 GMT
> Cache-Control: public, max-age=43200
> Expires: Mon, 06 Jul 2020 22:32:58 GMT
> ETag: "1594020863.76-1048576-4019326287"
> Accept-Ranges: bytes
> Accept-Ranges: bytes
> ```
> 
> but if I add range header to curl request, will get expected response.

Normally nginx does not send Accept-Ranges with partial responses.  The fact
that the response is partial is itself an indication that the server supports
ranges.  If the upstream server is nginx too, there should be no problem with
slice.  However if the upstream server sends Accept-Ranges with partial
responses, the client ends up with the two Accept-Ranges.

Attached is a patch that removes the original Accept-Ranges.  This brings
back the normal nginx behavior - one Accept-Ranges for full response and
none for partial.  Can you try it and report back if it works for you?

> Then I review the ngx_http_range_filter_module,  in `goto next_filter`( line
> 253) ,  should we handle  NGX_HTTP_OK response? like: 
> 
> ```
> next_filter:
> 
>     if (r->headers_out.status == NGX_HTTP_OK) {
>         r->headers_out.accept_ranges = NULL;
>         return ngx_http_next_header_filter(r);
>     }
> 
>     r->headers_out.accept_ranges = ngx_list_push(&r->headers_out.headers);
>     if (r->headers_out.accept_ranges == NULL) {
>         return NGX_ERROR;
>     }
> 
>     r->headers_out.accept_ranges->hash = 1;
>     ngx_str_set(&r->headers_out.accept_ranges->key, "Accept-Ranges");
>     ngx_str_set(&r->headers_out.accept_ranges->value, "bytes");
> 
>     return ngx_http_next_header_filter(r);
> ```
> 
> I am confused if it is a bug?
> 
> Posted at Nginx Forum: https://forum.nginx.org/read.php?2,288569,288569#msg-288569
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx

-- 
Roman Arutyunyan
-------------- next part --------------
# HG changeset patch
# User Roman Arutyunyan <arut at nginx.com>
# Date 1594115767 -10800
#      Tue Jul 07 12:56:07 2020 +0300
# Node ID 86cf2f78477d6ef8845608a9a36003899308f6e6
# Parent  1ece2ac2555a10f7a09ea2feca2db3149ccb470f
Slice filter: clear original Accept-Ranges.

The slice filter allows ranges for the response by setting the r->allow_ranges
flag, which enables the range filter.  If the range was not requested, the
range filter adds an Accept-Ranges header to the response to signal the
support for ranges.

Previously, if an Accept-Ranges header was already present in the first slice
response, client received two copies of this header.  Now, the slice filters
removes the Accept-Ranges header from the response prior to setting the
r->allow_ranges flag.

diff --git a/src/http/modules/ngx_http_slice_filter_module.c b/src/http/modules/ngx_http_slice_filter_module.c
--- a/src/http/modules/ngx_http_slice_filter_module.c
+++ b/src/http/modules/ngx_http_slice_filter_module.c
@@ -180,6 +180,11 @@ ngx_http_slice_header_filter(ngx_http_re
     r->headers_out.content_range->hash = 0;
     r->headers_out.content_range = NULL;
 
+    if (r->headers_out.accept_ranges) {
+        r->headers_out.accept_ranges->hash = 0;
+        r->headers_out.accept_ranges = NULL;
+    }
+
     r->allow_ranges = 1;
     r->subrequest_ranges = 1;
     r->single_range = 1;