Slice module 206 requirement

[Lucas Rolff]

Hi guys,

I’m having an nginx instance where I utilise the nginx slice module to slice upstream mp4 files when using proxy_cache.

However, I have an interesting origin where if sending a range request (which happens when the slice module is enabled), to a file that’s less than the slice range, the origin returns a 200 OK, but with the range related headers such as content-range, but obviously the full file is returned since it’s within the requested range.

When playing the MP4s through Google Chrome and Firefox it works fine when going through the nginx proxy instance, however, it somehow breaks Safari (both on MacOS, and iOS) - I guess Safari is more strict.
When playing directly through the origin it works fine in all browsers.

The md5 of response from the origin remains the same, so it’s not that the response itself is an invalid MP4 file, and even if you compare the cache files on disk with a “working” origin and the “broken” origin (one sends a 206 Partial Content, another sends 200 OK) - the content of the cache files remain the same, except obviously the header section of the cache file.

The origin returns a 206 status code, only if the file exceeds the slice size, so if I configure a slice size of 5 megabyte, only files above 5 megabytes will give 206s. Anything under 5 megabytes will result in a 200 OK with content-range and the correct content-length,

Looking in the slice module itself I see:
https://github.com/nginx/nginx/blob/master/src/http/modules/ngx_http_slice_filter_module.c#L116-L126


    if (r->headers_out.status != NGX_HTTP_PARTIAL_CONTENT) {
        if (r == r->main) {
            ngx_http_set_ctx(r, NULL, ngx_http_slice_filter_module);
            return ngx_http_next_header_filter(r);
        }

        ngx_log_error(NGX_LOG_ERR, r->connection->log, 0,
                      "unexpected status code %ui in slice response",
                      r->headers_out.status);
        return NGX_ERROR;
    }

This seems like the slice module expects a 206 status code to be returned, however, later in the same function https://github.com/nginx/nginx/blob/master/src/http/modules/ngx_http_slice_filter_module.c#L200-L211


    if (r->headers_out.status == NGX_HTTP_PARTIAL_CONTENT) {
        if (ctx->start + (off_t) slcf->size <= r->headers_out.content_offset) {
            ctx->start = slcf->size
                         * (r->headers_out.content_offset / slcf->size);
        }

        ctx->end = r->headers_out.content_offset
                   + r->headers_out.content_length_n;

    } else {
        ctx->end = cr.complete_length;
    }

There it will do an else statement if the status code isn’t 206.
So would this piece of code ever be reached, since there’s the initial error?

Additionally I don’t see in RFC7233 that 206 responses are an absolute requirement, additionally I don’t see content-range being prohibited/forbidden to be used for 200 OK responses.
Now, if one have a secondary proxy that modifies the response headers in between the origin returning 200 OK with the Content-Range header, and then strip out the Content-Range header, the nginx slice module seems to handle it fine, so somehow the combination of 200 OK and a Content-Range header being present seems to break the slice module from functioning.

I’m just curious why this happens within the slice module, and if there’s any possible solution for it (like allowing the combination of 200 OK and Content-Range, since those two would still indicate that the origin/upstream supports range requests) - obviously it would be nice to fix the upstream server but sometimes that’s sadly not possible.

I know the parts of the slice module haven’t been touched for years, so obviously it works for most people, just dipping my toes here to see if there’s a possible solution other than disabling slice when an origin returns 200 OK for files smaller than the slice size.

Thanks in advance

Best Regards,
Lucas Rolff
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20220708/d45be4b4/attachment.htm>