Slice module

[Martijn Berkvens]

Hi Roman,

> On 18 Feb 2016, at 18:14, Roman Arutyunyan <arut at nginx.com> wrote:
> 
> Hi Martijn,
> 
> On Wed, Feb 17, 2016 at 09:20:37AM +0000, Martijn Berkvens wrote:
>> Hi,
>> 
>> First off, thank you for the work on the slice module. We’re currently using a lua implementation to achieve something similar but this has its shortcomings and we would prefer to use this module. 
>> 
>> We’ve been testing this module and when using range requests it works as expected, but when doing normal requests (without a request range) the module still slices the content when the configured slice size is met resulting in corrupt data as it will only serve out the configured slice size and not the entire requested file.
> 
> Module slices all requests, whether they have Range header or not.
> This lets nginx reuse already cached slices.

This explains the behaviour I’m seeing then and there actually is no issue, it was my miss interpretation of how to implement it. I was doing something like this:

set $cachekey "$scheme$http_host$uri$is_args$args”;

if ($http_range)
{
set $cachekey “$cachekey $slice_range”;
}

slice 1m;
proxy_cache_key $cachekey;

After removing the unneeded if clause and just adding $slice_range to the cachekey for all requests everything worked as expected. Great to see that nginx reuses all slices to assemble a full file if requested.  

Thanks again for your work and apologies for the confusion. 

Martijn


> 
> Please describe in detail what exactly is corrupted and show your
> configuration.  Nothing should be corrupted when using the slice module, the
> entire file should be assembled of slices and sent to client once you
> configured nginx correctly.
> 
>> We run into these issues because a server or location configuration within nginx would service both normal requests as range requests and we have files exceeding the slice size in both cases. 
>> 
>> Ideally the slice module would only do its magic when a request contains $http_range. We’ve tried creating a map which would set the slice size to 0 (to disable slicing) when no $http_range is present, but unfortunately the slice directive currently does not support variables. 
>> 
>> For our use case it would most likely be sufficient to add variable support to the slice directive. It is understood and acceptable that we would have duplicate cached content in cases where we have both the sliced versions of the file as the entire file. 
> 
> You can have that by adding another location for files cached as a whole
> and rewriting your request to that location when $http_range is empty.
> 
> -- 
> Roman Arutyunyan
> 
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel