Multiple upstream_cache_status headers in response in a dual-cache configuration

[Peter Booth]

Why do you want to "realize a distributed caching layer based on disk-speed and storage?”

Providing that you are running nginx on a healthy host running linux then your HDD-cache 
 be faster (or the seem speed) as your SSD-cache. This because the cached file will be 
written though the Linux page cache, just as reads will return the file from Linux page cache 
and not touch either of the disks. 

This means that the effective performance of your server is largely decoupled from the 
physical performance of the physical drives. Of course you should monitor your host to 
ensure that it has sufficient memory and that no major page faults are occurring
(sar -B should return 0.0)


Peter



> On Oct 6, 2017, at 3:05 AM, rnmx18 <nginx-forum at forum.nginx.org> wrote:
> 
> Hi,
> 
> To realize a distributed caching layer based of disk-speed and storage, I
> have prepared the following configuration with an SSD-cache and HDD-cache.
> 
> http {
> 
>    add_header X-UpStream-Server-Cache-Status $upstream_cache_status;
> 
>    # proxy caching configurations
>    proxy_cache_path /tmp/myssd1 keys_zone=myssd1:1m levels=1:2
> inactive=60m;
> 
>    proxy_cache_path /tmp/myhdd1 keys_zone=myhdd1:10m levels=1:2
> inactive=60m;
> 
>    proxy_cache_key "$request_uri";
> 
>    upstream mylocalhdd {
>        server 127.0.0.1:82;
>    }
> 
>    upstream myorigin {
>        server 192.168.1.35:80;
>    }
> 
>    # Server-block1 - will cache in SSD if we get a minimum of 5 requests.
>    server {
>        listen       80;
>        server_name  example.com www.example.com;
> 
>        # Look up in ssd cache. If not found, goto hdd cache.
>        location / {
>            proxy_pass http://mylocalhdd;
>            proxy_cache myssd1;
>            proxy_cache_min_uses 5;
>        }
>    }
> 
>    # Server-block2 (acting as local upstream) - will fetch from origin and
> cache in HDD
>    server {
>        listen       82;
>        server_name  example.com www.example.com;
> 
>        # Look up in hdd cache. If not found, goto origin
>        location / {
>            proxy_pass http://myorigin;
>            proxy_cache myhdd1;
>        }
>    }
> 
> }
> 
> The smaller, yet faster SSD-cache will store content only if I get at least
> 5 requests for the URL. On the other hand, the larger HDD cache will every
> request.
> 
> The flow is straight-forward:
> 
> i) Client's first request is handled by server at port 80
> ii) It looks in SSD cache. Upon a cache-miss, it proxies to local upstream
> at port 82.
> iii) The local server at port 82, looks in HDD cache. upon cache miss, it
> fetches from origin, and adds to HDD-cache and sends the response back to
> first server.
> iv) Server1 does not add content yet to SSD-cache (as min_uses is not
> reached), and sends response to client.
> v) For the next 3 requests, the server1 would see an SSD-cache-miss, but
> server2 produces an HDD-cache-hit.
> vi) For the 5th request, the server1, will also add the response in
> SSD-cache, as min_uses criteria is met.
> vii) For the 6th request onwards, the server1 itself will serve the request
> from SSD-cache itself. No request is sent to local upstream.
> 
> I have added $upstream_cache_status in the response.
> 
> For the first request, I see the header twice in the response: This seem to
> correspond to MISS in both front-end SSD-cache and back-end HDD-cache.
> 
> < X-UpStream-Server-Cache-Status: MISS
> < X-UpStream-Server-Cache-Status: MISS
> 
> For the next 4 requests, I see the following: This seems to correspond to
> MISS in the from-end SSD-cache and HIT in the back-end HDD-cache.
> 
> < X-UpStream-Server-Cache-Status: HIT
> < X-UpStream-Server-Cache-Status: MISS
> 
> For the 6th request, I see the following:
> 
> < X-UpStream-Server-Cache-Status: HIT
> < X-UpStream-Server-Cache-Status: HIT
> 
> Why am I getting the header twice for the 6th request. In this case, the
> request is HIT by the SSD cache itself, and there is no request sent to
> local upstream also.
> 
> So, shouldn't I be getting only one instance of the header in the response?
> 
> Thanks
> Rajesh
> 
> Posted at Nginx Forum: https://forum.nginx.org/read.php?2,276727,276727#msg-276727
> 
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx