Memory usage in nginx proxy setup and use of min_uses

[Maxim Dounin]

Hello!

On Mon, May 17, 2021 at 02:47:33PM +0000, Lucas Rolff wrote:

> Hi Maxim,
> 
> Thanks a lot for your reply!
> 
> I'm indeed aware of the ~8k keys per mb of memory, I was just 
> wondering if it was handled differently when min_uses are in 
> use, but it does indeed make sense that nginx has to keep track 
> of it somehow, and the keys zone makes the most sense!
> 
> > Much like with any cache item, such keys are removed from the 
> > keys_zone if no matching requests are seen during the 
> > "inactive" time
> 
> That's a bummer, since that still allows memory "poisoning" - it 
> would be awesome to have another flag for proxy_cache_path to 
> control how long keys that have not yet reached min_uses are 
> kept in SHM.
> The benefit of this would be to say if min_uses have not been 
> reached within let's say 5 minutes, then we purge those keys 
> from SHM to clear up the memory.
> 
> For controlling the cache items - ideally we wanna use query 
> strings as a part of the cache key, but still ideally prevent 
> memory poisoning as above - the inactive flag for min_uses would 
> be pretty useful for this - while it won't prevent it fully, 
> we'd still be able to somewhat control memory even if people are 
> trying to do the cache/memory poisoning.

In no particular order:

- The attack you are considering is not about "poisoning".  At 
  most, it can be used to make the cache less efficient.

- The goal "to somewhat control memory" looks confusing: the 
  memory used by caching is hard-limited by the keys_zone size, 
  and it is not possible to use more memory than configured.  At 
  most, you can try to limit the number of keys an attacker will be 
  able to put into keys_zone.  But using separate inactive timer for 
  keys not reached min_uses won't help here: an attacker who is able 
  to do arbitrary amount of requests will be able to flush all cache 
  items anyway.

- Using proxy_cache_min_uses cannot help here, regardless of how 
  it is handled, since nothing stops the attacker from requesting 
  the same resource multiple times.

In general, I see two basic options to handle things if you don't 
want one to be able to reduce your cache efficiency:

1. Strictly limit which resources are to be cached, in particular, 
by using appropriate proxy_cache_key, as already suggested.

2. Limit the maximum number of requests an attacker can do, so it 
won't be able to cause noticeable degradation of cache efficiency.  
In particular, this can be done with limit_req 
(http://nginx.org/r/limit_req).

Also it is always a good idea to make sure your site works fine 
without caching at all.

-- 
Maxim Dounin
http://mdounin.ru/