nginx compression-at-edge (in front of a reverse proxy) compresses some content, not other ?

[Igor Sysoev]

On Wed, Apr 28, 2010 at 08:33:03AM -0700, Ben DJ wrote:

> > BTW, why do you use varnish instead of nginx built-in proxy cache ?
> 
> I'd originally had Pound in front of varnish;  Nginx was a logical
> choice to replace it, as a first step.
> 
> As for using it as a proxy cache ... Varnish provides all in-memory
> caching.  I simply don't know if nginx.

nginx uses file cache as persistent storage, however, once a response
has been served from nginx cache, the response is cached in OS VM cache,
and then nginx does not read from disk. If your hot content fits in
a host memory it will be served from OS VM cache without disk access as
in-memory cache. nginx usually uses sendfile() to send the cached responses,
so only one copy of file present in the host memory.

Varnish mmap()s a large file and stores responses in the mmap()ed memory,
i.e. in this file. However, since it mmap()s using MAP_NOSYNC flags,
so the stored data are not usually written to disk. The data will be
flushed to disk if there is memory shortage. Varnish uses sendfile()
too, so it also has a single copy of each response in the memory.
Varnish does not store cached responses just in memory (without file mmap),
because it can not use sendfile() in this case and then some responses
will be storage in OS memory in several copies: one in Varnish memory cache
and several in network buffers.

> Also, I use Varnish to provide a content-based parallel CDN.  Attempts
> to do it with nginx failed, and, it appeared others were having issue
> as well (http://groups.drupal.org/node/48182).  The nginx + varnish
> solution works well.

I did not understand what that parallel CDN does and what problem you
have encountered.

> > Also, "gzip_comp_level 1" is enough. 9th level simply eats more CPU, but
> > do not produce proportional cmpression ratio. Say, 1 decreases file 2 times
> > while 9 - only 2.5, but not 5 times.
> 
> I didn't realize that the difference was _that_ small ...

Try "gzip -1 some.js" and "gzip -9 some.js" to see the difference.
Also you may try to gzip a large file to see time difference.


-- 
Igor Sysoev
http://sysoev.ru/en/