Redirect on specific threshold !!

[Francis Daly]

On Mon, Jun 15, 2015 at 01:45:42PM +0300, Valentin V. Bartenev wrote:
> On Sunday 14 June 2015 22:12:37 shahzaib shahzaib wrote:

Hi there,

> > If there are exceeding 1K requests for http://storage.domain.com/test.mp4 ,
> > nginx should  construct a Redirect URL for rest of the requests related to
> > test.mp4 i.e http://cache.domain.com/test.mp4 and entertain the rest of
> > requests for test.mp4 from Caching Node while long tail would still be
> > served from storage.

> You can use limit_conn and limit_req modules to set limits:
> http://nginx.org/en/docs/http/ngx_http_limit_conn_module.html
> http://nginx.org/en/docs/http/ngx_http_limit_req_module.html
> 
> and the error_page directive to construct the redirect.

limit_conn and limit_req are the right answer if you care about concurrent
requests.

(For example: rate=1r/m with burst=1000 might do most of what you want,
without too much work on your part.)

I think you might care about historical requests, instead -- so if a
url is ever accessed 1K times, then it is "popular" and future requests
should be redirected.

To do that, you probably will find it simpler to do it outside of nginx,
at least initially.

Have something read the recent-enough log files[*], and whenever there are
more that 1K requests for the same resource, add a fragment like

  location = /test.mp4 { return 301 http://cache.domain.com/test.mp4; }

to nginx.conf (and remove similar fragments that are no longer currently
popular-enough, if appropriate), and do a no-downtime config reload.

You can probably come up with a module or a code config that does the
same thing, but I think it would take me longer to do that.


[*] or accesses the statistics by a method of your choice

	f
-- 
Francis Daly        francis at daoine.org