cache: move open to thread pool
Eran Kornblau
eran.kornblau at kaltura.com
Wed Aug 8 19:44:20 UTC 2018
Hi,
>
> - The code bypasses open file cache, and uses a direct call
> in the http cache code instead. While it might be ok in your
> setup, it looks like an incomplete solution from the generic point
> of view. A better solution would be to introduce a generic
> interface in ngx_open_cached_file() to allow use of thread
> pools.
>
A small comment on this - I wrote such an implementation a while ago
in my module. We've been using it on production for the last ~3 years.
Code is here -
https://github.com/kaltura/nginx-vod-module/blob/master/ngx_async_open_file_cache.c
(There's a lot of code there that was copied from ngx_open_file_cache.c,
I did that since those functions are static, and didn't want to change nginx core.
If implemented as part of nginx core, all this duplication can be avoided...)
In my implementation, I added a function similar to ngx_open_cached_file -
(ngx_async_open_cached_file), that gets a few extra params -
1. The thread pool
2. The thread task (optional) - this was done in order to reuse the task
when opening multiple files in a single request
3. Callback + context - invoked once the async open completes
This function first checks the cache, if there's a hit, it returns synchronously
(NGX_OK).
Otherwise, it posts a task that does ngx_open_and_stat_file (NGX_AGAIN).
When the task completes, the main nginx thread updates the cache, and invokes
the user callback.
Hope this helps...
Eran
More information about the nginx-devel
mailing list