cache: move open to thread pool

[Ka-Hing Cheung]

I spoke too soon, just realized our test colo is running the patches
without aio_open on. Flipping that switch now.

Also, including the patch that we applied on top in addition to the
massaging we did to resolve conflicts. I haven't dug too deep to see
if stock nginx also requires similar changes or they are only
necessary because of our other nginx changes:

--- src/http/ngx_http_file_cache.c
+++ src/http/ngx_http_file_cache.c
@@ -611,6 +611,12 @@ ngx_http_do_file_cache_open(ngx_http_request_t
*r, unsigned ignore_vary)
         return NGX_ERROR;
     }

+    c->buf = ngx_create_temp_buf(r->pool, c->body_start);
+    if (c->buf == NULL) {
+        return NGX_ERROR;
+    }
+    c->file.fd = NGX_INVALID_FILE;
+
     if (!test) {
         /*
          * Either the request is cacheable but the file doesn't
@@ -1148,6 +1154,7 @@ ngx_http_file_cache_aio_open(ngx_http_request_t
*r, ngx_http_cache_t *c)
         ngx_memzero(&r->open_file_info, sizeof(ngx_open_file_info_t));

         r->open_file_info.uniq = c->uniq;
+        r->open_file_info.fd = NGX_INVALID_FILE;
         r->open_file_info.valid = clcf->open_file_cache_valid;
         r->open_file_info.min_uses = clcf->open_file_cache_min_uses;
         r->open_file_info.events = clcf->open_file_cache_events;
@@ -1205,11 +1212,6 @@ ngx_http_file_cache_aio_open(ngx_http_request_t
*r, ngx_http_cache_t *c)
     c->length = r->open_file_info.size;
     c->fs_size = (r->open_file_info.fs_size + cache->bsize - 1) / cache->bsize;

-    c->buf = ngx_create_temp_buf(r->pool, c->body_start);
-    if (c->buf == NULL) {
-        return NGX_ERROR;
-    }
-
     return NGX_OK;
 }

On Tue, Jan 22, 2019 at 10:20 AM Ka-Hing Cheung <kahing at cloudflare.com> wrote:
>
> Hi Maxim and Roman,
>
> We've been running a version of your patches in our test colo for the
> past week with no ill effects, with the caveat that it's difficult to
> measure performance impacts in our test colo because of varying
> traffic. We had to massage the patches a bit because we don't run
> vanilla nginx. Also we've only tried with open file cache disabled.
>
> Let me know if you are looking for other feedback.
>
> - Ka-Hing
>
> On Mon, Jan 14, 2019 at 5:53 PM Ka-Hing Cheung <kahing at cloudflare.com> wrote:
> >
> > We didn't get to it last year because of other priorities but this is
> > being worked on. We don't use a vanilla nginx so it takes effort to
> > integrate your patches. Having a version of this in upstream nginx is
> > important to us and to me personally.
> >
> > On Thu, Jan 10, 2019 at 12:30 AM Maxim Konovalov <maxim at nginx.com> wrote:
> > >
> > > So guys, are you really interested in this work beyond shiny
> > > marketing blog posts?
> > >
> > > On 05/12/2018 12:43, Maxim Konovalov wrote:
> > > > Hello,
> > > >
> > > > just a reminder that we are looking for a tester for these patches.
> > > >
> > > > Thanks,
> > > >
> > > > Maxim
> > > >
> > > > On 16/11/2018 12:10, Maxim Konovalov wrote:
> > > >> Thanks, Ka-Hing, we'll wait for your feedback.
> > > >>
> > > >> On 16/11/2018 01:53, Ka-Hing Cheung wrote:
> > > >>> Hi,
> > > >>>
> > > >>> I didn't forget about this. I am pretty swamped at the moment and
> > > >>> there's a holiday freeze coming up. Will get to his in December.
> > > >>>
> > > >>> - Ka-Hing
> > > >>>
> > > >>> On Thu, Nov 8, 2018 at 6:19 AM Maxim Konovalov <maxim at nginx.com> wrote:
> > > >>>>
> > > >>>> Hi Ka-Hing,
> > > >>>>
> > > >>>> would you mind to test Roman's most recent patches that add
> > > >>>> "aio_open" directive?
> > > >>>>
> > > >>>> http://mailman.nginx.org/pipermail/nginx-devel/2018-November/011538.html
> > > >>>>
> > > >>>> We are looking for overall performance and stability metrics and
> > > >>>> feedback.
> > > >>>>
> > > >>>> Much appreciated,
> > > >>>>
> > > >>>> Maxim
> > > >>>>
> > > >>>> --
> > > >>>> Maxim Konovalov
> > > >>
> > > >>
> > > >
> > > >
> > >
> > >
> > > --
> > > Maxim Konovalov
> > > _______________________________________________
> > > nginx-devel mailing list
> > > nginx-devel at nginx.org
> > > http://mailman.nginx.org/mailman/listinfo/nginx-devel