[PATCH] Auth basic: Cache credentials if auth_basic_user_file is static

[Toshihito Kikuchi]

Hi,

On Tue, Oct 3, 2023 at 6:03 PM Maxim Dounin <mdounin at mdounin.ru> wrote:
>
> Hello!
>
> On Tue, Oct 03, 2023 at 03:46:05PM -0700, Toshihito Kikuchi wrote:
>
> > # HG changeset patch
> > # User Toshihito Kikuchi <leamovret at gmail.com>
> > # Date 1696359541 25200
> > #      Tue Oct 03 11:59:01 2023 -0700
> > # Node ID e397ea6cfa85e85ae0865c5061397dc295fb7df1
> > # Parent  3db945fda515014d220151046d02f3960bcfca0a
> > Auth basic: Cache credentials if auth_basic_user_file is static.
> >
> > In the current design, when auth_basic is on, every HTTP request triggers
> > file I/O (open, read, close) to the file specified in auth_basic_user_file.
> > Probably this is to allow auth_basic_user_file to contain variables.
> >
> > If the value is just a static text, however, there is no reason to read the
> > same file every request in every worker process.  It unnecessarily consumes
> > system resources.
> >
> > With this patch, if auth_basic_user_file does not have any variables, we
> > cache its content in the location context at configuration time and use it
> > in all subsequent requests.  If auth_basic_user_file contain variables, we keep
> > the original behavior.
>
> As currently implemented, auth_basic_user_file is read at runtime,
> making it possible to change users and their passwords - which is
> a relatively common task - without reloading nginx itself.  And
> this behaviour matches the one in Apache, which does the same.
> Changing this behaviour to read the password file while loading
> configuration (so any changes to the file won't be applied unless
> nginx is reloaded) would certainly break POLA, and needs some
> really good justification.

Thank you for your comments!  I understood the risk of changing the default
behavior.  Then the right approach would be to introduce a new directive such
as "auth_basic_preload_mode" for users to opt in.  Does that make sense?

For justification, let me explain our use case.  We're a blockchain node runner
hosting private endpoints to multiple blockchains such as Ethereum, Polygon,
and etc.  Technically each blockchain is a JSON-RPC server over HTTP, and we put
them behind Nginx with Basic Authentication to prevent free riders from using
our endpoints for free.

Keeping the best performance is crucial for our business.  We are observing
performance improvement after we deployed this patch to our infrastructure.
We believe this patch benefits other blockchain node runners and the industry.

>
> Further, in typical setups the file is effectively cached by the
> OS itself, making the I/O operations mentioned almost free,
> especially compared to costs of typical password hash
> calculations.

In our use case, prioritizing performance over the freerider risk, we use
the plaintext format in our credential file.  File I/O may be ignorable
compared to password hash calculations, but we cannot say it's almost free
because it still triggers context switches to execute syscalls, which is
expensive.

Thanks,
Toshihito Kikuchi