upstream 429 and non-idempotent request

[Frank Liu]

Hi,

I fully understand the concern and complexity of different cases. Making
any default assumption will have risks. That's why I suggested providing
config options since users themselves know their use case and whether it is
safe to retry.

Thanks!
Frank




On Tue, Jun 13, 2017 at 9:30 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Thu, Jun 08, 2017 at 09:55:05AM -0700, Frank Liu wrote:
>
> > I fully understand the rationale of not retrying non-idempotent requests
> if
> > they are already sent, but in case of 429 (maybe other cases as well), I
> > don't see an issue of retrying even if request is sent. It would be
> better
> > if we can selectively do something like "proxy_next_upstream
> > non-idempotent-http_429;" or whatever http code that we know safe.
>
> The problem is that we don't really know if the code is safe or
> not.
>
> In general, non-idempotent requests are not retried as we
> don't know if the request was processed or not.  The situation is
> obvious when we got a network error after sending a request: as
> the error might happen after the request was already processed, it
> is wise to refrain from retrying it.
>
> But what happen when we got a valid http error instead?  For
> example, 502 in most cases means that there was a network error
> somewhere else.  So, this basically means that 502 cannot be
> retried as well.  And the other 5xx error codes are more or less
> identical: we never know what really happened, and can't retry.
>
> With 4xx errors it seems safe to assume that the request was not
> processed, and hence retrying is possible.  But, for example, if
> error_page is used on the backend server, 404 might be returned
> if a network error happens and a corresponding error page cannot
> be found.  Similarly, 429 might be returned if limit_req rejects a
> request to the error page.
>
> We've considered adding a logic to always retry non-idempotent
> requests in case of 4xx errors when non-idempotence handing was
> introduced.  But decided to keep things simple and safe, and never
> retry non-idempotent requests.  On the other hand, introducing
> configuration options to fine tune if non-idempotent requests
> should be retried for each proxy_next_upstream case seems to be
> overkill.
>
> In general, current implementation assumes the following two
> options:
>
> - non-idempotent requests are not retried;
>
> - there is a duplicate request protection in the application, so
>   non-dempotent requests can be retried with "proxy_next_upstream
>   non_idempotent;".
>
> This seems to be enough for most, if not all, use cases.  If
> something more complex is really needed, it can be configured
> using error_page and additional error processing logic.
>
> --
> Maxim Dounin
> http://nginx.org/
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170613/eb9a7250/attachment.html>