gRPC reverse proxy closed tcp connection after 1000 rpc calls

jetchars nginx-forum at
Wed Mar 6 14:20:03 UTC 2019

Hey, genius

I've followed the official user guide to create a gRPC reverse proxy, config
as follow:

upstream grpcservers {
    server weight=3;
    server weight=3;
    server weight=3;
    keepalive 2000;
    keepalive_timeout 120;
    keepalive_requests 100000;

server {
    listen       8051 http2;
    error_log /home/nginx/log/s4_mongo_error.log;
    access_log /home/nginx/log/s4_mongo_access.log;
    grpc_socket_keepalive on;
    location / {
        grpc_pass grpc://grpcservers;
        error_page 502 = /error502grpc;

    location = /error502grpc {
        default_type application/grpc;
        add_header grpc-status 14;
        add_header grpc-message "unavailable";
        return 204;

- nginx keepalive with upstream servers successfully
- but when keepalive with gRPC client, after 1000 requests been processed,
nginx will close the tcp connection, because I can find `TIME_WAIT` on the
nginx side.
- then gRPC client will report tens of thousands of `TransientFailure` at
same time

Posted at Nginx Forum:,283297,283297#msg-283297

More information about the nginx mailing list