gRPC reverse proxy closed tcp connection after 1000 rpc calls
jetchars
nginx-forum at forum.nginx.org
Wed Mar 6 14:20:03 UTC 2019
Hey, genius
I've followed the official user guide to create a gRPC reverse proxy, config
as follow:
```
upstream grpcservers {
server 10.90.62.50:60080 weight=3;
server 10.90.62.51:60080 weight=3;
server 10.90.62.52:60080 weight=3;
keepalive 2000;
keepalive_timeout 120;
keepalive_requests 100000;
}
server {
listen 8051 http2;
error_log /home/nginx/log/s4_mongo_error.log;
access_log /home/nginx/log/s4_mongo_access.log;
grpc_socket_keepalive on;
location / {
grpc_pass grpc://grpcservers;
error_page 502 = /error502grpc;
}
location = /error502grpc {
internal;
default_type application/grpc;
add_header grpc-status 14;
add_header grpc-message "unavailable";
return 204;
}
}
```
- nginx keepalive with upstream servers successfully
- but when keepalive with gRPC client, after 1000 requests been processed,
nginx will close the tcp connection, because I can find `TIME_WAIT` on the
nginx side.
- then gRPC client will report tens of thousands of `TransientFailure` at
same time
Posted at Nginx Forum: https://forum.nginx.org/read.php?2,283297,283297#msg-283297
More information about the nginx
mailing list