Websocket tunnel broken with existing SSL session

[Maxim Dounin]

Hello!

On Tue, Jan 14, 2014 at 04:22:44PM +0000, Eiríkur Nilsson wrote:

> We've been debugging this issue for 3 days now and even though we have a
> temporary fix, we're still puzzled about it.
> 
> There is an iOS app, which opens a websocket connection to our server over
> SSL. Our server runs SmartOS and has nginx 1.5.0 (also happens on 1.4.1)
> proxying to a backend server running in NodeJS.
> 
> To reproduce, I start my app, a websocket connection is established and
> works well, then I put the app to sleep for awhile until nginx kills the
> connection. When I reopen the app, the following happens:
> 
> 1) App notices that the connection is dead and reconnects.
> 2) Behind the scenes, iOS reuses the SSL session from before and quickly
> opens a new socket.
> 3) A HTTP upgrade request and response flow across with no problems.
> 4) With a successful web-socket established on both sides, the client
> starts sending frames. However, none of these gets delivered to the backend
> server.
> 5) After a minute, nginx kills the connection even though the client is
> sending periodic pings.
> 6) Back to 1.
> 
> I haven't managed to reduce the test case or reproduce it in another
> environment yet. This only happens when using SSL. In wireshark I see the
> websocket frames being sent from the iPhone client and TCP acked properly.
> 
> What currently fixes the problem is to disable SSL session reuse in nginx.
> Then every websocket connection works like it should.
> 
> Here is the config before the fix:
> ###
>     server {
>         ### Server port and name ###

[...]

Which event method is used?  If eventport, try switching to 
/dev/poll instead (which is expected to be used by default on 
SmartOS and other Solaris variants), it should fix the issue.  The 
eventport event method is known to have problems when proxying and 
this may cause symptoms you see, it needs attention.

-- 
Maxim Dounin
http://nginx.org/