Websocket tunnel broken with existing SSL session

[Eiríkur Nilsson]

Thanks for the suggestion.

We're not specifying a event method currently, though I can't see if
eventport was the default. If I explicitly use /dev/poll, and turn
ssl_session_cache back on, the issue comes back.

I have verified that both the HTTP request and response are proxied
properly. It seems to me that when the upgrade is finished nginx enters
direct tunneling mode for the websocket data, which doesn't work for some
sockets, at least these recovered SSL sessions from iOS clients.

The event method issue would have explained why I can't reproduce the issue
on mac (with self signed cert). I also haven't reproduced it with an
Android client, although I did not verify with tcpdump if my android test
reused the SSL session.

Any other ideas?

- Eirikur




On Tue, Jan 14, 2014 at 10:53 PM, Maxim Dounin <mdounin at mdounin.ru> wrote:

> Hello!
>
> On Tue, Jan 14, 2014 at 04:22:44PM +0000, Eiríkur Nilsson wrote:
>
> > We've been debugging this issue for 3 days now and even though we have a
> > temporary fix, we're still puzzled about it.
> >
> > There is an iOS app, which opens a websocket connection to our server
> over
> > SSL. Our server runs SmartOS and has nginx 1.5.0 (also happens on 1.4.1)
> > proxying to a backend server running in NodeJS.
> >
> > To reproduce, I start my app, a websocket connection is established and
> > works well, then I put the app to sleep for awhile until nginx kills the
> > connection. When I reopen the app, the following happens:
> >
> > 1) App notices that the connection is dead and reconnects.
> > 2) Behind the scenes, iOS reuses the SSL session from before and quickly
> > opens a new socket.
> > 3) A HTTP upgrade request and response flow across with no problems.
> > 4) With a successful web-socket established on both sides, the client
> > starts sending frames. However, none of these gets delivered to the
> backend
> > server.
> > 5) After a minute, nginx kills the connection even though the client is
> > sending periodic pings.
> > 6) Back to 1.
> >
> > I haven't managed to reduce the test case or reproduce it in another
> > environment yet. This only happens when using SSL. In wireshark I see the
> > websocket frames being sent from the iPhone client and TCP acked
> properly.
> >
> > What currently fixes the problem is to disable SSL session reuse in
> nginx.
> > Then every websocket connection works like it should.
> >
> > Here is the config before the fix:
> > ###
> >     server {
> >         ### Server port and name ###
>
> [...]
>
> Which event method is used?  If eventport, try switching to
> /dev/poll instead (which is expected to be used by default on
> SmartOS and other Solaris variants), it should fix the issue.  The
> eventport event method is known to have problems when proxying and
> this may cause symptoms you see, it needs attention.
>
> --
> Maxim Dounin
> http://nginx.org/
>
> _______________________________________________
> nginx mailing list
> nginx at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20140115/62c0db5b/attachment.html>