Problem with TLS handshake in some browsers when OCSP stapling enabled

[Maxim Dounin]

Hello!

On Thu, Dec 12, 2013 at 11:59:26AM +0400, kyprizel wrote:

> Hi,
> we got a problem with OCSP stapling.
> 
> During the handshake some browsers send TLS extension "certificate status"
> with more than 5 bytes in it.
> In Nginx error_log it looks like:
> 
> [crit] 8721#0: *35 SSL_do_handshake() failed (SSL: error:0D0680A8:asn1
> encoding routines:ASN1_CHECK_TLEN:wrong tag error:0D08303A:asn1 enco
> ding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error error:1408A0E3:SSL
> routines:SSL3_GET_CLIENT_HELLO:parse tlsext) while SSL handshaking, client:
> 
> If we disable OCSP stapling - everything works fine. Looks like the problem
> is on the browser side and in OpenSSL tls ext parsing function. But can we
> make it just ignore the incorrect (?) tls extension than dropping SSL
> hanshake?

I don't think it's possible to do anything in nginx here.  Try 
looking at the relevant OpenSSL code - if the server status 
callback is set, it parses the extension, and if a parsing error 
happens - the error is returned.

It should be possible to work it around in OpenSSL code though.

-- 
Maxim Dounin
http://nginx.org/