NGINx async SSL handshake

Vakul Garg vakul.garg at
Mon Jan 9 07:51:35 UTC 2017

Hi Brian

We are using your nginx patch with our PKC accelerator (exposed through cryptodev-linux ioctls).
The OpenSSL cryptodev engine has been enhanced to make async PKC calls.

The system is running functionally well, but the performance is poor.

Last year, we used older OpenSSL versions (without ASYNC job infrastructure) with Geoff Thorpe's to achieve async crypto acceleration.
The performance was better than we are getting with OpenSSL-1.1.0 + Nginx patch in question.

On studying the Intel nginx changes, I feel that the way the asynchronous APIs have been integrated are somewhat heavy and there is possibility of improvement.
Specifically, the eventfd based job wakeup mechanism seems heavy.

I am looking towards making changes over your nginx patch to introduce a newer job wakeup method.
Further, I want to get rid of creating set of accelerator response poll threads in engine and provide an option to do this inline in nginx worker thread itself.

For this, I hoped your patches are submitted upstream and in process of review 
I can find your work in some github repo where we can submit our changes.

Let me know your comments.



-----Original Message-----
From: Will, Brian [mailto:brian.will at] 
Sent: Saturday, January 07, 2017 2:40 AM
To: Vakul Garg <vakul.garg at>
Cc: nginx-devel at; Michael Kardonik <michael.kardonik at>; Schuetze, Joel D <joel.d.schuetze at>; Yu, Ping Y <ping.y.yu at>; Finn, Coleman <coleman.finn at>
Subject: RE: NGINx async SSL handshake

Hello Vakul,
The link you have included is the latest patch for NGINX to support OpenSSL-1.1.0 that we are maintaining. We are working to try and get these changes included into the mainline of NGINX, but in the interim this patch should work for the OpenSSL-1.1.0 branch of code.
Were there specific issues that you have run into?


-----Original Message-----
From: Vakul Garg [mailto:vakul.garg at] 
Sent: Friday, January 06, 2017 12:10 PM
To: Will, Brian <brian.will at>
Cc: nginx-devel at; Michael Kardonik <michael.kardonik at>
Subject: RE: NGINx async SSL handshake

Hi Brian

I am using nginx openssl async: with my async RSA accelerator.
Where can I find the updated version of this work?

Can you share your plans to upstream this work?



-----Original Message-----
From: nginx-devel [mailto:nginx-devel-bounces at] On Behalf Of Alexey Ivanov
Sent: Saturday, November 19, 2016 2:00 PM
To: nginx-devel at
Cc: brian.will at
Subject: Re: NGINx async SSL handshake

+Brian Will from Intel, to correct me if I'm wrong.

My two cents here(Intel Quick Assist specific, based on conversations during Nginx.Conf):
1) Even without hardware offload async handshake helps in cases where you have high TLS connection rates, because right now handshake is basically a 2ms+ blocking operation(specific timing depend on AVX/AVX2 support[0]) inside the event loop. Therefore after some TLS connection rate nginx performance falls off the cliff.
2) Hardware offload numbers look very impressive[1](TL;DR: 5x improvement for RSA 2048, for ECDSA, imho, it is neither impressive, nor needed). Also they prove asymmetric part of the accelerator to be future proof, so that it is possible to add new handshake types(e.g. Ed25519). Disclaimer: we did not test that hardware yet.

As for patches, you can check for:
a) nginx openssl async:
b) zlib[2]:
(Full list of docs: )

Question for Brian/Maxim: are you planning on integrating it into mainline nginx? 1000+ line diffs are usually rather hard to integrate.

[0] FWIW, speaking about OpenSSL performance: using OpenSSL 1.0.2 + Intel Xeon v2 processors with AVX2 gives 2x performance boost(over OpenSSL 1.0.1 and v1).
[2] There are also cloudflare and intel patches for zlib for faster deflation (i.e. compression only)
> On Nov 18, 2016, at 8:12 PM, Vakul Garg <vakul.garg at> wrote:
> Hi
> I am a newbie to nginx.
> I am integrating a public key hardware accelerator in OpenSSL using engine interface.
> The engine is async capable.
> Recently openssl-1.1.0 has added support for ASYNC_JOB.
> IIUC, nginx would also require changes in order to do SSL handshake in async way.
> Any pointers where can I get the nginx code changes done for async openssl
> Regards
> Vakul
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at

More information about the nginx-devel mailing list