NGINx async SSL handshake

Fri Jan 13 08:51:40 UTC 2017

Hi Brian

Yes, one of the things I am looking for is a replacement for engine private receive thread with "poll inline" from nginx worker itself.
It might not necessarily require timer based mechanism to get the nginx event loop to poll accelerator responses.

The requirement is to give some time slice to the engine to do receive side processing from the accelerator.
This time slice can be given after completing ready list of FDs in event loop.

If any SSL connection is pending on WANT_ASYNC, then event poll loop should wakeup with zero timeout, because we know accelerator response would come soon and it makes sense to give timeslice for engine accelerator response processing.
For engine accelerator response processing an ENGINE control api can be built and directly called from nginx event loop.

Please let me know your comments.

Regards

Vakul

-----Original Message-----
From: Will, Brian [mailto:brian.will at intel.com] 
Sent: Friday, January 13, 2017 4:54 AM
To: Vakul Garg <vakul.garg at nxp.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>; Schuetze, Joel D <joel.d.schuetze at intel.com>; Yu, Ping Y <ping.y.yu at intel.com>; Finn, Coleman <coleman.finn at intel.com>; Pankaj Gupta <pankaj.gupta at nxp.com>; Arun Pathak <arun.pathak at nxp.com>; Chambers, Mark A <mark.a.chambers at intel.com>
Subject: RE: NGINx async SSL handshake

Hi Vakul,
Sorry for not getting back sooner. We are currently in the process of working with NGINX to get our patch integrated into the development tree. This process may result in a very different patch to what we have currently published, resulting in closer integration and fit into the infrastructure. As of right now we do not have the patch posted either as a repo or a review request. 

Your work on the wakeup mechanism would be very interesting, specifically what are you looking at as a replacement for the activation thread which the engine would own? On the option to "poll inline" from the nginx worker itself this is something we had explored and does result in increased performance, but then you would require some timer based mechanism to get the NGINX event loop to poll the SSL* API in question. This can be added and is supported through the current OpenSSL async features, but seemed like a different processing model to the way NGINX handles all other activity.

Thanks,
Brian

-----Original Message-----
From: Vakul Garg [mailto:vakul.garg at nxp.com]
Sent: Monday, January 09, 2017 12:52 AM
To: Will, Brian <brian.will at intel.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>; Schuetze, Joel D <joel.d.schuetze at intel.com>; Yu, Ping Y <ping.y.yu at intel.com>; Finn, Coleman <coleman.finn at intel.com>; Pankaj Gupta <pankaj.gupta at nxp.com>; Arun Pathak <arun.pathak at nxp.com>
Subject: RE: NGINx async SSL handshake

Hi Brian

We are using your nginx patch with our PKC accelerator (exposed through cryptodev-linux ioctls).
The OpenSSL cryptodev engine has been enhanced to make async PKC calls.

The system is running functionally well, but the performance is poor.

Last year, we used older OpenSSL versions (without ASYNC job infrastructure) with Geoff Thorpe's https://github.com/libfibre to achieve async crypto acceleration.
The performance was better than we are getting with OpenSSL-1.1.0 + Nginx patch in question.

On studying the Intel nginx changes, I feel that the way the asynchronous APIs have been integrated are somewhat heavy and there is possibility of improvement.
Specifically, the eventfd based job wakeup mechanism seems heavy.

I am looking towards making changes over your nginx patch to introduce a newer job wakeup method.
Further, I want to get rid of creating set of accelerator response poll threads in engine and provide an option to do this inline in nginx worker thread itself.

For this, I hoped your patches are submitted upstream and in process of review or I can find your work in some github repo where we can submit our changes.

Let me know your comments.

Regards

Vakul

-----Original Message-----
From: Will, Brian [mailto:brian.will at intel.com]
Sent: Saturday, January 07, 2017 2:40 AM
To: Vakul Garg <vakul.garg at nxp.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>; Schuetze, Joel D <joel.d.schuetze at intel.com>; Yu, Ping Y <ping.y.yu at intel.com>; Finn, Coleman <coleman.finn at intel.com>
Subject: RE: NGINx async SSL handshake

Hello Vakul,
The link you have included is the latest patch for NGINX to support OpenSSL-1.1.0 that we are maintaining. We are working to try and get these changes included into the mainline of NGINX, but in the interim this patch should work for the OpenSSL-1.1.0 branch of code.
Were there specific issues that you have run into?

Thanks,
Brian

-----Original Message-----
From: Vakul Garg [mailto:vakul.garg at nxp.com]
Sent: Friday, January 06, 2017 12:10 PM
To: Will, Brian <brian.will at intel.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>
Subject: RE: NGINx async SSL handshake

Hi Brian

I am using nginx openssl async: https://01.org/sites/default/files/page/nginx-1.10.0-async.l.0.3.0-001_0.tgz with my async RSA accelerator.
Where can I find the updated version of this work?

Can you share your plans to upstream this work?

Regards

Vakul

-----Original Message-----
From: nginx-devel [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Alexey Ivanov
Sent: Saturday, November 19, 2016 2:00 PM
To: nginx-devel at nginx.org
Cc: brian.will at intel.com
Subject: Re: NGINx async SSL handshake

+Brian Will from Intel, to correct me if I'm wrong.

My two cents here(Intel Quick Assist specific, based on conversations during Nginx.Conf):
1) Even without hardware offload async handshake helps in cases where you have high TLS connection rates, because right now handshake is basically a 2ms+ blocking operation(specific timing depend on AVX/AVX2 support[0]) inside the event loop. Therefore after some TLS connection rate nginx performance falls off the cliff.
2) Hardware offload numbers look very impressive[1](TL;DR: 5x improvement for RSA 2048, for ECDSA, imho, it is neither impressive, nor needed). Also they prove asymmetric part of the accelerator to be future proof, so that it is possible to add new handshake types(e.g. Ed25519). Disclaimer: we did not test that hardware yet.

As for patches, you can check 01.org for:
a) nginx openssl async: https://01.org/sites/default/files/page/nginx-1.10.0-async.l.0.3.0-001_0.tgz
b) zlib[2]: https://01.org/sites/default/files/page/zlib_shim_0.4.9-001.tgz
(Full list of docs: https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches )

Question for Brian/Maxim: are you planning on integrating it into mainline nginx? 1000+ line diffs are usually rather hard to integrate.

[0] FWIW, speaking about OpenSSL performance: using OpenSSL 1.0.2 + Intel Xeon v2 processors with AVX2 gives 2x performance boost(over OpenSSL 1.0.1 and v1).
[1] https://twitter.com/SaveTheRbtz/status/773962669166428161
[2] There are also cloudflare and intel patches for zlib for faster deflation (i.e. compression only)
> On Nov 18, 2016, at 8:12 PM, Vakul Garg <vakul.garg at nxp.com> wrote:
> 
> Hi
> 
> I am a newbie to nginx.
> I am integrating a public key hardware accelerator in OpenSSL using engine interface.
> The engine is async capable.
> 
> Recently openssl-1.1.0 has added support for ASYNC_JOB.
> IIUC, nginx would also require changes in order to do SSL handshake in async way.
> 
> Any pointers where can I get the nginx code changes done for async 
> openssl
> 
> Regards
> 
> Vakul
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel