NGINx async SSL handshake

Wed Feb 1 05:48:04 UTC 2017

Hi Brian

Sorry for the late response. I was on vacations.

I understand the point that if I call SSL_* API again with a given SSL session,  it will resume the ASYNC JOB associated with it.
>From there we can dequeue in the engine the responses from the crypto hardware.

But the moot point is how we are going to trigger nginx come out of epoll() so that it knows that SSL_ * API needs to be called for a given SSL session?

Regards

Vakul

-----Original Message-----
From: Will, Brian [mailto:brian.will at intel.com] 
Sent: Wednesday, January 25, 2017 4:08 AM
To: Vakul Garg <vakul.garg at nxp.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>; Schuetze, Joel D <joel.d.schuetze at intel.com>; Yu, Ping Y <ping.y.yu at intel.com>; Finn, Coleman <coleman.finn at intel.com>; Pankaj Gupta <pankaj.gupta at nxp.com>; Arun Pathak <arun.pathak at nxp.com>; Chambers, Mark A <mark.a.chambers at intel.com>
Subject: RE: NGINx async SSL handshake

Hi Vakul,
I look at OpenSSL and the asynch framework as separate to how the engine handles its submissions and responses. If you call the SSL_* API's as you stated it will resume the JOB at the point where it was PAUSED, namely in the engine allowing it to dequeue the responses from the HW. 

If the HW causes reordering of the responses it can simply dequeue the existing responses, store the ones that are out of order for this connection and either keep dequeuing until the correct response is seen or PAUSE again until the right response is available. This also means there is no need for a polling thread to be created by the engine. You would also disable the event notifications and FD creation that the engine currently owns.

I'm unable to post to the NGINX forum as I'm getting a permission error on this specific message.

Thanks,
Brian

-----Original Message-----
From: Vakul Garg [mailto:vakul.garg at nxp.com]
Sent: Monday, January 16, 2017 8:43 AM
To: Will, Brian <brian.will at intel.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>; Schuetze, Joel D <joel.d.schuetze at intel.com>; Yu, Ping Y <ping.y.yu at intel.com>; Finn, Coleman <coleman.finn at intel.com>; Pankaj Gupta <pankaj.gupta at nxp.com>; Arun Pathak <arun.pathak at nxp.com>; Chambers, Mark A <mark.a.chambers at intel.com>
Subject: RE: NGINx async SSL handshake

Hi Brian

> For the engine response processing another alternative is to simply recall the appropriate SSL_* API which will switch to the correct JOB context and complete the crypto processing.

IIUC, for this to work, the response from the accelerator corresponding to the paused job associate with SSL connection should have been dequeued prior. 
The engine's poll thread does just that. It dequeues accelerator responses and finds out wait_ctx to wakeup through its file descriptor.
This in turn wakes triggers NGINx's epoll_wait() to return. 

It then calls SSL_* API for the connection wokenup.
Now the SSL_* API would internally call ASYNC_start_job and it resumes previously paused job to complete the operation.

Thus, IMO, simply recalling SSL_* API is not sufficient unless you meant to say that you would dequeue accelerator response directly from its command rings after job gets resumed.
In that case, if we do this, then we would have a problem if accelerator reorders the responses. 

Regards

Vakul

-----Original Message-----
From: Will, Brian [mailto:brian.will at intel.com]
Sent: Saturday, January 14, 2017 1:51 AM
To: Vakul Garg <vakul.garg at nxp.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>; Schuetze, Joel D <joel.d.schuetze at intel.com>; Yu, Ping Y <ping.y.yu at intel.com>; Finn, Coleman <coleman.finn at intel.com>; Pankaj Gupta <pankaj.gupta at nxp.com>; Arun Pathak <arun.pathak at nxp.com>; Chambers, Mark A <mark.a.chambers at intel.com>
Subject: RE: NGINx async SSL handshake

Hi Vakul,
This sounds interesting and would give the capability to do the "polling inline". I'd like to see the changes you are proposing to the event loop.

For the engine response processing another alternative is to simply recall the appropriate SSL_* API which will switch to the correct JOB context and complete the crypto processing. This way you are able to continue using a standard OpenSSL API rather than having to create a new standardized ENGINE ctrl message. It shouldn't be much in the difference wrt performance as the context switch happens very soon after the SSL_* API invocations.

Thanks,
Brian

-----Original Message-----
From: Vakul Garg [mailto:vakul.garg at nxp.com]
Sent: Friday, January 13, 2017 1:52 AM
To: Will, Brian <brian.will at intel.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>; Schuetze, Joel D <joel.d.schuetze at intel.com>; Yu, Ping Y <ping.y.yu at intel.com>; Finn, Coleman <coleman.finn at intel.com>; Pankaj Gupta <pankaj.gupta at nxp.com>; Arun Pathak <arun.pathak at nxp.com>; Chambers, Mark A <mark.a.chambers at intel.com>
Subject: RE: NGINx async SSL handshake

Hi Brian

Yes, one of the things I am looking for is a replacement for engine private receive thread with "poll inline" from nginx worker itself.
It might not necessarily require timer based mechanism to get the nginx event loop to poll accelerator responses.

The requirement is to give some time slice to the engine to do receive side processing from the accelerator.
This time slice can be given after completing ready list of FDs in event loop.

If any SSL connection is pending on WANT_ASYNC, then event poll loop should wakeup with zero timeout, because we know accelerator response would come soon and it makes sense to give timeslice for engine accelerator response processing.
For engine accelerator response processing an ENGINE control api can be built and directly called from nginx event loop.

Please let me know your comments.

Regards

Vakul

-----Original Message-----
From: Will, Brian [mailto:brian.will at intel.com]
Sent: Friday, January 13, 2017 4:54 AM
To: Vakul Garg <vakul.garg at nxp.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>; Schuetze, Joel D <joel.d.schuetze at intel.com>; Yu, Ping Y <ping.y.yu at intel.com>; Finn, Coleman <coleman.finn at intel.com>; Pankaj Gupta <pankaj.gupta at nxp.com>; Arun Pathak <arun.pathak at nxp.com>; Chambers, Mark A <mark.a.chambers at intel.com>
Subject: RE: NGINx async SSL handshake

Hi Vakul,
Sorry for not getting back sooner. We are currently in the process of working with NGINX to get our patch integrated into the development tree. This process may result in a very different patch to what we have currently published, resulting in closer integration and fit into the infrastructure. As of right now we do not have the patch posted either as a repo or a review request. 

Your work on the wakeup mechanism would be very interesting, specifically what are you looking at as a replacement for the activation thread which the engine would own? On the option to "poll inline" from the nginx worker itself this is something we had explored and does result in increased performance, but then you would require some timer based mechanism to get the NGINX event loop to poll the SSL* API in question. This can be added and is supported through the current OpenSSL async features, but seemed like a different processing model to the way NGINX handles all other activity.

Thanks,
Brian

-----Original Message-----
From: Vakul Garg [mailto:vakul.garg at nxp.com]
Sent: Monday, January 09, 2017 12:52 AM
To: Will, Brian <brian.will at intel.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>; Schuetze, Joel D <joel.d.schuetze at intel.com>; Yu, Ping Y <ping.y.yu at intel.com>; Finn, Coleman <coleman.finn at intel.com>; Pankaj Gupta <pankaj.gupta at nxp.com>; Arun Pathak <arun.pathak at nxp.com>
Subject: RE: NGINx async SSL handshake

Hi Brian

We are using your nginx patch with our PKC accelerator (exposed through cryptodev-linux ioctls).
The OpenSSL cryptodev engine has been enhanced to make async PKC calls.

The system is running functionally well, but the performance is poor.

Last year, we used older OpenSSL versions (without ASYNC job infrastructure) with Geoff Thorpe's https://github.com/libfibre to achieve async crypto acceleration.
The performance was better than we are getting with OpenSSL-1.1.0 + Nginx patch in question.

On studying the Intel nginx changes, I feel that the way the asynchronous APIs have been integrated are somewhat heavy and there is possibility of improvement.
Specifically, the eventfd based job wakeup mechanism seems heavy.

I am looking towards making changes over your nginx patch to introduce a newer job wakeup method.
Further, I want to get rid of creating set of accelerator response poll threads in engine and provide an option to do this inline in nginx worker thread itself.

For this, I hoped your patches are submitted upstream and in process of review or I can find your work in some github repo where we can submit our changes.

Let me know your comments.

Regards

Vakul

-----Original Message-----
From: Will, Brian [mailto:brian.will at intel.com]
Sent: Saturday, January 07, 2017 2:40 AM
To: Vakul Garg <vakul.garg at nxp.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>; Schuetze, Joel D <joel.d.schuetze at intel.com>; Yu, Ping Y <ping.y.yu at intel.com>; Finn, Coleman <coleman.finn at intel.com>
Subject: RE: NGINx async SSL handshake

Hello Vakul,
The link you have included is the latest patch for NGINX to support OpenSSL-1.1.0 that we are maintaining. We are working to try and get these changes included into the mainline of NGINX, but in the interim this patch should work for the OpenSSL-1.1.0 branch of code.
Were there specific issues that you have run into?

Thanks,
Brian

-----Original Message-----
From: Vakul Garg [mailto:vakul.garg at nxp.com]
Sent: Friday, January 06, 2017 12:10 PM
To: Will, Brian <brian.will at intel.com>
Cc: nginx-devel at nginx.org; Michael Kardonik <michael.kardonik at nxp.com>
Subject: RE: NGINx async SSL handshake

Hi Brian

I am using nginx openssl async: https://01.org/sites/default/files/page/nginx-1.10.0-async.l.0.3.0-001_0.tgz with my async RSA accelerator.
Where can I find the updated version of this work?

Can you share your plans to upstream this work?

Regards

Vakul

-----Original Message-----
From: nginx-devel [mailto:nginx-devel-bounces at nginx.org] On Behalf Of Alexey Ivanov
Sent: Saturday, November 19, 2016 2:00 PM
To: nginx-devel at nginx.org
Cc: brian.will at intel.com
Subject: Re: NGINx async SSL handshake

+Brian Will from Intel, to correct me if I'm wrong.

My two cents here(Intel Quick Assist specific, based on conversations during Nginx.Conf):
1) Even without hardware offload async handshake helps in cases where you have high TLS connection rates, because right now handshake is basically a 2ms+ blocking operation(specific timing depend on AVX/AVX2 support[0]) inside the event loop. Therefore after some TLS connection rate nginx performance falls off the cliff.
2) Hardware offload numbers look very impressive[1](TL;DR: 5x improvement for RSA 2048, for ECDSA, imho, it is neither impressive, nor needed). Also they prove asymmetric part of the accelerator to be future proof, so that it is possible to add new handshake types(e.g. Ed25519). Disclaimer: we did not test that hardware yet.

As for patches, you can check 01.org for:
a) nginx openssl async: https://01.org/sites/default/files/page/nginx-1.10.0-async.l.0.3.0-001_0.tgz
b) zlib[2]: https://01.org/sites/default/files/page/zlib_shim_0.4.9-001.tgz
(Full list of docs: https://01.org/packet-processing/intel%C2%AE-quickassist-technology-drivers-and-patches )

Question for Brian/Maxim: are you planning on integrating it into mainline nginx? 1000+ line diffs are usually rather hard to integrate.

[0] FWIW, speaking about OpenSSL performance: using OpenSSL 1.0.2 + Intel Xeon v2 processors with AVX2 gives 2x performance boost(over OpenSSL 1.0.1 and v1).
[1] https://twitter.com/SaveTheRbtz/status/773962669166428161
[2] There are also cloudflare and intel patches for zlib for faster deflation (i.e. compression only)
> On Nov 18, 2016, at 8:12 PM, Vakul Garg <vakul.garg at nxp.com> wrote:
> 
> Hi
> 
> I am a newbie to nginx.
> I am integrating a public key hardware accelerator in OpenSSL using engine interface.
> The engine is async capable.
> 
> Recently openssl-1.1.0 has added support for ASYNC_JOB.
> IIUC, nginx would also require changes in order to do SSL handshake in async way.
> 
> Any pointers where can I get the nginx code changes done for async 
> openssl
> 
> Regards
> 
> Vakul
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel