回复:Re: Openssl 1.1.0f Support

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

回复:Re: Openssl 1.1.0f Support

iloveperl
With the patch, the ERR_clear_error() will only be called when the error occurs. In the normal situation, ERR_clear_error() will not be called, so the Err_get_state() will not be called and no lock contention in openssl 1.0.1 with the patch. 


----- 原始邮件 -----
发件人:Bryan Call <[hidden email]>
收件人:[hidden email]
抄送人:users <[hidden email]>
主题:Re: Openssl 1.1.0f Support
日期:2017年09月21日 23点37分

This only changes the order of the calls.  There is still going to be lock contention inside OpenSSL 1.0.1.

-Bryan

On Sep 20, 2017, at 11:37 PM, [hidden email] wrote:


The following traffic server patch can improve openssl 1.0.1 performance as openssl 1.1.0:

diff --git a/iocore/net/SSLUtils.cc b/iocore/net/SSLUtils.cc
index 5c9709c..5d306a1 100644
--- a/iocore/net/SSLUtils.cc
+++ b/iocore/net/SSLUtils.cc
@@ -1936,7 +1936,7 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t nbytes, int64_t &nwritten)
   if (unlikely(nbytes == 0)) {
     return SSL_ERROR_NONE;
   }
-  ERR_clear_error();
+
   int ret = SSL_write(ssl, buf, (int)nbytes);
   if (ret > 0) {
     nwritten = ret;
@@ -1953,6 +1953,9 @@ SSLWriteBuffer(SSL *ssl, const void *buf, int64_t nbytes, int64_t &nwritten)
     ERR_error_string_n(e, buf, sizeof(buf));
     Debug("ssl.error.write", "SSL write returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
   }
+
+  ERR_clear_error();
+
   return ssl_error;
 }
 
@@ -1964,7 +1967,7 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
   if (unlikely(nbytes == 0)) {
     return SSL_ERROR_NONE;
   }
-  ERR_clear_error();
+
   int ret = SSL_read(ssl, buf, (int)nbytes);
   if (ret > 0) {
     nread = ret;
@@ -1978,13 +1981,14 @@ SSLReadBuffer(SSL *ssl, void *buf, int64_t nbytes, int64_t &nread)
     Debug("ssl.error.read", "SSL read returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
   }
 
+  ERR_clear_error();
+
   return ssl_error;
 }
 
 ssl_error_t
 SSLAccept(SSL *ssl)
 {
-  ERR_clear_error();
   int ret = SSL_accept(ssl);
   if (ret > 0) {
     return SSL_ERROR_NONE;
@@ -1997,13 +2001,14 @@ SSLAccept(SSL *ssl)
     Debug("ssl.error.accept", "SSL accept returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
   }
 
+  ERR_clear_error();
+
   return ssl_error;
 }
 
 ssl_error_t
 SSLConnect(SSL *ssl)
 {
-  ERR_clear_error();
   int ret = SSL_connect(ssl);
   if (ret > 0) {
     return SSL_ERROR_NONE;
@@ -2016,5 +2021,7 @@ SSLConnect(SSL *ssl)
     Debug("ssl.error.connect", "SSL connect returned %d, ssl_error=%d, ERR_get_error=%ld (%s)", ret, ssl_error, e, buf);
   }
 
+  ERR_clear_error();
+
   return ssl_error;
 } 


From: Bryan Call <[hidden email]>
Reply-To: "[hidden email]" <[hidden email]>
Date: Thursday, September 21, 2017 at 8:38 AM
To: "[hidden email]" <[hidden email]>
Subject: Re: Openssl 1.1.0f Support

 

I meant to say 1.1.0. 

 

-Bryan

 

On Sep 20, 2017, at 3:54 PM, Bryan Call <[hidden email]> wrote:

 

I was see something like 2x the performance in my benchmarks with OpenSSL 1.0.1.  I have been doing all my development with OpenSSL 1.0.1 ATS since May, when I upgraded to Fedora 26.

 

-Bryan

 

On Sep 20, 2017, at 2:16 PM, Dave Thompson <[hidden email]> wrote:

 

Sorry Jeremy, my recollections were from 16 months ago which is fuzzy by now at best.   The gist of my recollection is that QAT is an IO based async engine, which of course ATS already has done extensively.   I recall the under-the-hood QAT longjumping was a non-starter in an ATS framework.   This was all static code analysis.  Integration looked like a non-starter, so no performance test done.

Regarding performance testing of "ATS + Openssl 1.1.0(x) + standard aes-ni acceleration", Susan (?Bryan?) was just telling me today of a measured order of magnitude improvement over with the same using Openssl 1.0.1(x) and small packet sizes...  Improvement attributed to lock contention issues in the older OpenSSL 1.0.1(x).

 

Dave

 

On Wed, Sep 20, 2017 at 3:22 PM, Jeremy Payne <[hidden email]> wrote:

Dave,


Did you run any comparison performance tests using the QAT engine ?

Specifically around these configurations(or similar)


1. ATS + Openssl 1.1.0(x) + QAT engine(sync)

2. ATS + Openssl 1.1.0(x) + standard aes-ni acceleration




On Wed, Sep 20, 2017 at 11:26 AM, Dave Thompson <[hidden email]> wrote:

> July 2016, I was evaluating the async Quick Assist in the context of ATS,

> and came away with the opinion it's value comes into play with a much

> simpler application.   It's effectively it's own async engine, long jumping

> across the stack, and doesn't play well or add  value to ATS's more

> extensive model to do similar.... not to mention mutually exclusive in their

> current forms.

>

> Dave

>

>

>

> On Wed, Sep 20, 2017 at 10:08 AM, Alan Carroll <[hidden email]>

> wrote:

>>

>> Susan and Dave Thompson were working on something related to that, "crypto

>> proxy". There's a small mention of it by Susan at the Fall 2016 summit in

>> the TLS state slides

>> (https://cwiki.apache.org/confluence/display/TS/Presentations+-+2016). I'd

>> start there and see if you can bug Susan or Good Dave*. Although that work

>> was designed to use an off box crypto engine, the implementation from the

>> ATS point of view is identical to what you're writing about. Susan will be

>> at the Fall 2017 Summit, I'd look her up then as well.

>>

>> * To distinguish from "Evil Dave" Carlin.

>>

>> On Wed, Sep 20, 2017 at 9:44 AM, Jeremy Payne <[hidden email]> wrote:

>>>

>>> Thanks guys.. Thats all I needed to know.. Now I can look closer at my

>>> end. Will let you know what I find.

>>>

>>> Also, any plans on supporting openssl async, which then allows for

>>> taking full advantage of the Intel QAT engine?

>>> Understood patches/commits are welcome, but just figured there may be

>>> some behind the scene works already started.

>>>

>>> Thanks!

>>>

>>> On Tue, Sep 19, 2017 at 6:31 PM, Alan Carroll <[hidden email]>

>>> wrote:

>>> > Susan has also run some performance tests with 7.1.x and openSSL 1.1

>>> > vs.

>>> > openSSL 1.0.2.

>>> >

>>> > On Tue, Sep 19, 2017 at 5:55 PM, Leif Hedstrom <[hidden email]>

>>> > wrote:

>>> >>

>>> >>

>>> >> On Sep 19, 2017, at 2:20 PM, Jeremy Payne <[hidden email]> wrote:

>>> >>

>>> >> I can link ATS 7.x and 8.x against openssl 1.1.0f, however, for some

>>> >> reason I can't establish a SSL/TLS connection.  Has anyone

>>> >> successfully linked ATS against openssl 1.1.0f  and successfully been

>>> >> able to establish a SSL/TLS session?

>>> >> In other words, is openssl 1.1.0f supported by ATS? If not, what about

>>> >> an earlier version of 1.1.0(x)??

>>> >>

>>> >>

>>> >>

>>> >> Yeh, we’re running current master with OpenSSL v1.1.0f on
>>> >> docs.trafficserver.apache.org. Maybe you have some mismatch / issues

>>> >> between

>>> >> headers (when compiling ATS) and runtime?

>>> >>

>>> >> Cheers,

>>> >>

>>> >> — Leif
>>> >>
>>> >
>>
>>
>


Reply | Threaded
Open this post in threaded view
|

Re: Openssl 1.1.0f Support

Leif Hedstrom

> On Sep 21, 2017, at 8:55 PM, [hidden email] wrote:
>
> With the patch, the ERR_clear_error() will only be called when the error occurs. In the normal situation, ERR_clear_error() will not be called, so the Err_get_state() will not be called and no lock contention in openssl 1.0.1 with the patch.
>
>
> ----- 原始邮件 -----
> 发件人:Bryan Call <[hidden email]>
> 收件人:[hidden email]
> 抄送人:users <[hidden email]>
> 主题:Re: Openssl 1.1.0f Support
> 日期:2017年09月21日 23点37分
>
> This only changes the order of the calls.  There is still going to be lock contention inside OpenSSL 1.0.1.
>
> -Bryan


I think this makes a lot of sense (but I haven’t tested it). Did you make a PR for this on Github?

Cheers,

— leif

Reply | Threaded
Open this post in threaded view
|

Re: Openssl 1.1.0f Support

Sudheer Vinukonda-2
Just to add some more context related to this -

IIRC, the patch proposed below (to clear the error queue only when an error occurred) was actually how this fix was implemented when it was first added (sometime during 5.3?) However, I think it was still resulting in some unexplained handshake/connect failures (which turned out to be not exactly connect failures, but were being marked so, due to stale error queue). And that was when the fix was changed to clear the error queue *before* calling connect.

All of this being a few years and releases ago, may no longer be the case and assuming the reordering of the clear queue tests good with no handshake failures or other side affects, this fix sounds fine.

- Sudheer



I agree this needs to be tested

> On Sep 26, 2017, at 7:51 AM, Leif Hedstrom <[hidden email]> wrote:
>
>
>> On Sep 21, 2017, at 8:55 PM, [hidden email] wrote:
>>
>> With the patch, the ERR_clear_error() will only be called when the error occurs. In the normal situation, ERR_clear_error() will not be called, so the Err_get_state() will not be called and no lock contention in openssl 1.0.1 with the patch.
>>
>>
>> ----- 原始邮件 -----
>> 发件人:Bryan Call <[hidden email]>
>> 收件人:[hidden email]
>> 抄送人:users <[hidden email]>
>> 主题:Re: Openssl 1.1.0f Support
>> 日期:2017年09月21日 23点37分
>>
>> This only changes the order of the calls.  There is still going to be lock contention inside OpenSSL 1.0.1.
>>
>> -Bryan
>
>
> I think this makes a lot of sense (but I haven’t tested it). Did you make a PR for this on Github?
>
> Cheers,
>
> — leif
>

Reply | Threaded
Open this post in threaded view
|

Re: Openssl 1.1.0f Support

Bryan Call-2
We should be able to test this by calling ERR_peek_error() or ERR_get_error() in a test build in production to see if they are picking up any errors that haven’t been cleared.

-Bryan



On Sep 26, 2017, at 8:05 AM, Sudheer Vinukonda <[hidden email]> wrote:

Just to add some more context related to this -

IIRC, the patch proposed below (to clear the error queue only when an error occurred) was actually how this fix was implemented when it was first added (sometime during 5.3?) However, I think it was still resulting in some unexplained handshake/connect failures (which turned out to be not exactly connect failures, but were being marked so, due to stale error queue). And that was when the fix was changed to clear the error queue *before* calling connect.

All of this being a few years and releases ago, may no longer be the case and assuming the reordering of the clear queue tests good with no handshake failures or other side affects, this fix sounds fine.

- Sudheer



I agree this needs to be tested

On Sep 26, 2017, at 7:51 AM, Leif Hedstrom <[hidden email]> wrote:


On Sep 21, 2017, at 8:55 PM, [hidden email] wrote:

With the patch, the ERR_clear_error() will only be called when the error occurs. In the normal situation, ERR_clear_error() will not be called, so the Err_get_state() will not be called and no lock contention in openssl 1.0.1 with the patch.


----- 原始邮件 -----
发件人:Bryan Call <[hidden email]>
收件人:[hidden email]
抄送人:users <[hidden email]>
主题:Re: Openssl 1.1.0f Support
日期:2017年09月21日 23点37分

This only changes the order of the calls.  There is still going to be lock contention inside OpenSSL 1.0.1.

-Bryan


I think this makes a lot of sense (but I haven’t tested it). Did you make a PR for this on Github?

Cheers,

— leif