Bug 13751

Summary: libopenssl (static): huge drop in performance in newer versions of buildroot
Product: buildroot Reporter: maksis33
Component: OtherAssignee: unassigned
Status: RESOLVED FIXED    
Severity: normal CC: buildroot, yann.morin.1998
Priority: P5    
Version: 2021.02.1   
Target Milestone: ---   
Hardware: All   
OS: Linux   
Host: Target:
Build:

Description maksis33 2021-04-12 09:23:15 UTC 
I've done some benchmarking with different TLS 1.3 ciphers with an Intel Celeron N3150 CPU that has support for the AES-NI instruction set. I used static builds of AirDC++ Web Client (http://airdcpp-web.github.io) that were built with different versions of buildroot. Target architecture is x86_64 (nocona). Transfer speeds are limited by the CPU (one core is maxed out).


Encrypted download speeds (Buildroot 2021.02.1):

TLS_AES_128_GCM_SHA256: 27 MB/s
TLS_AES_256_GCM_SHA384: 24 MB/s
TLS_CHACHA20_POLY1305_SHA256: 41 MB/s


Encrypted download speeds (Buildroot 2019.02):

TLS_AES_128_GCM_SHA256: 62 MB/s
TLS_AES_256_GCM_SHA384: 59 MB/s
TLS_CHACHA20_POLY1305_SHA256: 54 MB/s


Note that those are actual network file transfers with additional overhead, so the plain decrypt speed difference should be much bigger than that ("openssl speed -evp aes-128-cbc" is about 5x faster than "openssl speed aes-128-cbc" on the same system).

I'd assume that the issue has something to do with http://lists.busybox.net/pipermail/buildroot/2019-October/263030.html as the OpenSSL target arch is "gcc no-asm" in Buildroot 2021.02.1 while 2019.02 targets "linux-x86_64" and also has various *_ASM flags present in the Makefile. I also confirmed with objdump that the binary built with 2021.02.1 is missing all references to aesenc/aesdec/other AES-NI instructions while those instructions are present in the binary built with 2019.02.

I read https://github.com/openssl/openssl/commit/87bea6550ae0dda7c40937cff2e86cc2b0b09491 and https://github.com/openssl/openssl/issues/9839 but I don't fully understand why the target architecture was changed. I'd say that ASM support is a must for applications similar to AirDC++.
Comment 1 Arnout Vandecappelle 2021-04-12 20:07:52 UTC 
(In reply to maksis33 from comment #0)
> I'd assume that the issue has something to do with
> http://lists.busybox.net/pipermail/buildroot/2019-October/263030.html
> as the OpenSSL target arch is "gcc no-asm" in Buildroot 2021.02.1 while 2019.02
> targets "linux-x86_64" and also has various *_ASM flags present in the Makefile.

The culprit is actually the following commit:

commit 8c2c959b028d44f5518d4445f864aedae3d90406
Author: Fabrice Fontaine <fontaine.fabrice@gmail.com>
Date:   Fri May 31 15:39:28 2019

    package/libopenssl: fix static build
    
    no-dso option has been removed with
    https://github.com/openssl/openssl/commit/31b6ed76dfd53529b74e79830c81372d0b756929
    
    To fix this error, use "gcc" target in static builds. This target is
    very minimalistic, we need to manually pass -lpthread and
    -DOPENSSL_THREADS however we can also remove libdl workarounds
    
    Fixes:
     - http://autobuild.buildroot.org/results/96d6b89d20980e8f7fa450b832474a81d492b315
    
    Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
    Signed-off-by: Peter Korsgaard <peter@korsgaard.com>


The problem is that in a static build, you don't have shared libraries so also no DSO (the shared library provided by the kernel).


The upstream commit also adds a preprocessor symbol DSO_NONE (which is not set by anything). So you could try reverting buildroot commit 8c2c959b028d44f5518d4445f864aedae3d90406 and add -DDSO_NONE to LIBOPENSSL_CFLAGS.

Of course the revert is not entirely trivial, because the code was refactored and has moved to Config.in...
Comment 3 maksis33 2021-05-04 08:54:27 UTC 
I tested the patch and it seems to fix the issue. Performance is now similar to Buildroot 2019.02.
Comment 4 Yann E. MORIN 2021-05-04 19:26:13 UTC 
Hello, 

Thanks for your report. We believe this has now been fixed with commit
67d19f6014 (package/libopenssl: fix performance issue in static build)
as provided by Fabrice.

Thanks!

Regards,
Yann E. MORIN.
Comment 5 Peter Korsgaard 2021-05-07 08:40:25 UTC 
Also backported to 2021.02.x and will be part of the upcoming 2021.02.2 release