Bug 10551 - PowerPC SPE and Musl
Summary: PowerPC SPE and Musl
Status: RESOLVED WONTFIX
Alias: None
Product: buildroot
Classification: Unclassified
Component: Other (show other bugs)
Version: 2017.08
Hardware: PC Linux
: P5 normal
Target Milestone: ---
Assignee: Vincent Fazio
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-12-05 18:20 UTC by sigurd.segtnan
Modified: 2022-12-04 16:21 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:


Attachments
diff for package/gcc/gcc.mk (and Config.in in toolchain) (1.29 KB, text/plain)
2017-12-05 18:20 UTC, sigurd.segtnan
Details

Note You need to log in before you can comment on or make changes to this bug.
Description sigurd.segtnan 2017-12-05 18:20:12 UTC
Created attachment 7411 [details]
diff for package/gcc/gcc.mk (and Config.in in toolchain)

Musl is not supported for PowerPC SPE according to toolchain/toolchain-buildroot/Config.in line 74: "depens on !BR2_powerpc_SPE # not supported, build breaks". This breakage may be due the "--with-long-double-128" in package/gcc/gcc.mk, since we have not experienced any build breakage when we changed lines 241-245 to

ifeq ($(BR2_powerpc_SPE),y)
ifeq ($(BR2_TOOLCHAIN_USES_MUSL),n)
HOST_GCC_COMMON_CONF_OPTS += \
 	--enable-e500_double \
 	--with-long-double-128
endif
else
HOST_GCC_COMMON_CONF_OPTS += \
	--enable-e500_double \
	--with-long-double-64 \
	--enable-secureplt
endif

We have used this without problems since buildroot 2016.05 (tested with gcc 6.x and gcc 7.x).
Comment 1 Thomas Petazzoni 2018-01-10 20:20:44 UTC
Thanks for your suggestion. Indeed what you say seems to match the details at https://wiki.musl-libc.org/faq.html#Q:-My-dynamically-linked-program-crashes-on-PowerPC!.

There is one thing that is unclear to me: is this --with-long-double-{64,128} option only needed for PowerPC SPE, or for PowerPC in general ? What about --enable-secureplt ? Is it only for PowerPC SPE, or for PowerPC in general ?
Comment 2 sigurd.segtnan 2018-01-11 19:17:00 UTC
I have asked around, and we used the information from from https://wiki.musl-libc.org/supported-platforms.html. Since we found something that worked for the PowerPC we used, we did not investigate further.

Today the relevant part under Supported target architectures (ABIs) is:
* powerpc (needs gcc built with –enable-secureplt –with-long-double-64, and -Wl,–secure-plt to link dynamic binaries.)
* powerpc64

The "-Wl,–secure-plt" is probably for older versions of gcc (secure-plt is mentioned in the 900-musl-support.patch fotr gcc 4.9.4, and "Pass --secure-plt to the linker" seems to have been added to gcc in 2015 according to https://gcc.gnu.org/ml/gcc-patches/2015-10/msg01501.html).
Comment 3 Thomas Petazzoni 2018-01-16 22:28:43 UTC
Your patch only makes uses of those options for PowerPC SPE. The musl documentation doesn't mention this to be SPE specific. This is what confuses me.
Comment 4 sigurd.segtnan 2018-01-17 09:52:35 UTC
We have tested the patch on Power PC with and without SPE, and it works in both caes. In view of the original line 74 of buildroot/Config.in ("depens on !BR2_powerpc_SPE # not supported, build breaks") I suggest that we keep that patch as is-ish.

The -ish part is the change to Config.in. Instead of commenting out the line, a link to this discussion might be useful, so that anybody else that have a Power PC + musl issue can find it.
Comment 5 Thomas Petazzoni 2020-05-17 21:01:14 UTC
PowerPC SPE support has been dropped from gcc, so there's no real interest in fixing that further. See https://gcc.gnu.org/git/?p=gcc.git;a=commitdiff;h=b31d0348ddada49453e3edaaf93a423fdc61dc79.
Comment 6 Pali Rohár 2022-12-04 16:21:57 UTC
Hello!

Just to let you know, musl libc in this year 2022 in newly released version 1.2.3 added full support for PowerPC SPE FPU. See their release changelog: https://musl.libc.org/releases.html

Because SPE ABI is same as PowerPC soft-float ABI (floating point numbers are stored in general purpose registers and SPE instructions uses only those registers), it was pretty small change.

Here is list of all musl libc PowerPC SPE commits:
https://git.musl-libc.org/cgit/musl/commit/?id=66d1e31292a8c05d172872fe73880ca6d3b68104
https://git.musl-libc.org/cgit/musl/commit/?id=7be59733d71ada3a32a98622507399253f1d5e48
https://git.musl-libc.org/cgit/musl/commit/?id=8274aaaaa1948c50c661aa32e21b3db27a5c0eab

So PowerPC SPE is not dead and still developed in some way.

gcc 8.5.0 is the last gcc version which supports PowerPC SPE ISA.

But using gcc 8.x series with musl SPE is tricky because it has two major PowerPC bugs which cause crashing of compiled applications.

1. gcc does not set correct musl dynamic linker interpreter (musl SPE uses /lib/ld-musl-powerpc-sf.so.1)
2. gcc does not pass -Wl,--secure-plt when linking with musl libc as described at musl faq:
https://wiki.musl-libc.org/faq.html#Q:-My-dynamically-linked-program-crashes-on-PowerPC!

And --enable-secureplt configure flag in gcc 8.x for musl SPE is broken and does not work.

I helped to prepare gcc 8.x patches for both issues:
https://gitlab.nic.cz/turris/os/build/-/blob/42a1b3544f4712e10e100aad837aeb8f74945f99/patches/openwrt/to-upstream/0102-gcc-ppc-expect-sf-ld-suffix-even-with-spe-enabled.patch
https://gitlab.nic.cz/turris/os/build/-/blob/5d44c2123734174c21e8a28a143a9bec7cc9bf28/patches/openwrt/to-upstream/0103-gcc-ppc-enable_secureplt-yes-for-powerpc-linux-spe.patch

With these gcc patches it is possible to compile gcc (cross)compiler which can then compile target application for musl libc e500v2 SPE ISA correctly without crashes. I have verified and it works perfectly!

gcc at build time needs to be configured by flags:

./configure --enable-obsolete --target=powerpc-linux-muslspe --with-cpu=8548 --enable-e500-double

Note that it is needed to disable option --with-long-double-128 (by not specifying it) because it adds gcc emulation of IBM long double format where mantissa has 106 bits which musl libc does not support (yet). long double is basically unimplemented by musl libc.

For e500v1 SPE ISA (same as e500v2 but without DPFP SPE) flags are:

./configure --enable-obsolete --target=powerpc-linux-muslspe --with-cpu=8540

In both cases --enable-obsolete and --target=powerpc*-linux*-musl*spe* (* for wildcard) are important as this is the way how to build SPE-capable compiler. Without them you will get just non-SPE compiler. Other flags --with-cpu or --enable-e500-double (which choose e500v1 vs. e500v2 variant) can be overwritten at runtime when using compiler by gcc flags.

So to force SPE-capable compiler to produce e500v2 SPE binary:

gcc -mcpu=8548 -mabi=spe -mspe -mfloat-gprs=double

Or for e500v1 SPE binary:

gcc -mcpu=8540 -mabi=spe -mspe -mfloat-gprs=single

To summarize, e500v1 SPE brings: vector 32-bit integer instructions, single 32-bit floating point instructions, vector 32-bit floating point instructions and e500v2 SPE additionally brings single 64-bit (double) floating point instructions.


Note that GNU LD linker has two flags related to e500 isa: -me500 and -me500x2. Do not be confused by e500x2 vs e500v2. e500x2 is the old Motorola name for e500 core with first version of SPE. Very old name e500x1 was without SPE. e500x2 was probably later renamed to just e500 and e500x1 separated to e200 family (MPC5500). Later when double precision SPE was introduced, old e500 without DPFP was marked as e500v1 and new SPE with DPFP as e500v2. Hence e500x2 is technically e500v1 (and not v2). But because GNU ld does not distinguish between e500v1 and e500v2 it has only one -me500 switch, plus old-named -me500x2 which is just alias to -me500. So you should not use -me500x2 flag for e500v2 as it is very misleading.

Also gcc cpu name 8540 is misleading as it refers to SoC/processor mpc8540 which was the first released product with e500(v1) core. But in reality -mcpu=8540 specifies e500(v1) ISA. Same for 8548 which refers to the first release product with e500v2 core (mpc8548). gcc should have called them e500 or e500v1 and e500v2.