The find_library() function in the ctypes Python standard library is not working properly. https://docs.python.org/3/library/ctypes.html#finding-shared-libraries You can reproduce the issue by trying the following in the python shell... >>> from ctypes.util import find_library >>> find_library("c") This should return the string "libc.so.0" but returns nothing, this is the same for any library name I tried even though you can see the libraries are installed and "ldconfig -v" lists the libraries. This has the effect of causing libraries like pyusb to not work properly because they reply on ctypes & find_library() to find libusb. Digging a bit deeper on how find_library() works it calls either ldconfig -p, ld, or gcc. buildroot doesn't have ld or gcc installed so I assume that means it is using "ldconfig -p". ldconfig is part of the uclibc library and uclibc has caching turned off in its configuration so ldconfig -p just prints a message says, "Library cache disabled" instead of printing a list of libraries and their absolute paths. https://github.com/python/cpython/blob/main/Lib/ctypes/util.py There is an option in the uclibc menuconfig to enable the cache (LDSO_CACHE_SUPPORT) but when I did that it didn't seem to change anything. Any ideas how to fix this? I'm using the pc_x86_64_efi_defconfig with Python enabled. No other changes.
I have seen this bug, too, in building an image for the Beaglebone Black. It appears in the iio package in the python binding: In iio.py, CDLL calls find_library with the argument "iio" to get the name of the libiio.so.0 library, but fails. Some folks on #beagle said using find_library() this way was a bad idea because it produces unpredictable results (which I can attest to). Instead, it was recommended that CDLL be called with libiio.so.0 directly, removing find_libary(). This works and I am using a patch to implement it. BR currently uses version 0.19 of libiio but even in the most current version, 0.24, it still uses find_library(). I tried find_library() on a few other libs and got the same None result.
The problem is a bunch of dependencies rely on the find_library function so I don't really have a choice short of patching every library. I think the right way to fix this would be to enable the ldconfig cache mechanism in uclibc or mark libraries that depend on ctypes as dependent on glibc tthat does support the caching mechanism required for find_library
I was able to solve this by enabling the LDSO_CACHE_SUPPORT option in uclibc via a configuration fragment and then patching the file Lib/ctypes/util.py in the python3 package to change the mapping of "x86_64-64" from "libc6,x86_64" to "libc0". I wasn't able to find a way to generate the cache file during the build process so you have to run ldconfig on first boot, you could use and init script, before using Python to generate /etc/ls.so.cache and then ctypes will work correctly. The crux of the issue is that, on Linux, Python's ctypes module just isn't compatible with anything except glibc because is explicitly looks for "libc6" and other glibc specific items in the output of ldconfig -p which is uses to resolve paths to libraries loaded with ctypes. If you are using uclibc, of musl you are going to run into this.
This issue is indeed tricky. I think the right solution would be to patch Python to reimplement the find_library() function in a way that is compatible with Buildroot: (1) Walk the library directories to find the matching library. (2) Retrieve its SONAME. This might be a bit annoying to do in pure Python without invoking a separate Python library to read ELF files. A bit annoying to do, but probably the solution that is going to require the least amount of dependencies in Buildroot.
Created attachment 9341 [details] Python patch to support ctypes when using uclibc Also set LDSO_CACHE_SUPPORT=y un uclibc configuration. Then run ldconfig at boot prior to using Python / ctypes if ld.so.cache doesn't exist. if [[ ! -e "/etc/ld.so.cache" ]]; then ldconfig; fi
My understanding from reading some of the other posts is that the reason the ld.so.cache is disabled is because you can't reliably run it in the fakeroot environment without a bunch of patches.
Created attachment 9566 [details] libiio patch to fix python bindings I've just run into this problem with libiio python bindings as well. (On a glibc system though), the attached (libiio) patch fixes it, though I'm not sure if that's a desired solution.
Thank you for your report. The issue tracker for the Buildroot project has been moved to the Gitlab.com issue tracker: https://gitlab.com/buildroot.org/buildroot/-/issues We are taking this opportunity to close old issues in this old tracker. If you believe your issue is still relevant, please open one in the new issue tracker. Thank you!