Bug 7628

Summary: Python SSL does not get built for Raspberry Pi
Product: buildroot Reporter: tobias
Component: OtherAssignee: unassigned
Status: RESOLVED FIXED    
Severity: major CC: buildroot
Priority: P5    
Version: 2014.08   
Target Milestone: ---   
Hardware: PC   
OS: Linux   
Host: Target:
Build:
Attachments: My config

Description tobias 2014-11-10 10:34:02 UTC
I'm building for the raspberry pi and want to have the Python SSL package.

Steps i've taken:
- make distclean
- make raspberrypi_defconfig
- make nconfig
- Enable WCHAR support, python3 and python-ssl
- make
- grab coffee

However, there is no "_ssl.cpython-34m.so" ("_ssl.so" in the python2 case) to be found in the lib-dynload folder, thus the ssl module isn't working. I'm fairly new to (cross-)compiling so I might have missed something trivial, although I think I've doublechecked everything. I guess there just is a include or something like that missing, as the python compilation complains about ssl dependencies not being found.

I've also tried this with glibc 2.18 / 2.19 and gcc 4.8 / 4.9, binutils 2.22 / 2.24 and python2 and EABIhf. I actually need glibc + EABIhf for my application, but this problem seems to occur with every configuration.

Host: 3.2.0-70-virtual #105-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux
Buildroot: git version 2014.08-0-gfcd720d
Comment 1 Thomas Petazzoni 2014-11-13 23:02:54 UTC
Can you give your exact Buildroot .config? I've built the following defconfig here:

BR2_arm=y
BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
BR2_PACKAGE_PYTHON3=y
BR2_PACKAGE_PYTHON3_SSL=y

And I definitely do have the _ss.cpython-34m.so library built:

# find / -name '_ssl*'
/usr/lib/python3.4/lib-dynload/_ssl.cpython-34m.so

However, when I try to use the ssl module, it complains of an undefined instruction:

# python
Python 3.4.1 (default, Nov 13 2014, 22:55:45) 
[GCC 4.8.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
python (528): undefined instruction: pc=769e536c
Code: 00000000 f26ee1fe e12fff1e ee190f1d (e12fff1e) 
>>> 

So there is indeed an issue, but it does not appear to be the issue you reported.
Comment 2 tobias 2014-11-13 23:14:02 UTC
Created attachment 5774 [details]
My config
Comment 3 tobias 2014-11-13 23:21:11 UTC
I attached my config, highlighting the following:
BR2_arm=y
BR2_TARGET_GENERIC_GETTY_PORT="tty1"
BR2_PACKAGE_PYTHON3_PYC_ONLY=y
BR2_PACKAGE_PYTHON3_SSL=y
BR2_PACKAGE_PYTHON3_UNICODEDATA=y

find output/target/ -name "_ssl*" (in the buildroot main directory) definitely is empty with the given configuration (at least on my machine)

The getty port apparently differs but I don't think this causes any issues. I'm currently building with the modified getty port and PYC_ONLY=n, just in case, will edit results in here.

I don't know how well the env is cleaned when compiling, so here is the output of env:
TERM=xterm
SHELL=/bin/bash
XDG_SESSION_COOKIE=<cookie>
SSH_CLIENT=<ip & port>
SSH_TTY=/dev/pts/1
USER=<username>
LS_COLORS=<stuff>
MAIL=/var/mail/<username>
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
PWD=/home/<username>
LANG=de_DE.UTF-8
SHLVL=1
HOME=/home/<username>
LOGNAME=<username>
SSH_CONNECTION=<ip & port>
LESSOPEN=| /usr/bin/lesspipe %s
LESSCLOSE=/usr/bin/lesspipe %s %s
_=/usr/bin/env
Comment 4 tobias 2014-11-13 23:48:07 UTC
Apparently I can't edit my comments, so new comment: With the following changes to the attached config:

262c262
< BR2_TARGET_GENERIC_GETTY_PORT="ttyAMA0"
---
> BR2_TARGET_GENERIC_GETTY_PORT="tty1"
652,653c652,653
< BR2_PACKAGE_PYTHON3_PY_ONLY=y
< # BR2_PACKAGE_PYTHON3_PYC_ONLY is not set
---
> # BR2_PACKAGE_PYTHON3_PY_ONLY is not set
> BR2_PACKAGE_PYTHON3_PYC_ONLY=y

find output/target/ -name "_ssl*" still is empty (after make clean, make)

output/build/host-python3-3.4.1/build/lib.linux-x86_64-3.4/_ssl.cpython-34m.so (host-python ssl) gets built though (in every configuration I tried so far).
Comment 5 Thomas Petazzoni 2014-11-14 08:25:37 UTC
Thanks for your feedback. Interesting, it clearly doesn't match what I'm seeing here. Can you post to some pastebin site or some other location the complete build log? Something like:

  make 2>&1 | tee logfile

And then post 'logfile' somewhere. It would be even nicer if you could after the build do:

  find output/target/ | tee -a logfile

So that I can also see the full list of files in your target filesystem. I indeed think something in your host system might be causing the problem, we need to figure out what.

Thanks again!
Comment 6 Thomas De Schampheleire 2014-11-14 09:06:30 UTC
Possibly related to bug #7478
Comment 7 tobias 2014-11-14 10:29:36 UTC
Again, _ssl* missing in target but present in host. See line 61876, "The necessary bits to build these optional modules were not found:"

I uploaded this compressed file to my gdrive, the uncompressed log has ~23 mb, which probably does not get accepted by any pastebin-ish hoster.
https://drive (dot) google (dot) com/file/d/0BwB7SQYhX6MuZlIwYXdjSDZnZzA/view?usp=sharing

Note: /project is a symlink to /var/data/project on my filesystem (buildroot is too big for my root partition). I just tried compiling after switching to the "real" path, just in case, but same result.

I used the config mentioned in comment 4
Comment 8 tobias 2014-11-20 17:46:19 UTC
Update: I just setup a new virtual machine (3.13.0-32-generic, Ubuntu 14.04 LTS Server), and apparently there the SSL module gets built and I can run "import ssl" without a problem.

The used config differs from the one I uploaded but it didn't work with this particular one on my original system, too.
Comment 9 Thomas Petazzoni 2015-06-03 01:31:47 UTC
I believe it might have been a locale related issue, since having a non-english locale would make the Python add_gcc_paths() logic to fail, and therefore it wouldn't find the headers for OpenSSL.

This has been solved by commit http://git.buildroot.net/buildroot/commit/package/python?id=238bf198cf4f31a3301b81a924d5e22044945955.