Bug 13516

Summary:	utils/scanpypi: Failure when parsing diffoscope
Product:	buildroot	Reporter:	Conrad Ratschan <conrad.ratschan>
Component:	Other	Assignee:	unassigned
Status:	RESOLVED FIXED
Severity:	normal	CC:	buildroot, yann.morin.1998
Priority:	P5
Version:	2020.02.3
Target Milestone:	---
Hardware:	All
OS:	Linux
Host:		Target:
Build:

Description Conrad Ratschan 2021-02-03 19:07:03 UTC

https://pypi.org/project/diffoscope/

The first failure is when the interpreter selected is python2 (#!/usr/bin/env python) and the script attempts to parse diffoscopes' __init__.py which does not specify an encoding (not required in python3) and has a copyright symbol in a comment:

```
Traceback (most recent call last):
  File "./utils/scanpypi", line 754, in <module>
    main()
  File "./utils/scanpypi", line 702, in main
    package.load_setup()
  File "./utils/scanpypi", line 302, in load_setup
    setup = imp.load_module('setup', s_file, s_path, s_desc)
  File "/tmp/scanpypi-2xcRWc/python-diffoscope/diffoscope-166/setup.py", line 6, in <module>
  File "/tmp/scanpypi-2xcRWc/python-diffoscope/diffoscope-166/diffoscope/__init__.py", line 4
SyntaxError: Non-ASCII character '\xc2' in file /tmp/scanpypi-2xcRWc/python-diffoscope/diffoscope-166/diffoscope/__init__.py on line 4, but no encoding declared; see http://python.org/dev/peps/pep-0263/ for details
```

The second issue is if I force it to use python3 to get around the previous issue, diffoscope's setup.py contains a explicit check for python3 > 3.7 which fails out the scanpypi script

Comment 1 Thomas Petazzoni 2021-02-16 21:35:19 UTC

Regarding the first issue, I guess we could change scanpypi to use python3 explicitly. Maybe it will break with some Python modules that have a python2-only setup.py, but it should really be rare these days.

Regarding the second issue, I don't see what we can do. If I remember correctly how scanpypi works, it works by running  the setup.py, so if the setup.py has some requirement on a recent Python version... there's not much we can do about it. You could for example do:

$ make host-python3
$ export PATH=$(pwd)/output/host/bin:$PATH
$ ./utils/scanpipy

and that should use the python3 built by Buildroot, which is recent enough.

Comment 2 Yann E. MORIN 2021-02-20 16:45:17 UTC

Conrad, All,

The first half of the issue is now supposedly fixed with commit ee8b680816 by Thomas:
    https://git.buildroot.org/buildroot/commit/?id=ee8b6808169e1e61b4318b44fbc706ec888c605d

And as Thomas said, we can't do much about the second half, so I'm closing this as fixed.

Thanks for the report!

Regards,
Yann E. MORIN.