Bug 13321

Summary:	utils/scanpypi: fails with packages that define a package’s metadata in setup.cfg
Product:	buildroot	Reporter:	Bartosz Bilas <bartosz.bilas>
Component:	Other	Assignee:	unassigned
Status:	RESOLVED MOVED
Severity:	normal	CC:	buildroot, julian, kr886q, yann.morin.1998
Priority:	P5
Version:	2020.08.2
Target Milestone:	---
Hardware:	All
OS:	Linux
Host:		Target:
Build:

Description Bartosz Bilas 2020-11-18 19:02:24 UTC

There is a couple(or more than couple) of packages on PyPi that doesn't work with the scanpypi script (for example wheel/importlib_resources). The reason of that is that it uses setup.cfg in conjunction with setup.py to package metadata configuration. Use of scanpypi on these packages fails with the following errors:

utils/scanpypi:23: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
spdx_lookup module is not installed. This can lead to an inaccurate licence detection. Please install it via
pip install spdx_lookup
buildroot package name for wheel: python-wheel
Package: python-wheel
Fetching package wheel
Downloading package wheel from https://files.pythonhosted.org/packages/83/72/611c121b6bd15479cb62f1a425b2e3372e121b324228df28e64cc28b01c2/wheel-0.35.1.tar.gz...
Traceback (most recent call last):
  File "utils/scanpypi", line 754, in <module>
    main()
  File "utils/scanpypi", line 702, in main
    package.load_setup()
  File "utils/scanpypi", line 302, in load_setup
    setup = imp.load_module('setup', s_file, s_path, s_desc)
  File "/usr/lib/python3.8/imp.py", line 234, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.8/imp.py", line 171, in load_source
    module = _load(spec)
  File "<frozen importlib._bootstrap>", line 702, in _load
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/scanpypi-evf4l461/python-wheel/wheel-0.35.1/setup.py", line 4, in <module>
  File "utils/scanpypi", line 63, in closure
    BuildrootPackage.setup_args[kwargs['name']] = kwargs
KeyError: 'name'


To reproduce just type: 

utils/scanpypi wheel

Comment 1 kirby 2020-11-23 03:17:49 UTC

I'm seeing something similar trying to make a package for python-debugpy. 
I have yet to find a workaround unless anyone has any suggestions?

user@ubuntu:~/buildroot-2020.05.1$ python3 utils/scanpypi debugpy -o package
utils/scanpypi:23: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
spdx_lookup module is not installed. This can lead to an inaccurate licence detection. Please install it via
pip install spdx_lookup
buildroot package name for debugpy: python-debugpy
Package: python-debugpy
Fetching package debugpy
Downloading package debugpy from https://files.pythonhosted.org/packages/12/73/35e48dfeb1e26fbc6fe6e888c5b7507f7010488bb1d7363322edc38c03c6/debugpy-1.2.0.zip...
Error: Could not install package debugpy: module 'setup' has no attribute 'main'

Comment 2 Oliwia Zaremba 2021-01-15 14:00:29 UTC

To provide more background on this issue: this is the result of PEP-517 (https://www.python.org/dev/peps/pep-0517/) getting a wider adoption. According to the documentation: https://setuptools.readthedocs.io/en/latest/setuptools.html#id4 the recommended approach for Python packages is to define the dependencies decoratively in setup.cfg. The deprecated calls to setup.py remain backwards compatible by emulating a dummy setup.py with a setup() method call. In that case the method is called without any arguments, and this is ultimately the reason utils/scanpypi script fails for many packages that use this new approach. 

It seems to me that one possible solution to this would be to detect if the setup.cfg exists and retrieve the package's metadata from it (e.g. using the read_configuration method from setuptools.config). But I wonder if there exists a Python native solution to retrieve the metadata (including install_requires dependencies) from a Python package regardless whether PEP-517 convention is used or not, and more importantly, without installing that package. Does anyone know of such mechanism?

Also if anyone found a working solution to this, I would very much appreciate if it can be shared.

Comment 3 Yann E. MORIN 2021-11-06 11:04:25 UTC

*** Bug 14321 has been marked as a duplicate of this bug. ***

Comment 4 Yann E. MORIN 2024-06-15 14:55:42 UTC

Thank you for your report.

The issue tracker for the Buildroot project has been moved to
the Gitlab.com issue tracker:
    https://gitlab.com/buildroot.org/buildroot/-/issues

We are taking this opportunity to close old issues in this old
tracker. If you believe your issue is still relevant, please
open one in the new issue tracker.

Thank you!