Bug 9706

Summary:	Can't download newer revisions of package from PyPI
Product:	buildroot	Reporter:	J. Eric Mason <jem>
Component:	Outdated package	Assignee:	unassigned
Status:	RESOLVED INVALID
Severity:	normal	CC:	buildroot
Priority:	P5
Version:	2016.11
Target Milestone:	---
Hardware:	All
OS:	Linux
Host:		Target:
Build:

Description J. Eric Mason 2017-03-03 19:44:10 UTC

Newer versions of python projects (in my case I was attempting to update supervisor to supervisor-3.3.1) don't download with the URL scheme currently used. Not sure if this is only a supervisor problem or might affect other modules from PyPI. Recommendation from PyPI is to use their API

  API documentation is at https://warehouse.pypa.io/api-reference/legacy/
  It looks like buildroot should first make a request against /simple/supervisor/ and fetch links from the response.

For example if I hit https://pypi.python.org/simple/supervisor/ I get:

<!DOCTYPE html><html><head><title>Links for supervisor</title></head><body><h1>Links for supervisor</h1><a href="../../packages/2e/3b/f28afd29920be27c948923b276d1f131276880f6b4f9391323064b87b15b/supervisor-3.0a7.tar.gz#md5=d2e6c491fcb2606e4fd0afe4ebfd4b13" rel="internal">supervisor-3.0a7.tar.gz</a><br/>
<a href="../../packages/44/80/d28047d120bfcc8158b4e41127706731ee6a3419c661e0a858fb0e7c4b2d/supervisor-3.3.0.tar.gz#md5=46bac00378d1eddb616752b990c67416" rel="internal">supervisor-3.3.0.tar.gz</a><br/>
<a href="../../packages/4b/7d/cb3d91d6c65f73fd253c376fc0dbbc48b83e6606f61d9961d9ac7cf293f5/supervisor-3.0a9.tar.gz#md5=81d192acab20eef3578856651af7e851" rel="internal">supervisor-3.0a9.tar.gz</a><br/>

...etc.

Comment 1 Arnout Vandecappelle 2017-03-03 22:00:23 UTC

When bumping the package, SUPERVISOR_SITE should change from the current 

http://pypi.python.org/packages/source/s/supervisor

to

https://pypi.python.org/packages/44/80/d28047d120bfcc8158b4e41127706731ee6a3419c661e0a858fb0e7c4b2d

but it's not unusual for the _SITE to change when the version is bumped.

I don't see how this is a bug, so I'm closing as invalid. If you disagree, feel free to reopen.

PS if you do bump the package, don't forget to send the patch to the Buildroot mailing list for upstream inclusion.

Comment 2 Arnout Vandecappelle 2017-03-03 22:26:56 UTC

Correction, the URL for version 3.3.1 is:

https://pypi.python.org/packages/80/37/964c0d53cbd328796b1aeb7abea4c0f7b0e8c7197ea9b0b9967b7d004def

Comment 3 J. Eric Mason 2017-03-03 23:12:02 UTC

Semantically buildroot could (I won't go as far to say should yet) be using the simple repository protocol to find download URLs and hashes for a package generically https://www.python.org/dev/peps/pep-0503/ as the URLs could change over time. If I have the bandwidth I'll see if I can propose a changelist to do this resolution so this wouldn't be a problem in the future.

Comment 4 Thomas Petazzoni 2017-03-04 10:44:24 UTC

I think what you're proposing is exactly what Yegor Yefremov has implemented in https://patchwork.ozlabs.org/patch/708384/.