Bug 9796

Summary:	source-check broken for Git downloads
Product:	buildroot	Reporter:	Thomas Petazzoni <thomas.petazzoni>
Component:	Other	Assignee:	Yann E. MORIN <yann.morin.1998>
Status:	RESOLVED FIXED
Severity:	normal	CC:	buildroot
Priority:	P5
Version:	unspecified
Target Milestone:	---
Hardware:	All
OS:	Linux
Host:		Target:
Build:

Description Thomas Petazzoni 2017-04-07 09:37:18 UTC

source-check doesn't work for Github downloads it seems, see for example the Linux kernel download for the raspberrypi_defconfig:

$ make raspberrypi_defconfig
$ make source-check
[...]
Spider mode enabled. Check if remote file exists.
--2017-04-07 08:00:27--  https://github.com/raspberrypi/linux.git/linux-935c7ce84c982a26f567a03a58a1537424569938.tar.gz
Resolving github.com (github.com)... 192.30.253.112, 192.30.253.113
Connecting to github.com (github.com)|192.30.253.112|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
Remote file does not exist -- broken link!!!

Spider mode enabled. Check if remote file exists.
--2017-04-07 08:00:27--  http://sources.buildroot.net/linux-935c7ce84c982a26f567a03a58a1537424569938.tar.gz
Resolving sources.buildroot.net (sources.buildroot.net)... 176.9.16.109
Connecting to sources.buildroot.net (sources.buildroot.net)|176.9.16.109|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
Remote file does not exist -- broken link!!!

Some source-check fails because the kernel is not available. But in practice, if you do a build, it can download the kernel perfectly fine:

$ make linux-extract
>>> linux 935c7ce84c982a26f567a03a58a1537424569938 Downloading
Doing full clone
Cloning into 'linux-935c7ce84c982a26f567a03a58a1537424569938'...
remote: Counting objects: 5514757, done.
remote: Compressing objects: 100% (4789/4789), done.
remote: Total 5514757 (delta 3516), reused 11 (delta 11), pack-reused 5509948
Receiving objects: 100% (5514757/5514757), 1.55 GiB | 12.32 MiB/s, done.
Resolving deltas: 100% (4579570/4579570), done.
Checking connectivity... done.
Checking out files: 100% (56709/56709), done.
warning: refname '935c7ce84c982a26f567a03a58a1537424569938' is ambiguous.
Git normally never creates a ref that ends with 40 hex characters
because it will be ignored when you just specify 40-hex. These refs
may be created by mistake. For example,

  git checkout -b $br $(git rev-parse ...)

where "$br" is somehow empty and a 40-hex ref is created. Please
examine these refs and maybe delete them. Turn this message off by
running "git config advice.objectNameWarning false"
>>> linux 935c7ce84c982a26f567a03a58a1537424569938 Extracting
gzip -d -c /home/thomas/buildroot/dl/linux-935c7ce84c982a26f567a03a58a1537424569938.tar.gz | tar --strip-components=1 -C /home/thomas/buildroot/buildroot/output/build/linux-935c7ce84c982a26f567a03a58a1537424569938   -xf -

Comment 1 Peter Korsgaard 2017-04-08 08:28:25 UTC

But this has nothing to do with github, right? It is simply the source-check infrastructure that gets confused and tries to use wget instead of git.

Debugging a bit further, it is because $(PKG) isn't known in the foo-source-check step, so it doesn't pick up LINUX_SITE_METHOD=git and defaults to git.

Thomas, you reworked the source-check infrastructure, so I'll let you figure out how/where to get PKG defined ;)

Comment 2 Thomas Petazzoni 2017-04-08 08:30:51 UTC

Hum, I thought the defconfig was using an http download from Github, but indeed it's using a Git download instead. So indeed, it's an issue with the Git download helper.

Comment 3 Peter Korsgaard 2017-04-08 08:35:34 UTC

No, it is really a general problem with source-check (that we normally don't notice as an empty <pkg>_SITE_METHOD is handled with wget, which is correct for most packages). The git handler for source-check looks fine to me, but it never gets called.

Comment 4 Yann E. MORIN 2017-05-10 21:39:16 UTC

The underlying reason s that foo-source-check has no stamp file
associated to it, and eventually, $(PKG) is empty.

I'll fix soonish...

Comment 5 Yann E. MORIN 2017-05-12 15:55:31 UTC

Patch sent:
    https://patchwork.ozlabs.org/patch/761701/

Comment 6 Thomas Petazzoni 2017-05-13 13:13:32 UTC

Fixed by https://git.buildroot.org/buildroot/commit/?id=4c0aacbc317d298397cede593f74ece26d0a89f0. Thanks a lot Yann!