Bug 13616

Summary: tar file from git repository generation issue
Product: buildroot Reporter: Jean-pierre Cartal <jpcartal>
Component: OtherAssignee: Yann E. MORIN <yann.morin.1998>
Status: RESOLVED FIXED    
Severity: normal CC: buildroot, yann.morin.1998
Priority: P5    
Version: 2021.02   
Target Milestone: ---   
Hardware: All   
OS: Linux   
Host: Target:
Build:
Attachments: Master branch patch
2020.02.x branch patch

Description Jean-pierre Cartal 2021-03-15 10:36:01 UTC 
Created attachment 8791 [details]
Master branch patch

Hello,

The support/download/git script is generating tar files from git repositories using the following command :

# Create GNU-format tarballs, since that's the format of the tarballs on
# sources.buildroot.org and used in the *.hash files
tar cf - --transform="s#^\./#${basename}/#" \
         --numeric-owner --owner=0 --group=0 --mtime="${date}" --format=gnu \
         -T "${output}.list.sorted" >"${output}.tar"

In one of our repository, the --transform option is generating unusable tar files because of the bug described here : https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=775247

Links with the following pattern src/foo -> ./foo_bar/bar are replaced with 
src/foo -> ${basename}/foo_bar/bar and thus the generated tarball is not usable.

As explained in the debian bug report, adding the S flag to the transform pattern (i.e. --transform="s#^\./#${basename}/#" => --transform="s#^\./#${basename}/#S") solves this issue.

Attaching the corresponding patches for master and 2020.02.x branches 

Regards.
Comment 1 Jean-pierre Cartal 2021-03-15 10:36:39 UTC 
Created attachment 8796 [details]
2020.02.x branch patch
Comment 2 Vincent Fazio 2021-03-15 15:41:22 UTC 
This will potentially cause a rev change in the tarballs generated by both svn and git since hashes are likely to change for affected repositories.
Comment 3 Yann E. MORIN 2021-03-15 21:07:09 UTC 
Jean-Pierre, All,

Thanks for the report.

The use of --transform (in the current form or similar) has been introduced in
commit 6d938bcb5249 in 2018-04-02, and although we had a few fixups over the
following days, it has stayed like that ever since, until we introduced the
current generic helper, that reuses that exact same --transform pattern.

So, is this a new behaviour on your side, or is it a new package you are
introducing?

In any case, the use of the 'S' flag does indeed make sense.

Can you submit your patches to the mailing list, please?

Regards,
Yann E. MORIN.
Comment 4 Jean-pierre Cartal 2021-03-16 09:42:34 UTC 
(In reply to Yann E. MORIN from comment #3)
Yann,

I think this is not a new behaviour, neither a new package, but up to now we've been using the OVERRIDE_SRCDIR mechanism to generate binary tar files that were then used as part of the build. We just tried to make a full build from sources with this package and ended up hitting the reported issue.

I'll submit the patches to the mailing list as soon as possible.

Regards.