Bug 7172

Mike,

(In reply to comment #0)
> Ref:  man ld.so
> 
> Example:
> In menu -> tool chain -> Target Linker Options (BR2_TARGET_LDFLAGS) to:
> -Wl,-rpath='$ORIGIN/../lib'
> (A relative path from executable to pathname of its required libraries.
> The example in ld.so manual.).
> 
> Build BusyBox (which honors BR_TARGET_LDFLAGS, not everything does).
> 
> Use: readelf -dl on the resulting binary and find an entry similar to:
>  0x0000000f (RPATH)                      Library rpath: [outputRIGIN/../lib]
> 
> Where the $O of $ORIGIN has been expanded to an internal value by the time the
> content of BR2_TARGET_LDFLAGS is seen by the compiler.

Unfortunately, there is no trivial solution. This bug comes from the interaction between Buildroot and the packages' build-systems (especially those based on autotools).

I mean there no easy way to correctly escape "$ORIGIN" in makefiles (form Buildroot and the package itself) or/and in sh scripts.

> 
> This was when using the Buildroot generation of a toolchain.
> Other toolchain selections not tested.

The toolchain has nothing to do with this issue.


From the couple of experiments I run, the only reliable solution to correctly set RPATH is using chrpath or patchelf at the end of the build.


Regards,

Samuel

Not tested, but as $ORIGIN seems to be interpreted by make as ${O}RIGIN, then I guess using $$ORIGIN would work correctly, or if you prefer $${ORIGIN}.

This moves the responsibility of the correct escaping to the user, but as Samuel said there is no easy solution.

Once cleared out, we should document this in the manual though...

(In reply to comment #2)
> Not tested, but as $ORIGIN seems to be interpreted by make as ${O}RIGIN, then I
> guess using $$ORIGIN would work correctly, or if you prefer $${ORIGIN}.
> 
> This moves the responsibility of the correct escaping to the user, but as
> Samuel said there is no easy solution.
> 
> Once cleared out, we should document this in the manual though...
>

Sounds worth trying, will test and post (here) the results.

It is a rare use case, needing to get the ld.so tokens in BR_TARGET_LDFLAGS
to show up in the compiler command line as intended, but a valid one.

At the moment, the busybox package honors the setting of the 
"Target Linker Options" set in the config UI and the Lua package doesn't.
So those will be my test cases.

(and maybe fix the Lua package at the same time.)

Mike

These results do not make any sense to me:

Single escaped: ${ORIGIN}/../lib
 0x0000000f (RPATH)                      Library rpath: [/../lib]
 0x0000001d (RUNPATH)                    Library runpath: [/../lib]

Double escaped: $${ORIGIN}/../lib
 0x0000000f (RPATH)                      Library rpath: [/../lib]
 0x0000001d (RUNPATH)                    Library runpath: [/../lib]

Triple escaped: $$${ORIGIN}/../lib
 0x0000000f (RPATH)                      Library rpath: [../lib]
 0x0000001d (RUNPATH)                    Library runpath: [../lib]

At this rate, adding enough '$' characters might delete all of BR.
;)

Is what you wrote here correct? (No copy paste error?) Case one and two are identical?
Have you tried four dollar signs? Depending on how things are parsed it may be needed...

(In reply to comment #5)
> Is what you wrote here correct? (No copy paste error?) Case one and two are
> identical?
> Have you tried four dollar signs? Depending on how things are parsed it may be
> needed...
>

Yup - not what I expected either.*

Will recheck for cockpit error here and the $$$$ case.

Mike
* I have noticed that menuconfig does not always detect (and save) a configuration
change that only involves editing a string field.
I'll "less .config" before each test next time.

(In reply to comment #6)
> (In reply to comment #5)
> > Is what you wrote here correct? (No copy paste error?) Case one and two are
> > identical?
> > Have you tried four dollar signs? Depending on how things are parsed it may be
> > needed...
> >
> 
> Yup - not what I expected either.*
> 
> Will recheck for cockpit error here and the $$$$ case.

Well, I've already spent hours trying to figure out the right escape
sequence... in vain :-(

I even ended up with some mad sequences of '$' and '\'.
[1] is just an example of the dirty things I tried! My advice: don't go there
;-).

To me, escaping '$' is a dead end (or at least, really hard, except fo
CMake-based packages) because of:
- the make/shell escape character is '$' vs. '\';
- the unpredictable (or hard-to-predict) level/number of make/shell nested,
even for packages using the autotools (Makefile + libtool + ... + custom
commands).

That's why I end up thinking that a solution based on chrpath or patchelf is
the only sane way of handling the '$ORIGIN' support in RPATH (for both target
and host packages).

BTW, this is part of the "relocatable SDK" item, which is somewhere in my todo
list. I have a couple of branches (maybe more than just a couple) in my github
in which I test things from time to time, and that one day, I should gather and
post a real RFC.


[1]
https://github.com/tSed/buildroot/commit/7d53fe664afd3c98b3609994c5bf2addf2f4f692

(In reply to comment #7)
> (In reply to comment #6)
> > (In reply to comment #5)
> > > Is what you wrote here correct? (No copy paste error?) Case one and two are
> > > identical?
> > > Have you tried four dollar signs? Depending on how things are parsed it may be
> > > needed...
> > >
> > 
> > Yup - not what I expected either.*
> > 
> > Will recheck for cockpit error here and the $$$$ case.
> 
> Well, I've already spent hours trying to figure out the right escape
> sequence... in vain :-(
> 
> I even ended up with some mad sequences of '$' and '\'.
> [1] is just an example of the dirty things I tried! My advice: don't go there
> ;-).
> 
> To me, escaping '$' is a dead end (or at least, really hard, except fo
> CMake-based packages) because of:
> - the make/shell escape character is '$' vs. '\';
> - the unpredictable (or hard-to-predict) level/number of make/shell nested,
> even for packages using the autotools (Makefile + libtool + ... + custom
> commands).
> 
> That's why I end up thinking that a solution based on chrpath or patchelf is
> the only sane way of handling the '$ORIGIN' support in RPATH (for both target
> and host packages).
> 
> BTW, this is part of the "relocatable SDK" item, which is somewhere in my todo
> list. I have a couple of branches (maybe more than just a couple) in my github
> in which I test things from time to time, and that one day, I should gather and
> post a real RFC.
> 
> 
> [1]
> https://github.com/tSed/buildroot/commit/7d53fe664afd3c98b3609994c5bf2addf2f4f692

Agreed - add host-patchelf to the packages and let some's post-build
script deal with this.

If your working on a relocatable toolchain - note that your going to
have to diddle glibc and its support binaries also.

- - -

But as requested earlier - the additional tests for the record:

The earlier ${ORIGIN} and &&{ORIGIN} cases **ARE** the same result.

Four $ case: $$$${ORIGIN}/../lib
 
 0x0000000f (RPATH)                      Library rpath: [27445{ORIGIN}/../lib]
 0x0000001d (RUNPATH)                    Library runpath: [27445{ORIGIN}/../lib]

These are out-of-tree builds -
'/home/mszick/kA8/.' is the contents of $O

Single $ case: $ORIGIN/../lib

 0x0000000f (RPATH)                      Library rpath: [/home/mszick/kA8/.RIGIN/../lib]
 0x0000001d (RUNPATH)                    Library runpath: [/home/mszick/kA8/.RIGIN/../lib]

Double $ case $$ORIGIN/../lib

 0x0000000f (RPATH)                      Library rpath: [/../lib]
 0x0000001d (RUNPATH)                    Library runpath: [/../lib]

Triple $ case $$$ORIGIN/../lib

 0x0000000f (RPATH)                      Library rpath: [home/mszick/kA8/.RIGIN/../lib]
 0x0000001d (RUNPATH)                    Library runpath: [home/mszick/kA8/.RIGIN/../lib]

Quad $ case $$$$ORIGIN/../lib

 0x0000000f (RPATH)                      Library rpath: [3092ORIGIN/../lib]
 0x0000001d (RUNPATH)                    Library runpath: [3092ORIGIN/../lib]
 
The only pattern I see here is odd/even $ count.

PS: 
rpath and runpath can be relative paths (in glibc-2.19 at least).
but the path is resolved relative to cwd, not executable's location.

Ah,
That should be ${ORIGIN} and $${ORIGIN} are the same result.
Really.

Mike

Today's (non-)progress summary:

Two issues:

First, the "Target Linker Options" string field in the configuration tree:

This field is probably of too general a use to mark it "Broken".

Now:
But a note in the help text and in the manual that the internal handling
of any ld.so tokens ($ORIGIN, $PLATFORM, etc.) in the string will not be
handled as expected.

Later:
This will probably require that we capture the output of printf in a
variable rather than set it by assignment in critical locations of the
infrastructure.

I.E: BR_SOMETHING = `printf "%s" '$' 'ORIGIN'`
or even: BR_SOMETHING = `printf "%q" '$' 'ORIGIN'`

and possibly the equivalent Make command of that shell command.

I agree with adding the note(s) now because this will probably
take a long time to resolve.

Second, the BR_TARGET_LDFLAGS handling:

The result of this handling appears to be a per-package situation.

The cases so far:
*) Not recognized
*) Correctly recognized and handled
*) Added (first) in a rpath or runpath that includes the
trusted directories.

In my (non-developer) mind, this is a matter of "normal package
maintenance" and to be fixed as found.

Printing the shell significant character(s) into a variable is confirmed
to work at the manual entry shell level.

The Cliff's Notes example:

bin # export INT=`printf '%s' '/mnt/us/extensions/system/lib/ld-linux-armhf.so.3'`
bin # echo $INT
/mnt/us/extensions/system/lib/ld-linux-armhf.so.3
core2quad bin # patchelf --set-interpreter $INT busybox

bin # export LIB=`printf '%s' '$' 'ORIGIN/../lib' '$' 'ORIGIN/../usr/lib'`
bin # echo $LIB
$ORIGIN/../lib:$ORIGIN/../usr/lib
bin # patchelf --set-rpath $LIB busybox

====

bin # readelf -dl busybox
- - - -
Dynamic section at offset 0x13a180 contains 24 entries:
  Tag        Type                         Name/Value
 0x0000001d (RUNPATH)                    Library runpath: [$ORIGIN/../lib:$ORIGIN/../usr/lib]

====

[root@kindle root]# echo $PATH
/mnt/us/extensions/system/bin:/usr/local/bin:/bin:/usr/bin:/usr/sbin:/sbin:/usr/local/python/bin

[root@kindle root]# LD_DEBUG=libs busybox
     27571:     find library=libc.so.6 [0]; searching
- - - -
     27571:     calling init: /mnt/us/extensions/system/bin/../lib/libc.so.6
- - - -
BusyBox v1.22.1 (2014-06-11 02:23:01 CDT) multi-call binary.
BusyBox is copyrighted by many authors between 1998-2012.
etc.

(sorry incorrect bug for milestone)

Mike, 

Would you care to send patches to the mailing list to implement these changes?
In a first step, the 'Now' comment, and then the alternative you proposed. Having the changes on the mailing list will facilitate the discussion and review, as few developers look at the bug comments.

Thanks, Thomas

Ah, phooey, forgot to reply to the bug discussion thread.

The 'shell' code work around example can be found here:
http://www.mobileread.com/forums/showthread.php?p=2848930&postcount=18

I do not have the required knowledge of Buildroot infrastructure to
find where / how to incorporate that into the system properly.

Yann,
You submitted a patch to update the documentation and add patchelf, but currently these patches are marked as Changes Requested.
Any chance they could still be submitted for 2014.08?

Thomas,

Yes, I'll respin them.
Thanks for the reminder.

Regards,
Yann E. MORIN.

Bug closed after documentation and workaround commits:

http://git.buildroot.org/buildroot/commit/?id=a930d9cef6bdcf8de7d086ea66d977530200b8e0
http://git.buildroot.org/buildroot/commit/?id=8f570cd765d52da56b4cebd4e5149ab8ebceec7f