Bug 11366

Summary: [2018.08] SysV IPC not available for fakeroot on WSL
Product: buildroot Reporter: J.F. <jfdoyon>
Component: OtherAssignee: Yann E. MORIN <yann.morin.1998>
Status: RESOLVED FIXED    
Severity: normal CC: buildroot
Priority: P5    
Version: unspecified   
Target Milestone: ---   
Hardware: PC   
OS: Windows   
Host: Target:
Build:
Attachments: Proposed patch for configure.ac
New fakeroot makefile
Improved fakeroot makefile

Description J.F. 2018-09-23 20:33:25 UTC
When using Ubuntu on Windows Susbsystem for Linux (WSL) as your host build environment, the process fails on the execution of "fakeroot", with an error message to the effect that "fakeroot has failed due to SysV IPC not being available".

This is indeed correct, as the WSL system doesn't have the functionality.

A WSL bug has been filed: https://github.com/Microsoft/WSL/issues/2465

The workaround is to copy/link the host /usr/bin/fakeroot-tcp into the builroot host output, and re-run, it then succeeds no problem.

Suggestion: detect availability of SysV IPC and/or TCP, and build/use the appropriate fakeroot. If the host doesn't have either facility available, warn early (instead of finding out at the very end of a possibly long compilation run).

Thanks,
J.F.
Comment 1 Thomas Petazzoni 2018-09-23 20:46:08 UTC
Thanks for the bug report. I'd say this is something that should be improved in fakeroot configure script itself: it should detect if SysV IPC is available or not, and fallback to TCP if not.

Is this something you could have a look at ?
Comment 2 J.F. 2018-09-23 22:49:08 UTC
I'll give it a try!
Comment 3 J.F. 2018-09-24 01:47:25 UTC
findings so far:

OK, so WSL does have some SysV IPC, but no message Q's, which is the feature needed by fakeroot/faked by default.

The cleanest way to test this appears to be to use the ipcmk utility provided by the host util-linux package ... only if the user selects the host util package in his config however.  If ipcmk -Q returns 1, use TCP.

Also, the fakeroot package builds sysv OR tcp. In order to support both, you run 2 builds, presumably configured as two packages. Ideally, you would build only one, based on the availability opf message Qs, if at all possible.

So one option is to force the host-util-linux package with most programs (force the relevant BR2 option on, or mor eprecisely, remove the option and always have it on), and then re-package 2 fakeroots and then use ipcmk at run time to select the right fakeroot.

Another might be to simply give users the ability to select "TCP Fakeroot" in their config via a BR2 variable. Since this is an edge case, it seems it might be simpler to just do it that way, not unlike the way the host OpenSSL and libelf requirements exist for the kernel.

Since this dependency is on a kernel system call, there's no clean way to simply patch the existing fakeroot package and configure script ONLY, as Thomas suggested. Maybe I could patch the configure script to use ipcmk however ... ? I know *nothing* about how to work with autoconf however :P Because normally you would build this tool as part of a distro on a separate host, dynamic detection of IPC Message Q's would be of limited value "upstream" to the fakeroot developers anyways ...

I think i would tend towards providing a user selectable config for "TCP fakeroot" as a my favourite solution.

Thoughts?
Comment 4 J.F. 2018-09-24 01:59:38 UTC
Wait, I lied you can write custom tests in autoconf to test the actual presence of a function ... I can see whether I can get a handle on that ... although a user option still seems much simpler :)
Comment 5 Thomas Petazzoni 2018-09-24 06:48:48 UTC
Please don't use ipcmk, as requiring to build host-util-linux is clearly not nice.

Instead, we want a simple test program in C, that will be compiled/executed by fakeroot configure.ac script, to decide whether SysV message queues or TCP should be used. You can use autoconf AC_TRY_RUN() to achieve this.
Comment 6 J.F. 2018-09-24 15:18:20 UTC
Created attachment 7791 [details]
Proposed patch for configure.ac

Add a check to the configure script to test whether SysV IPC message queues are actually functional. Fall back to TCP if not.
Comment 7 J.F. 2018-09-24 15:21:34 UTC
Created attachment 7796 [details]
New fakeroot makefile

The new fakeroot makefile, which handles running autoreconf to re-generate the configure script after patch 0003 has been applied.
Comment 8 J.F. 2018-09-24 15:28:30 UTC
See recently attached files.

This will need to be tested on a normally functioning linux host with SysV IPC Message Q's running. Notably, use ipcs to make sure the message Q is cleaned up properly after the test. The code is there and quite simple, but I can't test it.

Also, the autoreconf setup is a little out of the ordinary, due to some out-of-the ordinary libtool requirements by fakeroot it seems.

This now works cleanly on current Ubuntu for WSL.

Let me know if you have more suggestions for improvements?

Cheers,
J.F.
Comment 9 J.F. 2018-09-24 23:57:34 UTC
Created attachment 7801 [details]
Improved fakeroot makefile

Improved makefile, added some missing package dependencies.
Comment 10 Thomas Petazzoni 2018-11-02 09:55:53 UTC
J.F, could you send a proper Buildroot patch on the mailing list, which includes the fakeroot source code changes and the fakeroot.mk changes, all with a proper description ?
Comment 11 Peter Korsgaard 2019-03-12 09:24:14 UTC
Or even better, send the patch to upstream fakeroot and get it included there
Comment 12 Peter Korsgaard 2019-11-04 21:29:03 UTC
Fixed by: https://git.buildroot.org/buildroot/commit/?id=fd1bcce9894773d7e1c1ca5be0f04b0b32128a1d

commit fd1bcce9894773d7e1c1ca5be0f04b0b32128a1d
Author: Matt Weber <matthew.weber@rockwellcollins.com>
Date:   Mon Sep 16 22:58:14 2019 -0500

    package/fakeroot: test for SYSV IPC support

    fakeroot can be built to either use SYSV IPC or TCP for message passing.

    A bug was discovered where Microsoft Windows 10 Services for Linux
    doesn't include support for SYSV IPC MsgQ.  This patch adds support to
    detect this case and automatically build fakeroot to use the TCP
    transport instead (It is assumed a TCP transport would definitely have
    more overhead then MsgQs so the default wasn't changed to TCP).

    Fixes
    https://bugs.busybox.net/show_bug.cgi?id=11366

    Cc: Yann E. MORIN <yann.morin.1998@free.fr>
    Signed-off-by: Jean-Francois Doyon <jfdoyon@gmail.com>
    Signed-off-by: Matthew Weber <matthew.weber@rockwellcollins.com>
    [Arnout: use a post-patch hook and AUTORECONF=YES]
    Signed-off-by: Arnout Vandecappelle (Essensium/Mind) <arnout@mind.be>