Created attachment 6781 [details] my experimental fix or workaround I'm unable to build a UBI image regardless of whether I use the provided ubinize.cfg or the new custom config option. In case it's useful, the build host is Fedora 24. Sanitized output: output/foo/build/_device_table.txt' iniparser: cannot open output/foo/build/ubinize.cfg ubinize: error!: cannot load the input ini file "output/build/ubinize.cfg" I think that the issue is that ubinize.cfg is being copied and edited under fakeroot and the permissions mask is wrong. As an experiment I added another routine that's called directly (rather than printed into _fakeroot.sh) and moved the ubi.mk _CMD to my new _SETUP_CMD and that resolves the problem. I'm not sure if something like this is a reasonable solution and I don't understand why copying/sed/etc is done under fakeroot to begin with: --- a/buildroot/fs/common.mk +++ b/buildroot/fs/common.mk @@ -95,6 +95,7 @@ $$(foreach s,$$(call qstrip,$$(BR2_ROOTFS_POST_FAKEROOT_SCRIPT)),\ echo "echo '$$(TERM_BOLD)>>> Executing fakeroot script $$(s)$$(TERM_RESET)'" >> $$(FAKEROOT_SCRIPT); \ echo $$(s) $$(TARGET_DIR) $$(BR2_ROOTFS_POST_SCRIPT_ARGS) >> $$(FAKEROOT_SCRIPT)$$(sep)) + $$(call $$(ROOTFS_$(2)_SETUP_CMD)) $$(call PRINTF,$$(ROOTFS_$(2)_CMD)) >> $$(FAKEROOT_SCRIPT) chmod a+x $$(FAKEROOT_SCRIPT) PATH=$$(BR_PATH) $$(HOST_DIR)/usr/bin/fakeroot -- $$(FAKEROOT_SCRIPT) --- a/buildroot/fs/ubifs/ubi.mk +++ b/buildroot/fs/ubifs/ubi.mk @@ -20,7 +20,7 @@ UBINIZE_CONFIG_FILE_PATH = fs/ubifs/ubinize.cfg endif -define ROOTFS_UBI_CMD +define ROOTFS_UBI_SETUP_CMD $(INSTALL) -m 0644 $(UBINIZE_CONFIG_FILE_PATH) $(BUILD_DIR)/ubinize.cfg $(SED) 's;BR2_ROOTFS_UBIFS_PATH;$@fs;' $(BUILD_DIR)/ubinize.cfg $(HOST_DIR)/usr/sbin/ubinize -o $@ $(UBI_UBINIZE_OPTS) $(BUILD_DIR)/ubinize.cfg
Andrey, Thanks for your report. I've tried to run atmel_sama5d4_xplained_defconfig that uses ubinize, but was not able to reproduce your issue. Note that we already had a very similar issue a while back, but it was marked as "worksforme" and closed because we had no precise explanations on how to reproduce: https://bugs.busybox.net/show_bug.cgi?id=9096 Since you have that issue now, would you care to provide a defconfig file that exhibits the problem, so we can try to reproduce and invetigate further, please? Regards, Yann E. MORIN.
Created attachment 6786 [details] approximate config I've attached an approximate defconfig (sanitized to remove paths that have to do with my product specifically, sorry for the inconvenience). The SoC is an Atmel AT91SAM9G20 so it's similar to the config you're testing with. One difference is there's both a CPIO/initramfs and a UBI image being generated but hopefully that's not important. I'm building with atmel_sama5d4_xplained_defconfig to see if I can reproduce the problem, sorry for not doing that first.
Andrey, There's not much we can do with this config file: all path are pointing to non-existing files, so the build fails horribly... Let's see if you can reproduce with the sama5d4 board. > sorry for not doing that first No problem! ;-) Regards, Yann E. MORIN.
Ok, I've reproduced it with your defconfig and I've left all paths in here. To build I just ran: make atmel_sama5d4_xplained_defconfig make And in the end the output is: >>> Generating root filesystem image rootfs.ubi rm -f /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs rm -f /home/ayurovsky/dev/build-system/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM rm -f /home/ayurovsky/dev/build-system/buildroot/output/build/_users_table.txt echo '#!/bin/sh' > /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs echo "set -e" >> /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs echo "chown -h -R 0:0 /home/ayurovsky/dev/build-system/buildroot/output/target" >> /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs printf ' \n' >> /home/ayurovsky/dev/build-system/buildroot/output/build/_users_table.txt PATH="/home/ayurovsky/dev/build-system/buildroot/output/host/bin:/home/ayurovsky/dev/build-system/buildroot/output/host/sbin:/home/ayurovsky/dev/build-system/buildroot/output/host/usr/bin:/home/ayurovsky/dev/build-system/buildroot/output/host/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/ayurovsky/.local/bin:/home/ayurovsky/bin" /home/ayurovsky/dev/build-system/buildroot/support/scripts/mkusers /home/ayurovsky/dev/build-system/buildroot/output/build/_users_table.txt /home/ayurovsky/dev/build-system/buildroot/output/target >> /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs cat system/device_table.txt > /home/ayurovsky/dev/build-system/buildroot/output/build/_device_table.txt printf ' /bin/busybox f 4755 0 0 - - - - -\n /dev/console c 622 0 0 5 1 - - -\n\n' >> /home/ayurovsky/dev/build-system/buildroot/output/build/_device_table.txt echo "/home/ayurovsky/dev/build-system/buildroot/output/host/usr/bin/makedevs -d /home/ayurovsky/dev/build-system/buildroot/output/build/_device_table.txt /home/ayurovsky/dev/build-system/buildroot/output/target" >> /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs printf ' /usr/bin/install -m 0644 fs/ubifs/ubinize.cfg /home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg\n /usr/bin/sed -i -e '\''s;BR2_ROOTFS_UBIFS_PATH;/home/ayurovsky/dev/build-system/buildroot/output/images/rootfs.ubifs;'\'' /home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg\n /home/ayurovsky/dev/build-system/buildroot/output/host/usr/sbin/ubinize -o /home/ayurovsky/dev/build-system/buildroot/output/images/rootfs.ubi -m 0x1000 -p 0x40000 /home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg\n rm /home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg\n' >> /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs chmod a+x /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs PATH="/home/ayurovsky/dev/build-system/buildroot/output/host/bin:/home/ayurovsky/dev/build-system/buildroot/output/host/sbin:/home/ayurovsky/dev/build-system/buildroot/output/host/usr/bin:/home/ayurovsky/dev/build-system/buildroot/output/host/usr/sbin:/usr/lib64/qt-3.3/bin:/usr/lib64/ccache:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/ayurovsky/.local/bin:/home/ayurovsky/bin" /home/ayurovsky/dev/build-system/buildroot/output/host/usr/bin/fakeroot -- /home/ayurovsky/dev/build-system/buildroot/output/build/_fakeroot.fs rootdir=/home/ayurovsky/dev/build-system/buildroot/output/target table='/home/ayurovsky/dev/build-system/buildroot/output/build/_device_table.txt' iniparser: cannot open /home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg ubinize: error!: cannot load the input ini file "/home/ayurovsky/dev/build-system/buildroot/output/build/ubinize.cfg" /usr/bin/install -m 0644 support/misc/target-dir-warning.txt /home/ayurovsky/dev/build-system/buildroot/output/target/THIS_IS_NOT_YOUR_ROOT_FILESYSTEM
Nothing weird in your log... Can you check the permissions of output/build/ubinize.cfg? And its contents? Do you have ACLs, SMACK, SELinux or anything like that? I really don't see how running ubinize from without or outside of fakeroot could make a difference... Maybe add an strace in front of the ubinize command in ubi.mk to see what is really going on? I do think it's a good idea to do the ubinize outside of fakeroot. However, I'd like to understand why this is happening - if it happens for ubinize, it could happen for something else as well...
I have SELinux but it's set to 'permissive' right now, nothing else is configured differently from a standard Fedora 24 install. ubi.mk rm's the ubinize.cfg file, if I comment that out it'll be left behind and then: $ stat output/build/ubinize.cfg File: 'output/build/ubinize.cfg' Size: 124 Blocks: 8 IO Block: 4096 regular file Device: fd03h/64771d Inode: 1596887 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 1000/ayurovsky) Gid: ( 1000/ayurovsky) Context: unconfined_u:object_r:user_home_t:s0 Access: 2016-10-27 15:10:37.142923074 -0700 Modify: 2016-10-27 15:10:37.142923074 -0700 Change: 2016-10-27 15:10:37.142923074 -0700 Birth: - What I can't explain is that if instead I leave everything alone and just insert a line like: stat $(BUILD_DIR)/ubinize.cf in ubi.mk (in its _CMD), stat reports that the path doesn't exist (as does ubinize). So why does it exist if I remove the last line in _CMD (the rm) but it doesn't exist after the first line (the $(INSTALL)) ... ?
Ok, I found a better workaround that may explain some of this (plus the original workaround I posted must be wrong, please ignore that). If I install (via dnf in my case) fakeroot and then in fs/common.mk change: PATH=$$(BR_PATH) $$(HOST_DIR)/usr/bin/fakeroot -- $$(FAKEROOT_SCRIPT) to just: fakeroot -- $$(FAKEROOT_SCRIPT) ...everything works fine. So the issue has something to do with the host tools provided fakeroot and/or the path handed to that. They're both version 1.20.2 so I'm guessing it's some path or environment issue. For now I'll patch common.mk to eliminate the buildroot-provided fakeroot but I would love to help get to the bottom of this (or understand what I'm doing wrong). Thanks!
Andrey, What distribution are you using? Do you have LD_LIBRARY_PATH and/or LD_PRELOAD in your environment? If so, that are their values? Regards, Yann E. MORIN.
Can reproduce on fc23. Upstream related bugs: https://bugzilla.redhat.com/show_bug.cgi?id=1238802 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=799858 It seems the selinux context isn't picked up by fakeroot and causes trouble. Unfortunately there seems to be no patch at this time to address this, so running ubinize outside fakeroot looks like the only option.
Gustavo, Andrey, OK, so this is a known issue with fakeroot relating to xattrs. There is nothing we can do in Buildroot, except as you said, to run outside fakeroot. But that solution means we need to tweak our fs infra just for the sake of a bug in fakeroot. That's not nice... However, following the bug report Gustavo copied, I got to: https://www.yoctoproject.org/tools-resources/projects/pseudo As they say: > Pseudo is a program that can either be used directly or as > an LD_PRELOAD, which allows these operations to succeed as > if the user did have system administrator privileges even > though they are an ordinary user. > > Pseudo has a lot of similarities to fakeroot but is a new > implementation that improves on the problems seen using > fakeroot. Maybe we should also go with pseudo in Buildroot, and ditch fakeroot? I'll try to have a look at that, and see if I can come up with a patch. Regards, Yann E. MORIN.
Andrey, All, I just sent two patches to try and fix this issue: https://patchwork.ozlabs.org/patch/688725/ https://patchwork.ozlabs.org/patch/688726/ Could you try them and report, please? Regards, Yann E. MORIN.
Fixed by switching from fakeroot to pseudo to build the root filesystem, done in commit https://git.buildroot.org/buildroot/commit/?id=8035ceb56cc3ba2541aba0d0ead3f29900d9e310. Thanks Yann for providing the patches, and thanks Andrey for reporting the problem in the first place!
I just tested the following defconfig: BR2_arm=y BR2_cortex_a8=y BR2_TOOLCHAIN_EXTERNAL=y # BR2_TARGET_ROOTFS_TAR is not set BR2_TARGET_ROOTFS_UBIFS=y BR2_TARGET_ROOTFS_UBI=y and built it under a Fedora 24 machine, and it built just fine, with Buildroot 2016.08.1 (which uses fakeroot) So I'm not sure why we have moved to pseudo because of this bug.
Hum, my apologies, there is indeed the message: iniparser: cannot open /home/test/buildroot/output/build/ubinize.cfg ubinize: error!: cannot load the input ini file "/home/test/buildroot/output/build/ubinize.cfg" and indeed, the UBIFS image is produced but not the UBI image. So I can reproduce the issue.
And indeed, installing Fedora's fakeroot, and changing $$(HOST_DIR)/usr/bin/fakeroot to just /usr/bin/fakeroot makes everything work. So there is a difference in the fakeroot source code, or the way we build it, we makes our fakeroot behave badly on SELinux enabled systems.
Switching to pseudo brought a number of issues, so I have reverted that and instead added a workaround to the ubifs package: https://git.buildroot.org/buildroot/commit/?id=01354e1a0f4333ffee40a20260a5689e3846560c
Thanks, that sed -i behavior issue makes a lot more sense and definitely explains it.