Bug 9531

Although you may observe the npm command has been modified to include "-d" and "--build-from-source" switches, these make no difference to the error output.

Please include the Buildroot version, and a minimal Buildroot .config file that allows to reproduce the issue. Thanks!

The version is specified in the ticket (2016.11).

Here is the config.

BR2_x86_64=y
BR2_WGET="wget --passive-ftp -nd -t 3 -nv"
BR2_TOOLCHAIN_EXTERNAL=y
BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y
BR2_TOOLCHAIN_EXTERNAL_PATH="$(HOME)/toolchain"
BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="$(ARCH)-nano-linux-uclibc"
BR2_TOOLCHAIN_EXTERNAL_GCC_5=y
BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_3=y
BR2_TOOLCHAIN_EXTERNAL_WCHAR=y
BR2_TOOLCHAIN_EXTERNAL_HAS_SSP=y
BR2_TOOLCHAIN_EXTERNAL_CXX=y
BR2_TARGET_GENERIC_HOSTNAME="Docker-nano"
BR2_TARGET_GENERIC_ISSUE="Docker nano"
BR2_INIT_NONE=y
# BR2_TARGET_GENERIC_GETTY is not set
BR2_ROOTFS_OVERLAY="rootfs_overlay"
BR2_ROOTFS_POST_BUILD_SCRIPT="./post_build.sh"
# BR2_PACKAGE_BUSYBOX is not set
BR2_PACKAGE_NODEJS=y
BR2_PACKAGE_NODEJS_MODULES_ADDITIONAL="ghost@0.11.3"
BR2_PACKAGE_OPENSSL=y
BR2_TARGET_ROOTFS_TAR_XZ=y

This uses the toolchain available from https://github.com/Docker-nano/crosstool-NG/releases/download/2.0.0/x86_64-nano-linux-uclibc.tar.xz

Further investigation reveals the culprit is the `-g` switch in the line, `$(NPM) install -g $(NODEJS_MODULES_LIST)` in package/nodejs/nodejs.mk.

Removing the `-g` switch allows the module and all its dependencies, including sqlite3, to build successfully. However, since they are now built in a different location, they are not included in the rootfs which defeats the purpose of building them!

My knowledge of Buildroot is too poor to know why an access denied violation occurs with `-g` or how to include the modules in the rootfs when the `-g` option is omitted, however I know the 2014.08 release of Buildroot did not rely on `-g` and this is the last known release version I could get this module working on.

I cannot reproduce it here. The error message refers to node-pre-grep which also uses /usr/bin/env, so presumably this is related to #9526?

Hi Peter, unfortunately I cannot stop producing this error. I do not believe this is related to `env` on the target at all because I just spent the entire day upgrading my toolchain and rebuilding everything to find exactly the same error is produced even when `env` is successfully built and installed to the target prior to running the NPM module build process.

NPM tries to run the following command:

PATH="/root/buildroot-2016.11/output/host/bin:/root/buildroot-2016.11/output/host/sbin:/root/buildroot-2016.11/output/host/usr/bin:/root/buildroot-2016.11/output/host/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" AR="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-ar" AS="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-as" LD="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-ld" NM="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-nm" CC="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-gcc" GCC="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-gcc" CPP="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-cpp" CXX="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-g++" FC="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-gfortran" F77="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-gfortran" RANLIB="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-ranlib" READELF="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-readelf" STRIP="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-strip" OBJCOPY="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-objcopy" OBJDUMP="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-objdump" AR_FOR_BUILD="/usr/bin/ar" AS_FOR_BUILD="/usr/bin/as" CC_FOR_BUILD="/usr/bin/gcc" GCC_FOR_BUILD="/usr/bin/gcc" CXX_FOR_BUILD="/usr/bin/g++" LD_FOR_BUILD="/usr/bin/ld" CPPFLAGS_FOR_BUILD="-I/root/buildroot-2016.11/output/host/usr/include" CFLAGS_FOR_BUILD="-O2 -I/root/buildroot-2016.11/output/host/usr/include" CXXFLAGS_FOR_BUILD="-O2 -I/root/buildroot-2016.11/output/host/usr/include" LDFLAGS_FOR_BUILD="-L/root/buildroot-2016.11/output/host/lib -L/root/buildroot-2016.11/output/host/usr/lib -Wl,-rpath,/root/buildroot-2016.11/output/host/usr/lib" FCFLAGS_FOR_BUILD="" DEFAULT_ASSEMBLER="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-as" DEFAULT_LINKER="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-ld" CPPFLAGS="-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64" CFLAGS="-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64  -Os " CXXFLAGS="-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64  -Os " LDFLAGS="" FCFLAGS=" -Os " FFLAGS=" -Os " PKG_CONFIG="/root/buildroot-2016.11/output/host/usr/bin/pkg-config" STAGING_DIR="/root/buildroot-2016.11/output/host/usr/x86_64-buildroot-linux-uclibc/sysroot" INTLTOOL_PERL=/usr/bin/perl LD="/root/buildroot-2016.11/output/host/usr/bin/x86_64-nano-linux-uclibc-g++" npm_config_arch=x64 npm_config_target_arch=x64 npm_config_build_from_source=true npm_config_nodedir=/root/buildroot-2016.11/output/build/nodejs-6.9.2 npm_config_prefix=/root/buildroot-2016.11/output/target/usr /root/buildroot-2016.11/output/host/usr/bin/npm install -g ghost@0.11.3

Which spits out the same errors mentioned previously:

npm WARN lifecycle ghost@0.11.3~preinstall: cannot run in wd %s %s (wd=%s) ghost@0.11.3 node core/server/utils/npm/preinstall.js /root/buildroot-2016.11/output/target/usr/lib/node_modules/.staging/ghost-88d37318

> sqlite3@3.1.8 install /root/buildroot-2016.11/output/target/usr/lib/node_modules/ghost/node_modules/sqlite3
> node-pre-gyp install --fallback-to-build

sh: 1: node-pre-gyp: Permission denied
/root/buildroot-2016.11/output/target/usr/lib
`-- (empty)

npm ERR! Linux 3.13.0-68-generic
npm ERR! argv "/root/buildroot-2016.11/output/host/usr/bin/node" "/root/buildroot-2016.11/output/host/usr/bin/npm" "install" "-g" "ghost@0.11.3"
npm ERR! node v6.9.2
npm ERR! npm  v3.10.9
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn

npm ERR! sqlite3@3.1.8 install: `node-pre-gyp install --fallback-to-build`
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the sqlite3@3.1.8 install script 'node-pre-gyp install --fallback-to-build'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the sqlite3 package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node-pre-gyp install --fallback-to-build
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs sqlite3
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls sqlite3
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /root/buildroot-2016.11/package/nodejs/npm-debug.log
npm ERR! code 1

As before, simply removing the `-g` flag results in a successful build in whatever the working directory happens to be. This is true for literally any directory because I'm root and I can write everywhere, so I do not understand where the "sh: 1: node-pre-gyp: Permission denied" error comes from when the `-g` flag is specified.

It seems this issue has been extensively documented at https://github.com/mapbox/node-sqlite3/issues/612 over the past 9 months. The solution, when building as root, is simply to specify the --unsafe-perm option to NPM. I include a patch, below.

+++ package/nodejs/nodejs.mk
@@ -160,7 +160,7 @@
 	# If you're having trouble with module installation, adding -d to the
 	# npm install call below and setting npm_config_rollback=false can both
 	# help in diagnosing the problem.
-	$(NPM) install -g $(NODEJS_MODULES_LIST)
+	$(NPM) install -g --unsafe-perm $(NODEJS_MODULES_LIST)
 endef
 endif

Bilge,

Nothing in Buildroot requires building as root. Building as root is not
supported (at least, not officially).

I'm not sure we want to take this patch. Playing with permissions (as
the name seems to imply) is not very clean IMHO. What does the doc about
--unsafe-perms says exactly?

Regards,
Yann E. MORIN.

If you don't want it, that's your prerogative, I'm just trying to help.

The documentation is here: https://docs.npmjs.com/misc/config#unsafe-perm
Copied in for completeness:

unsafe-perm

    Default: false if running as root, true otherwise
    Type: Boolean

Set to true to suppress the UID/GID switching when running package scripts. If set explicitly to false, then installing as a non-root user will fail.

(In reply to bilge from comment #7)

Why are you building as root in the first place? Is that some kind of side effect of docker?

It has nothing to do with Docker; you can think of Docker like any other virtual machine. I build as root (a) because I do not care about users or permissions in a VM and (b) because I had no reason to believe Buildroot would not work when running as root.

(In reply to bilge from comment #11)

Ok, but I would strongly suggest not building as root. Several packages behave differently when built as root (E.G. your npm issues), and Buildroot is specifically made to not require root permissions and be used from normal users. E.G. from the manual:

Important: you can and should build everything as a normal user. There is no need to be root to configure and use Buildroot. By running all commands as a regular user, you protect your system against packages behaving badly during compilation and installation.

I tested this and setting BR2_PACKAGE_NODEJS_MODULES_ADDITIONAL="ghost@0.11.3" builds fine when building as a non-root user.

What is happening here is npm is dropping it's root privileges before running node-pre-gyp which is why you see "node-pre-gyp: Permission denied" because it can't access the files under /root.  --unsafe-perm stops this behaviour which is why it then works.

I agree with Peter, building as root should be avoided.  I would recommend you modify your Dockerfile to build as a non-root user.

The one change which might be worth making is a recommendation against building as root in the manual and/or the Makefile.

(In reply to Martin from comment #13)

There is already a blurb in the manual that says that running as root is not recommended:

    https://buildroot.org/downloads/manual/manual.html#_buildroot_quick_start

Closing this bug, as it is not a bug in Buildroot but in the environment.

Thanks!

Regards,
Yann E. MORIN.