| Summary: | "Value too large for defined data type" error on 64 bit machine | ||
|---|---|---|---|
| Product: | Busybox | Reporter: | chanseok |
| Component: | Other | Assignee: | unassigned |
| Status: | REOPENED --- | ||
| Severity: | normal | CC: | busybox-cvs, busybox |
| Priority: | P5 | ||
| Version: | 1.31.x | ||
| Target Milestone: | --- | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Host: | Target: | ||
| Build: | |||
| Attachments: | patch | ||
|
Description
chanseok
2019-01-28 19:30:52 UTC
Someone from the Distroless thread confirmed that the latest 1.30.0 has the same issue. They believe that the issue is uclibc and confirmed that 1.30.0 glibc works. https://github.com/GoogleContainerTools/distroless/issues/225#issuecomment-458274789 If you need 64-bit file operations, select CONFIG_LFS=y
In your link:
> donbowman commented 11 days ago •
> The issue is the 64-bit inode.
> Busybox would need to be compiled with e.g. _FILE_OFFSET_BITS=64, but then it would need to be 64-bit as well (likely) since off_t etc would change, and wouldn't fit in an int anymore.
Wrong. The binary does not need to be 64-bit. Code carefully uses off_t, not ints, for file sizes everywhere.
Closing as WORKSFORME. Reopen if compiling with CONFIG_LFS=y does not help, in which case describe the situation in more detail. Could you provide the binary compiled with CONFIG_LFS=y at https://busybox.net/downloads/binaries? Distroless debug images depend on busybox and are created by simply pulling the binary from the link. All those binaries are built with CONFIG_LFS=y. Also, struct stat::st_ino seems to be 64-bit in these builds. It's unclear which field of struct stat is not wide enough. Can you find out the exact parameters of the file which can't be stat'ed? For example, by stracing some program which successfully stat's it: strace -v ls -l FILE 2>&1 | grep FILE I can reproduce this on Microsoft Azure, but there is no strace command in the busybox container(?) I guess I need to find a static version of strace to try. (In reply to db from comment #6) > I can reproduce this on Microsoft Azure, but there is no strace command https://busybox.net/downloads/binaries/strace_static_x86_64 Its not reproducing right now, i'm not sure why. It might relate to the uptime of the nodes etc. When it was happening, it was 100%. it was on the /proc filesystem (so e.g. 'ps' would return an error), and on the overlayfs (so e.g. ls / would yield: "ls: can't open '/': Value too large for defined data type") switching from the ucLibc version to the glibc version resolved the issue. I'll keep trying. OK here it is reproduced. Using https://github.com/GoogleContainerTools/distroless debug image (gcr.io/distroless/base:debug) # busybox BusyBox v1.27.1 (2017-07-18 19:24:44 CEST) multi-call binary. /tmp # /usr/bin/strace -f ps execve("/busybox/ps", ["ps"], 0x7ffcff53ba68 /* 16 vars */) = 0 /usr/bin/strace: [ Process PID=21 runs in 32 bit mode. ] /usr/bin/strace: WARNING: Proper structure decoding for this personality is not supported, please consider building strace with mpers support enabled. ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0 getuid32() = 0 brk(NULL) = 0x9d28000 brk(0x9d29000) = 0x9d29000 ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(0, TIOCGWINSZ, {ws_row=38, ws_col=120, ws_xpixel=0, ws_ypixel=0}) = 0 write(1, "PID USER TIME COMMAND\n", 30PID USER TIME COMMAND ) = 30 open("/proc", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3 fstat(3, 0xffa851a4) = -1 EOVERFLOW (Value too large for defined data type) close(3) = 0 write(2, "ps: can't open '/proc': Value to"..., 62ps: can't open '/proc': Value too large for defined data type ) = 62 exit(1) = ? +++ exited with 1 +++ and /proc # strace ls -l execve("/busybox/ls", ["ls", "-l"], 0x7ffcdc43ede8 /* 16 vars */) = 0 strace: [ Process PID=33 runs in 32 bit mode. ] strace: WARNING: Proper structure decoding for this personality is not supported, please consider building strace with mpers support enabled. ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0 ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0 getuid32() = 0 time([1551486393 /* 2019-03-02T00:26:33+0000 */]) = 1551486393 (2019-03-02T00:26:33+0000) ioctl(0, TIOCGWINSZ, {ws_row=38, ws_col=120, ws_xpixel=0, ws_ypixel=0}) = 0 ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0 brk(NULL) = 0x9768000 brk(0x9769000) = 0x9769000 lstat64(".", 0xffa6e374) = 0 open(".", O_RDONLY|O_NONBLOCK|O_CLOEXEC|O_DIRECTORY) = 3 fstat(3, 0xffa6e378) = -1 EOVERFLOW (Value too large for defined data type) close(3) = 0 write(2, "ls: can't open '.': Value too la"..., 58ls: can't open '.': Value too large for defined data type ) = 58 write(1, "total 0\n", 8total 0 ) = 8 exit(1) = ? +++ exited with 1 +++ # ./strace_static_x86_64 -v -v ls -l /proc
num_quals=312
--- stopped by SIGSTOP ---
--- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=73, si_uid=0} ---
execve("/busybox/ls", ["ls", "-l", "/proc"], ["KUBERNETES_SERVICE_PORT=443", "KUBERNETES_PORT=tcp://noc-noc-18"..., "HOSTNAME=test", "SHLVL=1", "OLDPWD=/usr", "HOME=/home", "SSL_CERT_FILE=/etc/ssl/certs/ca-"..., "TERM=xterm", "KUBERNETES_PORT_443_TCP_ADDR=noc"..., "PATH=/usr/local/sbin:/usr/local/"..., "KUBERNETES_PORT_443_TCP_PORT=443", "KUBERNETES_PORT_443_TCP_PROTO=tc"..., "KUBERNETES_SERVICE_PORT_HTTPS=44"..., "KUBERNETES_PORT_443_TCP=tcp://no"..., "PWD=/tmp", "KUBERNETES_SERVICE_HOST=noc-noc-"...]) = 0
[ Process PID=76 runs in 32 bit mode. ]
ioctl(0, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {c_iflags=0x500, c_oflags=0x5, c_cflags=0xbf, c_lflags=0x8a3b, c_line=0, c_cc="\x03\x1c\x7f\x15\x04\x00\x01\x00\x11\x13\x1a\x00\x12\x0f\x17\x16\x00\x00\x00"}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {c_iflags=0x500, c_oflags=0x5, c_cflags=0xbf, c_lflags=0x8a3b, c_line=0, c_cc="\x03\x1c\x7f\x15\x04\x00\x01\x00\x11\x13\x1a\x00\x12\x0f\x17\x16\x00\x00\x00"}) = 0
getuid32() = 0
time([1551486719]) = 1551486719
ioctl(0, TIOCGWINSZ, {ws_row=38, ws_col=120, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {c_iflags=0x500, c_oflags=0x5, c_cflags=0xbf, c_lflags=0x8a3b, c_line=0, c_cc="\x03\x1c\x7f\x15\x04\x00\x01\x00\x11\x13\x1a\x00\x12\x0f\x17\x16\x00\x00\x00"}) = 0
brk(0) = 0x9a97000
brk(0x9a98000) = 0x9a98000
lstat64("/proc", {st_dev=makedev(0, 259), st_ino=1, st_mode=S_IFDIR|0555, st_nlink=299, st_uid=0, st_gid=0, st_blksize=1024, st_blocks=0, st_size=0, st_atime=2019/03/02-00:23:48, st_mtime=2019/03/02-00:23:48, st_ctime=2019/03/02-00:23:48}) = 0
open("/proc", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
fstat(3, 0xffe905e8) = -1 EOVERFLOW (Value too large for defined data type)
close(3) = 0
write(2, "ls: can't open '/proc': Value to"..., 62ls: can't open '/proc': Value too large for defined data type
) = 62
write(1, "total 0\n", 8total 0
) = 8
_exit(1) = ?
+++ exited with 1 +++
Per db@donbowman.ca (https://github.com/GoogleContainerTools/distroless/issues/225#issuecomment-458274789 and https://github.com/GoogleContainerTools/distroless/issues/225#issuecomment-469741965), it is suggested that busybox using glibc don't suffer from this issue. Does busybox provide binaries complied with glibc at https://busybox.net/downloads/binaries/? (Doesn't seem likely.) If you provide them, I think we could use those binaries as a workaround. Here's an example with a regular file, from my desktop rather than Azure
BusyBox v1.30.0 (2018-12-30 22:25:27 CET) multi-call binary.
~ $ cd /etc
/etc $ /strace /busybox ls -l
num_quals=312
--- stopped by SIGSTOP ---
--- SIGCONT {si_signo=SIGCONT, si_code=SI_USER, si_pid=6, si_uid=1000} ---
execve("/busybox", ["/busybox", "ls", "-l"], [/* 8 vars */]) = 0
[ Process PID=9 runs in 32 bit mode. ]
ioctl(0, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
getuid32() = 1000
stat64("/etc/busybox.conf", 0xff81fe10) = -1 ENOENT (No such file or directory)
getgid32() = 1000
setgid32(1000) = 0
setuid32(1000) = 0
time([1552432528]) = 1552432528
ioctl(0, TIOCGWINSZ, {ws_row=57, ws_col=176, ws_xpixel=0, ws_ypixel=0}) = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, {B38400 opost isig icanon echo ...}) = 0
brk(0) = 0x987f000
brk(0x9880000) = 0x9880000
lstat64(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
open(".", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
fstat(3, 0xff81fe9c) = -1 EOVERFLOW (Value too large for defined data type)
close(3) = 0
write(2, "ls: can't open '.': Value too la"..., 58ls: can't open '.': Value too large for defined data type
) = 58
write(1, "total 0\n", 8total 0
) = 8
_exit(1) = ?
Created attachment 8091 [details]
patch
This is an uclibc bug
Reopening. Based on #comment13 on Jun 9 that says this was due to a bug in uClibc and that a new BusyBox 1.31.1 was released on Jun 10, I thought 1.31.1 contains the fix for this issue. However, this still happens: https://github.com/GoogleContainerTools/distroless/issues/225#issuecomment-574295852 The binary being used is https://busybox.net/downloads/binaries/1.31.0-i686-uclibc/busybox. Correction to #comment14: the version is 1.31.0, not 1.31.1. Still issue exists with v1.31.0 / $ busybox BusyBox v1.31.0 (2019-06-10 15:13:14 CEST) multi-call binary. BusyBox is copyrighted by many authors between 1998-2015. Licensed under GPLv2. See source distribution for detailed copyright notices. / $ ls ls: can't open '.': Value too large for defined data type |