Bug 11441 - xargs: fails with "argument line too long"
Summary: xargs: fails with "argument line too long"
Status: NEW
Alias: None
Product: Busybox
Classification: Unclassified
Component: Standard Compliance (show other bugs)
Version: 1.29.x
Hardware: All Linux
: P5 blocker
Target Milestone: ---
Assignee: unassigned
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-10-26 22:16 UTC by Steffen Nurpmeso
Modified: 2018-11-03 20:45 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:


Attachments
boiled down test script(s) (4.72 KB, application/gzip)
2018-11-02 22:34 UTC, Steffen Nurpmeso
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Steffen Nurpmeso 2018-10-26 22:16:36 UTC
On a newly installed AlpineLinux edge (without findutils) it happened that a perl script which creates quoted arguments and pipes them to a started $HOOK, a shell script which then calls "xargs tar .." fails with this.
(Installing findutils overcame this.)
Comment 1 Denys Vlasenko 2018-10-30 09:58:36 UTC
Can you attach the text file and exact command which causes this?

xargs tar {what args here?} <FILE
Comment 2 Steffen Nurpmeso 2018-10-31 20:53:05 UTC
Sigh, my mail did not get through?  Copy and pasting the content, please wait:

Hmm, the backup script does

   sub call{
      unless(open HOOK, "| $HOOK " . ($COMPLETE || $RESET) .
            " $^O >>$MFFN 2>&1"){
         ::err(1, "Failed to create hook pipe: $^E");
         ::do_exit(1)
      }else{
         my ($stop, $listref) = (0, Filelist::get_listref());
         local *hdl = sub{ $stop = 1 };
         local $SIG{PIPE} = \&hdl;
         foreach my $p (@$listref){
            last if $stop;
            $p =~ s/\"/\"\\\"\"/g;
            $p = '"' . $p . '"';
            print HOOK $p, "\n"
         }
      }
      close HOOK;

      seek $MFFH, 0, 2
   }

where $HOOK is an executable shell script.
The $HOOK then does (later on)

  xargs tar -v -r -f "${HOME}/traffic/${backup}.tar" || {
     rm -f "${HOME}/traffic/${backup}.tar"
     echo >&2 'Failed 11'
     exit 11
  }

and that does exit 11.  The "listref" above is an array of
absolute path names, and i did change the "print HOOK" to echo
some words first, and whereas the GNU xargs printed that three
times, the busybox one only once.

(And sorry for the late response, i have a new box and then
changed the old Apple box to native Linux, which turns out to be
very difficult for an inexperienced, and would plain fail without
the ArchLinux wiki -- thanks for that, whoever you are.)
Comment 3 Denys Vlasenko 2018-11-01 15:34:13 UTC
> The $HOOK then does (later on)
>
>  xargs tar -v -r -f "${HOME}/traffic/${backup}.tar" || {
>     rm -f "${HOME}/traffic/${backup}.tar"
>     echo >&2 'Failed 11'
>     exit 11
>  }
>
> and that does exit 11.

Insert ...| tee /tmp/FILE |... in a suitable location and thus save that input.
Comment 4 Steffen Nurpmeso 2018-11-01 20:40:42 UTC
No, sorry, no, this reveals the content of my entire file system?  You cannot reproduce that, really?  Well, i am still having no umlauts, and suspend does not really turn off cpu, and a fancontrol script is due, but thereafter i could find some time and look into that.  Note this is a new box with likely shorter paths, on bare metal not in VM, i have no idea.  (And i got no mail btw., i just looked because this on the plate)
Comment 5 Denys Vlasenko 2018-11-02 19:46:59 UTC
(In reply to Steffen Nurpmeso from comment #4)
> You cannot reproduce that, really?

Yes. I can pipe my entire output of "find /" to busybox xargs and it works. No error messages.
Comment 6 Steffen Nurpmeso 2018-11-02 22:34:09 UTC
Created attachment 7861 [details]
boiled down test script(s)
Comment 7 Steffen Nurpmeso 2018-11-02 22:38:40 UTC
grrrr!  Terrible!!  Adding an attachment thrown away the text in this s....y textbox!  once again...

It is very strange it is that way, i can reproduce it absolutely reliably even on my new pretty virgin AlpineLinux (same version numbers)!
I generated boiled down versions of my backup script and the currently used shell hook, you could unpack it in /tmp/ and it will send to $EMAIL the report of the failure, i would guess.  It will try to backup /usr, which has 19418 files on this new box now.  (I guess you look into the script yourself first anyway.)
I would not know what is wrong with what i do, maybe a fd issue?
Comment 8 Steffen Nurpmeso 2018-11-02 22:39:59 UTC
(It needs an installed mail(1), or anyway "mail -s" must work, you surely look into that thing yourself!)
Comment 9 Denys Vlasenko 2018-11-03 17:43:45 UTC
I need a simple reproducer: a file which causes your xargs to go belly up.

Not a multi-page perl script.

Can you garble your secret filenames? A-la

find /usr | tr '[a-z]' w | tr '[A-Z]' W | tr '[0-9]' 0 >FILE
Comment 10 Steffen Nurpmeso 2018-11-03 19:22:09 UTC
"--complete --reset" should have been used as arguments, sorry.
Yes, i can!  I will create the attachment with the original names from /usr/ in a second, and i can reproduce with "< .hooklist xargs echo bla".
Comment 11 Steffen Nurpmeso 2018-11-03 19:29:00 UTC
i cannot create the attachment, it is too large.
ha-ha-ha.
just run the script with "--complete --reset", it will not do any harm.
i can tell you the list is "19533 19533 769751 114" says wc -lwcL.
Comment 12 Steffen Nurpmeso 2018-11-03 20:42:08 UTC
Actually it is really easy, just as you say:

  find /usr -type f | while read i; do printf "\"%s\"\n" "$i"; done | /bin/busybox xargs echo bla

should do the trick.
Comment 13 Steffen Nurpmeso 2018-11-03 20:45:37 UTC
(And POSIX says that arguments may be quoted.)