Bug 3217 - wget dead slow
Summary: wget dead slow
Status: RESOLVED FIXED
Alias: None
Product: Busybox
Classification: Unclassified
Component: Networking (show other bugs)
Version: 1.18.x
Hardware: PC Linux
: P5 major
Target Milestone: ---
Assignee: unassigned
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-02-09 13:53 UTC by uli
Modified: 2011-03-07 11:37 UTC (History)
1 user (show)

See Also:
Host:
Target:
Build:


Attachments
log file for busybox 1.17.2 (38.93 KB, text/plain)
2011-02-10 09:24 UTC, uli
Details
log file for busybox 1.18.3 (37.98 KB, application/octet-stream)
2011-02-10 09:25 UTC, uli
Details

Note You need to log in before you can comment on or make changes to this bug.
Description uli 2011-02-09 13:53:38 UTC
Tried out busybox-based wget in v. 1.18.1 (in Tinycore Linux)
Takes 6.01 seconds for a 30KB file on a local server.

rolling back from 1.18.1 to 1.17.2 solves the problem and shows the expected 0.01 second. This has been tried out on a virtualbox, however a quick test showed similar behaviour on real hardware, too
Comment 1 Denys Vlasenko 2011-02-10 04:34:17 UTC
Can you produce strace logs of wget downloads using both 1.18.1 and 1.17.2 with these options:

strace -tt -T -oLOG busybox wget http://127.0.0.1/10M_of_zero_bytes -O/dev/null
Comment 2 Denys Vlasenko 2011-02-10 05:36:23 UTC
I did some tests and it looks like tiny 512-byte buffer is a culprit.
Newer version does additional poll() and time() calls around every read/write, that made overhead more visible in it.

Try this simplistic fix:

http://busybox.net/downloads/fixes-1.18.3/busybox-1.18.3-wget.patch

I did a better fix in git.
Comment 3 uli 2011-02-10 09:24:57 UTC
Created attachment 2941 [details]
log file for busybox 1.17.2
Comment 4 uli 2011-02-10 09:25:34 UTC
Created attachment 2947 [details]
log file for busybox 1.18.3
Comment 5 uli 2011-02-10 09:28:23 UTC
problem persists after patch, log files were taken with a server to which I've got a minimum of 100Mb bandwidth, again virtualbox, now running ubuntu, for reason of development environment.
Comment 6 uli 2011-02-10 12:07:29 UTC
did further tests, behaviour depends on server chosen and file that's fetched. With just fetching some index.html of some server somewhere in the world it always gets stuck (see logfiles above, only for 1.18.x, not 1.17.x), with large zero files so far no issue were seen, with just some old 10MB file I happened to have on disk there were hangups seen occasionally with 1.18.1, not so with the patched 1.18.3 yet.

The slowdown seems always related to some poll / timeout message in the logfile.

Conclusion:
The problem that actually triggered my report might have vanished in the patched 1.18.3 version. However, the fact that for any web server I tried to access by its web address only (ie. expecting to fetch some index file) the timeout occurs, and that's for 1.18.x, not so for 1.17.2, might perhaps indicate some remaining issue? 

... can't unfortunately debug any further, sorry...
Comment 7 Denys Vlasenko 2011-02-10 12:25:57 UTC
Ok, I see where we have a different problem. We don't send "Connection: close".

I updated busybox-1.18.3-wget.patch, please try it again
Comment 8 uli 2011-02-10 13:36:01 UTC
(In reply to comment #7)
> Ok, I see where we have a different problem. We don't send "Connection: close".
> I updated busybox-1.18.3-wget.patch, please try it again

issue seems fixed now, no more problems seen. thanks...
Comment 9 Denys Vlasenko 2011-03-07 11:37:52 UTC
Fixed, fix will be in 1.18.4 and 1.19.x