tahoe-lafs/trac-2024-07-25

download: speak to as few servers as possible #448

New issue

Closed

opened 2008-06-03 05:15:28 +00:00 by warner · 6 comments

warner commented

2008-06-03 05:15:28 +00:00

Owner

Our old roadmap.txt pointed out, in "Download Peer Selection step 2", that the current simplistic download code asks all known peers for the shares that they hold, whereas we really only need to ask enough peers to get "k" shares (plus a few extra in case we lose some during download, see #287).

This will probably occur about the same time we do a large-scale refactoring of the download code.

tahoe-lafs added the

labels 2008-06-03 05:15:28 +00:00

tahoe-lafs added this to the undecided milestone 2008-06-03 05:15:28 +00:00

tahoe-lafs modified the milestone from undecided to 1.7.0

2010-02-27 06:42:15 +00:00

tahoe-lafs changed title from ~~download: only speak to minimal number of peers~~ to download: speak to as few servers as possible

2010-02-27 17:48:44 +00:00

zooko commented

2010-05-08 19:43:50 +00:00

Author

Owner

#605 (two-hour delay to connect to a grid from Win32, if there are many storage servers unreachable) is blocked on this ticket because I want to see if fixing this ticket (#448) improves (or otherwise changes) the behavior of the #605 issue.

zooko commented

2010-05-08 19:45:44 +00:00

Author

Owner

If you like this ticket, you might also like the "Brian's New Downloader" bundle of tickets: #800 (improve alacrity by downloading only the part of the Merkle Tree that you need), #798 (improve random-access download to retrieve/decrypt less data), #809 (Measure how segment size affects upload/download speed.), #287 (download: tolerate lost or missing servers), and #605 (two-hour delay to connect to a grid from Win32, if there are many storage servers unreachable).

zooko commented

2010-05-08 22:48:46 +00:00

Author

Owner

Brian's New Downloader is now planned for v1.8.0.

tahoe-lafs modified the milestone from 1.7.0 to 1.8.0

2010-05-08 22:48:46 +00:00

davidsarah commented

2010-08-10 04:17:32 +00:00

Author

Owner

To what extent does the New Downloader address this ticket?

warner commented

2010-08-14 18:24:05 +00:00

Author

Owner

#798 resolves this ticket. The ShareFinder logic will send out multiple queries at once, so we'll wind up with more shares (and talk to more servers) that we strictly need, but it's limited to a window of 10 outstanding requests, which felt reasonable. (using something like 3*k might be more adaptive). And it will ask for more if it loses servers during the download. But it won't talk to the whole grid, which was the concern about this ticket.

So I'm closing this one.

#798 resolves this ticket. The `ShareFinder` logic will send out multiple queries at once, so we'll wind up with more shares (and talk to more servers) that we strictly need, but it's limited to a window of 10 outstanding requests, which felt reasonable. (using something like `3*k` might be more adaptive). And it will ask for more if it loses servers during the download. But it won't talk to the whole grid, which was the concern about this ticket. So I'm closing this one.

tahoe-lafs added the

fixed

label 2010-08-14 18:24:05 +00:00

warner closed this issue

2010-08-14 18:24:05 +00:00

zooko commented

2010-08-15 06:14:40 +00:00

Author

Owner

I created #1180 (put more DYHBs into flight at once when K is larger).