tahoe-lafs/trac-2024-07-25

implement CHK download helper #268

New issue

Open

opened 2008-01-09 23:25:31 +00:00 by warner · 4 comments

warner commented

2008-01-09 23:25:31 +00:00

Owner

To prevent the clients from making connections to all storage servers, we
need a helper that will also mediate CHK downloads and mutable uploads and
downloads. I'm making a separate ticket for this since it's more work than
I'd originally realized. I think I can still get it done in the original
schedule.

Zooko and I have talked about the need or lack of need for this
connection-minimization. It may be the case that minimizing connections is
not all that important, at least with a reasonable number of clients. So it's
possible that we'll want the #116 upload-only helper to get us resumability
and 1x uploads, but not the all-purpose helper described in this ticket to
get us minimal connections and isolation between clients and storage servers.

The upload-specific helper described in #116 is insufficient to achieve all the properties we're hoping for. The helper as originally envisioned in that ticket was specifically for uploading CHK files: it did not encompass downloading files (since there's no bandwidth benefit for those), nor did it include mutable files (since by our SDMF rules, these files are small and the bandwidth of them isn't too much of a burden). To prevent the clients from making connections to all storage servers, we need a helper that will also mediate CHK downloads and mutable uploads and downloads. I'm making a separate ticket for this since it's more work than I'd originally realized. I think I can still get it done in the original schedule. Zooko and I have talked about the need or lack of need for this connection-minimization. It may be the case that minimizing connections is not all that important, at least with a reasonable number of clients. So it's possible that we'll want the #116 upload-only helper to get us resumability and 1x uploads, but not the all-purpose helper described in this ticket to get us minimal connections and isolation between clients and storage servers.

tahoe-lafs added the

labels 2008-01-09 23:25:31 +00:00

tahoe-lafs added this to the 0.8.0 (Allmydata 3.0 Beta) milestone 2008-01-09 23:25:31 +00:00

warner commented

2008-01-24 01:48:13 +00:00

Author

Owner

I'm narrowing the focus of this ticket to just the CHK download helper. The
mutable-file helper has been moved off to #283, and the CHK-upload helper
remains in #116.

The benefit of implementing just the CHK-download helper is that downloads
will use a single TCP connection, improving bandwidth fairness. Specifically
this means that if both Tahoe and some other program are both downloading a
file over the same DSL line, they will get roughly the same bandwidth.
Without this feature, at our default 3-of-10 encoding, the Tahoe download
will get about three times as much bandwidth as the other program.

If all four helpers are implemented (#116, #268, and #283), then we can
change the introducer configuration to isolate the clients from the storage
servers.

I'm narrowing the focus of this ticket to just the CHK download helper. The mutable-file helper has been moved off to #283, and the CHK-upload helper remains in #116. The benefit of implementing just the CHK-download helper is that downloads will use a single TCP connection, improving bandwidth fairness. Specifically this means that if both Tahoe and some other program are both downloading a file over the same DSL line, they will get roughly the same bandwidth. Without this feature, at our default 3-of-10 encoding, the Tahoe download will get about three times as much bandwidth as the other program. If all four helpers are implemented (#116, #268, and #283), then we can change the introducer configuration to isolate the clients from the storage servers.

tahoe-lafs changed title from ~~implement rest of Helper~~ to implement CHK download helper

2008-01-24 01:48:13 +00:00

warner commented

2008-02-04 23:13:02 +00:00

Author

Owner

I'm pushing this out to 0.10.0 (i.e. post allmydata.com-final).

warner commented

2008-05-09 00:09:17 +00:00

Author

Owner

this one doesn't belong in 1.1.0