tahoe-lafs/trac-2024-07-25

implement URI:LIT encoding for very small files #81

New issue

Closed

opened 2007-07-08 15:48:34 +00:00 by warner · 5 comments

warner commented

2007-07-08 15:48:34 +00:00

Owner

Uploading a zero-length file causes a divide-by-zero error, because segment_size ends up being zero.

It would probably be sufficient to simply make segment_size always be at least one.

Uploading a zero-length file causes a divide-by-zero error, because segment_size ends up being zero. It would probably be sufficient to simply make segment_size always be at least one.

tahoe-lafs added the

labels 2007-07-08 15:48:34 +00:00

tahoe-lafs added this to the 0.5.0 milestone 2007-07-08 15:48:34 +00:00

warner commented

2007-07-11 22:31:43 +00:00

Author

Owner

upgrading priority because I really want this one fixed before the 0.5.0 release. Once the new webapi is in place and it is easy to upload a lot of files at once, this bug becomes a lot more annoying.

tahoe-lafs added

critical

and removed

major

labels 2007-07-11 22:31:43 +00:00

warner commented

2007-07-12 17:58:53 +00:00

Author

Owner

we're going to fix this one by implementing URI:LIT uris, which are used for very small files (less than about 40 bytes), in which the entire file contents are contained verbatim inside the URI itself.

tahoe-lafs changed title from ~~uploading an empty file causes an exception~~ to implement URI:LIT encoding for very small files

2007-07-12 17:58:53 +00:00

warner commented

2007-07-12 19:31:46 +00:00

Author

Owner

Our target for CHK URIs is URI:CHK:(base32(16byte AES key)):(base32(32byte URI Extension Hash)):needed_shares:total_shares:size

so that's 3+1+3+1+26+1+52+1+2+1+3+1+2 = 97 bytes (at least for <100B files).

Our plan for LIT uris is: URI:LIT:(base32(data))

So the breakeven point is 55 bytes. A 55 byte file will fit into a 3+1+3+1+88 = 96 byte LIT uri.

So if the file is 55 bytes or smaller, store it in a LIT, otherwise store it in the grid.

Our target for CHK URIs is URI:CHK:(base32(16byte AES key)):(base32(32byte URI Extension Hash)):needed_shares:total_shares:size so that's 3+1+3+1+26+1+52+1+2+1+3+1+2 = 97 bytes (at least for <100B files). Our plan for LIT uris is: URI:LIT:(base32(data)) So the breakeven point is 55 bytes. A 55 byte file will fit into a 3+1+3+1+88 = 96 byte LIT uri. So if the file is 55 bytes or smaller, store it in a LIT, otherwise store it in the grid.

warner commented

2007-07-12 19:32:49 +00:00

Author

Owner

Oh, and of course this also includes 0-byte files, which neatly fixes the problem of the encoding algorithm exploding when faced with 0-byte files.

warner commented

2007-07-12 20:25:39 +00:00

Author

Owner

done, plus tests.

tahoe-lafs added the

fixed