tahoe-lafs/trac-2024-07-25

tahoe make-verify-cap #713

New issue

Open

opened 2009-05-25 02:29:14 +00:00 by zooko · 10 comments

zooko commented

2009-05-25 02:29:14 +00:00

Owner

Make a tahoe cli command to compute an immutable verify cap for a given file and erasure coding params as cheaply as possible.

http://allmydata.org/pipermail/tahoe-dev/2009-May/001824.html

Make a tahoe cli command to compute an immutable verify cap for a given file and erasure coding params as cheaply as possible. <http://allmydata.org/pipermail/tahoe-dev/2009-May/001824.html>

tahoe-lafs added the

labels 2009-05-25 02:29:14 +00:00

tahoe-lafs added this to the undecided milestone 2009-05-25 02:29:14 +00:00

warner commented

2009-05-25 02:52:25 +00:00

Author

Owner

Does the existing "tahoe debug dump-cap FILECAP" do what you want? It ought to emit a verifycap after it emits a bunch of other information about the cap.

zooko commented

2009-05-25 03:04:40 +00:00

Author

Owner

I want a command that takes a filename in the local filesystem (or else reads file contents from stdin) rather than one that takes a capability to a file in the Tahoe filesystem. It is for uses such as Shawn Willden's experimentation in that tahoe-dev discussion -- benchmarking, for example.

zooko commented

2009-05-25 03:06:10 +00:00

Author

Owner

P.S. Actually it might be useful for it to produce the readcap, at least optionally, instead of the verifycap.

swillden commented

2009-05-25 13:36:11 +00:00

Author

Owner

Replying to zooko:

P.S. Actually it might be useful for it to produce the readcap, at least optionally,
instead of the verifycap.

Yes, if I were going to make us of it for my backup log, I would need the read cap.

Replying to [zooko](/tahoe-lafs/trac-2024-07-25/issues/713#issuecomment-113013): > P.S. Actually it might be useful for it to produce the readcap, at least optionally, > instead of the verifycap. Yes, if I were going to make us of it for my backup log, I would need the read cap.

warner commented

2009-05-31 23:00:05 +00:00

Author

Owner

Ah, ok, so it needs to be a lot like "tahoe put", but without the network part.

Would tahoe put --dry-run FILE be reasonable, if it emitted the filecap just like tahoe put FILE ? And then you could use dump-cap to derive the verifycap from it?

Ah, ok, so it needs to be a lot like "tahoe put", but without the network part. Would `tahoe put --dry-run FILE` be reasonable, if it emitted the filecap just like `tahoe put FILE` ? And then you could use dump-cap to derive the verifycap from it?

swillden commented

2009-06-01 01:12:46 +00:00

Author

Owner

Replying to warner:

Ah, ok, so it needs to be a lot like "tahoe put", but without the network part.

Would tahoe put --dry-run FILE be reasonable, if it emitted the filecap just like
tahoe put FILE?

That would work, though I'm using the web API rather than the command line interface.

I am still somewhat skeptical about whether or not performance would be acceptable, particularly on low-end backup appliances (think Linksys with USB-attached storage). SHA-256d hashing is CPU-bound on the one I've played with, and I get the idea that the full "PUT" operation would be around an order of magnitude slower.

The need to have read cap index files has driven me to find other uses for them as well, uses that I'd have to find some other way to implement if I didn't have them, so between the performance issue and that one, I'm not sure I'd use a "dry run" option, even if it existed.

Replying to [warner](/tahoe-lafs/trac-2024-07-25/issues/713#issuecomment-113015): > Ah, ok, so it needs to be a lot like "tahoe put", but without the network part. > > Would `tahoe put --dry-run FILE` be reasonable, if it emitted the filecap just like > `tahoe put FILE`? That would work, though I'm using the web API rather than the command line interface. I am still somewhat skeptical about whether or not performance would be acceptable, particularly on low-end backup appliances (think Linksys with USB-attached storage). SHA-256d hashing is CPU-bound on the one I've played with, and I get the idea that the full "PUT" operation would be around an order of magnitude slower. The need to have read cap index files has driven me to find other uses for them as well, uses that I'd have to find some other way to implement if I didn't have them, so between the performance issue and that one, I'm not sure I'd use a "dry run" option, even if it existed.

swillden commented

2009-07-05 20:25:58 +00:00

Author

Owner

Replying to [swillden]comment:6:

The need to have read cap index files has driven me to find other uses for them as well, uses that I'd have to find some other way to implement if I didn't have them, so between the performance issue and that one, I'm not sure I'd use a "dry run" option, even if it existed.

As zooko pointed out in response to a recent mailing list question, this would be a good solution for a need I have to verify that a grid-based immutable file matches a local file.

Replying to [swillden]comment:6: > The need to have read cap index files has driven me to find other uses for them as well, uses that I'd have to find some other way to implement if I didn't have them, so between the performance issue and that one, I'm not sure I'd use a "dry run" option, even if it existed. As zooko pointed out in response to a recent mailing list question, this would be a good solution for a need I have to verify that a grid-based immutable file matches a local file.

zooko commented

2009-07-05 21:41:32 +00:00

Author

Owner

Namely this mailing list thread: http://allmydata.org/pipermail/tahoe-dev/2009-July/002190.html

Namely this mailing list thread: <http://allmydata.org/pipermail/tahoe-dev/2009-July/002190.html>

davidsarah commented

2009-12-13 02:34:17 +00:00

Author

Owner

Replying to [swillden]comment:7:

... this would be a good solution for a need I have to verify that a grid-based immutable file matches a local file.

See also #658 for that.

Replying to [swillden]comment:7: > ... this would be a good solution for a need I have to verify that a grid-based immutable file matches a local file. See also #658 for that.

swillden commented

2009-12-13 03:50:38 +00:00

Author

Owner

Replying to [davidsarah]comment:9:

Replying to [swillden]comment:7:

... this would be a good solution for a need I have to verify that a grid-based immutable file matches a local file.

See also #658 for that.
#658 wouldn't help. I can't assume the backupdb would be there.

Replying to [davidsarah]comment:9: > Replying to [swillden]comment:7: > > ... this would be a good solution for a need I have to verify that a grid-based immutable file matches a local file. > > See also #658 for that. #658 wouldn't help. I can't assume the backupdb would be there.