"tahoe cp -r --mutable/--immutable": make mutable copy of immutable directories or vice versa #835

New issue

Open

opened 2009-11-20 04:02:57 +00:00 by warner · 9 comments

warner commented

2009-11-20 04:02:57 +00:00

There are a couple of interesting forms of copying that could be done. (In each case, we're talking about directories, and not files.)

original is immutable: make immutable copy (re-use same object)
original is immutable: make mutable copy
original is mutable: make mutable copy
original is mutable: make immutable copy

The default for cp -r should be to use the same type of object: mutable-to-mutable or immutable-to-immutable (and of course, immutable-to-immutable means we just re-use the original dircap).

I think that tahoe cp should acquire a --mutable flag which tells it to always create mutable directories, even if the original was immutable. This would be used to convert your "tahoe backup" -created immutable directories into a form that you can modify.

Likewise, I think it should have a --immutable flag which tells it to always create immutable directories.

I think that files should be handled differently: basically the default should be mutable-to-mutable and immutable-gets-shared. If you copy with --immutable, then clearly that will trigger mutable-to-immutable (since immutable dirnodes are deep-immutable, so we can't fill them with mutable files). But if you copy with --mutable, I think we should create mutable dirnodes with immutable files. A separate flag (maybe --mutable-files) could be used if you really do want to turn all of your immutable files into mutable ones.

Now that we have immutable directories (#607), we could use some CLI commands to take advantage of them. #828 is about having "tahoe backup" create immutable directories, but what if you want to convert those immutable directories into a form that you can modify again? `tahoe cp` seems like the most appropriate tool. There are a couple of interesting forms of copying that could be done. (In each case, we're talking about directories, and not files.) * original is immutable: make immutable copy (re-use same object) * original is immutable: make mutable copy * original is mutable: make mutable copy * original is mutable: make immutable copy The default for `cp -r` should be to use the same type of object: mutable-to-mutable or immutable-to-immutable (and of course, immutable-to-immutable means we just re-use the original dircap). I think that `tahoe cp` should acquire a `--mutable` flag which tells it to always create mutable directories, even if the original was immutable. This would be used to convert your "tahoe backup" -created immutable directories into a form that you can modify. Likewise, I think it should have a `--immutable` flag which tells it to always create immutable directories. I think that files should be handled differently: basically the default should be mutable-to-mutable and immutable-gets-shared. If you copy with `--immutable`, then clearly that will trigger mutable-to-immutable (since immutable dirnodes are deep-immutable, so we can't fill them with mutable files). But if you copy with `--mutable`, I think we should create mutable dirnodes with immutable files. A separate flag (maybe `--mutable-files`) could be used if you really do want to turn all of your immutable files into mutable ones.

warner added the

labels 2009-11-20 04:02:57 +00:00

warner added this to the undecided milestone 2009-11-20 04:02:57 +00:00

daira commented

2009-11-20 19:00:45 +00:00

But if you copy with --mutable, I think we should create mutable dirnodes with immutable files. A separate flag (maybe --mutable-files) could be used if you really do want to turn all of your immutable files into mutable ones.

That makes sense to me. But maybe the first option should be --mutable-dirs, since it isn't quite the inverse of --immutable. If you use --mutable then it should print a help text explaining the difference between --mutable-dirs and --mutable-files.

*But if you copy with `--mutable`, I think we should create mutable dirnodes with immutable files. A separate flag (maybe `--mutable-files`) could be used if you really do want to turn all of your immutable files into mutable ones.* That makes sense to me. But maybe the first option should be `--mutable-dirs`, since it isn't quite the inverse of `--immutable`. If you use `--mutable` then it should print a help text explaining the difference between `--mutable-dirs` and `--mutable-files`.

daira commented

2010-01-07 00:44:39 +00:00

This should probably be implemented in terms of #203.

daira modified the milestone from undecided to 1.7.0

2010-02-12 05:12:36 +00:00

zooko commented

2010-05-08 19:32:01 +00:00

Hm, can we implement this in the next week or so for v1.7? I doubt it, but if someone thinks so then please by all means go ahead!

zooko modified the milestone from 1.7.0 to soon

2010-06-18 23:46:05 +00:00

zooko commented

2010-07-30 23:42:20 +00:00

François Deppierraz indicated an interest in implementing this:
http://tahoe-lafs.org/pipermail/tahoe-dev/2010-July/004833.html

François Deppierraz indicated an interest in implementing this: <http://tahoe-lafs.org/pipermail/tahoe-dev/2010-July/004833.html>

daira commented

2011-11-26 14:00:29 +00:00

Kevin Reid wrote on tahoe-dev:

I want to experiment with running a webapp off a Tahoe grid. It would handle its own "database" read/write, but I need to bootstrap same-origin by uploading its source files to the grid.

It seems to me that the right thing here is a command which will upload the entire file tree (preferably with exclusions) immutably (to minimize duplication when I upload with changes) and return a dircap for it. What 'tahoe' arguments will do this?

'tahoe cp' requires a target directory and does not appear to produce immutable directories. 'tahoe put' does not appear to do directories. 'tahoe backup' requires a target directory and keeps old versions.

I suppose 'tahoe cp' is the closest in function I have found, but all else being equal I would like to have my 'install' function not actually use any of the user's pre-existing Tahoe capabilities. I could 'tahoe mkdir' a fresh directory and cp into it.

Any suggestions on how to do this more cleanly?

As well as the functionality of the proposed tahoe cp --immutable, Kevin's post also mentions exclusions, similar to tahoe backup's --exclude* options I assume. I've filed as #1597 to make tahoe cp support those options.

Kevin Reid [wrote on tahoe-dev](https://tahoe-lafs.org/pipermail/tahoe-dev/2011-November/006860.html): > I want to experiment with running a webapp off a Tahoe grid. It would handle its own "database" read/write, but I need to bootstrap same-origin by uploading its source files to the grid. > > It seems to me that the right thing here is a command which will upload the entire file tree (preferably with exclusions) immutably (to minimize duplication when I upload with changes) and return a dircap for it. What '`tahoe`' arguments will do this? > > '`tahoe cp`' requires a target directory and does not appear to produce immutable directories. '`tahoe put`' does not appear to do directories. '`tahoe backup`' requires a target directory and keeps old versions. > > I suppose '`tahoe cp`' is the closest in function I have found, but all else being equal I would like to have my 'install' function not actually use any of the user's pre-existing Tahoe capabilities. I could '`tahoe mkdir`' a fresh directory and cp into it. > > Any suggestions on how to do this more cleanly? As well as the functionality of the proposed `tahoe cp --immutable`, Kevin's post also mentions exclusions, similar to `tahoe backup`'s `--exclude*` options I assume. I've filed as #1597 to make `tahoe cp` support those options.

daira changed title from ~~"tahoe cp -r --mutable": make mutable copy of immutable directories, vice versa~~ to "tahoe cp -r --mutable/--immutable": make mutable copy of immutable directories or vice versa

2011-11-26 14:01:24 +00:00

daira commented

2011-11-26 14:05:42 +00:00

Kevin wrote:

'tahoe cp' requires a target directory and does not appear to produce immutable directories.

Maybe tahoe cp -r --immutable srcdir, with no destination argument, should produce an unlinked tree and output the root cap of that tree.

Kevin wrote: > '`tahoe cp`' requires a target directory and does not appear to produce immutable directories. Maybe `tahoe cp -r --immutable srcdir`, with no destination argument, should produce an unlinked tree and output the root cap of that tree.

daira commented

2012-11-26 01:02:34 +00:00

Note that there is no way to implement copying a Tahoe directory structure containing cycles (necessarily with a mutable link somewhere) using --immutable. That case should cause an error, and needs a test.

Note that there is no way to implement copying a Tahoe directory structure containing cycles (necessarily with a mutable link somewhere) using `--immutable`. That case should cause an error, and needs a test.

daira commented

2013-12-14 18:18:36 +00:00

amontero asked for this feature (cp -r --immutable) on #tahoe-lafs:

how do I put an entire tree into an immutable dir in one single command and get its URI?

tahoe-backup would be perfect just if it dumped the URI after the "creating directory for 'Archives/2013-12-14_14:24:38Z/'"

amontero asked for this feature (`cp -r --immutable`) on #tahoe-lafs: > how do I put an entire tree into an immutable dir in one single command and get its URI? > tahoe-backup would be perfect just if it dumped the URI after the "creating directory for 'Archives/2013-12-14_14:24:38Z/'"

amontero commented

2013-12-14 20:04:49 +00:00

Owner

Replying to daira:

amontero asked for this feature (cp -r --immutable) on #tahoe-lafs:

how do I put an entire tree into an immutable dir in one single command and get its URI?

tahoe-backup would be perfect just if it dumped the URI after the "creating directory for 'Archives/2013-12-14_14:24:38Z/'"

I've opened ticket #2135 with code proposal.

Replying to [daira](/tahoe-lafs/trac/issues/835#issuecomment-374465): > amontero asked for this feature (`cp -r --immutable`) on #tahoe-lafs: > > > how do I put an entire tree into an immutable dir in one single command and get its URI? > > > tahoe-backup would be perfect just if it dumped the URI after the "creating directory for 'Archives/2013-12-14_14:24:38Z/'" I've opened ticket #2135 with code proposal.