tahoe-lafs/trac-2024-07-25

"tahoe cp FILE ALIAS:" does unnecessary directory reads #2823

New issue

Open

opened 2016-09-03 22:11:48 +00:00 by warner · 0 comments

warner commented

2016-09-03 22:11:48 +00:00

Owner

I noticed that running tahoe cp FILE my: (where my: is an alias pointing at a directory with about 20 subdirectories) performs 20 directory reads before finally doing the expected upload and read+write assignment to the top-level directory. The CLI frontend is doing 20 separate WAPI GET /uri/CAP?t=json for each subdirectory.

It looks like targetting ALIAS: (rather than a fully-qualified ALIAS:FILE) is putting tahoe-cp in a mode where it finishes with a WAPI POST /uri/ALIAS?t=set_children. To do that, it wants to have the full contents of the target directory. However I think it should be able to get that information from the initial GET, rather than needing to fetch the contents of each subdirectory.

I don't remember why we're using set_children anyways. But I do remember an extensive investigation into how tahoe-cp ought to behave when the target is a directory (e.g. tahoe cp DIR ALIAS: or tahoe cp DIR ALIAS:SUBDIR). There might be a way to simplify this by transforming tahoe cp X ALIAS: into tahoe cp X ALIAS:X. Or at least an argument that the two ought to behave the same way.

I noticed that running `tahoe cp FILE my:` (where `my:` is an alias pointing at a directory with about 20 subdirectories) performs 20 directory reads before finally doing the expected upload and read+write assignment to the top-level directory. The CLI frontend is doing 20 separate WAPI `GET /uri/CAP?t=json` for each subdirectory. It looks like targetting `ALIAS:` (rather than a fully-qualified `ALIAS:FILE`) is putting tahoe-cp in a mode where it finishes with a WAPI `POST /uri/ALIAS?t=set_children`. To do that, it wants to have the full contents of the target directory. However I think it should be able to get that information from the initial GET, rather than needing to fetch the contents of each subdirectory. I don't remember why we're using `set_children` anyways. But I do remember an extensive investigation into how tahoe-cp ought to behave when the target is a directory (e.g. `tahoe cp DIR ALIAS:` or `tahoe cp DIR ALIAS:SUBDIR`). There might be a way to simplify this by transforming `tahoe cp X ALIAS:` into `tahoe cp X ALIAS:X`. Or at least an argument that the two ought to behave the same way.