tahoe-lafs/trac-2024-07-25

modifying SDMF file fails when tahoe.cfg's k/N differs from file #1510

New issue

Closed

opened 2011-08-27 20:44:07 +00:00 by warner · 3 comments

warner commented

2011-08-27 20:44:07 +00:00

Owner

The publish process has several dictionaries that are keyed by share number. It looks like some of them are being filled with shnums that depend upon what the current tahoe.cfg has for shares.total, whereas others are filled according to which existing shares were actually found. If tahoe.cfg has changed in the meantime, finish_publishing() will be called on shnums that were never handled in the earlier methods.

I've attached a unit test which exercises this case in a pretty narrow way.

This is a blocker for 1.9

I found a regression in the SDMF publish code when I happened to try to update an old directory that I'd created with k=3/N=20. My current tahoe.cfg uses k=3/N=10. The exception is a NotEnoughServersError that wraps an AssertionError in `mutable.layout.SDMFSlotWriteProxy.finish_publishing`, where it notices that it doesn't have all the share pieces it needs (it's missing sharedata, blockhashes, and sharehashes, but it has verification_key, encprivkey, and signature). The publish process has several dictionaries that are keyed by share number. It looks like some of them are being filled with shnums that depend upon what the current tahoe.cfg has for `shares.total`, whereas others are filled according to which existing shares were actually found. If tahoe.cfg has changed in the meantime, `finish_publishing()` will be called on shnums that were never handled in the earlier methods. I've attached a unit test which exercises this case in a pretty narrow way. This is a blocker for 1.9

tahoe-lafs added the

labels 2011-08-27 20:44:07 +00:00

tahoe-lafs added this to the 1.9.0 milestone 2011-08-27 20:44:07 +00:00

warner commented

2011-08-27 20:44:38 +00:00

Author

Owner

Attachment 1510-test.diff (1355 bytes) added

unit test to exercise the failure

**Attachment** 1510-test.diff (1355 bytes) added unit test to exercise the failure

1510-test.diff

1.3 KiB

warner commented

2011-08-27 21:15:03 +00:00

Author

Owner

Attachment update-parms.diff (780 bytes) added

here's the fix: update k/N after retrieval, so we use the same values on subsequent publish

**Attachment** update-parms.diff (780 bytes) added here's the fix: update k/N after retrieval, so we use the same values on subsequent publish

update-parms.diff

780 B

Brian Warner <warner@lothar.com> commented

2011-08-27 22:51:05 +00:00

Author

Owner

In changeset:370e6f271e40945b:

SDMF: update filenode with correct k/N after Retrieve. Fixes #1510.

Without this, we get a regression when modifying a mutable file that was
created with more shares (larger N) than our current tahoe.cfg . The
modification attempt creates new versions of the (0,1,..,newN-1) shares, but
leaves the old versions of the (newN,..,oldN-1) shares alone (and throws a
assertion error in SDMFSlotWriteProxy.finish_publishing in the process).

The mixed versions that result (some shares with e.g. N=10, some with N=20,
such that both versions are recoverable) cause problems for the Publish code,
even before MDMF landed. Might be related to refs #1390 and refs #1042.

In changeset:370e6f271e40945b: ``` SDMF: update filenode with correct k/N after Retrieve. Fixes #1510. Without this, we get a regression when modifying a mutable file that was created with more shares (larger N) than our current tahoe.cfg . The modification attempt creates new versions of the (0,1,..,newN-1) shares, but leaves the old versions of the (newN,..,oldN-1) shares alone (and throws a assertion error in SDMFSlotWriteProxy.finish_publishing in the process). The mixed versions that result (some shares with e.g. N=10, some with N=20, such that both versions are recoverable) cause problems for the Publish code, even before MDMF landed. Might be related to refs #1390 and refs #1042. ```

tahoe-lafs added the

fixed

label 2011-08-27 22:51:05 +00:00

Brian Warner <warner@lothar.com> closed this issue

2011-08-27 22:51:05 +00:00