escalation of authority from knowing a storage index to being able to delete corresponding shares #1528

New issue

Closed

opened 2011-09-06 23:27:20 +00:00 by zooko · 12 comments

zooko commented

2011-09-06 23:27:20 +00:00

In Tahoe-LAFS, each file is encoded into a redundant set of "shares" (like in RAID-5 or RAID-6), and each share is stored on a different server. There is a secret string called the "cancellation secret" which is stored on the server by being appended to the end of the share data. The bug is that the server allows a client to read past the end of the share data and thus learn the cancellation secret. A client that knows the cancellation secret can use it to cause that server to delete the shares it stores of that file.

We have prepared a set of patches that do three things:

Fix the bounds violation in reading of immutable files that allowed the clients to learn the cancellation secrets.
Remove the function that takes a cancellation secret and deletes shares. This function (named "remote_cancel_lease") was not actually used, as all users currently rely on a different mechanism for deleting unused data (a garbage collection mechanism in which unused shares get deleted by the server once no client has renewed its lease on them in more than a month).
Fix some similar bounds violations in mutable files that could potentially lead to similar vulnerability. This vulnerability is probably not a concern in practice, because it doesn't arise unless the legitimate, authorized client deliberately writes a "hole" into the mutable file (by seeking past the end of the current data and not writing over all the bytes thus uncovered). No extant version of Tahoe-LAFS does this, so presumably no legitimate user would be exposed to that vulnerability.

[known_issues.rst for 1.8.3]source:1.8.3/docs/known_issues.rst#unauthorized-deletion-of-an-immutable-file-by-its-storage-index has more details, but I'll paste the most relevant bit here:

This vulnerability does not enable anyone to read file contents without authorization (confidentiality), nor to change the contents of a file (integrity).

A person could learn the storage index of a file in several ways:

By being granted the authority to read the immutable file—i.e. by being granted a read capability to the file. They can determine the file's storage index from its read capability.
By being granted a verify capability to the file. They can determine the file's storage index from its verify capability. This case probably doesn't happen often because users typically don't share verify caps.
By operating a storage server, and receiving a request from a client that has a read cap or a verify cap. If the client attempts to upload, download, or verify the file with their storage server, even if it doesn't actually have the file, then they can learn the storage index of the file.
By gaining read access to an existing storage server's local filesystem, and inspecting the directory structure that it stores its shares in. They can thus learn the storage indexes of all files that the server is holding at least one share of. Normally only the operator of an existing storage server would be able to inspect its local filesystem, so this requires either being such an operator of an existing storage server, or somehow gaining the ability to inspect the local filesystem of an existing storage server.

The Tahoe-LAFS core team has discovered a bug in Tahoe-LAFS v1.8.2 and all earlier versions starting with Tahoe-LAFS v1.3.0 that could allow users to unauthorizedly delete immutable files in some cases. In Tahoe-LAFS, each file is encoded into a redundant set of "shares" (like in RAID-5 or RAID-6), and each share is stored on a different server. There is a secret string called the "cancellation secret" which is stored on the server by being appended to the end of the share data. The bug is that the server allows a client to read past the end of the share data and thus learn the cancellation secret. A client that knows the cancellation secret can use it to cause that server to delete the shares it stores of that file. We have prepared a set of patches that do three things: 1. Fix the bounds violation in reading of immutable files that allowed the clients to learn the cancellation secrets. 2. Remove the function that takes a cancellation secret and deletes shares. This function (named "remote_cancel_lease") was not actually used, as all users currently rely on a different mechanism for deleting unused data (a garbage collection mechanism in which unused shares get deleted by the server once no client has renewed its lease on them in more than a month). 3. Fix some similar bounds violations in mutable files that could potentially lead to similar vulnerability. This vulnerability is probably not a concern in practice, because it doesn't arise unless the legitimate, authorized client deliberately writes a "hole" into the mutable file (by seeking past the end of the current data and not writing over all the bytes thus uncovered). No extant version of Tahoe-LAFS does this, so presumably no legitimate user would be exposed to that vulnerability. [known_issues.rst for 1.8.3]source:1.8.3/docs/known_issues.rst#unauthorized-deletion-of-an-immutable-file-by-its-storage-index has more details, but I'll paste the most relevant bit here: This vulnerability does not enable anyone to read file contents without authorization (confidentiality), nor to change the contents of a file (integrity). A person could learn the storage index of a file in several ways: 1. By being granted the authority to read the immutable file—i.e. by being granted a read capability to the file. They can determine the file's storage index from its read capability. 2. By being granted a verify capability to the file. They can determine the file's storage index from its verify capability. This case probably doesn't happen often because users typically don't share verify caps. 3. By operating a storage server, and receiving a request from a client that has a read cap or a verify cap. If the client attempts to upload, download, or verify the file with their storage server, even if it doesn't actually have the file, then they can learn the storage index of the file. 4. By gaining read access to an existing storage server's local filesystem, and inspecting the directory structure that it stores its shares in. They can thus learn the storage indexes of all files that the server is holding at least one share of. Normally only the operator of an existing storage server would be able to inspect its local filesystem, so this requires either being such an operator of an existing storage server, or somehow gaining the ability to inspect the local filesystem of an existing storage server.

zooko added the

labels 2011-09-06 23:27:20 +00:00

zooko added this to the undecided milestone 2011-09-06 23:27:20 +00:00

zooko commented

2011-09-13 16:25:36 +00:00

Author

In [5006/1.8.3]:

storage: remove the storage server's "remote_cancel_lease" function
We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file.
fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)

In [5006/1.8.3]: ``` storage: remove the storage server's "remote_cancel_lease" function We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file. fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them) ```

zooko added the

r/fixed

label 2011-09-13 16:25:36 +00:00

zooko closed this issue

2011-09-13 16:25:36 +00:00

zooko commented

2011-09-13 16:25:36 +00:00

Author

In [5007/1.8.3]:

immutable: prevent clients from reading past the end of share data, which would allow them to learn the cancellation secret
Declare explicitly that we prevent this problem in the server's version dict.
fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)

In [5007/1.8.3]: ``` immutable: prevent clients from reading past the end of share data, which would allow them to learn the cancellation secret Declare explicitly that we prevent this problem in the server's version dict. fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them) ```

daira commented

2011-09-13 16:50:11 +00:00

In [5013/1.8.3]:

interfaces: document that the 'fills-holes-with-zero-bytes' key should be used to detect whether a storage server has that behavior. refs #1528

In [5013/1.8.3]: ``` interfaces: document that the 'fills-holes-with-zero-bytes' key should be used to detect whether a storage server has that behavior. refs #1528 ```

daira added

and removed

labels 2011-09-13 18:34:31 +00:00

daira modified the milestone from undecided to 1.8.3

2011-09-13 18:34:31 +00:00

daira reopened this issue

2011-09-13 18:34:31 +00:00

daira commented

2011-09-13 18:48:36 +00:00

Reassigning to me to apply the fix to trunk.

daira changed title from ~~placeholder ticket~~ to escalation of authority from knowing a storage index to being able to delete corresponding shares

2011-09-13 18:48:36 +00:00

zooko commented

2011-09-13 22:10:04 +00:00

Author

In changeset:5476f67dc1177a26:

storage: remove the storage server's "remote_cancel_lease" function
We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file.
fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)

In changeset:5476f67dc1177a26: ``` storage: remove the storage server's "remote_cancel_lease" function We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file. fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them) ```

zooko added the

r/fixed

label 2011-09-13 22:10:04 +00:00

zooko closed this issue

2011-09-13 22:10:04 +00:00

zooko commented

2011-09-13 22:10:04 +00:00

Author

In changeset:20e2910c616531c9:

immutable: prevent clients from reading past the end of share data, which would allow them to learn the cancellation secret
Declare explicitly that we prevent this problem in the server's version dict.
fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)

In changeset:20e2910c616531c9: ``` immutable: prevent clients from reading past the end of share data, which would allow them to learn the cancellation secret Declare explicitly that we prevent this problem in the server's version dict. fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them) ```

zooko commented

2011-09-13 22:10:04 +00:00

Author

In changeset:401d0e7f69ddeaef:

merge the NEWS about the security fix (#1528) with the work-in-progress NEWS

In changeset:401d0e7f69ddeaef: ``` merge the NEWS about the security fix (#1528) with the work-in-progress NEWS ```

daira commented

2011-09-13 22:10:05 +00:00

In changeset:c10099f982ee0803:

interfaces: document that the 'fills-holes-with-zero-bytes' key should be used to detect whether a storage server has that behavior. refs #1528

In changeset:c10099f982ee0803: ``` interfaces: document that the 'fills-holes-with-zero-bytes' key should be used to detect whether a storage server has that behavior. refs #1528 ```

daira commented

2011-09-13 22:23:32 +00:00

Note that changesets [5004/1.8.3], [5005/1.8.3], and [5008/1.8.3]--[5012/1.8.3] inclusive on the 1.8.3 branch, and changesets changeset:cffc98780414760c, changeset:65de17245da26a4c, changeset:942c5e5162fc3d9c, changeset:32f80625c912be45, changeset:48f56dab6fb9cc20, changeset:7a98abeb3a7b1efb, changeset:eb26075da077404e, changeset:b15bd674c3f1a73e, and changeset:4c33d855d1fbfb51 on trunk, are also related to this ticket. (They didn't get posted here automatically because the 'refs' syntax in the patch descriptions was wrong.)

warner commented

2011-09-14 16:52:13 +00:00

For future code-archaeologists, this bug was introduced in changeset:6c4019ec33e7a253, which
removed a precondition check in read_share_data() (because it used
the original 4-byte size field, which was deprecated in favor of
measuring the length of the container file with os.stat), but didn't
provide a replacement. This was 536 patches after the 1.2.0 release, and
about 295 patches before the 1.3.0 release.

For future code-archaeologists, this bug was introduced in changeset:6c4019ec33e7a253, which removed a precondition check in `read_share_data()` (because it used the original 4-byte size field, which was deprecated in favor of measuring the length of the container file with os.stat), but didn't provide a replacement. This was 536 patches after the 1.2.0 release, and about 295 patches before the 1.3.0 release.

zooko commented

2011-09-19 05:15:27 +00:00

Author

Oh, I see that it is actually something more complicated:

http://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/src/allmydata/history.py?annotate=blame&rev=4046

Brian pointed out to me that there is another way that someone can learn the storage index of a file. It is shown on the "Recent Uploads and Downloads" page of a gateway. If someone can access your gateway, and you've uploaded or downloaded the file recently (if I recall correctly it is a FIFO queue of the most recent 20 uploads or downloads)... Oh, I see that it is actually something more complicated: <http://tahoe-lafs.org/trac/tahoe-lafs/browser/trunk/src/allmydata/history.py?annotate=blame&rev=4046>

zooko commented

2012-01-13 17:50:17 +00:00

Author

According to this post by Josh Bressers from RedHat on the oss-sec mailing list, we should use the identifier CVE-2011-3617 for this bug.

According to [this post](http://openwall.com/lists/oss-security/2011/10/10/10) by Josh Bressers from RedHat on the oss-sec mailing list, we should use the identifier `CVE-2011-3617` for this bug.