extendable Introducer protocol: dictionary-based, signed announcements #466

Closed
opened 2008-06-17 01:39:56 +00:00 by warner · 29 comments

Zooko and I have discussed a new API for the Introducer that would be more
extensible for future uses. We need three features out of the new design:

  • announce dictionaries rather than tuples, so that we can add new keys to
    the announcements without affecting older subscribers who won't understand
    them
  • handle signed announcements: eventually we'll be using "blessed servers"
    to allow clients to restrict which storage servers they'll use (they can
    specify a public key, and only pay attention to announcements that were
    signed with the corresponding private key)
  • be compatible with v1.0/v1.1 clients, by using new remote_ methods and
    leaving the old methods in place.

This is one of the pre-requisites for implementing Accounting, since when we
require clients to use rights-amplification to obtain a reference to an
attenuated/labeled storage server object (instead of the current scheme in
which everybody gets access to a full-powered server reference), we'll be
announcing the "login" facet via a different interface name.

We're also planning on using the introducer to allow storage servers to
update their announcement when they get full (or when they free enough space
to resume accepting shares). This is a performance optimization: if a server
is going to reject all shares, it is faster to remove it from the list of
available writers than to ask and be rejected for each upload. The server
should remain available for reading, though. We're not yet sure how we'll do
this:

  • announce a "writeable" flag for each server
  • announce read-server and write-server separately

We should keep the idea of "updating your earlier announcement" in mind as we
implement this improvement.

Zooko and I have discussed a new API for the Introducer that would be more extensible for future uses. We need three features out of the new design: * announce dictionaries rather than tuples, so that we can add new keys to the announcements without affecting older subscribers who won't understand them * handle signed announcements: eventually we'll be using "blessed servers" to allow clients to restrict which storage servers they'll use (they can specify a public key, and only pay attention to announcements that were signed with the corresponding private key) * be compatible with v1.0/v1.1 clients, by using new remote_ methods and leaving the old methods in place. This is one of the pre-requisites for implementing Accounting, since when we require clients to use rights-amplification to obtain a reference to an attenuated/labeled storage server object (instead of the current scheme in which everybody gets access to a full-powered server reference), we'll be announcing the "login" facet via a different interface name. We're also planning on using the introducer to allow storage servers to update their announcement when they get full (or when they free enough space to resume accepting shares). This is a performance optimization: if a server is going to reject all shares, it is faster to remove it from the list of available writers than to ask and be rejected for each upload. The server should remain available for reading, though. We're not yet sure how we'll do this: * announce a "writeable" flag for each server * announce read-server and write-server separately We should keep the idea of "updating your earlier announcement" in mind as we implement this improvement.
warner added the
c/code-network
p/major
t/enhancement
v/1.1.0
labels 2008-06-17 01:39:56 +00:00
warner added this to the undecided milestone 2008-06-17 01:39:56 +00:00
Author

More design notes:

  • each announcement is a certificate chain (to allow key-rotation, and
    probably some forms of extendability), so a variable-length list of
    certificates.
  • I'm thinking that we could use the same cert syntax for both introducer
    announcements and storage authority, so I'm kind of front-loading the
    design here.
  • Each certificate is a triple of (encoded-message, signature,
    pubkey-identifier).
  • The encoded-message is the actual content, probably a JSON-encoded
    dictionary (but of course then we must base64-encode any binary strings,
    since JSON can only carry unicode strings). Note that the encoded-message
    is a single string, and is generally left in encoded form until someone
    needs to know the contents. The only time encoding takes place is just
    before a signature is created. The encoded form is the canonical form.
  • The signature is an EC-DSA-192 signature of the encoded-message, using
    some public key. This is a single string. If the cert is unsigned, this
    will be a 0-length string.
  • The pubkey-identifier is optional: if omitted, it is a 0-length string.
    This helps a verifier figure out which pubkey was used to compute the
    signature. If omitted, the verifier must attempt to verify the signature
    against all root certs. If present, it is a strict prefix of the full
    serialized pubkey. Since we don't expect to have more than a handful of
    root certs, let's say this should be 4 bytes long.:
   for rootcert in roots:
       pubkey = rootcert.pubkey
       pubkey_s = pubkey.serialize()
       if pubkey_s.startswith(cert[2]):
           if pubkey.verify(cert[0], cert[1]):
               return (cert[0], rootcert) # valid, this is the parent cert
   raise InvalidCertificate("no matching root certificate found")
  • The announcement protocol will use a signature like:
EncodedMessage = StringConstraint(regexp=PRINTABLE)
Certificate = TupleOf(EncodedMessage, str, str)
CertificateChain = ListOf(Certificate)
class RIIntroducerPublisher(RemoteInterface):
    def publish(announcement=CertificateChain):
        return None
  • We'll need to define more serialization details for storage authority, to
    get the whole certificate chain down to a single string. The goals for
    that use case are:
  • tight encoding: no double-base32 encoding of anything, short strings,
    give up generality in deference to brevity
  • the final string should be base32- or base62- safe (so it can fit
    comfortably in a URL's query argument)
  • we need the encoded message to be serialized in some well-defined form
    (so that we know how to create its signature), but we also want
    flexibility for what goes into the encoded message, hence the JSON. For
    storage authority cert chains, we might not actually want that
    flexibility: if the client doesn't recognize one of the attenuations,
    it should fail-safe by rejecting the cert. There are a couple of
    competing goals here, and I'm not sure what the best answer is.
More design notes: * each announcement is a certificate chain (to allow key-rotation, and probably some forms of extendability), so a variable-length list of certificates. * I'm thinking that we could use the same cert syntax for both introducer announcements and storage authority, so I'm kind of front-loading the design here. * Each certificate is a triple of (encoded-message, signature, pubkey-identifier). * The encoded-message is the actual content, probably a JSON-encoded dictionary (but of course then we must base64-encode any binary strings, since JSON can only carry unicode strings). Note that the encoded-message is a single string, and is generally left in encoded form until someone needs to know the contents. The only time encoding takes place is just before a signature is created. The encoded form *is* the canonical form. * The signature is an EC-DSA-192 signature of the encoded-message, using some public key. This is a single string. If the cert is unsigned, this will be a 0-length string. * The pubkey-identifier is optional: if omitted, it is a 0-length string. This helps a verifier figure out which pubkey was used to compute the signature. If omitted, the verifier must attempt to verify the signature against all root certs. If present, it is a strict prefix of the full serialized pubkey. Since we don't expect to have more than a handful of root certs, let's say this should be 4 bytes long.: ``` for rootcert in roots: pubkey = rootcert.pubkey pubkey_s = pubkey.serialize() if pubkey_s.startswith(cert[2]): if pubkey.verify(cert[0], cert[1]): return (cert[0], rootcert) # valid, this is the parent cert raise InvalidCertificate("no matching root certificate found") ``` * The announcement protocol will use a signature like: ``` EncodedMessage = StringConstraint(regexp=PRINTABLE) Certificate = TupleOf(EncodedMessage, str, str) CertificateChain = ListOf(Certificate) class RIIntroducerPublisher(RemoteInterface): def publish(announcement=CertificateChain): return None ``` * We'll need to define more serialization details for storage authority, to get the whole certificate chain down to a single string. The goals for that use case are: * tight encoding: no double-base32 encoding of anything, short strings, give up generality in deference to brevity * the final string should be base32- or base62- safe (so it can fit comfortably in a URL's query argument) * we need the encoded message to be serialized in some well-defined form (so that we know how to create its signature), but we also want flexibility for what goes into the encoded message, hence the JSON. For storage authority cert chains, we might not actually want that flexibility: if the client doesn't recognize one of the attenuations, it should fail-safe by rejecting the cert. There are a couple of competing goals here, and I'm not sure what the best answer is.
Author

Versioning: the announcer API (possibly called publish_v2) will
implicitly define the version of the certificate chain: publish_v2 is
always called with a chain in a specific format. If we switch to different
signature schemes (perhaps a different key length) in the future, they can
use a new API call (publish_v3) which will include additional arguments
as necessary, or which will deliver certs with a different format (possibly
including additional data, like which signature scheme is in use).

This technique can be generalized: clients can attempt to call
methname_vNN for the highest ```NN}} they know, if they get a
NameError then they can fall back to older interfaces (or fail), and
they should remember which NN worked so they can avoid repeating the failure
again the next time.

When the cert chain is serialized into a single string (for storage
authority), we'll need to put a magic number and version identifier at the
beginning of that string.

Versioning: the announcer API (possibly called `publish_v2`) will implicitly define the version of the certificate chain: `publish_v2` is always called with a chain in a specific format. If we switch to different signature schemes (perhaps a different key length) in the future, they can use a new API call (`publish_v3`) which will include additional arguments as necessary, or which will deliver certs with a different format (possibly including additional data, like which signature scheme is in use). This technique can be generalized: clients can attempt to call `methname_vNN` for the highest ```NN}} they know, if they get a `NameError` then they can fall back to older interfaces (or fail), and they should remember which NN worked so they can avoid repeating the failure again the next time. When the cert chain is serialized into a single string (for storage authority), we'll need to put a magic number and version identifier at the beginning of that string.
Author

Actually, this is not really a prerequisite for accounting. We can just advertise "storage-login" instead of "storage" through the existing tuple-based interface.

This ticket is really related to "blessed storage servers", which are about protecting the client against low-reliability servers. Accounting is about protecting the servers against unauthorized clients.

So we can put this one off for a while.

Actually, this is not really a prerequisite for accounting. We can just advertise "storage-login" instead of "storage" through the existing tuple-based interface. This ticket is really related to "blessed storage servers", which are about protecting the client against low-reliability servers. Accounting is about protecting the servers against unauthorized clients. So we can put this one off for a while.

Is this a duplicate of #295?

Is this a duplicate of #295?
Author

not exactly, let's say that #295 is about distributed introduction, and this one (#466) is about signed/extendable announcements. I've updated the summary on #295 to match.

not exactly, let's say that #295 is about distributed introduction, and this one (#466) is about signed/extendable announcements. I've updated the summary on #295 to match.
Author

The code for this is pretty much done: my sandbox has a tree which does everything I want it to do, except for:

  • web display should show cert chains, not just last-cert pubkey
  • web display should clearly show which announcements are acceptable and which are not

I'm holding off on pushing it into trunk, though, for two reasons:

  • the ecdsa API in pycryptopp is going to change soon, in incompatible ways, so the tahoe code will need to be changed. Pushing this feature now would result in tahoe-1.3.0 being compatible with only pycryptopp-0.5.7 or older, which would be unfortunate (I don't like it when upgrading a supporting library causes the main application to break or start failing unit tests).
  • the ecdsa serialization format in pycryptopp is going to change soon (the #331 change), and if we can manage to not release a version of tahoe that uses the older (more fluffy) serialization scheme, then we can ease the versioning hassles a little bit.

I'll attach a diff with my changes, for safekeeping.

So this ticket is blocked on #331, and can be pushed to trunk (and then, barring problems, closed) shortly after #331 is finished and a new version of pycryptopp is released.

The code for this is pretty much done: my sandbox has a tree which does everything I want it to do, except for: * web display should show cert chains, not just last-cert pubkey * web display should clearly show which announcements are acceptable and which are not I'm holding off on pushing it into trunk, though, for two reasons: * the ecdsa API in pycryptopp is going to change soon, in incompatible ways, so the tahoe code will need to be changed. Pushing this feature now would result in tahoe-1.3.0 being compatible with only pycryptopp-0.5.7 or older, which would be unfortunate (I don't like it when upgrading a supporting library causes the main application to break or start failing unit tests). * the ecdsa serialization format in pycryptopp is going to change soon (the #331 change), and if we can manage to not release a version of tahoe that uses the older (more fluffy) serialization scheme, then we can ease the versioning hassles a little bit. I'll attach a diff with my changes, for safekeeping. So this ticket is blocked on #331, and can be pushed to trunk (and then, barring problems, closed) shortly after #331 is finished and a new version of pycryptopp is released.
Author

Just an update.. my patch for this will have bitrotted in the last 5 months.. enough trunk code has changed that it will no longer apply cleanly. I estimate it would take me about 3 or 4 days to bring it back up-to-date.

It is still quite blocked on ECDSA (#331). I can do about half the updating work without it (I'd have to create a fake ecdsa module and guess at what the eventual API will be).

Just an update.. my patch for this will have bitrotted in the last 5 months.. enough trunk code has changed that it will no longer apply cleanly. I estimate it would take me about 3 or 4 days to bring it back up-to-date. It is still quite blocked on ECDSA (#331). I can do about half the updating work without it (I'd have to create a fake ecdsa module and guess at what the eventual API will be).
Author

Ok, that -ver6 patch is worth reviewing. I think there are a few more items I
want to add before landing (more tests, mainly), but it's certainly worth
talking about.

This version drops the certchain that was in the previous one: each
announcement is either signed with a single pubkey, or not signed at all. It
uses an embedded copy of my python-ecdsa library
(https://github.com/warner/python-ecdsa), which is a bit on the slow side,
but still fast enough for announcement/introducer use. It adds code to the
IntroducerClient to add, distribute, and verify signatures, but does
not add any code to the Client or StorageBroker to use those
features: actual UI/tahoe.cfg switches to enable signing or require
signatures is left for a future patch.

The patch creates a new version of the Introducer, as well as its protocols.
V2 servers (i.e. Introducers) can accept connections from either V1 or V2
clients, and V2 clients can tolerate talking to a V1 server, so all
combinations are covered. Signatures can only be passed from a V2 client,
through a V2 server, off to another V2 client: any V1 components along the
way will lose the signature.

Each announcement is a dictionary with keys to replace everything that was in
the V1 protocol's tuples. I added some more version information (the full
app-versions dict), which may be too much (especially if you're trying to be
anonymous).

The quirkiest thing about this scheme is the relationship between FURL tubids
and ECDSA pubkeys. The Introducer is supposed to recognize multiple
announcements from the same source and let the new one replace the old one.
When both are V1 tuples with the same tubid in their FURLs, or when both are
V2 dicts signed by the same pubkey, the relationship is easy. If a client is
upgraded from V1 to V2 and starts signing its announcements, the V2
announcement won't replace its old V1 announcement. I'm not sure how to
handle this yet.

In addition, we need to think about how/if we want to transition serverids
from being FURL-based to being pubkey-based. Since serverids are baked into
shares (both via the permuted serverlist and, more directly, by the
Write-Enablers embedded in each mutable share), we can't just casually change
an existing server's id. For brand-new servers, we could switch to using the
pubkey as the serverid: this might make non-Foolscap-based share-transport
protocols easier to secure. For old servers with pre-existing shares that
start signing their announcements, we should probably keep using their tubid
as a serverid, perhaps even after we switch away from foolscap and to some
ECDSA-signature based protocol. But we must validate it: a bad server could
publish a FURL that they don't actually control, with a tubid that matches
the server they wish to impersonate, knowing that we'll never actually
connect to the FURL and discover the problem. I think the validation protocol
will involve connecting to the FURL and receiving a copy of the pubkey back,
to prove that the owner of the FURL really does want to be associated with
that pubkey.

Ok, that -ver6 patch is worth reviewing. I think there are a few more items I want to add before landing (more tests, mainly), but it's certainly worth talking about. This version drops the certchain that was in the previous one: each announcement is either signed with a single pubkey, or not signed at all. It uses an embedded copy of my python-ecdsa library (<https://github.com/warner/python-ecdsa>), which is a bit on the slow side, but still fast enough for announcement/introducer use. It adds code to the `IntroducerClient` to add, distribute, and verify signatures, but does not add any code to the `Client` or `StorageBroker` to use those features: actual UI/tahoe.cfg switches to enable signing or require signatures is left for a future patch. The patch creates a new version of the Introducer, as well as its protocols. V2 servers (i.e. Introducers) can accept connections from either V1 or V2 clients, and V2 clients can tolerate talking to a V1 server, so all combinations are covered. Signatures can only be passed from a V2 client, through a V2 server, off to another V2 client: any V1 components along the way will lose the signature. Each announcement is a dictionary with keys to replace everything that was in the V1 protocol's tuples. I added some more version information (the full app-versions dict), which may be too much (especially if you're trying to be anonymous). The quirkiest thing about this scheme is the relationship between FURL tubids and ECDSA pubkeys. The Introducer is supposed to recognize multiple announcements from the same source and let the new one replace the old one. When both are V1 tuples with the same tubid in their FURLs, or when both are V2 dicts signed by the same pubkey, the relationship is easy. If a client is upgraded from V1 to V2 and starts signing its announcements, the V2 announcement won't replace its old V1 announcement. I'm not sure how to handle this yet. In addition, we need to think about how/if we want to transition serverids from being FURL-based to being pubkey-based. Since serverids are baked into shares (both via the permuted serverlist and, more directly, by the Write-Enablers embedded in each mutable share), we can't just casually change an existing server's id. For brand-new servers, we could switch to using the pubkey as the serverid: this might make non-Foolscap-based share-transport protocols easier to secure. For old servers with pre-existing shares that start signing their announcements, we should probably keep using their tubid as a serverid, perhaps even after we switch away from foolscap and to some ECDSA-signature based protocol. But we must validate it: a bad server could publish a FURL that they don't actually control, with a tubid that matches the server they wish to impersonate, knowing that we'll never actually connect to the FURL and discover the problem. I think the validation protocol will involve connecting to the FURL and receiving a copy of the pubkey back, to prove that the owner of the FURL really does want to be associated with that pubkey.
Author

oops, forgot to set the flag

oops, forgot to set the flag
Author

I think I've got an idea to clean up the "what is your serverid anyway?"
issue. For reference, this is the problem:

  • a server which is signing its Announcements holds two secrets:

    • the Foolscap Tub private key, which is paired with the public TubID
    • the ECDSA signing key, which is paired with the public verifying-key /
      keyid that shows up in the signed announcements
  • For many aspects of Tahoe's share-placement behavior, we need a reliable,

stable, and unspoofable "serverid":

  • serverlist permutation

  • write-enabler secrets

  • consistent server reliability (if you think your share is safe because
    it's on server A, but someone else was able to make you put the share on
    server B instead, then you aren't getting this. Note that server A is
    allowed to unilaterally delegate its storage duties: the failure mode is
    if someone other than you and/or server A is able to make your share go
    elsewhere).

  • and eventually accounting stuff

  • To be unspoofable, the serverid needs to be the public half of some

cryptographic pair, so either the TubID or the ECDSA keyid would do.
To be stable for all current servers, it needs to be the TubID, since
that's what we've been using since the beginning. But eventually we hope to
get away from Foolscap and move to an easier-to-port-to-other-languages
protocol, so eventually we'd prefer to use the ECDSA keyid instead.

  • To use the serverid safely, the client must validate it before allowing any

of the storage-client code (!Uploader/Downloader) to rely upon it.

So today's thought is:

  • the Introducer Announcement dictionary includes two new keys:
    'serverid' and 'serverid-type'
  • the StorageFarmBroker (which subscribes to hear about storage

announcements and manages RIStorageServer connections) keeps two
lists: unvalidated announcements and validated connections. It only tells
!Uploader/Downloader about the validated connections, in a table indexed by
serverid.

  • future world: new servers (who have never been known by their tubid) will

set serverid-type to "pubkey_v1".

  • The broker will validate the announcement by merely comparing
    serverid against the keyid which was used to sign the announcement.
    (IntroducerClient already checks signatures and discards invalid
    ones, so the subscribe interface's callback is invoked with a keyid
    and an announcement dictionary).
  • these new-world servers can still accept Foolscap connections, but
    clients will connect to whatever FURL is in the announcement and won't
    pay attention to that FURL's tubid: they will strictly use the pubkey as
    a serverid.
  • current world: servers who have been known as a tubid will continue to do

so: they will set serverid-type to "FURL_v1". When the broker sees
this, it will:

  • compare the TubID of the claimed FURL against the announcement's
    serverid, abort if they do not match
  • connect to the FURL listed in the announcement
  • invoke a new remote method, maybe named "get_pubkey"
  • abort if the returned pubkey does not match the one used to sign the
    announcement
  • if they do match, validate the connection under the serverid

So when a get_announcement method returns a pubkey, that Tub is
delegating all its authority to the signing key. (this authority includes the
right to know the write-enablers, and the right to decide where its shares
are placed).

The first check (FURL.TubID vs serverid is nominally redundant
(why not just extract the tubid and use it as the serverid), but I expect it
will be convenient to keep a list of validated announcements around, and we'd
like to be able to trust the serverid fields therein. It also provides
useful non-validated data in some places (like the Introducer's web status
page).

The second check (get_pubkey()==serverid) blocks a bunch of noise:
bogus announcements made by someone other than the Tub operator, but pointing
at that Tub. These announcements could contain false information about server
capabilities, free space, versioning, etc (everything in the announcement
dictionary). By making sure that the Tub in question is actually willing to
be represented (in announcements) by that pubkey, we prevent anyone else from
making these false claims.

The legacy authority we're trying to protect here is the tubid-based
serverid. By restricting that authority to servers who can actually provide
an object at that Tub, we prevent other parties from being able to claim that
authority.

It's still an open question in my mind whether this transition is a good idea
overall. I think it will help Tahoe if we can write clients in languages
other than Python, and relying upon Foolscap is a barrier to that (we could
port Foolscap to other languages, but that feels like more work for less gain
than changing Tahoe). If that's a good direction to go, then switching to a
serverid based on some cryptographic quality other than a foolscap tubid is
a necessary step.

I just worry about about the existing servers and the long-term baggage that
they'll need to carry around (since all their mutable shares are tied to the
serverid, they'll need to keep a Tub around forever, to prove knowledge of
the Tub private key, even if all the connections are using ECDSA-signed HTTP
messages or whatnot). (incidentally, this might be relaxed somewhat if
foolscap#19 were implemented,
allowing the SSL key to be used for signing arbitrary data, because then we
could publish the public cert and its signature on the ECDSA pubkey in the
announcement. Both sides would still need to do some SSL work to verify the
signature, but they wouldn't strictly need to use Foolscap to do it, nor
would they need to use foolscap connections to move shares around.

I think I've got an idea to clean up the "what is your serverid anyway?" issue. For reference, this is the problem: * a server which is signing its Announcements holds two secrets: * the Foolscap Tub private key, which is paired with the public `TubID` * the ECDSA signing key, which is paired with the public verifying-key / keyid that shows up in the signed announcements * For many aspects of Tahoe's share-placement behavior, we need a reliable, > stable, and unspoofable "serverid": * serverlist permutation * write-enabler secrets * consistent server reliability (if you think your share is safe because it's on server A, but someone else was able to make you put the share on server B instead, then you aren't getting this. Note that server A is allowed to unilaterally delegate its storage duties: the failure mode is if someone other than you and/or server A is able to make your share go elsewhere). * and eventually accounting stuff * To be unspoofable, the serverid needs to be the public half of some > cryptographic pair, so either the `TubID` or the ECDSA keyid would do. > To be stable for all current servers, it needs to be the `TubID`, since > that's what we've been using since the beginning. But eventually we hope to > get away from Foolscap and move to an easier-to-port-to-other-languages > protocol, so eventually we'd prefer to use the ECDSA keyid instead. * To use the serverid safely, the client must validate it before allowing any > of the storage-client code (!Uploader/Downloader) to rely upon it. So today's thought is: * the Introducer Announcement dictionary includes two new keys: '`serverid`' and '`serverid-type`' * the `StorageFarmBroker` (which subscribes to hear about storage > announcements and manages `RIStorageServer` connections) keeps two > lists: unvalidated announcements and validated connections. It only tells !Uploader/Downloader about the validated connections, in a table indexed by `serverid`. * future world: new servers (who have never been known by their tubid) will > set `serverid-type` to "pubkey_v1". * The broker will validate the announcement by merely comparing `serverid` against the keyid which was used to sign the announcement. (`IntroducerClient` already checks signatures and discards invalid ones, so the `subscribe` interface's callback is invoked with a keyid and an announcement dictionary). * these new-world servers can still accept Foolscap connections, but clients will connect to whatever FURL is in the announcement and won't pay attention to that FURL's tubid: they will strictly use the pubkey as a serverid. * current world: servers who *have* been known as a tubid will continue to do > so: they will set `serverid-type` to "FURL_v1". When the broker sees > this, it will: * compare the `TubID` of the claimed FURL against the announcement's `serverid`, abort if they do not match * connect to the FURL listed in the announcement * invoke a new remote method, maybe named "`get_pubkey`" * abort if the returned pubkey does not match the one used to sign the announcement * if they do match, validate the connection under the `serverid` So when a `get_announcement` method returns a pubkey, that Tub is delegating all its authority to the signing key. (this authority includes the right to know the write-enablers, and the right to decide where its shares are placed). The first check (`FURL.TubID` vs `serverid` is nominally redundant (why not just extract the tubid and use it as the serverid), but I expect it will be convenient to keep a list of validated announcements around, and we'd like to be able to trust the `serverid` fields therein. It also provides useful non-validated data in some places (like the Introducer's web status page). The second check (`get_pubkey()==serverid`) blocks a bunch of noise: bogus announcements made by someone other than the Tub operator, but pointing at that Tub. These announcements could contain false information about server capabilities, free space, versioning, etc (everything in the announcement dictionary). By making sure that the Tub in question is actually willing to be represented (in announcements) by that pubkey, we prevent anyone else from making these false claims. The legacy authority we're trying to protect here is the tubid-based serverid. By restricting that authority to servers who can actually provide an object at that Tub, we prevent other parties from being able to claim that authority. It's still an open question in my mind whether this transition is a good idea overall. I think it will help Tahoe if we can write clients in languages other than Python, and relying upon Foolscap is a barrier to that (we could port Foolscap to other languages, but that feels like more work for less gain than changing Tahoe). If that's a good direction to go, then switching to a serverid based on some cryptographic quality *other* than a foolscap tubid is a necessary step. I just worry about about the existing servers and the long-term baggage that they'll need to carry around (since all their mutable shares are tied to the serverid, they'll need to keep a Tub around forever, to prove knowledge of the Tub private key, even if all the connections are using ECDSA-signed HTTP messages or whatnot). (incidentally, this might be relaxed somewhat if [foolscap#19](http://foolscap.lothar.com/trac/ticket/19) were implemented, allowing the SSL key to be used for signing arbitrary data, because then we could publish the public cert and its signature on the ECDSA pubkey in the announcement. Both sides would still need to do some SSL work to verify the signature, but they wouldn't strictly need to use Foolscap to do it, nor would they need to use foolscap connections to move shares around.

I've begun a review of this ticket. Some initial questions from the first design comment follow. I may answer some of these by reading the patch and the trunk version.

Q1. Why does the design specify EC-DSA-192? What are the requirements which drive this algorithm selection?

Zooko suggested on IRC that the primary goal is small public keys.

Q2. Why is the pubkey-identifier optional? What use case does this facilitate?

Q3. If the identifier is absent, verification is checked against all "root certs". How are these managed?

I've begun a review of this ticket. Some initial questions from the first design comment follow. I may answer some of these by reading the patch and the trunk version. Q1. Why does the design specify EC-DSA-192? What are the requirements which drive this algorithm selection? Zooko suggested on IRC that the primary goal is small public keys. Q2. Why is the pubkey-identifier optional? What use case does this facilitate? Q3. If the identifier is absent, verification is checked against all "root certs". How are these managed?
nejucomo self-assigned this 2011-01-02 19:44:08 +00:00
Author

nejucomo: please start your review with the latest patch (466-ver6.diff). The design has changed a lot since this ticket was first opened, and most of those questions are no longer applicable.

nejucomo: please start your review with the latest patch (466-ver6.diff). The design has changed a lot since this ticket was first opened, and most of those questions are no longer applicable.
daira modified the milestone from undecided to 1.9.0 2011-01-06 00:40:35 +00:00
Author

Attachment 2011-02-p1.diff (100975 bytes) added

copy python-ecdsa-0.7 into tree as allmydata.util.ecdsa

**Attachment** 2011-02-p1.diff (100975 bytes) added copy python-ecdsa-0.7 into tree as allmydata.util.ecdsa
Author

Attachment 2011-02-p2.diff (9476 bytes) added

add 'tahoe admin generate-keypair/derive-pubkey' commands, apply on top of p1

**Attachment** 2011-02-p2.diff (9476 bytes) added add 'tahoe admin generate-keypair/derive-pubkey' commands, apply on top of p1
Author

Attachment 2011-02-p3.diff (111773 bytes) added

new Introducer. see https://github.com/warner/tahoe-lafs/tree/466-introducer-take2 for updates

**Attachment** 2011-02-p3.diff (111773 bytes) added new Introducer. see <https://github.com/warner/tahoe-lafs/tree/466-introducer-take2> for updates
Author

State of the patch

I'd say this project is about 80% complete. The patch currently attached
for review is split up into three pieces. This note is to explain what
those pieces do, what the overall design is like, and what's left to
design/build.

Why Do We Want This?

The goal of this ticket is add signatures to our introducer
announcements, and to make them extensible. Current announcements are a
fixed 6-tuple: (FURL, service_name, remote_interface_name, nickname,
version, oldest_supported). The new announcements will be an arbitrary
JSON-serializable dictionary, embedded in a 3-tuple of (ann_json,
signature, pubkey), where the sig/pubkey are optional.

The utility of extensibility is pretty obvious: there are additional
services and features we could enable (or make more efficient) if we
could safely advertise them ahead of time through the introducer. In
general, extensible protocol formats (dicts, not tuples) improves
flexibility and enables change, since it's awfully difficult to make
changes to all nodes simultaneously. Flexible announcements aren't
strictly necessary: we could instead e.g. add new methods to the
RIStorageServer object, and have clients speculatively attempt to invoke
them, and gently tolerate NameErrors, but that seems inelegant, and
requires a roundtrip: by putting slowly-changing things in the
announcements, clients learn about them earlier.

The value of adding signatures is great, but not immediate. Signatures
would bind the contents of an announcement to some "serverid",
preventing other parties (other grid members, or the Introducer itself)
from forging those contents. This would turn the introduction system
into a secure channel from publisher to subscriber, indexed by serverid.

Without signatures, anybody in the grid can publish anything they like,
such as a record with Alice's real storage-server FURL but with a
nickname of "Bob", which would currently replace Alice's real
announcement.

Currently we use Foolscap Tub IDs (i.e. hash of the tub certificate,
which appears in each FURL) as server IDs. The only way to verify
possession of the corresponding secret is to connect to a FURL that uses
this tubid: Foolscap ensures that the object you connect to (and
subsequent send callRemote messages to) is selected by the
secret-holder. This requires an online check, whereas a signed message
could be verified offline.

We currently use serverids for three things:

  • to distinguish between storage servers who wish to be treated
    separately, sending separate shares to each
  • as a stable long-term seed for the permuted peerlist, used to decide
    how to distribute shares of each file
  • to calculate several shared secrets: the mutable-file
    "write-enabler", and the renew/expire lease tokens. These need to be
    different for each server, so that when a client exercises their
    authority on server A, that doesn't enable A to exercise the client's
    corresponding authority on server B

In the future, we would like to also use serverids to:

  • enable explicit server selection: tahoe.cfg could list the serverids
    that uploads will use, ignoring all others, to protect the user's
    upstream bandwidth and reliability choices. This selection could be
    delegated to a central party, allowing the server list to change over
    time without constant user involvement.
  • correlate reciprocal Accounting relationships

I think that a fully distributed introducer requires announcement
signatures. With a single central Introducer, we could achieve some
measure of control over the grid by restricting publishing access to
certain servers. But with a highly distributed log-flood -based
introduction system, we'd give up central control, and I think
individually-traceable announcements would make up for that loss.

Finally, a long-term goal is to move away from Foolscap to an HTTP-based
protocol that is easier to implement in non-Python languages, to
facilitate multiple implementations of the Tahoe protocol. Without
Foolscap, we'll need a different mechanism to securely identify a
server, for which the announcement signing keypair is appropriate.

patch 1: python-ecdsa

https://github.com/warner/python-ecdsa/ is where I maintain a
pure-python ECDSA library, with a not-too-bad API. I think I may want to
make some changes to the API still. It's fast enough for use by signed
announcements, since sign/verify operations occur only once per server.
Once pycryptopp acquires ECDSA support, we should move to that, for the
30x speedup.

The library is embedded into allmydata/src/util/ecdsa/*.py, rather
than being added as a dependency, for two reasons. First, extra
dependencies are really making packagers lives difficult, slowing
packaging efforts and thus hurting adoption. Second, changes to the
upstream API will be easier to accomodate by using a fixed version of
python-ecdsa, so I think it makes sense to copy it wholesale into the
tahoe tree until the API stabilizes (which needs to be driven by using
it and learning what works and what doesn't).

patch 2: keypair generation

This adds tahoe admin generate-keypair and derive-pubkey,
basic userspace tools to work with keys. The basic idea is that some day
you might use them to creates a value that you then paste into a
tahoe.cfg file. They are currently unused.

patch 3: everything else

I'll split this into:

  • terminology
  • V2 introducer protocol
  • announcement signatures
  • backwards compatibility with V1 protocol
  • serverid computation

Terminology

The IntroducerServer is an object that lives in the introducer
process, the one (just one, so far, but #68 will change that) identified
by the introducer.furl. Most of this note calls this the "server".
The IntroducerClient is an object that lives in each
non-introducer process, both tahoe storage servers and tahoe clients,
which manages the connection to the IntroducerServer.

The "publisher" is a Tahoe node (usually a storage server) which wants
to broadcast information about themselves to the whole grid. Each time
they do this, they are said to "announce" their information. The bundle
of information is called an "announcement".

The "subscriber" is a Tahoe node (usually a client/gateway) that wants
to receive announcements.

Subscribers call their local IntroducerClient.subscribe_to to sign
up to hear announcements, and provide a callback that will be invoked
multiple times as they arrive. This provokes the IntroducerClient
to send a "subscribe" message to the server. Later, the server will send
an "announce" message to the client, and the IntroducerClient will
fire the callback.

Publishers call IntroducerClient.publish to deliver announcements,
which provokes the IntroducerClient to send a "publish" message to
the server, which provokes the server to send "announce" messages to all
interested clients.

V2 Introducer Protocol

The old V1 protocol used 6-tuples as announcements: (FURL, service_name,
remoteinterface_name, nickname, my_version, oldest_supported). The new
V2 protocol uses an open-ended JSON-serializable dictionary, with a
number of top-level keys that are expected to be present.

The V2 client->server "publish" message adds a "canary" argument, which
allows the server to detect when the publisher has disconnected. This is
unused so far, but the intent is to display liveness status on the
server's "introweb" page, and to let them stop publishing data for
servers which are offline (or perhaps have remained offline for several
days).

The V2 client->server "subscribe" message adds a "subscriber_info"
argument, which lets the server's introweb page show information about
each subscriber (mostly nickname and version). In the V1 protocol, this
was accomplished by having each subscriber also "announce" a special
"stub_client" service, which didn't correspond to a real service, but
included enough information to build the status display. The V2
subscriber_info field is defined (by foolscap schema) to be a dictionary
with string keys, with at least "nickname", "my-version",
"app-versions", and "oldest-supported".

Announcement Signatures

V2 announcements on the wire are a 3-tuple (ann_d_json, sig_hex,
pubkey_hex), in which the announcement is serialized with JSON. Unsigned
announcements have sig_hex == pubkey_hex == None.

We use 256-bit ECDSA keys (from the NIST256p curve, since that seemed to
be the most widely implemented, in openssl/nss). pubkey_hex is a
base16-encoded uncompressed raw binary key (TODO: versioning), the
output of VerifyingKey.to_string().encode("hex"). This is 512 bits
long, and neither includes an OID nor a 0x04 "uncompressed" flag byte.
Signatures are computed with SHA1 (TODO: given NIST256p, let's use
SHA256), with an algorithm that is compatible with openssl (verified by
the python-ecdsa test suite). sig_hex is the output of
SigningKey.sign(ann_d_json.encode("utf-8")).encode("hex") which
uses python-ecdsa's minimal binary string encoding (no versioning
information).

Publishers give their IntroducerClient.publish both an
announcement dictionary and a SigningKey instance (or None).
Subscribers receive an announcement dictionary and a VerifyfingKey
instance (which will be None if the announcement did not have a matching
valid signature, which includes both unsigned announcements,
forged/invalid signatures, and valid signatures from some different
pubkey).

Backwards Compatibility with V1

There are four interesting V1+V2 compatibility cases, two on the
publishing half, and two on the subscribing/announcement half.

The V2 IntroducerServer provides all the same method names as the V1
server, plus additional announce_v2/subscribe_v2 methods that are only
used by V2 clients. On the V1 methods, the V2 server accepts the same
message format as the V1 server. The server seeks to hide the client
versions from each other: a V1 client receives only V1-format
announcements, and a V2 client receives only V2-format announcements,
regardless of what client version generated those announcements.

When a V1 client publishes to a V2 server, it uses the old "publish"
method name, allowing the server to detect the client's old version. The
server upconverts the V1-format announcement tuple into an unsigned
V2-format dictionary, leaving some fields empty (like
ann_d["app_versions"]={}) when necessary. It then dispatches this
V2-format announcement internally as if it was received from a real V2
client. When a V1 client subscribes to a V2 server, the old server-side
"subscribe" method wraps the remote reference in a !SubscriberAdapter_v1,
which behaves just like a remote reference to a modern V2 subscriber,
but downconverts the messages to old-style V1 tuples before sending them
over the wire.

When a V2 client tries to publish an announcement, it first tries to
invoke the new "publish_v2" method, with a V2-style announcement
dictionary (maybe signed). If this callRemote fails in a way that looks
like the server does not implement publish_v2, the client concludes that
it is dealing with a V1 server. It then downconverts its annoucement to
V1-style and sends it to the old V1 "publish" method.

When a V2 client wants to subscribe, it first tries the new
"subscribe_v2" method, passing itself as the desired recipient of
"announce_v2" messages (the remote callback, in a sense). If the
"subscribe_v2" method fails, the client concludes that it's dealing with
a V1 server, and calls the old V1 "subscribe" method instead, passing a
different object that can accept V1-style announcements. This
client-side object upconverts each V1 announcement into a V2-format
dictionary before internal delivery. The client also publishes a
"stub_client" announcement, so that the V1 server can display nicknames
and version numbers of all subscribed clients.

=== Server ID Computation ===

This is the trickiest part, and ties into our eventual goals for having
signed announcements.

In the ideal future world, as we envision it today, we no longer use
Foolscap, have no Tub IDs, and use HTTP to send signed unencrypted
storage-protocol messages from client to server. In that world, the
ECDSA pubkey is the serverid, and clients use signed announcements to
learn genuine information about servers.

But since we've been using foolscap tubids as serverids, and since we've
been using serverids to compute both serverlist-permutation and
shared-secrets, there is a compatibility concern. Any shares we've
uploaded to tubid-based servers must continue to be accessible with the
old serverid. I think this means that, even when we add an ECDSA pubkey
to those servers, they need to continue to use their tubid as serverid,
and we need to allow an announcement signed with pubkey-A to correctly
claim to have a serverid tubid-B. This will require a verification step,
in which the client connects to the tubid-B-bearing FURL, and the server
is expected to announce (over that channel) that it uses pubkey-A. Once
that is accomplished, the client can believe other metadata included in
the signed announcement.

This feels complex, so I'm uneasy about it, but here's the protocol I
have in mind:

  • new-style announcements include two keys "serverid" and "serverid-type"
  • for brand-new servers on pure-V2 grids, serverid-type is "pubkey_v1",
    and serverid is the ASCII pubkey
  • for all other servers, serverid-type is "furl_v1", and serverid is
    base32-encoded tubid
  • when client receives announcement from Introducer:
  • serverid-type="pubkey_v1": assert ann[serverid] matches signature
  • serverid-type="furl_v1": connect to ann[furl], send a
    "getAnnouncement" message, response is a new announcement (hopefully
    identical to the one from the Introducer), assert that serverid in
    new ann matches furl, then accept new announcement (ignoring the
    original).

== remaining work ==

  • do we need version numbers in serverids? in signatures?
  • serverid format: they are used in human-readable config files,
    permutation hash. Current serverids are binary. Make new ones ASCII?
  • switch ECDSA hash step from SHA1 to SHA256
  • clean up ascii/unicode transitions
    • plan for HTTP protocol
    • allow unicode in announcement dictionary (especially in nickname)
    • sign only well-defined binary data
    • manage signature handling well enough to allow variety of transport
      protocols (i.e. stick to ASCII)
  • should we replace all data arguments with a single JSON string per
    message, in anticipation of switching the Introducer protocol to HTTP?
    (i.e. design the HTTP protocol first, then map it to Foolscap)
  • turning it all on
**State of the patch** I'd say this project is about 80% complete. The patch currently attached for review is split up into three pieces. This note is to explain what those pieces do, what the overall design is like, and what's left to design/build. ## Why Do We Want This? The goal of this ticket is add signatures to our introducer announcements, and to make them extensible. Current announcements are a fixed 6-tuple: (FURL, service_name, remote_interface_name, nickname, version, oldest_supported). The new announcements will be an arbitrary JSON-serializable dictionary, embedded in a 3-tuple of (ann_json, signature, pubkey), where the sig/pubkey are optional. The utility of extensibility is pretty obvious: there are additional services and features we could enable (or make more efficient) if we could safely advertise them ahead of time through the introducer. In general, extensible protocol formats (dicts, not tuples) improves flexibility and enables change, since it's awfully difficult to make changes to all nodes simultaneously. Flexible announcements aren't strictly necessary: we could instead e.g. add new methods to the RIStorageServer object, and have clients speculatively attempt to invoke them, and gently tolerate NameErrors, but that seems inelegant, and requires a roundtrip: by putting slowly-changing things in the announcements, clients learn about them earlier. The value of adding signatures is great, but not immediate. Signatures would bind the contents of an announcement to some "serverid", preventing other parties (other grid members, or the Introducer itself) from forging those contents. This would turn the introduction system into a secure channel from publisher to subscriber, indexed by serverid. Without signatures, anybody in the grid can publish anything they like, such as a record with Alice's real storage-server FURL but with a nickname of "Bob", which would currently replace Alice's real announcement. Currently we use Foolscap Tub IDs (i.e. hash of the tub certificate, which appears in each FURL) as server IDs. The only way to verify possession of the corresponding secret is to connect to a FURL that uses this tubid: Foolscap ensures that the object you connect to (and subsequent send callRemote messages to) is selected by the secret-holder. This requires an online check, whereas a signed message could be verified offline. We currently use serverids for three things: * to distinguish between storage servers who wish to be treated separately, sending separate shares to each * as a stable long-term seed for the permuted peerlist, used to decide how to distribute shares of each file * to calculate several shared secrets: the mutable-file "write-enabler", and the renew/expire lease tokens. These need to be different for each server, so that when a client exercises their authority on server A, that doesn't enable A to exercise the client's corresponding authority on server B In the future, we would like to also use serverids to: * enable explicit server selection: tahoe.cfg could list the serverids that uploads will use, ignoring all others, to protect the user's upstream bandwidth and reliability choices. This selection could be delegated to a central party, allowing the server list to change over time without constant user involvement. * correlate reciprocal Accounting relationships I *think* that a fully distributed introducer requires announcement signatures. With a single central Introducer, we could achieve some measure of control over the grid by restricting publishing access to certain servers. But with a highly distributed log-flood -based introduction system, we'd give up central control, and I think individually-traceable announcements would make up for that loss. Finally, a long-term goal is to move away from Foolscap to an HTTP-based protocol that is easier to implement in non-Python languages, to facilitate multiple implementations of the Tahoe protocol. Without Foolscap, we'll need a different mechanism to securely identify a server, for which the announcement signing keypair is appropriate. ## patch 1: python-ecdsa <https://github.com/warner/python-ecdsa/> is where I maintain a pure-python ECDSA library, with a not-too-bad API. I think I may want to make some changes to the API still. It's fast enough for use by signed announcements, since sign/verify operations occur only once per server. Once pycryptopp acquires ECDSA support, we should move to that, for the 30x speedup. The library is embedded into `allmydata/src/util/ecdsa/*.py`, rather than being added as a dependency, for two reasons. First, extra dependencies are really making packagers lives difficult, slowing packaging efforts and thus hurting adoption. Second, changes to the upstream API will be easier to accomodate by using a fixed version of python-ecdsa, so I think it makes sense to copy it wholesale into the tahoe tree until the API stabilizes (which needs to be driven by using it and learning what works and what doesn't). ## patch 2: keypair generation This adds `tahoe admin generate-keypair` and `derive-pubkey`, basic userspace tools to work with keys. The basic idea is that some day you might use them to creates a value that you then paste into a tahoe.cfg file. They are currently unused. ## patch 3: everything else I'll split this into: * terminology * V2 introducer protocol * announcement signatures * backwards compatibility with V1 protocol * serverid computation ### Terminology The `IntroducerServer` is an object that lives in the introducer process, the one (just one, so far, but #68 will change that) identified by the `introducer.furl`. Most of this note calls this the "server". The `IntroducerClient` is an object that lives in each non-introducer process, both tahoe storage servers and tahoe clients, which manages the connection to the `IntroducerServer`. The "publisher" is a Tahoe node (usually a storage server) which wants to broadcast information about themselves to the whole grid. Each time they do this, they are said to "announce" their information. The bundle of information is called an "announcement". The "subscriber" is a Tahoe node (usually a client/gateway) that wants to receive announcements. Subscribers call their local `IntroducerClient.subscribe_to` to sign up to hear announcements, and provide a callback that will be invoked multiple times as they arrive. This provokes the `IntroducerClient` to send a "subscribe" message to the server. Later, the server will send an "announce" message to the client, and the `IntroducerClient` will fire the callback. Publishers call `IntroducerClient.publish` to deliver announcements, which provokes the `IntroducerClient` to send a "publish" message to the server, which provokes the server to send "announce" messages to all interested clients. ### V2 Introducer Protocol The old V1 protocol used 6-tuples as announcements: (FURL, service_name, remoteinterface_name, nickname, my_version, oldest_supported). The new V2 protocol uses an open-ended JSON-serializable dictionary, with a number of top-level keys that are expected to be present. The V2 client->server "publish" message adds a "canary" argument, which allows the server to detect when the publisher has disconnected. This is unused so far, but the intent is to display liveness status on the server's "introweb" page, and to let them stop publishing data for servers which are offline (or perhaps have remained offline for several days). The V2 client->server "subscribe" message adds a "subscriber_info" argument, which lets the server's introweb page show information about each subscriber (mostly nickname and version). In the V1 protocol, this was accomplished by having each subscriber also "announce" a special "stub_client" service, which didn't correspond to a real service, but included enough information to build the status display. The V2 subscriber_info field is defined (by foolscap schema) to be a dictionary with string keys, with at least "nickname", "my-version", "app-versions", and "oldest-supported". ### Announcement Signatures V2 announcements on the wire are a 3-tuple (ann_d_json, sig_hex, pubkey_hex), in which the announcement is serialized with JSON. Unsigned announcements have `sig_hex == pubkey_hex == None`. We use 256-bit ECDSA keys (from the NIST256p curve, since that seemed to be the most widely implemented, in openssl/nss). `pubkey_hex` is a base16-encoded uncompressed raw binary key (TODO: versioning), the output of `VerifyingKey.to_string().encode("hex")`. This is 512 bits long, and neither includes an OID nor a 0x04 "uncompressed" flag byte. Signatures are computed with SHA1 (TODO: given NIST256p, let's use SHA256), with an algorithm that is compatible with openssl (verified by the python-ecdsa test suite). `sig_hex` is the output of `SigningKey.sign(ann_d_json.encode("utf-8")).encode("hex")` which uses python-ecdsa's minimal binary string encoding (no versioning information). Publishers give their `IntroducerClient.publish` both an announcement dictionary and a `SigningKey` instance (or None). Subscribers receive an announcement dictionary and a `VerifyfingKey` instance (which will be None if the announcement did not have a matching valid signature, which includes both unsigned announcements, forged/invalid signatures, and valid signatures from some different pubkey). ### Backwards Compatibility with V1 There are four interesting V1+V2 compatibility cases, two on the publishing half, and two on the subscribing/announcement half. The V2 IntroducerServer provides all the same method names as the V1 server, plus additional announce_v2/subscribe_v2 methods that are only used by V2 clients. On the V1 methods, the V2 server accepts the same message format as the V1 server. The server seeks to hide the client versions from each other: a V1 client receives only V1-format announcements, and a V2 client receives only V2-format announcements, regardless of what client version generated those announcements. When a V1 client publishes to a V2 server, it uses the old "publish" method name, allowing the server to detect the client's old version. The server upconverts the V1-format announcement tuple into an unsigned V2-format dictionary, leaving some fields empty (like `ann_d["app_versions"]={`}) when necessary. It then dispatches this V2-format announcement internally as if it was received from a real V2 client. When a V1 client subscribes to a V2 server, the old server-side "subscribe" method wraps the remote reference in a !SubscriberAdapter_v1, which behaves just like a remote reference to a modern V2 subscriber, but downconverts the messages to old-style V1 tuples before sending them over the wire. When a V2 client tries to publish an announcement, it first tries to invoke the new "publish_v2" method, with a V2-style announcement dictionary (maybe signed). If this callRemote fails in a way that looks like the server does not implement publish_v2, the client concludes that it is dealing with a V1 server. It then downconverts its annoucement to V1-style and sends it to the old V1 "publish" method. When a V2 client wants to subscribe, it first tries the new "subscribe_v2" method, passing itself as the desired recipient of "announce_v2" messages (the remote callback, in a sense). If the "subscribe_v2" method fails, the client concludes that it's dealing with a V1 server, and calls the old V1 "subscribe" method instead, passing a different object that can accept V1-style announcements. This client-side object upconverts each V1 announcement into a V2-format dictionary before internal delivery. The client also publishes a "stub_client" announcement, so that the V1 server can display nicknames and version numbers of all subscribed clients. === Server ID Computation === This is the trickiest part, and ties into our eventual goals for having signed announcements. In the ideal future world, as we envision it today, we no longer use Foolscap, have no Tub IDs, and use HTTP to send signed unencrypted storage-protocol messages from client to server. In that world, the ECDSA pubkey is the serverid, and clients use signed announcements to learn genuine information about servers. But since we've been using foolscap tubids as serverids, and since we've been using serverids to compute both serverlist-permutation and shared-secrets, there is a compatibility concern. Any shares we've uploaded to tubid-based servers must continue to be accessible with the old serverid. I think this means that, even when we add an ECDSA pubkey to those servers, they need to continue to use their tubid as serverid, and we need to allow an announcement signed with pubkey-A to correctly claim to have a serverid tubid-B. This will require a verification step, in which the client connects to the tubid-B-bearing FURL, and the server is expected to announce (over that channel) that it uses pubkey-A. Once that is accomplished, the client can believe other metadata included in the signed announcement. This feels complex, so I'm uneasy about it, but here's the protocol I have in mind: * new-style announcements include two keys "serverid" and "serverid-type" - for brand-new servers on pure-V2 grids, serverid-type is "pubkey_v1", and serverid is the ASCII pubkey - for all other servers, serverid-type is "furl_v1", and serverid is base32-encoded tubid * when client receives announcement from Introducer: - serverid-type="pubkey_v1": assert ann[serverid] matches signature - serverid-type="furl_v1": connect to ann[furl], send a "getAnnouncement" message, response is a new announcement (hopefully identical to the one from the Introducer), assert that serverid in new ann matches furl, then accept new announcement (ignoring the original). == remaining work == * do we need version numbers in serverids? in signatures? * serverid format: they are used in human-readable config files, permutation hash. Current serverids are binary. Make new ones ASCII? * switch ECDSA hash step from SHA1 to SHA256 * clean up ascii/unicode transitions - plan for HTTP protocol - allow unicode in announcement dictionary (especially in nickname) - sign only well-defined binary data - manage signature handling well enough to allow variety of transport protocols (i.e. stick to ASCII) * should we replace all data arguments with a single JSON string per message, in anticipation of switching the Introducer protocol to HTTP? (i.e. design the HTTP protocol first, then map it to Foolscap) * turning it all on

Brian's opinion needed on whether this will be ready for 1.9.

Brian's opinion needed on whether this will be ready for 1.9.
nejucomo was unassigned by daira 2011-07-16 21:28:13 +00:00
warner was assigned by daira 2011-07-16 21:28:13 +00:00
Author

sigh, no. I grow increasingly less hopeful that this will ever see the light of day. Bumping out of 1.9

sigh, no. I grow increasingly less hopeful that this will ever see the light of day. Bumping out of 1.9
warner modified the milestone from 1.9.0 to 1.10.0 2011-07-25 06:04:40 +00:00
Author

ok, we're getting close here. The current work is on my github branch:

https://github.com/warner/tahoe-lafs/tree/466-take8

(note: I rebase this branch frequently. Also, I switch to subsequent "takeNN" branches when the need arises)

We went (quickly) over the design and code at the Summit, and I fixed all the issues that were raised there. So the branch is ready to land once the necessary dependencies are in place. The sequence from here is:

  • Zooko lands the patch in pycryptopp#75
  • Zooko makes a new release of pycryptopp (maybe named 0.6?)
  • we update tahoe's _autodeps.py to depend on the new pycryptopp
  • we land my 466-take8 branch
  • ???
  • profit!
ok, we're getting close here. The current work is on my github branch: <https://github.com/warner/tahoe-lafs/tree/466-take8> (note: I rebase this branch frequently. Also, I switch to subsequent "takeNN" branches when the need arises) We went (quickly) over the design and code at the Summit, and I fixed all the issues that were raised there. So the branch is ready to land once the necessary dependencies are in place. The sequence from here is: * Zooko lands the patch in [pycryptopp#75](https://tahoe-lafs.org/trac/pycryptopp/ticket/75) * Zooko makes a new release of pycryptopp (maybe named 0.6?) * we update tahoe's `_autodeps.py` to depend on the new pycryptopp * we land my `466-take8` branch * ??? * profit!

A darcs patch against trunk corresponding to the git branch is at https://tahoe-lafs.org/~davidsarah/patches/introducer-signed-messages.darcs.patch.

I made some minor changes to the code and tests for the tahoe admin commands (see the patch descriptions), and fixed some duplicate umids; apart from that the code in the darcs patch is the same. I confirmed that it passes tests on Ubuntu Maverick x86-64, provided that pycryptopp with the https://tahoe-lafs.org/trac/pycryptopp/ticket/75 patch is installed, and modulo some failures due to #1586.

A darcs patch against trunk corresponding to the git branch is at <https://tahoe-lafs.org/~davidsarah/patches/introducer-signed-messages.darcs.patch>. I made some minor changes to the code and tests for the `tahoe admin` commands (see the patch descriptions), and fixed some duplicate umids; apart from that the code in the darcs patch is the same. I confirmed that it passes tests on Ubuntu Maverick x86-64, provided that pycryptopp with the <https://tahoe-lafs.org/trac/pycryptopp/ticket/75> patch is installed, and modulo some failures due to #1586.

At warner's request, I rerecorded the darcs patches so that my changes are in separate patches to the changes from the git branch. They are at https://tahoe-lafs.org/~davidsarah/patches/introducer-signed-messages-v2.darcs.patch.

Note that 'two.diff' (output of git show a604a) is still split into two darcs patches, for code and test changes.

At warner's request, I rerecorded the darcs patches so that my changes are in separate patches to the changes from the git branch. They are at <https://tahoe-lafs.org/~davidsarah/patches/introducer-signed-messages-v2.darcs.patch>. Note that 'two.diff' (output of `git show a604a`) is still split into two darcs patches, for code and test changes.
Brian Warner <warner@lothar.com> commented 2012-03-14 01:26:57 +00:00
Owner

In changeset:bc21726dfd73b434:

new introducer: signed extensible dictionary-based messages! refs #466

This introduces new client and server halves to the Introducer (renaming the
old one with a _V1 suffix). Both have fallbacks to accomodate talking to a
different version: the publishing client switches on whether the server's
.get_version() advertises V2 support, the server switches on which
subscription method was invoked by the subscribing client.

The V2 protocol sends a three-tuple of (serialized announcement dictionary,
signature, pubkey) for each announcement. The V2 server dispatches messages
to subscribers according to the service-name, and throws errors for invalid
signatures, but does not otherwise examine the messages. The V2 receiver's
subscription callback will receive a (serverid, ann_dict) pair. The
'serverid' will be equal to the pubkey if all of the following are true:

  the originating client is V2, and was told a privkey to use
  the announcement went through a V2 server
  the signature is valid

If not, 'serverid' will be equal to the tubid portion of the announced FURL,
as was the case for V1 receivers.

Servers will create a keypair if one does not exist yet, stored in
private/server.privkey .

The signed announcement dictionary puts the server FURL in a key named
"anonymous-storage-FURL", which anticipates upcoming Accounting-related
changes in the server advertisements. It also provides a key named
"permutation-seed-base32" to tell clients what permutation seed to use. This
is computed at startup, using tubid if there are existing shares, otherwise
the pubkey, to retain share-order compatibility for existing servers.
In changeset:bc21726dfd73b434: ``` new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers. ```
Brian Warner <warner@lothar.com> commented 2012-03-14 01:27:42 +00:00
Owner

In changeset:bc21726dfd73b434:

new introducer: signed extensible dictionary-based messages! refs #466

This introduces new client and server halves to the Introducer (renaming the
old one with a _V1 suffix). Both have fallbacks to accomodate talking to a
different version: the publishing client switches on whether the server's
.get_version() advertises V2 support, the server switches on which
subscription method was invoked by the subscribing client.

The V2 protocol sends a three-tuple of (serialized announcement dictionary,
signature, pubkey) for each announcement. The V2 server dispatches messages
to subscribers according to the service-name, and throws errors for invalid
signatures, but does not otherwise examine the messages. The V2 receiver's
subscription callback will receive a (serverid, ann_dict) pair. The
'serverid' will be equal to the pubkey if all of the following are true:

  the originating client is V2, and was told a privkey to use
  the announcement went through a V2 server
  the signature is valid

If not, 'serverid' will be equal to the tubid portion of the announced FURL,
as was the case for V1 receivers.

Servers will create a keypair if one does not exist yet, stored in
private/server.privkey .

The signed announcement dictionary puts the server FURL in a key named
"anonymous-storage-FURL", which anticipates upcoming Accounting-related
changes in the server advertisements. It also provides a key named
"permutation-seed-base32" to tell clients what permutation seed to use. This
is computed at startup, using tubid if there are existing shares, otherwise
the pubkey, to retain share-order compatibility for existing servers.
In changeset:bc21726dfd73b434: ``` new introducer: signed extensible dictionary-based messages! refs #466 This introduces new client and server halves to the Introducer (renaming the old one with a _V1 suffix). Both have fallbacks to accomodate talking to a different version: the publishing client switches on whether the server's .get_version() advertises V2 support, the server switches on which subscription method was invoked by the subscribing client. The V2 protocol sends a three-tuple of (serialized announcement dictionary, signature, pubkey) for each announcement. The V2 server dispatches messages to subscribers according to the service-name, and throws errors for invalid signatures, but does not otherwise examine the messages. The V2 receiver's subscription callback will receive a (serverid, ann_dict) pair. The 'serverid' will be equal to the pubkey if all of the following are true: the originating client is V2, and was told a privkey to use the announcement went through a V2 server the signature is valid If not, 'serverid' will be equal to the tubid portion of the announced FURL, as was the case for V1 receivers. Servers will create a keypair if one does not exist yet, stored in private/server.privkey . The signed announcement dictionary puts the server FURL in a key named "anonymous-storage-FURL", which anticipates upcoming Accounting-related changes in the server advertisements. It also provides a key named "permutation-seed-base32" to tell clients what permutation seed to use. This is computed at startup, using tubid if there are existing shares, otherwise the pubkey, to retain share-order compatibility for existing servers. ```

Servers will create a keypair if one does not exist yet, stored in private/server.privkey .

Existing private key files are called *.pem, e.g. node.pem.

> Servers will create a keypair if one does not exist yet, stored in private/server.privkey . Existing private key files are called `*.pem`, e.g. `node.pem`.

Don't we need to make Tahoe dependent on "pycryptopp >= 0.6.0"?

Don't we need to make Tahoe dependent on "pycryptopp >= 0.6.0"?

Replying to davidsarah:

Don't we need to make Tahoe dependent on "pycryptopp >= 0.6.0"?

Oh, it is, but the addition to install_requires is done in the require_more function of source:src/allmydata/_auto_deps.py. It could just as well be in the static definition of install_requires, since it's unconditional.

Replying to [davidsarah](/tahoe-lafs/trac/issues/466#issuecomment-367919): > Don't we need to make Tahoe dependent on "pycryptopp >= 0.6.0"? Oh, it is, but the addition to `install_requires` is done in the `require_more` function of source:src/allmydata/_auto_deps.py. It could just as well be in the static definition of `install_requires`, since it's unconditional.
Author

Replying to davidsarah:

Servers will create a keypair if one does not exist yet, stored in private/server.privkey .

Existing private key files are called *.pem, e.g. node.pem.

This new server key is an Ed25519 key (stored as 32 binary bytes), not an X509/SSL key, so .pem didn't seem appropriate. I'm happy to use a different name, though.

It could just as well be in the static definition of install_requires, since it's unconditional.

Good point, I'll move it there.

Replying to [davidsarah](/tahoe-lafs/trac/issues/466#issuecomment-367918): > > Servers will create a keypair if one does not exist yet, stored in private/server.privkey . > > Existing private key files are called `*.pem`, e.g. `node.pem`. This new server key is an Ed25519 key (stored as 32 binary bytes), not an X509/SSL key, so .pem didn't seem appropriate. I'm happy to use a different name, though. > It could just as well be in the static definition of install_requires, since it's unconditional. Good point, I'll move it there.

Replying to [warner]comment:30:

Replying to davidsarah:

Existing private key files are called *.pem, e.g. node.pem.

This new server key is an Ed25519 key (stored as 32 binary bytes), not an X509/SSL key, so .pem didn't seem appropriate. I'm happy to use a different name, though.

Oh, in that case server.privkey is fine.

Replying to [warner]comment:30: > Replying to [davidsarah](/tahoe-lafs/trac/issues/466#issuecomment-367918): > > Existing private key files are called `*.pem`, e.g. `node.pem`. > > This new server key is an Ed25519 key (stored as 32 binary bytes), not an X509/SSL key, so .pem didn't seem appropriate. I'm happy to use a different name, though. Oh, in that case `server.privkey` is fine.
daira modified the milestone from 1.11.0 to 1.10.0 2012-04-01 00:31:20 +00:00
Author

Ok, time to close this one out. The code landed a while ago, and things look
stable. A few more notes about the final protocol that we settled on (and how it
differs from the big explanation in comment:367907) :

  • announcements contain separate fields for the different uses of a "server

id". This turned out to work better than having a "serverid-type" marker.

  • anonymous-storage-FURL: points at the foolscap object that provides
    non-Accounting-based storage service, using the same protocol we've been
    using for years. It has the "anonymous-" prefix to distinguish it from the
    non-anonymous thing that Accounting (#666) will provide. When a server
    requires accounting-based connections, its announcement will omit
    anonymous-storage-FURL.
  • permutation-seed-base32: tells clients where to place this server in
    the permuted ring, for share-placement purposes. The server gets to decide
    this placement independently of its serverid or tubid (this is marginally
    more power than they had before, but we decided it was safe). New servers
    use the public-key -based serverid for this. Old servers (those which have
    published shares under their tubid) keep using their tubid for this, to
    maintain stability (clients keep looking for shares in the same old
    place). The code in client.py:_init_permutation_seed() holds this
    "Am I old or new" logic.
  • the IntroducerClient which receives these announcements produces an
    IServer object, specifically a
    allmydata.storage_client.NativeStorageServer, which offers methods

that client code can use to figure out how to talk to the server.

  • get_permutation_seed(): maps directly to permutation-seed-base32
  • get_lease_seed() and get_foolscap_write_enabler_seed(): both
    return the tubid, never the pubkey-based serverid, because these are
    used for shared-secret authorization of
    add-lease/remove-lease/modify-share operations, and shared-secrets are only
    safe to inside a channel that's tied to those secrets. Since these
    operations are closely tied to Foolscap, it's ok to leave them using the
    tubid. When we move to a non-Foolscap transport layer, we'll need to use
    different authorization mechanisms for leases and write-enablers, and
    these will go away.
  • the pubkey-based serverid will be used for all future

explicit-server-selection and Accounting needs, never the tubid

  • the code uses ed25519 signatures (from pycryptopp), not python-ecdsa. These

signatures are shorter, faster, and more secure.

  • serverids have "v0-" version prefixes, as do the signatures. The pubkey and

signatures are sent as base32-encoded strings, to make non-binary-safe
transports easier to accomodate. When presenting serverids to users, the
v0- prefix is removed.

  • announcements are JSONable dictionaries with unicode fields. The signature

is computed over its UTF-8 encoding.

  • the new protocol is enabled by default. As soon as you start a node with

the new code, it will generate a new key and start using it.

  • a copy of the old (V1) introducer client+server is retained in

src/allmydata/introducer/old.py for testing purposes. Once we're no longer
concerned with maintaining compatibility with V1 introducers or clients, we
can remove it.

Ok, time to close this one out. The code landed a while ago, and things look stable. A few more notes about the final protocol that we settled on (and how it differs from the big explanation in [comment:367907](/tahoe-lafs/trac/issues/466#issuecomment-367907)) : * announcements contain separate fields for the different uses of a "server > id". This turned out to work better than having a "serverid-type" marker. * `anonymous-storage-FURL`: points at the foolscap object that provides non-Accounting-based storage service, using the same protocol we've been using for years. It has the "anonymous-" prefix to distinguish it from the non-anonymous thing that Accounting (#666) will provide. When a server requires accounting-based connections, its announcement will omit `anonymous-storage-FURL`. * `permutation-seed-base32`: tells clients where to place this server in the permuted ring, for share-placement purposes. The server gets to decide this placement independently of its serverid or tubid (this is marginally more power than they had before, but we decided it was safe). New servers use the public-key -based serverid for this. Old servers (those which have published shares under their tubid) keep using their tubid for this, to maintain stability (clients keep looking for shares in the same old place). The code in `client.py:_init_permutation_seed()` holds this "Am I old or new" logic. * the `IntroducerClient` which receives these announcements produces an `IServer` object, specifically a `allmydata.storage_client.NativeStorageServer`, which offers methods > that client code can use to figure out how to talk to the server. * `get_permutation_seed()`: maps directly to `permutation-seed-base32` * `get_lease_seed()` and `get_foolscap_write_enabler_seed()`: both return the *tubid*, never the pubkey-based serverid, because these are used for shared-secret authorization of add-lease/remove-lease/modify-share operations, and shared-secrets are only safe to inside a channel that's tied to those secrets. Since these operations are closely tied to Foolscap, it's ok to leave them using the tubid. When we move to a non-Foolscap transport layer, we'll need to use different authorization mechanisms for leases and write-enablers, and these will go away. * the pubkey-based serverid will be used for all future > explicit-server-selection and Accounting needs, never the tubid * the code uses ed25519 signatures (from pycryptopp), not python-ecdsa. These > signatures are shorter, faster, and more secure. * serverids have "v0-" version prefixes, as do the signatures. The pubkey and > signatures are sent as base32-encoded strings, to make non-binary-safe > transports easier to accomodate. When presenting serverids to users, the > v0- prefix is removed. * announcements are JSONable dictionaries with unicode fields. The signature > is computed over its UTF-8 encoding. * the new protocol is enabled by default. As soon as you start a node with > the new code, it will generate a new key and start using it. * a copy of the old (V1) introducer client+server is retained in > src/allmydata/introducer/old.py for testing purposes. Once we're no longer > concerned with maintaining compatibility with V1 introducers or clients, we > can remove it.
warner added the
r/fixed
label 2012-05-14 23:18:09 +00:00
Sign in to join this conversation.
No labels
c/code
c/code-dirnodes
c/code-encoding
c/code-frontend
c/code-frontend-cli
c/code-frontend-ftp-sftp
c/code-frontend-magic-folder
c/code-frontend-web
c/code-mutable
c/code-network
c/code-nodeadmin
c/code-peerselection
c/code-storage
c/contrib
c/dev-infrastructure
c/docs
c/operational
c/packaging
c/unknown
c/website
kw:2pc
kw:410
kw:9p
kw:ActivePerl
kw:AttributeError
kw:DataUnavailable
kw:DeadReferenceError
kw:DoS
kw:FileZilla
kw:GetLastError
kw:IFinishableConsumer
kw:K
kw:LeastAuthority
kw:Makefile
kw:RIStorageServer
kw:StringIO
kw:UncoordinatedWriteError
kw:about
kw:access
kw:access-control
kw:accessibility
kw:accounting
kw:accounting-crawler
kw:add-only
kw:aes
kw:aesthetics
kw:alias
kw:aliases
kw:aliens
kw:allmydata
kw:amazon
kw:ambient
kw:annotations
kw:anonymity
kw:anonymous
kw:anti-censorship
kw:api_auth_token
kw:appearance
kw:appname
kw:apport
kw:archive
kw:archlinux
kw:argparse
kw:arm
kw:assertion
kw:attachment
kw:auth
kw:authentication
kw:automation
kw:avahi
kw:availability
kw:aws
kw:azure
kw:backend
kw:backoff
kw:backup
kw:backupdb
kw:backward-compatibility
kw:bandwidth
kw:basedir
kw:bayes
kw:bbfreeze
kw:beta
kw:binaries
kw:binutils
kw:bitcoin
kw:bitrot
kw:blacklist
kw:blocker
kw:blocks-cloud-deployment
kw:blocks-cloud-merge
kw:blocks-magic-folder-merge
kw:blocks-merge
kw:blocks-raic
kw:blocks-release
kw:blog
kw:bom
kw:bonjour
kw:branch
kw:branding
kw:breadcrumbs
kw:brians-opinion-needed
kw:browser
kw:bsd
kw:build
kw:build-helpers
kw:buildbot
kw:builders
kw:buildslave
kw:buildslaves
kw:cache
kw:cap
kw:capleak
kw:captcha
kw:cast
kw:centos
kw:cffi
kw:chacha
kw:charset
kw:check
kw:checker
kw:chroot
kw:ci
kw:clean
kw:cleanup
kw:cli
kw:cloud
kw:cloud-backend
kw:cmdline
kw:code
kw:code-checks
kw:coding-standards
kw:coding-tools
kw:coding_tools
kw:collection
kw:compatibility
kw:completion
kw:compression
kw:confidentiality
kw:config
kw:configuration
kw:configuration.txt
kw:conflict
kw:connection
kw:connectivity
kw:consistency
kw:content
kw:control
kw:control.furl
kw:convergence
kw:coordination
kw:copyright
kw:corruption
kw:cors
kw:cost
kw:coverage
kw:coveralls
kw:coveralls.io
kw:cpu-watcher
kw:cpyext
kw:crash
kw:crawler
kw:crawlers
kw:create-container
kw:cruft
kw:crypto
kw:cryptography
kw:cryptography-lib
kw:cryptopp
kw:csp
kw:curl
kw:cutoff-date
kw:cycle
kw:cygwin
kw:d3
kw:daemon
kw:darcs
kw:darcsver
kw:database
kw:dataloss
kw:db
kw:dead-code
kw:deb
kw:debian
kw:debug
kw:deep-check
kw:defaults
kw:deferred
kw:delete
kw:deletion
kw:denial-of-service
kw:dependency
kw:deployment
kw:deprecation
kw:desert-island
kw:desert-island-build
kw:design
kw:design-review-needed
kw:detection
kw:dev-infrastructure
kw:devpay
kw:directory
kw:directory-page
kw:dirnode
kw:dirnodes
kw:disconnect
kw:discovery
kw:disk
kw:disk-backend
kw:distribute
kw:distutils
kw:dns
kw:do_http
kw:doc-needed
kw:docker
kw:docs
kw:docs-needed
kw:dokan
kw:dos
kw:download
kw:downloader
kw:dragonfly
kw:drop-upload
kw:duplicity
kw:dusty
kw:earth-dragon
kw:easy
kw:ec2
kw:ecdsa
kw:ed25519
kw:egg-needed
kw:eggs
kw:eliot
kw:email
kw:empty
kw:encoding
kw:endpoint
kw:enterprise
kw:enum34
kw:environment
kw:erasure
kw:erasure-coding
kw:error
kw:escaping
kw:etag
kw:etch
kw:evangelism
kw:eventual
kw:example
kw:excess-authority
kw:exec
kw:exocet
kw:expiration
kw:extensibility
kw:extension
kw:failure
kw:fedora
kw:ffp
kw:fhs
kw:figleaf
kw:file
kw:file-descriptor
kw:filename
kw:filesystem
kw:fileutil
kw:fips
kw:firewall
kw:first
kw:floatingpoint
kw:flog
kw:foolscap
kw:forward-compatibility
kw:forward-secrecy
kw:forwarding
kw:free
kw:freebsd
kw:frontend
kw:fsevents
kw:ftp
kw:ftpd
kw:full
kw:furl
kw:fuse
kw:garbage
kw:garbage-collection
kw:gateway
kw:gatherer
kw:gc
kw:gcc
kw:gentoo
kw:get
kw:git
kw:git-annex
kw:github
kw:glacier
kw:globalcaps
kw:glossary
kw:google-cloud-storage
kw:google-drive-backend
kw:gossip
kw:governance
kw:grid
kw:grid-manager
kw:gridid
kw:gridsync
kw:grsec
kw:gsoc
kw:gvfs
kw:hackfest
kw:hacktahoe
kw:hang
kw:hardlink
kw:heartbleed
kw:heisenbug
kw:help
kw:helper
kw:hint
kw:hooks
kw:how
kw:how-to
kw:howto
kw:hp
kw:hp-cloud
kw:html
kw:http
kw:https
kw:i18n
kw:i2p
kw:i2p-collab
kw:illustration
kw:image
kw:immutable
kw:impressions
kw:incentives
kw:incident
kw:init
kw:inlineCallbacks
kw:inotify
kw:install
kw:installer
kw:integration
kw:integration-test
kw:integrity
kw:interactive
kw:interface
kw:interfaces
kw:interoperability
kw:interstellar-exploration
kw:introducer
kw:introduction
kw:iphone
kw:ipkg
kw:iputil
kw:ipv6
kw:irc
kw:jail
kw:javascript
kw:joke
kw:jquery
kw:json
kw:jsui
kw:junk
kw:key-value-store
kw:kfreebsd
kw:known-issue
kw:konqueror
kw:kpreid
kw:kvm
kw:l10n
kw:lae
kw:large
kw:latency
kw:leak
kw:leasedb
kw:leases
kw:libgmp
kw:license
kw:licenss
kw:linecount
kw:link
kw:linux
kw:lit
kw:localhost
kw:location
kw:locking
kw:logging
kw:logo
kw:loopback
kw:lucid
kw:mac
kw:macintosh
kw:magic-folder
kw:manhole
kw:manifest
kw:manual-test-needed
kw:map
kw:mapupdate
kw:max_space
kw:mdmf
kw:memcheck
kw:memory
kw:memory-leak
kw:mesh
kw:metadata
kw:meter
kw:migration
kw:mime
kw:mingw
kw:minimal
kw:misc
kw:miscapture
kw:mlp
kw:mock
kw:more-info-needed
kw:mountain-lion
kw:move
kw:multi-users
kw:multiple
kw:multiuser-gateway
kw:munin
kw:music
kw:mutability
kw:mutable
kw:mystery
kw:names
kw:naming
kw:nas
kw:navigation
kw:needs-review
kw:needs-spawn
kw:netbsd
kw:network
kw:nevow
kw:new-user
kw:newcaps
kw:news
kw:news-done
kw:news-needed
kw:newsletter
kw:newurls
kw:nfc
kw:nginx
kw:nixos
kw:no-clobber
kw:node
kw:node-url
kw:notification
kw:notifyOnDisconnect
kw:nsa310
kw:nsa320
kw:nsa325
kw:numpy
kw:objects
kw:old
kw:openbsd
kw:openitp-packaging
kw:openssl
kw:openstack
kw:opensuse
kw:operation-helpers
kw:operational
kw:operations
kw:ophandle
kw:ophandles
kw:ops
kw:optimization
kw:optional
kw:options
kw:organization
kw:os
kw:os.abort
kw:ostrom
kw:osx
kw:osxfuse
kw:otf-magic-folder-objective1
kw:otf-magic-folder-objective2
kw:otf-magic-folder-objective3
kw:otf-magic-folder-objective4
kw:otf-magic-folder-objective5
kw:otf-magic-folder-objective6
kw:p2p
kw:packaging
kw:partial
kw:password
kw:path
kw:paths
kw:pause
kw:peer-selection
kw:performance
kw:permalink
kw:permissions
kw:persistence
kw:phone
kw:pickle
kw:pip
kw:pipermail
kw:pkg_resources
kw:placement
kw:planning
kw:policy
kw:port
kw:portability
kw:portal
kw:posthook
kw:pratchett
kw:preformance
kw:preservation
kw:privacy
kw:process
kw:profile
kw:profiling
kw:progress
kw:proxy
kw:publish
kw:pyOpenSSL
kw:pyasn1
kw:pycparser
kw:pycrypto
kw:pycrypto-lib
kw:pycryptopp
kw:pyfilesystem
kw:pyflakes
kw:pylint
kw:pypi
kw:pypy
kw:pysqlite
kw:python
kw:python3
kw:pythonpath
kw:pyutil
kw:pywin32
kw:quickstart
kw:quiet
kw:quotas
kw:quoting
kw:raic
kw:rainhill
kw:random
kw:random-access
kw:range
kw:raspberry-pi
kw:reactor
kw:readonly
kw:rebalancing
kw:recovery
kw:recursive
kw:redhat
kw:redirect
kw:redressing
kw:refactor
kw:referer
kw:referrer
kw:regression
kw:rekey
kw:relay
kw:release
kw:release-blocker
kw:reliability
kw:relnotes
kw:remote
kw:removable
kw:removable-disk
kw:rename
kw:renew
kw:repair
kw:replace
kw:report
kw:repository
kw:research
kw:reserved_space
kw:response-needed
kw:response-time
kw:restore
kw:retrieve
kw:retry
kw:review
kw:review-needed
kw:reviewed
kw:revocation
kw:roadmap
kw:rollback
kw:rpm
kw:rsa
kw:rss
kw:rst
kw:rsync
kw:rusty
kw:s3
kw:s3-backend
kw:s3-frontend
kw:s4
kw:same-origin
kw:sandbox
kw:scalability
kw:scaling
kw:scheduling
kw:schema
kw:scheme
kw:scp
kw:scripts
kw:sdist
kw:sdmf
kw:security
kw:self-contained
kw:server
kw:servermap
kw:servers-of-happiness
kw:service
kw:setup
kw:setup.py
kw:setup_requires
kw:setuptools
kw:setuptools_darcs
kw:sftp
kw:shared
kw:shareset
kw:shell
kw:signals
kw:simultaneous
kw:six
kw:size
kw:slackware
kw:slashes
kw:smb
kw:sneakernet
kw:snowleopard
kw:socket
kw:solaris
kw:space
kw:space-efficiency
kw:spam
kw:spec
kw:speed
kw:sqlite
kw:ssh
kw:ssh-keygen
kw:sshfs
kw:ssl
kw:stability
kw:standards
kw:start
kw:startup
kw:static
kw:static-analysis
kw:statistics
kw:stats
kw:stats_gatherer
kw:status
kw:stdeb
kw:storage
kw:streaming
kw:strports
kw:style
kw:stylesheet
kw:subprocess
kw:sumo
kw:survey
kw:svg
kw:symlink
kw:synchronous
kw:tac
kw:tahoe-*
kw:tahoe-add-alias
kw:tahoe-admin
kw:tahoe-archive
kw:tahoe-backup
kw:tahoe-check
kw:tahoe-cp
kw:tahoe-create-alias
kw:tahoe-create-introducer
kw:tahoe-debug
kw:tahoe-deep-check
kw:tahoe-deepcheck
kw:tahoe-lafs-trac-stream
kw:tahoe-list-aliases
kw:tahoe-ls
kw:tahoe-magic-folder
kw:tahoe-manifest
kw:tahoe-mkdir
kw:tahoe-mount
kw:tahoe-mv
kw:tahoe-put
kw:tahoe-restart
kw:tahoe-rm
kw:tahoe-run
kw:tahoe-start
kw:tahoe-stats
kw:tahoe-unlink
kw:tahoe-webopen
kw:tahoe.css
kw:tahoe_files
kw:tahoewapi
kw:tarball
kw:tarballs
kw:tempfile
kw:templates
kw:terminology
kw:test
kw:test-and-set
kw:test-from-egg
kw:test-needed
kw:testgrid
kw:testing
kw:tests
kw:throttling
kw:ticket999-s3-backend
kw:tiddly
kw:time
kw:timeout
kw:timing
kw:to
kw:to-be-closed-on-2011-08-01
kw:tor
kw:tor-protocol
kw:torsocks
kw:tox
kw:trac
kw:transparency
kw:travis
kw:travis-ci
kw:trial
kw:trickle
kw:trivial
kw:truckee
kw:tub
kw:tub.location
kw:twine
kw:twistd
kw:twistd.log
kw:twisted
kw:twisted-14
kw:twisted-trial
kw:twitter
kw:twn
kw:txaws
kw:type
kw:typeerror
kw:ubuntu
kw:ucwe
kw:ueb
kw:ui
kw:unclean
kw:uncoordinated-writes
kw:undeletable
kw:unfinished-business
kw:unhandled-error
kw:unhappy
kw:unicode
kw:unit
kw:unix
kw:unlink
kw:update
kw:upgrade
kw:upload
kw:upload-helper
kw:uri
kw:url
kw:usability
kw:use-case
kw:utf-8
kw:util
kw:uwsgi
kw:ux
kw:validation
kw:variables
kw:vdrive
kw:verify
kw:verlib
kw:version
kw:versioning
kw:versions
kw:video
kw:virtualbox
kw:virtualenv
kw:vista
kw:visualization
kw:visualizer
kw:vm
kw:volunteergrid2
kw:volunteers
kw:vpn
kw:wapi
kw:warners-opinion-needed
kw:warning
kw:weapi
kw:web
kw:web.port
kw:webapi
kw:webdav
kw:webdrive
kw:webport
kw:websec
kw:website
kw:websocket
kw:welcome
kw:welcome-page
kw:welcomepage
kw:wiki
kw:win32
kw:win64
kw:windows
kw:windows-related
kw:winscp
kw:workaround
kw:world-domination
kw:wrapper
kw:write-enabler
kw:wui
kw:x86
kw:x86-64
kw:xhtml
kw:xml
kw:xss
kw:zbase32
kw:zetuptoolz
kw:zfec
kw:zookos-opinion-needed
kw:zope
kw:zope.interface
p/blocker
p/critical
p/major
p/minor
p/normal
p/supercritical
p/trivial
r/cannot reproduce
r/duplicate
r/fixed
r/invalid
r/somebody else's problem
r/was already fixed
r/wontfix
r/worksforme
t/defect
t/enhancement
t/task
v/0.2.0
v/0.3.0
v/0.4.0
v/0.5.0
v/0.5.1
v/0.6.0
v/0.6.1
v/0.7.0
v/0.8.0
v/0.9.0
v/1.0.0
v/1.1.0
v/1.10.0
v/1.10.1
v/1.10.2
v/1.10a2
v/1.11.0
v/1.12.0
v/1.12.1
v/1.13.0
v/1.14.0
v/1.15.0
v/1.15.1
v/1.2.0
v/1.3.0
v/1.4.1
v/1.5.0
v/1.6.0
v/1.6.1
v/1.7.0
v/1.7.1
v/1.7β
v/1.8.0
v/1.8.1
v/1.8.2
v/1.8.3
v/1.8β
v/1.9.0
v/1.9.0-s3branch
v/1.9.0a1
v/1.9.0a2
v/1.9.0b1
v/1.9.1
v/1.9.2
v/1.9.2a1
v/cloud-branch
v/unknown
No milestone
No project
No assignees
5 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: tahoe-lafs/trac#466
No description provided.