update for Ed25519, and fix a bug that would allow meet-in-the-middle attacks against the private exponent

[Imported from Trac: page NewMutableEncodingDesign, version 21]

NewMutableEncodingDesign.md

@@ -32,7 +32,7 @@ signatures use 1216-*byte* signing keys, 292-byte verifying keys, and
 
 The RSA fields are so large that we clearly cannot put them in the filecaps,
 so the encoding scheme requires them to be stored in the shares, encrypted
-and hashed as necessary. The ECDSA keys are short enough (in most cases) to put
+and hashed as necessary. The Ed25519 keys are short enough (in most cases) to put
 directly in the filecap, although note that this still increases the cap length
 relative to using a hash truncated to K+T bits.
 
@@ -84,7 +84,7 @@ Adding 2 metadata characters and a clear separator gives us:
  * once we get the ciphertext, it gets segmented and erasure-coded in the
    same way as immutable files. Shares include a merkle tree over the share
    blocks, and a second one over the ciphertext segments.
- * we'd like to add a merkle tree over the plaintext, without reintroducing
+ * we'd like to add a Merkle tree over the plaintext, without reintroducing
    the partial-information-guessing attack that prompted us to remove it.
    This means encrypting the nodes of this merkle tree with a key derived
    from the readcap.
@@ -106,7 +106,7 @@ Adding 2 metadata characters and a clear separator gives us:
    hash of the pubkey. The SI must be long enough to meet our
    collision-resistance criteria.
 
-## ECDSA, semi-private keys, no traversalcap
+## ECDSA or Ed25519, semi-private keys, no traversalcap
 
 Zooko captured the current leading semi-private-key-using mutable file design
 nicely in the ["StorageSS08" paper](http://allmydata.org/~zooko/lafs.pdf)
@@ -147,11 +147,12 @@ Without semi-private keys, we need something more complicated to protect the
 readkey: the only thing that can be mathematically derived from the writecap
 is the pubkey, and that can't be used to protect the data because it's public
 (and used by the server to validate shares). One approach is to use the
-current (discrete-log DSA) mutable file structure, and merely move the
-private key out of the share and into the writecap:
+current mutable file structure (with any signature algorithm), and merely
+move the private key out of the share and into the writecap:
 
- * (K + T) writecap = (K+T)-bit random string = privkey
- * (2K + T) readcap = H(writecap)[:K] + H(pubkey)[:K+T]
+ * (K + T) writecap = (K+T)-bit random string
+ * (2K) privkey = H(writecap)
+ * (2K + T) readcap = H(privkey)[:K] + H(pubkey)[:K+T]
  * (K + T) verifycap = H(pubkey)[:K+T]
  * storage-index = verifycap
 
@@ -163,31 +164,35 @@ on H(pubkey) to attain K-bit second-preimage resistance for 2^T^ targets. (To
 obtain collision-resistance, set T = K, although that shouldn't be necessary
 for mutable files.) The verifycap is K+T bits.
 
-### include ECDSA pubkey in cap
+### include ECDSA or Ed25519 pubkey in cap
 
 Or, if the pubkey is short enough, include it in the cap rather than
 requiring the client to fetch a copy:
 
- * (K + T) writecap = (K+T)-bit random string = privkey
- * (minimum 3K) readcap = H(writecap)[:K] + pubkey
+ * (K + T) writecap = (K+T)-bit random string
+ * (2K) privkey = H(writecap)
+ * (minimum 3K) readcap = H(privkey)[:K] + pubkey
  * (minimum 2K) verifycap = pubkey
  * storage-index = H(pubkey)
 
-ECDSA pubkeys are slightly more than 2*K long, so this would increase the
-length of the readcaps whenever K > T. The advantage would be simplifying/speeding up
-the download process. It is highly unlikely that there is any public key algorithm
+The hash to obtain the privkey is necessary because directly using a (K+T)-bit
+exponent would allow meet-in-the-middle attacks. ECDSA or Ed25519 pubkeys are
+slightly more than 2*K long, so this would increase the length of the readcaps
+because 2*K > T. The advantage would be simplifying/speeding up the download
+process. It is highly unlikely that there is any public key algorithm
 with keys shorter than 2*K for a K-bit security level. Since we can use shorter
 hashes than public keys, the H(pubkey) design above gives us shorter read caps,
 although they are not shorter than using semi-private keys.
 
 ### Any signature algorithm, no semi-private keys, with traversalcap
 
-Since a secure pubkey identifier (either H(pubkey)[:K+T] or the original privkey)
+Since a secure pubkey identifier (either H(pubkey)[:K+T] or the original writecap)
 is present in all caps, it's easy to insert arbitrary intermediate levels. It
 doesn't even change the way the existing caps are used:
 
- * (K + T) writecap = (K+T)-bit random string = privkey
- * (2K + T) readcap = H(writecap)[:K] + H(pubkey)[:K+T]
+ * (K + T) writecap = (K+T)-bit random string
+ * (2K) privkey = H(writecap)
+ * (2K + T) readcap = H(privkey)[:K] + H(pubkey)[:K+T]
  * (2K + T) traversalcap: H(readcap)[:K] + H(pubkey)[:K+T]
  * (K + T) verifycap = H(pubkey)[:K+T]
  * storage-index = verifycap