extendable Introducer protocol: dictionary-based, signed announcements #466
Labels
No labels
c/code
c/code-dirnodes
c/code-encoding
c/code-frontend
c/code-frontend-cli
c/code-frontend-ftp-sftp
c/code-frontend-magic-folder
c/code-frontend-web
c/code-mutable
c/code-network
c/code-nodeadmin
c/code-peerselection
c/code-storage
c/contrib
c/dev-infrastructure
c/docs
c/operational
c/packaging
c/unknown
c/website
kw:2pc
kw:410
kw:9p
kw:ActivePerl
kw:AttributeError
kw:DataUnavailable
kw:DeadReferenceError
kw:DoS
kw:FileZilla
kw:GetLastError
kw:IFinishableConsumer
kw:K
kw:LeastAuthority
kw:Makefile
kw:RIStorageServer
kw:StringIO
kw:UncoordinatedWriteError
kw:about
kw:access
kw:access-control
kw:accessibility
kw:accounting
kw:accounting-crawler
kw:add-only
kw:aes
kw:aesthetics
kw:alias
kw:aliases
kw:aliens
kw:allmydata
kw:amazon
kw:ambient
kw:annotations
kw:anonymity
kw:anonymous
kw:anti-censorship
kw:api_auth_token
kw:appearance
kw:appname
kw:apport
kw:archive
kw:archlinux
kw:argparse
kw:arm
kw:assertion
kw:attachment
kw:auth
kw:authentication
kw:automation
kw:avahi
kw:availability
kw:aws
kw:azure
kw:backend
kw:backoff
kw:backup
kw:backupdb
kw:backward-compatibility
kw:bandwidth
kw:basedir
kw:bayes
kw:bbfreeze
kw:beta
kw:binaries
kw:binutils
kw:bitcoin
kw:bitrot
kw:blacklist
kw:blocker
kw:blocks-cloud-deployment
kw:blocks-cloud-merge
kw:blocks-magic-folder-merge
kw:blocks-merge
kw:blocks-raic
kw:blocks-release
kw:blog
kw:bom
kw:bonjour
kw:branch
kw:branding
kw:breadcrumbs
kw:brians-opinion-needed
kw:browser
kw:bsd
kw:build
kw:build-helpers
kw:buildbot
kw:builders
kw:buildslave
kw:buildslaves
kw:cache
kw:cap
kw:capleak
kw:captcha
kw:cast
kw:centos
kw:cffi
kw:chacha
kw:charset
kw:check
kw:checker
kw:chroot
kw:ci
kw:clean
kw:cleanup
kw:cli
kw:cloud
kw:cloud-backend
kw:cmdline
kw:code
kw:code-checks
kw:coding-standards
kw:coding-tools
kw:coding_tools
kw:collection
kw:compatibility
kw:completion
kw:compression
kw:confidentiality
kw:config
kw:configuration
kw:configuration.txt
kw:conflict
kw:connection
kw:connectivity
kw:consistency
kw:content
kw:control
kw:control.furl
kw:convergence
kw:coordination
kw:copyright
kw:corruption
kw:cors
kw:cost
kw:coverage
kw:coveralls
kw:coveralls.io
kw:cpu-watcher
kw:cpyext
kw:crash
kw:crawler
kw:crawlers
kw:create-container
kw:cruft
kw:crypto
kw:cryptography
kw:cryptography-lib
kw:cryptopp
kw:csp
kw:curl
kw:cutoff-date
kw:cycle
kw:cygwin
kw:d3
kw:daemon
kw:darcs
kw:darcsver
kw:database
kw:dataloss
kw:db
kw:dead-code
kw:deb
kw:debian
kw:debug
kw:deep-check
kw:defaults
kw:deferred
kw:delete
kw:deletion
kw:denial-of-service
kw:dependency
kw:deployment
kw:deprecation
kw:desert-island
kw:desert-island-build
kw:design
kw:design-review-needed
kw:detection
kw:dev-infrastructure
kw:devpay
kw:directory
kw:directory-page
kw:dirnode
kw:dirnodes
kw:disconnect
kw:discovery
kw:disk
kw:disk-backend
kw:distribute
kw:distutils
kw:dns
kw:do_http
kw:doc-needed
kw:docker
kw:docs
kw:docs-needed
kw:dokan
kw:dos
kw:download
kw:downloader
kw:dragonfly
kw:drop-upload
kw:duplicity
kw:dusty
kw:earth-dragon
kw:easy
kw:ec2
kw:ecdsa
kw:ed25519
kw:egg-needed
kw:eggs
kw:eliot
kw:email
kw:empty
kw:encoding
kw:endpoint
kw:enterprise
kw:enum34
kw:environment
kw:erasure
kw:erasure-coding
kw:error
kw:escaping
kw:etag
kw:etch
kw:evangelism
kw:eventual
kw:example
kw:excess-authority
kw:exec
kw:exocet
kw:expiration
kw:extensibility
kw:extension
kw:failure
kw:fedora
kw:ffp
kw:fhs
kw:figleaf
kw:file
kw:file-descriptor
kw:filename
kw:filesystem
kw:fileutil
kw:fips
kw:firewall
kw:first
kw:floatingpoint
kw:flog
kw:foolscap
kw:forward-compatibility
kw:forward-secrecy
kw:forwarding
kw:free
kw:freebsd
kw:frontend
kw:fsevents
kw:ftp
kw:ftpd
kw:full
kw:furl
kw:fuse
kw:garbage
kw:garbage-collection
kw:gateway
kw:gatherer
kw:gc
kw:gcc
kw:gentoo
kw:get
kw:git
kw:git-annex
kw:github
kw:glacier
kw:globalcaps
kw:glossary
kw:google-cloud-storage
kw:google-drive-backend
kw:gossip
kw:governance
kw:grid
kw:grid-manager
kw:gridid
kw:gridsync
kw:grsec
kw:gsoc
kw:gvfs
kw:hackfest
kw:hacktahoe
kw:hang
kw:hardlink
kw:heartbleed
kw:heisenbug
kw:help
kw:helper
kw:hint
kw:hooks
kw:how
kw:how-to
kw:howto
kw:hp
kw:hp-cloud
kw:html
kw:http
kw:https
kw:i18n
kw:i2p
kw:i2p-collab
kw:illustration
kw:image
kw:immutable
kw:impressions
kw:incentives
kw:incident
kw:init
kw:inlineCallbacks
kw:inotify
kw:install
kw:installer
kw:integration
kw:integration-test
kw:integrity
kw:interactive
kw:interface
kw:interfaces
kw:interoperability
kw:interstellar-exploration
kw:introducer
kw:introduction
kw:iphone
kw:ipkg
kw:iputil
kw:ipv6
kw:irc
kw:jail
kw:javascript
kw:joke
kw:jquery
kw:json
kw:jsui
kw:junk
kw:key-value-store
kw:kfreebsd
kw:known-issue
kw:konqueror
kw:kpreid
kw:kvm
kw:l10n
kw:lae
kw:large
kw:latency
kw:leak
kw:leasedb
kw:leases
kw:libgmp
kw:license
kw:licenss
kw:linecount
kw:link
kw:linux
kw:lit
kw:localhost
kw:location
kw:locking
kw:logging
kw:logo
kw:loopback
kw:lucid
kw:mac
kw:macintosh
kw:magic-folder
kw:manhole
kw:manifest
kw:manual-test-needed
kw:map
kw:mapupdate
kw:max_space
kw:mdmf
kw:memcheck
kw:memory
kw:memory-leak
kw:mesh
kw:metadata
kw:meter
kw:migration
kw:mime
kw:mingw
kw:minimal
kw:misc
kw:miscapture
kw:mlp
kw:mock
kw:more-info-needed
kw:mountain-lion
kw:move
kw:multi-users
kw:multiple
kw:multiuser-gateway
kw:munin
kw:music
kw:mutability
kw:mutable
kw:mystery
kw:names
kw:naming
kw:nas
kw:navigation
kw:needs-review
kw:needs-spawn
kw:netbsd
kw:network
kw:nevow
kw:new-user
kw:newcaps
kw:news
kw:news-done
kw:news-needed
kw:newsletter
kw:newurls
kw:nfc
kw:nginx
kw:nixos
kw:no-clobber
kw:node
kw:node-url
kw:notification
kw:notifyOnDisconnect
kw:nsa310
kw:nsa320
kw:nsa325
kw:numpy
kw:objects
kw:old
kw:openbsd
kw:openitp-packaging
kw:openssl
kw:openstack
kw:opensuse
kw:operation-helpers
kw:operational
kw:operations
kw:ophandle
kw:ophandles
kw:ops
kw:optimization
kw:optional
kw:options
kw:organization
kw:os
kw:os.abort
kw:ostrom
kw:osx
kw:osxfuse
kw:otf-magic-folder-objective1
kw:otf-magic-folder-objective2
kw:otf-magic-folder-objective3
kw:otf-magic-folder-objective4
kw:otf-magic-folder-objective5
kw:otf-magic-folder-objective6
kw:p2p
kw:packaging
kw:partial
kw:password
kw:path
kw:paths
kw:pause
kw:peer-selection
kw:performance
kw:permalink
kw:permissions
kw:persistence
kw:phone
kw:pickle
kw:pip
kw:pipermail
kw:pkg_resources
kw:placement
kw:planning
kw:policy
kw:port
kw:portability
kw:portal
kw:posthook
kw:pratchett
kw:preformance
kw:preservation
kw:privacy
kw:process
kw:profile
kw:profiling
kw:progress
kw:proxy
kw:publish
kw:pyOpenSSL
kw:pyasn1
kw:pycparser
kw:pycrypto
kw:pycrypto-lib
kw:pycryptopp
kw:pyfilesystem
kw:pyflakes
kw:pylint
kw:pypi
kw:pypy
kw:pysqlite
kw:python
kw:python3
kw:pythonpath
kw:pyutil
kw:pywin32
kw:quickstart
kw:quiet
kw:quotas
kw:quoting
kw:raic
kw:rainhill
kw:random
kw:random-access
kw:range
kw:raspberry-pi
kw:reactor
kw:readonly
kw:rebalancing
kw:recovery
kw:recursive
kw:redhat
kw:redirect
kw:redressing
kw:refactor
kw:referer
kw:referrer
kw:regression
kw:rekey
kw:relay
kw:release
kw:release-blocker
kw:reliability
kw:relnotes
kw:remote
kw:removable
kw:removable-disk
kw:rename
kw:renew
kw:repair
kw:replace
kw:report
kw:repository
kw:research
kw:reserved_space
kw:response-needed
kw:response-time
kw:restore
kw:retrieve
kw:retry
kw:review
kw:review-needed
kw:reviewed
kw:revocation
kw:roadmap
kw:rollback
kw:rpm
kw:rsa
kw:rss
kw:rst
kw:rsync
kw:rusty
kw:s3
kw:s3-backend
kw:s3-frontend
kw:s4
kw:same-origin
kw:sandbox
kw:scalability
kw:scaling
kw:scheduling
kw:schema
kw:scheme
kw:scp
kw:scripts
kw:sdist
kw:sdmf
kw:security
kw:self-contained
kw:server
kw:servermap
kw:servers-of-happiness
kw:service
kw:setup
kw:setup.py
kw:setup_requires
kw:setuptools
kw:setuptools_darcs
kw:sftp
kw:shared
kw:shareset
kw:shell
kw:signals
kw:simultaneous
kw:six
kw:size
kw:slackware
kw:slashes
kw:smb
kw:sneakernet
kw:snowleopard
kw:socket
kw:solaris
kw:space
kw:space-efficiency
kw:spam
kw:spec
kw:speed
kw:sqlite
kw:ssh
kw:ssh-keygen
kw:sshfs
kw:ssl
kw:stability
kw:standards
kw:start
kw:startup
kw:static
kw:static-analysis
kw:statistics
kw:stats
kw:stats_gatherer
kw:status
kw:stdeb
kw:storage
kw:streaming
kw:strports
kw:style
kw:stylesheet
kw:subprocess
kw:sumo
kw:survey
kw:svg
kw:symlink
kw:synchronous
kw:tac
kw:tahoe-*
kw:tahoe-add-alias
kw:tahoe-admin
kw:tahoe-archive
kw:tahoe-backup
kw:tahoe-check
kw:tahoe-cp
kw:tahoe-create-alias
kw:tahoe-create-introducer
kw:tahoe-debug
kw:tahoe-deep-check
kw:tahoe-deepcheck
kw:tahoe-lafs-trac-stream
kw:tahoe-list-aliases
kw:tahoe-ls
kw:tahoe-magic-folder
kw:tahoe-manifest
kw:tahoe-mkdir
kw:tahoe-mount
kw:tahoe-mv
kw:tahoe-put
kw:tahoe-restart
kw:tahoe-rm
kw:tahoe-run
kw:tahoe-start
kw:tahoe-stats
kw:tahoe-unlink
kw:tahoe-webopen
kw:tahoe.css
kw:tahoe_files
kw:tahoewapi
kw:tarball
kw:tarballs
kw:tempfile
kw:templates
kw:terminology
kw:test
kw:test-and-set
kw:test-from-egg
kw:test-needed
kw:testgrid
kw:testing
kw:tests
kw:throttling
kw:ticket999-s3-backend
kw:tiddly
kw:time
kw:timeout
kw:timing
kw:to
kw:to-be-closed-on-2011-08-01
kw:tor
kw:tor-protocol
kw:torsocks
kw:tox
kw:trac
kw:transparency
kw:travis
kw:travis-ci
kw:trial
kw:trickle
kw:trivial
kw:truckee
kw:tub
kw:tub.location
kw:twine
kw:twistd
kw:twistd.log
kw:twisted
kw:twisted-14
kw:twisted-trial
kw:twitter
kw:twn
kw:txaws
kw:type
kw:typeerror
kw:ubuntu
kw:ucwe
kw:ueb
kw:ui
kw:unclean
kw:uncoordinated-writes
kw:undeletable
kw:unfinished-business
kw:unhandled-error
kw:unhappy
kw:unicode
kw:unit
kw:unix
kw:unlink
kw:update
kw:upgrade
kw:upload
kw:upload-helper
kw:uri
kw:url
kw:usability
kw:use-case
kw:utf-8
kw:util
kw:uwsgi
kw:ux
kw:validation
kw:variables
kw:vdrive
kw:verify
kw:verlib
kw:version
kw:versioning
kw:versions
kw:video
kw:virtualbox
kw:virtualenv
kw:vista
kw:visualization
kw:visualizer
kw:vm
kw:volunteergrid2
kw:volunteers
kw:vpn
kw:wapi
kw:warners-opinion-needed
kw:warning
kw:weapi
kw:web
kw:web.port
kw:webapi
kw:webdav
kw:webdrive
kw:webport
kw:websec
kw:website
kw:websocket
kw:welcome
kw:welcome-page
kw:welcomepage
kw:wiki
kw:win32
kw:win64
kw:windows
kw:windows-related
kw:winscp
kw:workaround
kw:world-domination
kw:wrapper
kw:write-enabler
kw:wui
kw:x86
kw:x86-64
kw:xhtml
kw:xml
kw:xss
kw:zbase32
kw:zetuptoolz
kw:zfec
kw:zookos-opinion-needed
kw:zope
kw:zope.interface
p/blocker
p/critical
p/major
p/minor
p/normal
p/supercritical
p/trivial
r/cannot reproduce
r/duplicate
r/fixed
r/invalid
r/somebody else's problem
r/was already fixed
r/wontfix
r/worksforme
t/defect
t/enhancement
t/task
v/0.2.0
v/0.3.0
v/0.4.0
v/0.5.0
v/0.5.1
v/0.6.0
v/0.6.1
v/0.7.0
v/0.8.0
v/0.9.0
v/1.0.0
v/1.1.0
v/1.10.0
v/1.10.1
v/1.10.2
v/1.10a2
v/1.11.0
v/1.12.0
v/1.12.1
v/1.13.0
v/1.14.0
v/1.15.0
v/1.15.1
v/1.2.0
v/1.3.0
v/1.4.1
v/1.5.0
v/1.6.0
v/1.6.1
v/1.7.0
v/1.7.1
v/1.7β
v/1.8.0
v/1.8.1
v/1.8.2
v/1.8.3
v/1.8β
v/1.9.0
v/1.9.0-s3branch
v/1.9.0a1
v/1.9.0a2
v/1.9.0b1
v/1.9.1
v/1.9.2
v/1.9.2a1
v/cloud-branch
v/unknown
No milestone
No project
No assignees
5 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: tahoe-lafs/trac#466
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Zooko and I have discussed a new API for the Introducer that would be more
extensible for future uses. We need three features out of the new design:
the announcements without affecting older subscribers who won't understand
them
to allow clients to restrict which storage servers they'll use (they can
specify a public key, and only pay attention to announcements that were
signed with the corresponding private key)
leaving the old methods in place.
This is one of the pre-requisites for implementing Accounting, since when we
require clients to use rights-amplification to obtain a reference to an
attenuated/labeled storage server object (instead of the current scheme in
which everybody gets access to a full-powered server reference), we'll be
announcing the "login" facet via a different interface name.
We're also planning on using the introducer to allow storage servers to
update their announcement when they get full (or when they free enough space
to resume accepting shares). This is a performance optimization: if a server
is going to reject all shares, it is faster to remove it from the list of
available writers than to ask and be rejected for each upload. The server
should remain available for reading, though. We're not yet sure how we'll do
this:
We should keep the idea of "updating your earlier announcement" in mind as we
implement this improvement.
More design notes:
probably some forms of extendability), so a variable-length list of
certificates.
announcements and storage authority, so I'm kind of front-loading the
design here.
pubkey-identifier).
dictionary (but of course then we must base64-encode any binary strings,
since JSON can only carry unicode strings). Note that the encoded-message
is a single string, and is generally left in encoded form until someone
needs to know the contents. The only time encoding takes place is just
before a signature is created. The encoded form is the canonical form.
some public key. This is a single string. If the cert is unsigned, this
will be a 0-length string.
This helps a verifier figure out which pubkey was used to compute the
signature. If omitted, the verifier must attempt to verify the signature
against all root certs. If present, it is a strict prefix of the full
serialized pubkey. Since we don't expect to have more than a handful of
root certs, let's say this should be 4 bytes long.:
get the whole certificate chain down to a single string. The goals for
that use case are:
give up generality in deference to brevity
comfortably in a URL's query argument)
(so that we know how to create its signature), but we also want
flexibility for what goes into the encoded message, hence the JSON. For
storage authority cert chains, we might not actually want that
flexibility: if the client doesn't recognize one of the attenuations,
it should fail-safe by rejecting the cert. There are a couple of
competing goals here, and I'm not sure what the best answer is.
Versioning: the announcer API (possibly called
publish_v2
) willimplicitly define the version of the certificate chain:
publish_v2
isalways called with a chain in a specific format. If we switch to different
signature schemes (perhaps a different key length) in the future, they can
use a new API call (
publish_v3
) which will include additional argumentsas necessary, or which will deliver certs with a different format (possibly
including additional data, like which signature scheme is in use).
This technique can be generalized: clients can attempt to call
methname_vNN
for the highest ```NN}} they know, if they get aNameError
then they can fall back to older interfaces (or fail), andthey should remember which NN worked so they can avoid repeating the failure
again the next time.
When the cert chain is serialized into a single string (for storage
authority), we'll need to put a magic number and version identifier at the
beginning of that string.
Actually, this is not really a prerequisite for accounting. We can just advertise "storage-login" instead of "storage" through the existing tuple-based interface.
This ticket is really related to "blessed storage servers", which are about protecting the client against low-reliability servers. Accounting is about protecting the servers against unauthorized clients.
So we can put this one off for a while.
Is this a duplicate of #295?
not exactly, let's say that #295 is about distributed introduction, and this one (#466) is about signed/extendable announcements. I've updated the summary on #295 to match.
The code for this is pretty much done: my sandbox has a tree which does everything I want it to do, except for:
I'm holding off on pushing it into trunk, though, for two reasons:
I'll attach a diff with my changes, for safekeeping.
So this ticket is blocked on #331, and can be pushed to trunk (and then, barring problems, closed) shortly after #331 is finished and a new version of pycryptopp is released.
Just an update.. my patch for this will have bitrotted in the last 5 months.. enough trunk code has changed that it will no longer apply cleanly. I estimate it would take me about 3 or 4 days to bring it back up-to-date.
It is still quite blocked on ECDSA (#331). I can do about half the updating work without it (I'd have to create a fake ecdsa module and guess at what the eventual API will be).
Ok, that -ver6 patch is worth reviewing. I think there are a few more items I
want to add before landing (more tests, mainly), but it's certainly worth
talking about.
This version drops the certchain that was in the previous one: each
announcement is either signed with a single pubkey, or not signed at all. It
uses an embedded copy of my python-ecdsa library
(https://github.com/warner/python-ecdsa), which is a bit on the slow side,
but still fast enough for announcement/introducer use. It adds code to the
IntroducerClient
to add, distribute, and verify signatures, but doesnot add any code to the
Client
orStorageBroker
to use thosefeatures: actual UI/tahoe.cfg switches to enable signing or require
signatures is left for a future patch.
The patch creates a new version of the Introducer, as well as its protocols.
V2 servers (i.e. Introducers) can accept connections from either V1 or V2
clients, and V2 clients can tolerate talking to a V1 server, so all
combinations are covered. Signatures can only be passed from a V2 client,
through a V2 server, off to another V2 client: any V1 components along the
way will lose the signature.
Each announcement is a dictionary with keys to replace everything that was in
the V1 protocol's tuples. I added some more version information (the full
app-versions dict), which may be too much (especially if you're trying to be
anonymous).
The quirkiest thing about this scheme is the relationship between FURL tubids
and ECDSA pubkeys. The Introducer is supposed to recognize multiple
announcements from the same source and let the new one replace the old one.
When both are V1 tuples with the same tubid in their FURLs, or when both are
V2 dicts signed by the same pubkey, the relationship is easy. If a client is
upgraded from V1 to V2 and starts signing its announcements, the V2
announcement won't replace its old V1 announcement. I'm not sure how to
handle this yet.
In addition, we need to think about how/if we want to transition serverids
from being FURL-based to being pubkey-based. Since serverids are baked into
shares (both via the permuted serverlist and, more directly, by the
Write-Enablers embedded in each mutable share), we can't just casually change
an existing server's id. For brand-new servers, we could switch to using the
pubkey as the serverid: this might make non-Foolscap-based share-transport
protocols easier to secure. For old servers with pre-existing shares that
start signing their announcements, we should probably keep using their tubid
as a serverid, perhaps even after we switch away from foolscap and to some
ECDSA-signature based protocol. But we must validate it: a bad server could
publish a FURL that they don't actually control, with a tubid that matches
the server they wish to impersonate, knowing that we'll never actually
connect to the FURL and discover the problem. I think the validation protocol
will involve connecting to the FURL and receiving a copy of the pubkey back,
to prove that the owner of the FURL really does want to be associated with
that pubkey.
oops, forgot to set the flag
I think I've got an idea to clean up the "what is your serverid anyway?"
issue. For reference, this is the problem:
a server which is signing its Announcements holds two secrets:
TubID
keyid that shows up in the signed announcements
For many aspects of Tahoe's share-placement behavior, we need a reliable,
serverlist permutation
write-enabler secrets
consistent server reliability (if you think your share is safe because
it's on server A, but someone else was able to make you put the share on
server B instead, then you aren't getting this. Note that server A is
allowed to unilaterally delegate its storage duties: the failure mode is
if someone other than you and/or server A is able to make your share go
elsewhere).
and eventually accounting stuff
To be unspoofable, the serverid needs to be the public half of some
So today's thought is:
'
serverid
' and 'serverid-type
'StorageFarmBroker
(which subscribes to hear about storageserverid
against the keyid which was used to sign the announcement.(
IntroducerClient
already checks signatures and discards invalidones, so the
subscribe
interface's callback is invoked with a keyidand an announcement dictionary).
clients will connect to whatever FURL is in the announcement and won't
pay attention to that FURL's tubid: they will strictly use the pubkey as
a serverid.
TubID
of the claimed FURL against the announcement'sserverid
, abort if they do not matchget_pubkey
"announcement
serverid
So when a
get_announcement
method returns a pubkey, that Tub isdelegating all its authority to the signing key. (this authority includes the
right to know the write-enablers, and the right to decide where its shares
are placed).
The first check (
FURL.TubID
vsserverid
is nominally redundant(why not just extract the tubid and use it as the serverid), but I expect it
will be convenient to keep a list of validated announcements around, and we'd
like to be able to trust the
serverid
fields therein. It also providesuseful non-validated data in some places (like the Introducer's web status
page).
The second check (
get_pubkey()==serverid
) blocks a bunch of noise:bogus announcements made by someone other than the Tub operator, but pointing
at that Tub. These announcements could contain false information about server
capabilities, free space, versioning, etc (everything in the announcement
dictionary). By making sure that the Tub in question is actually willing to
be represented (in announcements) by that pubkey, we prevent anyone else from
making these false claims.
The legacy authority we're trying to protect here is the tubid-based
serverid. By restricting that authority to servers who can actually provide
an object at that Tub, we prevent other parties from being able to claim that
authority.
It's still an open question in my mind whether this transition is a good idea
overall. I think it will help Tahoe if we can write clients in languages
other than Python, and relying upon Foolscap is a barrier to that (we could
port Foolscap to other languages, but that feels like more work for less gain
than changing Tahoe). If that's a good direction to go, then switching to a
serverid based on some cryptographic quality other than a foolscap tubid is
a necessary step.
I just worry about about the existing servers and the long-term baggage that
they'll need to carry around (since all their mutable shares are tied to the
serverid, they'll need to keep a Tub around forever, to prove knowledge of
the Tub private key, even if all the connections are using ECDSA-signed HTTP
messages or whatnot). (incidentally, this might be relaxed somewhat if
foolscap#19 were implemented,
allowing the SSL key to be used for signing arbitrary data, because then we
could publish the public cert and its signature on the ECDSA pubkey in the
announcement. Both sides would still need to do some SSL work to verify the
signature, but they wouldn't strictly need to use Foolscap to do it, nor
would they need to use foolscap connections to move shares around.
I've begun a review of this ticket. Some initial questions from the first design comment follow. I may answer some of these by reading the patch and the trunk version.
Q1. Why does the design specify EC-DSA-192? What are the requirements which drive this algorithm selection?
Zooko suggested on IRC that the primary goal is small public keys.
Q2. Why is the pubkey-identifier optional? What use case does this facilitate?
Q3. If the identifier is absent, verification is checked against all "root certs". How are these managed?
nejucomo: please start your review with the latest patch (466-ver6.diff). The design has changed a lot since this ticket was first opened, and most of those questions are no longer applicable.
Attachment 2011-02-p1.diff (100975 bytes) added
copy python-ecdsa-0.7 into tree as allmydata.util.ecdsa
Attachment 2011-02-p2.diff (9476 bytes) added
add 'tahoe admin generate-keypair/derive-pubkey' commands, apply on top of p1
Attachment 2011-02-p3.diff (111773 bytes) added
new Introducer. see https://github.com/warner/tahoe-lafs/tree/466-introducer-take2 for updates
State of the patch
I'd say this project is about 80% complete. The patch currently attached
for review is split up into three pieces. This note is to explain what
those pieces do, what the overall design is like, and what's left to
design/build.
Why Do We Want This?
The goal of this ticket is add signatures to our introducer
announcements, and to make them extensible. Current announcements are a
fixed 6-tuple: (FURL, service_name, remote_interface_name, nickname,
version, oldest_supported). The new announcements will be an arbitrary
JSON-serializable dictionary, embedded in a 3-tuple of (ann_json,
signature, pubkey), where the sig/pubkey are optional.
The utility of extensibility is pretty obvious: there are additional
services and features we could enable (or make more efficient) if we
could safely advertise them ahead of time through the introducer. In
general, extensible protocol formats (dicts, not tuples) improves
flexibility and enables change, since it's awfully difficult to make
changes to all nodes simultaneously. Flexible announcements aren't
strictly necessary: we could instead e.g. add new methods to the
RIStorageServer object, and have clients speculatively attempt to invoke
them, and gently tolerate NameErrors, but that seems inelegant, and
requires a roundtrip: by putting slowly-changing things in the
announcements, clients learn about them earlier.
The value of adding signatures is great, but not immediate. Signatures
would bind the contents of an announcement to some "serverid",
preventing other parties (other grid members, or the Introducer itself)
from forging those contents. This would turn the introduction system
into a secure channel from publisher to subscriber, indexed by serverid.
Without signatures, anybody in the grid can publish anything they like,
such as a record with Alice's real storage-server FURL but with a
nickname of "Bob", which would currently replace Alice's real
announcement.
Currently we use Foolscap Tub IDs (i.e. hash of the tub certificate,
which appears in each FURL) as server IDs. The only way to verify
possession of the corresponding secret is to connect to a FURL that uses
this tubid: Foolscap ensures that the object you connect to (and
subsequent send callRemote messages to) is selected by the
secret-holder. This requires an online check, whereas a signed message
could be verified offline.
We currently use serverids for three things:
separately, sending separate shares to each
how to distribute shares of each file
"write-enabler", and the renew/expire lease tokens. These need to be
different for each server, so that when a client exercises their
authority on server A, that doesn't enable A to exercise the client's
corresponding authority on server B
In the future, we would like to also use serverids to:
that uploads will use, ignoring all others, to protect the user's
upstream bandwidth and reliability choices. This selection could be
delegated to a central party, allowing the server list to change over
time without constant user involvement.
I think that a fully distributed introducer requires announcement
signatures. With a single central Introducer, we could achieve some
measure of control over the grid by restricting publishing access to
certain servers. But with a highly distributed log-flood -based
introduction system, we'd give up central control, and I think
individually-traceable announcements would make up for that loss.
Finally, a long-term goal is to move away from Foolscap to an HTTP-based
protocol that is easier to implement in non-Python languages, to
facilitate multiple implementations of the Tahoe protocol. Without
Foolscap, we'll need a different mechanism to securely identify a
server, for which the announcement signing keypair is appropriate.
patch 1: python-ecdsa
https://github.com/warner/python-ecdsa/ is where I maintain a
pure-python ECDSA library, with a not-too-bad API. I think I may want to
make some changes to the API still. It's fast enough for use by signed
announcements, since sign/verify operations occur only once per server.
Once pycryptopp acquires ECDSA support, we should move to that, for the
30x speedup.
The library is embedded into
allmydata/src/util/ecdsa/*.py
, ratherthan being added as a dependency, for two reasons. First, extra
dependencies are really making packagers lives difficult, slowing
packaging efforts and thus hurting adoption. Second, changes to the
upstream API will be easier to accomodate by using a fixed version of
python-ecdsa, so I think it makes sense to copy it wholesale into the
tahoe tree until the API stabilizes (which needs to be driven by using
it and learning what works and what doesn't).
patch 2: keypair generation
This adds
tahoe admin generate-keypair
andderive-pubkey
,basic userspace tools to work with keys. The basic idea is that some day
you might use them to creates a value that you then paste into a
tahoe.cfg file. They are currently unused.
patch 3: everything else
I'll split this into:
Terminology
The
IntroducerServer
is an object that lives in the introducerprocess, the one (just one, so far, but #68 will change that) identified
by the
introducer.furl
. Most of this note calls this the "server".The
IntroducerClient
is an object that lives in eachnon-introducer process, both tahoe storage servers and tahoe clients,
which manages the connection to the
IntroducerServer
.The "publisher" is a Tahoe node (usually a storage server) which wants
to broadcast information about themselves to the whole grid. Each time
they do this, they are said to "announce" their information. The bundle
of information is called an "announcement".
The "subscriber" is a Tahoe node (usually a client/gateway) that wants
to receive announcements.
Subscribers call their local
IntroducerClient.subscribe_to
to signup to hear announcements, and provide a callback that will be invoked
multiple times as they arrive. This provokes the
IntroducerClient
to send a "subscribe" message to the server. Later, the server will send
an "announce" message to the client, and the
IntroducerClient
willfire the callback.
Publishers call
IntroducerClient.publish
to deliver announcements,which provokes the
IntroducerClient
to send a "publish" message tothe server, which provokes the server to send "announce" messages to all
interested clients.
V2 Introducer Protocol
The old V1 protocol used 6-tuples as announcements: (FURL, service_name,
remoteinterface_name, nickname, my_version, oldest_supported). The new
V2 protocol uses an open-ended JSON-serializable dictionary, with a
number of top-level keys that are expected to be present.
The V2 client->server "publish" message adds a "canary" argument, which
allows the server to detect when the publisher has disconnected. This is
unused so far, but the intent is to display liveness status on the
server's "introweb" page, and to let them stop publishing data for
servers which are offline (or perhaps have remained offline for several
days).
The V2 client->server "subscribe" message adds a "subscriber_info"
argument, which lets the server's introweb page show information about
each subscriber (mostly nickname and version). In the V1 protocol, this
was accomplished by having each subscriber also "announce" a special
"stub_client" service, which didn't correspond to a real service, but
included enough information to build the status display. The V2
subscriber_info field is defined (by foolscap schema) to be a dictionary
with string keys, with at least "nickname", "my-version",
"app-versions", and "oldest-supported".
Announcement Signatures
V2 announcements on the wire are a 3-tuple (ann_d_json, sig_hex,
pubkey_hex), in which the announcement is serialized with JSON. Unsigned
announcements have
sig_hex == pubkey_hex == None
.We use 256-bit ECDSA keys (from the NIST256p curve, since that seemed to
be the most widely implemented, in openssl/nss).
pubkey_hex
is abase16-encoded uncompressed raw binary key (TODO: versioning), the
output of
VerifyingKey.to_string().encode("hex")
. This is 512 bitslong, and neither includes an OID nor a 0x04 "uncompressed" flag byte.
Signatures are computed with SHA1 (TODO: given NIST256p, let's use
SHA256), with an algorithm that is compatible with openssl (verified by
the python-ecdsa test suite).
sig_hex
is the output ofSigningKey.sign(ann_d_json.encode("utf-8")).encode("hex")
whichuses python-ecdsa's minimal binary string encoding (no versioning
information).
Publishers give their
IntroducerClient.publish
both anannouncement dictionary and a
SigningKey
instance (or None).Subscribers receive an announcement dictionary and a
VerifyfingKey
instance (which will be None if the announcement did not have a matching
valid signature, which includes both unsigned announcements,
forged/invalid signatures, and valid signatures from some different
pubkey).
Backwards Compatibility with V1
There are four interesting V1+V2 compatibility cases, two on the
publishing half, and two on the subscribing/announcement half.
The V2 IntroducerServer provides all the same method names as the V1
server, plus additional announce_v2/subscribe_v2 methods that are only
used by V2 clients. On the V1 methods, the V2 server accepts the same
message format as the V1 server. The server seeks to hide the client
versions from each other: a V1 client receives only V1-format
announcements, and a V2 client receives only V2-format announcements,
regardless of what client version generated those announcements.
When a V1 client publishes to a V2 server, it uses the old "publish"
method name, allowing the server to detect the client's old version. The
server upconverts the V1-format announcement tuple into an unsigned
V2-format dictionary, leaving some fields empty (like
ann_d["app_versions"]={
}) when necessary. It then dispatches thisV2-format announcement internally as if it was received from a real V2
client. When a V1 client subscribes to a V2 server, the old server-side
"subscribe" method wraps the remote reference in a !SubscriberAdapter_v1,
which behaves just like a remote reference to a modern V2 subscriber,
but downconverts the messages to old-style V1 tuples before sending them
over the wire.
When a V2 client tries to publish an announcement, it first tries to
invoke the new "publish_v2" method, with a V2-style announcement
dictionary (maybe signed). If this callRemote fails in a way that looks
like the server does not implement publish_v2, the client concludes that
it is dealing with a V1 server. It then downconverts its annoucement to
V1-style and sends it to the old V1 "publish" method.
When a V2 client wants to subscribe, it first tries the new
"subscribe_v2" method, passing itself as the desired recipient of
"announce_v2" messages (the remote callback, in a sense). If the
"subscribe_v2" method fails, the client concludes that it's dealing with
a V1 server, and calls the old V1 "subscribe" method instead, passing a
different object that can accept V1-style announcements. This
client-side object upconverts each V1 announcement into a V2-format
dictionary before internal delivery. The client also publishes a
"stub_client" announcement, so that the V1 server can display nicknames
and version numbers of all subscribed clients.
=== Server ID Computation ===
This is the trickiest part, and ties into our eventual goals for having
signed announcements.
In the ideal future world, as we envision it today, we no longer use
Foolscap, have no Tub IDs, and use HTTP to send signed unencrypted
storage-protocol messages from client to server. In that world, the
ECDSA pubkey is the serverid, and clients use signed announcements to
learn genuine information about servers.
But since we've been using foolscap tubids as serverids, and since we've
been using serverids to compute both serverlist-permutation and
shared-secrets, there is a compatibility concern. Any shares we've
uploaded to tubid-based servers must continue to be accessible with the
old serverid. I think this means that, even when we add an ECDSA pubkey
to those servers, they need to continue to use their tubid as serverid,
and we need to allow an announcement signed with pubkey-A to correctly
claim to have a serverid tubid-B. This will require a verification step,
in which the client connects to the tubid-B-bearing FURL, and the server
is expected to announce (over that channel) that it uses pubkey-A. Once
that is accomplished, the client can believe other metadata included in
the signed announcement.
This feels complex, so I'm uneasy about it, but here's the protocol I
have in mind:
and serverid is the ASCII pubkey
base32-encoded tubid
"getAnnouncement" message, response is a new announcement (hopefully
identical to the one from the Introducer), assert that serverid in
new ann matches furl, then accept new announcement (ignoring the
original).
== remaining work ==
permutation hash. Current serverids are binary. Make new ones ASCII?
protocols (i.e. stick to ASCII)
message, in anticipation of switching the Introducer protocol to HTTP?
(i.e. design the HTTP protocol first, then map it to Foolscap)
Brian's opinion needed on whether this will be ready for 1.9.
sigh, no. I grow increasingly less hopeful that this will ever see the light of day. Bumping out of 1.9
ok, we're getting close here. The current work is on my github branch:
https://github.com/warner/tahoe-lafs/tree/466-take8
(note: I rebase this branch frequently. Also, I switch to subsequent "takeNN" branches when the need arises)
We went (quickly) over the design and code at the Summit, and I fixed all the issues that were raised there. So the branch is ready to land once the necessary dependencies are in place. The sequence from here is:
_autodeps.py
to depend on the new pycryptopp466-take8
branchA darcs patch against trunk corresponding to the git branch is at https://tahoe-lafs.org/~davidsarah/patches/introducer-signed-messages.darcs.patch.
I made some minor changes to the code and tests for the
tahoe admin
commands (see the patch descriptions), and fixed some duplicate umids; apart from that the code in the darcs patch is the same. I confirmed that it passes tests on Ubuntu Maverick x86-64, provided that pycryptopp with the https://tahoe-lafs.org/trac/pycryptopp/ticket/75 patch is installed, and modulo some failures due to #1586.At warner's request, I rerecorded the darcs patches so that my changes are in separate patches to the changes from the git branch. They are at https://tahoe-lafs.org/~davidsarah/patches/introducer-signed-messages-v2.darcs.patch.
Note that 'two.diff' (output of
git show a604a
) is still split into two darcs patches, for code and test changes.In changeset:bc21726dfd73b434:
In changeset:bc21726dfd73b434:
Existing private key files are called
*.pem
, e.g.node.pem
.Don't we need to make Tahoe dependent on "pycryptopp >= 0.6.0"?
Replying to davidsarah:
Oh, it is, but the addition to
install_requires
is done in therequire_more
function of source:src/allmydata/_auto_deps.py. It could just as well be in the static definition ofinstall_requires
, since it's unconditional.Replying to davidsarah:
This new server key is an Ed25519 key (stored as 32 binary bytes), not an X509/SSL key, so .pem didn't seem appropriate. I'm happy to use a different name, though.
Good point, I'll move it there.
Replying to [warner]comment:30:
Oh, in that case
server.privkey
is fine.Ok, time to close this one out. The code landed a while ago, and things look
stable. A few more notes about the final protocol that we settled on (and how it
differs from the big explanation in comment:367907) :
anonymous-storage-FURL
: points at the foolscap object that providesnon-Accounting-based storage service, using the same protocol we've been
using for years. It has the "anonymous-" prefix to distinguish it from the
non-anonymous thing that Accounting (#666) will provide. When a server
requires accounting-based connections, its announcement will omit
anonymous-storage-FURL
.permutation-seed-base32
: tells clients where to place this server inthe permuted ring, for share-placement purposes. The server gets to decide
this placement independently of its serverid or tubid (this is marginally
more power than they had before, but we decided it was safe). New servers
use the public-key -based serverid for this. Old servers (those which have
published shares under their tubid) keep using their tubid for this, to
maintain stability (clients keep looking for shares in the same old
place). The code in
client.py:_init_permutation_seed()
holds this"Am I old or new" logic.
IntroducerClient
which receives these announcements produces anIServer
object, specifically aallmydata.storage_client.NativeStorageServer
, which offers methodsget_permutation_seed()
: maps directly topermutation-seed-base32
get_lease_seed()
andget_foolscap_write_enabler_seed()
: bothreturn the tubid, never the pubkey-based serverid, because these are
used for shared-secret authorization of
add-lease/remove-lease/modify-share operations, and shared-secrets are only
safe to inside a channel that's tied to those secrets. Since these
operations are closely tied to Foolscap, it's ok to leave them using the
tubid. When we move to a non-Foolscap transport layer, we'll need to use
different authorization mechanisms for leases and write-enablers, and
these will go away.