Mon Mar 28 20:00:59 PDT 2011 Kevan Carstensen * interfaces: Create "IPeerSelector", an interface for an abstract peer selection class. New patches: [interfaces: Create "IPeerSelector", an interface for an abstract peer selection class. Kevan Carstensen **20110329030059 Ignore-this: 6bd8ad4056a7d75c7349876c59de8848 ] hunk ./src/allmydata/interfaces.py 602 """I am a node which represents a file: a sequence of bytes. I am not a container, like IDirectoryNode.""" +class IPeerSelector(Interface): + """ + I select peers for an upload, maximizing some measure of health. + + I keep track of the state of a grid relative to a file. This means + that I know about all of the peers that parts of that file could be + placed on, and about shares that have been placed on those peers. + Given this, I assign shares to peers in a way that maximizes the + file's health according to whichever definition of health I am + programmed with. I tell the uploader whether or not my assignment is + healthy. I keep track of failures during the process and update my + conclusions appropriately. + """ + def add_peer_with_share(peerid, shnum, existing=False): + """ + Update my internal state to reflect the fact that peer peerid + holds share shnum. Called for shares that are detected before + peer selection begins, and for shares that are successfully + allocated after peer selection ends. + + I take an optional parameter called existing. This is a hint + that the peer => share relationship I'm being told about already + exists. I may use this hint in heuristics designed to minimize + the amount of share data uploaded to the grid. + """ + + def add_peers(peerids=set): + """ + Update my internal state to include the peers in peerids as + potential candidates for storing a file. + """ + + def mark_full_peer(peerid): + """ + Mark the peer peerid as full. This means that any + peer-with-share relationships I know about for peerid remain + valid, but that peerid will not be assigned any new shares. + """ + + def mark_bad_peer(peerid): + """ + Mark the peer peerid as bad. This is typically called when an + error is encountered when communicating with a peer. I will + disregard any existing peer => share relationships associated + with peerid, and will not attempt to assign it any more shares. + """ + + def get_share_assignments(shallow=False): + """ + Given what I know about the state of the grid, I'll attempt to + assign shares to peers in a way that maximizes my definition of + file health. I'll return a list of (share, peerid) tuples with + my decision. + + I have an optional shallow parameter. If True, I will not + attempt to assign any new shares before checking happiness; + instead, I will rely only on the shares that I know are already + placed. + """ + + def is_healthy(): + """ + I return whether the share assignments I'm currently using + reflect a healthy file, based on my internal definitions. + """ + + def needs_recomputation(): + """ + I return True if the share assignments I last returned may have + become stale. This is a hint to the caller that they should call + get_share_assignments again. + """ + class IImmutableFileNode(IFileNode): def read(consumer, offset=0, size=None): """Download a portion (possibly all) of the file's contents, making Context: [NEWS: 'top' for node processes, WUI formatting, removal of GUI apps, documentation updates, foolscap dependency. refs #174, #1219, #1225 david-sarah@jacaranda.org**20110106005727 Ignore-this: f61ac58b4d10e635feb6f7391b1b48fe ] [Makefile: update 'clean' target for files in bin/ david-sarah@jacaranda.org**20110103052738 Ignore-this: 2bdbc4a50e13e508b66d0f65718c79b2 ] [docs: update performance.rst to describe the difference between already-uploaded and not-already-uploaded, to parameterize segment size, and to use "~A" to mean "approximately A" zooko@zooko.com**20110104065455 Ignore-this: 8df0d79a062ee19854c0211bd202f606 ] [bin/tahoe-script.template: On non-Windows, invoke support/bin/tahoe directly as a script (rather than via python), so that 'top' for example will show it as 'tahoe'. On Windows, simplify some code that set argv[0], which is never used. fixes #174 david-sarah@jacaranda.org**20101127232650 Ignore-this: 42a86f3eecfdc1ea7b76a7cc68626898 ] [test_runner: avoid unnecessary use of non-ASCII. david-sarah@jacaranda.org**20110101100101 Ignore-this: e2ff40dce6bb3b021306f2913d4e75df ] [docs/quickstart.html: fix redundant, badly nested tag. refs #1284 david-sarah@jacaranda.org**20110102175159 Ignore-this: 2ae9cc0b47d2e87b9eb64a0f517c4eef ] [docs/quickstart.html: information about 'troublesome dependencies' and 'verified systems' de-emphasized by smaller italic font. Re-wrap so that the HTML source is readable (just about) as text. Minor wording tweaks. Improve organization by adding 'Windows Caveats' subsection. fixes #1284 david-sarah@jacaranda.org**20110102174212 Ignore-this: e9dc57983974478200856651c5318fee ] [NEWS: update entry for removal of Mac and Windows apps. refs #1282 david-sarah@jacaranda.org**20101226042245 Ignore-this: c8099bc6e8235718d042c9a13c1e2425 ] [Move dependency imports from windows/depends.py (which has gone away) into src/allmydata/windows/tahoesvc.py. Also fix a pyflakes warning, and change the service display name from 'Allmydata Tahoe Node' to 'Tahoe-LAFS node'. refs #1282 david-sarah@jacaranda.org**20101226042100 Ignore-this: ee45f324934e1251380206dbee6346d0 ] [Remove unmaintained Windows GUI app, except for windows/tahoesvc.py which is moved to src/allmydata/windows. refs #1282 david-sarah@jacaranda.org**20101226040237 Ignore-this: cae37b6622a7dd5940acc7d3e6a98b90 ] [Remove the Makefile targets relating to the Mac GUI app. refs #1282 david-sarah@jacaranda.org**20101226025859 Ignore-this: 75303be783974b41138744ec62b07965 ] [NEWS: remove unmaintained Mac GUI app. refs #1282 david-sarah@jacaranda.org**20101226020858 Ignore-this: 40474a07f4a550b48563d35350be7ab5 ] [Remove unmaintained Mac GUI app. fixes #1282 david-sarah@jacaranda.org**20101226020508 Ignore-this: b3613bf1abfd284d542bf7c753ec557a ] [Remove src/allmydata/util/find_exe.py which is no longer used. fixes #1150 david-sarah@jacaranda.org**20101226023206 Ignore-this: 7436c9b53bf210aed34a1a973cd9cace ] [status_web_pages_review.darcs.patch freestorm77@gmail.com**20110102034214 Ignore-this: 29f1ecb36177f10f3f846b3d56b313b2 I make some changes on status web pages status.xhtml: - Delete unused webform_css link - Align tables on the left tahoe-css: - Do some minor changes on code synthax - changes table.status-download-events style to look like other tables status.py: - Align table on the left - Changes table header - Add heading tags - Modify google api graph: add image border, calculate height to feet data signed-off-by: zooko@zooko.com fixes #1219 ] [test_storage.py: fix a pyflakes unused import warning. david-sarah@jacaranda.org**20101231220756 Ignore-this: df08231540cb7dff9d2b038e47ab30ee ] [test_storage.py: leave at least 512 MiB free when running test_large_share. refs #1195 david-sarah@jacaranda.org**20101231203215 Ignore-this: b2144c0341c3452b5d4ba219e284ea0e ] [storage: use fileutil's version of get_disk_stats() and get_available_space(), use mockery/fakery in tests, enable large share test on platforms with sparse files and if > 4 GiB of disk space is currently available zooko@zooko.com**20100910173629 Ignore-this: 1304f1164c661de6d5304f993eb9b27b ] [fileutil: copy in the get_disk_stats() and get_available_space() functions from storage/server.py zooko@zooko.com**20100910173520 Ignore-this: 8b15569715f710f4fc5092f7ca109253 ] [Update foolscap version requirement to 0.6.0, to address http://foolscap.lothar.com/trac/ticket/167 david-sarah@jacaranda.org**20101231060039 Ignore-this: 98d2b8086a1a500b9f4565bca5a3810 ] [docs/webapi.rst: typos. david-sarah@jacaranda.org**20101230034422 Ignore-this: d1f5166d72cc711f7e0d9981eac9105e ] [docs/webapi.rst: capitalization, formatting of section on URL character encoding, and a correction about Internet Explorer. david-sarah@jacaranda.org**20101230034049 Ignore-this: b3b9819d2fb264b4cdc5c8afd4e8c48d ] [docs: corrections and clarifications. david-sarah@jacaranda.org**20101227051056 Ignore-this: e33202858c7644c58f3f924b164294b6 ] [docs: more formatting cleanups and corrections. Spell webapi and wapi as web-API. david-sarah@jacaranda.org**20101227050533 Ignore-this: 18b23cbfb780df585d8a722a1ec63e94 ] [docs/debian.rst: bring description of building dependencies from source up-to-date, and change hostname from allmydata.com to tahoe-lafs.org. david-sarah@jacaranda.org**20101212222912 Ignore-this: f38462afc88b4475195610385a28391c ] [docs/architecture.rst: correct rst syntax. david-sarah@jacaranda.org**20101212202003 Ignore-this: 3fbe12feb28bec6f1c63aedbc79aad21 ] [docs/architecture.rst: formatting. david-sarah@jacaranda.org**20101212201719 Ignore-this: 305fa5dfc2939355eaf6d0d2161eb1ff ] [docs: linkification, wording improvements. david-sarah@jacaranda.org**20101212201234 Ignore-this: 4e67287f527a8bc728cfbd93255d2aae ] [docs: formatting. david-sarah@jacaranda.org**20101212201115 Ignore-this: 2e0ed394ac7726651d3a4f2c4b0d3798 ] [docs/configuration.rst: more formatting tweaks; which -> that. david-sarah@jacaranda.org**20101212195522 Ignore-this: a7becb7021854ca5a90edd892b36fdd7 ] [docs/configuration.rst: more changes to formatting. david-sarah@jacaranda.org**20101212194511 Ignore-this: 491aac33e5f5268d224359f1447d10be ] [docs/configuration.rst: changes to formatting (mainly putting commands and filenames in monospace). david-sarah@jacaranda.org**20101212181828 Ignore-this: 8a1480e2d5f43bee678476424615b50f ] [scripts/backupdb.py: more accurate comment about path field. david-sarah@jacaranda.org**20101212170320 Ignore-this: 50e47a2228a85207bbcd188a78a0d4e6 ] [scripts/cli.py: fix missing 'put' in usage example for 'tahoe put'. david-sarah@jacaranda.org**20101212170207 Ignore-this: 2cbadf066fff611fc03d3c0ff97ce6ec ] [docs/frontends/CLI.rst: changes to formatting (mainly putting commands and filenames in monospace), and to command syntax to reflect that DIRCAP/... is accepted. Clarify the syntax of 'tahoe put' and other minor corrections. Tahoe -> Tahoe-LAFS. david-sarah@jacaranda.org**20101212165800 Ignore-this: a123ef6b564aa8624d1e79c97068ea12 ] [docs/frontends/CLI.rst: Unicode arguments to 'tahoe' work on Windows as of v1.7.1. david-sarah@jacaranda.org**20101212063740 Ignore-this: 3977a99dfa86ac33a44171deaf43aaab ] [docs/known_issues.rst: fix title and linkify another URL. refs #1225 david-sarah@jacaranda.org**20101212062817 Ignore-this: cc91287f7fb51c23440b3d2fe79c449c ] [docs/known_issues.rst: fix an external link. refs #1225 david-sarah@jacaranda.org**20101212062435 Ignore-this: b8cbf12f353131756c358965c48060ec ] [Fix a link from uri.rst to dirnodes.rst. refs #1225 david-sarah@jacaranda.org**20101212054502 Ignore-this: af6205299f5c9a33229cab259c00f9d5 ] [Fix a link from webapi.rst to FTP-and-SFTP.rst. refs #1225 david-sarah@jacaranda.org**20101212053435 Ignore-this: 2b9f88678c3447ea860d6b61e8799858 ] [More specific hyperlink to architecture.rst from helper.rst. refs #1225 david-sarah@jacaranda.org**20101212052607 Ignore-this: 50424c768fca481252fabf58424852dc ] [Update hyperlinks between docs, and linkify some external references. refs #1225 david-sarah@jacaranda.org**20101212051459 Ignore-this: cd43a4c3d3de1f832abfa88d5fc4ace1 ] [docs/specifications/dirnodes.rst: fix references to mutable.rst. refs #1225 david-sarah@jacaranda.org**20101212012720 Ignore-this: 6819b4b4e06e947ee48b365e840db37d ] [docs/specifications/mutable.rst: correct the magic string for v1 mutable containers. refs #1225 david-sarah@jacaranda.org**20101212011400 Ignore-this: 99a5fcdd40cef83dbb08f323f6cdaaca ] [Move .txt files in docs/frontends and docs/specifications to .rst. refs #1225 david-sarah@jacaranda.org**20101212010251 Ignore-this: 8796d35d928370f7dc6ad2dafdc1c0fe ] [Convert docs/frontends and docs/specifications to reStructuredText format (not including file moves). david-sarah@jacaranda.org**20101212004632 Ignore-this: e3ceb2d832d73875abe48624ddbb5622 ] [scripts/cli.py: remove the disclaimer in the help for 'tahoe cp' that it does not handle non-ASCII filenames well. (At least, we intend to handle them.) david-sarah@jacaranda.org**20101130002145 Ignore-this: 94c003efaa20b9eb4a83503d79844ca ] [relnotes.txt: fifth -> sixth labor-of-love release zooko@zooko.com**20101129045647 Ignore-this: 21c245015268b38916e3a138d256c09d ] [Makefile: BB_BRANCH is set to the empty string for trunk, not the string 'trunk'. david-sarah@jacaranda.org**20101128233512 Ignore-this: 5a7ef8eb10475636d21b91e25b56c369 ] [relnotes.txt: eleventh -> twelfth release. david-sarah@jacaranda.org**20101128223321 Ignore-this: 1e26410156a665271c1170803dea2c0d ] [relnotes.tst: point to known_issues.rst, not known_issues.txt. david-sarah@jacaranda.org**20101128222918 Ignore-this: 60194eb4544cac446fe4f60b3e34b887 ] [quickstart.html: fix link to point to allmydata-tahoe-1.8.1.zip. david-sarah@jacaranda.org**20101128221728 Ignore-this: 7b3ee86f8256aa12f5d862f689f3ee29 ] [TAG allmydata-tahoe-1.8.1 david-sarah@jacaranda.org**20101128212336 Ignore-this: 9c18bdeaef4822f590d2a0d879e00621 ] Patch bundle hash: 9a0a487d93c7ef13e5e40e21603f9feeb68466ca