1 patch for repository http://tahoe-lafs.org/source/tahoe/trunk: Fri Oct 7 03:15:29 BST 2011 david-sarah@jacaranda.org * Add misc/coding_tools/check-miscaptures.py to detect incorrect captures of variables declared in a for loop, and a 'make check-miscaptures' Makefile target to run it. (It is also run by 'make code-checks'.) refs #1555 New patches: [Add misc/coding_tools/check-miscaptures.py to detect incorrect captures of variables declared in a for loop, and a 'make check-miscaptures' Makefile target to run it. (It is also run by 'make code-checks'.) refs #1555 david-sarah@jacaranda.org**20111007021529 Ignore-this: 212f4f6039af508716e09b8952ae7ab ] { hunk ./Makefile 124 false endif -code-checks: build version-and-path check-interfaces -find-trailing-spaces -check-umids pyflakes +code-checks: build version-and-path check-interfaces check-miscaptures -find-trailing-spaces -check-umids pyflakes version-and-path: $(TAHOE) --version-and-path hunk ./Makefile 133 $(TAHOE) @misc/coding_tools/check-interfaces.py 2>&1 |tee violations.txt @echo +check-miscaptures: + $(PYTHON) misc/coding_tools/check-miscaptures.py $(SOURCES) 2>&1 |tee miscaptures.txt + @echo + pyflakes: $(PYTHON) -OOu `which pyflakes` $(SOURCES) |sort |uniq @echo addfile ./misc/coding_tools/check-miscaptures.py hunk ./misc/coding_tools/check-miscaptures.py 1 +#! /usr/bin/python + +import os, sys, compiler +from compiler.ast import Node, For, AssName, Name, Lambda, Function + +def check_file(filename): + results = [] + check_ast(compiler.parseFile(filename), results) + return results + +def check_source(source): + results = [] + check_ast(compiler.parse(source), results) + return results + +def check_ast(ast, results): + if isinstance(ast, For): + check_for(ast, results) + else: + for child in ast.getChildNodes(): + if isinstance(ast, Node): + check_ast(child, results) + +def check_for(ast, results): + declared = set() + captured = set() + collect_declared(ast, declared) + collect_captured(ast, captured, set(), False) + for name in captured & declared: + results.append((name, ast.lineno)) + +def collect_declared(ast, declared): + if isinstance(ast, AssName): + declared.add(ast.name) + else: + for child in ast.getChildNodes(): + if isinstance(ast, Node): + collect_declared(child, declared) + +def collect_captured(ast, captured, excluded, in_func): + if isinstance(ast, Name): + if in_func and ast.name not in excluded: + captured.add(ast.name) + elif isinstance(ast, (Lambda, Function)): + # Formal parameters of the function are not captures + newly_excluded = excluded.copy() + for argname in ast.argnames: + newly_excluded.add(argname) + + childnodes = ast.getChildNodes() + # ... and neither are variable uses in default argument + # expressions, provided we weren't already in a function. + if not in_func: + childnodes = childnodes[len(ast.defaults):] + + for child in childnodes: + if isinstance(ast, Node): + collect_captured(child, captured, newly_excluded, True) + else: + for child in ast.getChildNodes(): + if isinstance(ast, Node): + collect_captured(child, captured, excluded, in_func) + +def report(path, results, out): + for (name, lineno) in results: + print >>out, "%s:%d %r" % (path, lineno, name) + + +def check(sources, out): + class Counts: + n = 0 + processed_files = 0 + suspect_files = 0 + counts = Counts() + + def _process(path): + results = check_file(path) + report(path, results, out) + counts.n += len(results) + counts.processed_files += 1 + if len(results) > 0: + counts.suspect_files += 1 + + for source in sources: + print >>out, "Checking %s..." % (source,) + if os.path.isfile(source): + _process(source) + else: + for (dirpath, dirnames, filenames) in os.walk(source): + for fn in filenames: + (basename, ext) = os.path.splitext(fn) + if ext == '.py': + _process(os.path.join(dirpath, fn)) + + print >>out, ("%d suspiciously captured variables in %d out of %d files" + % (counts.n, counts.suspect_files, counts.processed_files)) + + +sources = ['src'] +if len(sys.argv) > 1: + sources = sys.argv[1:] +check(sources, sys.stderr) + + +# TODO: self-tests } Context: [no_network.py: Clean up whitespace around code changed by previous patch. david-sarah@jacaranda.org**20111004010407 Ignore-this: 647ec8a9346dca1a41212ab250619b72 ] [no_network.py: Fix potential bugs in some tests due to capture of slots in for loops. david-sarah@jacaranda.org**20111004010231 Ignore-this: 9c496877613a3befd54979e5de6e63d2 ] [docs: fix the rst formatting of COPYING.TGPPL.rst zooko@zooko.com**20111003043333 Ignore-this: c5fbc83f4a3db81a0c95b27053c463c5 Now it renders correctly both on trac and with rst2html --verbose from docutils v0.8.1. ] [MDMF: remove extension fields from caps, tolerate arbitrary ones. Fixes #1526 Brian Warner **20111001233553 Ignore-this: 335e1690aef1146a2c0b8d8c18c1cb21 The filecaps used to be produced with hints for 'k' and segsize, but they weren't actually used, and doing so had the potential to limit how we change those filecaps in the future. Also the parsing code had some problems dealing with other numbers of extensions. Removing the existing fields and making the parser tolerate (and ignore) extra ones makes MDMF more future-proof. ] [test/test_runner.py: BinTahoe.test_path has rare nondeterministic failures; this patch probably fixes a problem where the actual cause of failure is masked by a string conversion error. david-sarah@jacaranda.org**20110927225336 Ignore-this: 6f1ad68004194cc9cea55ace3745e4af ] [docs/configuration.rst: add section about the types of node, and clarify when setting web.port enables web-API service. fixes #1444 zooko@zooko.com**20110926203801 Ignore-this: ab94d470c68e720101a7ff3c207a719e ] [TAG allmydata-tahoe-1.9.0a2 warner@lothar.com**20110925234811 Ignore-this: e9649c58f9c9017a7d55008938dba64f ] Patch bundle hash: 83fda893e4e4585dd65a793d1d45225f7b8177ee