tahoe-lafs/trac-2024-07-25

CLI: allow dispatch functions to return a Deferred #2826

New issue

Closed

opened 2016-09-09 17:37:49 +00:00 by warner · 2 comments

warner commented

2016-09-09 17:37:49 +00:00

Owner

This will enable #2490 to run (async) Tor operations during tahoe create-node --listen=tor. Currently all CLI dispatch functions must finish synchronously.

The file-operation subcommands, which use blocking httplib calls, must be run from unit tests under deferToThread. #2825 is about fixing that, but this ticket will do some cleanup that will help. Specifically it will move the deferToThread down into runner.py, using the thread only on the subcommands which really need it. After this is fixed, there should be no deferToThread calls in the unit tests.

This will enable #2490 to run (async) Tor operations during `tahoe create-node --listen=tor`. Currently all CLI dispatch functions must finish synchronously. The file-operation subcommands, which use blocking `httplib` calls, must be run from unit tests under `deferToThread`. #2825 is about fixing that, but this ticket will do some cleanup that will help. Specifically it will move the deferToThread down into `runner.py`, using the thread only on the subcommands which really need it. After this is fixed, there should be no deferToThread calls in the unit tests.

tahoe-lafs added the

labels 2016-09-09 17:37:49 +00:00

tahoe-lafs added this to the 1.12.0 milestone 2016-09-09 17:37:49 +00:00

warner commented

2016-09-09 23:30:00 +00:00

Author

Owner

The branch is ready to land. It looks like there's one deferToThread left in the test suite: test_runner.RunBinTahoeMixin.run_bintahoe uses subprocess.Popen underneath deferToThread. I think this can be replaced with a reactor.spawnProcess, but I'm not 100% sure, and I don't want to go messing with it right now, so I'm going to leave that cleanup for another day.

The branch is ready to land. It looks like there's one `deferToThread` left in the test suite: `test_runner.RunBinTahoeMixin.run_bintahoe` uses `subprocess.Popen` underneath `deferToThread`. I *think* this can be replaced with a `reactor.spawnProcess`, but I'm not 100% sure, and I don't want to go messing with it right now, so I'm going to leave that cleanup for another day.

Brian Warner <warner@lothar.com> commented

2016-09-09 23:30:26 +00:00

Author

Owner

In 802cfc8/trunk:

CLI: allow dispatch functions to return Deferred

In addition, CLI functions are allowed to use sys.exit() instead of
always needing to return the exit code as an integer.

runner.py now knows about the blocking httplib calls in scripts/cli and
scripts/magic_folder, and uses deferToThread() to invoke them. Those
functions cannot return a Deferred: when rewrite them to use twisted.web
or treq, we'll remove this deferToThread call.

Option parsing was split out to a separate function for testing. We now
use twisted.internet.task.react() to start the reactor, which required
changing the way runner.py is tested.

closes ticket:2826

In [802cfc8/trunk](/tahoe-lafs/trac-2024-07-25/commit/802cfc87fe8c9343f39e4ef80100e0d61817292a): ``` CLI: allow dispatch functions to return Deferred In addition, CLI functions are allowed to use sys.exit() instead of always needing to return the exit code as an integer. runner.py now knows about the blocking httplib calls in scripts/cli and scripts/magic_folder, and uses deferToThread() to invoke them. Those functions cannot return a Deferred: when rewrite them to use twisted.web or treq, we'll remove this deferToThread call. Option parsing was split out to a separate function for testing. We now use twisted.internet.task.react() to start the reactor, which required changing the way runner.py is tested. closes ticket:2826 ```

tahoe-lafs added the

fixed

label 2016-09-09 23:30:26 +00:00

Brian Warner <warner@lothar.com> closed this issue

2016-09-09 23:30:26 +00:00