tahoe-lafs/trac-2024-07-25

intermittent DirtyReactor failures in test_system.Connections.test_rref #2768

New issue

Open

opened 2016-04-04 05:23:25 +00:00 by warner · 0 comments

warner commented

2016-04-04 05:23:25 +00:00

Owner

[ERROR]
Traceback (most recent call last):
Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean.
DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug)
<DelayedCall 0x102f15440 [0.0305078029633s] called=0 cancelled=0 LoopingCall<0.3>(iterateEndpoint, *(), **{})()>
<DelayedCall 0x102f3dea8 [0.0598268508911s] called=0 cancelled=0 LoopingCall<0.3>(iterateEndpoint, *(), **{})()>
<DelayedCall 0x102fcb248 [0.18648314476s] called=0 cancelled=0 LoopingCall<0.3>(iterateEndpoint, *(), **{})()>
<DelayedCall 0x102fc72d8 [0.186053037643s] called=0 cancelled=0 LoopingCall<0.3>(iterateEndpoint, *(), **{})()>

allmydata.test.test_system.Connections.test_rref

This is a consequence of Twisted#8223, which is fixed in the upcoming Twisted-16.1.0 .

Foolscap-0.11.0 is necessary to expose the bug because that's the first version to use Twisted's HostnameEndpoint
Twisted-16.0.0 is necessary to expose the bug because previous versions didn't work at all (Twisted#8014)
multiple network addresses are necessary to expose the bug because 8223 involves cancelling an endpoint.connect, which only happens when Foolscap succeeds with one hint (and thus cancels the others).

I don't know if we need to fix anything in Tahoe to deal with this. I suppose we could bump the Twisted dependency to 16.1.0, once it comes out (probably in the next few weeks). I'm mostly just filing this ticket to help others who run into this same problem before it goes away by itself.

With Foolscap-0.11.0 and Twisted-16.0.0, and a computer that is online (and thus has an external IP address, and thus auto-detected FURLs have two or more location hints {including localhost}), I occasionally see the following DirtyReactorError in `test_system.Connections.test_rref`: ``` [ERROR] Traceback (most recent call last): Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean. DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug) <DelayedCall 0x102f15440 [0.0305078029633s] called=0 cancelled=0 LoopingCall<0.3>(iterateEndpoint, *(), **{})()> <DelayedCall 0x102f3dea8 [0.0598268508911s] called=0 cancelled=0 LoopingCall<0.3>(iterateEndpoint, *(), **{})()> <DelayedCall 0x102fcb248 [0.18648314476s] called=0 cancelled=0 LoopingCall<0.3>(iterateEndpoint, *(), **{})()> <DelayedCall 0x102fc72d8 [0.186053037643s] called=0 cancelled=0 LoopingCall<0.3>(iterateEndpoint, *(), **{})()> allmydata.test.test_system.Connections.test_rref ``` This is a consequence of [Twisted#8223](https://twistedmatrix.com/trac/ticket/8223), which is fixed in the upcoming Twisted-16.1.0 . * Foolscap-0.11.0 is necessary to expose the bug because that's the first version to use Twisted's `HostnameEndpoint` * Twisted-16.0.0 is necessary to expose the bug because previous versions didn't work at all ([Twisted#8014](https://twistedmatrix.com/trac/ticket/8014)) * multiple network addresses are necessary to expose the bug because 8223 involves cancelling an endpoint.connect, which only happens when Foolscap succeeds with one hint (and thus cancels the others). I don't know if we need to fix anything in Tahoe to deal with this. I suppose we could bump the Twisted dependency to 16.1.0, once it comes out (probably in the next few weeks). I'm mostly just filing this ticket to help others who run into this same problem before it goes away by itself.