tahoe-lafs/trac-2024-07-25

don't let "Calling Stale Broke" trigger an incident -- that's a false alarm #1998

New issue

Open

opened 2013-06-07 21:25:23 +00:00 by zooko · 0 comments

zooko commented

2013-06-07 21:25:23 +00:00

Owner

A customer (thedod) sent me an incident report file. It turned out to be triggered by this:


    19:33:48.913 [237]: WEIRD Error in Segmentation FAILURE:

    [CopiedFailure instance: Traceback from remote host -- Traceback (most recent call last):
    Failure: allmydata.interfaces.NoSharesError: no shares (need 1). Last failure: [Failure instance: Traceback: <class 'foolscap.ipb.DeadReferenceError'>: Calling Stale Broker
    /home/nonboong/src/tahoe-lafs/src/allmydata/immutable/downloader/share.py:269:_do_loop
    /home/nonboong/src/tahoe-lafs/src/allmydata/immutable/downloader/share.py:744:_send_requests
    /home/nonboong/src/tahoe-lafs/src/allmydata/immutable/downloader/share.py:754:_send_request
    /usr/lib/python2.7/dist-packages/foolscap/referenceable.py:415:callRemote
    --- <exception caught here> ---
    /usr/local/lib/python2.7/site-packages/Twisted-12.0.0-py2.7-linux-i686.egg/twisted/internet/defer.py:134:maybeDeferred
    /usr/lib/python2.7/dist-packages/foolscap/referenceable.py:455:_callRemote
    /usr/lib/python2.7/dist-packages/foolscap/broker.py:477:newRequestID
    ]
    ]

    [INCIDENT-TRIGGER]

I'm pretty sure there's nothing out of the ordinary about this. It's just what happens whenever a network connection to a storage server fails during an operation. Is that right?

So I'm asking the customer to look for other incident report files to send to me. But if this was a false alarm, then it has impeded our investigation and should be silenced.

Assigning to Brian to confirm my understanding that this is common and harmless.

A customer (thedod) sent me an incident report file. It turned out to be triggered by this: ``` 19:33:48.913 [237]: WEIRD Error in Segmentation FAILURE: [CopiedFailure instance: Traceback from remote host -- Traceback (most recent call last): Failure: allmydata.interfaces.NoSharesError: no shares (need 1). Last failure: [Failure instance: Traceback: <class 'foolscap.ipb.DeadReferenceError'>: Calling Stale Broker /home/nonboong/src/tahoe-lafs/src/allmydata/immutable/downloader/share.py:269:_do_loop /home/nonboong/src/tahoe-lafs/src/allmydata/immutable/downloader/share.py:744:_send_requests /home/nonboong/src/tahoe-lafs/src/allmydata/immutable/downloader/share.py:754:_send_request /usr/lib/python2.7/dist-packages/foolscap/referenceable.py:415:callRemote --- <exception caught here> --- /usr/local/lib/python2.7/site-packages/Twisted-12.0.0-py2.7-linux-i686.egg/twisted/internet/defer.py:134:maybeDeferred /usr/lib/python2.7/dist-packages/foolscap/referenceable.py:455:_callRemote /usr/lib/python2.7/dist-packages/foolscap/broker.py:477:newRequestID ] ] [INCIDENT-TRIGGER] ``` I'm pretty sure there's nothing out of the ordinary about this. It's just what happens whenever a network connection to a storage server fails during an operation. Is that right? So I'm asking the customer to look for *other* incident report files to send to me. But if this was a false alarm, then it has impeded our investigation and should be silenced. Assigning to Brian to confirm my understanding that this is common and harmless.