Deferred Callback Propagation¶
Before a Deferred has resolved, it can have a number of callbacks and an errbacks added to it, that will tell it what to do in the case of a failure.
In many cases, these callbacks or errbacks can also return a Deferred, which may in turn return yet more Deferreds, each with their own callbacks and errbacks.
Callbacks may even be called by errbacks and vice versa, all the way along the chain.
At the end of this tangled (dare I say, ‘twisted’) process, the final returned value of the final callback or errback in the chain is the value that the initial Deferred will be passed.
While the callback chain sounds like a difficult process to grasp, it is usually very straightforward. For a more visual explanation, consider the following diagram:
If a chain of callbacks goes d
-> cbA
-> ebB
,
the final value of A
will be the value returned by ebB
.
In code, the above diagram would look similar to this:
def testCallbackPropagationExample(self):
def cbA(data):
d = defer.Deferred()
d.addCallback(cbB)
d.callback(True)
return d
def cbB(data):
d = defer.Deferred()
d.addErrback(ebC)
d.errback(1)
return d
def ebC(data):
raise Exception("NO")
d = defer.Deferred()
d.addCallback(cbA)
d.callback("OK")
return d
Notice that in the diagram, matching errbacks for cbA
and cbB
, and
a matching callback for cbC
were not set or used, but still existed.
This is because callbacks and errbacks are always created in pairs, much like
try
-catch
blocks. This behaviour is demonstrated below.
Callbacks and errbacks are always created in pairs¶
Note
Calling addCallbacks()
explicitly adds both a callback
and an
errback
, whereas addCallback()
or addErrback()
will
only explicitly add one or the other.
In this diagram, the Deferred, d
was given a callback via the function
Deferred.addCallback
, and the function it called back (cbA
) added an
errback via the function Deferred.addErrback
.
In both of these cases, only either a callback or an errback were explicitly added by the user, but in both cases a matching placeholder opposite was added by Twisted. This is because errbacks and callbacks are always created in pairs.
Notice that cbB
adds both a callback (cbC
) and an
errback (ebC
) explicitly via the function Deferred.addCallbacks
.
If the cbB
function in the first example diagram had added both a callback
and errback to its Deferred as was done in this example, then the final value of
d
would be the return of whichever of cbC
or ebC
was
eventually called (or whichever was called last).
The implementation of the above diagram would look something like this:
# Demonstrate that callbacks and errbacks are created in pairs
def testCallbackErrbackPairs(self):
def cbA(data):
d = defer.Deferred()
d.addErrback(ebB)
#do something that will call d.errback(x)
return d
def ebB(data):
d = defer.Deferred()
d.addCallbacks(cbC, ebC)
#do things that will call either:
# d.callback(x) or d.errback(x)
return d
def cbC(data):
return True
def ebC(data):
return False
d = defer.Deferred()
d.addCallback(cbA)
#do something that will eventually call d.callback(x)
return d
When a chain of callbacks and errbacks is useful¶
Callback chains are useful for handling situations where a branching tree is an ideal
way to handle data. (“Run x. If x fails, do y unless the failure is z, then do…”)
They are especially suited to the task because they are modular, easy to read, and do
not result in large, unwieldy walls of if
statements.
Multiple Callbacks for a single Deferred¶
It is worth noting that a single Deferred can have multiple callbacks.
Below, both returnA
and returnB
are added as callbacks to
d
. Because it has multiple callbacks, d
is assigned the value
returned by the last callback it fires. This means that the eventual resolved
value of d
is “B”.
class ExampleTests(unittest.TestCase):
# Demonstrate that an individual Deferred can have multiple callbacks
def testMultipleCallbacks(self):
def returnA(d):
print(d)
return "A"
def returnB(d):
print(d)
return "B"
d = defer.Deferred()
d.addCallback(returnA)
d.addCallback(returnB)
d.callback("Start")
print(d)
return d
When run, it should output something like this:
Ran 1 test in 0.125s
OK
Process finished with exit code 0
Start
A
<Deferred at 0x10b20ef98 current result: 'B'>
An errback calling a callback and continuing as-normal¶
Just because a Deferred’s errback is called, it does not necessarily mean that it will continue to errback, or that it will finally resolve into a Failure instance.
Errbacks can be safely caught and called back, returning the program to normal operation. Think of errbacks as being similar to try-except blocks in standard python.
The below example creates a Deferred and has it callback willErrback
, which
does errback, but the function that it errbacks (willCallback
) returns a
callback. As a result, the errback that was created by willErrback
is
considered to be handled, allowing the program to continue.
Eventually, the original Deferred resolves with the value “It was touch and go for a bit there” because that is what the last callback (or errback) in the chain returns.
# Demonstrate an Errback continuing as a callback
def testErrbackContinues(self):
def ebA(data):
d = defer.Deferred()
d.addCallback(cbB)
d.callback("Can you handle it?")
return d
def cbB(data):
print(data)
return "The Failure was handled!"
d = defer.Deferred()
d.addErrback(ebA)
d.addCallback(print)
d.errback(1)
return d
The output should look similar to this:
Ran 1 test in 0.113s
OK
Can you handle it?
The Failure was handled!
Another possible outcome could have been our errback continuing to errback any number of times before eventually being handled and calling back, returning the program to the normal flow.