Python|Decorators Don't Have to be (that) Scary
Python Decorators Don't Have to be (that) Scary - Siafoo
Python Decorators Don't Have to be (that) Scary
Edit 
Add to Library 
Subscribe 
Send 
Contents
Decorators. The shear mention of them brings fear to even the seasoned Python programmer.
Okay, maybe not. But decorators, at least in this author's opinion, have a weird syntax and inevitably complicated implementations that are especially foreign (I think) to many who have gotten used to the simplicity of Python.
1 The Basics
Decorators modify functions. More specifically, a decorator is a function that transforms another function.
When you use a decorator, Python passes the decorated function -- we'll call this the target function -- to the decorator function, and replaces it with the result. Without decorators, it would look something like this:
1def decorator_function(target):
2 # Do something with the target function
3 target.attribute = 1
4 return target
5
6def target(a,b):
7 return a + b
8
9# This is what the decorator actually does
10target = decorator_function(target)
This code has the exact same functionality, but uses decorators. Note that I can name my decorator function whatever I want. Here, I've chosen 'decorator_function':
1def decorator_function(target):
2 # Do something with the target function
3 target.attribute = 1
4 return target
5
6# Here is the decorator, with the syntax '@function_name'
7@decorator_function
8def target(a,b):
9 return a + b
As you can see, you need to put the decorator function's name, prefaced with a @, on the line before the target function definition. Python internally will transform the target by applying the decorator to it and replacing it with the returned value.
Both of the above examples will have the same results:
1>>> target(1,2)
23
3>>> target.attribute
41
1.1 Does a decorator function have to return a function?
No. The decorator function can return absolutely anything, and Python will replace the target function with that return value. For example, you could do something like this:
1def decorator_evil(target):
2 return False
3
4@decorator_evil
5def target(a,b):
6 return a + b
7
8>>> target
9False
10
11>>> target(1,2)
12TypeError: 'bool' object is not callable
This is really not something you want to be doing on a regular basis though -- I'm pretty sure that a basic design principal is to not have functions randomly turning into other sorts of things. It makes good sense to at least return some sort of callable.
2 Run-Time Transformations
"But," I hear you saying, "I thought decorators did more than that. I want to do things at run-time, like conditionally calling the function and transforming the arguments and return value."
Can decorators do these things? Yes. Is that really something 'more' than we talked about above? Not really. It's important here not to get bogged down in the details -- you already know all there is to know about decorators. To do one of these more complex things, we're really just adding some plain old Python to the mix.
2.1 The Wrapper Function
Remember, your decorator function can return an arbitrary function. We'll call it the wrapper function, for reasons which will become clear in a second. The trick here is to define the wrapper function inside the decorator function, giving it access to the decorator function's variable scope, including the target function.
1def decorator(target):
2
3 def wrapper():
4 print 'Calling function "%s"' % target.__name__
5 return target()
6
7 # Since the wrapper is replacing the target function, assigning an attribute to the target function won't do anything.
8 # We need to assign it to the *wrapper function*.
9 wrapper.attribute = 1
10 return wrapper
11
12@decorator
13def target():
14 print 'I am the target function'
15
16>>> target()
17Calling function "target"
18I am the target function
19
20>>> target.attribute
211
As you can see, the wrapper function can do whatever it wants to the target function, including the simple case of returning the target's return value. But what happens to any arguments passed to the target function?
2.2 Getting the Arguments
Since the returned wrapper function replaces the target function, the wrapper function will receive the arguments intended for the target function. Assuming you want your decorator to work for any target function, your wrapper function then should accept arbitrary non-keyword arguments and arbitrary keyword arguments, add, remove, or modify arguments if necessary, and pass the arguments to the target function.
1def decorator(target):
2
3 def wrapper(*args, **kwargs):
4 kwargs.update({'debug': True}) # Edit the keyword arguments -- here, enable debug mode no matter what
5 print 'Calling function "%s" with arguments %s and keyword arguments %s' % (target.__name__, args, kwargs)
6 return target(*args, **kwargs)
7
8 wrapper.attribute = 1
9 return wrapper
10
11@decorator
12def target(a, b, debug=False):
13 if debug: print '[Debug] I am the target function'
14 return a+b
15
16>>> target(1,2)
17Calling function "target" with arguments (1, 2) and keyword arguments {'debug': True}
18[Debug] I am the target function
193
20
21>>> target.attribute
221
Note
You can also apply a decorator to a class method. If your decorator is always going to be used this way, and you need access to the current instance, your wrapper function can assume the first argument is always self:
1def wrapper(self, *args, **kwargs):
2 # Do something with 'self'
3 print self
4 return target(self, *args, **kwargs)
2.3 Summing It Up
So, we have a wrapper function that accepts arbitrary arguments defined inside our decorator function. The wrapper function can call the target function if and when it wants, get the result, do something with it, and return whatever it wants.
Say I want certain function calls to require positive confirmation before they are executed, and then stringify the result of the function before returning it. Note that the built-in function raw_input prints a message and then waits for a response from stdin.
1def decorator(target): # Python passes the target function to the decorator
2
3 def wrapper(*args, **kwargs):
4
5 choice = raw_input('Are you sure you want to call the function "%s"? ' % target.__name__)
6
7 if choice and choice[0].lower() == 'y':
8 # If input starts with a 'y', call the function with the arguments
9 result = target(*args, **kwargs)
10 return str(result)
11
12 else:
13 print 'Call to %s cancelled' % target.__name__
14
15 return wrapper
16
17@decorator
18def target(a,b):
19 return a+b
20
21>>> test.target(1,2)
22Are you sure you want to call the function "target"? n
23Call to target cancelled
24
25>>> test.target(1,2)
26Are you sure you want to call the function "target"? y
27'3'
3 Dynamic Decorators
Sometimes you might want to customize behavior by passing arbitrary options to your decorator function. A cursory look at decorator syntax suggests there's no way to do that. You could just abandon the decorator idea altogether, but you certainly don't have to.
The solution is define your decorator function inside another function -- call it the options function. Right before the target function definition, where you would normally list the decorator function (prepended with an @), call this options function (prepended with an @) instead. The options function then returns your decorator function, which Python will use as the passes the target function to as before.
3.1 Passing Options to the Decorator
Your options function can accept any arguments you want it to. Since the decorator function is defined inside the options function, the decorator function has access to any of the arguments passed to the options function.
1def options(value):
2
3 def decorator(target):
4 # Do something with the target function
5 target.attribute = value
6 return target
7 return decorator
8
9@options('value')
10def target(a,b):
11 return a + b
12
13>>> target(1,2)
143
15
16>>> target.attribute
17'value'
As you can see, nothing here about the decorator syntax itself has changed. Our decorator function is just in a dynamic scope instead of a static one.
3.2 Run-Time Tranformations
You can do Run-time Transformations by returning a wrapper function from your decorator function, just like before. For better or worse, though, there now must be three levels of functions:
1def options(debug_level):
2
3 def decorator(target):
4
5 def wrapper(*args, **kwargs):
6 kwargs.update({'debug_level': debug_level}) # Edit the keyword arguments
7 # here, set debug level to whatever specified in the options
8
9 print 'Calling function "%s" with arguments %s and keyword arguments %s' % (target.__name__, args, kwargs)
10 return target(*args, **kwargs)
11
12 return wrapper
13
14 return decorator
15
16@options(5)
17def target(a, b, debug_level=0):
18 if debug_level: print '[Debug Level %s] I am the target function' % debug_level
19 return a+b
20
21>>> target(1,2)
22Calling function "target" with arguments (1, 2) and keyword arguments {'debug_level': 5}
23[Debug Level 5] I am the target function
243
4 Caveat: Function Signatures
Phew. Understand everything you can do with decorators now? Good :). However, there is one drawback that must be mentioned.
The function returned from the decorator function -- usually a wrapper function -- replaces the target function completely. Any later introspection into what appears to be the target function will actually be into the wrapper function.
Most of the time, this is okay. Generally you just call a function with some options. Your program doesn't check to see what the function's __name__ or what arguments it accepts. So usually this problem won't be a problem.
However sometimes you care if the function you are calling supports a certain option, supports arbitrary options, or, perhaps, what its __name__ is. Or maybe you are interested in one if the function's attributes. If you look at a function that has been decorated, you will actually be looking at the wrapper function.
In the example below, note that the getargspec function of the inspect module gets the names and default values of a function's arguments.
1# This function is the same as the function 'target', except for the name
2def standalone_function(a,b):
3 return a+b
4
5def decorator(target):
6
7 def wrapper(*args, **kwargs):
8 return target()
9
10 return wrapper
11
12@decorator
13def target(a,b):
14 return a+b
15
16>>> from inspect import getargspec
17
18>>> standalone_function.__name__
19'standalone_function'
20
21>>> getargspec(standalone_function)
22(['a', 'b'], None, None, None)
23
24>>> target.__name__
25'wrapper'
26
27>>> getargspec(target)
28([], 'args', 'kwargs', None)
As you can see, the wrapper function reports that it accepts different arguments than the original target function Its call signature has changed.
4.1 A Solution
This is not an easy problem to solve. The update_wrapper method of the functools module provides a partial solution, copying the __name__ and other attributes from one function to another. But it does not solve what might be the largest problem of all: the changed call signature.
The decorator function of the decorator module provides the best solution: it can wrap your wrapper function in a dynamically-evaluated function with the correct arguments, restoring the original call signature. Similar to the update_wrapper function, it can also update your wrapper function with the __name__ and other attributes from the target function.
Note
For the remainder of this section, when I speak of the decorator function, I mean the one from this module, not one of the decorator functions that we've been using to transform our target functions.
Another way to create decorators
Unfortunately, though, the decorator function wasn't written with this use in mind. Instead it was written to turn standalone wrapper functions into full-fledged decorators, without having to worry about the function nesting described in Run-Time Transformations, above.
While this technique is often useful, it is much less customizable. Everything must be done at run-time, each time the function is executed. You cannot do any work when the target function is defined, including assigning the target or wrapper functions attributes or passing options to the decorator.
Also, in this author's opinion it is a bit of a black box; I'd rather know what my decorators are doing even if it is a little messier.
But we can make it work for us to solve this problem.
Ideally you would just call decorator(wrapper) and be done with it. However, things are never as simple as we'd like. As described above, the decorator function wraps the function passed to it -- our wrapper function -- in a dynamic function to fix the signature. But we still have a few problems:
- Problem #1:
- The dynamic function calls our wrapper function with (func, *args, **kwargs)
- Solution #1:
- Make our wrapper function accept (func, *args, **kwargs) instead of just (*args, **kwargs).
- Problem #2:
- The dynamic function is then wrapped in another function that expects to be used as an actual decorator -- it expects to be called with the target function, and will return the wrapper function.
- Solution #2:
- Call decorator's return value with the target function to get back to the dynamic function, which has the right signature.
This technique is a bit of a hack, and is a bit hard to explain, but it is easy to implement and works well.
This is the same example as before, but now with the decorator function (and a name change so things don't get too confusing):
1from decorator import decorator
2
3def my_decorator(target):
4
5 def wrapper(target, *args, **kwargs): # the target function has been prepended to the list of arguments
6 return target(*args, **kwargs)
7
8 # We are calling the returned value with the target function to get a 'proper' wrapper function back
9 return decorator(wrapper)(target)
10
11
12@my_decorator
13def target(a,b):
14 return a+b
15
16>>> from inspect import getargspec
17
18>>> target.__name__
19'target'
20
21>>> getargspec(target)
22(['a', 'b'], None, None, None)
5 Putting it All Together
Sometimes, you really need a customizable decorator that does work both at parse-time and run-time, and has the signature of the original target function.
Here's an example that ties everything together. Expanding on the example from earlier, say you want certain function calls to require positive confirmation before they are executed, and you want to be able to customize the confirmation string for each target function. Furthermore, for some reason [1], you need the decorated function's signature to match the target function.
Here we go:
1from decorator import decorator
2
3# The 'options' function. Recieves options and returns a decorator.
4def confirm(text):
5 '''
6 Pass a string to be sent as a confirmation message. Returns a decorator.
7 '''
8
9 # The actual decorator. Recieves the target function.
10 def my_decorator(target):
11 # Anything not in the wrapper function is done when the target function is initially parsed
12
13 # This is okay because the decorator function will copy the attribute to the wrapper function
14 target.attribute = 1
15
16 # The wrapper function. Replaces the target function and receives its arguments
17 def wrapper(target, *args, **kwargs):
18 # You could do something with the args or kwargs here
19
20 choice = raw_input(text)
21
22 if choice and choice[0].lower() == 'y':
23 # If input starts with a 'y', call the function with the arguments
24 result = target(*args, **kwargs)
25 # You could do something with the result here
26 return result
27
28 else:
29 print 'Call to %s cancelled' % target.__name__
30
31 # Fix the wrapper's call signature
32 return decorator(wrapper)(target)
33
34 return my_decorator
35
36@confirm('Are you sure you want to add these numbers? ')
37def target(a,b):
38 return a+b
39
40>>> Are you sure you want to add these numbers? yes
413
42
43>>> target.attribute
441
Hey, what do you know, it actually works.
6 Conclusion
As always, if you have a better way to do anything mentioned here, or if I've left anything out, leave a comment or feel free edit this article to fix the problem.
7 References
Michele Simionato's 'decorator' Module
Python Tips, Tricks, and Hacks - Decorators
| [1] | For example, Pylons checking your method's signature to decide what arguments to pass it. |
