Course
Decorators are a powerful and elegant feature in Python that let you modify or extend the behavior of functions and methods without changing their code.
A decorator is a design pattern in Python that allows a user to add new functionality to an existing object without modifying its structure. Decorators are typically applied to functions, and they play a crucial role in enhancing or modifying the behavior of functions. Traditionally, decorators are placed before the definition of a function you want to decorate. In this tutorial, we'll demonstrate how to effectively use decorators in Python functions.
Functions as First-Class Objects
Functions in Python are first class citizens. This means that they support operations such as being passed as an argument, returned from a function, modified, and assigned to a variable. This property is crucial as it allows functions to be treated like any other object in Python, enabling greater flexibility in programming.
To easily run all the example code in this tutorial yourself, you can create a DataLab workbook for free that has Python pre-installed and contains all code samples. For more practice on decorators, check out this hands-on DataCamp exercise.
Learn Python From Scratch
Assigning functions to variables
To kick us off we create a function that will add one to a number whenever it is called. We'll then assign the function to a variable and use this variable to call the function.
def plus_one(number):
return number + 1
add_one = plus_one
add_one(5)
6
Defining functions inside other functions
Defining functions inside other functions is a powerful feature in Python—and it's essential for building decorators. Let’s look at another core idea: passing functions as arguments. This will bring us one step closer to writing decorators.
def plus_one(number):
def add_one(number):
return number + 1
result = add_one(number)
return result
plus_one(4)
5
Passing functions as arguments to other functions
Functions can also be passed as parameters to other functions. Let's illustrate that below.
def plus_one(number):
return number + 1
def function_call(function):
number_to_add = 5
return function(number_to_add)
function_call(plus_one)
6
Functions returning other functions
A function can also generate another function. We'll show that below using an example.
def hello_function():
def say_hi():
return "Hi"
return say_hi
hello = hello_function()
hello()
'Hi'
Inner Functions and Closures
Python allows a nested function to access the outer scope of the enclosing function. This is a critical concept in decorators, known as a closure.
A closure in Python is a function that remembers the environment in which it was created, even after that environment is no longer active. This means a nested function can "close over" variables from its enclosing scope and continue to use them.
Closures are essential for understanding decorators because decorators rely on the ability of a nested wrapper function to access and modify the state of the enclosing decorator function.
Example of a closure:
def outer_function(message):
def inner_function():
print(f"Message from closure: {message}")
return inner_function
closure_function = outer_function("Hello, closures!")
closure_function()
# Output: Message from closure: Hello, closures!
In this example:
inner_function
is a closure because it accessesmessage
, a variable from its enclosing scope (outer_function
).- Even though
outer_function
has finished executing,inner_function
retains access tomessage
.
When you create a decorator, the wrapper function (inside the decorator) is a closure. It retains access to the function being decorated and any additional state or arguments defined in the decorator function. For example:
def simple_decorator(func):
def wrapper():
print("Before the function call")
func()
print("After the function call")
return wrapper
@simple_decorator
def greet():
print("Hello!")
greet()
# Output:
# Before the function call
# Hello!
# After the function call
Here, wrapper
is a closure that remembers the function greet
and adds behavior before and after its execution.
Creating Your First Decorator
Now that you understand closures—the ability of a function to remember variables from its enclosing scope—we’re ready to create our first real decorator. Closures are the secret sauce that allows decorators to work behind the scenes.
Let's go ahead and create a simple decorator that will convert a sentence to uppercase. We do this by defining a wrapper inside an enclosed function. As you can see it very similar to the function inside another function that we created earlier.
def uppercase_decorator(function):
def wrapper():
func = function()
make_uppercase = func.upper()
return make_uppercase
return wrapper
Since our decorator takes a function as an argument, we’ll define a new function and pass it to the decorator. We learned earlier that we could assign a function to a variable. We'll use that trick to call our decorator function.
def say_hi():
return 'hello there'
decorate = uppercase_decorator(say_hi)
decorate()
'HELLO THERE'
Using the @ syntax
However, Python provides a much easier way for us to apply decorators. We simply use the @ symbol before the function we'd like to decorate. Let's show that in practice below.
@uppercase_decorator
def say_hi():
return 'hello there'
say_hi()
'HELLO THERE'
Stacking Multiple Decorators
Once you’re comfortable using the @
syntax for a single decorator, you can take it a step further and stack multiple decorators on the same function. Just keep in mind: the order matters!
Below we'll define another decorator that splits the sentence into a list. We'll then apply the uppercase_decorator
and split_string
decorator to a single function.
import functools
def split_string(function):
@functools.wraps(function)
def wrapper():
func = function()
splitted_string = func.split()
return splitted_string
return wrapper
@split_string
@uppercase_decorator
def say_hi():
return 'hello there'
say_hi()
['HELLO', 'THERE']
From the above output, we notice that the application of decorators is from the bottom up. Had we interchanged the order, we'd have seen an error since lists don't have an upper
attribute. The sentence has first been converted to uppercase and then split into a list.
Note: When stacking decorators, it's a common practice to use functools.wraps
to ensure that the metadata of the original function is preserved throughout the stacking process. This helps maintain clarity and consistency in debugging and understanding the properties of the decorated function.
Accepting Arguments in Decorators
So far, we’ve seen decorators that only wrap a function. But what if you want to configure the decorator itself—like passing parameters into it? That’s where decorator factories come in.
def decorator_with_arguments(function):
def wrapper_accepting_arguments(arg1, arg2):
print("My arguments are: {0}, {1}".format(arg1,arg2))
function(arg1, arg2)
return wrapper_accepting_arguments
@decorator_with_arguments
def cities(city_one, city_two):
print("Cities I love are {0} and {1}".format(city_one, city_two))
cities("Nairobi", "Accra")
My arguments are: Nairobi, Accra
Cities I love are Nairobi and Accra
Note: It's essential to ensure that the number of arguments in the decorator (arg1, arg2
in this example) matches the number of arguments in the wrapped function (cities
in this example). This alignment is crucial to avoid errors and ensure proper functionality when using decorators with arguments.
General-Purpose Decorators with *args and **kwargs
To define a general purpose decorator that can be applied to any function we use args
and **kwargs
. args
and **kwargs
collect all positional and keyword arguments and stores them in the args and kwargs variables. args
and kwargs
allow us to pass as many arguments as we would like during function calls.
def a_decorator_passing_arbitrary_arguments(function_to_decorate):
def a_wrapper_accepting_arbitrary_arguments(*args,**kwargs):
print('The positional arguments are', args)
print('The keyword arguments are', kwargs)
function_to_decorate(*args)
return a_wrapper_accepting_arbitrary_arguments
@a_decorator_passing_arbitrary_arguments
def function_with_no_argument():
print("No arguments here.")
function_with_no_argument()
The positional arguments are ()
The keyword arguments are {}
No arguments here.
Let's see how we'd use the decorator using positional arguments.
@a_decorator_passing_arbitrary_arguments
def function_with_arguments(a, b, c):
print(a, b, c)
function_with_arguments(1,2,3)
The positional arguments are (1, 2, 3)
The keyword arguments are {}
1 2 3
Here’s how you can pass keyword arguments to a decorated function:
@a_decorator_passing_arbitrary_arguments
def function_with_keyword_arguments():
print("This has shown keyword arguments")
function_with_keyword_arguments(first_name="Derrick", last_name="Mwiti")
The positional arguments are ()
The keyword arguments are {'first_name': 'Derrick', 'last_name': 'Mwiti'}
This has shown keyword arguments
Note: The use of **kwargs
in the decorator allows it to handle keyword arguments. This makes the general-purpose decorator versatile and capable of handling a variety of argument types during function calls.
Passing Arguments to Decorators
Now let's see how we'd pass arguments to the decorator itself. In order to achieve this, we define a decorator maker that accepts arguments then define a decorator inside it. We then define a wrapper function inside the decorator as we did earlier.
def decorator_maker_with_arguments(decorator_arg1, decorator_arg2, decorator_arg3):
def decorator(func):
def wrapper(function_arg1, function_arg2, function_arg3) :
"This is the wrapper function"
print("The wrapper can access all the variables\n"
"\t- from the decorator maker: {0} {1} {2}\n"
"\t- from the function call: {3} {4} {5}\n"
"and pass them to the decorated function"
.format(decorator_arg1, decorator_arg2,decorator_arg3,
function_arg1, function_arg2,function_arg3))
return func(function_arg1, function_arg2,function_arg3)
return wrapper
return decorator
pandas = "Pandas"
@decorator_maker_with_arguments(pandas, "Numpy","Scikit-learn")
def decorated_function_with_arguments(function_arg1, function_arg2,function_arg3):
print("This is the decorated function and it only knows about its arguments: {0}"
" {1}" " {2}".format(function_arg1, function_arg2,function_arg3))
decorated_function_with_arguments(pandas, "Science", "Tools")
The wrapper can access all the variables
- from the decorator maker: Pandas Numpy Scikit-learn
- from the function call: Pandas Science Tools
and pass them to the decorated function
This is the decorated function, and it only knows about its arguments: Pandas Science Tools
Debugging Decorators
As we have noticed, decorators wrap functions. The original function name, its docstring, and parameter list are all hidden by the wrapper closure: For example, when we try to access the decorated_function_with_arguments
metadata, we'll see the wrapper closure's metadata. This presents a challenge when debugging.
decorated_function_with_arguments.__name__
'wrapper'
decorated_function_with_arguments.__doc__
'This is the wrapper function'
In order to solve this challenge Python provides a functools.wraps
decorator. This decorator copies the lost metadata from the undecorated function to the decorated closure. Let's show how we'd do that.
import functools
def uppercase_decorator(func):
@functools.wraps(func)
def wrapper():
return func().upper()
return wrapper
@uppercase_decorator
def say_hi():
"This will say hi"
return 'hello there'
say_hi()
'HELLO THERE'
Now, when we check say_hi
's metadata, we see that it correctly reflects the original function—not the wrapper.
say_hi.__name__
'say_hi'
say_hi.__doc__
'This will say hi'
It is advisable and good practice to always use functools.wraps
when defining decorators. It will save you a lot of headache in debugging.
Class-Based Decorators
While function-based decorators are common, Python also allows you to create class-based decorators, which provide greater flexibility and maintainability, especially for complex use cases. A class-based decorator is a class with a __call__
method that allows it to behave like a function.
class UppercaseDecorator:
def __init__(self, function):
self.function = function
def __call__(self, *args, **kwargs):
result = self.function(*args, **kwargs)
return result.upper()
@UppercaseDecorator
def greet():
return "hello there"
print(greet())
# Output: HELLO THERE
How it works:
- The
__init__
method initializes the decorator with the function to be decorated. - The
__call__
method is invoked when the decorated function is called, allowing the decorator to modify its behavior.
Advantages of class-based decorators:
- Stateful decorators: Class-based decorators can maintain state using instance variables, unlike function-based decorators which require closures or global variables.
- Readability: For complex decorators, encapsulating logic in a class can make the code more organized and easier to understand.
Example of a stateful decorator:
class CallCounter:
def __init__(self, function):
self.function = function
self.count = 0
def __call__(self, *args, **kwargs):
self.count += 1
print(f"Function {self.function.__name__} has been called {self.count} times.")
return self.function(*args, **kwargs)
@CallCounter
def say_hello():
print("Hello!")
say_hello()
say_hello()
# Output:
# Function say_hello has been called 1 times.
# Hello!
# Function say_hello has been called 2 times.
# Hello!
Real-World Decorator Use Case: Caching
The lru_cache
decorator is a built-in tool in Python that caches the results of expensive function calls. This improves performance by avoiding redundant calculations for repeated inputs.
Example:
from functools import lru_cache
@lru_cache(maxsize=128)
def fibonacci(n):
if n < 2:
return n
return fibonacci(n - 1) + fibonacci(n - 2)
print(fibonacci(50)) # Subsequent calls with the same argument are much faster
Other common uses for decorators:
-
Logging: Track function calls, arguments, and return values for debugging or auditing.
-
Authentication: Enforce access control in web applications like Flask or Django.
-
Execution timing: Measure and optimize function execution time for performance-critical tasks.
-
Retry mechanism: Automatically retry failed function calls, useful in network operations.
-
Input validation: Validate function arguments before execution.
Python Decorators Summary
Decorators dynamically alter the functionality of a function, method, or class without having to directly use subclasses or change the source code of the function being decorated. Using decorators in Python also ensures that your code is DRY(Don't Repeat Yourself). Decorators have several use cases such as:
- Authorization in Python frameworks such as Flask and Django
- Logging
- Measuring execution time
- Synchronization
To learn more about Python decorators check out Python's Decorator Library.
Become a Python Developer
FAQs
Are there any performance considerations when using decorators?
Yes, decorators can add overhead because they introduce additional function calls. When performance is critical, it's important to consider this overhead, especially if the decorated function is called frequently in a performance-sensitive context.
Can decorators be used with class methods, and if so, how?
Yes, decorators can be applied to class methods just like regular functions. The decorator will receive the method as its argument and return a new method or a modified version of the method. This is commonly used for logging, access control, or enforcing preconditions.
How can decorators be used for logging purposes?
Decorators can be used to log function calls, their arguments, and return values by wrapping the function execution with code that records these details to a logging system. This helps in tracing and debugging.
What is the significance of the @ symbol in decorators?
The @
symbol is syntactic sugar in Python that simplifies applying a decorator to a function. It allows you to apply a decorator to a function directly above its definition, making the code cleaner and more readable.
Can a decorator modify the return value of a function, and how would that work?
Yes, a decorator can modify the return value of a function by altering the return statement within the wrapper function. For instance, it could transform the output data type, format it, or add additional processing before returning the final result.
How does Python handle variable scope when a nested function accesses a variable from its enclosing function?
Python uses a LEGB (Local, Enclosing, Global, Built-in) scope rule. In the case of nested functions, the nested function can access variables from its enclosing function's scope, which allows for closures where the inner function retains access to the outer function's variables even after the outer function has finished executing.
What’s the difference between a closure and a decorator?
A closure is a function that remembers variables from its outer scope. A decorator is a function that uses closures to enhance or wrap other functions.
When should I use functools.wraps?
Always use functools.wraps
when writing decorators. It preserves the original function’s metadata (like name and docstring), which is helpful for debugging and documentation.
Can decorators accept arguments?
Yes! You can pass arguments to decorators by wrapping them in another function (a decorator factory). This lets you customize decorator behavior.
What’s the advantage of class-based decorators?
Class-based decorators allow you to maintain state across function calls and organize more complex logic in an object-oriented way.