Introducing Python Magic Methods

One of the contributing factors that make Python such a powerful programming language is magic methods. In some Pythonista circles, you may hear them being referred to as ‘dunder methods’ in light of their double underscores construct. 

Their role in Python is to carry out a procedure known as ‘operator overloading,’ which simply means that a predefined operator is provided with extended meaning. In other words, operator overloading is used to add custom behaviors to our classes to enable their usage with Python’s operators and built-in functionality.  

In this article, we will delve deeper into Python magic methods by: 

• Distinguishing the difference between methods and functions 
• Explaining what magic methods are
• Providing some Python examples. 

Check out this DataLab workbook to follow along with the code used in this tutorial. 

Python Methods vs. Functions

Before we can get into what magic methods are, we must understand the differences between a method and a function in Python. 

Python uses methods to represent the behavior of an object. Thus, methods are associated with an object and will perform some operations upon the data within an object. Another key distinguishing factor of a method is the self parameter that must always occur as the first parameter passed to the method. 

Note: It doesn’t have to be the word  self as the first parameter, but Pythonistas universally agreed it’s a best practice. 

# Example of a method
class Human:
    # This is a method
    def walk(self):
        print("I am walking") 

Methods are dependent on the class they are associated with. Thus, to call a method, an instance of the class must be created before the method can be accessed with the dot operator. 

# This will work
joe = Human()
joe.walk()

"""
I am walking
"""




# This will throw a NameError
walk()

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
/tmp/ipykernel_153/2327344440.py in <module>
      4
      5 # This will throw a NameError
----> 6 walk()

NameError: name 'walk' is not defined

A function, on the other hand, is different: we do not have to create an instance of a class. 

Python functions are steps executed as a block of code only when called. They are defined independently of any class object, meaning they can be called directly by their name. Since they are not associated with any class, we do not have to pass self as a parameter – parameters are optional. 

In essence, a function is simply a set of well-defined instructions the interpreter uses to perform a task. 

# Example of a function
def run():
  print("I am running.")
​
run()

"""
I am running.
"""

As explored in our How to call and write functions article, functions also come in two different flavors: built-ins and user-defined. 

• Built-in functions: Functions that are included in the Python standard library. 
• User-defined functions: Functions defined by the user - Python permits users to define and use their own functions that may be accessed in a Python program.  

In many respects, methods and functions are identical - i.e., they are both defined with a def statement. The underlying difference is that methods are associated with an object and are used to operate on the data that’s contained within a class, whereas functions are defined independently and may be called by their name anywhere in a program. 

What are Python magic methods?

At the beginning of the tutorial, we mentioned that magic methods are also referred to as dunder methods due to their double underscore construct. What we meant by this is all Python magic methods begin and end with two underscores. 

The most well-recognized magic method is the __init__ method, which enables a class to be initialized with specific attributes. 

class Employee:
    def __init__(self, name, age, role):
        self.name = name
        self.age = age
        self.role = role

To instantiate an instance of this class, we must specify the instance's name, age, and role attributes. For example:

user_1 = Employee(name="John", age=24, role="Data Scientist")

Failure to do so would raise a TypeError. 

The __init__ method falls under the initialization and construction magic method category, but it’s only one of many. The table below details some other initialization and construction magic methods. 

The simplest way to conceptualize magic methods may be as a contact between your implementation and the python interpreter. One of the terms in the contract requires Python to perform some actions under the hood from some given circumstance, such as initializing a class instance with specific objects. 

In other words, it’s not your responsibility to invoke a magic method directly by calling it - Python does it for you behind the scenes. 

String representation methods

Python is an object-oriented programming (OOP) language – everything in Python is an object. When a new object is created, we implicitly create a related object because all classes inherit from object.  

Note: you can also make the inheritance explicit by passing the class object to your class. 

# This works
class Car:
  pass
​
# This also works
class Vehicle(object):
  pass
​
car = Car()
vehicle = Vehicle()
​
print(car, vehicle, sep="\n")

"""
<__main__.Car object at 0x7f80449dc0a0>
<__main__.Vehicle object at 0x7f80449dc1f0>
"""

In the code above, we created two empty classes, but the Python interpreter still knew what to print when we called the print statement. This is possible because all classes inherit from the object class (the parent of all classes in Python), and it has a magic method called __repr__, of which the print statement calls to return a value back to our main program.  

Those familiar with inheritance would know that a child class can override its parent class by defining its own method with the same name. 

# Overriding the object __repr__
class Vehicle:
  def __repr__(self):
      return f"{self.__class__.__qualname__}"
​
vehicle = Vehicle()
print(vehicle)

"""
Vehicle
"""

You may also use the __str__ magic method to create a string representation that is called with a print statement. 

In fact, the __repr__ magic method serves as the backup behavior to the __str__ magic method. Thus, when we call the print statement, Python first looks for the __str__ method defined in your class before falling back on the __repr__ method. 

# Using __str__ for string representation
class Vehicle:
  def __str__(self):
      return f"{self.__class__.__qualname__}"
     
vehicle = Vehicle()
print(vehicle)

"""
Vehicle
"""

The table below consists of a list of other string representation magic methods. 

Encapsulation “magic” methods 

Encapsulation is one of the fundamental concepts in object-oriented programming (OOP). It refers to the bundling of data with the methods that operate on that data or restricting direct access to certain components within an object.

Developers coming from other OOP languages regularly contest Python’s failure to embrace encapsulation for classes truly. These people are accustomed to having explicit modifiers for methods or fields that enable them to define things such as private attributes with public getters and setters. The difference in Python is that much of the existing encapsulation occurs via magic methods. 

For example, __setattr__ is an encapsulation solution that enables us to define behavior for assignment to an attribute despite its presence. In other words, rules can be defined for changes in the values of attributes. 

import string

class Alphabet():
    def __setattr__(self, name, value):
        self.__dict__[name] = value.upper()

a = Alphabet()

# Assinging an attribute that does not exist
# The setattr should convert it to caps despite being set as lowercase
a.alphabet = string.ascii_lowercase
print(a.alphabet)

In the code above, we created a class called Alphabet with no attributes, but we told Python we wanted to capitalize any attribute of the class instance we set.

We understand OOP may be a little out of reach for some readers. To get up to scratch with OOP in Python, check out DataCamp’s Python Programmer Track.   

Math magic methods

Another category of Python magic methods are the math magic methods (also known as normal arithmetic operators). Expressions are created using special symbols we call ‘operators’ in Python. Using such operators in an expression involves creating objects with math magic methods. Failure to abide by this rule will raise a TypeError. 

class Numbers:
  def __init__(self, a, b):
      self.a = a
      self.b = b
   
set_a = Numbers(2, 4)
set_b = Numbers(3, 5)
​
print(set_a + set_b)
"""
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_152/3639634723.py in <module>
      7 set_b = Numbers(3, 5)
      8
----> 9 print(set_a + set_b)

TypeError: unsupported operand type(s) for +: 'Numbers' and 'Numbers'
"""

In the code above, we attempted to create multiple instances of the class Numbers and add them together which raised an error as expected. One solution to this error could be to create a method in our class to add numbers together, but this would prevent us from creating expressions. 

A better solution would be to implement a __add__ magic method as follows:  

# Using add operator
class Numbers:
    def __init__(self, a, b):
        self.a = a
        self.b = b
   
    def __add__(self, other):
        # Only permit Numbers objects to be added
        if not isinstance(other, Numbers):
            return NotImplemented
        return Numbers(other.a + self.a, other.b + self.b)
   
    def __repr__(self):
      return f"{self.__class__.__qualname__}({self.a}, {self.b})"
   
a = Numbers(2, 4)
b = Numbers(3, 5)
print(a + b) # Should result in Numbers(5, 9)


"""
RandomNumbers(5, 9)
"""

The code example above only enables us to add two Numbers objects together. Let’s implement another math magic method that permits us to multiply a Numbers object with an integer. 

# Multiplying Numbers with an integer
class Numbers:
    def __init__(self, a, b):
        self.a = a
        self.b = b
   
    def __add__(self, other):
        # Only permit Numbers objects to be added
        if not isinstance(other, Numbers):
            return NotImplemented
        return Numbers(other.a + self.a, other.b + self.b)
   
    def __mul__(self, other):
      if not isinstance(other, int):
          return NotImplemented
         
      return Numbers(self.a * other, self.b * other)
   
    def __repr__(self):
      return f"{self.__class__.__qualname__}({self.a}, {self.b})"
   
a = Numbers(2, 4)
print(a * 5) # Should return 10, 20




"""
RandomNumbers(10, 20)
"""

In the code above, we added a __mul__ magic method to allow us to multiply the numbers of our custom class by an integer. 

Notice the Numbers object occurs on the left side of the * operator - this is not by accident. Let’s put it on the right side of the operand and observe what happens. 

- snip --

a = Numbers(2, 4)
print(3 * a)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/tmp/ipykernel_152/2857655049.py in <module>
      1 a = Numbers(2, 4)
----> 2 print(3 * a)

TypeError: unsupported operand type(s) for *: 'int' and 'Numbers'

Python throws a TypeError. 

The __mul__ magic method we implemented only accounts for when the Numbers object is on the left-hand side of the * operator. To extend this functionality, we must add the reverse magic method, __rmul__. 

# Enabling reverse multiplication
class Numbers:
    def __init__(self, a, b):
        self.a = a
        self.b = b
   
    def __add__(self, other):
        # Only permit Numbers objects to be added
        if not isinstance(other, Numbers):
            return NotImplemented
        return Numbers(other.a + self.a, other.b + self.b)
   
    def __mul__(self, other):
      if not isinstance(other, int):
          return NotImplemented
      return Numbers(self.a * other, self.b * other)

    def __rmul__(self, other):
        return self.__mul__(other)
   
    def __repr__(self):
      return f"{self.__class__.__qualname__}({self.a}, {self.b})"
   
a = Numbers(2, 4)
print(5 * a) # Should return 10, 20

"""
Numbers(10, 20)
"""

Now we can create a multiplication expression without having to consider what side of the operand the Numbers object appears on. 

Wrap up

The basic principle for leveraging Python magic methods stays the same: create a class and override the object class magic method. There are so many magic methods in Python, meaning it would be an extremely long tutorial if we decided to cover them all individually. We highly recommend you check out the Python documentation to get an idea of the full scope of Python magic methods you could use to enhance your Python programming skills.