Skip to main content

Logging in Python Tutorial

Learn about the fundamentals of Logging in Python.
Mar 2020  · 9 min read

If you are just getting started in Python and would like to learn more, take DataCamp's Introduction to Data Science in Python course.

Levels of Logging
Levels of Logging

In general, logging is a way of keeping a log generated from a computer program. A log file usually has a set of records of events that occur in your code ( for example, a python code ) or an operating system. These events or messages are written to a log file, a message, or an event can also be a communication between different users.

It usually depends on the developers on how they would like to generate logs. Maybe they would like to log every operation of an application they developed, log only when an error occurs, or monitor essential operations done by a user and log them. An essential operation could be to check the application or the server was accessed by which end-user or IP address, and in case it was an unsuccessful attempt, the developer then could seek help from the logs and provide better insights than a mere stack trace.

A nice example would be when you try to do an installation on Linux as a sudo user when you are not a sudo user. You see a message "Xyz is not in the sudoers file. This incident will be reported", which is written in a log file.

The good news is that Python comes with a built-in logging module, and you can use it off the shelf without any installation.

How logging is different from a Print Function?

Logging is similar in spirit to a Print Function. However, a Print function lacks a lot of information that might be useful to a developer. Logging, on the other hand, can log the timestamps and line number at which the error occurred. It can log errors or any information to files, sockets, etc., and it offers you five types of severity based on which you can differentiate your logging.

Five Levels of Logging as shown here, are:

  • Debug (10): Useful for diagnosing issues in the code.

  • Info (20): It can act as an acknowledgment that there are no bugs in the code. One good use-case of Info level logging is the progress of training a machine learning model.

  • Warning (30): Indicative of a problem that could occur in the future. For example, a warning of a module that might be discontinued in the future or low-ram warning.

  • Error (40): A serious bug in the code, could be a syntax error, out of memory error, exceptions.

  • Critical (50): An error due to which the program might stop functioning or might exit abruptly.

Enough of theory, right?

Let's now understand how you can implement logging in a Python code.

But before that, let's see the logging attribute table, you would be using some of the below logging attributes in today's tutorial.

logging attribute table
Logging Attributes</a
import logging

Now you will call the basicConfig() method, which helps you in creating a basic configuration for the logging system to work smoothly.

It allows you to set the following parameters:

  • the severity of the logging: the five levels of logging.
  • log the events into a file.
  • log only the current logs by overwriting the old ones using the filemode.
  • file format in which logs are stored.

Remember that the basicConfig() method is optional, and you might want to skip it.

logging.basicConfig()

Let's now build the five levels of severity.

logging.debug("A Debug Logging Message")
logging.info("A Info Logging Message")
logging.warning("A Warning Logging Message")
WARNING:root:A Warning Logging Message
logging.error("An Error Logging Message")
ERROR:root:An Error Logging Message
logging.critical("A Critical Logging Message")
CRITICAL:root:A Critical Logging Message

As you can observe from the above outputs, only the ones which have severity level greater than 20 were successfully logged since that is the default configuration of logging in Python.

However, as discussed above, you can modify the severity levels using the basicConfig() method. Here you would change the severity level to 20, which means logging levels having severity greater than 10 will be logged.

Note: Kindly restart the kernel before running the below cells to see the correct output.

import logging
logging.basicConfig(level=logging.INFO)
logging.debug("A Debug Logging Message")
logging.info("A Info Logging Message")
INFO:root:A Info Logging Message
logging.warning("A Warning Logging Message")
WARNING:root:A Warning Logging Message
logging.error("An Error Logging Message")
ERROR:root:An Error Logging Message
logging.critical("A Critical Logging Message")
CRITICAL:root:A Critical Logging Message

Let's now log the above outputs in a log file. To achieve this, you would use the basicConfig() method.

import logging
logging.basicConfig(level = logging.INFO, filename = 'datacamp.log')
logging.debug("A Debug Logging Message")
logging.info("A Info Logging Message")
logging.warning("A Warning Logging Message")
logging.error("An Error Logging Message")
logging.critical("A Critical Logging Message")
output

As you can see, now the logs are being generated in a log file datacamp.log. Upon running the above cells multiple times, you would notice that logs will be added in the log file that many times. By default, the file is in append mode.

output

You can make use of the filemode keyword to prevent the log file from being written with multiple copies of logs i.e., append mode. The filemode can be changed to write mode, which will overwrite the previous logs and only save the current ones. Since the filemode is set to w, this means that the log file will be opened in write mode each time basicConfig() is run, which will ultimately overwrite the file.

The file mode functionality in logging is similar to a standard file handling that Python offers.

import logging
logging.basicConfig(level = logging.INFO, filename = 'datacamp.log', filemode = 'w')
logging.debug("A Debug Logging Message")
logging.info("A Info Logging Message")
logging.warning("A Warning Logging Message")
logging.error("An Error Logging Message")
logging.critical("A Critical Logging Message")

Let's run the above lines of code multiple times. You will notice that this time only the current log entries will be saved, and the previous ones will be overwritten, as shown below.

output

Let's look at some more logging attributes like date, time, line number at which warning or error was generated. Well, as you might agree by now that all these are advantages over a simple Print function and is far more capable than a mere Print function.

In order for you to accomplish this, you will pass the logging attributes asctime and lineno. Also you will pass the message attribute, which will be a placeholder for the message you would like to display for the logs. All these attributes will be passed to the format method.

import logging
logging.basicConfig(format='Date-Time : %(asctime)s : Line No. : %(lineno)d - %(message)s', \
                    level = logging.DEBUG)
logging.debug("A Debug Logging Message")
logging.info("A Info Logging Message")
logging.warning("A Warning Logging Message")
logging.error("An Error Logging Message")
logging.critical("A Critical Logging Message")
Date-Time : 2020-03-18 23:58:12,429 : Line No. : 1 - A Debug Logging Message
Date-Time : 2020-03-18 23:58:12,436 : Line No. : 2 - A Info Logging Message
Date-Time : 2020-03-18 23:58:12,437 : Line No. : 3 - A Warning Logging Message
Date-Time : 2020-03-18 23:58:12,440 : Line No. : 4 - An Error Logging Message
Date-Time : 2020-03-18 23:58:12,441 : Line No. : 5 - A Critical Logging Message

Let's look at the definitions of each attribute straight from this table.

  • asctime: Human-readable time when the LogRecord was generated. By default, this is of the form 2020-03-18 22:24:42,670 (the numbers after the comma are millisecond portion of the time). Since this attribute is a string, hence, you use s keyword after it.

  • lineno: Source line number where the logging call was called. Since this attribute is an integer, hence, you use d keyword after it.

  • message: The logged message, computed as msg % args. This is set when Formatter.format() is invoked. If you do not specify this attribute while using the format() parameter, the messages you wish to log will not be generated. Similarly, for a message attribute, you use s since its a string.

  • funcName: This will output the module or the function name in which the log is generated. It is useful when you have a lot of functions in your code, and you want to segregate the logs based on the function or method they belong to.

Let's take a simple example and see how you can plug in the logging debug function in place of the print function.

import logging

logging.basicConfig(format='%(asctime)s :: %(levelname)s :: %(funcName)s :: %(lineno)d \
:: %(message)s', level = logging.INFO)

def addition(x, y):
    add = x + y
    return add


def subtract(x, y):
    sub = x - y
    return sub


def multiply(x, y):
    mul = x * y
    return mul


def divide(x, y):
    div = x / y
    return div

def exponent(x, y):
    exp = x ** y
    return exp


num1 = 20
num2 = 2

def main():
    add_result = addition(num1, num2)
    logging.info('Add: {} + {} = {}'.format(num1, num2, add_result))

    sub_result = subtract(num1, num2)
    logging.info('Sub: {} - {} = {}'.format(num1, num2, sub_result))

    mul_result = multiply(num1, num2)
    logging.info('Mul: {} * {} = {}'.format(num1, num2, mul_result))

    div_result = divide(num1, num2)
    logging.info('Div: {} / {} = {}'.format(num1, num2, div_result))

    exp_result = exponent(num1, num2)
    logging.info('Exp: {} ** {} = {}'.format(num1, num2, exp_result))


main()
2020-03-19 00:01:27,372 :: INFO :: main :: 35 :: Add: 20 + 2 = 22
2020-03-19 00:01:27,376 :: INFO :: main :: 38 :: Sub: 20 - 2 = 18
2020-03-19 00:01:27,377 :: INFO :: main :: 41 :: Mul: 20 * 2 = 40
2020-03-19 00:01:27,378 :: INFO :: main :: 44 :: Div: 20 / 2 = 10.0
2020-03-19 00:01:27,379 :: INFO :: main :: 47 :: Exp: 20 ** 2 = 400

Conclusion

Congratulations on finishing the tutorial.

You might want to try out other logging attributes and understand how they can be leveraged in your code.

Please feel free to ask any questions related to this tutorial in the comments section below.

If you are just getting started in Python and would like to learn more, take DataCamp's Introduction to Data Science in Python course.

Introduction to Python

Beginner
4 hours
4,596,359
Master the basics of data analysis with Python in just four hours. This online course will introduce the Python interface and explore popular packages.
See DetailsRight Arrow
Start Course

Intermediate Python

Beginner
4 hours
883,215
Level up your data science skills by creating visualizations using Matplotlib and manipulating DataFrames with pandas.

Python Data Science Toolbox (Part 2)

Beginner
4 hours
225,461
Continue to build your modern Data Science skills by learning about iterators and list comprehensions.
See all coursesRight Arrow
Related

The 23 Top Python Interview Questions & Answers

Essential Python interview questions with examples for job seekers, final-year students, and data professionals.
Abid Ali Awan's photo

Abid Ali Awan

22 min

Working with Dates and Times in Python Cheat Sheet

Working with dates and times is essential when manipulating data in Python. Learn the basics of working with datetime data in this cheat sheet.
DataCamp Team's photo

DataCamp Team

Plotly Express Cheat Sheet

Plotly is one of the most widely used data visualization packages in Python. Learn more about it in this cheat sheet.
DataCamp Team's photo

DataCamp Team

0 min

Getting started with Python cheat sheet

Python is the most popular programming language in data science. Use this cheat sheet to jumpstart your Python learning journey.
DataCamp Team's photo

DataCamp Team

8 min

Python pandas tutorial: The ultimate guide for beginners

Are you ready to begin your pandas journey? Here’s a step-by-step guide on how to get started. [Updated November 2022]
Vidhi Chugh's photo

Vidhi Chugh

15 min

Python Iterators and Generators Tutorial

Explore the difference between Python Iterators and Generators and learn which are the best to use in various situations.
Kurtis Pykes 's photo

Kurtis Pykes

10 min

See MoreSee More