Working with Modules in Python Tutorial

As a beginner, you start working with Python on the interpreter, later when you need to write longer programs you start writing scripts. As your program grows more in the size you may want to split it into several files for easier maintenance as well as reusability of the code. The solution to this is Modules. You can define your most used functions in a module and import it, instead of copying their definitions into different programs. A module can be imported by another program to make use of its functionality. This is how you can use the Python standard library as well.

Simply put, a module is a file consisting of Python code. It can define functions, classes, and variables, and can also include runnable code. Any Python file can be referenced as a module. A file containing Python code, for example: test.py, is called a module, and its name would be test.

There are various methods of writing modules, but the simplest way is to create a file with a .py extension which contains functions and variables.

Before you start writing your own modules, I recommend you take a look at our Intro to Python for Data Science course, which covers the basics of the Python programming language.

• The import statement in Python
• Writing Modules
• More on Import Statements
• Module Search Path
• Byte Compiled Files
• The dir() function

The import statement

To use the functionality present in any module, you have to import it into your current program. You need to use the import keyword along with the desired module name. When interpreter comes across an import statement, it imports the module to your current program. You can use the functions inside a module by using a dot(.) operator along with the module name. First, let's see how to use the standard library modules. In the example below,math module is imported into the program so that you can use sqrt() function defined in it.

import math             #You need to put this command,`import` keyword along with the name of the module you want to import
num = 4
print(math.sqrt(num))   #Use dot operator to access sqrt() inside module "math"

For efficiency reasons, each module is only imported once per interpreter session. Therefore, if you change your modules, you must restart the interpreter, if it’s just one module you want to test interactively, use reload(), for example: reload(module_name).

Writing Modules

Now that you have learned how to import a module in your program, it is time to write your own, and use it in another program. Writing a module is just like writing any other Python file. Let's start by writing a function to add/subtract two numbers in a file calculation.py.

def add(x,y):
    return (x+y)
def sub(x,y):
    return (x-y)

If you try to execute this script on the command line, nothing will happen because you have not instructed the program to do anything. Create another python script in the same directory with name module_test.py and write following code into it.

import calculation            #Importing calculation module
print(calculation.add(1,2))   #Calling function defined in add module.

If you execute module_test.py, you will see "3" as output. When the interpreter came across the import statement, it imported the calculation module in your code and then by using the dot operator, you were able to access the add() function.

More on Import Statements

Tere are more ways to import modules:

• from .. import statement
• from .. import * statement
• renaming the imported module

from .. import statement

The from..import statement allows you to import specific functions/variables from a module instead of importing everything. In the previous example, when you imported calculation into module_test.py, both the add() and sub() functions were imported. But what if you only needed the add() function in your code?

Here is an example to illustrate the use of from..import

from calculation import add
print(add(1,2))

In above example, only the add() function is imported and used. Notice the use of add()? You can now access it directly without using the module name. You can import multiple attributes as well, separating them with a comma in the import statement. Take a look at the following example:

from calculation import add,sub

from .. import * statement

You can import all attributes of a module using this statement. This will make all attributes of imported module visible in your code.

Here is an example to illustrate the use of from .. import *:

from calculation import *
print(add(1,2))
print(sub(3,2))

Note that in the professional world, you should avoid using from..import and from..import*, as it makes your code less readable.

Renaming the imported module

You can rename the module you are importing, which can be useful in cases when you want to give a more meaningful name to the module or the module name is too large to use repeatedly. You can use the as keyword to rename it. The following example explains how to use it in your program.

import calculation as cal
print(cal.add(1,2))

You saved yourself some typing time by renaming calculation as cal.
Note that you now can't use calculation.add(1,2) anymore, as calculation is no longer recognized in your program.

Module Search Path

You may need your modules to be used in different programs/projects and their physical location in the directory can be different. If you want to use a module residing in some other directory, you have some options provided by Python.
When you import a module named calculation, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named calculation.py in a list of directories given by the variable sys.path.
sys.path contains these locations:

• the directory containing the input script (or the current directory).
• PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
• the installation-dependent default.

Assume module_test.py is in the /home/datacamp/ directory, and you moved calculation.py to /home/test/. You can modify sys.path to include /home/test/ in the list of paths, in which the Python interpreter will search for the module. For this, You need to modify module_test.py in following way:

import sys
sys.path.append('/home/test/')

import calculation
print(calculation.add(1,2))

Byte Compiled Files

Importing a module increases the execution time of programs, so Python has some tricks to speed it up. One way is to create byte-compiled files with the extension .pyc.
Internally, Python converts the source code into an intermediate form called bytecode, it then translates this into the native language of your computer and then runs it. This .pyc file is useful when you import the module the next time from a different program - it will be much faster since a portion of the processing required for importing a module is already done. Also, these byte-compiled files are platform-independent.
Note that these .pyc files are usually created in the same directory as the corresponding .py files. If Python does not have permission to write to files in that directory, then the .pyc files will not be created.

The dir() function

The dir() function is used to find out all the names defined in a module. It returns a sorted list of strings containing the names defined in a module.

import calculation
print(test.add(1,2))
print(dir(calculation))

In the output, you can see the names of the functions you defined in the module, add & sub. Attribute __name__ contains the name of the module. All attributes beginning with an underscore are default python attributes associated with a module.

Conclusion

Creating a module is required for better management of code and reusability. Python provides you with some built-in modules, which can be imported by using the import keyword. Python also allows you to create your own modules and use them in your programs. It gives you byte-compiled files to overcome the cost of using modules, which makes execution faster. You can use dir() to know the attributes defined in the module being used in a program, this can be used both on pre-defined modules as well as user-defined modules.

If you would like to learn more about Python, you can take a look at our Intermediate Python for Data Science course.