Skip to main content

How to Drop Columns in Pandas Tutorial

Learn how to drop columns in a pandas DataFrame.
Aug 2020  · 3 min read

Often, a DataFrame will contain columns that are not useful to your analysis. Such columns should be dropped from the DataFrame to make it easier for you to focus on the remaining columns.

The columns can be removed by specifying label names and corresponding axis, or by specifying index or column names directly. When using a multi-index, labels on different levels can be removed by specifying the level.

.drop() Method

Let's compare missing value counts with the shape of the dataframe. You will notice that the county_name column contains as many missing values as rows, meaning that it only contains missing values.

ri.isnull().sum()
state                            0
stop_date                        0
stop_time                        0
county_name                  91741
driver_gender                 5205
driver_race                   5202
...
ri.shape
91741, 15

Since it contains no useful information, this column can be dropped using the .drop() method.

Besides specifying the column name, you need to specify that you are dropping from the columns axis and that you want the operation to occur in place, which avoids an assignment statement as shown below:

ri.drop('county_name',
  axis='columns', inplace=True)

.dropna() Method

The .dropna() method is a great way to drop rows based on the presence of missing values in that row.

For example, using the dataset above, let's assume the stop_date and stop_time columns are critical to our analysis, and thus a row is useless to us without that data.

ri.head()
    state   stop_date    stop_time    driver_gender   driver_race
0      RI  2005-01-04        12:55                M         White
1      RI  2005-01-23        23:15                M         White
2      RI  2005-02-17        04:15                M         White
3      RI  2005-02-20        17:15                M         White
4      RI  2005-02-24        01:20                F         White

We can tell pandas to drop all rows that have a missing value in either the stop_date or stop_time column. Because we specify a subset, the .dropna() method only takes these two columns into account when deciding which rows to drop.

ri.dropna(subset=['stop_date', 'stop_time'], inplace=True)

Interactive Example of Dropping Columns

In this example, you will drop the county_name column because it only contains missing values, and you'll drop the state column because all of the traffic stops took place in one state (Rhode Island). Thus, these columns can be dropped because they contain no useful information. The number of missing values in each column has been printed to the console for you.

  • Examine the DataFrame's .shape to find out the number of rows and columns.
  • Drop both the county_name and state columns by passing the column names to the .drop() method as a list of strings.
  • Examine the .shape again to verify that there are now two fewer columns.
# Examine the shape of the DataFrame
print(ri.shape)

# Drop the 'county_name' and 'state' columns
ri.drop(['county_name', 'state'], axis='columns', inplace=True)

# Examine the shape of the DataFrame (again)
print(ri.shape)

When you run the above code, it produces the following result:

(91741, 15)
(91741, 13)

Try it for yourself.

To learn more about dropping columns in pandas, please see this video from our course, Analyzing Police Activity with pandas.

This content is taken from DataCamp’s Analyzing Police Activity with pandas course by Kevin Markham.

Check out our Pandas Add Column Tutorial.

Introduction to Python

Beginner
4 hours
4,591,248
Master the basics of data analysis with Python in just four hours. This online course will introduce the Python interface and explore popular packages.
See DetailsRight Arrow
Start Course

Intermediate Python

Beginner
4 hours
882,021
Level up your data science skills by creating visualizations using Matplotlib and manipulating DataFrames with pandas.

Analyzing Police Activity with pandas

Beginner
4 hours
58,868
Explore the Stanford Open Policing Project dataset and analyze the impact of gender on police behavior using pandas.
See all coursesRight Arrow
Related

The 23 Top Python Interview Questions & Answers

Essential Python interview questions with examples for job seekers, final-year students, and data professionals.
Abid Ali Awan's photo

Abid Ali Awan

22 min

Working with Dates and Times in Python Cheat Sheet

Working with dates and times is essential when manipulating data in Python. Learn the basics of working with datetime data in this cheat sheet.
DataCamp Team's photo

DataCamp Team

Plotly Express Cheat Sheet

Plotly is one of the most widely used data visualization packages in Python. Learn more about it in this cheat sheet.
DataCamp Team's photo

DataCamp Team

0 min

Getting started with Python cheat sheet

Python is the most popular programming language in data science. Use this cheat sheet to jumpstart your Python learning journey.
DataCamp Team's photo

DataCamp Team

8 min

Python pandas tutorial: The ultimate guide for beginners

Are you ready to begin your pandas journey? Here’s a step-by-step guide on how to get started. [Updated November 2022]
Vidhi Chugh's photo

Vidhi Chugh

15 min

Python Iterators and Generators Tutorial

Explore the difference between Python Iterators and Generators and learn which are the best to use in various situations.
Kurtis Pykes 's photo

Kurtis Pykes

10 min

See MoreSee More