Skip to main content

Python Select Columns Tutorial

Use Python Pandas and select columns from DataFrames. Follow our tutorial with code examples and learn different ways to select your data today!
Updated Nov 25, 2024  · 7 min read

To easily run all the example code in this tutorial yourself, you can create a DataLab workbookfor free that has Python pre-installed and contains all code samples.

If you have a DataFrame and would like to access or select a specific few rows/columns from that DataFrame, you can use square brackets or other advanced methods such as loc and iloc.

Selecting Columns Using Square Brackets

Now, suppose that you want to select the country column from the brics DataFrame. To achieve this, you will type brics and then the column label inside the square brackets.

Selecting a column

          country     capital        area    population
BR         Brazil    Brasilia       8.516        200.40
RU         Russia      Moscow      17.100        143.50
IN         India    New Dehli       3.286       1252.00
CH         China      Beijing       9.597       1357.00
SA  South Africa     Pretoria       1.221         52.98

brics["country"]

BR         Brazil
RU         Russia
IN         India
CH         China
SA  South Africa
Name: country, dtype: object

Checking the type of the object

Let's check the type of the object that gets returned with the type() function.

type(brics["country"])

pandas.core.series.Series

As we can see from the above output, we are dealing with a pandas series here! Series could be thought of as a one-dimensional array that could be labeled just like a DataFrame.

If you want to select data and keep it in a DataFrame, you will need to use double square brackets:

brics[["country"]]

BR         Brazil
RU         Russia
IN         India
CH         China
SA  South Africa

If we check the type of this output, it's a DataFrame! With only one column, though.

type(brics[["country"]])

pandas.core.frame.DataFrame

Understanding the .shape attribute

Let's now look at the .shape attribute. The .shape attribute in pandas provides a quick way to see the structure of a DataFrame or Series. It returns a tuple that indicates the number of rows and columns in the DataFrame or just the number of rows in a Series.

Example: Using .shape with a DataFrame

Let’s look again at our DataFrame called brics:

import pandas as pd

# Sample DataFrame
data = {
    "country": ["Brazil", "Russia", "India", "China", "South Africa"],
    "capital": ["Brasilia", "Moscow", "New Delhi", "Beijing", "Pretoria"],
    "area": [8.516, 17.1, 3.286, 9.597, 1.221],
    "population": [200.4, 143.5, 1252, 1357, 52.98],
}
brics = pd.DataFrame(data, index=["BR", "RU", "IN", "CH", "SA"])

# Check the shape
print(brics.shape)

Output:

(5, 4)

This tells us that the brics DataFrame has 5 rows and 4 columns.

Example: Using .shape with a Series

When selecting a single column from the DataFrame, such as the country column, the result is a Series. Here’s how .shape behaves in that case:

country_series = brics["country"]
print(country_series.shape)

Output:

(5,)

Here, Series has 5 rows and no columns since a Series is one-dimensional.

To recap: For a DataFrame, .shape returns a tuple with the number of rows and columns. For a Series, it returns the number of rows. Use .shape to quickly check the size of your dataset or verify the results of filtering and selection. Also, remember that .shape is an attribute, so you don’t need parentheses to access it. 

Selecting multiple columns

You can extend this call to select two columns. Let's try to select country and capital.

brics[["country", "capital"]]

          country     capital
BR         Brazil    Brasilia
RU         Russia      Moscow
IN         India    New Dehli
CH         China      Beijing
SA  South Africa     Pretoria

If you look at this closely, you are actually putting a list with column labels inside another set of square brackets and end up with a sub DataFrame containing only the country and capital columns.

Learn Python From Scratch

Master Python for data science and gain in-demand skills.
Start Learning for Free

Selecting Rows Using Square Brackets

Square brackets can do more than just selecting columns. You can also use them to get rows, or observations, from a DataFrame.

Example

You can only select rows using square brackets if you specify a slice, like 0:4. Also, you're using the integer indexes of the rows here, not the row labels!

To get the second, third, and fourth rows of brics DataFrame, we use the slice 1 through 4. Remember that end the of the slice is exclusive, and the index starts at zero.

brics[1:4]

          country     capital     area    population
RU         Russia      Moscow   17.100        143.50
IN         India    New Delhi    3.286       1252.00
CH         China      Beijing    9.597       1357.00

These square brackets work, but they only offer limited functionality. Ideally, we would want something similar to 2D Numpy arrays, where you also use square brackets. The index, or slice, before the comma refers to the rows, and the slice after the comma refers to the columns.

Example of 2D Numpy array:

my_array[rows, columns]

If you want to do something similar with pandas, you need to look at using the loc and iloc functions.

  • loc: label-based

  • iloc: integer position-based

loc Function

loc is a technique to select parts of your data based on labels. Let's look at the brics DataFrame and get the rows for Russia.

To achieve this, you will put the label of interest in square brackets after loc.

Selecting rows

brics.loc["RU"]

country     Russia
capital     Moscow
area          17.1
population   143.5
Name: RU, dtype: object

We get a pandas series containing all of the rows information; inconveniently, though, it is shown on different lines. To get a DataFrame, we have to put the RU sting in another pair of brackets. We can also select multiple rows at the same time. Suppose you want to also include India and China. Simply add those row labels to the list.

brics.loc[["RU", "IN", "CH"]]

          country     capital     area    population
RU         Russia      Moscow   17.100        143.50
IN         India    New Delhi    3.286       1252.00
CH         China      Beijing    9.597       1357.00

The difference between using a loc and basic square brackets is that you can extend the selection with a comma and a specification of the columns of interest.

Selecting rows and columns

Let's extend the previous call to only include the country and capital columns. We add a comma and list the column labels we want to keep. The intersection gets returned.

brics.loc[["RU", "IN", "CH"], ["country", "capital"]]

          country     capital
RU         Russia      Moscow
IN         India    New Delhi
CH         China      Beijing

You can also use loc to select all rows but only a specific number of columns. Simply replace the first list that specifies the row labels with a colon. A slice going from beginning to end. This time, we get back all of the rows but only two columns.

Selecting All Rows and Specific Columns

brics.loc[:, ["country", "capital"]]

          country     capital
BR         Brazil    Brasilia
RU         Russia      Moscow
IN         India    New Delhi
CH         China      Beijing
SA  South Africa     Pretoria

iloc Function

The iloc function allows you to subset pandas DataFrames based on their position or index.

Selecting rows

Let's use the same data and similar examples as we did for loc. Let's start by getting the row for Russia.

brics.iloc[[1]]

          country     capital     area    population
RU         Russia      Moscow   17.100        143.50

To get the rows for Russia, India, and China. You can now use a list of index 1, 2, 3.

brics.iloc[[1, 2, 3]]

          country     capital     area    population
RU         Russia      Moscow   17.100        143.50
IN         India    New Delhi    3.286       1252.00
CH         China      Beijing    9.597       1357.00

Selecting rows and columns

Similar to loc, we can also select both rows and columns using iloc. Here, we will select rows for Russia, India, and China and columns country and capital.

brics.iloc[[1, 2, 3], [0, 1]]

          country     capital
RU         Russia      Moscow
IN         India    New Delhi
CH         China      Beijing

Selecting all rows and specific columns

Finally, if you wanted to select all rows but just keep the country and capital columns, you can:

brics.loc[:, [0, 1]]

          country     capital
BR         Brazil    Brasilia
RU         Russia      Moscow
IN         India    New Delhi
CH         China      Beijing
SA  South Africa     Pretoria

loc and iloc functions are pretty similar. The only difference is how you refer to columns and rows.

Interactive Example on Selecting a Subset of Data

In the following example, the cars data is imported from a CSV files as a Pandas DataFrame. To select only the cars_per_cap column from cars, you can use:

cars['cars_per_cap']
cars[['cars_per_cap']]

The single bracket version gives a Pandas Series; the double bracket version gives a Pandas DataFrame.

  • You will use single square brackets to print out the country column of cars as a Pandas Series.
  • Then use double square brackets to print out the country column of cars as a Pandas DataFrame.
  • Finally, use the double square brackets to print out a DataFrame with both the country and drives_right columns of cars, in this order.
# Import cars data
import pandas as pd
cars = pd.read_csv('cars.csv', index_col = 0)

# Print out country column as Pandas Series
print(cars['country'])

# Print out country column as Pandas DataFrame
print(cars[['country']])

# Print out DataFrame with country and drives_right columns
print(cars[['country', 'drives_right']])

When we run the above code, it produces the following result:

US     United States
AUS        Australia
JPN            Japan
IN             India
RU            Russia
MOR          Morocco
EG             Egypt
Name: country, dtype: object
           country
US   United States
AUS      Australia
JPN          Japan
IN           India
RU          Russia
MOR        Morocco
EG           Egypt
           country  drives_right
US   United States          True
AUS      Australia         False
JPN          Japan         False
IN           India         False
RU          Russia          True
MOR        Morocco          True
EG           Egypt          True

Try it for yourself.

To learn more about pandas, please see this video from our course Intermediate Python.

This content is taken from DataCamp’s Intermediate Python course by Hugo Bowne-Anderson.

Topics
Python
Data Analysis
Data Literacy

Python Courses

course

Intermediate Python

4 hr
1.2M
Level up your data science skills by creating visualizations using Matplotlib and manipulating DataFrames with pandas.
See DetailsRight Arrow
Start Course
See MoreRight Arrow
Related

cheat-sheet

Pandas Cheat Sheet for Data Science in Python

A quick guide to the basics of the Python data analysis library Pandas, including code samples.
Karlijn Willems's photo

Karlijn Willems

4 min

tutorial

Pandas Tutorial: DataFrames in Python

Explore data analysis with Python. Pandas DataFrames make manipulating your data easy, from selecting or replacing columns and indices to reshaping your data.
Karlijn Willems's photo

Karlijn Willems

20 min

tutorial

How to Drop Columns in Pandas Tutorial

Learn how to drop columns in a pandas DataFrame.
DataCamp Team's photo

DataCamp Team

3 min

tutorial

Pandas Add Column Tutorial

You are never stuck with just the data you are given. Instead, you can add new columns to a DataFrame.
DataCamp Team's photo

DataCamp Team

4 min

tutorial

Pandas Apply Tutorial

Learn what Python pandas .apply is and how to use it for DataFrames. Learn how to iterate over DataFrames using the .apply() function today!
DataCamp Team's photo

DataCamp Team

3 min

tutorial

Joining DataFrames in pandas Tutorial

In this tutorial, you’ll learn various ways in which multiple DataFrames could be merged in python using Pandas library.
DataCamp Team's photo

DataCamp Team

19 min

See MoreSee More