Skip to main content

Web Scraping in Python

Learn to retrieve and parse information from the internet using the Python library scrapy.

Start Course for Free
4 Hours17 Videos56 Exercises45,331 Learners
4500 XP

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA. You confirm you are at least 16 years old (13 if you are an authorized Classrooms user).

Loved by learners at thousands of companies


Course Description

The ability to build tools capable of retrieving and parsing information stored across the internet has been and continues to be valuable in many veins of data science. In this course, you will learn to navigate and parse html code, and build tools to crawl websites automatically. Although our scraping will be conducted using the versatile Python library scrapy, many of the techniques you learn in this course can be applied to other popular Python libraries as well, including BeautifulSoup and Selenium. Upon the completion of this course, you will have a strong mental model of html structure, will be able to build tools to parse html code and access desired information, and create a simple scrapy spiders to crawl the web at scale.

  1. 1

    Introduction to HTML

    Free

    Learn the structure of HTML. We begin by explaining why web scraping can be a valuable addition to your data science toolbox and then delving into some basics of HTML. We end the chapter by giving a brief introduction on XPath notation, which is used to navigate the elements within HTML code.

    Play Chapter Now
    Web Scraping Overview
    50 xp
    Web-scraping is not nonsense!
    50 xp
    HyperText Markup Language
    50 xp
    HTML tree wordy navigation
    50 xp
    From Tree to HTML
    100 xp
    Attributes
    50 xp
    Keep it Classy
    100 xp
    Finding href
    50 xp
    Crash Course in XPath
    50 xp
    Where am I?
    100 xp
    It's Time to P
    100 xp
    A classy span
    100 xp
  2. 3

    CSS Locators, Chaining, and Responses

    Learn CSS Locator syntax and begin playing with the idea of chaining together CSS Locators with XPath. We also introduce Response objects, which behave like Selectors but give us extra tools to mobilize our scraping efforts across multiple websites.

    Play Chapter Now
  3. 4

    Spiders

    Learn to create web crawlers with scrapy. These scrapy spiders will crawl the web through multiple pages, following links to scrape each of those pages automatically according to the procedures we've learned in the previous chapters.

    Play Chapter Now

In the following tracks

Python Programmer

Collaborators

David CamposMari NazaryShon Inouye

Prerequisites

Intermediate Python
Thomas Laetsch Headshot

Thomas Laetsch

Data Scientist at New York University

Since January 2016, Thomas Laetsch has been a Moore-Sloan Post-Doctoral Associate in the Center for Data Science at NYU. In 2012, he received his PhD in mathematics from the University of California, San Diego, specializing in probability, differential geometry, and functional analysis. From 2012 through 2015, he was a Visiting Assistant Professor at the University of Connecticut, working on central tendency theorems for random walks in degenerate spaces.
See More

What do other learners have to say?

I've used other sites—Coursera, Udacity, things like that—but DataCamp's been the one that I've stuck with.

Devon Edwards Joseph
Lloyds Banking Group

DataCamp is the top resource I recommend for learning data science.

Louis Maiden
Harvard Business School

DataCamp is by far my favorite website to learn from.

Ronald Bowers
Decision Science Analytics, USAA