Project Description

In this Project, you'll scrape the novel Moby Dick from the website Project Gutenberg (which contains a large corpus of books) using the Python package requests. Then you'll extract words from this web data using BeautifulSoup. Finally, we'll dive into analyzing the distribution of words using the Natural Language ToolKit (nltk). The natural language processing tools used here apply to much of the data that data scientists encounter as a vast proportion of the world's data is unstructured data and includes a great deal of text.

To complete this project, you need to know how to import web data into python and how to work with natural language text. Before starting this project we recommend that you have completed the following courses:

This Project is based on a live screencast by DataCamp's own Hugo Bowne-Anderson. When you've finished the Project, or if you get stuck, do check out the screencast with Hugo's solution (the screencast starts 12 minutes into the video). You can also find Hugo's solution notebook here.

Project Tasks

  • 1 Tools for text processing
  • 2 Request Moby Dick
  • 3 Get the text from the HTML
  • 4 Extract the words
  • 5 Make the words lowercase
  • 6 Load in stop words
  • 7 Remove stop words in Moby Dick
  • 8 We have the answer
  • 9 The most common word
Hugo Bowne-Anderson
Hugo Bowne-Anderson

Data Scientist at DataCamp

Hugo hearts all things Pythonic and is charged with building out DataCamp’s Python curriculum. He can be found at hackathons, meetups & code sprints, primarily in NYC. Before joining the ranks of DataCamp, he worked in applied mathematics (biology) research at Yale University.

See More
Rasmus Bååth
Rasmus Bååth

Instructor at DataCamp

Rasmus Bååth is an instructor at DataCamp and in charge of building new DataCamp Projects. Before joining DataCamp he did a PHD in Cognitive Science at Lund University, Sweden. Follow him at @rabaath on Twitter or on his blog, Publishable Stuff.

See More
  • Python