Skip to main content
Premium project

Word Frequency in Classic Novels

Use web scraping and NLP to find the most frequent words in one of two pieces of classic literature: Herman Melville's novel, Moby Dick, or Peter Pan by J. M. Barrie.

Start Project
9 Tasks1,500 XP

Loved by learners at thousands of companies

Project Description

In this Project, you'll scrape a novel from the website Project Gutenberg (which contains a large corpus of books) using the Python package `requests`. Then you'll extract words from this web data using `BeautifulSoup`. Finally, we'll dive into analyzing the distribution of words using the Natural Language ToolKit (`nltk`). The natural language processing tools used here apply to much of the data that data scientists encounter as a vast proportion of the world's data is unstructured data and includes a great deal of text. To complete this Project, you need to know how to import web data into Python and how to work with natural language text.

Project Tasks

  1. 1
    Tools for text processing
  2. 2
    Request Moby Dick
  3. 3
    Get the text from the HTML
  4. 4
    Extract the words
  5. 5
    Make the words lowercase
  6. 6
    Load in stop words
  7. 7
    Remove stop words in Moby Dick
  8. 8
    We have the answer
  9. 9
    The most common word

Technologies

Python Python

Topics

Importing & Cleaning DataCase Studies
Hugo Bowne-Anderson Headshot

Hugo Bowne-Anderson

Data Scientist at DataCamp

Hugo is a data scientist, educator, writer and podcaster at DataCamp. His main interests are promoting data & AI literacy, helping to spread data skills through organizations and society and doing amateur stand up comedy in NYC. If you want to know what he likes to talk about, definitely check out DataFramed, the DataCamp podcast, which he hosts and produces: https://www.datacamp.com/community/podcast
See More

What do other learners have to say?

I've used other sites—Coursera, Udacity, things like that—but DataCamp's been the one that I've stuck with.

Devon Edwards Joseph
Lloyds Banking Group

DataCamp is the top resource I recommend for learning data science.

Louis Maiden
Harvard Business School

DataCamp is by far my favorite website to learn from.

Ronald Bowers
Decision Science Analytics, USAA