Premium project
Word Frequency in Classic Novels
Use web scraping and NLP to find the most frequent words in one of two pieces of classic literature: Herman Melville's novel, Moby Dick, or Peter Pan by J. M. Barrie.
Start Project9 Tasks1,500 XP
Loved by learners at thousands of companies
Project Description
In this Project, you'll scrape a novel from the website Project Gutenberg (which contains a large corpus of books) using the Python package requests. Then you'll extract words from this web data using BeautifulSoup. Finally, we'll dive into analyzing the distribution of words using the Natural Language ToolKit (nltk).
The natural language processing tools used here apply to much of the data that data scientists encounter as a vast proportion of the world's data is unstructured data and includes a great deal of text. To complete this Project, you need to know how to import web data into Python and how to work with natural language text.
Project Tasks
- 1Tools for text processing
- 2Request Moby Dick
- 3Get the text from the HTML
- 4Extract the words
- 5Make the words lowercase
- 6Load in stop words
- 7Remove stop words in Moby Dick
- 8We have the answer
- 9The most common word
Technologies
Python
Hugo Bowne-Anderson
Data Scientist at DataCamp
Hugo is a data scientist, educator, writer and podcaster at DataCamp. His main interests are promoting data & AI literacy, helping to spread data skills through organizations and society and doing amateur stand up comedy in NYC. If you want to know what he likes to talk about, definitely check out DataFramed, the DataCamp podcast, which he hosts and produces: https://www.datacamp.com/community/podcast