课程

Web Scraping in R

中级技能水平

更新时间 2024年4月

Learn how to efficiently collect and download data from any website using R.

免费开始课程

RData Preparation4 小时13 视频45 练习3,600 经验值14,861成就声明

创建您的免费帐户

或

继续操作即表示您接受我们的《使用条款》和《隐私政策》，并同意您的数据存储在美国。

深受数千家公司学习者的喜爱

培训2人或更多？

试用DataCamp for Business

课程描述

Have you ever come across a website that displays a lot of data such as statistics, product reviews, or prices in a format that’s not data analysis-ready? Often, authorities and other data providers publish their data in neatly formatted tables. However, not all of these sites include a download button, but don’t despair. In this course, you’ll learn how to efficiently collect and download data from any website using R. You'll learn how to automate the scraping and parsing of Wikipedia using the rvest and httr packages. Through hands-on exercises, you’ll also expand your understanding of HTML and CSS, the building blocks of web pages, as you make your data harvesting workflows less error-prone and more efficient.

先决条件

Intermediate R Introduction to the Tidyverse

1

Introduction to HTML and Web Scraping

In this chapter, you'll be introduced to Hyper Text Markup Language (HTML), a declarative language used to structure modern websites. Using the rvest library, you'll learn how to query simple HTML elements and scrape your first table.

Introduction to HTML

50 经验值

Read in HTML

100 经验值

Beware of syntax errors!

50 经验值

Navigating HTML

50 经验值

Select all children of a list

100 经验值

Parse hyperlinks into a data frame

100 经验值

Scrape your first table

50 经验值

The right order of table elements

100 经验值

Turn a table into a data frame with html_table()

100 经验值

2

Navigation and Selection with CSS

Cascading Style Sheets (CSS) describe how HTML elements are displayed on a web page, including colors, fonts, and general layout. In this chapter, you'll learn why CSS selectors and combinators are a crucial ingredient for web scraping.

Introduction to CSS

50 经验值

Select multiple HTML types

100 经验值

Order CSS selectors by the number of results

100 经验值

CSS classes and IDs

50 经验值

Identify the correct selector types

100 经验值

Leverage the uniqueness of IDs

100 经验值

Select the last child with a pseudo-class

100 经验值

CSS combinators

50 经验值

Select direct descendants with the child combinator

100 经验值

How many elements get returned?

50 经验值

Simply the best!

100 经验值

Not every sibling is the same

100 经验值

3

Advanced Selection with XPATH

The CSS selectors you got to know in the last chapter are powerful but have their limitations. For example, if you want to select nodes based on the properties of their descendants. XPath to the rescue! Using this query language, you can navigate and scrape even the most hideous HTML.

Introduction to XPATH

50 经验值

Find the correct CSS equivalent

100 经验值

Select by class and ID with XPATH

100 经验值

Use predicates to select nodes based on their children

100 经验值

XPATH functions and advanced predicates

50 经验值

Find a more elegant XPATH alternative

50 经验值

Get to know the position() function

100 经验值

Extract nodes based on the number of their children

100 经验值

The XPATH text() function

50 经验值

The shortcomings of html_table() with badly structured tables

100 经验值

Select directly from a parent element with XPATH's text()

100 经验值

Combine extracted data into a data frame

100 经验值

Scrape an element based on its text

100 经验值

4

Scraping Best Practices

Now that you know how to extract content from web pages, it's time to look behind the curtains. In this final chapter, you’ll learn why HTTP requests are the foundation of every scraping action and how they can be customized to comply with best practices in web scraping.

The nature of HTTP requests

50 经验值

Which of these statements about HTTP is false?

50 经验值

Do it the httr way

100 经验值

Houston, we got a 404!

100 经验值

Telling who you are with custom user agents

50 经验值

Check out your user agent

100 经验值

Add a custom user agent

100 经验值

How to be gentle and slow down your requests

50 经验值

Custom arguments for throttled functions

50 经验值

Apply throttling to a multi-page crawler

100 经验值

Recap: Web Scraping in R

50 经验值

Web Scraping in R

课程完成

获得成就证明

将此证书添加到你的 LinkedIn 档案、简历或履历中
在社交媒体和绩效评估中分享立即注册

加入超过19百万学习者，今天就开始Web Scraping in R！

创建您的免费帐户

或

继续操作即表示您接受我们的《使用条款》和《隐私政策》，并同意您的数据存储在美国。

通过 DataCamp for Mobile 提升您的数据技能

随时随地通过我们的移动课程和每日 5 分钟编程挑战提升技能。