课程

Data Processing in Shell

中级技能水平

更新时间 2025年10月

Learn powerful command-line skills to download, process, and transform data, including machine learning pipeline.

免费开始课程

ShellData Manipulation4 小时13 视频46 练习3,550 经验值22,736成就声明

创建您的免费帐户

或

继续操作即表示您接受我们的《使用条款》和《隐私政策》，并同意您的数据存储在美国。

深受数千家公司学习者的喜爱

培训2人或更多？

试用DataCamp for Business

课程描述

We live in a busy world with tight deadlines. As a result, we fall back on what is familiar and easy, favoring GUI interfaces like Visual Studio and RStudio. However, taking the time to learn data analysis on the command line is a great long-term investment because it makes us stronger and more productive data people.In this course, we will take a practical approach to learn simple, powerful, and data-specific command-line skills. Using publicly available Spotify datasets, we will learn how to download, process, clean, and transform data, all via the command line. We will also learn advanced techniques such as command-line based SQL database operations. Finally, we will combine the powers of command line and Python to build a data pipeline for automating a predictive model.

先决条件

Introduction to Shell Intermediate Python Intermediate SQL

1

Downloading Data on the Command Line

In this chapter, we learn how to download data files from web servers via the command line. In the process, we also learn about documentation manuals, option flags, and multi-file processing.

Downloading data using curl

50 经验值

Using curl documentation

50 经验值

Downloading single file using curl

100 经验值

Downloading multiple files using curl

100 经验值

Downloading data using Wget

50 经验值

Installing Wget

50 经验值

Downloading single file using wget

100 经验值

Advanced downloading using Wget

50 经验值

Setting constraints for multiple file downloads

50 经验值

Creating wait time using Wget

100 经验值

Data downloading with Wget and curl

100 经验值

2

Data Cleaning and Munging on the Command Line

We continue our data journey from data downloading to data processing. In this chapter, we utilize the command line library csvkit to convert, preview, filter and manipulate files to prepare our data for further analyses.

Getting started with csvkit

50 经验值

Installation and documentation for csvkit

100 经验值

Converting and previewing data with csvkit

100 经验值

File conversion and summary statistics with csvkit

100 经验值

Filtering data using csvkit

50 经验值

Printing column headers with csvkit

100 经验值

Filtering data by column with csvkit

100 经验值

Filtering data by row with csvkit

100 经验值

Stacking data and chaining commands with csvkit

50 经验值

Stacking files with csvkit

100 经验值

Chaining commands using operators

100 经验值

Data processing with csvkit

100 经验值

3

Database Operations on the Command Line

In this chapter, we dig deeper into all that csvkit library has to offer. In particular, we focus on database operations we can do on the command line, including table creation, data pull, and various ETL transformation.

Pulling data from database

50 经验值

Using sql2csv documentation

50 经验值

Understand sql2csv connectors

50 经验值

Practice pulling data from database

100 经验值

Manipulating data using SQL syntax

50 经验值

Applying SQL to a local CSV file

100 经验值

Cleaner scripting via shell variables

100 经验值

Joining local CSV files using SQL

100 经验值

Pushing data back to database

50 经验值

Practice pushing data back to database

100 经验值

Database and SQL with csvkit

100 经验值

4

Data Pipeline on the Command Line

In the last chapter, we bridge the connection between command line and other data science languages and learn how they can work together. Using Python as a case study, we learn to execute Python on the command line, to install dependencies using the package manager pip, and to build an entire model pipeline using the command line.

Python on the command line

50 经验值

Finding Python version on the command line

50 经验值

Executing Python script on the command line

100 经验值

Python package installation with pip

50 经验值

Understanding pip's capabilities

50 经验值

Installing Python dependencies

100 经验值

Running a Python model

100 经验值

Data job automation with cron

50 经验值

Understanding cron scheduling syntax

50 经验值

Scheduling a job with crontab

100 经验值

Model production on the command line

100 经验值

Course recap

50 经验值

Data Processing in Shell

课程完成

获得成就证明

将此证书添加到你的 LinkedIn 档案、简历或履历中
在社交媒体和绩效评估中分享立即注册

加入超过19百万学习者，今天就开始Data Processing in Shell！

创建您的免费帐户

或

继续操作即表示您接受我们的《使用条款》和《隐私政策》，并同意您的数据存储在美国。

通过 DataCamp for Mobile 提升您的数据技能

随时随地通过我们的移动课程和每日 5 分钟编程挑战提升技能。