In 2019, the average salary for data engineers overtook data scientists. How did this happen? Companies wanting to find the gold within their data realized it wasn’t possible if they hadn’t yet built the mine. Data engineers lay the foundations that make data science possible.
<br /> <br />
In this course, you’ll learn about a data engineer’s core responsibilities, how they differ from data scientists, and facilitate the flow of data through an organization. Through hands-on exercises you’ll follow Spotflix, a fictional music streaming company, to understand how their data engineers collect, clean, and catalog their data. By the end of the course, you’ll understand what your company's data engineers do, be ready to have a conversation with a data engineer, and have a solid foundation to start your own data engineer journey.


Discover how data engineers lay the groundwork that makes data science possible. No coding involved!

Data Engineering for Everyone

As a Data Scientist, the majority of your time should be spent gleaning actionable insights from data -- not waiting for your code to finish running. Writing efficient Python code can help reduce runtime and save computational resources, ultimately freeing you up to do the things you love as a Data Scientist. In this course, you'll learn how to use Python's built-in data structures, functions, and modules to write cleaner, faster, and more efficient code. We'll explore how to time and profile code in order to find bottlenecks. Then, you'll practice eliminating these bottlenecks, and other bad design patterns, using Python's Standard Library, NumPy, and pandas. After completing this course, you'll have the necessary tools to start writing efficient Python code!

Learn to write efficient code that executes quickly and allocates resources skillfully to avoid unnecessary overhead.

Writing Efficient Python Code

You've done your analysis, built your report, and trained a model. What's next? Well, if you want to deploy your model into production, your code will need to be more reliable than exploratory scripts in a Jupyter notebook. Writing Functions in Python will give you a strong foundation in writing complex and beautiful functions so that you can contribute research and engineering skills to your team. You'll learn useful tricks, like how to write context managers and decorators. You'll also learn best practices around how to write maintainable reusable functions with good documentation. They say that people who can do good research and write high-quality code are unicorns. Take this course and discover the magic!

Learn to use best practices to write maintainable, reusable, complex functions with good documentation.

Writing Functions in Python

Have you heard people talk about data engineers and wonder what it is they do? Do you know what data engineers do but you're not sure how to become one yourself? This course is the perfect introduction. It touches upon all things you need to know to streamline your data processing. This introductory course will give you enough context to start exploring the world of data engineering. It's perfect for people who work at a company with several data sources and don't have a clear idea of how to use all those data sources in a scalable way. Be the first one to introduce these techniques to your company and become the company star employee.

Learn about the world of data engineering with an overview of all its relevant topics and tools!

Introduction to Data Engineering

The Unix command line has survived and thrived for almost 50 years because it lets people do complex things with just a few keystrokes. Sometimes called "the universal glue of programming," it helps users combine existing programs in new ways, automate repetitive tasks, and run programs on clusters and clouds that may be halfway around the world. This course will introduce its key elements and show you how to use them efficiently.

The Unix command line helps users combine existing programs in new ways, automate repetitive tasks, and run programs on clusters and clouds. 

Discover the Unix command line and how to use it to combine existing programs, automate repetitive tasks, and run programs on clusters and clouds in Shell.

Introduction to Shell

Before you can analyze data, you first have to acquire it. This course teaches you how to build pipelines to import data kept in common storage formats. You’ll use pandas, a major Python library for analytics, to get data from a variety of sources, from spreadsheets of survey responses, to a database of public service requests, to an API for a popular review site. Along the way, you’ll learn how to fine-tune imports to get only what you need and to address issues like incorrect data types. Finally, you’ll assemble a custom dataset from a mix of sources.

Learn to acquire data from common file formats and systems such as CSV files, spreadsheets, JSON, SQL databases, and APIs.

Streamlined Data Ingestion with pandas

Every data science project needs unit testing. It comes with huge benefits - saving a lot of development and maintenance time, improving documentation, increasing end-user trust and reducing downtime of productive systems. As a result, unit testing has become a must-have skill in the industry, used by almost every company. This course teaches unit testing in Python using the most popular testing framework pytest. By the end of this course, you will have written a complete test suite for a data science project. In the process, you will learn to write unit tests for data preprocessors, models and visualizations, interpret test results and fix any buggy code. You will also learn advanced concepts like TDD, test organization, fixtures and mocking so that you can test your own data science projects properly.

Learn how to write unit tests for your Data Science projects in Python using pytest.

Unit Testing for Data Science in Python

Delivering data on a schedule can be a manual process. You write scripts, add complex cron tasks, and try various ways to meet an ever-changing set of requirements—and it’s even trickier to manage everything when working with teammates. Airflow can remove this headache by adding scheduling, error handling, and reporting to your workflows. In this course, you’ll master the basics of Airflow and learn how to implement complex data engineering pipelines in production. You'll also learn how to use Directed Acyclic Graphs (DAGs), automate data engineering workflows, and implement data engineering tasks in an easy and repeatable fashion—helping you to maintain your sanity.

Learn how to implement and schedule data engineering workflows.

Learn to implement and schedule data engineering workflows by adding scheduling, error handling, and reporting using Airflow and Airflow DAGs in Python. 

Introduction to Airflow in Python

Twitter produces hundreds of million messages per day, with people around the world discussing sports, politics, business, and entertainment. You can access thousands of messages flowing in this stream in a matter of minutes. In this course, you will learn how to collect Twitter data and analyze tweet text, Twitter networks, and the geographical origin of the tweet. We'll be doing this with datasets on tech companies, data science hashtags, and the 2018 State of the Union address. Using these methods, you will be able to inform business and political decision-making by discovering the prevalence of important topics, the diversity of discussion networks, and a topic's geographical reach.

In this course, you'll learn how to collect Twitter data and analyze Twitter text, networks, and geographical origin.

Analyzing Social Media Data in Python

In this introduction to R course, you'll master the basics of this widely used open source language&mdash;including vectors, factors, lists, and data frames. With the coding skills you'll gain in this course, you'll be ready to undertake your own data analysis in R. There are millions of R users worldwide, cementing it as a leading programming language in statistics and data science. More and more organizations are using R, so begin your coding journey in one of DataCamp's most popular courses today!

Master the basics of data analysis by manipulating common data structures such as vectors, matrices, and data frames.

DataCamp's Into to R training course teaches you how to use R programming for data science at your own pace with video tutorials & interactive challenges.

Introduction to R Online

Introduction to R

You’ve already used SQL to query data from databases. But did you know that there's a lot more you can do with databases? You can model different phenomena in your data, as well as the relationships between them. This gives your data structure and consistency, which results in better data quality. In this course, you'll experience this firsthand by working with a real-life dataset that was used to investigate questionable university affiliations. Column by column, table by table, you'll get to unlock and admire the full potential of databases. You'll learn how to create tables and specify their relationships, as well as how to enforce data integrity. You'll also discover other unique features of database systems, such as constraints.

Learn how to create one of the most efficient ways of storing data - relational databases! 

Learn how to create tables and specify their relationships in SQL, as well as how to enforce data integrity and other unique features of database systems.

Introduction to Relational Databases in SQL

We live in a busy world with tight deadlines.  As a result, we fall back on what is familiar and easy, favoring GUI interfaces like Anaconda and RStudio. However, taking the time to learn data analysis on the command line is a great long-term investment because it makes us stronger and more productive data people.
 <br />
<br />
In this course, we will take a practical approach to learn simple, powerful, and data-specific command-line skills.  Using publicly available Spotify datasets, we will learn how to download, process, clean, and transform data, all via the command line.  We will also learn advanced techniques such as command-line based SQL database operations.  Finally, we will combine the powers of command line and Python to build a data pipeline for automating a predictive model.

Learn powerful command-line skills to download, process,  and transform data, including machine learning pipeline.

Learn powerful command-line skills to download, process, and transform data, including a machine learning pipeline.

Data Processing in Shell

A good database design is crucial for a high-performance application. Just like you wouldn't start building a house without the benefit of a blueprint, you need to think about how your data will be stored beforehand. Taking the time to design a database saves time and frustration later on, and a well-designed database ensures ease of access and retrieval of information. While choosing a design, a lot of considerations have to be accounted for. In this course, you'll learn how to process, store, and organize data in an efficient way. You'll see how to structure data through normalization and present your data with views. Finally, you'll learn how to manage your database and all of this will be done on a variety of datasets from book sales, car rentals, to music reviews.

Learn to design databases using star and snoflake schemas, and OLTP and OLAP. You will gain essential skills for creating and managing databases in SQL.

Data Science Courses

Data Science Tracks

Our Learning Experience

@drleon

Workspace Publications (0)

Completed Tracks (0)

Completed courses (9)

Completed Projects (0)