Course
Scalable Data Processing in R
AdvancedSkill Level
Updated 08/2024RProgramming4 hr15 videos49 Exercises3,950 XP6,127Statement of Accomplishment
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.Loved by learners at thousands of companies
Training 2 or more people?
Try DataCamp for BusinessCourse Description
Prerequisites
Writing Efficient R Code1
Working with increasingly large data sets
In this chapter, we cover the reasons you need to apply new techniques when data sets are larger than available RAM. We show that importing and exporting data using the base R functions can be slow and some easy ways to remedy this. Finally, we introduce the bigmemory package.
2
Processing and Analyzing Data with bigmemory
Now that you've got some experience using bigmemory, we're going to go through some simple data exploration and analysis techniques. In particular, we'll see how to create tables and implement the split-apply-combine approach.
3
Working with iotools
We'll use the iotools package that can process both numeric and string data, and introduce the concept of chunk-wise processing.
4
Case Study: A Preliminary Analysis of the Housing Data
In the previous chapters, we've introduced the housing data and shown how to compute with data that is about as big, or bigger than, the amount of RAM on a single machine. In this chapter, we'll go through a preliminary analysis of the data, comparing various trends over time.
Scalable Data Processing in R
Course Complete
Earn Statement of Accomplishment
Add this credential to your LinkedIn profile, resume, or CVShare it on social media and in your performance reviewEnroll Now
FAQs
What real-world dataset is used throughout this course?
The course uses Federal Housing Finance Agency data covering all mortgages held or securitized by Fannie Mae and Freddie Mac from 2009 to 2015.
Which R packages does this course teach for handling large datasets?
The course focuses on the bigmemory package for working with data larger than RAM and the iotools package for chunk-wise processing of both numeric and string data.
Why can't I just use standard R functions for very large datasets?
R stores all variables in memory by default, so datasets larger than available RAM cannot be handled with base R. This course teaches disk-based and chunk-wise processing as alternatives.
Does the course cover the split-apply-combine approach for big data?
Yes, you will implement split-apply-combine using bigmemory, learning how to explore and analyze large datasets through table creation and grouped computations.
What prior R experience is recommended before taking this course?
You should have completed Introduction to R, Intermediate R, and Writing Efficient R Code. Familiarity with writing efficient code is especially important for this course.
Join over 19 million learners and start Scalable Data Processing in R today!
Create Your Free Account
or
By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.Grow your data skills with DataCamp for Mobile
Make progress on the go with our mobile courses and daily 5-minute coding challenges.