Learn, Practice, Apply!
At DataCamp, our mission has always been about helping people learn data science by doing. Our interactive courses help people learn the fundamentals of R, Python, and SQL as well as more advanced topics like Machine Learning and Deep Learning.
We have been pretty successful in helping people learn, as evidenced by our rapid growth. We have close to 1.5 million users on our platform, learning data science from an expansive library of 90 courses. While the completion rates for MOOCs typically linger around 5 - 10%, the completion rates for DataCamp courses are much higher, ranging from 30% for our free courses, going up to 80% for some of the paid ones. This is a testament to our focus on helping people learn by doing, keeping them highly engaged during the learning process.
While learning for the sake of learning might be the end-goal for some of our users, a large percentage of our user base is learning so that they can take what they have learnt and apply it to problems they encounter as a part of their daily work life. Consequently, we realized that our mission does not just stop with helping our users learn, but goes beyond that, to help them attain proficiency, and apply the skills they have learnt.
Carnegie Mellon summarizes the extensive research done on learning to conclude that:
To develop mastery, students must acquire component skills, practice integrating them, and know when to apply what they have learned.
In other words, to support our mission of helping users apply data science skills in real life, we need to help them learn, practice and apply!
Interactive courses support our mission to help people learn data science by doing. Over the last four years we have grown our course library to nearly 100 courses, spread across R, Python, and SQL. In order to provide users more guidance towards gaining a specific set of skills, we introduced the idea of tracks, that lets students work on a sequence of courses, to build proficiency in a specific language and/or skill.
It is clear that before we can help our users apply their skills in real life, we need to help them gain proficiency and mastery of the topics they have learnt, through practice. Importantly, not just any kind of practice, but the kind that can systematically help users focus on concepts, understand at a granular level, what they don’t know, iterate on their learning cycle, and repeat practice.
Deliberate practice requires that one identify certain sharply defined elements of performance that need to be improved, and then work intently on them. … Consistently and overwhelmingly, the evidence showed that experts are always made, not born.
When we set out to design Practice Mode, our goal was to help our users retain knowledge, and build fluency seamlessly by immersing themselves in deliberate practice.
Practice Mode allows DataCamp users to practice the skills they have learnt in a course by repeatedly taking short sessions consisting of 5 - 10 challenges. The secret to deliberate practice is consistency and repetition, and so we designed the system such that it is easy for everyone to get started!
When a student completes a course on DataCamp, they are directed to take a practice session to challenge their skills. Every session consists of byte-sized challenges, each of which focuses on one concept. A student can complete a session by getting five challenges right. The idea here is to ensure that students can complete a session within a short time and make regular practice a habit!
Once a student starts a practice session, they are presented with challenge questions, that can be one of three types. We designed these types keeping in mind that there are two kinds of fluency that we want our users to develop; passive and active.
Match Code with Output (or vice-versa)
While doing exercises, students get so busy writing code, that they sometimes stop to think about why they are writing something. The idea of this type of challenge is to make students reason out the output of code without really executing it. We have found that this helps uncover a lot of misconceptions that do not become apparent during the learning process.
This challenge type has a variant, where makes students match output with code, where they have to reason out the output of multiple code chunks, and select the one that produces the output shown.
This type of challenge helps students achieve passive fluency, as they learn how to reason about code without actually writing it. It provides a stepping stone to achieving active fluency.
Complete the Script
This challenge type requires the student to write code, but is fundamentally different from exercises in that they are only required to fill in missing piece(s) in the script. While this might seem sub-optimal, a major advantage of this challenge type is that it allows us to break down a concept into smaller units, and challenge students on each unit.
Our system allows us to programatically vary the missing piece(s), thereby letting students practice filling different variations. This helps students gain mastery and proficiency through repetition.
This challenge type has a variant where users are presented with a set of code snippets that they can drag and drop into the blank spaces to complete the script. This helps uncover misconceptions like the difference between the use of single bracket,
[, and double brackets,
[[, in R.
If a student answers a question incorrectly, they are shown the correct solution, allowing them to digress further on the mistake they made, and correct their understanding of the underlying concept.
This challenge type helps students build active fluency as they are actually writing code to complete it. By focusing on writing byte-sized chunks of code repeatedly, they are able to build and flex their muscle memory, which goes a long way in helping them achieve mastery.
Finally, there are some concepts that cannot be expressed as code. The multiple choice type allows us to challenge students on such concepts, and test their learning.
A critical element of deliberate practice is to identify certain sharply defined elements of performance that need to be improved, and then work intently on them. We achieve this by following a careful set of guidelines while writing practice questions.
- Start with the learning objective.
- Break it down into units.
- Write questions that test each unit.
Let me illustrate the idea with an example. Suppose that the objective is to learn how to use the
select function in
dplyr. This function can accept arguments in different flavors: names, indices, negative indices, range, functions etc. In order to help students achieve mastery of this function, we design practice questions that test each of these variants. This helps students uncover at a very granular level what they are struggling with, learn it better, and try to solve it again.
A key feature of Practice Mode is the ability to generate near-infinite variants of a question based on a template. The system also provides content developers great control over how these variants are generated. This feature ensures that we never run out of good questions, and our students can repeatedly practice to their heart’s content and sharpen their skills.
Since launching Practice Mode in June 2017, we now have practice sessions available for 17 of our courses. The plan is to expand this out to more courses so that our users can gain proficiency across the board. Our students are completing 4000+ practice sessions every day, and this number has been growing steadily. Of the users who take a practice session, more than 50% take another session.
As we strive to improve Practice Mode, there are several enhancements in the works that should make it more useful.
One of the important elements of deliberate practice is feedback. We do provide feedback after completion of every question. However, sometimes, it is useful to look at feedback for all questions after completing a session. We are working on a new mode that will let users see feedback for all questions completed during a session, so that they can assess their shortcomings and make plans to address their knowledge gaps.
Currently, questions are drawn at random from a big pool, for every practice session. We can do better than this by making the system more adaptive. This implies learning from each practice session, so the next one can be personalized based on questions that a user got wrong. The ultimate plan is to provide users a detailed dashboard of their skill levels on different units of learning, and create a personalized learning plan for them to achieve mastery. We are diligently working towards this goal.
If you want to take a practice session, you can sign up for DataCamp and complete one of our free courses. If you have already taken some practice sessions and have suggestions on how we can improve it, please leave us a comment here.