Skip to main content
HomeBlogCertification

Introducing the DataCamp Certification Technical Manual

Learn more about the new DataCamp Certification Technical Manual.
May 2023  · 5 min read

As part of our ongoing commitment to show that DataCamp Certifications can be trusted, are credible, and are industry-recognized, we are happy to introduce the DataCamp Certification Technical Manual. This technical manual is the reference on how all DataCamp certifications are implemented. It is a living document that contains detailed information about the design, development, validity, reliability, and fairness of all certifications available. At DataCamp, we offer certifications for Data Scientists and Data Analysts at the associate and professional levels and for Data Engineers at the associate level. These certifications reflect your proficiencies across a set of competencies defined for each level and role. Now let’s take a brief tour of what our technical manual covers!

The Certification Development Cycle

Both at the design and implementation levels, we follow a rigorous process to determine which competencies belong to a particular role and how we measure them. We draw from the best practices in measurement and psychometrics to design and implement each certification. In addition, each live certification undergoes a continuous quality assurance process to guarantee that each certification remains at the high-quality standards we aspire them to be at.

Certification design and development

You might have wondered how we determine the competencies measured in each certification. Based on an initial job task analysis and interviewing experts in the field, we develop a detailed certification specification in consultation with our certification advisory panel. Each specification contains detailed information on the purpose and audience of the certification, which competencies are being measured, and what the specific exam structure of the certification will be. Did you know that all specifications are publicly available? You can find them here:

The Item Writing Process

For each certification, we work together with item writers who are professionals in the field. By doing so, we ensure that each item reflects the actual skills expected on the job. We have a rigorous item review process (see Figure 1 below), where items are reviewed by other item writers (SME review), internally by our certification content team (internal review), and finally, just before they are ready to be launched into an exam. For both the timed and practical exams, there are detailed guidelines on how to write items specific to the competencies being measured, what language to use (or not use), etc., to ensure all items are fair and devoid of bias.

The two-stage item review process.

Figure 1. The two-stage item review process.

Scoring Exams

Have you ever wondered how the timed and practical exams work and how they are scored? In the technical manual, we explain how we, for the timed exams, rely on item response theory and computerized adaptive testing to obtain a quick and accurate estimate of your skill level in each of the domains measured. Indeed, each question you get on the exam is tailored to your current performance, allowing us to efficiently determine whether you are at the level required for the certification you are trying to obtain. The score you obtain is then compared to a target score you must exceed to pass the exam. This score is calculated by simulating hundreds of exams of a hypothetical candidate that is exactly at the level we expect our candidates to be. Based on the distribution of expected scores for a qualified candidate, we then derive a target score we use during the timed exams.

Quality Monitoring

We continuously track quality metrics for all timed exams, including the pass rates and the reliability of each pool of items used in the timed exams. By doing so, we consistently provide reliable assessments of the domains we certify for.

While all timed exams are scored automatically, the practical exams are scored either automatically (Data Engineer Associate) or by human graders. To ensure grading is done reliably, we apply a double grading process where approximately 30% of submissions are graded by at least two graders, either both external graders or one external grader and one internal grader. We then compare how both graders score the rubric, and in case of disagreement, a third internal grader is assigned to rule out any disagreements. This process creates a continuous feedback loop for graders to ensure they are consistently grading against the rubric and awarding certifications to candidates that meet the qualifications.

Curious what more you can find in our technical manual? Go on and have a look!

Topics
Related

The 5 Best Cloud Certifications to Kick-Start Your Career in 2024

Explore the best cloud certifications for 2024 in our comprehensive guide. Discover how certifications like AWS, Azure, and CompTIA Cloud+ can boost your career.
Matt Crabtree's photo

Matt Crabtree

9 min

Introducing New SQL Associate Certification

Prove your SQL skills are job-ready with a new industry-leading certification.
DataCamp Team's photo

DataCamp Team

4 min

Top 32 AWS Interview Questions and Answers For 2024

A complete guide to exploring the basic, intermediate, and advanced AWS interview questions, along with questions based on real-world situations. It covers all the areas, ensuring a well-rounded preparation strategy.
Zoumana Keita 's photo

Zoumana Keita

15 min

A Guide to PyTorch Certifications & Certificates

Unlock the potential of AI with our guide on PyTorch Certifications & Certificates. Learn key differences, explore top courses, and discover how PyTorch skills can elevate your career in ML & DL.
Adel Nehme's photo

Adel Nehme

8 min

Avoiding Burnout for Data Professionals with Jen Fisher, Human Sustainability Leader at Deloitte

Jen and Adel cover Jen’s own personal experience with burnout, the role of a Chief Wellbeing Officer, the impact of work on our overall well-being, the patterns that lead to burnout, the future of human sustainability in the workplace and much more.
Adel Nehme's photo

Adel Nehme

44 min

Becoming Remarkable with Guy Kawasaki, Author and Chief Evangelist at Canva

Richie and Guy explore the concept of being remarkable, growth, grit and grace, the importance of experiential learning, imposter syndrome, finding your passion, how to network and find remarkable people, measuring success through benevolent impact and much more. 
Richie Cotton's photo

Richie Cotton

55 min

See MoreSee More