Skip to main content

A/B Testing in Python

Learn the practical uses of A/B testing in Python to run and analyze experiments. Master p-values, sanity checks, and analysis to guide business decisions.

Start Course for Free
4 Hours16 Videos51 Exercises
3,143 LearnersTrophyStatement of Accomplishment

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.

Loved by learners at thousands of companies


Course Description

In this course, you will dive into the world of A/B testing, gain a deep understanding of the practical use cases, and learn to design, run, and analyze these A/B tests in Python.

Discover How A/B Tests Work



Did you know that you are almost guaranteed to participate in an A/B test every time you browse the internet? From search engines and e-commerce sites to social networks and marketing campaigns — all businesses hire the best data analysts, scientists, and engineers to leverage the power of AB testing. Testing different variants can help optimize the customer experience, maximize profits, inform the next best design, and much more.

Learn About A/B Testing in Python



You’ll start by learning how to define the right metrics before learning how to estimate the appropriate sample size and duration to yield conclusive results. Throughout this course, you’ll use a range of Python packages to help with A/B testing, including statsmodels, scipy, and pingouin.

By the end of the course, you will be able to run the necessary checks that guarantee accurate results, master the art of p-values, and analyze the results of A/B tests with ease and confidence to guide the most critical business decisions.
  1. 1

    Overview of A/B Testing

    Free

    In this chapter, you’ll learn the foundations of A/B testing. You’ll explore clear steps and use cases, learn the reasons and value of designing and running A/B tests, and discover the most commonly used metrics design and estimation frameworks.

    Play Chapter Now
    What is A/B testing?
    50 xp
    When an A/B test is not best
    50 xp
    A/B testing steps
    100 xp
    Randomization effects
    100 xp
    Why run experiments?
    50 xp
    Correlation visualization
    100 xp
    Correlation or causation?
    50 xp
    Metrics design and estimation
    50 xp
    Means and proportions
    100 xp
    Ad impressions metrics
    100 xp
  2. 2

    Experiment Design and Planning

    In Chapter 2, you’ll cover the experiment design process. Starting with learning how to formulate strong A/B testing hypotheses, you’ll also cover statistical concepts such as power, error rates, and minimum detectable effects. You’ll finish the chapter by learning to estimate the appropriate sample size needed to yield conclusive results and tackle scenarios with multiple comparisons.

    Play Chapter Now
  3. 3

    Data Processing, Sanity Checks, and Results Analysis

    Here, you’ll discover a concrete workflow for cleaning, preprocessing, and exploring AB testing data, as well as learn the necessary sanity checks we need to follow to ensure valid results. You’ll explore a detailed explanation and example of analyzing difference in proportions A/B tests.

    Play Chapter Now
  4. 4

    Practical Considerations and Making Decisions

    In the final chapter, you’ll develop frameworks for analyzing differences in means and leveraging non-parametric tests when several assumptions aren't met. You’ll also learn how to apply the Delta method when analyzing ratio metrics and discover the best practices and some advanced topics to continue the A/B testing mastery journey.

    Play Chapter Now

Datasets

checkoutadmissions

Collaborators

Collaborator's avatar
Jasmin Ludolf
Collaborator's avatar
George Boorman
Collaborator's avatar
Maham Khan
Moe Lotfy, PhD HeadshotMoe Lotfy, PhD

Principal Data Science Manager

Moe is a Principal Data Science Manager with 10+ years of experience working in the fields of data science and analytics in various settings including research, academia, and industry. He combines his physics/engineering domain knowledge with data science expertise to uncover insights in massive datasets, influence critical design decisions, and drive product improvements. Over his career, he has advised and built analytics and experimentation functions for several fortune 500 companies. He fuels his passion for Data Science/AI through teaching and giving invited lectures/talks. Moe has a PhD in Nuclear Fusion Engineering from UCLA with a focus on experimentation and computational analysis, and a BSc in Mechanical and Aerospace Engineering from the University of Illinois at Urbana-Champaign. His areas of expertise include energy systems, AI, autonomous driving, shipping high-impact products, and all things data and experimentation.
See More

What do other learners have to say?

FAQs

Join over 13 million learners and start A/B Testing in Python today!

Create Your Free Account

GoogleLinkedInFacebook

or

By continuing, you accept our Terms of Use, our Privacy Policy and that your data is stored in the USA.