Skip to content
Project: Hypothesis Testing in Healthcare
  • AI Chat
  • Code
  • Report
  • Hypothesis Testing in Healthcare: Drug Safety

    A pharmaceutical company GlobalXYZ has just completed a randomized controlled drug trial. To promote transparency and reproducibility of the drug's outcome, they (GlobalXYZ) have presented the dataset to your organization, a non-profit that focuses primarily on drug safety.

    The dataset provided contained five adverse effects, demographic data, vital signs, etc. Your organization is primarily interested in the drug's adverse reactions. It wants to know if the adverse reactions, if any, are of significant proportions. It has asked you to explore and answer some questions from the data.

    The dataset drug_safety.csv was obtained from Hbiostat courtesy of the Vanderbilt University Department of Biostatistics. It contained five adverse effects: headache, abdominal pain, dyspepsia, upper respiratory infection, chronic obstructive airway disease (COAD), demographic data, vital signs, lab measures, etc. The ratio of drug observations to placebo observations is 2 to 1.

    For this project, the dataset has been modified to reflect the presence and absence of adverse effects adverse_effects and the number of adverse effects in a single individual num_effects.

    The columns in the modified dataset are:

    ColumnDescription
    sexThe gender of the individual
    ageThe age of the individual
    weekThe week of the drug testing
    trxThe treatment (Drug) and control (Placebo) groups
    wbcThe count of white blood cells
    rbcThe count of red blood cells
    adverse_effectsThe presence of at least a single adverse effect
    num_effectsThe number of adverse effects experienced by a single individual

    The original dataset can be found here.

    Your organization has asked you to explore and answer some questions from the data collected. See the project instructions.

    # Import packages
    import numpy as np
    import pandas as pd
    from statsmodels.stats.proportion import proportions_ztest
    import pingouin
    import seaborn as sns
    import matplotlib.pyplot as plt
    
    # Load the dataset
    drug_safety = pd.read_csv("drug_safety.csv")
    
    # Start coding here...
    drug_safety
    # Foe two sample z test divide the tex in to two sample OR two groups
    drug_group = drug_safety[drug_safety['trx']== 'Drug']
    placebo_group = drug_safety[drug_safety['trx']== 'Placebo']
    
    # Count the number of adverse effects in each group
    adverse_effects_drug = drug_group['adverse_effects'].value_counts().get('Yes',0)
    adverse_effects_placebo = placebo_group['adverse_effects'].value_counts().get('Yes',0)
    
    # Count the total number of observations in each group
    num_drugs = len(drug_group)
    num_placebo = len(placebo_group)
    
    adverse_effects_count = np.array([adverse_effects_drug,adverse_effects_placebo])
    drugs_placebo_count = np.array([num_drugs, num_placebo])
    
    stats, two_sample_p_value = proportions_ztest(adverse_effects_count , drugs_placebo_count )
    
    print('Stats:', stats)
    print('p-value:', two_sample_p_value)
    
    
    # performing chi-squared independence of test to know the relation between x and y (categorical variables)
    num_effects_groups = pingouin.chi2_independence(data = drug_safety, x = 'num_effects', y= 'trx')
    num_effects_groups
    
    #Find out if the number of adverse effects is independent of the treatment and control groups
    num_effects_p_value = num_effects_groups[2]["pval"][0]
    num_effects_p_value
    
    #inspecting wether the ages are normally distributed or not by plotting histogram
    # Plotting the histograms
    plt.figure(figsize=(14, 6))
    
    # Histogram for the drug group
    plt.subplot(1, 2, 1)
    plt.hist(drug_group['age'], bins=5, edgecolor='red')
    plt.title('Age Distribution for Drug Group')
    plt.xlabel('Age')
    plt.ylabel('Frequency')
    
    # Histogram for the placebo group
    plt.subplot(1, 2, 2)
    plt.hist(placebo_group['age'], bins=5, edgecolor='red')
    plt.title('Age Distribution for Placebo Group')
    plt.xlabel('Age')
    plt.ylabel('Frequency')
    
    # Display the plots
    plt.tight_layout()
    plt.show()
    normality = pingouin.normality(
        data=drug_safety,
        dv='age',
        group='trx',
        method='shapiro')
    
    age_drug = drug_safety[drug_safety['trx'] == 'Drug'][['trx', 'age']]
    age_placebo = drug_safety[drug_safety['trx'] == 'Placebo'][['trx', 'age']]
    # Perform Mann-Whitney U test
    mwu_test = pingouin.mwu(age_drug['age'], age_placebo['age'])
    #Store p-value
    age_group_effects_p_value = mwu_test['p-val']