Hypothesis Testing in Healthcare: Drug Safety
A pharmaceutical company GlobalXYZ has just completed a randomized controlled drug trial. To promote transparency and reproducibility of the drug's outcome, they (GlobalXYZ) have presented the dataset to your organization, a non-profit that focuses primarily on drug safety.
The dataset provided contained five adverse effects, demographic data, vital signs, etc. Your organization is primarily interested in the drug's adverse reactions. It wants to know if the adverse reactions, if any, are of significant proportions. It has asked you to explore and answer some questions from the data.
The dataset drug_safety.csv was obtained from Hbiostat courtesy of the Vanderbilt University Department of Biostatistics. It contained five adverse effects: headache, abdominal pain, dyspepsia, upper respiratory infection, chronic obstructive airway disease (COAD), demographic data, vital signs, lab measures, etc. The ratio of drug observations to placebo observations is 2 to 1.
For this project, the dataset has been modified to reflect the presence and absence of adverse effects adverse_effects and the number of adverse effects in a single individual num_effects.
The columns in the modified dataset are:
| Column | Description |
|---|---|
sex | The gender of the individual |
age | The age of the individual |
week | The week of the drug testing |
trx | The treatment (Drug) and control (Placebo) groups |
wbc | The count of white blood cells |
rbc | The count of red blood cells |
adverse_effects | The presence of at least a single adverse effect |
num_effects | The number of adverse effects experienced by a single individual |
The original dataset can be found here.
# Import packages
import numpy as np
import pandas as pd
from statsmodels.stats.proportion import proportions_ztest
import pingouin
import seaborn as sns
import matplotlib.pyplot as plt
# Load the dataset
drug_safety = pd.read_csv("drug_safety.csv")drug_safety.head()Test of Adverse Effects Proportions
To determine whether there is a statistically significant difference in the proportion of adverse effects between participants who received the Drug and those who received the Placebo, we performed a two-sample z-test. This test compares the proportions of adverse effects in the two groups, providing a z-score and a p-value to assess the difference.
Methodology
We first calculated the number of participants experiencing adverse effects in both the Drug and Placebo groups. Using these counts, we performed the two-sample z-test. A low p-value (typically < 0.05) would indicate a significant difference in the proportions of adverse effects between the Drug and Placebo groups.
# Group the data by treatment type ("trx") and count the occurrences of adverse effects ("adverse_effects")
drug_n_placebo = drug_safety.groupby("trx")["adverse_effects"].value_counts()
# Extract the counts of adverse effects for the Drug group
drug_advrese = drug_n_placebo[("Drug", "Yes")]
# Calculate the total number of participants in the Drug group
drug_total = drug_n_placebo[("Drug", "Yes")] + drug_n_placebo[("Drug", "No")]
# Extract the counts of adverse effects for the Placebo group
placebo_advrese = drug_n_placebo[("Placebo", "Yes")]
# Calculate the total number of participants in the Placebo group
placebo_total = drug_n_placebo[("Placebo", "Yes")] + drug_n_placebo[("Placebo", "No")]
# Create arrays for the counts of adverse effects and the total number of participants in each group
counts = np.array([drug_advrese, placebo_advrese])
nobs = np.array([drug_total, placebo_total])
# Perform a two-sample z-test to compare the proportions of adverse effects between the Drug and Placebo groups
two_sample_z_score, two_sample_p_value = proportions_ztest(count=counts, nobs=nobs, alternative="two-sided")
print([two_sample_z_score, two_sample_p_value])Results
The z-test yielded the following results:
- Z-score: 0.0452
- P-value: 0.9639
Interpretation
Based on the p-value obtained, we can conclude whether the difference in adverse effects is statistically significant. Since the p-value is greater than 0.05, we fail to reject the null hypothesis, indicating no significant difference in the proportions of adverse effects between the Drug and Placebo groups.
Test for Association Between Treatment and Adverse Effects
To further investigate the relationship between the treatment group (trx) and the number of adverse effects (num_effects), we performed a Chi-square test of independence. This test helps us determine if there is a significant association between these two categorical variables.
Methodology
We used the drug_safety dataset for this analysis. The Chi-square test was applied to the trx and num_effects columns to evaluate the independence of these variables.
test_result = pingouin.chi2_independence(data=drug_safety, y="trx", x="num_effects")[2].query("test == 'pearson'")
num_effects_p_value = test_result.loc[0, "pval"]
test_resultResults
The Chi-square test yielded the following results:
- Chi-square Statistic: 1.799
- Degrees of Freedom: 3
- P-value: 0.615
Interpretation
The p-value obtained from the Chi-square test is greater than the conventional threshold of 0.05. Therefore, we fail to reject the null hypothesis, indicating that there is no significant association between the treatment group and the number of adverse effects.
This result aligns with our previous findings from the z-test, further supporting the conclusion that the treatment and placebo groups do not differ significantly in terms of adverse effects.
Mann-Whitney U Test for Age Distribution Between Treatment Groups
To further explore potential differences between the treatment (Drug) and placebo (Placebo) groups, we conducted a Mann-Whitney U test. This non-parametric test is used to determine if there is a significant difference in the distribution of ages between the two groups.
groups_ages = drug_safety.pivot(columns="trx", values="age")
test_result = pingouin.mwu(x=groups_ages["Drug"], y=groups_ages["Placebo"], alternative="two-sided")
age_group_effects_p_value = test_result["p-val"]
test_resultResults
The Mann-Whitney U test yielded the following results:
- U-value: 29149339.5
- P-value: 0.257
Interpretation
The p-value obtained from the Mann-Whitney U test is greater than the conventional threshold of 0.05. Therefore, we fail to reject the null hypothesis, indicating that there is no significant difference in the age distribution between the treatment and placebo groups.
Conclusion
Based on the results of the Mann-Whitney U test, we conclude that the age distribution does not significantly differ between the treatment and placebo groups. This finding suggests that age is not a confounding factor in the analysis of adverse effects between these groups.