Hypothesis Testing in Healthcare: Drug Safety
A pharmaceutical company GlobalXYZ has just completed a randomized controlled drug trial. To promote transparency and reproducibility of the drug's outcome, they (GlobalXYZ) have presented the dataset to your organization, a non-profit that focuses primarily on drug safety.
The dataset provided contained five adverse effects, demographic data, vital signs, etc. Your organization is primarily interested in the drug's adverse reactions. It wants to know if the adverse reactions, if any, are of significant proportions. It has asked you to explore and answer some questions from the data.
The dataset drug_safety.csv was obtained from Hbiostat courtesy of the Vanderbilt University Department of Biostatistics. It contained five adverse effects: headache, abdominal pain, dyspepsia, upper respiratory infection, chronic obstructive airway disease (COAD), demographic data, vital signs, lab measures, etc. The ratio of drug observations to placebo observations is 2 to 1.
For this project, the dataset has been modified to reflect the presence and absence of adverse effects adverse_effects and the number of adverse effects in a single individual num_effects.
The columns in the modified dataset are:
| Column | Description |
|---|---|
sex | The gender of the individual |
age | The age of the individual |
week | The week of the drug testing |
trx | The treatment (Drug) and control (Placebo) groups |
wbc | The count of white blood cells |
rbc | The count of red blood cells |
adverse_effects | The presence of at least a single adverse effect |
num_effects | The number of adverse effects experienced by a single individual |
The original dataset can be found here.
Your organization has asked you to explore and answer some questions from the data collected. See the project instructions.
# Import packages
import numpy as np
import pandas as pd
from statsmodels.stats.proportion import proportions_ztest
import pingouin
import seaborn as sns
import matplotlib.pyplot as plt
# Load the dataset
drug_safety = pd.read_csv("drug_safety.csv")
# Start coding here...
drug_safety.head()# Proportion of Adverse Effects by Treatment & Countrol Groups
adverse_prop = drug_safety.groupby('trx')['num_effects'].value_counts(normalize=True)
print(adverse_prop)
# Visualise the prop
# Convert the Series 'a' into a DataFrame for easier plotting
a_df = adverse_prop.reset_index(name='proportion')
# Plot the proportions of adverse effects for each treatment group
plt.figure(figsize=(8,4))
sns.barplot(data=a_df, x='num_effects', y='proportion', hue='trx')
plt.title('Proportion of Adverse Effects by Treatment & Control Group')
plt.xlabel('The number of adverse effects')
plt.ylabel('Proportion')
plt.show()From the graph, we can make the following observations:
- The proportions of adverse effects are identical in both the Treatment and Placebo groups for all numbers of adverse effects.
- Approximately 90% of participants in both groups did not experience any adverse effects. Among those who did suffer from adverse effects, a large proportion experienced only one adverse effect, at nearly 1%.
Conduct a two-sample Z-test to determine if the proportion of adverse effects differs significantly between the Drug and Placebo groups.
Call
# Count of adverse affects by Treatment & Countrol groups
adverse_count_by_group = drug_safety.groupby('trx')['adverse_effects'].value_counts()
# Create an array of the "Yes" count for each group in adverse_effects
success_count = np.array([adverse_count_by_group[('Drug', 'Yes')],
adverse_count_by_group[('Placebo', 'Yes')]])
# Create an array of the total number of rows in each group in adverse_effects
n = np.array([adverse_count_by_group['Drug'].sum(),
adverse_count_by_group['Placebo'].sum()])
two_sample_p_value = proportions_ztest(success_count, n)[1]
print('p-value: ', two_sample_p_value)With a p-value of 0.964, which is larger than the significance level of 0.05, we fail to reject the null hypothesis. This suggests that there is no significant difference between the Drug and Placebo groups.
Test if num_effects and trx are independent to determine whether trx influences the number of effects.
num_effects and trx are independent to determine whether trx influences the number of effects.trx and num_effects are independent.
trx and num_effects are associated.
# chi-square of independence
expected, observed, stats = pingouin.chi2_independence(
data=drug_safety, x='trx', y='num_effects'
)
print(expected, '\n')
print(observed, '\n')
# extracting the pearson row from `stats`
num_effects_p_value = stats.loc[0, 'pval']
num_effects_p_valueBased on the result of the chi-square test of independence, we can conclude the following:
- Test statistic:
chi2= 1.799644 - Degrees of freedom:
dof= 3 - p-value:
pval= 0.615012
Since the p-value is 0.615012, which is greater than the significance level (0.01), we fail to reject the null hypothesis. Therefore, we do have enough evidence to suggest an association between the variables trx and num_effects.
# Check whether the counts of each group in `trx` are big enough
counts = drug_safety['trx'].value_counts()
(counts >= 30).all()# Create subplots with 1 row and 2 columns
g, ax = plt.subplots(1, 2)
# Define the bins for the histograms
bins = np.arange(np.min(drug_safety['age']), np.max(drug_safety['age']), step=2)
# Plot the histogram for the 'age' column in the 'Drug' group
ax[0].hist(data=drug_safety[drug_safety['trx']=='Drug'], x='age', bins=bins)
# Plot the histogram for the 'age' column in the 'Placebo' group
ax[1].hist(data=drug_safety[drug_safety['trx']=='Placebo'], x='age', bins=bins)
# Display the plot
ax[0].set_ylabel('Count')
ax[0].set_xlabel('Age (Drug)')
ax[1].set_xlabel('Age (Placebo)')
g.suptitle('Age distribution by Drug and Placebo groups')
plt.show()pingouin.normality(drug_safety, 'age', 'trx')Significant difference between the ages of both groups
To ensure age wasn't a confounder, conduct a Mann-Whitney test to determine if age differed significantly between the trx groups:
# Perform an Mann-Whitney test to compare the ages between the "Drug" and "Placebo" groups
two_ind_samp_results = pingouin.mwu(
x=drug_safety.loc[drug_safety['trx']=='Drug', 'age'],
y=drug_safety.loc[drug_safety['trx']=='Placebo', 'age']
)
# Print the p-value from t-test result
age_group_effects_p_value = two_ind_samp_results[ 'p-val']
age_group_effects_p_value['MWU']