You're working as a sports journalist at a major online sports media company, specializing in soccer analysis and reporting. You've been watching both men's and women's international soccer matches for a number of years, and your gut instinct tells you that more goals are scored in women's international football matches than men's. This would make an interesting investigative article that your subscribers are bound to love, but you'll need to perform a valid statistical hypothesis test to be sure!
While scoping this project, you acknowledge that the sport has changed a lot over the years, and performances likely vary a lot depending on the tournament, so you decide to limit the data used in the analysis to only official FIFA World Cup matches (not including qualifiers) since 2002-01-01.
You create two datasets containing the results of every official men's and women's international football match since the 19th century, which you scraped from a reliable online source. This data is stored in two CSV files: women_results.csv and men_results.csv.
The question you are trying to determine the answer to is:
Are more goals scored in women's international soccer matches than men's?
You assume a 10% significance level, and use the following null and alternative hypotheses:
# Start your code here
import pandas as pd
import matplotlib.pyplot as plt
import pingouin
from scipy.stats import mannwhitneyu
women = pd.read_csv('women_results.csv', index_col=0)
men = pd.read_csv('men_results.csv', index_col=0)
print(women.head(), '\n')
print(men.head())#exploring the data to get insight into it
print(women.dtypes, "\n")
print(men.dtypes, "\n")
print(women.columns, "\n")
print(men.columns, "\n")
print(men.value_counts(), "\n")
print(women.value_counts(), "\n")
print(men.values, "\n") # Removed parentheses after 'values'
print(women.values, "\n")#filtering the data frame
df_women = women.loc[(women['tournament'] == 'FIFA World Cup') & (women['date'] > '2002-01-01')]
df_men = men.loc[(men['tournament'] == 'FIFA World Cup') & (men['date'] > '2002-01-01')]
print(df_women.head(), '\n')
print(df_men.head())#calculating the total goals scored by each group
df_women['goals_scored'] = df_women['home_score'] + df_women['away_score']
df_men['goals_scored'] = df_men['home_score'] + df_men['away_score']
print(df_women['goals_scored'].sum(), "\n")
print(df_men['goals_scored'].sum())import matplotlib.pyplot as plt
plt.hist(x=df_women['goals_scored'], bins=10)
plt.hist(x=df_men['goals_scored'], bins=10)men_subset["group"] = "men"
women_subset["group"] = "women"
men_subset["goals_scored"] = men_subset["home_score"] + men_subset["away_score"]
women_subset["goals_scored"] = women_subset["home_score"] + women_subset["away_score"]
## Filter the data to only include official FIFA World Cup matches that took place after 2002-01-01.
men["date"] = pd.to_datetime(men["date"])
women["date"] = pd.to_datetime(women["date"])
men_data = men[(men["date"] > "2002-01-01") & (men["tournament"].isin(["FIFA World Cup"]))]
women_data = women[(women["date"] > "2002-01-01") & (women["tournament"].isin(["FIFA World Cup"]))]
## Choosing the correct hypothesis test
men_data["group"] = "men"
women_data["group"] = "women"
men_data["goals_scored"] = men_data["home_score"] + men_data["away_score"]
women_data["goals_scored"] = women_data["home_score"] + women_data["away_score"]
## Determine normality using histograms
men_data["goals_scored"].hist()
plt.show()
plt.clf()
## Goals scored is not normally distributed, so use Wilcoxon-Mann-Whitney test of two groups
men_data["goals_scored"].hist()
plt.show()
plt.clf()
## Combine women's and men's data using pd.concat() and calculate goals scored in each match
both = pd.concat([women_data, men_data], axis=0, ignore_index=True)
## Transform the data for the pingouin Mann-Whitney U t-test
both_data = both[["goals_scored", "group"]]
both_subset_wide = both_data.pivot(columns="group", values="goals_scored")
## Perform right-tailed "greater" Wilcoxon-Mann-Whitney test with "pingouin"
results = pingouin.mwu(x=both_subset_wide["women"],
y=both_subset_wide["men"],
alternative="greater")
## Alternative SciPy solution: Perform right-tailed Wilcoxon-Mann-Whitney test with "scipy"
results_scipy = mannwhitneyu(x=women_subset["goals_scored"],
y=men_subset["goals_scored"],
alternative="greater")
## p-value.
p_val = results["p-val"].values[0]
## Hypoth. testing using sig. level 10 %
if p_val <= 0.01:
result = "reject"
else:
result = "fail to reject"
result_dict = {"p_val": p_val, "result": result}
print(result_dict)
#Reject null hypotheses