Skip to content
Soccer Data
  • AI Chat
  • Code
  • Report
  • Soccer Data

    This dataset contains data of every game from the 2018-2019 season in the English Premier League.

    Not sure where to begin? Scroll to the bottom to find challenges!

    #This are a few questions to answer:
    #Are shots related to goals?
    #Are shots on target related to goals?
    #Are shots on target related to wins?
    #Does winning at HT increases wins?
    #Are corners related to goals?
    
    import pandas as pd
    import seaborn as sns
    import numpy as np
    import matplotlib.pyplot as plt
    from statsmodels.formula.api import ols
    
    soccer = pd.read_csv("soccer18-19.csv")
    soccer
    

    Data Dictionary

    ColumnExplanation
    DivDivision the game was played in
    DateThe date the game was played
    HomeTeamThe home team
    AwayTeamThe away team
    FTHGFull time home goals
    FTAGFull time away goals
    FTRFull time result
    HTHGHalf time home goals
    HTAGHalf time away goals
    HTRHalf time result
    RefereeThe referee of the game
    HSNumber of shots taken by home team
    ASNumber of shots taken by away team
    HSTNumber of shots taken by home team on target
    ASTNumber of shots taken by away team on target
    HFNumber of fouls made by home team
    AFNumber of fouls made by away team
    HCNumber of corners taken by home team
    ACNumber of corners taken by away team
    HYNumber of yellow cards received by home team
    AYNumber of yellow cards received by away team
    HRNumber of red cards received by home team
    ARNumber of red cards received by away team

    Source of dataset.

    Don't know where to start?

    Challenges are brief tasks designed to help you practice specific skills:

    • 🗺️ Explore: What team commits the most fouls?
    • 📊 Visualize: Plot the percentage of games that ended in a draw over time.
    • 🔎 Analyze: Does the number of red cards a team receives have an effect on its probability of winning a game?

    Scenarios are broader questions to help you develop an end-to-end project for your portfolio:

    You have just been hired as a data analyst for a local soccer team. The team has recently signed on some junior players and wants to give them as much experience as possible without losing games. If the head coach could be confident in the outcome of a game by halftime, they would be more likely to give the junior players time on the field.

    The coach has asked you whether you can predict the outcome of a game by the results at halftime and how confident you would be in the prediction.

    You will need to prepare a report that is accessible to a broad audience. It should outline your motivation, steps, findings, and conclusions.

    #Exploratory Analysis
    
    #Converting Date column to datetime object
    soccer['Date'] = pd.to_datetime(soccer['Date'])
    print(soccer.describe())
    
    #Full time result to binary
    
    #Team with most fouls
    Hfouls = soccer.groupby('HomeTeam')['HF'].sum()
    Hfouls = pd.DataFrame(Hfouls,columns = ['HF'])
    Afouls = soccer.groupby('AwayTeam')['AF'].sum()
    Afouls = pd.DataFrame(Afouls,columns = ['AF'])
    Total_fouls = Hfouls.merge(Afouls, how = 'left', left_index = True, right_index = True)
    Total_fouls['Total'] = Total_fouls['HF'] + Total_fouls['AF'] 
    print(Total_fouls)
    print(Total_fouls.loc[Total_fouls['Total'] == max(Total_fouls['Total'])])
    
    #Percentage of draws over time
    soccer['draw'] = (soccer['FTHG'] == soccer['FTAG']).astype(int)
    draws = soccer['draw'].sum()
    print(draws)
    
    # Defining wins
    soccer['Home_win'] = (soccer['FTHG'] > soccer['FTAG']).astype(int) 
    soccer['Away_win'] = (soccer['FTHG'] < soccer['FTAG']).astype(int) 
    
    # Calculating Total Wins
    Hwins = soccer.groupby('HomeTeam')['Home_win'].sum()
    Hwins = pd.DataFrame(Hwins,columns = ['Home_win'])
    Awins = soccer.groupby('AwayTeam')['Away_win'].sum()
    Awins = pd.DataFrame(Awins, columns = ['Away_win'] )
    Total_wins = Hwins.merge(Awins, how = 'left', left_index = True, right_index = True)
    Total_wins.index.names = ['Team']
    
    #Home_and_total = Total_wins.merge(Hwins, how = 'left', on = 'HomeTeam')
    print(Total_wins)
    
    #Calculating Total Shots
    Hc = soccer.groupby('HomeTeam')['HC'].sum()
    Hc = pd.DataFrame(Hc,columns = ['HC'])
    Ac = soccer.groupby('AwayTeam')['AC'].sum()
    Ac = pd.DataFrame(Ac,columns = ['AC'])
    Total_corners = Hc.merge(Ac, how = 'left', left_index = True, right_index = True)
    Total_corners.index.names = ['Team']
    print(Total_corners)
    
    #Calculating Total Red Cards
    Hred = soccer.groupby('HomeTeam')['HR'].sum()
    Hred = pd.DataFrame(Hred,columns = ['HR'])
    Ared = soccer.groupby('AwayTeam')['AR'].sum()
    Ared = pd.DataFrame(Ared,columns = ['AR'])
    Total_reds = Hred.merge(Ared, how = 'left', left_index = True, right_index = True)
    Total_reds.index.names = ['Team']
    print(Total_reds)
    
    #Corners
    Hshots = soccer.groupby('HomeTeam')['HS'].sum()
    Hshots = pd.DataFrame(Hshots,columns = ['HS'])
    Ashots = soccer.groupby('AwayTeam')['AS'].sum()
    Ashots = pd.DataFrame(Ashots,columns = ['AS'])
    Total_shots = Hshots.merge(Ashots, how = 'left', left_index = True, right_index = True)
    Total_shots.index.names = ['Team']
    print(Total_shots)
    
    #Merging all stats
    Total_stats = Total_wins.merge(Total_shots, how = 'left', left_index = True, right_index = True)
    Total_stats.columns = ['Home_win', 'Away_win', 'Home_shots', 'Away_shots']
    Total_stats = Total_stats.merge(Total_reds, how = 'left', left_index = True, right_index = True)
    Total_stats['Total_wins'] = Total_stats['Home_win'] + Total_stats['Away_win']
    Total_stats['Total_shots'] = Total_stats['Home_shots'] + Total_stats['Away_shots']
    Total_stats['Total_reds'] = Total_stats['HR'] + Total_stats['AR']
    #Total_stats['Total_corners'] = Total_corners['HC'] = Total_corners['AC']
    print(Total_stats)
    
    
    
    
    
    
    #Percentage of draws over time
    soccer['draw'] = (soccer['FTHG'] == soccer['FTAG']).astype(int)
    draws = soccer['draw'].sum()
    Wins = Total_stats['Total_wins'].sum()
    Percentage = draws/Wins*100
    print('The percentage of draws over time is' ,Percentage)
    
    #Total Wins vs Shots
    plt.figure()
    sns.scatterplot(data = Total_stats, x = 'Total_wins', y = 'Total_shots')
    sns.regplot(data = Total_stats, x = 'Total_wins', y = 'Total_shots')
    plt.title('Relation between shots and wins')
    plt.xlabel('Wins')
    plt.ylabel('Shots')
    
    #Total wins vs red cards
    plt.figure()
    sns.scatterplot(data = Total_stats, x = 'Total_wins', y = 'Total_reds')
    sns.regplot(data = Total_stats, x = 'Total_wins', y = 'Total_reds')
    plt.title('Relation between red cards and wins')
    plt.xlabel('Wins')
    plt.ylabel('Red cards')
    
    #Linear Regression more shots -> more wins
    shots_vs_wins = ols('Total_wins ~ Total_shots', data = Total_stats)
    shots_vs_wins = shots_vs_wins.fit()
    print(shots_vs_wins.summary())
    
    #Linear Regression more shots -> more wins
    reds_vs_wins = ols('Total_wins ~ Total_reds', data = Total_stats)
    reds_vs_wins = reds_vs_wins.fit()
    print(reds_vs_wins.summary())
    
    #Convert to tidy data
    #soccer_tidy = pd.melt(soccer[['HomeTeam', 'Home_win']], ['HomeTeam'])
    #soccer_tidy = soccer_tidy.sort_values(by = 'HomeTeam')
    #rint(soccer_tidy)
    
    #Same steps as before using shots on target
    Hshots_target = soccer.groupby('HomeTeam')['HST'].sum()
    Hshots_target = pd.DataFrame(Hshots_target,columns = ['HST'])
    Ashots_target = soccer.groupby('AwayTeam')['AST'].sum()
    Ashots_target = pd.DataFrame(Ashots_target,columns = ['AST'])
    Total_shots_target = Hshots_target.merge(Ashots_target, how = 'left', left_index = True, right_index = True)
    Total_shots_target.index.names = ['Team']
    print(Total_shots)
    
    Total_stats_target = Total_wins.merge(Total_shots_target, how = 'left', left_index = True, right_index = True)
    Total_stats_target.columns = ['Home_win', 'Away_win', 'Home_shots', 'Away_shots']
    Total_stats_target['Total_wins'] = Total_stats_target['Home_win'] + Total_stats_target['Away_win']
    Total_stats_target['Total_shots_on_target'] = Total_stats_target['Home_shots'] + Total_stats_target['Away_shots']
    Total_stats_target['Total_corners'] = Total_corners['HC'] = Total_corners['AC']
    Total_stats_target['Total_reds'] = Total_reds['HR'] + Total_reds['AR']
    Total_stats_target['Total_fouls'] = Total_fouls['Total']
    print(Total_stats_target)
    
    #Scatterplot with regression line Shots on target vs wins
    sns.scatterplot(data = Total_stats_target, x = 'Total_wins', y = 'Total_shots_on_target')
    sns.regplot(data = Total_stats_target, x = 'Total_wins', y = 'Total_shots_on_target')
    plt.title('Relation between shots on target and wins')
    plt.xlabel('Wins')
    plt.ylabel('Shots on target')
    plt.show()
    
    #Scatterplot with regression line fouls vs wins
    sns.scatterplot(data = Total_stats_target, x = 'Total_wins', y = 'Total_fouls')
    sns.regplot(data = Total_stats_target, x = 'Total_wins', y = 'Total_fouls')
    plt.title('Relation between fouls and wins')
    plt.xlabel('Wins')
    plt.ylabel('Fouls')
    plt.show()
    
    #Linear Regression more shots on target -> more wins
    shots_vs_wins = ols('Total_wins ~ Total_shots_on_target', data = Total_stats_target)
    shots_vs_wins = shots_vs_wins.fit()
    print(shots_vs_wins.summary())
    #Home teams
    total_home_wins = soccer.groupby('HomeTeam')['Home_win'].sum()
    total_away_wins = soccer.groupby('AwayTeam')['Away_win'].sum()
    
    #Lineplot of home and away wins
    ax = sns.lineplot(data = total_home_wins, palette = 'blue', label="Home wins")
    ax = sns.lineplot(data = total_away_wins, palette = 'blue', label="Away wins")
    ax.set_xlabel('Team')
    ax.set_ylabel('Wins')
    ax.set_title('Comparison between home and away wins')
    ax.tick_params(axis = 'x', rotation = 90)
    
    print(soccer.groupby('HomeTeam')['Home_win'].mean())
    
    
    
    
    #most_goals = soccer.groupby('HomeTeam')['FTHG'].sum()
    #print(most_goals)
    
    #print(soccer)
    #print(soccer_tidy)
    #ax = sns.lineplot(data = foul_sum, x = 'HomeTeam', y = 'HF')
    #ax.tick_params(axis='x', rotation=90)
    
    #Machine Learning
    #from sklearn.neighbors import KNeighborsClassifier
    from sklearn.model_selection import train_test_split
    from sklearn.linear_model import LinearRegression
    from sklearn.metrics import mean_squared_error
    Total_stats_full = Total_stats_target[['Total_wins', 'Total_shots_on_target', 'Total_reds', 'Total_corners']]
    print(Total_stats_full)
    X = Total_stats_full.drop(['Total_wins', 'Total_reds'], axis = 1).values
    print(X)
    #X = X.reshape(-1,1)
    y = Total_stats_full['Total_wins'].values
    print(y)
    print(X.shape, y.shape)
    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.3, random_state=42)
    reg = LinearRegression()
    reg.fit(X_train, y_train)
    y_predict = reg.predict(X_test)
    reg_score = reg.score(X_test, y_test)
    reg_mse = mean_squared_error(y_test, y_predict)
    print(reg_score)
    print(reg_mse)
    
    #Model performance
    from sklearn.model_selection import cross_val_score, KFold
    kf = KFold(n_splits=4, shuffle=True, random_state=42)
    cv_results = cross_val_score(reg, X, y,cv= kf)
    print(cv_results)
    print(np.mean(cv_results))
    
    #from sklearn.metrics import classification_report, confusion_matrix
    #print(confusion_matrix(y_test, y_predict))
    #print(y_test)
    #print(y_predict)
    
    #GridSearchCV 
    from sklearn.model_selection import GridSearchCV
    kf = KFold(n_splits=5, shuffle=True, random_state=42)
    param_grid = {'alpha' : np.arange(0.0001,1,10), 'solver':['sag, lsqr']}
    ridge = Rid