Project: Exploring NYC Public School Test Result Scores

Photo by Jannis Lucas on Unsplash.

Every year, American high school students take SATs, which are standardized tests intended to measure literacy, numeracy, and writing skills. There are three sections - reading, math, and writing, each with a maximum score of 800 points. These tests are extremely important for students and colleges, as they play a pivotal role in the admissions process.

Analyzing the performance of schools is important for a variety of stakeholders, including policy and education professionals, researchers, government, and even parents considering which school their children should attend.

You have been provided with a dataset called schools.csv, which is previewed below.

You have been tasked with answering three key questions about New York City (NYC) public school SAT performance.

# Re-run this cell 
import pandas as pd

# Read in the data
schools = pd.read_csv("schools.csv")

# Preview the data
schools.head()

# Start coding here...
# Add as many cells as you like...

# 1. Finding schools with the best math scores
# Calculate the threshold for 80% of the maximum possible score of 800
math_threshold = 0.8 * 800

# Filter the data for schools meeting this threshold for "average_math"
best_math_schools = schools[schools["average_math"] >= math_threshold][["school_name", "average_math"]]

# Sort the results by "average_math" in descending order
best_math_schools = best_math_schools.sort_values(by="average_math", ascending=False)

# Round the numeric values to two decimal places
best_math_schools["average_math"] = best_math_schools["average_math"].round(2)

# 2. Identifying the top 10 performing schools
# Add a new column called "total_SAT" to the original DataFrame
schools["total_SAT"] = schools["average_math"] + schools["average_reading"] + schools["average_writing"]

# Sort the values of the DataFrame by "total_SAT" in descending order
top_10_schools = schools.sort_values(by="total_SAT", ascending=False)[["school_name", "total_SAT"]].head(10)

# Round the numeric values to two decimal places
top_10_schools["total_SAT"] = top_10_schools["total_SAT"].round(2)

# 3. Locating the NYC borough with the largest standard deviation in SAT performance
# Group by "borough" and calculate the standard deviation of "total_SAT"
borough_std = schools.groupby("borough")["total_SAT"].std().reset_index()

# Add a column for the number of schools in each borough
borough_std["num_schools"] = schools.groupby("borough")["school_name"].count().values

# Add a column for the average SAT score in each borough
borough_std["average_SAT"] = schools.groupby("borough")["total_SAT"].mean().values

# Rename the standard deviation column for clarity
borough_std = borough_std.rename(columns={"total_SAT": "std_SAT"})

# Find the borough with the largest standard deviation
largest_std_dev = borough_std.sort_values(by="std_SAT", ascending=False).head(1)

# Round the numeric values to two decimal places
largest_std_dev["std_SAT"] = largest_std_dev["std_SAT"].round(2)
largest_std_dev["average_SAT"] = largest_std_dev["average_SAT"].round(2)

# 4. Create the appropriate data visualization to show the insights
import matplotlib.pyplot as plt
import seaborn as sns

# Plot the top 10 performing schools
plt.figure(figsize=(10, 6))
sns.barplot(x="total_SAT", y="school_name", data=top_10_schools, palette="viridis")
plt.title("Top 10 Performing Schools Based on Total SAT Scores")
plt.xlabel("Total SAT Score")
plt.ylabel("School Name")
plt.show()

# Plot the standard deviation of SAT scores by borough
plt.figure(figsize=(10, 6))
sns.barplot(x="std_SAT", y="borough", data=borough_std.sort_values(by="std_SAT", ascending=False), palette="magma")
plt.title("Standard Deviation of SAT Scores by Borough")
plt.xlabel("Standard Deviation of Total SAT Score")
plt.ylabel("Borough")
plt.show()

# Draw Conclusions
conclusions = """
Key Insights:
1. The schools with the best math results have average math scores of at least 80% of the maximum possible score of 800.
2. The top 10 performing schools based on the combined SAT scores have been identified and visualized.
3. The borough with the largest standard deviation in the combined SAT score has been identified, indicating variability in performance within that borough.

These insights can help stakeholders understand which schools are excelling in math, which schools have the highest overall SAT performance, and which boroughs have the most variability in SAT scores.
"""
print(conclusions)