Photo by Jannis Lucas on Unsplash.
Every year, American high school students take SATs, which are standardized tests intended to measure literacy, numeracy, and writing skills. There are three sections - reading, math, and writing, each with a maximum score of 800 points. These tests are extremely important for students and colleges, as they play a pivotal role in the admissions process.
Analyzing the performance of schools is important for a variety of stakeholders, including policy and education professionals, researchers, government, and even parents considering which school their children should attend.
You have been provided with a dataset called schools.csv, which is previewed below.
You have been tasked with answering three key questions about New York City (NYC) public school SAT performance.
# Re-run this cell
import pandas as pd
# Read in the data
schools = pd.read_csv("schools.csv")
# Preview the data
schools.head()
# Load dataset
df = pd.read_csv('schools.csv')
# Convert SAT columns to numeric, if needed
df['average_math'] = pd.to_numeric(df['average_math'], errors='coerce')
df['average_reading'] = pd.to_numeric(df['average_reading'], errors='coerce')
df['average_writing'] = pd.to_numeric(df['average_writing'], errors='coerce')Schools with the Best Math Results We define the best math results as scores equal to or greater than 80% of the maximum possible (640 out of 800).
We filter the dataset to only include these schools and sort them in descending order of their average math score.
# Filter schools with average math score ≥ 640
best_math_schools = df[df['average_math'] >= 640][['school_name', 'average_math']]
best_math_schools = best_math_schools.sort_values(by='average_math', ascending=False)
best_math_schools.head()Top 10 Schools by Total SAT Score
We create a new column, total_SAT, which is the sum of the three SAT components.
Then, we select the top 10 schools based on this total
# Calculate total SAT score
df['total_SAT'] = df['average_math'] + df['average_reading'] + df['average_writing']
# Select and sort
top_10_schools = df[['school_name', 'total_SAT']].sort_values(by='total_SAT', ascending=False).head(10)
top_10_schoolsBorough with Highest Variation in Total SAT
We group the data by borough and calculate:
- Number of schools
- Mean total SAT score
- Standard deviation of total SAT score
We identify the borough with the highest standard deviation.
# Group by borough
borough_stats = df.groupby('borough')['total_SAT'].agg(['count', 'mean', 'std']).reset_index()
# Rename and round values
borough_stats.columns = ['borough', 'num_schools', 'average_SAT', 'std_SAT']
borough_stats['average_SAT'] = borough_stats['average_SAT'].round(2)
borough_stats['std_SAT'] = borough_stats['std_SAT'].round(2)
# Get borough with highest std deviation
largest_std_dev = borough_stats.sort_values(by='std_SAT', ascending=False).head(1)
largest_std_dev