Skip to content

Welcome to New York City, one of the most-visited cities in the world. There are many Airbnb listings in New York City to meet the high demand for temporary lodging for travelers, which can be anywhere between a few nights to many months. In this notebook, we will take a closer look at the New York Airbnb market by combining data from multiple file types like .csv, .tsv, and .xlsx.

Recall that CSV, TSV, and Excel files are three common formats for storing data. Three files containing data on 2019 Airbnb listings are available to you:

data/airbnb_price.csv

  • listing_id: unique identifier of listing
  • price: nightly listing price in USD
  • nbhood_full: name of borough and neighborhood where listing is located

data/airbnb_room_type.xlsx This is an Excel file containing data on Airbnb listing descriptions and room types.

  • listing_id: unique identifier of listing
  • description: listing description
  • room_type: Airbnb has three types of rooms: shared rooms, private rooms, and entire homes/apartments

data/airbnb_last_review.tsv This is a TSV file containing data on Airbnb host names and review dates.

  • listing_id: unique identifier of listing
  • host_name: name of listing host
  • last_review: date when the listing was last reviewed

Our goals are to convert untidy data into appropriate formats to analyze, and answer key questions including:

  • What is the average price, per night, of an Airbnb listing in NYC?
  • How does the average price of an Airbnb listing, per month, compare to the private rental market?
  • How many adverts are for private rooms?
  • How do Airbnb listing prices compare across the five NYC boroughs?
# 1
# We've loaded your first package for you! You can add as many cells as you need.
import numpy as np
import pandas as pd
import datetime as dt

# Load Data
prices = pd.read_csv("data/airbnb_price.csv")
xls = pd.ExcelFile("data/airbnb_room_type.xlsx")
room_types = xls.parse(0)
reviews = pd.read_csv("data/airbnb_last_review.tsv", sep = '\t')
# 1 Solution 
import numpy as np
import pandas as pd
import datetime as dt

# Load airbnb_price.csv, prices
prices = pd.read_csv("data/airbnb_price.csv")

# Load airbnb_room_type.xlsx, xls
xls = pd.ExcelFile("data/airbnb_room_type.xlsx")

# Parse the first sheet from xls, room_types
room_types = xls.parse(0)

# Load airbnb_last_review.tsv, reviews
reviews = pd.read_csv("data/airbnb_last_review.tsv", sep="\t")
# 2
# Clean price column
prices.info()

prices['price'] = prices['price'].str.strip('dollars')
prices['price'] = prices['price'].str.strip()
prices['price'] = prices['price'].astype('int')

assert prices['price'].dtype == 'int'
# 2 Solution
# Remove whitespace and string characters from prices column
prices["price"] = prices["price"].str.replace(" dollars", "")

# Convert prices column to numeric datatype
prices["price"] = pd.to_numeric(prices["price"])

print(prices.info())
Run cancelled
prices.describe()
# 3
# Calculate average price
avg_price = prices.query('price >0')
avg_price = avg_price['price'].mean()
avg_price = round(avg_price,2)
print('Average Price: ',avg_price)
#3 solution
# Subset prices for listings costing $0, free_listings
free_listings = prices["price"] == 0

# Update prices by removing all free listings from prices
prices = prices.loc[~free_listings]

# Calculate the average price, avg_price
avg_price = round(prices["price"].mean(), 2)

print(avg_price)
#4 
# Calculate price per month
prices['price_per_month'] = prices['price'] * 365 / 12

# Calculate avg price per month
average_price_per_month = round(prices['price_per_month'].mean(),2)
print('Average price per month: ', average_price_per_month)

# Calculate difference between avg price per month airbnb and privat market 
difference = round(average_price_per_month - 3100,2)
print('Difference: ', difference)
#4 Solution
# Add a new column to the prices DataFrame, price_per_month
prices["price_per_month"] = prices["price"] * 365 / 12

# Calculate average_price_per_month
average_price_per_month = round(prices["price_per_month"].mean(), 2)
difference = round((average_price_per_month - 3100),2)

print('Average price per month: ', average_price_per_month)
print(difference)
#5
# Cleaning room_type column 
room_types['room_type'] = room_types['room_type'].str.lower()
room_types['room_type'].astype('object')
# Number of Rooms 
room_frequencies = room_types['room_type'].value_counts()

print(room_frequencies)
# 5 Solution 
# Convert the room_type column to lowercase
room_types["room_type"] = room_types["room_type"].str.lower()

# Update the room_type column to category data type
room_types["room_type"] = room_types["room_type"].astype("category")

# Create the variable room_frequencies
room_frequencies = room_types["room_type"].value_counts()

print(room_frequencies)
#6
# Clean last_review column 
reviews["last_review"] = pd.to_datetime(reviews["last_review"]).dt.date

# first review
first_reviewed = reviews["last_review"].min()
print('Date of First Review: ', first_reviewed)

# last review
last_reviewed = reviews["last_review"].max()
print('Date of Last Review: ', last_reviewed)
# 6 Solution 
# Change the data type of the last_review column to datetime
reviews["last_review"] = pd.to_datetime(reviews["last_review"])

# Create first_reviewed, the earliest review date
first_reviewed = reviews["last_review"].dt.date.min()

# Create last_reviewed, the most recent review date
last_reviewed = reviews["last_review"].dt.date.max()

print('Date of First Review: ', first_reviewed)
print('Date of Last Review: ', last_reviewed)
Run cancelled
help(prices.merge)
‌
‌
‌