Welcome to New York City, one of the most-visited cities in the world. There are many Airbnb listings in New York City to meet the high demand for temporary lodging for travelers, which can be anywhere between a few nights to many months. In this notebook, we will take a closer look at the New York Airbnb market by combining data from multiple file types like .csv, .tsv, and .xlsx.
Recall that CSV, TSV, and Excel files are three common formats for storing data. Three files containing data on 2019 Airbnb listings are available to you:
data/airbnb_price.csv
listing_id: unique identifier of listingprice: nightly listing price in USDnbhood_full: name of borough and neighborhood where listing is located
data/airbnb_room_type.xlsx This is an Excel file containing data on Airbnb listing descriptions and room types.
listing_id: unique identifier of listingdescription: listing descriptionroom_type: Airbnb has three types of rooms: shared rooms, private rooms, and entire homes/apartments
data/airbnb_last_review.tsv This is a TSV file containing data on Airbnb host names and review dates.
listing_id: unique identifier of listinghost_name: name of listing hostlast_review: date when the listing was last reviewed
Our goals are to convert untidy data into appropriate formats to analyze, and answer key questions including:
- What is the average price, per night, of an Airbnb listing in NYC?
- How does the average price of an Airbnb listing, per month, compare to the private rental market?
- How many adverts are for private rooms?
- How do Airbnb listing prices compare across the five NYC boroughs?
# Import important libraries
import numpy as np
import pandas as pd
import datetime as dt# Import relevant files
prices = pd.read_csv('data/airbnb_price.csv')
room_types = pd.read_excel('data/airbnb_room_type.xlsx')
reviews = pd.read_table('data/airbnb_last_review.tsv')# Save files as csv
room_types.to_csv('data/airbnb_room_type.csv', index=None)
reviews.to_csv('data/airbnb_last_review.csv', index=None)# Read converted files
room_types = pd.read_csv('data/airbnb_room_type.csv')
reviews = pd.read_csv('data/airbnb_last_review.csv')# Delete everything after space in price column
prices['price'] = prices['price'].str.replace(' dollars', '')# Change price column datayype to numeric type
prices['price'] = pd.to_numeric(prices['price'])# We notice that there are some outliers when we look at 'min' and 'max'
prices['price'].describe()# We update prices data, deleting where price is zero
zero_prices = prices['price'] == 0
prices = prices.loc[~zero_prices]# Average price
avg_price = round(prices['price'].mean(),2)
# Price per month
prices['price_per_month'] = prices['price']*365/12
# Average price per month
avg_price_per_month = round(prices['price_per_month'].mean(), 2)
# Difference between private market
difference = round((avg_price_per_month-3100), 2)# Change room_type values to lower case
room_types['room_type'] = room_types['room_type'].str.lower()
# Change room_type data type to category
room_types['room_type'] = room_types['room_type'].astype('category')
# Count room_type values as percentile
room_frequencies = room_types['room_type'].value_counts()# Change last_review datatype from string to datetime
reviews['last_review'] = pd.to_datetime(reviews['last_review'])
# Find first review
first_reviewed = reviews['last_review'].dt.date.min()
# Find last review
last_reviewed = reviews['last_review'].dt.date.max()
#print("First review: ", first_reviewed)
#print("Last review: ", last_reviewed)# Merge files based on listing_id column
rooms_and_prices = prices.merge(room_types, how='outer', on='listing_id')
airbnb_merged = rooms_and_prices.merge(reviews, how='outer', on='listing_id')# Drop rows where there is NA values
airbnb_merged.dropna(inplace=True)# Checking is there any duplicate
airbnb_merged.duplicated().sum()