Loan Data
This dataset consists of more than 9,500 loans with information on the loan structure, the borrower, and whether the loan was pain back in full. This data was extracted from LendingClub.com, which is a company that connects borrowers with investors.
Not sure where to begin? Scroll to the bottom to find challenges!
import pandas as pd
loan_data = pd.read_csv("loan_data.csv")
print(loan_data.shape)
loan_data.head(100)Data dictionary
| Variable | Explanation | |
|---|---|---|
| 0 | credit_policy | 1 if the customer meets the credit underwriting criteria; 0 otherwise. |
| 1 | purpose | The purpose of the loan. |
| 2 | int_rate | The interest rate of the loan (more risky borrowers are assigned higher interest rates). |
| 3 | installment | The monthly installments owed by the borrower if the loan is funded. |
| 4 | log_annual_inc | The natural log of the self-reported annual income of the borrower. |
| 5 | dti | The debt-to-income ratio of the borrower (amount of debt divided by annual income). |
| 6 | fico | The FICO credit score of the borrower. |
| 7 | days_with_cr_line | The number of days the borrower has had a credit line. |
| 8 | revol_bal | The borrower's revolving balance (amount unpaid at the end of the credit card billing cycle). |
| 9 | revol_util | The borrower's revolving line utilization rate (the amount of the credit line used relative to total credit available). |
| 10 | inq_last_6mths | The borrower's number of inquiries by creditors in the last 6 months. |
| 11 | delinq_2yrs | The number of times the borrower had been 30+ days past due on a payment in the past 2 years. |
| 12 | pub_rec | The borrower's number of derogatory public records. |
| 13 | not_fully_paid | 1 if the loan is not fully paid; 0 otherwise. |
Source of dataset.
Don't know where to start?
Challenges are brief tasks designed to help you practice specific skills:
- 🗺️ Explore: Generate a correlation matrix between the numeric columns. What columns are positively and negatively correlated with each other? Does it change if you segment it by the purpose of the loan?
- 📊 Visualize: Plot histograms for every numeric column with a color element to segment the bars by
not_fully_paid. - 🔎 Analyze: Do loans with the same purpose have similar qualities not shared by loans with differing purposes? You can consider only fully paid loans.
Scenarios are broader questions to help you develop an end-to-end project for your portfolio:
You recently got a job as a machine learning scientist at a startup that wants to automate loan approvals. As your first project, your manager would like you to build a classifier to predict whether a loan will be paid back based on this data. There are two things to note. First, there is class imbalance; there are fewer examples of loans not fully paid. Second, it's more important to accurately predict whether a loan will not be paid back rather than if a loan is paid back. Your manager will want to know how you accounted for this in training and evaluation your model.
You will need to prepare a report that is accessible to a broad audience. It will need to outline your motivation, analysis steps, findings, and conclusions.
Data Cleaning
# get info
print(loan_data.info())
display(loan_data.head())
loan_data['purpose']=loan_data['purpose'].astype('category')
print(loan_data['purpose'].unique())Cross correlate numeric columns and display as heatmap
import matplotlib.pyplot as plt
import seaborn as sns
loan_datanum=loan_data.drop('purpose',axis=1)
loan_corr=loan_datanum.corr()
sns.heatmap(loan_corr)convert to numeric columns and cross correlate purpose categories
#get dummies
flip_loan = pd.get_dummies(loan_data['purpose'], drop_first=True)
flip_loan['credit_policy']=loan_data['credit.policy']
sns.heatmap(flip_loan.corr(), annot=True)
plt.show()
Remark:
Credit_policy has no correlation with purpose of credit so we can drop purpose column from further analysis.
=========================
Plot histograms for each numerical feature
cols=loan_datanum.columns
cols=cols.drop('not.fully.paid')
ncol=len(cols)
for i in range(ncol):
col=cols[i]
sns.histplot(data=loan_datanum, x=col, hue='not.fully.paid')
plt.show()
Remark:
Different features have different distributions but overall unpaid percent remains at about 10-20 percent.
Examine if there are differences between purpose categories