http://datacamp-community-prod.s3.amazonaws.com/af9bb467-170d-41c9-a0bd-26e675384c4e

Make your own neural networks with this Keras cheat sheet to deep learning in Python for beginners, with code samples!

Keras Cheat Sheet: Neural Networks in Python

http://datacamp-community-prod.s3.amazonaws.com/eb807da5-dce5-4b97-a54d-74e89f14266b

A handy scikit-learn cheat sheet to machine learning with Python, including steps to implement machine learning algorithms successfully and code samples. 

Scikit-Learn Cheat Sheet: Python Machine Learning

https://www.datacamp.com/community/podcast/scaling-machine-learning-adoption-a-pragmatic-approach

<p>In this episode of DataFramed, we speak with Noah Gift, founder of Pragmatic AI Labs and prolific author about operationalizing machine learning in organizations and his new book Practical MLOPs.&nbsp;</p>

In this episode of DataFramed, we speak with Noah Gift, founder of Pragmatic AI Labs and prolific author about operationalizing machine learning in organizations and his new book Practical MLOPs. 

Scaling Machine Learning Adoption: A Pragmatic Approach

https://www.datacamp.com/community/blog/scaling-machine-learning-adoption-a-pragmatic-approach

https://www.datacamp.com/community/podcast/the-future-of-responsible-ai

<p>In this episode of DataFramed, Adel speaks with Maria Luciana Axente, Responsible AI and AI for Good Lead at PwC UK on the state and future of responsible AI.</p>

In this episode of DataFramed, Adel speaks with Maria Luciana Axente, Responsible AI and AI for Good Lead at PwC UK on the state and future of responsible AI.

The Future of Responsible AI

https://www.datacamp.com/community/blog/the-future-of-responsible-ai

https://www.datacamp.com/community/news/data-science-gambling-and-bookmaking-with-marco-blume-h3szzd7jfyl

Marco Blume, Trading Director at Pinnacle Sports talks about the role of data science in large-scale bets and bookmaking, how Marco is training an army of data scientists and much more.

Data Science, Gambling and Bookmaking (with Marco Blume)

https://www.datacamp.com/community/news/managing-data-science-teams-with-angela-bassa-nktiamvwi6

Angela Bassa, the Director of Data Science at iRobot, talks about managing data science teams and much more.

Managing Data Science Teams (with Angela Bassa)

https://www.datacamp.com/community/news/k-means-clustering-in-r-tutorial-cq3latvdphb

Learn all about clustering and, more specifically, k-means in this R Tutorial, where you'll focus on a case study with Uber data.

K-Means Clustering in R Tutorial

https://www.datacamp.com/community/news/the-10-statistical-techniques-data-scientists-need-to-master-ed6o27bk1b9

In this article, I present 10 statistical techniques which a data scientist needs to master. Build up your toolbox of data science tools by having a look at this great overview post.

The 10 Statistical Techniques Data Scientists Need to Master

<p>Today, high-performing insurance organizations are leveraging data science to advance their businesses in a competitive environment. In a <a target="_blank" href="https://www.datacamp.com/resources/webinars/future-of-data-science-in-insurance-1">recent webinar</a>, Sudaman Thoppan Mohanchandralal, Regional Chief Data and Analytics Officer at Allianz Benelux discussed the current state of data science in insurance and exciting use-cases of machine learning in the industry.     </p>
<h1 id="why-data-science-is-valuable-in-the-insurance-industry-today">Why Data Science is Valuable in the Insurance Industry Today</h1>
<p>Recently, the industry’s demand has shifted to more personalized services rather than those that appeal to mass markets. This <a target="_blank" href="https://www.mckinsey.com/industries/financial-services/our-insights/how-traditional-insurance-carriers-can-disrupt-through-personalized-marketing">McKinsey report</a> describes this shift Sudaman refers to. It explains how deep one-to-one relationships can be built with consumers at scale through strategic personalized marketing and service offerings.</p>
<p>These changes in needs and relationships are made possible by the massive growth of available data for individual customers as well as advances in machine learning and resources. In <a target="_blank" href="https://www.ibm.com/thought-leadership/institute-business-value/report/insurance-data-personalization">this article</a>, IBM discusses how insurers observe a 22-25% improvement in time to profitability when creating personalized services relative to mass-appealing products. These products can only be developed at scale using machine learning.  </p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1638880876/image3_lya3kn.png" /></center><br>
<br><center><a target="_blank" href="https://www.datacamp.com/groups/business"><img src="https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1624651228/schedule_a_demo_bfhlx1.png" /></a></center>
<h1 id="the-state-of-data-science-in-insurance">The state of data science in insurance</h1>
<p>Sudaman explains in the webinar that machine learning provides value when it is leveraged for predictions rather than causal inference and when the problem is sufficiently self-contained. As of today, machine learning fails when organizations attempt to use these algorithms as a crystal ball or black box rather than a tool to inform humans on directions to explore. Here’s a glimpse of some of the use-cases covered in the webinar:</p>
<h3 id="1-fraud-detection-in-insurance-claims">1. Fraud Detection in insurance Claims</h3>
<p>Machine learning performs very well on prediction tasks with a lot of data. Therefore, <a target="_blank" href="https://www.datacamp.com/courses/fraud-detection-in-python">fraud detection</a> is a great use case to train and deploy a classification algorithm such as logistic regression or a decision tree. This system can be used to flag claims that look suspicious. This makes fraud prevention significantly more manageable for an analyst as it will augment their workflow by surfacing the claims most likely to be fraudulent.</p>
<h3 id="2-identifying-customers-for-retention-activities">2. Identifying Customers for Retention Activities</h3>
<p>Another important use case for improving the scale of any organization is improving <a target="_blank" href="https://www.datacamp.com/courses/machine-learning-for-marketing-in-python">customer retention</a>. Machine learning can help to identify which customers are at risk of leaving your organization. Identifying these customers at scale and providing incentives for them to stay will lead to better customer lifetime value in the long run—enabling deeper personalization.</p>
<h3 id="3-optimizing-pricing-with-time-series-data-risk-premium-modeling-">3. Optimizing Pricing with Time Series Data (Risk Premium Modeling)</h3>
<p>Understanding a customer’s risk to file a claim is critical in determining the correct price and structure of a plan to provide them. All insurers are doing this at some level. With the expansion of available data, model accuracy to determine a customer’s risk profile can be improved to offer more <a target="_blank" href="https://www.datacamp.com/courses/life-insurance-products-valuation-in-r">competitive pricing</a> to individuals.</p>
<h3 id="4-future-disease-prediction">4. Future Disease Prediction</h3>
<p>Similar to the previous use case, understanding future disease risks is critical in portfolio optimization and pricing. <a target="_blank" href="https://www.lloyds.com/~/media/files/news-and-insight/coronavirus-hub/product-simplification-report/lloyds_product_simplification_report_final.pdf?la=en">This Lloyd’s of London report</a> discusses how the insurance industry will have to pay out upwards of $100 billion in COVID-related claims. While predicting the pandemic in advance would have been challenging, understanding and predicting health outcomes will enable insurers to prioritize and personalize their services.</p>
<h3 id="5-portfolio-optimization">5. Portfolio Optimization</h3>
<p>The final use case we will discuss is portfolio optimization. <a href="https://www.datacamp.com/courses/introduction-to-portfolio-risk-management-in-python">Portfolio management</a> consists of a few steps: identifying meaningful groupings of risk to be analyzed, informing optimal decisions, and identifying opportunities. Each of these subtasks is solvable through data science optimization techniques.</p>
<p>If you want to learn more about the future of data science in insurance, make sure to tune in to the webinar <a target="_blank" href="https://www.datacamp.com/resources/webinars/future-of-data-science-in-insurance-1">here</a>.</p>
<br><center><a target="_blank" href="https://www.datacamp.com/groups/business"><img src="https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1624651228/schedule_a_demo_bfhlx1.png" /></a></center>

https://www.datacamp.com/community/blog/data-science-in-insurance-today

The insurance industry is rife with data and data science use-cases that provide value. In a recent webinar, Allianz Benelux Regional Chief Data & Analytics Officer Sudaman T M explored the state of data science in the industry today.

Data Science in Insurance Today

https://www.datacamp.com/community/blog/scaling-ai-adoption-in-financial-services

In this episode of DataFramed, we speak with Shameek Kundu,  Chief Strategy Officer & Head of Financial Services at TruEra Inc about Scaling AI Adoption throughout financial services.

Scaling AI Adoption in Financial Services

<h2>Introduction to Python for Data Analysis</h2><p>This chapter will get you started with Python for Data Analysis. We will cover the reasons to learn Data Science using Python, provide an overview of the Python ecosystem and get you to write your first code in Python!</p><h2>Python Libraries and data structures</h2><p>In this chapter, we will introduce some of the most common data structures in Python to you and take you through some of the libraries we commonly use in data analysis.</p><h2>Exploratory analysis in Python using Pandas</h2><p>We start with the first step of data analysis - the exploratory data analysis.</p><h2>Data Munging in Python using Pandas</h2><p>Pandas is at the heart of data analysis in Python. This chapter gets you started with Data Munging in Python using Pandas</p><h2>Building a Predictive model in Python</h2><p>We build our predictive models and make submissions to the AV DataHack platform in this section.</p><h2>Expert advice to improve model performance</h2><p>This chapter will help to understand the approach of data science experts, "How they do approach a challenge?", "How to select a right algorithm?", "How to combine outputs of multiple algorithms?" and "How to select the right value of model parameter also known as parameter tuning?".</p>

https://www.datacamp.com/community/open-courses/introduction-to-python-machine-learning-with-analytics-vidhya-hackathons

This course introduces you to data science concepts, data exploration, and preparation in Python to prepare you to participate in machine learning competitions on Analytics Vidhya.

Introduction to Python & Machine Learning (with Analytics Vidhya Hackathons)

<h2>Getting Started with Python</h2><p>In this chapter we will go trough the essential steps that you will need to take before beginning to build predictive models.</p><h2>Predicting with Decision Trees</h2><p>After making your first predictions in the previous chapter, it's time to bring you to the next level. In chapter 2 you</p><h2>Improving your predictions through Random Forests</h2><p>What techniques can you use to improve your predictions even more? One possible way is by making use of the machine learning method Random Forest. Namely, a forest is just a collection of trees...</p>

https://www.datacamp.com/community/open-courses/kaggle-python-tutorial-on-machine-learning

Always wanted to compete in a Kaggle competition but not sure you have the right skill set? In this course, you'll learn to predict survival rate for Kaggle's Titanic competition.

Kaggle Python Tutorial on Machine Learning

<h2 id="table-of-contents-">Table of Contents:</h2>
<ul>
<li><a href="#what-is-the-normal-equation-">What is the Normal Equation?</li>
<li><a href="#the-normal-equation-vs-gradient-descent">The Normal Equation vs Gradient Descent<ul>
<li><a href="#gradient-descent">Gradient Descent</li>
<li><a href="#how-do-they-differ-">How do they differ?</li>
</ul>
</li>
<li><a href="#deciphering-when-to-use-the-normal-equation">Deciphering When to Use the Normal Equation</li>
<li><a href="#the-normal-equation-from-scratch-in-python">The Normal Equation from Scratch in Python</li>
<li><a href="#frequently-asked-questions-faqs-">FAQs</li>
</ul>
<p>Most problems we encounter have several ways they can be solved. For example, if we wanted to get from one side of a room to another, we may decide to walk around the room until we arrive at the opposing side, or we could just cut across.</p>
<p>The normal equation is just an emphasis of this concept. It’s just another way to solve a problem. What problem did you ask? We’ll cover that in the remainder of this article. For now, all you need to know is that it&#39;s an effective approach that can help you save lots of time when implementing linear regression under certain conditions. </p>
<p>Let’s dive deeper…</p>
<h2 id="what-is-the-normal-equation-">What is the Normal Equation?</h2>
<p>The normal equation is a closed-form solution used to find the value of θ that minimizes the cost function. Another way to describe the normal equation is as a one-step algorithm used to analytically find the coefficients that minimize the loss function. Both descriptions work, but what exactly do they mean? We will start with linear regression.</p>
<p>Linear regression makes a prediction, y_hat, by computing the weighted sum of input features plus a bias term. Mathematically it can be represented as follows:</p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238228/image5_iengil.png"></center>
<br>
<p>Where θ represents the parameters and n is the number of features.</p>
<p>Essentially, all that occurs in the above equation is the dot product of θ, and x is being summed. Thus, a more concise way to represent this is to use its vectorized form:</p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238228/image6_t3omdh.png"></center>
<br>
<p>h(θ) is the hypothesis function.</p>
<p>Given this approximate target function, we can use our model to make predictions. To determine if our model has learned well, it’s important we measure the performance of our model on the training data. For this purpose, we compute a loss function. The goal of the training process is to find the values of theta (θ) that minimize the loss function.  </p>
<p>Here’s how we can represent our loss function mathematically:</p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238227/image3_p2c9gn.png"></center>
<br>
<p>In the above equation, theta (θ) is a n + 1 dimensional vector, and our loss function is a function of the vector value. Consequently, the partial derivative of the loss function, J, has to be taken with respect to every parameter of θ_j in turn. All of them must equal zero. Following this process and solving for all of the values of θ from θ_0 to θ_n will result in the values of θ that minimize the loss function.</p>
<p>Working through the solution to the parameters θ_0 to θ_n using the process described above results in an extremely involved derivation procedure. There is indeed a faster solution.</p>
<p>Take a look at the formula for the normal equation:</p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238228/image4_sckrl4.png"></center>

<p>Where:</p>
<p>θ → The parameters that minimize the loss function
X → The input feature values for each instance
y → The vector of output values for each instance</p>
<h2 id="the-normal-equation-vs-gradient-descent">The Normal Equation vs Gradient Descent</h2>
<p>While both methods seek to find the parameters theta (θ) that minimize the loss function, the method of approach differs greatly between the two solutions.</p>
<p>Since we’ve already covered how the normal equation works in ‘<a href="#what-is-the-normal-equation-">What is the Normal Equation?</a>’, in this section, we will briefly touch on gradient descent and then provide ways in which the two techniques differ.</p>
<h3 id="gradient-descent">Gradient Descent</h3>
<p>Gradient descent is one of the most used machine learning algorithms in machine learning. It’s deployed to iteratively find the parameters theta (θ) that minimize the loss function.</p>
<p>The process starts by first evaluating the model&#39;s performance. Next, the partial derivative is calculated from the loss function which is used to reference the slope at its current point. Lastly, we take steps proportional to the negative gradient to make a descent to the minimum of the loss function by updating the current set of parameters - see formula below.</p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238227/image1_b93ju9.png"></center>
<br>
<p>This process is repeated until convergence at the minimum of the loss function.</p>
<h3 id="how-do-they-differ-">How do they Differ?</h3>
<p>The most obvious way in which the normal equation differs from gradient descent is that it’s analytical. Gradient descent takes an iterative approach which means our parameters are updated gradually until convergence. Another subtle difference baked into this is that gradient descent requires us to define a learning rate that controls the size of the steps taken towards the minimum of the loss function. The normal equation doesn’t require us to define a learning rate because we are not taking iterative steps - we get the results directly.</p>
<p>Also, feature scaling is not required when we use the normal equation approach; we typically perform feature scaling to ensure our features have a similar range of values. This is because gradient descent is sensitive to the ranges of our data points. Failing to normalize our features when we use gradient descent may introduce skewness into the contour plot of the loss function, but the normal equation does not suffer from this problem.</p>
<h2 id="deciphering-when-to-use-the-normal-equation">Deciphering When to Use the Normal Equation</h2>
<p>The best way to know if you should use the normal equation over gradient descent is to understand its disadvantages.</p>
<p>Computing the normal equation becomes computationally challenging when the number of features in our dataset is large. The reason for this is that in order to solve for the parameters θ, the term (X’ X)^-1 must be computed. Computing X’ X produces an n x n matrix and for most computer implementations, converting a matrix grows approximately as the cube of the dimensions of the matrix. This means the inverse operation runs in O(n^3) runtime complexity which makes the normal equation run extremely slow when n is very large - learn more about <a target="_blank" href="https://en.wikipedia.org/wiki/Time_complexity">time complexity</a>.</p>
<p>Thus, it’s best to use gradient descent when the number of features in the dataset is large. Andrew Ng, a prominent machine learning and AI expert, recommends you should consider using gradient descent when the number of features, n, is greater than 10,000. For 10,000 features or less, you may be better off using the normal equation since you’re not required to select a value for the learning rate which means you have one less hyperparameter to tune.</p>
<h2 id="the-normal-equation-from-scratch-in-python">The Normal Equation from Scratch in Python</h2>
<p>Let’s generate a regression problem to test this equation:</p>
<pre><code class="lang-python">import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_regression

# Generate a regression problem
X, y = make_regression(
    n_samples=100,
    n_features=2,
    n_informative=2,
    noise = 10,
    random_state=25
    )

# Visualize feature at index 1 vs target
plt.subplots(figsize=(8, 5))
plt.scatter(X[:, 1], y, marker=&#39;o&#39;)
plt.xlabel(&quot;Feature at Index 1&quot;)
plt.ylabel(&quot;Target&quot;)
plt.show()
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238227/image2_c75gme.png"></center>
<br>
<p>Here’s where we will implement the normal equation:</p>
<pre><code class="lang-python"># adds x0 = 1 to each instance
X_b = np.concatenate([np.ones((len(X), 1)), X], axis=1)
# calculate normal equation
theta_best = np.linalg.inv(X_b.T.dot(X_b)).dot(X_b.T).dot(y)
# best values for theta
intercept, *coef = theta_best
print(f&quot;Intercept: {intercept}\n\
Coefficients: {coef}&quot;)

Intercept: 0.35921242677977794
Coefficients: [6.129199175400593, 96.44309685893134]
</code></pre><p>Let’s put our model to the test by making a prediction:</p>
<pre><code class="lang-python"># making a new sample
new_sample = np.array([[-2, 0.25]])
# adding a bias term to the instance
new_sample_b = np.concatenate([np.ones((len(new_sample), 1)), new_sample], axis=1)
# predicting the value of our new sample
new_sample_pred = new_sample_b.dot(theta_best)
print(f&quot;Prediction: {new_sample_pred}&quot;)
Prediction: [12.21158829]
</code></pre><p>Whenever you implement a machine learning algorithm from scratch, it’s always helpful to have a method of validating your solution; <a target="_blank" href="https://scikit-learn.org/">Scikit-learn</a> is one of the most popular machine learning libraries in Python. It features several implementations of different algorithms, including linear regression, which we will be using to validate our normal equation.</p>
<pre><code class="lang-python">from sklearn.linear_model import LinearRegression

lr = LinearRegression()
lr.fit(X, y)

print(f&quot;Intercept: {lr.intercept_}\n\
Coefficients: {lr.coef_}&quot;)
print(f&quot;Prediction: {lr.predict(new_sample)}&quot;)

Intercept: 0.3592124267797807
Coefficients: [ 6.12919918 96.44309686]
Prediction: [12.21158829]
</code></pre><p>The solutions are approximately equal so we can confirm our solution is correct.</p>
<h2 id="frequently-asked-questions-faqs-">Frequently Asked Questions (FAQs)</h2>
<h4 id="what-is-the-normal-equation-in-machine-learning-">What is the normal equation in machine learning?</h4>
<p>The normal equation is an analytical approach to finding the value of θ that minimizes the loss function without having to iterate.</p>
<h4 id="when-should-i-use-the-normal-equation-instead-of-gradient-descent-">When should I use the normal equation instead of gradient descent?</h4>
<p>It’s better to use the normal equation when we have fewer features. Computing the normal equation becomes computationally challenging when the number of features in our dataset is large.</p>
<h4 id="can-the-normal-equation-be-used-for-logistic-regression-">Can the normal equation be used for logistic regression?</h4>
<p>Unfortunately no. There’s only one conditional model in classification theory that has a closed-form solution - linear regression.</p>
<h4 id="what-s-the-difference-between-the-normal-equation-and-gradient-descent-">What’s the difference between the normal equation and gradient descent?</h4>
<p>The obvious difference between the normal equation and gradient descent is that the normal equation uses an analytical approach to find the minimum of the loss function, whereas gradient descent uses an iterative approach. Another difference is that you don’t have to tune a learning rate to compute the normal equation since only one step is taken to find the parameters θ.</p>


https://www.datacamp.com/community/tutorials/tutorial-normal-equation-for-linear-regression

Learn what the normal equation is and how can you use it to build machine learning models. 

Normal Equation for Linear Regression Tutorial

<ul>
<li>Introducing Linear Models</li>
<li>Regularization</li>
<li>Introduction to Lasso Regression<ul>
<li>Lasso Regression for (L1) Regularization</li>
<li>Lasso Regression for model selection</li>
</ul>
</li>
<li>Ridge Regression<ul>
<li>L2 Regularization</li>
</ul>
</li>
<li>Differences between Lasso and Ridge Regression Models<ul>
<li>How Lasso Regression leads to model selection but not a ridge.</li>
</ul>
</li>
<li>Implementation in Python<ul>
<li>Data Importation and Exploratory Data Analysis</li>
<li>Data Transformation</li>
<li>Linear Regression</li>
<li>Ridge Model</li>
<li>Lasso Model</li>
<li>Selecting Optimal alpha values using Cross-Validation in sklearn</li>
</ul>
</li>
<li>Summary and Key takeaways</li>
</ul>
<h2 id="introducing-linear-models">Introducing Linear Models</h2>
<p>Linear regression is a type of linear model that is considered the most basic and commonly used predictive algorithm. This can not be dissociated from its simple, yet effective architecture. A linear model assumes a linear relationship between input variable(s) 𝑥 and an output variable y. The equation for a linear model looks like this:</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image15_duqnl4.png"></center>

<p>In this equation 1.1, we show a linear model with n number of features. w is considered the coefficient (or weights) assigned to each feature - an indicator of their significance to the outcome y. For example, we assume that temperature is a larger driver of ice cream sales than whether it&#39;s a public holiday. The weight assigned to temperature in our linear model will be larger than the public holiday variable.</p>
<p>The goal for a linear model then becomes to optimize the weight (b) via the cost function in equation 1.2. The cost function calculates the error between predictions and actual values, represented as a single real-valued number. The cost function is the average error across n samples in the dataset, represented below as:</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image14_elbrgp.png"></center>
<center>Cost function of a linear regression model</center>


<p>In the equation above yi is the actual value and thati is the predicted value from our linear equation, where M is the number of rows and P is the number of features.</p>
<h2 id="regularization">Regularization</h2>
<p>When it comes to training models, there are two major problems one can encounter: overfitting and underfitting.</p>
<ul>
<li>Overfitting happens when the model performs well on the training set but not so well on unseen (test) data.</li>
<li>Underfitting happens when it neither performs well on the train set nor on the test set.</li>
</ul>
<p>Particularly, regularization is implemented to avoid overfitting of the data, especially when there is a large variance between train and test set performances. With regularization, the number of features used in training is kept constant, yet the magnitude of the coefficients (w) as seen in equation 1.1, is reduced.</p>
<p>Consider the image of coefficients below to predict house prices. While there are quite a number of predictors, RM and RAD have the largest coefficients. The implication of this will be that housing prices will be driven more significantly by these two features leading to overfitting, where generalizable patterns have not been learned.</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205670/image9_qate2m.png"></center>

<p>There are different ways of reducing model complexity and preventing overfitting in linear models. This includes ridge and lasso regression models.</p>
<h2 id="introduction-to-lasso-regression">Introduction to Lasso Regression</h2>
<p>This is a regularization technique used in feature selection using a Shrinkage method also referred to as the <strong>penalized regression method</strong>. Lasso is short for <strong>L</strong>east <strong>A</strong>bsolute <strong>S</strong>hrinkage and <strong>S</strong>election <strong>O</strong>perator, which is used both for regularization and model selection. If a model uses the <strong>L1 regularization</strong> technique, then it is called lasso regression.</p>
<h3 id="lasso-regression-for-regularization">Lasso Regression for Regularization</h3>
<p>In this shrinkage technique, the coefficients determined in the linear model from equation 1.1. above are shrunk towards the central point as the mean by introducing a penalization factor called the alpha α (or sometimes lamda) values.</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image17_ryqgum.png"></center>


<p>Alpha (α) is the penalty term that denotes the amount of shrinkage (or constraint) that will be implemented in the equation. With alpha set to zero, you will find that this is the equivalent of the linear regression model from equation 1.2, and a larger value penalizes the optimization function. Therefore, lasso regression shrinks the coefficients and helps to reduce the model complexity and multi-collinearity.</p>
<p>Alpha (α) can be any real-valued number between zero and infinity; the larger the value, the more aggressive the penalization is.</p>
<h3 id="lasso-regression-for-model-selection">Lasso Regression for Model Selection</h3>
<p>Due to the fact that coefficients will be shrunk towards a mean of zero, less important features in a dataset are eliminated when penalized. The shrinkage of these coefficients based on the alpha value provided leads to some form of automatic feature selection, as input variables are removed in an effective approach.</p>
<h2 id="ridge-regression">Ridge Regression</h2>
<p>Similar to the lasso regression, ridge regression puts a similar constraint on the coefficients by introducing a penalty factor. However, while lasso regression takes the magnitude of the coefficients, ridge regression takes the square.</p>
 <center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image13_ftroyj.png"></center>


<p>Ridge regression is also referred to as <strong>L2 Regularization</strong>.</p>
<h3 id="why-lasso-can-be-used-for-model-selection-but-not-ridge-regression">Why Lasso can be Used for Model Selection, but not Ridge Regression</h3>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205672/image18_a3zz7y.png"></center>
<a target="_blank" href="https://online.stat.psu.edu/stat508/book/export/html/749">Source</a>

<p>Considering the geometry of both the lasso (left) and ridge (right) models, the elliptical contours (red circles) are the cost functions for each. Relaxing the constraints introduced by the penalty factor leads to an increase in the constrained region (diamond, circle). Doing this continually, we will hit the center of the ellipse, where the results of both lasso and ridge models are similar to a linear regression model.</p>

<p>However, both methods determine coefficients by finding the first point where the elliptical contours hit the region of constraints. Since lasso regression takes a diamond shape in the plot for the constrained region, each time the elliptical regions intersect with these corners, at least one of the coefficients becomes zero. This is impossible in the ridge regression model as it forms a circular shape and therefore values can be shrunk close to zero, but never equal to zero.</p>

<h3 id="python implementation"> Python Implementation </h3>
<p>For this implementation, we will use the Boston housing dataset found in sklearn. What we intend to see is:</p>
<ol>
    <li>How to perform ridge and lasso regression in Python</li>
    <li>Compare the results with a linear regression model</li>
</ol>

<h4 id="data importation and eda"> Data Importation and EDA </h4>

<pre><code>#libraries
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.linear_model import Ridge, RidgeCV, Lasso
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import load_boston

#data
boston = load_boston()
boston_df=pd.DataFrame(boston.data,columns=boston.feature_names)
#target variable
boston_df[&#39;Price&#39;]=boston.target
#preview
boston_df.head()

#Exploration
plt.figure(figsize = (10, 10))
sns.heatmap(boston_df.corr(), annot = True)</code></pre>

<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image12_erlhn0.png"></center>

<pre><code>#There are cases of multicolinearity, we will drop a few columns
boston_df.drop(columns = [&quot;INDUS&quot;, &quot;NOX&quot;], inplace = True)

#pairplot
sns.pairplot(boston_df)

#we will log the LSTAT Column
boston_df.LSTAT = np.log(boston_df.LSTAT)
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205672/image10_p83sh7.png"></center>


<p>Note that we logged the LSTAT column as it doesn’t have a linear relationship with the price column. Linear models assume a linear relationship between x and y variables.</p>
<h3 id="data-splitting-and-scaling">Data Splitting and Scaling</h3>
<pre><code>#preview
features = boston_df.columns[0:11]
target = boston_df.columns[-1]

#X and y values
X = boston_df[features].values
y = boston_df[target].values

#splot
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=17)

print(&quot;The dimension of X_train is {}&quot;.format(X_train.shape))
print(&quot;The dimension of X_test is {}&quot;.format(X_test.shape))
#Scale features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
</code></pre><p>  Output:</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image20_mkxrik.png"></center>


<h3 id="linear-and-ridge-regression-models">Linear and Ridge Regression Models</h3>
<p>We will build a linear and a ridge regression model and then compare the coefficients in a plot. The score of the train and test sets will also help us evaluate how well the model performs.</p>
<pre><code>#Model
lr = LinearRegression()

#Fit model
lr.fit(X_train, y_train)

#predict
#prediction = lr.predict(X_test)

#actual
actual = y_test

train_score_lr = lr.score(X_train, y_train)
test_score_lr = lr.score(X_test, y_test)

print(&quot;The train score for lr model is {}&quot;.format(train_score_lr))
print(&quot;The test score for lr model is {}&quot;.format(test_score_lr))


#Ridge Regression Model
ridgeReg = Ridge(alpha=10)

ridgeReg.fit(X_train,y_train)

#train and test scorefor ridge regression
train_score_ridge = ridgeReg.score(X_train, y_train)
test_score_ridge = ridgeReg.score(X_test, y_test)

print(&quot;\nRidge Model............................................\n&quot;)
print(&quot;The train score for ridge model is {}&quot;.format(train_score_ridge))
print(&quot;The test score for ridge model is {}&quot;.format(test_score_ridge))
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205670/image8_bdldbf.png"></center>

<p>Using an alpha value of 10, the evaluation on the model on the train and test data indicates a better performance on the ridge model than on the linear regression model.</p>
<p>We can also plot the coefficients for both the linear and ridge models.</p>
<pre><code>plt.figure(figsize = (10, 10))
plt.plot(features,ridgeReg.coef_,alpha=0.7,linestyle=&#39;none&#39;,marker=&#39;*&#39;,markersize=5,color=&#39;red&#39;,label=r&#39;Ridge; $\alpha = 10$&#39;,zorder=7)
#plt.plot(rr100.coef_,alpha=0.5,linestyle=&#39;none&#39;,marker=&#39;d&#39;,markersize=6,color=&#39;blue&#39;,label=r&#39;Ridge; $\alpha = 100$&#39;)
plt.plot(features,lr.coef_,alpha=0.4,linestyle=&#39;none&#39;,marker=&#39;o&#39;,markersize=7,color=&#39;green&#39;,label=&#39;Linear Regression&#39;)
plt.xticks(rotation = 90)
plt.legend()
plt.show()
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205672/image22_jnzw4w.png"></center>


<h2 id="lasso-regression">Lasso Regression</h2>
<pre><code>#Lasso regression model
print(&quot;\nLasso Model............................................\n&quot;)
lasso = Lasso(alpha = 10)
lasso.fit(X_train,y_train)
train_score_ls =lasso.score(X_train,y_train)
test_score_ls =lasso.score(X_test,y_test)

print(&quot;The train score for ls model is {}&quot;.format(train_score_ls))
print(&quot;The test score for ls model is {}&quot;.format(test_score_ls))
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205670/image11_jgroco.png"></center>

<p>We can visualize the coefficents too.</p>
<pre><code>pd.Series(lasso.coef_, features).sort_values(ascending = True).plot(kind = &quot;bar&quot;)
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image16_gdsari.png"></center>


<p>Earlier on, we established that the lasso model can inert to zero due to the diamond shape of the constraint region. In this case, using an alpha value of 10 over penalizes the model and shrinks all the values to zero. We can see this effectively by visualizing the coefficients of the model as shown the figure above.</p>
<h3 id="selecting-optimal-alpha-values-using-cross-validation-in-sklearn">Selecting Optimal Alpha Values Using Cross Validation in Sklearn</h3>
<p>We may need to try out different alpha values to find the optimal constraint value. For this case, we can use the cross validation model in the sklearn package. This will try out different combinations of alpha values and then choose the best model.</p>
<pre><code>#Using the linear CV model
from sklearn.linear_model import LassoCV

#Lasso Cross validation
lasso_cv = LassoCV(alphas = [0.0001, 0.001,0.01, 0.1, 1, 10], random_state=0).fit(X_train, y_train)


#score
print(lasso_cv.score(X_train, y_train))
print(lasso_cv.score(X_test, y_test))
</code></pre><p>Output:</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205672/image19_qmmbjw.png"></center>

<p>The model will be trained on different alpha values that I have specified in the LassoCV function. We can observe a better performance of the model, removing the tedious effort of manually changing alpha values.</p>
<p>We can compare the coefficients from the lasso model with the rest of the models (linear and ridge).</p>
<pre><code>#plot size
plt.figure(figsize = (10, 10))
#add plot for ridge regression
plt.plot(features,ridgeReg.coef_,alpha=0.7,linestyle=&#39;none&#39;,marker=&#39;*&#39;,markersize=5,color=&#39;red&#39;,label=r&#39;Ridge; $\alpha = 10$&#39;,zorder=7)

#addd plot for lasso regression
plt.plot(lasso_cv.coef_,alpha=0.5,linestyle=&#39;none&#39;,marker=&#39;d&#39;,markersize=6,color=&#39;blue&#39;,label=r&#39;lasso; $\alpha = grid$&#39;)

#add plot for linear model
plt.plot(features,lr.coef_,alpha=0.4,linestyle=&#39;none&#39;,marker=&#39;o&#39;,markersize=7,color=&#39;green&#39;,label=&#39;Linear Regression&#39;)

#rotate axis
plt.xticks(rotation = 90)
plt.legend()
plt.title(&quot;Comparison plot of Ridge, Lasso and Linear regression model&quot;)
plt.show()
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205672/image21_jxceie.png"></center>

<p><strong>Note</strong>: A similar approach could be employed for the ridge regression model, which could lead to better results. In the sklearn package, the function RidgeCV performs similarly.</p>
<pre><code>#Using the linear CV model
from sklearn.linear_model import RidgeCV

#Lasso Cross validation
ridge_cv = RidgeCV(alphas = [0.0001, 0.001,0.01, 0.1, 1, 10]).fit(X_train, y_train)

#score
print(&quot;The train score for ridge model is {}&quot;.format(ridge_cv.score(X_train, y_train)))
print(&quot;The train score for ridge model is {}&quot;.format(ridge_cv.score(X_test, y_test)))
</code></pre><p>Summary</p>
<p>We have seen an implementation of ridge and lasso regression models and the theoretical and mathematical concepts behind these techniques. Some of the key takeaways from this tutorial include:</p>
<ol>
<li>The cost function for both ridge and lasso regression are similar. However, ridge regression takes the square of the coefficients and lasso takes the magnitude.</li>
<li>Lasso regression can be used for automatic feature selection, as the geometry of its constrained region allows coefficient values to inert to zero.</li>
<li>An alpha value of zero in either ridge or lasso model will have results similar to the regression model.</li>
<li>The larger the alpha value, the more aggressive the penalization.</li>
</ol>
<p>You can find a more robust and complete notebook for the python implementation <a target="_blank" href="https://app.datacamp.com/workspace/w/e09a2743-6536-4d44-88e9-088be0edc682">here</a>.</p>


https://www.datacamp.com/community/tutorials/tutorial-lasso-ridge-regression

Learn about the lasso and ridge techniques of regression.  Compare and analyse the methods in detail. 

community

datacamp

machine learning

News (155)

Data Science, Gambling and Bookmaking (with Marco Blume)

K-Means Clustering in R Tutorial

Data Science, Gambling and Bookmaking (with Marco Blume)

Tutorials (66)

Normal Equation for Linear Regression Tutorial

Lasso and Ridge Regression Tutorial

Cheat Sheets (2)

Keras Cheat Sheet: Neural Networks in Python

Open Courses (3)

Introduction to Python & Machine Learning (with Analytics Vidhya Hackathons)

Kaggle Python Tutorial on Machine Learning

Podcast (38)

Scaling Machine Learning Adoption: A Pragmatic Approach (with Noah Gift)

Blog Posts (93)

Data Science in Insurance Today

Tech Thoughts (0)