http://datacamp-community-prod.s3.amazonaws.com/dfdb6d58-e044-4b38-bab3-5de0b825909b

This Python cheat sheet is a handy reference for doing linear algebra with SciPy and interacting with NumPy!

SciPy Cheat Sheet: Linear Algebra in Python

https://www.datacamp.com/community/news/solving-internal-co-variate-shift-in-deep-learning-with-linked-neurons-gng0ulnq4sr

Solving internal co-variate shift in deep learning with linked Neurons with Interactive Code [ Manual Back Prop in TF ]

Solving internal co-variate shift in deep learning with linked Neurons

https://www.datacamp.com/community/news/my-take-on-25-questions-to-test-a-data-scientist-on-image-processing-w-tori1tikwag

My take on 25 Questions to test a Data Scientist on Image Processing with Interactive Code- Part 2

My take on 25 Questions to test a Data Scientist on Image Processing w

https://www.datacamp.com/community/news/mandalas-colored-ezgio7jpm1

Doing art is a nice and effective way to learn some important features of R as recursivity or ggplot. This experiment will show you that learning can be very entertaining.

Mandalas Colored

https://www.datacamp.com/community/news/depth-breath-first-search-matrix-traversal-in-python-with-interactiv-4539p6ju1wu

Depth / Breath First Search Matrix Traversal in Python with Interactive Code [ Back to Basics ]

Depth / Breath First Search Matrix Traversal in Python with Interactiv

<p>By now, you will have already learned that NumPy, one of the fundamental packages for scientific computing, forms at least for a part&nbsp;the fundament of other important&nbsp;packages that you might use used for data manipulation and machine learning with Python. One of those packages is SciPy, another one&nbsp;of the core packages for scientific computing in Python that provides mathematical algorithms and convenience functions built on the&nbsp;NumPy extension of Python.&nbsp;</p>

<p>You might now wonder why this library might come in handy for data science.&nbsp;</p>

<p>Well, SciPy has many modules that will help you to understand some of the basic components that you need to master when you&#39;re learning data science, namely, math, stats and machine learning. You can find out what other things you need to tackle to learn data science <a href="https://www.datacamp.com/community/tutorials/learn-data-science-infographic/">here</a>. You&#39;ll see that for statistics, for example, a module like&nbsp;<code>scipy.stats</code>, etc. will definitely be of interest to you.</p>

<p>The other topic that was mentioned was machine learning: here, the&nbsp;<code>scipy.linalg</code>&nbsp;and <code>scipy.sparse</code> modules will offer everything that you&#39;re looking for to understand machine learning concepts such as eigenvalues, regression, and matrix multiplication...</p>

<p>But, what is maybe the most obvious is that most machine learning techniques deal with high-dimensional data and that data is often represented as matrices. What&#39;s more, you&#39;ll need to understand how to manipulate these matrices. &nbsp;</p>

<p>That is why DataCamp has made a SciPy cheat sheet that will help you to master linear algebra with Python.&nbsp;</p>

<p>Take a look by clicking on the button below:</p>

<p><a href="http://datacamp-community-prod.s3.amazonaws.com/dfdb6d58-e044-4b38-bab3-5de0b825909b"><img alt="python scipy cheat sheet" src="https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1625838166/SciPy_Cheat_Sheet-_Linear_Algebra_in_Python_ff0oa8.png" style="width: 800px; height: 551px;" /></a></p>

<p>&nbsp;</p>

<p>You&#39;ll see that this SciPy cheat sheet covers the basics of linear algebra&nbsp;that you need to get started: it provides a brief explanation of what the&nbsp;library has to offer and how you can use it to interact with NumPy, and goes on to summarize topics in linear algebra, such as matrix creation, matrix functions,&nbsp;basic routines that you can perform with matrices, and matrix decompositions from <code>scipy.linalg</code>. Sparse matrices are also included, with their own routines, functions, and decompositions from the <code>scipy.sparse</code> module.&nbsp;</p>
<p><strong>(Above is the printable version of this cheat sheet)</strong></p>

<div id="scoped-content">
<style type="text/css">:target:before {
content:"";
display:block;
height:150px;
margin:-150px 0 0;
}

h3 {font-weight:normal;
}
h4 {
	font-weight: lighter;
}
table {
    width: 100%;
    table-layout: fixed;
}

th {
    height: 50px;
}

th, td {
    padding: 5px;
    text-align: left;}
tr {background-color:white}
tr:hover {background-color: #f5f5f5}

</style>
<h2>Python for Data-Science <em>Cheat Sheet</em>&#58; SciPy - Linear Algebra</h2>
<h2>SciPy</h2>
<p>The SciPy library is one of the core packages for scientific computing that provides mathematical algorithms and convenience functions built on the NumPy extension of Python.</p>
<h2>Asking For Help</h2>
<tr><td><code>&#62;&#62;&#62; help&#40;scipy.linalg.diagsvd&#41;</code></td></tr>
<h2>Interacting With NumPy</h2>
<table>
<tr><td><code>&#62;&#62;&#62; import numpy as np </code></td></tr>
<tr><td><code>&#62;&#62;&#62; a &#58; np.array&#40;&#91;1&#44;2&#44;3&#93;&#41;</code></td></tr>
<tr><td><code>&#62;&#62;&#62; b &#58; np.array&#40;&#91;&#40;1+5j&#44;2j&#44;3j&#41;&#44; &#40;4j&#44;5j&#44;6j&#41;&#93;&#41;</code></td></tr>
<tr><td><code>&#62;&#62;&#62; c &#58; np.array&#40;&#91;&#91;&#40;1.5&#44;2&#44;3&#41;&#44; &#40;4&#44;5&#44;6&#41;&#93;&#44; &#91;&#40;3&#44;2&#44;1&#41;&#44; &#40;4&#44;5&#44;6&#41;&#93;&#93;&#41;</code></td></tr>
</table>
<h3>Index Tricks</h3>
<table>
<tr><td><code>&#62;&#62;&#62; np.mgrid&#91;0&#58;5&#44;0&#58;5&#93;</code></td><td> Create a dense meshgrid</td></tr>
<tr><td><code>&#62;&#62;&#62; np.ogrid&#91;0&#58;2&#44;0&#58;2&#93; </code></td></tr>
<tr><td><code>&#62;&#62;&#62; np.r_&#91;3&#44;&#91;0&#93;*5&#44;-1&#58;1&#58;10j&#93;</code></td><td>Stack arrays vertically &#40;row-wise&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; np.c_&#91;b&#44;c&#93;</code></td></tr>
</table>
<h3>Shape Manipulation</h3>
<table>
<tr><td><code>&#62;&#62;&#62; np.transpose&#40;b&#41;</code></td><td> Permute array dimensions</td></tr>
<tr><td><code>&#62;&#62;&#62; b.flatten&#40;&#41;</code></td><td>Flatten the array</td></tr>
<tr><td><code>&#62;&#62;&#62; np.hstack&#40;&#40;b&#44;c&#41;&#41;</code></td><td>Stack arrays horizontally &#40;column-wise&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; np.vstack&#40;&#40;a&#44;b&#41;&#41;</code></td><td>Stack arrays vertically &#40;row-wise&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; np.hsplit&#40;c&#44;2&#41;</code></td><td>Split the array horizontally at the 2nd index</td></tr>
<tr><td><code>&#62;&#62;&#62; np.vpslit&#40;d&#44;2&#41;</code></td></tr>
</table>
<h3>Polynomials </h3>
<table>
<tr><td><code>&#62;&#62;&#62; from numpy import poly1d</code></td></tr>
<tr><td><code>&#62;&#62;&#62; p &#58; poly1d&#40;&#91;3&#44;4&#44;5&#93;&#41;</code></td><td>Create a polynomial object</td></tr>
</table>
<h3>Vectorizing Functions</h3>
<table>
	<table>
	<tr><td><code>&#62;&#62;&#62; def myfunc&#40;a&#41;&#58;</code></td></tr>
	<tr><td><code>&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;if a &#8249; 0&#58; </code></td></tr>
	<tr><td><code>&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;return a*2</code></td></tr>
	<tr><td><code>&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;else&#58;</code></td></tr>
	<tr><td><code>&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;return a/2</code></td></tr>
	<tr><td><code>&#62;&#62;&#62; np.vectorize&#40;myfunc&#41;</code></td><td> Vectorize functions</td></tr>
</table>
<h3>Type Handling </h3>
<table>
<tr><td><code>&#62;&#62;&#62; np.real&#40;b&#41;</code></td><td>Return the real part of the array elements </td></tr>
<tr><td><code>&#62;&#62;&#62; np.imag&#40;b&#41;</code></td><td>Return the imaginary part of the array elements</td></tr>
<tr><td><code>&#62;&#62;&#62; np.real_if_close&#40;c&#44;tol&#58;1000&#41; </code></td><td>Return a real array if complex parts close to 0</td></tr>
<tr><td><code>&#62;&#62;&#62; np.cast&#91;'f'&#93;&#40;np.pi&#41;</code></td><td>Cast object to a data type</td></tr>
</table>
<h3>Other Useful Functions</h3>
<table>
<tr><td><code>&#62;&#62;&#62; np.angle&#40;b&#44;deg&#58;True&#41;</code></td><td>Return the angle of the complex argument</td></tr>
<tr><td><code>&#62;&#62;&#62; g &#58; np.linspace&#40;0&#44;np.pi&#44;num&#58;5&#41;</code></td><td>Create an array of evenly spaced values &#40;number of samples&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; g &#91;3&#58;&#93; +&#58; np.pi</code></td></tr>
<tr><td><code>&#62;&#62;&#62; np.unwrap&#40;g&#41; </code></td></tr>
<tr><td><code>&#62;&#62;&#62; np.logspace&#40;0&#44;10&#44;3&#41;</code></td><td>Create an array of evenly spaced values &#40;log scale&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; np.select&#40;&#91;c&#60;4&#93;&#44;&#91;c*2&#93;&#41;</code></td><td>Return values from a list of arrays depending on conditions</td></tr>
<tr><td><code>&#62;&#62;&#62; misc.factorial&#40;a&#41;</code></td><td>Factorial </td></tr>
<tr><td><code>&#62;&#62;&#62; misc.comb&#40;10&#44;3&#44;exact&#58;True&#41;</code></td><td>Combine N things taken at k time</td></tr>
<tr><td><code>&#62;&#62;&#62; misc.central_diff_weights&#40;3&#41;</code></td><td>Weights for Np-point central derivative</td></tr>
<tr><td><code>&#62;&#62;&#62; misc.derivative&#40;myfunc&#44;1.0&#41;</code></td><td>Find the n-th derivative of a function at a point</td></tr>
</table>

<h3>﻿Linear Algebra</h3>
<p>You&#39;ll use the linalg and sparse modules. Note thatscipy.linalg contains and expands onnumpy.linalg. </p>
<table>
<tr><td><code>&#62;&#62;&#62; from scipy import linalg&#44; sparse</code></td></tr>
</table>
<h3>Creating Matrices</h3>
<table>
<tr><td><code>&#62;&#62;&#62; A &#58; np.matrix&#40;np.random.random&#40;&#40;2&#44;2&#41;&#41;&#41;</code></td></tr>
<tr><td><code>&#62;&#62;&#62; B &#58; np.asmatrix&#40;b&#41;</code></td></tr>
<tr><td><code>&#62;&#62;&#62; C &#58; np.mat&#40;np.random.random&#40;&#40;10&#44;5&#41;&#41;&#41;</code></td></tr>
<tr><td><code>&#62;&#62;&#62; D &#58; np.mat&#40;&#91;&#91;3&#44;4&#93;&#44; &#91;5&#44;6&#93;&#93;&#41;</code></td></tr>
</table>



<table>
</table>
<h3>Basic Matrix Routines</h3>
<table>
</table>
<h4>Inverse</h4>
<table>
<tr><td><code>&#62;&#62;&#62; A.I</code></td><td>Inverse</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.inv&#40;A&#41;</code></td><td>Inverse</td></tr>
</table>
<h4>Transposition</h4>
<table>
<tr><td><code>&#62;&#62;&#62; A.T</code></td><td>Tranpose matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; A.H</code></td><td>Conjugate transposition</td></tr>
</table>
<h4>Trace</h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.trace&#40;A&#41;</code></td><td>Trace</td></tr>
</table>
<h4>Norm</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.norm&#40;A&#41;</code></td><td> Frobenius norm</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.norm&#40;A&#44;1&#41;</code></td><td> L1 norm &#40;max column sum&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.norm&#40;A&#44;np.inf&#41;</code></td><td> L inf norm &#40;max row sum&#41;</td></tr>
</table>
<h4>Rank</h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.linalg.matrix_rank&#40;C&#41;</code></td><td> Matrix rank</td></tr>
</table>
<h4>Determinant</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.det&#40;A&#41;</code></td><td>Determinant</td></tr>
</table>
<h4>Solving linear problems</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.solve&#40;A&#44;b&#41;</code></td><td>Solver for dense matrices</td></tr>
<tr><td><code>&#62;&#62;&#62; E &#58; np.mat&#40;a&#41;.T</code></td><td>Solver for dense matrices</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.lstsq&#40;F&#44;E&#41;</code></td><td>Least-squares solution to linear matrix equation</td></tr>
</table>
<h4> Generalized inverse</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.pinv&#40;C&#41;</code></td><td> Compute the pseudo-inverse of a matrix &#40;least-squares solver</td></tr>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.pinv2&#40;C&#41;</code></td><td>Compute the pseudo-inverse of a matrix &#40;SVD&#41;</td></tr>
</table>

<h3>Creating Sparse Matrices</h3>
<table>
<tr><td><code>&#62;&#62;&#62; F &#58; np.eye&#40;3&#44; k&#58;1&#41;</code></td><td> Create a 2X2 identity matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; G &#58; np.mat&#40;np.identity&#40;2&#41;&#41; </code></td><td>Create a 2x2 identity matrix </td></tr>
<tr><td><code>&#62;&#62;&#62; C&#91;C &#62; 0.5&#93; &#58; 0 </code></td></tr>
<tr><td><code>&#62;&#62;&#62; H &#58; sparse.csr_matrix&#40;C&#41; </code></td><td>Compressed Sparse Row matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; I &#58; sparse.csc_matrix&#40;D&#41; </code></td><td>Compressed Sparse Column matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; J &#58; sparse.dok_matrix&#40;A&#41; </code></td><td>Dictionary Of Keys matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; E.todense&#40;&#41;</code></td><td>Sparse matrix to full matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; sparse.isspmatrix_csc&#40;A&#41; </code></td><td>Identify sparse matrix</td></tr>
</table>
<h3>Sparse Matrix Routines</h3>
<table>
</table>
<h4>Inverse</h4>
<table>
<tr><td><code>&#62;&#62;&#62; sparse.linalg.inv&#40;I&#41;</code></td><td>Inverse</td></tr>
</table>
<h4>Norm </h4>
<table>
<tr><td><code>&#62;&#62;&#62; sparse.linalg.norm&#40;I&#41; </code></td><td>Norm</td></tr>
</table>
<h4> Solving linear problems </h4>
<table>
<tr><td><code>&#62;&#62;&#62; sparse.linalg.spsolve&#40;H&#44;I&#41;</code></td><td>Solver for sparse matrices</td></tr>
</table>
<h3>Sparse Matrix Functions</h3>
<table>
<tr><td><code>&#62;&#62;&#62; la&#44; v &#58; sparse.linalg.eigs&#40;F&#44;1&#41;</code></td><td>Eigenvalues and eigenvectors</td></tr>
<tr><td><code>&#62;&#62;&#62; sparse.linalg.svds&#40;H&#44; 2&#41;</code></td><td> SVD</td></tr>
<tr><td><code>&#62;&#62;&#62; sparse.linalg.expm&#40;I&#41; </code></td><td>Sparse matrix exponential</td></tr>
</table>

<h3>Matrix Functions</h3>
<h4>Addition</h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.add&#40;A&#44;D&#41;</code></td><td>Addition</td></tr>
</table>
<h4> Subtraction</h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.subtract&#40;A&#44;D&#41;</code></td><td> Subtraction</td></tr>
</table>
<h4> Division</h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.divide&#40;A&#44;D&#41;</code></td><td> Division</td></tr>
</table>
<h4> Multiplication</h4>
<table>
<tr><td><code>&#62;&#62;&#62; A @ D</code></td><td>Multiplication operator &#40;Python 3&#41;</td></tr>

<tr><td><code>&#62;&#62;&#62; np.multiply&#40;D&#44;A&#41;</code></td><td> Multiplication</td></tr>
<tr><td><code>&#62;&#62;&#62; np.dot&#40;A&#44;D&#41;</code></td><td>Dot product</td></tr>
<tr><td><code>&#62;&#62;&#62; np.vdot&#40;A&#44;D&#41;</code></td><td> Vector dot product</td></tr>
<tr><td><code>&#62;&#62;&#62; np.inner&#40;A&#44;D&#41;</code></td><td>Inner product</td></tr>
<tr><td><code>&#62;&#62;&#62; np.outer&#40;A&#44;D&#41;</code></td><td>Outer product</td></tr>
<tr><td><code>&#62;&#62;&#62; np.tensordot&#40;A&#44;D&#41;</code></td><td>Tensor dot product</td></tr>
<tr><td><code>&#62;&#62;&#62; np.kron&#40;A&#44;D&#41;</code></td><td> Kronecker product</td></tr>
</table>
<h4>Exponential Functions</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.expm&#40;A&#41;</code></td><td> Matrix exponential</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.expm2&#40;A&#41;</code></td><td>Matrix exponential &#40;Taylor Series&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.expm3&#40;D&#41;</code></td><td>Matrix exponential &#40;eigenvalue decomposition&#41;</td></tr>
</table>
<h4>Logarithm Function</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.logm&#40;A&#41;</code></td><td> Matrix logarithm </td></tr>
</table>
<h4>Trigonometric Functions</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.sinm&#40;D&#41;</code></td><td> Matrix sine</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.cosm&#40;D&#41;</code></td><td> Matrix cosine</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.tanm&#40;A&#41;</code></td><td> Matrix tangent</td></tr>
</table>
<h4>Hyperbolic Trigonometric Functions</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.sinhm&#40;D&#41;</code></td><td>Hypberbolic matrix sine</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.coshm&#40;D&#41;</code></td><td>Hyperbolic matrix cosine</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.tanhm&#40;A&#41;</code></td><td>Hyperbolic matrix tangent</td></tr>
</table>
<h4>Matrix Sign Function </h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.signm&#40;A&#41;</code></td><td>Matrix sign function</td></tr>
</table>
<h4>Matrix Square Root</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.sqrtm&#40;A&#41;</code></td><td>Matrix square root</td></tr>
</table>
<h4>Arbitrary Functions</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.funm&#40;A&#44; lambda x&#58; x*x&#41;</code></td><td>Evaluate matrix function</td></tr>
</table>

<h3>Decompositions</h3>
<table>
</table>
<h4>Eigenvalues and Eigenvectors</h4>
<table>
<tr><td><code>&#62;&#62;&#62; la&#44; v &#58; linalg.eig&#40;A&#41; </code></td><td>Solve ordinary or generalized eigenvalue problem for square matrix </td></tr>
<tr><td><code>&#62;&#62;&#62; l1&#44; l2 &#58; la</code></td><td> Unpack eigenvalues</td></tr>
<tr><td><code>&#62;&#62;&#62; v&#91;&#58;&#44;0&#93;</code></td><td>First eigenvector</td></tr>
<tr><td><code>&#62;&#62;&#62; v&#91;&#58;&#44;1&#93;</code></td><td>Second eigenvector</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.eigvals&#40;A&#41;</code></td><td> Unpack eigenvalues</td></tr>
</table>
<h4>Singular Value Decomposition </h4>
<table>
<tr><td><code>&#62;&#62;&#62; U&#44;s&#44;Vh &#58; linalg.svd&#40;B&#41;</code></td><td>Singular Value Decomposition &#40;SVD&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; M&#44;N &#58; B.shape </code></td></tr>
<tr><td><code>&#62;&#62;&#62; Sig &#58; linalg.diagsvd&#40;s&#44;M&#44;N&#41;</code></td><td>Construct sigma matrix in SVD</td></tr>
<h3>LU Decomposition</h3>
<table><tr><td><code>&#62;&#62;&#62; P&#44;L&#44;U &#58; linalg.lu&#40;C&#41;</code></td><td>LU Decomposition</td></tr></table>


<h3>Sparse Matrix Decompositions</h3>
<tr><td><code>&#62;&#62;&#62; np.info&#40;np.matrix&#41;</code></td></tr>
<br>
<br>
<br>
<br>
<p>PS.&nbsp;Don&#39;t miss our other Python cheat sheets for data science that cover <a href="https://www.datacamp.com/community/blog/python-numpy-cheat-sheet/">Numpy</a>&#44;&nbsp;<a href="https://www.datacamp.com/community/blog/scikit-learn-cheat-sheet/">Scikit-Learn</a>&#44;&nbsp;<a href="https://www.datacamp.com/community/blog/bokeh-cheat-sheet-python/">Bokeh</a>,&nbsp;<a href="https://www.datacamp.com/community/blog/python-pandas-cheat-sheet/">Pandas</a>&nbsp;and the&nbsp;<a href="https://www.datacamp.com/community/tutorials/python-data-science-cheat-sheet-basics/">Python basics</a>.</p>

</div>


https://www.datacamp.com/community/blog/python-scipy-cheat-sheet

This Python cheat sheet is a handy reference with code samples for doing linear algebra with SciPy and interacting with NumPy.

SciPy Cheat Sheet: Linear Algebra in Python 

<h2 id="table-of-contents-">Table of Contents:</h2>
<ul>
<li><a href="#what-is-the-normal-equation-">What is the Normal Equation?</li>
<li><a href="#the-normal-equation-vs-gradient-descent">The Normal Equation vs Gradient Descent<ul>
<li><a href="#gradient-descent">Gradient Descent</li>
<li><a href="#how-do-they-differ-">How do they differ?</li>
</ul>
</li>
<li><a href="#deciphering-when-to-use-the-normal-equation">Deciphering When to Use the Normal Equation</li>
<li><a href="#the-normal-equation-from-scratch-in-python">The Normal Equation from Scratch in Python</li>
<li><a href="#frequently-asked-questions-faqs-">FAQs</li>
</ul>
<p>Most problems we encounter have several ways they can be solved. For example, if we wanted to get from one side of a room to another, we may decide to walk around the room until we arrive at the opposing side, or we could just cut across.</p>
<p>The normal equation is just an emphasis of this concept. It’s just another way to solve a problem. What problem did you ask? We’ll cover that in the remainder of this article. For now, all you need to know is that it&#39;s an effective approach that can help you save lots of time when implementing linear regression under certain conditions. </p>
<p>Let’s dive deeper…</p>
<h2 id="what-is-the-normal-equation-">What is the Normal Equation?</h2>
<p>The normal equation is a closed-form solution used to find the value of θ that minimizes the cost function. Another way to describe the normal equation is as a one-step algorithm used to analytically find the coefficients that minimize the loss function. Both descriptions work, but what exactly do they mean? We will start with linear regression.</p>
<p>Linear regression makes a prediction, y_hat, by computing the weighted sum of input features plus a bias term. Mathematically it can be represented as follows:</p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238228/image5_iengil.png"></center>
<br>
<p>Where θ represents the parameters and n is the number of features.</p>
<p>Essentially, all that occurs in the above equation is the dot product of θ, and x is being summed. Thus, a more concise way to represent this is to use its vectorized form:</p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238228/image6_t3omdh.png"></center>
<br>
<p>h(θ) is the hypothesis function.</p>
<p>Given this approximate target function, we can use our model to make predictions. To determine if our model has learned well, it’s important we measure the performance of our model on the training data. For this purpose, we compute a loss function. The goal of the training process is to find the values of theta (θ) that minimize the loss function.  </p>
<p>Here’s how we can represent our loss function mathematically:</p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238227/image3_p2c9gn.png"></center>
<br>
<p>In the above equation, theta (θ) is a n + 1 dimensional vector, and our loss function is a function of the vector value. Consequently, the partial derivative of the loss function, J, has to be taken with respect to every parameter of θ_j in turn. All of them must equal zero. Following this process and solving for all of the values of θ from θ_0 to θ_n will result in the values of θ that minimize the loss function.</p>
<p>Working through the solution to the parameters θ_0 to θ_n using the process described above results in an extremely involved derivation procedure. There is indeed a faster solution.</p>
<p>Take a look at the formula for the normal equation:</p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238228/image4_sckrl4.png"></center>

<p>Where:</p>
<p>θ → The parameters that minimize the loss function
X → The input feature values for each instance
y → The vector of output values for each instance</p>
<h2 id="the-normal-equation-vs-gradient-descent">The Normal Equation vs Gradient Descent</h2>
<p>While both methods seek to find the parameters theta (θ) that minimize the loss function, the method of approach differs greatly between the two solutions.</p>
<p>Since we’ve already covered how the normal equation works in ‘<a href="#what-is-the-normal-equation-">What is the Normal Equation?</a>’, in this section, we will briefly touch on gradient descent and then provide ways in which the two techniques differ.</p>
<h3 id="gradient-descent">Gradient Descent</h3>
<p>Gradient descent is one of the most used machine learning algorithms in machine learning. It’s deployed to iteratively find the parameters theta (θ) that minimize the loss function.</p>
<p>The process starts by first evaluating the model&#39;s performance. Next, the partial derivative is calculated from the loss function which is used to reference the slope at its current point. Lastly, we take steps proportional to the negative gradient to make a descent to the minimum of the loss function by updating the current set of parameters - see formula below.</p>
<br><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238227/image1_b93ju9.png"></center>
<br>
<p>This process is repeated until convergence at the minimum of the loss function.</p>
<h3 id="how-do-they-differ-">How do they Differ?</h3>
<p>The most obvious way in which the normal equation differs from gradient descent is that it’s analytical. Gradient descent takes an iterative approach which means our parameters are updated gradually until convergence. Another subtle difference baked into this is that gradient descent requires us to define a learning rate that controls the size of the steps taken towards the minimum of the loss function. The normal equation doesn’t require us to define a learning rate because we are not taking iterative steps - we get the results directly.</p>
<p>Also, feature scaling is not required when we use the normal equation approach; we typically perform feature scaling to ensure our features have a similar range of values. This is because gradient descent is sensitive to the ranges of our data points. Failing to normalize our features when we use gradient descent may introduce skewness into the contour plot of the loss function, but the normal equation does not suffer from this problem.</p>
<h2 id="deciphering-when-to-use-the-normal-equation">Deciphering When to Use the Normal Equation</h2>
<p>The best way to know if you should use the normal equation over gradient descent is to understand its disadvantages.</p>
<p>Computing the normal equation becomes computationally challenging when the number of features in our dataset is large. The reason for this is that in order to solve for the parameters θ, the term (X’ X)^-1 must be computed. Computing X’ X produces an n x n matrix and for most computer implementations, converting a matrix grows approximately as the cube of the dimensions of the matrix. This means the inverse operation runs in O(n^3) runtime complexity which makes the normal equation run extremely slow when n is very large - learn more about <a target="_blank" href="https://en.wikipedia.org/wiki/Time_complexity">time complexity</a>.</p>
<p>Thus, it’s best to use gradient descent when the number of features in the dataset is large. Andrew Ng, a prominent machine learning and AI expert, recommends you should consider using gradient descent when the number of features, n, is greater than 10,000. For 10,000 features or less, you may be better off using the normal equation since you’re not required to select a value for the learning rate which means you have one less hyperparameter to tune.</p>
<h2 id="the-normal-equation-from-scratch-in-python">The Normal Equation from Scratch in Python</h2>
<p>Let’s generate a regression problem to test this equation:</p>
<pre><code class="lang-python">import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_regression

# Generate a regression problem
X, y = make_regression(
    n_samples=100,
    n_features=2,
    n_informative=2,
    noise = 10,
    random_state=25
    )

# Visualize feature at index 1 vs target
plt.subplots(figsize=(8, 5))
plt.scatter(X[:, 1], y, marker=&#39;o&#39;)
plt.xlabel(&quot;Feature at Index 1&quot;)
plt.ylabel(&quot;Target&quot;)
plt.show()
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1649238227/image2_c75gme.png"></center>
<br>
<p>Here’s where we will implement the normal equation:</p>
<pre><code class="lang-python"># adds x0 = 1 to each instance
X_b = np.concatenate([np.ones((len(X), 1)), X], axis=1)
# calculate normal equation
theta_best = np.linalg.inv(X_b.T.dot(X_b)).dot(X_b.T).dot(y)
# best values for theta
intercept, *coef = theta_best
print(f&quot;Intercept: {intercept}\n\
Coefficients: {coef}&quot;)

Intercept: 0.35921242677977794
Coefficients: [6.129199175400593, 96.44309685893134]
</code></pre><p>Let’s put our model to the test by making a prediction:</p>
<pre><code class="lang-python"># making a new sample
new_sample = np.array([[-2, 0.25]])
# adding a bias term to the instance
new_sample_b = np.concatenate([np.ones((len(new_sample), 1)), new_sample], axis=1)
# predicting the value of our new sample
new_sample_pred = new_sample_b.dot(theta_best)
print(f&quot;Prediction: {new_sample_pred}&quot;)
Prediction: [12.21158829]
</code></pre><p>Whenever you implement a machine learning algorithm from scratch, it’s always helpful to have a method of validating your solution; <a target="_blank" href="https://scikit-learn.org/">Scikit-learn</a> is one of the most popular machine learning libraries in Python. It features several implementations of different algorithms, including linear regression, which we will be using to validate our normal equation.</p>
<pre><code class="lang-python">from sklearn.linear_model import LinearRegression

lr = LinearRegression()
lr.fit(X, y)

print(f&quot;Intercept: {lr.intercept_}\n\
Coefficients: {lr.coef_}&quot;)
print(f&quot;Prediction: {lr.predict(new_sample)}&quot;)

Intercept: 0.3592124267797807
Coefficients: [ 6.12919918 96.44309686]
Prediction: [12.21158829]
</code></pre><p>The solutions are approximately equal so we can confirm our solution is correct.</p>
<h2 id="frequently-asked-questions-faqs-">Frequently Asked Questions (FAQs)</h2>
<h4 id="what-is-the-normal-equation-in-machine-learning-">What is the normal equation in machine learning?</h4>
<p>The normal equation is an analytical approach to finding the value of θ that minimizes the loss function without having to iterate.</p>
<h4 id="when-should-i-use-the-normal-equation-instead-of-gradient-descent-">When should I use the normal equation instead of gradient descent?</h4>
<p>It’s better to use the normal equation when we have fewer features. Computing the normal equation becomes computationally challenging when the number of features in our dataset is large.</p>
<h4 id="can-the-normal-equation-be-used-for-logistic-regression-">Can the normal equation be used for logistic regression?</h4>
<p>Unfortunately no. There’s only one conditional model in classification theory that has a closed-form solution - linear regression.</p>
<h4 id="what-s-the-difference-between-the-normal-equation-and-gradient-descent-">What’s the difference between the normal equation and gradient descent?</h4>
<p>The obvious difference between the normal equation and gradient descent is that the normal equation uses an analytical approach to find the minimum of the loss function, whereas gradient descent uses an iterative approach. Another difference is that you don’t have to tune a learning rate to compute the normal equation since only one step is taken to find the parameters θ.</p>


https://www.datacamp.com/community/tutorials/tutorial-normal-equation-for-linear-regression

Learn what the normal equation is and how can you use it to build machine learning models. 

Normal Equation for Linear Regression Tutorial

<ul>
<li>Introducing Linear Models</li>
<li>Regularization</li>
<li>Introduction to Lasso Regression<ul>
<li>Lasso Regression for (L1) Regularization</li>
<li>Lasso Regression for model selection</li>
</ul>
</li>
<li>Ridge Regression<ul>
<li>L2 Regularization</li>
</ul>
</li>
<li>Differences between Lasso and Ridge Regression Models<ul>
<li>How Lasso Regression leads to model selection but not a ridge.</li>
</ul>
</li>
<li>Implementation in Python<ul>
<li>Data Importation and Exploratory Data Analysis</li>
<li>Data Transformation</li>
<li>Linear Regression</li>
<li>Ridge Model</li>
<li>Lasso Model</li>
<li>Selecting Optimal alpha values using Cross-Validation in sklearn</li>
</ul>
</li>
<li>Summary and Key takeaways</li>
</ul>
<h2 id="introducing-linear-models">Introducing Linear Models</h2>
<p>Linear regression is a type of linear model that is considered the most basic and commonly used predictive algorithm. This can not be dissociated from its simple, yet effective architecture. A linear model assumes a linear relationship between input variable(s) 𝑥 and an output variable y. The equation for a linear model looks like this:</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image15_duqnl4.png"></center>

<p>In this equation 1.1, we show a linear model with n number of features. w is considered the coefficient (or weights) assigned to each feature - an indicator of their significance to the outcome y. For example, we assume that temperature is a larger driver of ice cream sales than whether it&#39;s a public holiday. The weight assigned to temperature in our linear model will be larger than the public holiday variable.</p>
<p>The goal for a linear model then becomes to optimize the weight (b) via the cost function in equation 1.2. The cost function calculates the error between predictions and actual values, represented as a single real-valued number. The cost function is the average error across n samples in the dataset, represented below as:</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image14_elbrgp.png"></center>
<center>Cost function of a linear regression model</center>


<p>In the equation above yi is the actual value and thati is the predicted value from our linear equation, where M is the number of rows and P is the number of features.</p>
<h2 id="regularization">Regularization</h2>
<p>When it comes to training models, there are two major problems one can encounter: overfitting and underfitting.</p>
<ul>
<li>Overfitting happens when the model performs well on the training set but not so well on unseen (test) data.</li>
<li>Underfitting happens when it neither performs well on the train set nor on the test set.</li>
</ul>
<p>Particularly, regularization is implemented to avoid overfitting of the data, especially when there is a large variance between train and test set performances. With regularization, the number of features used in training is kept constant, yet the magnitude of the coefficients (w) as seen in equation 1.1, is reduced.</p>
<p>Consider the image of coefficients below to predict house prices. While there are quite a number of predictors, RM and RAD have the largest coefficients. The implication of this will be that housing prices will be driven more significantly by these two features leading to overfitting, where generalizable patterns have not been learned.</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205670/image9_qate2m.png"></center>

<p>There are different ways of reducing model complexity and preventing overfitting in linear models. This includes ridge and lasso regression models.</p>
<h2 id="introduction-to-lasso-regression">Introduction to Lasso Regression</h2>
<p>This is a regularization technique used in feature selection using a Shrinkage method also referred to as the <strong>penalized regression method</strong>. Lasso is short for <strong>L</strong>east <strong>A</strong>bsolute <strong>S</strong>hrinkage and <strong>S</strong>election <strong>O</strong>perator, which is used both for regularization and model selection. If a model uses the <strong>L1 regularization</strong> technique, then it is called lasso regression.</p>
<h3 id="lasso-regression-for-regularization">Lasso Regression for Regularization</h3>
<p>In this shrinkage technique, the coefficients determined in the linear model from equation 1.1. above are shrunk towards the central point as the mean by introducing a penalization factor called the alpha α (or sometimes lamda) values.</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image17_ryqgum.png"></center>


<p>Alpha (α) is the penalty term that denotes the amount of shrinkage (or constraint) that will be implemented in the equation. With alpha set to zero, you will find that this is the equivalent of the linear regression model from equation 1.2, and a larger value penalizes the optimization function. Therefore, lasso regression shrinks the coefficients and helps to reduce the model complexity and multi-collinearity.</p>
<p>Alpha (α) can be any real-valued number between zero and infinity; the larger the value, the more aggressive the penalization is.</p>
<h3 id="lasso-regression-for-model-selection">Lasso Regression for Model Selection</h3>
<p>Due to the fact that coefficients will be shrunk towards a mean of zero, less important features in a dataset are eliminated when penalized. The shrinkage of these coefficients based on the alpha value provided leads to some form of automatic feature selection, as input variables are removed in an effective approach.</p>
<h2 id="ridge-regression">Ridge Regression</h2>
<p>Similar to the lasso regression, ridge regression puts a similar constraint on the coefficients by introducing a penalty factor. However, while lasso regression takes the magnitude of the coefficients, ridge regression takes the square.</p>
 <center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image13_ftroyj.png"></center>


<p>Ridge regression is also referred to as <strong>L2 Regularization</strong>.</p>
<h3 id="why-lasso-can-be-used-for-model-selection-but-not-ridge-regression">Why Lasso can be Used for Model Selection, but not Ridge Regression</h3>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205672/image18_a3zz7y.png"></center>
<a target="_blank" href="https://online.stat.psu.edu/stat508/book/export/html/749">Source</a>

<p>Considering the geometry of both the lasso (left) and ridge (right) models, the elliptical contours (red circles) are the cost functions for each. Relaxing the constraints introduced by the penalty factor leads to an increase in the constrained region (diamond, circle). Doing this continually, we will hit the center of the ellipse, where the results of both lasso and ridge models are similar to a linear regression model.</p>

<p>However, both methods determine coefficients by finding the first point where the elliptical contours hit the region of constraints. Since lasso regression takes a diamond shape in the plot for the constrained region, each time the elliptical regions intersect with these corners, at least one of the coefficients becomes zero. This is impossible in the ridge regression model as it forms a circular shape and therefore values can be shrunk close to zero, but never equal to zero.</p>

<h3 id="python implementation"> Python Implementation </h3>
<p>For this implementation, we will use the Boston housing dataset found in sklearn. What we intend to see is:</p>
<ol>
    <li>How to perform ridge and lasso regression in Python</li>
    <li>Compare the results with a linear regression model</li>
</ol>

<h4 id="data importation and eda"> Data Importation and EDA </h4>

<pre><code>#libraries
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from sklearn.linear_model import Ridge, RidgeCV, Lasso
from sklearn.preprocessing import StandardScaler
from sklearn.datasets import load_boston

#data
boston = load_boston()
boston_df=pd.DataFrame(boston.data,columns=boston.feature_names)
#target variable
boston_df[&#39;Price&#39;]=boston.target
#preview
boston_df.head()

#Exploration
plt.figure(figsize = (10, 10))
sns.heatmap(boston_df.corr(), annot = True)</code></pre>

<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image12_erlhn0.png"></center>

<pre><code>#There are cases of multicolinearity, we will drop a few columns
boston_df.drop(columns = [&quot;INDUS&quot;, &quot;NOX&quot;], inplace = True)

#pairplot
sns.pairplot(boston_df)

#we will log the LSTAT Column
boston_df.LSTAT = np.log(boston_df.LSTAT)
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205672/image10_p83sh7.png"></center>


<p>Note that we logged the LSTAT column as it doesn’t have a linear relationship with the price column. Linear models assume a linear relationship between x and y variables.</p>
<h3 id="data-splitting-and-scaling">Data Splitting and Scaling</h3>
<pre><code>#preview
features = boston_df.columns[0:11]
target = boston_df.columns[-1]

#X and y values
X = boston_df[features].values
y = boston_df[target].values

#splot
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=17)

print(&quot;The dimension of X_train is {}&quot;.format(X_train.shape))
print(&quot;The dimension of X_test is {}&quot;.format(X_test.shape))
#Scale features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)
</code></pre><p>  Output:</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image20_mkxrik.png"></center>


<h3 id="linear-and-ridge-regression-models">Linear and Ridge Regression Models</h3>
<p>We will build a linear and a ridge regression model and then compare the coefficients in a plot. The score of the train and test sets will also help us evaluate how well the model performs.</p>
<pre><code>#Model
lr = LinearRegression()

#Fit model
lr.fit(X_train, y_train)

#predict
#prediction = lr.predict(X_test)

#actual
actual = y_test

train_score_lr = lr.score(X_train, y_train)
test_score_lr = lr.score(X_test, y_test)

print(&quot;The train score for lr model is {}&quot;.format(train_score_lr))
print(&quot;The test score for lr model is {}&quot;.format(test_score_lr))


#Ridge Regression Model
ridgeReg = Ridge(alpha=10)

ridgeReg.fit(X_train,y_train)

#train and test scorefor ridge regression
train_score_ridge = ridgeReg.score(X_train, y_train)
test_score_ridge = ridgeReg.score(X_test, y_test)

print(&quot;\nRidge Model............................................\n&quot;)
print(&quot;The train score for ridge model is {}&quot;.format(train_score_ridge))
print(&quot;The test score for ridge model is {}&quot;.format(test_score_ridge))
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205670/image8_bdldbf.png"></center>

<p>Using an alpha value of 10, the evaluation on the model on the train and test data indicates a better performance on the ridge model than on the linear regression model.</p>
<p>We can also plot the coefficients for both the linear and ridge models.</p>
<pre><code>plt.figure(figsize = (10, 10))
plt.plot(features,ridgeReg.coef_,alpha=0.7,linestyle=&#39;none&#39;,marker=&#39;*&#39;,markersize=5,color=&#39;red&#39;,label=r&#39;Ridge; $\alpha = 10$&#39;,zorder=7)
#plt.plot(rr100.coef_,alpha=0.5,linestyle=&#39;none&#39;,marker=&#39;d&#39;,markersize=6,color=&#39;blue&#39;,label=r&#39;Ridge; $\alpha = 100$&#39;)
plt.plot(features,lr.coef_,alpha=0.4,linestyle=&#39;none&#39;,marker=&#39;o&#39;,markersize=7,color=&#39;green&#39;,label=&#39;Linear Regression&#39;)
plt.xticks(rotation = 90)
plt.legend()
plt.show()
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205672/image22_jnzw4w.png"></center>


<h2 id="lasso-regression">Lasso Regression</h2>
<pre><code>#Lasso regression model
print(&quot;\nLasso Model............................................\n&quot;)
lasso = Lasso(alpha = 10)
lasso.fit(X_train,y_train)
train_score_ls =lasso.score(X_train,y_train)
test_score_ls =lasso.score(X_test,y_test)

print(&quot;The train score for ls model is {}&quot;.format(train_score_ls))
print(&quot;The test score for ls model is {}&quot;.format(test_score_ls))
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205670/image11_jgroco.png"></center>

<p>We can visualize the coefficents too.</p>
<pre><code>pd.Series(lasso.coef_, features).sort_values(ascending = True).plot(kind = &quot;bar&quot;)
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205671/image16_gdsari.png"></center>


<p>Earlier on, we established that the lasso model can inert to zero due to the diamond shape of the constraint region. In this case, using an alpha value of 10 over penalizes the model and shrinks all the values to zero. We can see this effectively by visualizing the coefficients of the model as shown the figure above.</p>
<h3 id="selecting-optimal-alpha-values-using-cross-validation-in-sklearn">Selecting Optimal Alpha Values Using Cross Validation in Sklearn</h3>
<p>We may need to try out different alpha values to find the optimal constraint value. For this case, we can use the cross validation model in the sklearn package. This will try out different combinations of alpha values and then choose the best model.</p>
<pre><code>#Using the linear CV model
from sklearn.linear_model import LassoCV

#Lasso Cross validation
lasso_cv = LassoCV(alphas = [0.0001, 0.001,0.01, 0.1, 1, 10], random_state=0).fit(X_train, y_train)


#score
print(lasso_cv.score(X_train, y_train))
print(lasso_cv.score(X_test, y_test))
</code></pre><p>Output:</p>
<center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205672/image19_qmmbjw.png"></center>

<p>The model will be trained on different alpha values that I have specified in the LassoCV function. We can observe a better performance of the model, removing the tedious effort of manually changing alpha values.</p>
<p>We can compare the coefficients from the lasso model with the rest of the models (linear and ridge).</p>
<pre><code>#plot size
plt.figure(figsize = (10, 10))
#add plot for ridge regression
plt.plot(features,ridgeReg.coef_,alpha=0.7,linestyle=&#39;none&#39;,marker=&#39;*&#39;,markersize=5,color=&#39;red&#39;,label=r&#39;Ridge; $\alpha = 10$&#39;,zorder=7)

#addd plot for lasso regression
plt.plot(lasso_cv.coef_,alpha=0.5,linestyle=&#39;none&#39;,marker=&#39;d&#39;,markersize=6,color=&#39;blue&#39;,label=r&#39;lasso; $\alpha = grid$&#39;)

#add plot for linear model
plt.plot(features,lr.coef_,alpha=0.4,linestyle=&#39;none&#39;,marker=&#39;o&#39;,markersize=7,color=&#39;green&#39;,label=&#39;Linear Regression&#39;)

#rotate axis
plt.xticks(rotation = 90)
plt.legend()
plt.title(&quot;Comparison plot of Ridge, Lasso and Linear regression model&quot;)
plt.show()
</code></pre><center><img src="https://res.cloudinary.com/dyd911kmh/image/upload/v1648205672/image21_jxceie.png"></center>

<p><strong>Note</strong>: A similar approach could be employed for the ridge regression model, which could lead to better results. In the sklearn package, the function RidgeCV performs similarly.</p>
<pre><code>#Using the linear CV model
from sklearn.linear_model import RidgeCV

#Lasso Cross validation
ridge_cv = RidgeCV(alphas = [0.0001, 0.001,0.01, 0.1, 1, 10]).fit(X_train, y_train)

#score
print(&quot;The train score for ridge model is {}&quot;.format(ridge_cv.score(X_train, y_train)))
print(&quot;The train score for ridge model is {}&quot;.format(ridge_cv.score(X_test, y_test)))
</code></pre><p>Summary</p>
<p>We have seen an implementation of ridge and lasso regression models and the theoretical and mathematical concepts behind these techniques. Some of the key takeaways from this tutorial include:</p>
<ol>
<li>The cost function for both ridge and lasso regression are similar. However, ridge regression takes the square of the coefficients and lasso takes the magnitude.</li>
<li>Lasso regression can be used for automatic feature selection, as the geometry of its constrained region allows coefficient values to inert to zero.</li>
<li>An alpha value of zero in either ridge or lasso model will have results similar to the regression model.</li>
<li>The larger the alpha value, the more aggressive the penalization.</li>
</ol>
<p>You can find a more robust and complete notebook for the python implementation <a target="_blank" href="https://app.datacamp.com/workspace/w/e09a2743-6536-4d44-88e9-088be0edc682">here</a>.</p>


https://www.datacamp.com/community/tutorials/tutorial-lasso-ridge-regression

Learn about the lasso and ridge techniques of regression.  Compare and analyse the methods in detail. 

community

datacamp

mathematics

News (10)

Solving internal co-variate shift in deep learning with linked Neurons

Mandalas Colored

Tutorials (5)

Normal Equation for Linear Regression Tutorial

Lasso and Ridge Regression Tutorial

Cheat Sheets (1)

SciPy Cheat Sheet: Linear Algebra in Python

Open Courses (0)

Podcast (0)

Blog Posts (1)

SciPy Cheat Sheet: Linear Algebra in Python

Tech Thoughts (0)