http://datacamp-community-prod.s3.amazonaws.com/dfdb6d58-e044-4b38-bab3-5de0b825909b

This Python cheat sheet is a handy reference for doing linear algebra with SciPy and interacting with NumPy!

SciPy Cheat Sheet: Linear Algebra in Python

https://www.datacamp.com/community/news/solving-internal-co-variate-shift-in-deep-learning-with-linked-neurons-gng0ulnq4sr

Solving internal co-variate shift in deep learning with linked Neurons with Interactive Code [ Manual Back Prop in TF ]

Solving internal co-variate shift in deep learning with linked Neurons

https://www.datacamp.com/community/news/my-take-on-25-questions-to-test-a-data-scientist-on-image-processing-w-tori1tikwag

My take on 25 Questions to test a Data Scientist on Image Processing with Interactive Code- Part 2

My take on 25 Questions to test a Data Scientist on Image Processing w

https://www.datacamp.com/community/news/mandalas-colored-ezgio7jpm1

Doing art is a nice and effective way to learn some important features of R as recursivity or ggplot. This experiment will show you that learning can be very entertaining.

Mandalas Colored

https://www.datacamp.com/community/news/depth-breath-first-search-matrix-traversal-in-python-with-interactiv-4539p6ju1wu

Depth / Breath First Search Matrix Traversal in Python with Interactive Code [ Back to Basics ]

Depth / Breath First Search Matrix Traversal in Python with Interactiv

<p>By now, you will have already learned that NumPy, one of the fundamental packages for scientific computing, forms at least for a part&nbsp;the fundament of other important&nbsp;packages that you might use used for data manipulation and machine learning with Python. One of those packages is SciPy, another one&nbsp;of the core packages for scientific computing in Python that provides mathematical algorithms and convenience functions built on the&nbsp;NumPy extension of Python.&nbsp;</p>

<p>You might now wonder why this library might come in handy for data science.&nbsp;</p>

<p>Well, SciPy has many modules that will help you to understand some of the basic components that you need to master when you&#39;re learning data science, namely, math, stats and machine learning. You can find out what other things you need to tackle to learn data science <a href="https://www.datacamp.com/community/tutorials/learn-data-science-infographic/">here</a>. You&#39;ll see that for statistics, for example, a module like&nbsp;<code>scipy.stats</code>, etc. will definitely be of interest to you.</p>

<p>The other topic that was mentioned was machine learning: here, the&nbsp;<code>scipy.linalg</code>&nbsp;and <code>scipy.sparse</code> modules will offer everything that you&#39;re looking for to understand machine learning concepts such as eigenvalues, regression, and matrix multiplication...</p>

<p>But, what is maybe the most obvious is that most machine learning techniques deal with high-dimensional data and that data is often represented as matrices. What&#39;s more, you&#39;ll need to understand how to manipulate these matrices. &nbsp;</p>

<p>That is why DataCamp has made a SciPy cheat sheet that will help you to master linear algebra with Python.&nbsp;</p>

<p>Take a look by clicking on the button below:</p>

<p><a href="http://datacamp-community-prod.s3.amazonaws.com/dfdb6d58-e044-4b38-bab3-5de0b825909b"><img alt="python scipy cheat sheet" src="https://res.cloudinary.com/dyd911kmh/image/upload/f_auto,q_auto:best/v1625838166/SciPy_Cheat_Sheet-_Linear_Algebra_in_Python_ff0oa8.png" style="width: 800px; height: 551px;" /></a></p>

<p>&nbsp;</p>

<p>You&#39;ll see that this SciPy cheat sheet covers the basics of linear algebra&nbsp;that you need to get started: it provides a brief explanation of what the&nbsp;library has to offer and how you can use it to interact with NumPy, and goes on to summarize topics in linear algebra, such as matrix creation, matrix functions,&nbsp;basic routines that you can perform with matrices, and matrix decompositions from <code>scipy.linalg</code>. Sparse matrices are also included, with their own routines, functions, and decompositions from the <code>scipy.sparse</code> module.&nbsp;</p>
<p><strong>(Above is the printable version of this cheat sheet)</strong></p>

<div id="scoped-content">
<style type="text/css">:target:before {
content:"";
display:block;
height:150px;
margin:-150px 0 0;
}

h3 {font-weight:normal;
}
h4 {
	font-weight: lighter;
}
table {
    width: 100%;
    table-layout: fixed;
}

th {
    height: 50px;
}

th, td {
    padding: 5px;
    text-align: left;}
tr {background-color:white}
tr:hover {background-color: #f5f5f5}

</style>
<h2>Python for Data-Science <em>Cheat Sheet</em>&#58; SciPy - Linear Algebra</h2>
<h2>SciPy</h2>
<p>The SciPy library is one of the core packages for scientific computing that provides mathematical algorithms and convenience functions built on the NumPy extension of Python.</p>
<h2>Asking For Help</h2>
<tr><td><code>&#62;&#62;&#62; help&#40;scipy.linalg.diagsvd&#41;</code></td></tr>
<h2>Interacting With NumPy</h2>
<table>
<tr><td><code>&#62;&#62;&#62; import numpy as np </code></td></tr>
<tr><td><code>&#62;&#62;&#62; a &#58; np.array&#40;&#91;1&#44;2&#44;3&#93;&#41;</code></td></tr>
<tr><td><code>&#62;&#62;&#62; b &#58; np.array&#40;&#91;&#40;1+5j&#44;2j&#44;3j&#41;&#44; &#40;4j&#44;5j&#44;6j&#41;&#93;&#41;</code></td></tr>
<tr><td><code>&#62;&#62;&#62; c &#58; np.array&#40;&#91;&#91;&#40;1.5&#44;2&#44;3&#41;&#44; &#40;4&#44;5&#44;6&#41;&#93;&#44; &#91;&#40;3&#44;2&#44;1&#41;&#44; &#40;4&#44;5&#44;6&#41;&#93;&#93;&#41;</code></td></tr>
</table>
<h3>Index Tricks</h3>
<table>
<tr><td><code>&#62;&#62;&#62; np.mgrid&#91;0&#58;5&#44;0&#58;5&#93;</code></td><td> Create a dense meshgrid</td></tr>
<tr><td><code>&#62;&#62;&#62; np.ogrid&#91;0&#58;2&#44;0&#58;2&#93; </code></td></tr>
<tr><td><code>&#62;&#62;&#62; np.r_&#91;3&#44;&#91;0&#93;*5&#44;-1&#58;1&#58;10j&#93;</code></td><td>Stack arrays vertically &#40;row-wise&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; np.c_&#91;b&#44;c&#93;</code></td></tr>
</table>
<h3>Shape Manipulation</h3>
<table>
<tr><td><code>&#62;&#62;&#62; np.transpose&#40;b&#41;</code></td><td> Permute array dimensions</td></tr>
<tr><td><code>&#62;&#62;&#62; b.flatten&#40;&#41;</code></td><td>Flatten the array</td></tr>
<tr><td><code>&#62;&#62;&#62; np.hstack&#40;&#40;b&#44;c&#41;&#41;</code></td><td>Stack arrays horizontally &#40;column-wise&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; np.vstack&#40;&#40;a&#44;b&#41;&#41;</code></td><td>Stack arrays vertically &#40;row-wise&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; np.hsplit&#40;c&#44;2&#41;</code></td><td>Split the array horizontally at the 2nd index</td></tr>
<tr><td><code>&#62;&#62;&#62; np.vpslit&#40;d&#44;2&#41;</code></td></tr>
</table>
<h3>Polynomials </h3>
<table>
<tr><td><code>&#62;&#62;&#62; from numpy import poly1d</code></td></tr>
<tr><td><code>&#62;&#62;&#62; p &#58; poly1d&#40;&#91;3&#44;4&#44;5&#93;&#41;</code></td><td>Create a polynomial object</td></tr>
</table>
<h3>Vectorizing Functions</h3>
<table>
	<table>
	<tr><td><code>&#62;&#62;&#62; def myfunc&#40;a&#41;&#58;</code></td></tr>
	<tr><td><code>&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;if a &#8249; 0&#58; </code></td></tr>
	<tr><td><code>&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;return a*2</code></td></tr>
	<tr><td><code>&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;else&#58;</code></td></tr>
	<tr><td><code>&emsp;&emsp;&emsp;&emsp;&emsp;&emsp;return a/2</code></td></tr>
	<tr><td><code>&#62;&#62;&#62; np.vectorize&#40;myfunc&#41;</code></td><td> Vectorize functions</td></tr>
</table>
<h3>Type Handling </h3>
<table>
<tr><td><code>&#62;&#62;&#62; np.real&#40;b&#41;</code></td><td>Return the real part of the array elements </td></tr>
<tr><td><code>&#62;&#62;&#62; np.imag&#40;b&#41;</code></td><td>Return the imaginary part of the array elements</td></tr>
<tr><td><code>&#62;&#62;&#62; np.real_if_close&#40;c&#44;tol&#58;1000&#41; </code></td><td>Return a real array if complex parts close to 0</td></tr>
<tr><td><code>&#62;&#62;&#62; np.cast&#91;'f'&#93;&#40;np.pi&#41;</code></td><td>Cast object to a data type</td></tr>
</table>
<h3>Other Useful Functions</h3>
<table>
<tr><td><code>&#62;&#62;&#62; np.angle&#40;b&#44;deg&#58;True&#41;</code></td><td>Return the angle of the complex argument</td></tr>
<tr><td><code>&#62;&#62;&#62; g &#58; np.linspace&#40;0&#44;np.pi&#44;num&#58;5&#41;</code></td><td>Create an array of evenly spaced values &#40;number of samples&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; g &#91;3&#58;&#93; +&#58; np.pi</code></td></tr>
<tr><td><code>&#62;&#62;&#62; np.unwrap&#40;g&#41; </code></td></tr>
<tr><td><code>&#62;&#62;&#62; np.logspace&#40;0&#44;10&#44;3&#41;</code></td><td>Create an array of evenly spaced values &#40;log scale&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; np.select&#40;&#91;c&#60;4&#93;&#44;&#91;c*2&#93;&#41;</code></td><td>Return values from a list of arrays depending on conditions</td></tr>
<tr><td><code>&#62;&#62;&#62; misc.factorial&#40;a&#41;</code></td><td>Factorial </td></tr>
<tr><td><code>&#62;&#62;&#62; misc.comb&#40;10&#44;3&#44;exact&#58;True&#41;</code></td><td>Combine N things taken at k time</td></tr>
<tr><td><code>&#62;&#62;&#62; misc.central_diff_weights&#40;3&#41;</code></td><td>Weights for Np-point central derivative</td></tr>
<tr><td><code>&#62;&#62;&#62; misc.derivative&#40;myfunc&#44;1.0&#41;</code></td><td>Find the n-th derivative of a function at a point</td></tr>
</table>

<h3>﻿Linear Algebra</h3>
<p>You&#39;ll use the linalg and sparse modules. Note thatscipy.linalg contains and expands onnumpy.linalg. </p>
<table>
<tr><td><code>&#62;&#62;&#62; from scipy import linalg&#44; sparse</code></td></tr>
</table>
<h3>Creating Matrices</h3>
<table>
<tr><td><code>&#62;&#62;&#62; A &#58; np.matrix&#40;np.random.random&#40;&#40;2&#44;2&#41;&#41;&#41;</code></td></tr>
<tr><td><code>&#62;&#62;&#62; B &#58; np.asmatrix&#40;b&#41;</code></td></tr>
<tr><td><code>&#62;&#62;&#62; C &#58; np.mat&#40;np.random.random&#40;&#40;10&#44;5&#41;&#41;&#41;</code></td></tr>
<tr><td><code>&#62;&#62;&#62; D &#58; np.mat&#40;&#91;&#91;3&#44;4&#93;&#44; &#91;5&#44;6&#93;&#93;&#41;</code></td></tr>
</table>



<table>
</table>
<h3>Basic Matrix Routines</h3>
<table>
</table>
<h4>Inverse</h4>
<table>
<tr><td><code>&#62;&#62;&#62; A.I</code></td><td>Inverse</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.inv&#40;A&#41;</code></td><td>Inverse</td></tr>
</table>
<h4>Transposition</h4>
<table>
<tr><td><code>&#62;&#62;&#62; A.T</code></td><td>Tranpose matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; A.H</code></td><td>Conjugate transposition</td></tr>
</table>
<h4>Trace</h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.trace&#40;A&#41;</code></td><td>Trace</td></tr>
</table>
<h4>Norm</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.norm&#40;A&#41;</code></td><td> Frobenius norm</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.norm&#40;A&#44;1&#41;</code></td><td> L1 norm &#40;max column sum&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.norm&#40;A&#44;np.inf&#41;</code></td><td> L inf norm &#40;max row sum&#41;</td></tr>
</table>
<h4>Rank</h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.linalg.matrix_rank&#40;C&#41;</code></td><td> Matrix rank</td></tr>
</table>
<h4>Determinant</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.det&#40;A&#41;</code></td><td>Determinant</td></tr>
</table>
<h4>Solving linear problems</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.solve&#40;A&#44;b&#41;</code></td><td>Solver for dense matrices</td></tr>
<tr><td><code>&#62;&#62;&#62; E &#58; np.mat&#40;a&#41;.T</code></td><td>Solver for dense matrices</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.lstsq&#40;F&#44;E&#41;</code></td><td>Least-squares solution to linear matrix equation</td></tr>
</table>
<h4> Generalized inverse</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.pinv&#40;C&#41;</code></td><td> Compute the pseudo-inverse of a matrix &#40;least-squares solver</td></tr>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.pinv2&#40;C&#41;</code></td><td>Compute the pseudo-inverse of a matrix &#40;SVD&#41;</td></tr>
</table>

<h3>Creating Sparse Matrices</h3>
<table>
<tr><td><code>&#62;&#62;&#62; F &#58; np.eye&#40;3&#44; k&#58;1&#41;</code></td><td> Create a 2X2 identity matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; G &#58; np.mat&#40;np.identity&#40;2&#41;&#41; </code></td><td>Create a 2x2 identity matrix </td></tr>
<tr><td><code>&#62;&#62;&#62; C&#91;C &#62; 0.5&#93; &#58; 0 </code></td></tr>
<tr><td><code>&#62;&#62;&#62; H &#58; sparse.csr_matrix&#40;C&#41; </code></td><td>Compressed Sparse Row matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; I &#58; sparse.csc_matrix&#40;D&#41; </code></td><td>Compressed Sparse Column matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; J &#58; sparse.dok_matrix&#40;A&#41; </code></td><td>Dictionary Of Keys matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; E.todense&#40;&#41;</code></td><td>Sparse matrix to full matrix</td></tr>
<tr><td><code>&#62;&#62;&#62; sparse.isspmatrix_csc&#40;A&#41; </code></td><td>Identify sparse matrix</td></tr>
</table>
<h3>Sparse Matrix Routines</h3>
<table>
</table>
<h4>Inverse</h4>
<table>
<tr><td><code>&#62;&#62;&#62; sparse.linalg.inv&#40;I&#41;</code></td><td>Inverse</td></tr>
</table>
<h4>Norm </h4>
<table>
<tr><td><code>&#62;&#62;&#62; sparse.linalg.norm&#40;I&#41; </code></td><td>Norm</td></tr>
</table>
<h4> Solving linear problems </h4>
<table>
<tr><td><code>&#62;&#62;&#62; sparse.linalg.spsolve&#40;H&#44;I&#41;</code></td><td>Solver for sparse matrices</td></tr>
</table>
<h3>Sparse Matrix Functions</h3>
<table>
<tr><td><code>&#62;&#62;&#62; la&#44; v &#58; sparse.linalg.eigs&#40;F&#44;1&#41;</code></td><td>Eigenvalues and eigenvectors</td></tr>
<tr><td><code>&#62;&#62;&#62; sparse.linalg.svds&#40;H&#44; 2&#41;</code></td><td> SVD</td></tr>
<tr><td><code>&#62;&#62;&#62; sparse.linalg.expm&#40;I&#41; </code></td><td>Sparse matrix exponential</td></tr>
</table>

<h3>Matrix Functions</h3>
<h4>Addition</h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.add&#40;A&#44;D&#41;</code></td><td>Addition</td></tr>
</table>
<h4> Subtraction</h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.subtract&#40;A&#44;D&#41;</code></td><td> Subtraction</td></tr>
</table>
<h4> Division</h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.divide&#40;A&#44;D&#41;</code></td><td> Division</td></tr>
</table>
<h4> Multiplication</h4>
<table>
<tr><td><code>&#62;&#62;&#62; A @ D</code></td><td>Multiplication operator &#40;Python 3&#41;</td></tr>

<tr><td><code>&#62;&#62;&#62; np.multiply&#40;D&#44;A&#41;</code></td><td> Multiplication</td></tr>
<tr><td><code>&#62;&#62;&#62; np.dot&#40;A&#44;D&#41;</code></td><td>Dot product</td></tr>
<tr><td><code>&#62;&#62;&#62; np.vdot&#40;A&#44;D&#41;</code></td><td> Vector dot product</td></tr>
<tr><td><code>&#62;&#62;&#62; np.inner&#40;A&#44;D&#41;</code></td><td>Inner product</td></tr>
<tr><td><code>&#62;&#62;&#62; np.outer&#40;A&#44;D&#41;</code></td><td>Outer product</td></tr>
<tr><td><code>&#62;&#62;&#62; np.tensordot&#40;A&#44;D&#41;</code></td><td>Tensor dot product</td></tr>
<tr><td><code>&#62;&#62;&#62; np.kron&#40;A&#44;D&#41;</code></td><td> Kronecker product</td></tr>
</table>
<h4>Exponential Functions</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.expm&#40;A&#41;</code></td><td> Matrix exponential</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.expm2&#40;A&#41;</code></td><td>Matrix exponential &#40;Taylor Series&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.expm3&#40;D&#41;</code></td><td>Matrix exponential &#40;eigenvalue decomposition&#41;</td></tr>
</table>
<h4>Logarithm Function</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.logm&#40;A&#41;</code></td><td> Matrix logarithm </td></tr>
</table>
<h4>Trigonometric Functions</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.sinm&#40;D&#41;</code></td><td> Matrix sine</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.cosm&#40;D&#41;</code></td><td> Matrix cosine</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.tanm&#40;A&#41;</code></td><td> Matrix tangent</td></tr>
</table>
<h4>Hyperbolic Trigonometric Functions</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.sinhm&#40;D&#41;</code></td><td>Hypberbolic matrix sine</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.coshm&#40;D&#41;</code></td><td>Hyperbolic matrix cosine</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.tanhm&#40;A&#41;</code></td><td>Hyperbolic matrix tangent</td></tr>
</table>
<h4>Matrix Sign Function </h4>
<table>
<tr><td><code>&#62;&#62;&#62; np.signm&#40;A&#41;</code></td><td>Matrix sign function</td></tr>
</table>
<h4>Matrix Square Root</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.sqrtm&#40;A&#41;</code></td><td>Matrix square root</td></tr>
</table>
<h4>Arbitrary Functions</h4>
<table>
<tr><td><code>&#62;&#62;&#62; linalg.funm&#40;A&#44; lambda x&#58; x*x&#41;</code></td><td>Evaluate matrix function</td></tr>
</table>

<h3>Decompositions</h3>
<table>
</table>
<h4>Eigenvalues and Eigenvectors</h4>
<table>
<tr><td><code>&#62;&#62;&#62; la&#44; v &#58; linalg.eig&#40;A&#41; </code></td><td>Solve ordinary or generalized eigenvalue problem for square matrix </td></tr>
<tr><td><code>&#62;&#62;&#62; l1&#44; l2 &#58; la</code></td><td> Unpack eigenvalues</td></tr>
<tr><td><code>&#62;&#62;&#62; v&#91;&#58;&#44;0&#93;</code></td><td>First eigenvector</td></tr>
<tr><td><code>&#62;&#62;&#62; v&#91;&#58;&#44;1&#93;</code></td><td>Second eigenvector</td></tr>
<tr><td><code>&#62;&#62;&#62; linalg.eigvals&#40;A&#41;</code></td><td> Unpack eigenvalues</td></tr>
</table>
<h4>Singular Value Decomposition </h4>
<table>
<tr><td><code>&#62;&#62;&#62; U&#44;s&#44;Vh &#58; linalg.svd&#40;B&#41;</code></td><td>Singular Value Decomposition &#40;SVD&#41;</td></tr>
<tr><td><code>&#62;&#62;&#62; M&#44;N &#58; B.shape </code></td></tr>
<tr><td><code>&#62;&#62;&#62; Sig &#58; linalg.diagsvd&#40;s&#44;M&#44;N&#41;</code></td><td>Construct sigma matrix in SVD</td></tr>
<h3>LU Decomposition</h3>
<table><tr><td><code>&#62;&#62;&#62; P&#44;L&#44;U &#58; linalg.lu&#40;C&#41;</code></td><td>LU Decomposition</td></tr></table>


<h3>Sparse Matrix Decompositions</h3>
<tr><td><code>&#62;&#62;&#62; np.info&#40;np.matrix&#41;</code></td></tr>
<br>
<br>
<br>
<br>
<p>PS.&nbsp;Don&#39;t miss our other Python cheat sheets for data science that cover <a href="https://www.datacamp.com/community/blog/python-numpy-cheat-sheet/">Numpy</a>&#44;&nbsp;<a href="https://www.datacamp.com/community/blog/scikit-learn-cheat-sheet/">Scikit-Learn</a>&#44;&nbsp;<a href="https://www.datacamp.com/community/blog/bokeh-cheat-sheet-python/">Bokeh</a>,&nbsp;<a href="https://www.datacamp.com/community/blog/python-pandas-cheat-sheet/">Pandas</a>&nbsp;and the&nbsp;<a href="https://www.datacamp.com/community/tutorials/python-data-science-cheat-sheet-basics/">Python basics</a>.</p>

</div>


https://www.datacamp.com/community/blog/python-scipy-cheat-sheet

This Python cheat sheet is a handy reference with code samples for doing linear algebra with SciPy and interacting with NumPy.

SciPy Cheat Sheet: Linear Algebra in Python 

<p>Data science is an interdisciplinary field that uses mathematics and advanced statistics to make predictions. All data science algorithms directly or indirectly use mathematical concepts. Solid understanding of math will help you develop innovative data science solutions such as a recommender system. If you are good at mathematics, it will make your transition into data science easier. As a data scientist, you have to utilize the fundamental concepts of mathematics to solve problems.</p>
<p>Apart from mathematics, you also need domain knowledge, programming skills, business skills, analytical skills, and a curious mindset. There is no way of escaping mathematics for a data scientist. You have to inculcate and teach yourself the basics of mathematics and statistics to become a data scientist.</p>
<p>In this tutorial, you are going to explore basic math concepts for data science (or especially deep learning) such as:</p>
<ul>
<li>Scalar and vector</li>
<li>Matrix and tensor</li>
<li>Determinant</li>
<li>Eigenvalues and vectors</li>
<li>NORM function</li>
<li>Matrix factorization</li>
<li>Singular value decomposition</li>
<li>Moore-Penrose Pseudoinverse</li>
<li>Hadamard product</li>
<li>Entropy</li>
<li>Kullback-Leibler Divergence</li>
<li>Gradient Descent</li>
</ul>
<h2 id="scalar-and-vector">Scalar and Vector</h2>
<ul>
<li><strong>Scalars:</strong> A scalar is a single number that deals with the vector in space via scalar multiplication.</li>
</ul>
<ul>
<li><strong>Vectors:</strong> A vector V is an ordered set of items. Vector is an array of numbers that can be a row or a column. Vectors can be added together and can be multiplied by a real number known as a scaler.</li>
</ul>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714959/image1.png" /></p>
<p><a href="https://medium.com/@manish54.thapliyal/machine-learning-basics-scalars-vectors-matrices-and-tensors-e120ecd0e6f7">Image Source</a></p>
<pre><code class="lang-python"># Import numpy module
import numpy as np

# creating a vector
v = np.array([1, 2, 3, 4, 5])
print(v)
</code></pre>
<pre><code>[1 2 3 4 5]
</code></pre><pre><code class="lang-python"># Vector Operations
# Import numpy module
import numpy as np

# Create two vector
a = np.array([1, 2, 3, 4, 5])
b = np.array([1, 2, 3, 4, 5])

# adding two vectors
add = a + b
print(&quot;Addition:&quot;,add)

# Vector Subtraction
sub = a - b
print(&quot;Subtraction:&quot;,sub)

# Vector Multiplication
mul = a * b
print(&quot;Multiplication:&quot;,mul)

# Vector Division
div = a / b
print(&quot;division&quot;,div)
</code></pre>
<pre><code>Addition: [ 2  4  6  8 10]
Subtraction: [0 0 0 0 0]
Multiplication: [ 1  4  9 16 25]
division [1. 1. 1. 1. 1.]
</code></pre><h2 id="matrix-and-tensor">Matrix and Tensor</h2>
<ul>
<li><strong>Matrices:</strong> A matrix is an N-D array of numbers which represent transformations. You can think of a matrix as a transformation such as &quot;scale&quot;, &quot;rotate&quot;, &quot;shear&quot;, and &quot;flip&quot;.  It turns single points in space into different points in space.</li>
</ul>
<pre><code class="lang-python"># Import numpy module
import numpy as np

# create 2*2 matrix
a1=np.array([[1, 2], [3, 4]])
a2=np.array([[1, 2], [3, 4]])

# Dot Product
dot_product = np.dot(a1,a2)
print(&quot;Dot Product: \n&quot;,dot_product)

# Cross Product
cross_product = np.cross(a1,a2)
print(&quot;Cross Product: \n&quot;, cross_product)
</code></pre>
<pre><code>Dot Product:
 [[ 7 10]
 [15 22]]
Cross Product:
 [0 0]
</code></pre><p>Dot Product of two vectors represents the projection of one vector on another vector and Cross Product of two vectors enables one vector to identify the plane on which both the vectors can lie.</p>
<ul>
<li><strong>Tensor:</strong> Sometimes, you will need an array with more than two dimensions and with a different dimension of each array, arranged in a grid known as a tensor. Tensor dimension is known as its rank. Scalars and vectors are also a type of tensor. Zero-order tensors are scalers, and first-order tensors are vectors.</li>
</ul>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714959/image2.png" /></p>
<p><a href="https://subscription.packtpub.com/book/big_data_and_business_intelligence/9781787125933/14/ch14lvl1sec85/tensorflow-ranks-and-tensors">Image source</a></p>
<h2 id="determinant">Determinant</h2>
<p>Determinant is a scalar value that represents a factor of a matrix by which length(in 1-dimension), area(2-dimension), volume (3-dimension) can be explored. If the determinant is 2, it is twice the volume(in 3-dimension), or if the determinant is 1, it will not affect the volume(in 3-dimension). If the determinant is 0, it has no inverse because zero multiplied with anything will give you zero.</p>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714959/image3.png" /></p>
<p><strong>Properties:</strong></p>
<ul>
<li><p>The product of matrices is the product of the determinant of the matrices: det (M1M2)=det (M1) det (M2).</p>
</li>
<li><p>The determinant of a matrix is equivalent to the product of eigenvalues.</p>
</li>
<li><p>If you multiply a matrix by a constant then the determinant changes by det(cM)=cN det (M). N is the dimension of the matrix.</p>
</li>
</ul>
<p><em>If your matrix represents a stretchable toy than a determinant of this matrix shows how much you’ve stretched your toy.</em></p>
<pre><code class="lang-python"># Import numpy module
import numpy as np

# Create 2*2 matrix
arr=np.array([[1,2],[3,4]])

# Compute Determinant of a matrix
arr_det=np.linalg.det(arr)

# Print the Computed Determinant
print(&quot;Determinant:&quot;,arr_det)
</code></pre>
<pre><code>Determinant: -2.0000000000000004
</code></pre><h2 id="eigenvalues-and-eigenvectors">Eigenvalues and Eigenvectors</h2>
<p>An eigenvector of square matrix A is a nonzero vector such that multiplication by A alters only the scale of v.</p>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714959/image4.png" /></p>
<p><a href="https://www.sharetechnote.com/html/Handbook_EngMath_Matrix_Eigen.html">Image Source</a></p>
<p>The eigenvector is also known as the characteristic vector. It is a non-zero vector that only changes with the scalar factor when a linear transformation is applied to it.</p>
<p>Eigenvectors are rotational axes of the linear transformation. These axes are fixed in direction, and eigenvalue is the scale factor by which the matrix is scaled up or down. Eigenvalues are also known as characteristic values or characteristic roots.
In other words, you can say eigenvalues are fixed line or plane which limits the behavior of the linear transformation and eigenvalues of a linear transformation are the factor of distortion.  </p>
<p>Determinant tells you the area of shapes that scaled up and down in linear transformation. That&#39;s why the multiplication of eigenvalues is equal to the determinant.</p>
<pre><code class="lang-python"># Import numpy module
import numpy as np

# Create 2*2 matrix
arr=np.array([[1,2],[3,4]])

# Find eigenvalues and eigenvectors
eigenvalues, eigenvectors = np.linalg.eig(arr)

# print the eigenvalues and eigenvectors
print(&quot;Eigen Values: \n&quot;,eigenvalues)
print(&quot;Eigen Vectors:\n&quot;, eigenvectors)
</code></pre>
<pre><code>Eigen Values:
 [-0.37228132  5.37228132]
Eigen Vectors:
 [[-0.82456484 -0.41597356]
 [ 0.56576746 -0.90937671]]
</code></pre><h2 id="norm-function">NORM Function</h2>
<p>Sometimes you want to measure the size of a vector. Norm function helps you measure the size of a vector. It assigns a strict positive length to a vector in a vector space except for zero vector. It includes the L^p norm. It maps vectors to non-negative values. It is equivalent to Euclidean distance for a vector and a matrix. It is equal to the largest singular value.</p>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714959/image5.png" /></p>
<pre><code class="lang-python"># import numpy module
import numpy as np

# Create 3*3 Matrix
a = np.array([[1,2,3],[4,5,6],[7,8,9]])

# Compute norm
a_norm = np.linalg.norm(a)

# print the norm of function
print(a_norm)
</code></pre>
<pre><code>16.881943016134134
</code></pre><h2 id="matrix-factorization">Matrix Factorization</h2>
<p>Matrix factorization, also known as matrix decomposition. It is used to split a matrix into its constituent parts. Matrix factorization is equivalent to the factoring of numbers, such as the factoring of 10 into 2 x 5.  It is used to solve linear equations.</p>
<p>The following matrix factorization techniques are available:</p>
<ul>
<li><strong>LU Decomposition</strong> is for square matrices and decomposes a matrix into L and U components.</li>
</ul>
<ul>
<li><strong>QR Decomposition</strong> is for m x n matrices (not limited to square matrices) and decomposes a matrix into Q and R components. It is not limited to square matrices like LU decomposition.</li>
</ul>
<ul>
<li><strong>Cholesky Decomposition</strong> is used for solving linear least squares for linear regression, as well as simulation and optimization methods.</li>
</ul>
<ul>
<li><strong>Singular Value Decomposition</strong> explained in the next section.</li>
</ul>
<h2 id="singular-value-decomposition">Singular Value Decomposition</h2>
<p>In the previous section, we have seen eigen-decomposition of a matrix that decomposes into eigenvectors and eigenvalues. Singular value decomposition is a type of matrix factorization method that decomposes into singular vectors and singular values. It offers various useful applications in signal processing, psychology, sociology, climate, atmospheric science, statistics, and astronomy.</p>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714959/image6.png" /></p>
<ul>
<li>M is an m×m matrix</li>
<li>U is a m×n left singular matrix</li>
<li>Σ is a n×n diagonal matrix with non-negative real numbers.</li>
<li>V is a m×n right singular matrix</li>
<li>V* is n×m matrix, which is the transpose of the V.</li>
</ul>
<pre><code class="lang-python"># Import numpy module
import numpy as np

# Create 3*3 matrix
a = np.array([[1, 3, 4], [5, 6, 9], [1, 2, 3], [7, 6, 8]])

# Decomposition of matrix using SVD
U, s, Vh = np.linalg.svd(a, full_matrices=False)

U,s,Vh
</code></pre>
<pre><code>(array([[-0.27067357, -0.61678044,  0.69789573],
        [-0.65939972, -0.2937857 , -0.62152182],
        [-0.20244298, -0.3480035 , -0.06765408],
        [-0.67152413,  0.64200111,  0.34939247]]),
 array([18.0376394 ,  2.34360292,  0.38870323]),
 array([[-0.46961711, -0.51018039, -0.72053851],
        [ 0.87911451, -0.19502274, -0.43488368],
        [-0.08134773,  0.83766467, -0.54009299]]))
</code></pre><pre><code class="lang-python"># Generate the initial matrix
new_a = np.dot(U, np.dot(np.diag(s), Vh))

# Print original matrix
print(new_a)
</code></pre>
<pre><code>[[1. 3. 4.]
 [5. 6. 9.]
 [1. 2. 3.]
 [7. 6. 8.]]
</code></pre><h2 id="moore-penrose-pseudoinverse">Moore-Penrose Pseudoinverse</h2>
<p>Pseudoinverse of a matrix is generalized of the inverse matrix. It is utilized in computing the least square solutions. Moore-Penrose inverse is the most popular form of matrix pseudoinverse.  </p>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714959/image7.png" /></p>
<pre><code class="lang-python"># Import numpy module
import numpy as np

# Create 3*3 matrix
a = np.array([[1, 3, 4], [5, 6, 9], [1, 2, 3], [7, 6, 8]])

# Compute the (Moore-Penrose) pseudo-inverse of a matrix.
inv=np.linalg.pinv(a)

# Print pseudo-inverse of a matrix.
print(inv)
</code></pre>
<pre><code>[[-0.37037037  0.03703704 -0.11111111  0.18518519]
 [ 1.56296296 -1.2962963  -0.11111111  0.71851852]
 [-0.84444444  0.94444444  0.16666667 -0.57777778]]
</code></pre><h2 id="hadamard-product">Hadamard Product</h2>
<p>Hadamard product or Schur product is an element-wise product of two same dimensional original matrices. It is also known as the element-wise product. It is simpler than the matrix product. Hadamard product is utilized in JPEG lossy compression algorithms. Hadamard product is commutative, associative, and distributive. It easily obtains the inverse and simplifies the computation of power matrices.</p>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714959/image8.png" /></p>
<p>Hadamard product is utilized in various fields such as code correction in satellite transmissions, information theory, cryptography, pattern recognition, neural network, maximum likelihood estimation, JPEG lossy compression, multivariate statistical analysis, and linear modeling.</p>
<pre><code class="lang-python"># Import numpy module
import numpy as np

# Create 2*2 matrix a1 and a2
a1=np.array([[1, 2], [3, 4]])
a2=np.array([[1, 2], [3, 4]])

# Element wise multiplication
hadamard_product = np.multiply(a1,a2)

# Print hadamard distance
print(&quot;Hadamard Product: \n&quot;, hadamard_product)
</code></pre>
<pre><code>Hadamard Product:
 [[ 1  4]
 [ 9 16]]
</code></pre><h2 id="entropy">Entropy</h2>
<p>&quot;The entropy of a random variable is a function which attempts to characterize the unpredictability of a random variable.&quot; (<a href="https://people.cs.umass.edu/~elm/Teaching/Docs/mutInf.pdf">Entropy and Mutual Information</a>)&quot; It is used for the construction of an automatic decision tree at each step of tree building; feature selection is done such using entropy criteria. Model selection is based on the principle of the maximum entropy, which states from the conflicting models, the one with the highest entropy is the best.</p>
<p>&quot;If a random variable X takes on values in a set χ={x1, x2,..., xn}, and is defined by a probability distribution P(X), then we will write the entropy of the random variable as,&quot; (<a href="https://people.cs.umass.edu/~elm/Teaching/Docs/mutInf.pdf">Entropy and Mutual Information</a>)   </p>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714959/image9.png" /></p>
<p>&quot;If the log in the above equation is taken to be to the base 2, then the entropy is expressed in bits. If the log is taken to be the natural log, then the entropy is expressed in nats. More commonly, entropy is expressed in bits.&quot; (<a href="https://people.cs.umass.edu/~elm/Teaching/Docs/mutInf.pdf">Entropy and Mutual Information</a>)</p>
<pre><code class="lang-python"># Import scipy and numpy module
import scipy.stats
import numpy as np

# Create an array
a=np.array([1,1,2,3,1,3,4,2,5,6,3,2,4,3])


# Compute probability distribution
a_pdf=scipy.stats.norm.pdf(a)

# Calculate the entropy of a distribution for given probability values.
entropy = scipy.stats.entropy(a_pdf) # get entropy from probability values
print(&quot;Entropy: &quot;,entropy)
</code></pre>
<pre><code>Entropy:  1.6688066853941022
</code></pre><h2 id="kullback-leibler-divergence">Kullback-Leibler Divergence</h2>
<p>Kullback–Leibler divergence is the relative entropy of two probability distributions. It measures the distance(similarity or dissimilarity) of one distribution from another reference probability distribution. 0 Value of Kullback–Leibler divergence indicates that both distributions are identical. It can be expressed as,</p>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714959/image10.png" /></p>
<p>It does sound like a distance measure, but it is not. This is because it is asymmetric in nature, which means the metric isn’t commutative. In general, you can say that D(p, q) ≠D(q, p). KL divergence is frequently used in the un-supervised machine learning technique “Variational Autoencoders”.</p>
<pre><code class="lang-python"># Import scipy.stats and numpy module
import scipy.stats
import numpy as np

# Create numpy arrays
a=np.array([1,1,2,3,1,3,4,2,5,6,3,2,4,3])
b=np.array([1,1,3,4,2,4,5,2,5,6,3,2,4,3])

# Compute probability distribution
a_pdf=scipy.stats.norm.pdf(a)
b_pdf=scipy.stats.norm.pdf(b)

# compute relative entropy or KL Divergence
kl_div=scipy.stats.entropy(a_pdf,b_pdf)

print(&quot;KL Divergence: &quot;,kl_div)
</code></pre>
<pre><code>KL Divergence:  0.26732496641464365
</code></pre><h2 id="gradient-descent">Gradient Descent</h2>
<p>Gradient descent is one of the most well-known algorithms used to optimize coefficients and bias for linear regression, logistic regression, and neural networks. It is an iterative process that finds the minimum of any given function.</p>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714960/image11.png" /></p>
<p><a href="https://ml-cheatsheet.readthedocs.io/en/latest/gradient_descent.html">Image Source</a></p>
<p><center><img src="https://res.cloudinary.com/dchysltjf/image/upload/f_auto,q_auto:best/v1560714960/image12.png" /></p>
<p><a href="https://jacklee.work/%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/2019-03-28-Understanding%20the%20Mathematics%20behind%20Gradient%20Descent.html">Image Source</a></p>
<p>Three types of the gradient descent algorithm: full batch, stochastic, and mini-batch gradient descent. Full batch gradient descent uses the whole dataset for computing gradient while stochastic gradient descent uses a sample of the dataset for computing gradient. Mini-batch gradient descent is a combination of both stochastic and batch gradient descent. The training set is split into various small groups called batches. These small batches compute the loss one by one and average the final loss result.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Congratulations, you have made it to the end of this tutorial!</p>
<p>You learned basic mathematical concepts for deep learning such as scalar, vector, matrix, tensor, determinant eigenvalues, eigenvectors, NORM function, singular value decomposition(SVD), Moore-Penrose Pseudoinverse, Hadamard product, Entropy Kullback-Leibler Divergence, and Gradient Descent.</p>
<p>Along the road, you have also practiced these concepts in python using NumPy and SciPy.</p>
<p>If you would like to learn more about Python, take the following DataCamp courses:</p>
<ul>
<li><a href="https://www.datacamp.com/courses/clustering-methods-with-scipy">Clustering Methods with SciPy</a></li>
<li><a href="https://www.datacamp.com/courses/unsupervised-learning-in-python">Unsupervised Learning in Python</a></li>
<li><a href="https://www.datacamp.com/courses/statistical-thinking-in-python-part-1">Statistical Thinking in Python (Part 1)</a></li>
<li><a href="https://www.datacamp.com/courses/statistical-simulation-in-python">Statistical Simulation in Python</a></li>
</ul>


https://www.datacamp.com/community/tutorials/demystifying-mathematics-concepts-deep-learning

Explore basic math concepts for data science and deep learning such as scalar and vector, determinant, singular value decomposition, and more.

Demystifying Mathematical Concepts for Deep Learning

https://www.datacamp.com/community/tutorials/python-scipy-tutorial

A SciPy tutorial in which you'll learn the basics of linear algebra that you need for machine learning in Python, with a focus how to with NumPy.  

community

datacamp

mathematics

News (10)

Solving internal co-variate shift in deep learning with linked Neurons

Mandalas Colored

Tutorials (2)

Demystifying Mathematical Concepts for Deep Learning

Scipy Tutorial: Vectors and Arrays (Linear Algebra)

Cheat Sheets (1)

SciPy Cheat Sheet: Linear Algebra in Python

Open Courses (0)

Podcast (0)

Blog Posts (1)

SciPy Cheat Sheet: Linear Algebra in Python

Tech Thoughts (0)