Scipy Tutorial: Vectors and Arrays (Linear Algebra)

The Essentials of NumPy ndarray Objects

An array is, structurally speaking, nothing but pointers. It’s a combination of a memory address, a data type, a shape and strides. It contains information about the raw data, how to locate an element and how to interpret an element.

The memory address and strides are important when you dive deeper into the lower-level details of arrays, while the data type and shape are things that beginners should surely know and understand. Two other attributes that you might want to consider are the data and size, which allow you to gather even more information on your array.

Refresh the usage of the ndarray attributes in the following DataCamp Light chunk. The array myArray has already been loaded in. You can inspect it by typing it in into the IPython shell and by pressing ENTER.

You’ll see in the results of the code that is included in the code chunk above that the data type of myArray is int64. When you’re intensively working with arrays, you will definitely remember that there are ways to convert your arrays from one data type to another with the astype() method.

Nevertheless, when you’re using SciPy and NumPy together, you might also find the following type handling NumPy functions very useful, especially when you’re working with complex numbers:

Try to add print() calls to see the results of the code that is given above. Then, you’ll see that complex numbers have a real and an imaginary part to them. The np.real() and np.imag() functions are designed to return these parts to the user, respectively.

Alternatively, you might also be able to use np.cast to cast an array object to a different data type, such as float in the example above.

The only thing that really stands out in difficulty in the above code chunk is the np.real_if_close() function. When you give it a complex input, such as myArray, you’ll get a real array back if the complex parts are close to zero. This last part, “close to 0”, can be adjusted by yourself with the tol argument that you can pass to the function. Play around with it the treshold to see what happens!

Array Creation

You have now seen how to inspect your array and to make adjustments in the data type of it, but you haven’t explicitly seen hwo to create arrays. You should already know that you can use np.array() to to this, but there are other routines for array creation that you should know about: np.eye() and np.identity().

The np.eye() funcion allows you to create a square matrix with dimensions that are equal to the positive integer that you give as an argument to the function. The entries are generally filled with zeros, only the matrix diagonal is filled with ones. The np.identity() function works and does the same and also returns an identity array.

However, note that np.eye() can take an additional argument k that you can specify to pick the index of the diagonal that you want to populate with ones.

Other array creation functions that will most definitely come in handy when you’re working with the matrices for linear algebra are the following:

• The np.arange() function creates an array with uniformly spaced values between two numbers. You can specify the spacing between the elements,
• The latter also holds for np.linspace(), but with this function you specify the number of elements that you want in your array.
• Lastly, the np.logspace() function also creates arrays with uniformly spaced values, but this time in a logarithmic scale. This means that the spacing is now logarithmical: two numbers are evenly spaced between the logarithms of these two to the base of 10.

These functions create arrays in the form of meshes, which can be used to create mesh grids, about which you read earlier in this post.

Note that numpy is already imported as np in the code chunk below!

Now that you have refreshed your memory and you know how to handle the data types of your arrays, it’s time to also tackle the topic of indexing and slicing.

Install SciPy

Of course you first need to make sure firstly that you have Python installed. Go to this page if you still need to do this :) If you’re working on Windows, make sure that you have added Python to the PATH environment variable. In addition, don’t forget to install a package manager, such as pip, which will ensure that you’re able to use Python’s open-source libraries.

Note that recent versions of Python 3 come with pip, so double check if you have it and if you do, upgrade it before you install any other packages:

 pip install pip --upgrade pip --version

But just installing a package manager is not enough; You also need to download to download the wheel for the library: go here to get your SciPy wheel. After the download, open up the terminal at the download directory on your pc and install it. Additionally, you can check whether the installation was successful and confirm the package version that you’re running:

# Install the wheel install "scipy‑0.18.1‑cp36‑cp36m‑win_amd64.whl"# Confirm successful installimport scipy# Check package versionscipy.__version__

After these steps, you’re ready to goy!

Tip: install the package by downloading the Anaconda Python distribution. It’s an easy way to get started quickly, as Anaconda not only includes 100 of the most popular Python, R and Scala packages for data science, but also includes several open course development environments such as Jupyter and Spyder. If you’d like to start working with Jupyter Notebook, check out this Jupyter notebook tutorial.

If you haven’t downloaded it already, go here to get it.

Vectors and Matrices: The Basics

Now that you have made sure that your workspace is prepped, you can finally get started with linear algebra in Python. In essence, this discipline is occupied with the study of vector spaces and the linear mappings that exist between them. These linear mappings can be described with matrices, which also makes it easier to calculate.

Remember that a vector space is a fundamental concept in linear algebra. It’s a space where you have a collection of objects (vectors) and where you can add or scale two vectors without the resulting vector leaving the space. Remember also that vectors are rows (or columns) of a matrix.

But how does this work in Python?

You can easily create a vector with the np.array() function. Similarly, you can give a matrix structure to every one-or two-dimensional ndarray with either the np.matrix() or np.mat() commands.

Try it out in the following code chunk:

So arrays and matrices are the same, besides from the formatting?

Well, not exactly. There are some differences:

• A matrix is 2-D, while arrays are usually n-D,
• As the functions above already implied, the matrix is a subclass of ndarray,
• Both arrays and matrices have .T(), but only matrices have .H() and .I(),
• Matrix multiplication works differently from element-wise array multiplication, and
• To add to this, the ** operation has different results for matrices and arrays

When you’re working with matrices, you might sometimes have some in which most of the elements are zero. These matrices are called “sparse matrices”, while the ones that have mostly non-zero elements are called “dense matrices”.

In itself, this seems trivial, but when you’re working with SciPy for linear algebra, this can sometimes make a difference in the modules that you use to get certain things done. More concretely, you can use scipy.linalg for dense matrices, but when you’re working with sparse matrices, you might also want to consider checking up on the scipy.sparse module, which also contains its own scipy.sparse.linalg.

For sparse matrices, there are quite a number of options to create them. The code chunk below lists some:

Additionally, there are also some other functions that you might be able to use to create sparse matrices: Block Sparse Row matrices with bsr_matrix(), COOrdinate format sparse matrices with coo_matrix(), DIAgonal storage sparse matrices with dia_matrix(), and Row-based linked list sparse matrices with lil_matrix().

There are really a lot of options, but which one should you choose if you’re making a sparse matrix yourself?

It’s not that hard.

Basically, it boils down to first is how you’re going to initialize it. Next, consider what you want to be doing with your sparse matrix.

More concretely, you can go through the following checklist to decide what type of sparse matrix you want to use:

• If you plan to fill the matrix with numbers one by one, pick a coo_matrix() or dok_matrix() to create your matrix.
• If you want to initialize the matrix with an array as the diagonal, pick dia_matrix() to initialize your matrix.
• For sliced-based matrices, use lil_matrix().
• If you’re constructing the matrix from blocks of smaller matrices, consider using bsr_matrix().
• If you want to have fast access to your rows and columns, convert your matrices by using the csr_matrix() and csc_matrix() functions, respectively. The last two functions are not great to pick when you need to initialize your matrices, but when you’re multiplying, you’ll definitely notice the difference in speed.

Easy peasy!

Eigenvalues and Eigenvectors

The first topic that you will tackle are the eigenvalues and eigenvectors.

Eigenvalues are a new way to see into the heart of a matrix. But before you go more into that, let’s explain first what eigenvectors are. Almost all vectors change direction, when they are multiplied by a matrix. However, certain exceptional, resulting vectors are in the same direction as the vectors that are the result of the multiplication. These are the eigenvectors.

In other words, multiply an eigenvector by a matrix, and the resulting vector of that multiplication is equal to a multiplication of the original eigenvector with \(\lambda\), the eigenvalue: \[Ax = \lambda x. \]

This means that the eigenvalue gives you very valuable information: it tells you whether one of the eigenvectors is stretched, shrunk, reversed, or left unchanged—when it is multiplied by a matrix.

You use the eig() function from the linalg SciPy module to solve ordinary or generalized eigenvalue problems for square matrices.

Note that the eigvals() function is another way of unpacking the eigenvalues of a matrix.

When you’re working with sparse matrices, you can fall back on the module scipy.sparse to provide you with the correct functions to find the eigenvalues and eigenvectors:

la, v = sparse.linalg.eigs(myMatrix,1)

Note that the code above specifies the number of eigenvalues and eigenvectors that has to be retrieved, namely, 1.

The eigenvalues and eigenvectors are important concepts in many computer vision and machine learning techniques, such as Principal Component Analysis (PCA) for dimensionality reduction and EigenFaces for face recognition.

Singular Value Decomposition (SVD)

Next, you need to know about SVD if you want to really learn data science. The singular value decomposition of a matrix \(A\) is the decomposition or facorization of \(A\) into the product of three matrices: \(A = U * \Sigma * V^t\).

The size of the individual matrices is as follows if you know that matrix \(A\) is of size \(M\) x \(N\):

• Matrix \(U\) is of size \(M\) x \(M\)
• Matrix \(V\) is of size \(N\) x \(N\)
• Matrix \(\Sigma\) is of size \(M\) x \(N\)

The \(*\) indicates that the matrices are multiplied and the \(^t\) that you see in \(V^t\) means that the matrix is transposed, which means that the rows and columns are interchanged.

Simply stated, singular value decomposition provides a way to break a matrix into simpler, meaningful pieces. These pieces may contain some data we are interested in.

Note that for sparse matrices, you can use the sparse.linalg.svds() function to perform the decomposition.

If you’re new to data science, the matrix decomposition will be quite opaque for you: you might not immediately see any use cases to apply this. But SVD is useful in many tasks, such as data compression, noise reduction and data analysis. In the following, you will see how SVD can be used to compress images:

# Import the necessary packagesimport numpy as npfrom scipy import linalgfrom skimage import dataimport matplotlib.pyplot as plt# Get an image from `skimage`img= data.camera()# Check number of singular valueslinalg.svdvals(img)# Singular Value DecompositionU, s, Vh = linalg.svd(img)# Use only 32 singular valuesA = np.dot(U[:,0:32],           np.dot(np.diag(s[0:32]), Vh[0:32,:]))fig = plt.figure(figsize=(8, 3))# Add a subplot to the figureax = fig.add_subplot(121)# Plot `img` on grayscaleax.imshow(img, cmap='gray')# Add a second subplot to the figureax2 = fig.add_subplot(122)# Plot `A` in the second subplotax2.imshow(A)# Add a titlefig.suptitle('Image Compression with SVD', fontsize=14, fontweight='bold')# Show the plotplt.show()

Which will give you the following result:

Consider also the following examples where SVD is used:

• SVD is closely linked to Principal Component Analysis (PCA), which is used for dimensionality reduction: both result in a set of “new axes” that are constructed from linear combinations of the the feature space axes of your data. These “new axes” break down the variance in the data points based on each direction’s contribution to the variance in the data. To see a concrete example of how PCA works on data, go to our Scikit-Learn Tutorial.
• Another link is one with data mining and natural language processing (NLP): Latent Semantic Indexing (LSI). It s a technique that is used in document retrieval and word similarity. Latent semantic indexing uses SVD to group documents to the concepts that could consist of different words found in those documents. Various words can be grouped into a concept. Also here, SVD reduces the noisy correlation between words and their documents, and it decreases the number of dimensions that the original data has.

You see, SVD is an important concept in your data science journey that you must cover. That’s why you should consider going deeper into SVD than what this tutorial covers: for example, go to this page to read more about this matrix decomposition.