Course Notes: Introduction to Deep Learning with Keras

add text here

# Write and run code here

You're good at spotting lies! Keras is a wrapper around a backend library, so a backend like TensorFlow, Theano, CNTK, etc must be provided.

Hello nets!

You're going to build a simple neural network to get a feeling of how quickly it is to accomplish this in Keras.

You will build a network that takes two numbers as an input, passes them through a hidden layer of 10 neurons, and finally outputs a single non-constrained number.

A non-constrained output can be obtained by avoiding setting an activation function in the output layer. This is useful for problems like regression, when we want our output to be able to take any non-constrained value.

Import the Sequential model from tensorflow.keras.models and the Denselayer from tensorflow.keras.layers. Create an instance of the Sequential model. Add a 10-neuron hidden Dense layer with an input_shape of two neurons. Add a final 1-neuron output layer and summarize your model with summary().

Import the Sequential model and Dense layer

from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense

Create a Sequential model

model = Sequential()

Add an input layer and a hidden layer with 10 neurons

model.add(Dense(10, input_shape=(2,), activation="relu"))

Add a 1-neuron output layer

model.add(Dense(1))

Summarise your model

model.summary()

Counting parameters

You've just created a neural network. But you're going to create a new one now, taking some time to think about the weights of each layer. The Keras Dense layer and the Sequential model are already loaded for you to use.

This is the network you will be creating:

Instantiate a new Sequential() model. Add a Dense() layer with five neurons and three neurons as input. Add a final dense layer with one neuron and no activation.

Instantiate a new Sequential model

model = Sequential()

Add a Dense layer with five neurons and three inputs

model.add(Dense(5, input_shape=(3,), activation="relu"))

Add a final Dense layer with one neuron and no activation

model.add(Dense(1))

Summarize your model

model.summary()

Build as shown!

You will take on a final challenge before moving on to the next lesson. Build the network shown in the picture below. Prove your mastered Keras basics in no time!

from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense

Instantiate a Sequential model

model = Sequential()

Build the input and hidden layer

model.add(Dense(3, input_shape = (2,)))

Add the ouput layer

model.add(Dense(1))

Specifying a model

You will build a simple regression model to predict the orbit of the meteor!

Your training data consist of measurements taken at time steps from -10 minutes before the impact region to +10 minutes after. Each time step can be viewed as an X coordinate in our graph, which has an associated position Y for the meteor orbit at that time step.

Note that you can view this problem as approximating a quadratic function via the use of neural networks.

This data is stored in two numpy arrays: one called time_steps , what we call features, and another called y_positions, with the labels. Go on and build your model! It should be able to predict the y positions for the meteor orbit at future time steps.

Keras Sequential model and Dense layers are available for you to use.

Instantiate a Sequential model. Add a Dense layer of 50 neurons with an input shape of 1 neuron. Add two Dense layers of 50 neurons each and 'relu' activation. End your model with a Dense layer with a single neuron and no activation.

Instantiate a Sequential model

model = Sequential()

Add a Dense layer with 50 neurons and an input of 1 neuron

model.add(Dense(50, input_shape=(1,), activation='relu'))

Add two Dense layers with 50 neurons and relu activation

model.add(Dense(50, activation='relu')) model.add(Dense(50, activation='relu'))

End your model with a Dense layer and no activation

model.add(Dense(1))

Instantiate a Sequential model

model = Sequential()

Add a Dense layer with 50 neurons and an input of 1 neuron

model.add(Dense(50, input_shape=(1,), activation='relu'))

Add two Dense layers with 50 neurons and relu activation

model.add(Dense(50, activation='relu')) model.add(Dense(50, activation='relu'))

End your model with a Dense layer and no activation

model.add(Dense(1))

Training

You're going to train your first model in this course, and for a good cause!

Remember that before training your Keras models you need to compile them. This can be done with the .compile() method. The .compile() method takes arguments such as the optimizer, used for weight updating, and the loss function, which is what we want to minimize. Training your model is as easy as calling the .fit() method, passing on the features, labels and a number of epochs to train for.

The regression model you built in the previous exercise is loaded for you to use, along with the time_steps and y_positions data. Train it and evaluate it on this very same data, let's see if your model can learn the meteor's trajectory.

Compile your model making use of the 'adam' optimizer and 'mse' as your loss function. Fit your model using the features and labels for 30 epochs. Evaluate your model with the .evaluate() method, passing the features and labels used during training.

Compile your model

model.compile(optimizer = 'adam', loss = 'mse')

print("Training started..., this can take a while:")

Fit your model on your data for 30 epochs

model.fit(time_steps,y_positions, epochs = 30)

Evaluate your model

print("Final loss value:",model.evaluate(time_steps, y_positions))

1/63 [..............................] - ETA: 20s - loss: 3152.8167 61/63 [============================>.] - ETA: 0s - loss: 1527.4290 63/63 [==============================] - 0s 843us/step - loss: 1512.0977 Epoch 2/30

1/63 [..............................] - ETA: 0s - loss: 565.0184 55/63 [=========================>....] - ETA: 0s - loss: 258.5826 63/63 [==============================] - 0s 942us/step - loss: 246.1549 Epoch 3/30

1/63 [..............................] - ETA: 0s - loss: 115.9736 60/63 [===========================>..] - ETA: 0s - loss: 139.0434 63/63 [==============================] - 0s 858us/step - loss: 138.9797 Epoch 4/30

1/63 [..............................] - ETA: 0s - loss: 129.5075 57/63 [==========================>...] - ETA: 0s - loss: 122.8491 63/63 [==============================] - 0s 898us/step - loss: 122.5375 Epoch 5/30

1/63 [..............................] - ETA: 0s - loss: 134.7303 56/63 [=========================>....] - ETA: 0s - loss: 106.2992 63/63 [==============================] - 0s 919us/step - loss: 105.0882 Epoch 6/30

1/63 [..............................] - ETA: 0s - loss: 83.1582 61/63 [============================>.] - ETA: 0s - loss: 89.2547 63/63 [==============================] - 0s 841us/step - loss: 89.0923 Epoch 7/30

1/63 [..............................] - ETA: 0s - loss: 104.2328 51/63 [=======================>......] - ETA: 0s - loss: 70.8678 63/63 [==============================] - 0s 1ms/step - loss: 71.1999 Epoch 8/30

1/63 [..............................] - ETA: 0s - loss: 69.8733 60/63 [===========================>..] - ETA: 0s - loss: 56.4643 63/63 [==============================] - 0s 848us/step - loss: 56.3744 Epoch 9/30

1/63 [..............................] - ETA: 0s - loss: 59.7633 56/63 [=========================>....] - ETA: 0s - loss: 41.3658 63/63 [==============================] - 0s 920us/step - loss: 40.4538 Epoch 10/30

1/63 [..............................] - ETA: 0s - loss: 28.6026 53/63 [========================>.....] - ETA: 0s - loss: 29.7555 63/63 [==============================] - 0s 950us/step - loss: 28.8388 Epoch 11/30

1/63 [..............................] - ETA: 0s - loss: 22.3008 61/63 [============================>.] - ETA: 0s - loss: 19.9023 63/63 [==============================] - 0s 838us/step - loss: 19.7445 Epoch 12/30

1/63 [..............................] - ETA: 0s - loss: 12.6029 50/63 [======================>.......] - ETA: 0s - loss: 14.0550 63/63 [==============================] - 0s 1ms/step - loss: 13.8041 Epoch 13/30

1/63 [..............................] - ETA: 0s - loss: 9.4913 47/63 [=====================>........] - ETA: 0s - loss: 9.7622 63/63 [==============================] - 0s 1ms/step - loss: 9.1781 Epoch 14/30

1/63 [..............................] - ETA: 0s - loss: 3.9090 33/63 [==============>...............] - ETA: 0s - loss: 6.6329 63/63 [==============================] - 0s 1ms/step - loss: 6.3003 Epoch 15/30

1/63 [..............................] - ETA: 0s - loss: 7.3084 42/63 [===================>..........] - ETA: 0s - loss: 4.4178 63/63 [==============================] - 0s 1ms/step - loss: 4.3138 Epoch 16/30

1/63 [..............................] - ETA: 0s - loss: 2.9824 47/63 [=====================>........] - ETA: 0s - loss: 3.3751 63/63 [==============================] - 0s 1ms/step - loss: 3.1214 Epoch 17/30

1/63 [..............................] - ETA: 0s - loss: 4.3195 43/63 [===================>..........] - ETA: 0s - loss: 2.6009 63/63 [==============================] - 0s 1ms/step - loss: 2.4626 Epoch 18/30

1/63 [..............................] - ETA: 0s - loss: 1.9220 54/63 [========================>.....] - ETA: 0s - loss: 1.7962 63/63 [==============================] - 0s 959us/step - loss: 1.7754 Epoch 19/30

1/63 [..............................] - ETA: 0s - loss: 1.6610 53/63 [========================>.....] - ETA: 0s - loss: 1.4779 63/63 [==============================] - 0s 987us/step - loss: 1.4794 Epoch 20/30

1/63 [..............................] - ETA: 0s - loss: 1.3279 51/63 [=======================>......] - ETA: 0s - loss: 1.1764 63/63 [==============================] - 0s 1ms/step - loss: 1.1292 Epoch 21/30

1/63 [..............................] - ETA: 0s - loss: 0.9747 53/63 [========================>.....] - ETA: 0s - loss: 0.9902 63/63 [==============================] - 0s 970us/step - loss: 0.9846 Epoch 22/30

1/63 [..............................] - ETA: 0s - loss: 0.4731 48/63 [=====================>........] - ETA: 0s - loss: 0.7588 63/63 [==============================] - 0s 1ms/step - loss: 0.7527 Epoch 23/30

1/63 [..............................] - ETA: 0s - loss: 0.7435 53/63 [========================>.....] - ETA: 0s - loss: 0.6727 63/63 [==============================] - 0s 956us/step - loss: 0.6508 Epoch 24/30

1/63 [..............................] - ETA: 0s - loss: 0.1424 63/63 [==============================] - 0s 805us/step - loss: 0.4869 Epoch 25/30

1/63 [..............................] - ETA: 0s - loss: 0.3779 62/63 [============================>.] - ETA: 0s - loss: 0.4361 63/63 [==============================] - 0s 827us/step - loss: 0.4348 Epoch 26/30

1/63 [..............................] - ETA: 0s - loss: 0.3613 62/63 [============================>.] - ETA: 0s - loss: 0.3747 63/63 [==============================] - 0s 830us/step - loss: 0.3721 Epoch 27/30

1/63 [..............................] - ETA: 0s - loss: 0.4469 57/63 [==========================>...] - ETA: 0s - loss: 0.3130 60/63 [===========================>..] - ETA: 0s - loss: 0.3103 63/63 [==============================] - 2s 38ms/step - loss: 0.3065 Epoch 28/30

1/63 [..............................] - ETA: 0s - loss: 0.2232 22/63 [=========>....................] - ETA: 0s - loss: 0.2827 46/63 [====================>.........] - ETA: 0s - loss: 0.2970 63/63 [==============================] - 0s 2ms/step - loss: 0.2996 Epoch 29/30

1/63 [..............................] - ETA: 0s - loss: 0.3858 27/63 [===========>..................] - ETA: 0s - loss: 0.3191 39/63 [=================>............] - ETA: 0s - loss: 0.2951 54/63 [========================>.....] - ETA: 0s - loss: 0.2777 63/63 [==============================] - 0s 3ms/step - loss: 0.2626 Epoch 30/30

1/63 [..............................] - ETA: 0s - loss: 0.1238 21/63 [=========>....................] - ETA: 0s - loss: 0.1821 45/63 [====================>.........] - ETA: 0s - loss: 0.2095 63/63 [==============================] - 0s 2ms/step - loss: 0.2012

1/63 [..............................] - ETA: 6s - loss: 5.0286 63/63 [==============================] - 0s 592us/step - loss: 0.2253 Final loss value: 0.2253316193819046

Amazing! You can check the console to see how the loss function decreased as epochs went by. Your model is now ready to make predictions on unseen data.

Predicting the orbit! You've already trained a model that approximates the orbit of the meteor approaching Earth and it's loaded for you to use.

Since you trained your model for values between -10 and 10 minutes, your model hasn't yet seen any other values for different time steps. You will now visualize how your model behaves on unseen data.

If you want to check the source code of plot_orbit, paste show_code(plot_orbit) into the console.

Hurry up, the Earth is running out of time!

Remember np.arange(x,y) produces a range of values from x to y-1. That is the [x, y) interval.

Use the model's .predict() method to predict from -10 to 10 minutes.

Predict the twenty minutes orbit

twenty_min_orbit = model.predict(np.arange(-10, 11))

Plot the twenty minute orbit

plot_orbit(twenty_min_orbit)

Use the model's .predict() method to predict from -40 to 40 minutes.

Predict the eighty minute orbit

eighty_min_orbit = model.predict(np.arange(-40, 41))

Plot the eighty minute orbit

plot_orbit(eighty_min_orbit)

Your model fits perfectly to the scientists trajectory for time values between -10 to +10, the region where the meteor crosses the impact region, so we won't be hit! However, it starts to diverge when predicting for new values we haven't trained for. This shows neural networks learn according to the data they are fed with. Data quality and diversity are very important. You've barely scratched the surface of what neural networks can do. Are you prepared for the next chapter?

Exploring dollar bills

You will practice building classification models in Keras with the Banknote Authentication dataset.

Your goal is to distinguish between real and fake dollar bills. In order to do this, the dataset comes with 4 features: variance,skewness,kurtosis and entropy. These features are calculated by applying mathematical operations over the dollar bill images. The labels are found in the dataframe's class column.

A pandas DataFrame named banknotes is ready to use, let's do some data exploration!

Import seaborn as sns. Use seaborn's pairplot() on banknotes and set hue to be the name of the column containing the labels. Generate descriptive statistics for the banknotes authentication data. Count the number of observations per label with .value_counts().

Import seaborn

import seaborn as sns

Use pairplot and set the hue to be our class column

sns.pairplot(banknotes, hue="class")

Show the plot

plt.show()

Describe the data

print('Dataset stats: \n', banknotes.describe())

Count the number of observations per class

print('Observations per class: \n', banknotes["class"].value_counts())

Your pairplot shows that there are features for which the classes spread out noticeably. This gives us an intuition about our classes being easily separable. Let's build a model to find out what it can do!

A binary classification model

Now that you know what the Banknote Authentication dataset looks like, we'll build a simple model to distinguish between real and fake bills.

You will perform binary classification by using a single neuron as an output. The input layer will have 4 neurons since we have 4 features in our dataset. The model's output will be a value constrained between 0 and 1.

We will interpret this output number as the probability of our input variables coming from a fake dollar bill, with 1 meaning we are certain it's a fake bill.

Import the Sequential model and Dense layer from tensorflow.keras. Create a sequential model. Add a 4 neuron input layer with the input_shape parameter and a 1 neuron output layer with sigmoid activation. Compile your model using sgd as an optimizer.

Import the sequential model and dense layer

from tensorflow.keras.models import Sequential from tensorflow.keras.layers import Dense

Create a sequential model

model = Sequential()

Add a dense layer

model.add(Dense(1, input_shape=(4,), activation= 'sigmoid'))

Compile your model

model.compile(loss='binary_crossentropy', optimizer='sgd', metrics=['accuracy'])

Display a summary of your model

model.summary()

That was fast! Let's use this classification model to detect fake dollar bills!

‌
‌
‌

Course Notes: Introduction to Deep Learning with Keras

.mfe-app-workspace-kj242g{position:absolute;top:-8px;}.mfe-app-workspace-11ezf91{display:inline-block;}.mfe-app-workspace-11ezf91:hover .Anchor__copyLink{visibility:visible;}Import the Sequential model and Dense layer

Create a Sequential model

Add an input layer and a hidden layer with 10 neurons

Add a 1-neuron output layer

Summarise your model

Instantiate a new Sequential model

Add a Dense layer with five neurons and three inputs

Add a final Dense layer with one neuron and no activation

Summarize your model

Instantiate a Sequential model

Build the input and hidden layer

Add the ouput layer

Instantiate a Sequential model

Add a Dense layer with 50 neurons and an input of 1 neuron

Add two Dense layers with 50 neurons and relu activation

End your model with a Dense layer and no activation

Instantiate a Sequential model

Add a Dense layer with 50 neurons and an input of 1 neuron

Add two Dense layers with 50 neurons and relu activation

End your model with a Dense layer and no activation

Compile your model

Fit your model on your data for 30 epochs

Evaluate your model

Predict the twenty minutes orbit

Plot the twenty minute orbit

Predict the eighty minute orbit

Plot the eighty minute orbit

Import seaborn

Use pairplot and set the hue to be our class column

Show the plot

Describe the data

Count the number of observations per class

Import the sequential model and dense layer

Create a sequential model

Add a dense layer

Compile your model

Display a summary of your model

Import the Sequential model and Dense layer