Google DeepMind: Design And Train Neural Networks

In this module, you will explore the concept of generalization. You will begin by considering the concepts of signal and noise, and the role they play in model performance. You will then investigate how these concepts relate to overfitting and underfitting. Finally, you will also consider the course learning objectives and how to learn most effectively on this course.

In this module, you will be formally introduced to generalization and the trade-off between bias and variance. You will explore training and test splits and the use of loss curves as a diagnostic tool. You will then move on to explore anticipation as a key research skill through a research case study. While you cannot predict every impact your technologies may have, lessons from places where the risks of AI have already been revealed can help you anticipate how similar challenges might emerge in your region.

In this module, you will investigate the anatomy of neural networks, starting with single-neuron neural networks before moving on to the multilayer perceptron (MLP). You will learn why neural networks consist of multiple layers and how this allows them to do complex classification problems. Finally, you will implement an MLP from scratch and see how it can be applied to the task of predicting the next token.

In this module, you will learn about different settings of the training process called hyperparameters. You will undertake hyperparameter tuning and explore methods to mitigate overfitting including the use of a validation dataset. Moreover, you will consider alignment and safety to ensure that AI models act in ways consistent with human values, ethical principles, and intended goals. This will help you to reduce the risks of undesirable, unintended, or biased outcomes and make AI more trustworthy and beneficial for society.

In this module, you will discover how gradients minimize error when training artificial neural networks. You will investigate gradient descent and compute the gradient using the chain rule. You will then explore how automatic differentiation works in the deep learning library JAX by applying gradient updates to a single-layer neural network model.

In this module, you will examine a fundamental concept of training neural networks - backpropagation. You will then move on to consider stochastic gradient descent and more sophisticated methods for training neural networks, such as Adam (Adaptive Moment Estimation). Finally, you will train a neural network using optimizers implemented in the Python deep learning framework Keras.

In this module, you will focus on the responsible innovation principle of anticipation. Anticipation is the forward-thinking practice of exploring the potential impacts of new technologies. It does not attempt to predict the future, but rather to understand a range of plausible outcomes. You will draw on this skill to create an impact statement card that involves ethically reflecting on your work to increase the benefits of your project and reduce any potential risks.

In this module, you will have the opportunity to consult additional resources and further reading to investigate the topics you have covered in more detail. Finally, you will consider your next steps and how you can build on what you have learned in the course.