Most popular programs
Trending now
"Deep Learning" gives an overview of the basic types of neural networks and how to train them.
Starting with Multi-Layer Perceptrons (MLPs), the course covers the most important training tricks, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), as well as Transformers, and gives an outlook on effective training and efficient fine-tuning of large multi-modal models.
After a course session ends, it will be archived.
Artificial neural networks form the foundation of modern AI systems. “Deep Learning” offers participants a comprehensive introduction to the core principles and fundamental building blocks used in today’s neural networks. The course covers the most important types of neural networks, like MLPs, CNNs, RNNs, and Transformers, as well as practical techniques for efficient training and the reuse large pre-trained models.
Throughout the course, students will gain a robust understanding of the general training process and key differences between different network types, as well as practical knowledge through hands-on programming exercises.
By the end of the course, students will be equipped with the knowledge and skills to understand, train, and apply deep neural networks to a variety of problems, laying a strong foundation for advanced exploration of the field.
You will need an introductory knowledge of linear algebra and machine learning. For the programming exercises, you will need a working knowledge of Python.
Week 1 – Introduction to Deep Learning
In the first week, we will give an overview of the history of deep learning, covering important milestones and factors of rapid progress that the field has experienced in recent years. In addition to this overview, you will learn about the essentials of neural network training: multi-layer perceptrons, activation functions, error functions, and the backpropagation algorithm.
Week 2 – Practical Deep Learning
In week two, we will have a closer look at some of the important practical aspects of training neural networks. In particular, we will cover data preprocessing and weight initialization, adaptive first-order optimization methods, and some of the typical tricks that enable reliable training and good task performance.
Week 3 - Convolutional Neural Networks
Week 3 introduces Convolutional Neural Networks (CNNs) as a practical way to process image data with neural networks. We will motivate CNNs as parameter-efficient learnable image filters, present typical CNN operators like pooling layers and give an overview of tried and proven architectures for image classification. Additionally, we introduce the encoder-decoder architecture, which is not only a popular pattern to tackle dense image processing tasks, but also a generally useful way to map inputs to outputs.
Week 4 - Recurrent Neural Networks
In week 4, we will present network architectures for natural language processing. In particular, we will introduce Recurrent Neural Networks (RNNs) for sequence processing and analyse their training behaviour, uncovering reasons for unstable training. We will also study Long Short-Term Memory (LSTM) networks, which exhibit more stable training behaviour, and the attention mechanism as a way to improve network architectures for sequence-to-sequence processing.
Week 5 - Transformers
In week 5, we will introduce the fundamental architecture behind modern deep neural networks, the Transformer. Starting from a learnable key-value storage mechanism as a motivating example, we present the basic building blocks of the architecture, covering self- and cross-attention, positional encodings, Transformer Encoder and Decoder, as well as three examples of powerful applications of this versatile architecture.
Week 6 - Large-Scale Learning
In the final week, we will cover large-scale learning systems. Motivated from empirical findings on the benefits of scaling to large models and large amounts of data (so-called scaling laws), we present the typical pretraining and fine-tuning paradigm. Additionally, we show how these large-scale models enable easy construction of multi-modal models from existing architectural building blocks.
Who can take this course?
Unfortunately, learners residing in one or more of the following countries or regions will not be able to register for this course: Iran, Cuba and the Crimea region of Ukraine. While edX has sought licenses from the U.S. Office of Foreign Assets Control (OFAC) to offer our courses to learners in these countries and regions, the licenses we have received are not broad enough to allow us to offer this course in all locations. edX truly regrets that U.S. sanctions prevent us from offering all of our courses to everyone, no matter where they live.
Who can take this course?
Unfortunately, learners residing in one or more of the following countries or regions will not be able to register for this course: Iran, Cuba and the Crimea region of Ukraine. While edX has sought licenses from the U.S. Office of Foreign Assets Control (OFAC) to offer our courses to learners in these countries and regions, the licenses we have received are not broad enough to allow us to offer this course in all locations. edX truly regrets that U.S. sanctions prevent us from offering all of our courses to everyone, no matter where they live.
