Databricks: Large Language Models: Foundation Models from the Ground Up
This course dives into the details of foundation models in large language models (LLMs). You will learn the innovations that led to the proliferation of transformer-based models, including BERT, GPT, and T5, and the key breakthroughs that led to applications such as ChatGPT. Additionally, you will gain understanding about the latest advances that continue to improve LLM functionality including Flash Attention, LoRa, AliBi, and PEFT methods.
Large Language Models: Foundation Models from the Ground Up
About this courseSkip About this course
This course dives into the details of LLM foundation models. You will learn the innovations that led to the proliferation of transformer-based architectures, from encoder models (BERT), to decoder models (GPT), to encoder-decoder models (T5). You will also learn about the recent breakthroughs that led to applications like ChatGPT. You will gain understanding about the latest advances that continue to improve LLM functionality including Flash Attention, LoRa, AliBi, and PEFT methods. The course concludes with an overview of multi-modal LLM developments to address NLP problems involving a combination of text, audio, and visual components.
At a glance
- Institution: Databricks
- Subject: Computer Science
- Level: Advanced
Intermediate-level experience with Python
Understanding of deep learning concepts and hands-on experience with PyTorch
Completing the LLM: Application through Production course is highly recommended, but not strictly required prior to taking this course.
- Language: English
- Video Transcript: English
- Associated programs:
- Professional Certificate in Large Language Models
- Associated skills: Natural Language Processing, Decision Making, Transfer Learning
What you'll learnSkip What you'll learn
- Describe the components and theory behind foundation models, including the attention mechanism, encoder and decoder architectures.
- Articulate the developments in the evolution of GPT models that were critical in the creation of popular LLMs like ChatGPT.
- Explain and implement the latest advances that improve LLM functionality, including Fast Attention, AliBi, and PEFT methods.
- Gain insights into multi-modal applications of LLMs involving a combination of text, audio, and visual elements.
Module 1 - Transformer Architecture: Attention & Transformer Fundamentals
Module 2 - Efficient Fine Tuning
Module 3 - Deployment and Hardware Considerations
Module 4 - Beyond Text-Based LLMs: Multi-Modality