Get in Touch

Course Outline

Introduction to Large Language Models (LLMs)

  • Overview of LLMs
  • Definition and significance
  • Applications in AI today

Transformer Architecture

  • Understanding what a transformer is and its operational mechanism
  • Key components and features
  • Embedding and positional encoding
  • Multi-head attention
  • Feed-forward neural network
  • Normalization and residual connections

Transformer Models

  • Self-attention mechanism
  • Encoder-decoder architecture
  • Positional embeddings
  • BERT (Bidirectional Encoder Representations from Transformers)
  • GPT (Generative Pretrained Transformer)

Performance Optimization and Pitfalls

  • Context length
  • Mamba and state-space models
  • Flash attention
  • Sparse transformers
  • Vision transformers
  • The importance of quantization

Enhancing Transformers

  • Retrieval augmented text generation
  • Mixture of models
  • Tree of thoughts

Fine-Tuning

  • Theory of low-rank adaptation
  • Fine-Tuning with QLora

Scaling Laws and Optimization in LLMs

  • Significance of scaling laws for LLMs
  • Scaling of data and model size
  • Computational scaling
  • Parameter efficiency scaling

Optimization

  • Interplay between model size, data volume, compute budget, and inference requirements
  • Strategies for optimizing LLM performance and efficiency
  • Best practices and tools for training and fine-tuning LLMs

Training and Fine-Tuning LLMs

  • Procedural steps and challenges of training LLMs from scratch
  • Data acquisition and maintenance
  • Requirements for large-scale data, CPU, and memory
  • Optimization challenges
  • Overview of open-source LLM landscape

Fundamentals of Reinforcement Learning (RL)

  • Introduction to Reinforcement Learning
  • Learning through positive reinforcement
  • Definition and core concepts
  • Markov Decision Process (MDP)
  • Dynamic programming
  • Monte Carlo methods
  • Temporal Difference Learning

Deep Reinforcement Learning

  • Deep Q-Networks (DQN)
  • Proximal Policy Optimization (PPO)
  • Core elements of Reinforcement Learning

Integration of LLMs and Reinforcement Learning

  • Combining LLMs with Reinforcement Learning
  • Application of RL in LLMs
  • Reinforcement Learning with Human Feedback (RLHF)
  • Alternatives to RLHF

Case Studies and Applications

  • Real-world applications
  • Success stories and challenges

Advanced Topics

  • Advanced techniques
  • Advanced optimization methods
  • Cutting-edge research and developments

Summary and Next Steps

Requirements

  • Foundational knowledge of Machine Learning

Target Audience

  • Data scientists
  • Software engineers
 21 Hours

Number of participants


Price per participant

Upcoming Courses

Related Categories