Curriculum

A complete, hands-on curriculum for mastering large language model engineering — from GPU fundamentals to production deployment. All 64 notebooks are freely available on GitHub.

Foundations

Build the core skills and mental models needed to understand and implement modern LLM systems from first principles.

01: GPU Fundamentals

Master GPU memory management, tensor operations, and CUDA kernels. Understand hardware bottlenecks and optimization techniques from first principles.

02: Transformer Architecture

Implement transformers from scratch. Learn self-attention, multi-head mechanisms, positional encoding, and why they work. Build intuition for modern LLM design.

03: Tokenization

Understand byte-pair encoding (BPE), SentencePiece, and other tokenization strategies. Build a tokenizer from scratch and understand how vocabulary design impacts model performance.

04: Pre-training

Learn the mechanics of pre-training language models. Cover data preparation, training objectives, learning rate schedules, and common pitfalls in large-scale training runs.

Training & Adaptation

Learn how to adapt and fine-tune foundation models efficiently for specific tasks and domains.

05: Supervised Fine-tuning

Hands-on supervised fine-tuning of foundation models. Cover instruction tuning, dataset preparation, and the practical differences between full fine-tuning and parameter-efficient approaches.

06: LoRA & PEFT

Hands-on with LoRA, QLoRA, and PEFT techniques. Adapt foundation models efficiently for specific use cases without massive computational overhead.

07: Quantization

Reduce model memory footprint with INT8, INT4, and GPTQ quantization. Understand the accuracy-efficiency trade-offs and choose the right strategy for your use case.

08: Prompting & Context Engineering

Master prompt engineering, chain-of-thought reasoning, few-shot learning, and system prompt design. Learn how context structure affects model outputs and reliability.

Alignment & Safety

Explore advanced techniques for building safer, more helpful AI systems through retrieval, evaluation, and alignment.

09: RAG Pipelines

Build retrieval-augmented generation systems from basic implementations to agentic RAG architectures.

10: Evaluation & Benchmarks

Design and implement evaluation metrics, benchmarks, and testing strategies for LLM applications.

11: Alignment

Learn alignment techniques including DPO (Direct Preference Optimization) and RLHF (Reinforcement Learning from Human Feedback).

Production

Master the tools and techniques needed to deploy and operate LLM systems at scale in production environments.

12: Scaling Laws

Understand model and data scaling laws, and how to predict performance at different scales.

13: Data Engineering

Build robust data pipelines for preprocessing, validation, and efficient large-scale data handling.

14: Distributed Training

Implement distributed training techniques for scaling across multiple GPUs and nodes.

15: Serving

Deploy models at scale using vLLM, SGLang, and other serving frameworks for low-latency inference.

16: Agents & Tool Use

Build intelligent agents that can plan, reason, and use tools to accomplish complex tasks.

17: Multimodal Models

Work with multimodal architectures that process text, images, and other modalities together.

18: Mixture of Experts

Implement mixture of experts models for efficient scaling and specialized expert routing.

19: Deployment & MLOps

Manage the complete deployment lifecycle including monitoring, logging, version control, and model management.

Capstone Projects

Apply everything you have learned in 20 end-to-end projects. Build real systems — from fine-tuned domain-specific models to full RAG applications and deployed inference APIs. Each capstone project is designed to consolidate multiple modules and produce something you can showcase.

Extras

Additional resources and supplementary materials to extend your learning.

20: Library Cookbook

Practical recipes and patterns for popular libraries including LangChain, LlamaIndex, and other tools.

21: Supplements

Supplementary materials, advanced topics, and additional resources for deeper learning.