RAG Systems Course
Why RAG Latency Is a Prefill Problem, Not a Retrieval Problem
Part 13: Why prefill dominates RAG latency, and how to reuse KV caches that share no prefix, with implementations
Akshay Pachaar
90 posts published
Reinforcement Learning Course
How do AI teams use RL in production?
RL Part 13: An exploration of real-world RL case studies.
Reinforcement Learning Course
Agentic RL: Environments, Trajectories, and the Training
RL Part 12: From a single judged group to a full multi-step training loop with ART and RULER.
Reinforcement Learning Course
The Reward Signal Problem for Agents
RL Part 11: From verifiable rewards to LLM-as-a-judge.
Reinforcement Learning Course
Verifiable Rewards and GRPO
RL Part 10: Dropping two models from the four-model pipeline, and building rewards you can trust.
Reinforcement Learning Course
RLHF: Aligning Language Models with Human Feedback
Part 9: From human preferences to a trained reward signal, and the four-model PPO pipeline.
Reinforcement Learning Course
Proximal Policy Optimization
RL Part 8: Trust regions, the clipped surrogate, and the workhorse of modern RL.
Reinforcement Learning Course
Policy Gradients: REINFORCE and Actor-Critic
RL Part 7: Learning the policy directly, from REINFORCE to actor-critic.
Reinforcement Learning Course
Introduction to Deep RL and DQN
RL Part 6: From linear features to neural networks, and the engineering choices that makes deep value-based RL possible.
Reinforcement Learning Course
Function Approximation
RL Part 5: From tables to parameterized value functions.
Reinforcement Learning Course
Model-Free Learning
RL Part 4: Learning value functions and policies without a model. Monte Carlo methods, TD(0), SARSA, Q-learning, and the bias-variance bridge between them.
Reinforcement Learning Course
Bellman Equations and Dynamic Programming
RL Part 3: Bellman expectation and optimality equations, policy iteration, value iteration, and why dynamic programming needs a model.
Reinforcement Learning Course
Markov Decision Processes and Value Functions
RL Part 2: Markov decision processes, returns, policies, and value functions.
Reinforcement Learning Course
Foundations of Reinforcement Learning
RL Part 1: Agents, environments, rewards, and why RL is different from supervised learning.
Classical ML and Deep Learning
Diffusion LLMs from the Ground Up: Training, Inference, and Practical Engineering
Diffusion LLMs Part 2: How dLLMs scale to 100B parameters, the inference stack that makes them fast, hands-on code, and when to actually use them.
Classical ML and Deep Learning
Diffusion LLMs from the Ground Up: Theory, Math, and Why They Work
Diffusion LLMs Part 1: Understanding how diffusion language models work from first principles, the math behind masked diffusion, and why they represent a fundamentally different approach to text generation.
MLOps/LLMOps Course
MLOps and LLMOps: Case Studies
An exploration of real-world MLOps and LLMOps case studies, examining the importance of reliable ML and AI engineering and their significance for business outcomes.
MLOps/LLMOps Course
Concepts of LLM Serving
LLMOps Part 14: An overview of the fundamentals of LLM serving, including API-based access, inference with vLLM, and practical decisions.
MLOps/LLMOps Course
LLM Inference and Optimization: Fundamentals, Bottlenecks, and Techniques
LLMOps Part 13: Exploring the mechanics of LLM inference, from prefill and decode phases to KV caching, batching, and optimization techniques that improve latency and throughput.
MLOps/LLMOps Course
LLM Fine-tuning: Techniques for Adapting Language Models
LLMOps Part 12: Understanding LLM fine-tuning, parameter-efficient methods like LoRA and QLoRA, and alignment techniques such as RLHF, DPO, and GRPO.
MLOps/LLMOps Course
Evaluation: Multi-turn Conversations, Tool Use, Tracing, and Red Teaming
LLMOps Part 11: Understanding evaluation of conversational LLM systems, tool evaluations, tracing with Langfuse, and automated red teaming.
MLOps/LLMOps Course
Evaluation: Model Benchmarks and LLM Application Assessment
LLMOps Part 10: Understanding model benchmarks, LLM application evaluation, and tooling.
MLOps/LLMOps Course
Evaluation: Fundamentals
LLMOps Part 9: A foundational guide to the evaluation of LLM applications, covering challenges and a practical taxonomy of evaluation methods.
MLOps/LLMOps Course
Context Engineering: Memory and Temporal Context
LLMOps Part 8: A concise overview of memory, dynamic and temporal context in LLM systems, covering short and long-term memory, dynamic context injection, and some of the common context failure modes in agentic applications.
MLOps/LLMOps Course
Context Engineering: An Introduction to the Information Environment for LLMs
LLMOps Part 7: A conceptual overview of context engineering, covering context types, context construction principles, and retrieval-centric techniques for building high-signal inputs.
MLOps/LLMOps Course
Context Engineering: Prompt Management, Defense, and Control
LLMOps Part 6: Exploring prompt versioning, defensive prompting, and techniques such as verbalized sampling, role prompting and more.
MLOps/LLMOps Course
Context Engineering: Foundations, Categories, and Techniques of Prompt Engineering
LLMOps Part 5: An introduction to prompt engineering (a subset of context engineering), covering prompt types, the prompt development workflow, and key techniques in the field.
MLOps/LLMOps Course
Building Blocks of LLMs: Decoding, Generation Parameters, and the LLM Application Lifecycle
LLMOps Part 4: An exploration of key decoding strategies, sampling parameters, and the general lifecycle of LLM-based applications.
MLOps/LLMOps Course
Building Blocks of LLMs: Attention, Architectural Designs and Training
LLMOps Part 3: A focused look at the core ideas behind attention mechanism, transformer and mixture-of-experts architectures, and model pretraining and fine-tuning.
MLOps/LLMOps Course
Building Blocks of LLMs: Tokenization and Embeddings
LLMOps Part 2: A detailed walkthrough of tokenization, embeddings, and positional representations, building the foundational translation layer that enables LLMs to process and reason over text.
