Reinforcement Learning Course
Introduction to Deep RL and DQN
RL Part 6: From linear features to neural networks, and the engineering choices that makes deep value-based RL possible.
Akshay Pachaar
82 posts published
Reinforcement Learning Course
Function Approximation
RL Part 5: From tables to parameterized value functions.
Reinforcement Learning Course
Model-Free Learning
RL Part 4: Learning value functions and policies without a model. Monte Carlo methods, TD(0), SARSA, Q-learning, and the bias-variance bridge between them.
Reinforcement Learning Course
Bellman Equations and Dynamic Programming
RL Part 3: Bellman expectation and optimality equations, policy iteration, value iteration, and why dynamic programming needs a model.
Reinforcement Learning Course
Markov Decision Processes and Value Functions
RL Part 2: Markov decision processes, returns, policies, and value functions.
Reinforcement Learning Course
Foundations of Reinforcement Learning
RL Part 1: Agents, environments, rewards, and why RL is different from supervised learning.
Classical ML and Deep Learning
Diffusion LLMs from the Ground Up: Training, Inference, and Practical Engineering
Diffusion LLMs Part 2: How dLLMs scale to 100B parameters, the inference stack that makes them fast, hands-on code, and when to actually use them.
Classical ML and Deep Learning
Diffusion LLMs from the Ground Up: Theory, Math, and Why They Work
Diffusion LLMs Part 1: Understanding how diffusion language models work from first principles, the math behind masked diffusion, and why they represent a fundamentally different approach to text generation.
MLOps/LLMOps Course
MLOps and LLMOps: Case Studies
An exploration of real-world MLOps and LLMOps case studies, examining the importance of reliable ML and AI engineering and their significance for business outcomes.
MLOps/LLMOps Course
Concepts of LLM Serving
LLMOps Part 14: An overview of the fundamentals of LLM serving, including API-based access, inference with vLLM, and practical decisions.
MLOps/LLMOps Course
LLM Inference and Optimization: Fundamentals, Bottlenecks, and Techniques
LLMOps Part 13: Exploring the mechanics of LLM inference, from prefill and decode phases to KV caching, batching, and optimization techniques that improve latency and throughput.
MLOps/LLMOps Course
LLM Fine-tuning: Techniques for Adapting Language Models
LLMOps Part 12: Understanding LLM fine-tuning, parameter-efficient methods like LoRA and QLoRA, and alignment techniques such as RLHF, DPO, and GRPO.
MLOps/LLMOps Course
Evaluation: Multi-turn Conversations, Tool Use, Tracing, and Red Teaming
LLMOps Part 11: Understanding evaluation of conversational LLM systems, tool evaluations, tracing with Langfuse, and automated red teaming.
MLOps/LLMOps Course
Evaluation: Model Benchmarks and LLM Application Assessment
LLMOps Part 10: Understanding model benchmarks, LLM application evaluation, and tooling.
MLOps/LLMOps Course
Evaluation: Fundamentals
LLMOps Part 9: A foundational guide to the evaluation of LLM applications, covering challenges and a practical taxonomy of evaluation methods.
MLOps/LLMOps Course
Context Engineering: Memory and Temporal Context
LLMOps Part 8: A concise overview of memory, dynamic and temporal context in LLM systems, covering short and long-term memory, dynamic context injection, and some of the common context failure modes in agentic applications.
MLOps/LLMOps Course
Context Engineering: An Introduction to the Information Environment for LLMs
LLMOps Part 7: A conceptual overview of context engineering, covering context types, context construction principles, and retrieval-centric techniques for building high-signal inputs.
MLOps/LLMOps Course
Context Engineering: Prompt Management, Defense, and Control
LLMOps Part 6: Exploring prompt versioning, defensive prompting, and techniques such as verbalized sampling, role prompting and more.
MLOps/LLMOps Course
Context Engineering: Foundations, Categories, and Techniques of Prompt Engineering
LLMOps Part 5: An introduction to prompt engineering (a subset of context engineering), covering prompt types, the prompt development workflow, and key techniques in the field.
MLOps/LLMOps Course
Building Blocks of LLMs: Decoding, Generation Parameters, and the LLM Application Lifecycle
LLMOps Part 4: An exploration of key decoding strategies, sampling parameters, and the general lifecycle of LLM-based applications.
MLOps/LLMOps Course
Building Blocks of LLMs: Attention, Architectural Designs and Training
LLMOps Part 3: A focused look at the core ideas behind attention mechanism, transformer and mixture-of-experts architectures, and model pretraining and fine-tuning.
MLOps/LLMOps Course
Building Blocks of LLMs: Tokenization and Embeddings
LLMOps Part 2: A detailed walkthrough of tokenization, embeddings, and positional representations, building the foundational translation layer that enables LLMs to process and reason over text.
AI Agents Course
A Practical Deep Dive Into Memory Optimization for Agentic Systems (Part C)
AI Agents Crash Course—Part 17 (with implementation).
AI Agents Course
A Practical Deep Dive Into Memory Optimization for Agentic Systems (Part B)
AI Agents Crash Course—Part 16 (with implementation).
MLOps/LLMOps Course
Foundations of AI Engineering and LLMs
LLMOps Part 1: An overview of AI engineering and LLMOps, and the core dimensions that define modern AI systems.
LLM and Fine-tuning
A Practical Guide to Integrate Evaluation and Observability into LLM Apps
A comprehensive guide to Opik, an open-source LLM evaluation and observability framework.
MLOps/LLMOps Course
CI/CD Workflows
MLOps Part 18: A hands-on guide to CI/CD in MLOps with DVC, Docker, GitHub Actions, and GitOps-based Kubernetes delivery on Amazon EKS.
MLOps/LLMOps Course
Monitoring and Observability: Practical Tooling with Evidently, Prometheus, and Grafana
MLOps Part 17: ML monitoring in practice with Evidently, Prometheus and Grafana, stitched into a FastAPI inference service with drift reports, metrics scraping, and dashboards.
AI Agents Course
A Practical Deep Dive Into Memory Optimization for Agentic Systems (Part A)
AI Agents Crash Course—Part 15 (with implementation).
MLOps/LLMOps Course
Monitoring and Observability: Core Fundamentals
MLOps Part 16: A comprehensive overview of drift detection using statistical techniques, and how logging and observability keep ML systems healthy.
