LLMOps
KV Cache Engineering for LLM Serving
12 techniques to manage KV cache in production.
LLMOps
A collection of 25 posts
LLMOps
KV vs Prefix vs Prompt vs Semantic Caching
...explained with best practices in production.
LLMOps
How a GPU Actually Works
The intuition an LLM engineer needs, without the hardware manual. By the end, techniques like quantization, speculative decoding, and continuous batching should stop looking like a list of tricks to memorize.
LLMOps
5 LLM Quantization Techniques
...explained visually.
LLMOps
6 LLM Deployment Formats in Production
...explained visually.
LLMOps
Rethinking KV Caching For Production Inference
A practitioner's guide to KV cache management in production.
LLMOps
72 Techniques to Optimize LLMs in Production
...explained with usage.
MLOps/LLMOps Course
MLOps and LLMOps: Case Studies
An exploration of real-world MLOps and LLMOps case studies, examining the importance of reliable ML and AI engineering and their significance for business outcomes.
MLOps/LLMOps Course
Concepts of LLM Serving
LLMOps Part 14: An overview of the fundamentals of LLM serving, including API-based access, inference with vLLM, and practical decisions.
MLOps/LLMOps Course
LLM Inference and Optimization: Fundamentals, Bottlenecks, and Techniques
LLMOps Part 13: Exploring the mechanics of LLM inference, from prefill and decode phases to KV caching, batching, and optimization techniques that improve latency and throughput.
MLOps/LLMOps Course
LLM Fine-tuning: Techniques for Adapting Language Models
LLMOps Part 12: Understanding LLM fine-tuning, parameter-efficient methods like LoRA and QLoRA, and alignment techniques such as RLHF, DPO, and GRPO.
MLOps/LLMOps Course
Evaluation: Model Benchmarks and LLM Application Assessment
LLMOps Part 10: Understanding model benchmarks, LLM application evaluation, and tooling.
MLOps/LLMOps Course
Evaluation: Fundamentals
LLMOps Part 9: A foundational guide to the evaluation of LLM applications, covering challenges and a practical taxonomy of evaluation methods.
MLOps/LLMOps Course
Context Engineering: Memory and Temporal Context
LLMOps Part 8: A concise overview of memory, dynamic and temporal context in LLM systems, covering short and long-term memory, dynamic context injection, and some of the common context failure modes in agentic applications.
MLOps/LLMOps Course
Context Engineering: An Introduction to the Information Environment for LLMs
LLMOps Part 7: A conceptual overview of context engineering, covering context types, context construction principles, and retrieval-centric techniques for building high-signal inputs.
MLOps/LLMOps Course
Context Engineering: Prompt Management, Defense, and Control
LLMOps Part 6: Exploring prompt versioning, defensive prompting, and techniques such as verbalized sampling, role prompting and more.
MLOps/LLMOps Course
Context Engineering: Foundations, Categories, and Techniques of Prompt Engineering
LLMOps Part 5: An introduction to prompt engineering (a subset of context engineering), covering prompt types, the prompt development workflow, and key techniques in the field.
MLOps/LLMOps Course
Building Blocks of LLMs: Decoding, Generation Parameters, and the LLM Application Lifecycle
LLMOps Part 4: An exploration of key decoding strategies, sampling parameters, and the general lifecycle of LLM-based applications.
MLOps/LLMOps Course
Building Blocks of LLMs: Tokenization and Embeddings
LLMOps Part 2: A detailed walkthrough of tokenization, embeddings, and positional representations, building the foundational translation layer that enables LLMs to process and reason over text.
MLOps/LLMOps Course
Foundations of AI Engineering and LLMs
LLMOps Part 1: An overview of AI engineering and LLMOps, and the core dimensions that define modern AI systems.
MLOps/LLMOps Course
Data and Pipeline Engineering: Data Sources, Formats, and ETL Foundations
MLOps Part 5: A detailed walkthrough of data engineering for MLOps, covering data sources, format performance trade-offs, and ETL/ELT pipelines.
MLOps/LLMOps Course
Reproducibility and Versioning in ML Systems: Weights and Biases for Reproducible ML
MLOps Part 4: A practical walkthrough of W&B-powered reproducibility.
MLOps/LLMOps Course
Reproducibility and Versioning in ML Systems: Fundamentals of Repeatable and Traceable Setups
MLOps Part 3: A practical exploration of reproducibility and versioning, covering deterministic training, data and model versioning, and experiment tracking.
MLOps/LLMOps Course
The Machine Learning System Lifecycle
MLOps Part 2: A deeper look at the ML lifecycle, plus a minimal train-to-API and containerization demo using FastAPI and Docker.
MLOps/LLMOps Course
Background and Foundations for ML in Production
MLOps Part 1: An introduction to machine learning in production, covering pitfalls, system-level concerns, and an overview of the full ML lifecycle.
