Collection

LLM Inference & Serving

Understand how prompts become tokens, how attention and KV cache work, and how serving engines scale inference.

4 notes in this path

Note 01 1 of 4

Mar 7, 2026 LARGE LANGUAGE MODELS

From Prompt to Response: A Step-by-Step Walkthrough of LLM Inference

A cleaner mental model for LLM inference: prompt templating, tokenization, scheduling, prefill, KV cache, decode, and the split between the inference engine and the model.

Read

Note 02 2 of 4

Mar 15, 2026 LARGE LANGUAGE MODELS

Attention Dilution

Why longer LLM context windows can make answers worse: softmax dilution, positional bias, and attention sinks.

Read

Note 03 3 of 4

Jul 5, 2026 AI ENGINEERING

KV-Centric LLM Serving: vLLM, SGLang, and Disaggregated Attention

A practical mental model for KV cache memory, vLLM PagedAttention, SGLang RadixAttention, and prefill/decode disaggregation in modern LLM serving.

Read

Note 04 4 of 4

Jul 3, 2026 AI ENGINEERING

Reproducing vLLM and LMCache KV Cache Reuse on a CPU-Only MacBook

A reproducible CPU-only vLLM and LMCache workspace for understanding KV cache reuse, transfer modes, and local inference debugging on Apple Silicon.

Read