Bin Zhang's Field Notes
Practical essays, implementation notes, and mental models for LLMs, infrastructure, data platforms, and the craft of building durable software.
Pinned dispatch
Update: For a deeper systems-level treatment of LLM inference, especially the interaction between request scheduling, prefill, decode, and KV-cache reuse, see arXiv:2606.24937.
Read articleFresh from the notebook
I became interested in LMCache because it sits in the part of LLM serving that feels both very practical and very under-discussed: KV cache...
Read noteI wanted one Databricks-hosted model to work in two developer surfaces:
Read noteMy previous local development workflow was simple:
Read noteAttention dilution (also called context dilution) is one of the fundamental limitations of transformer-based LLMs when dealing with long...
Read noteHow many of these terms do you actually recognize?
Read noteChatGPT Stats ChatGPT Growth ChatGPT Revenue
Read note