From the archive
Years of learning, building, and writing things down.
A cleaner mental model for LLM inference: prompt templating, tokenization, scheduling, prefill, KV cache, decode, and the split between the inference engine and the model.
Carry authorization through retrieval, context, caches, and citations. Define trust boundaries, revocation behavior, and operational controls.
Separate retrieval coverage, context quality, and answer support. Calculate ranking metrics and design reproducible regression gates.
Separate schema validity, citation identity, and factual support; design bounded repair, honest abstention, and streaming behavior around those distinct guarantees.
Translate questions into bounded retrieval plans, preserve metadata constraints, and verify SQL result semantics with a worked incident-table example.
Work through reciprocal rank fusion, distinguish full-text search from BM25, and evaluate reranking and corrective retrieval without treating scores as confidence.
Turn retrieved candidates into useful context by preserving qualifiers, tracking expansion provenance, budgeting tokens, and separating image retrieval from visual answering.
Understand vector geometry, pin the full encoding contract, and evaluate approximate search under real filters before migrating an embedding index.
Design document ingestion around immutable representations, idempotent jobs, and atomic publication so failed updates cannot silently replace searchable evidence.
Compare chunk boundaries on the same source, preserve qualifiers and table structure, and evaluate retrieval against stable evidence rather than arbitrary chunk IDs.
Understand how RAG connects documents to an LLM: build an index, retrieve passages, assemble context, and return an answer with traceable sources.
A system-level guide to the twelve classic RAG failure modes, the techniques now used to address them, and the limits that still remain.
How two OpenUsage projects inspired me to investigate a Codex JSONL edge case and build a local-first dashboard for AI coding agents.
Learn how to choose and use statistical tests without turning analysis into a p-value checklist—from experimental design and assumptions to effect sizes, confidence intervals, and...
How to connect VS Code Copilot Chat to local OpenAI-compatible models served by LM Studio or Unsloth Studio, including the BYOK utility-model setting that prevents agent errors.
A practical mental model for KV cache memory, vLLM PagedAttention, SGLang RadixAttention, and prefill/decode disaggregation in modern LLM serving.
A reproducible CPU-only vLLM and LMCache workspace for understanding KV cache reuse, transfer modes, and local inference debugging on Apple Silicon.
How to use one local proxy to connect Databricks Model Serving to both VS Code Copilot Custom Endpoint and GitHub Copilot CLI.
How I use a Git ref as a local message channel between Claude Code and Codex for implementation, independent review, and iterative bug fixing.
Why longer LLM context windows can make answers worse: softmax dilution, positional bias, and attention sinks.
A practical mental model for AI terminology: prompt, context, memory, agents, RAG, function calling, MCP, workflows, skills, and sub-agents.
A visual recap of my year with ChatGPT in 2025, collecting usage statistics and personalized summary cards.
Over the next 12 to 24 months, the differentiator among engineers will shift from mastery of programming languages like Rust, Go, or Python, or the volume of code produced, to the...
Language models learn distributions over tokens; useful applications also depend on sound evaluation, efficient training and inference, and reliable access to evidence. These...
As large language models (LLMs) scale up, researchers have begun to notice a growing imbalance between model size and the availability of high-quality training tokens. The...
vLLM pre-allocates a block of GPU memory for KV-cache based on the --gpu_memory_utilization parameter. By increasing this parameter, you free up more contiguous space for KV...
Reflection is related to agent self-improvement or reasoning feedback loops.
A LangGraph sample project with separate research, analysis, and supervisor services, REST communication, and retry handling.
LangGraph’s core philosophy is to orchestrate multi-step, stateful agent workflows using a directed graph (DAG or DG) structure. Each step is modeled as a node, while data and...
The goal of this project is to design, implement, and train a small-scale Large Language Model (LLM) from scratch, progressing through the full training lifecycle:
Currently, our different projects are using various MCP servers. To streamline and unify the process, we plan to implement a HUB MCP server that can handle multiple MCP Servers and...
LLM call is not stable and may fail due to network issues or other reasons, therefore, retry logic is necessary.
Compare stdio, SSE, and Streamable HTTP transports for MCP, with Python examples for configuring servers and clients.
A smolagents text-to-SQL example with query execution logs, answers from a receipts database, and tracing with Phoenix.
Beside the vscode extension hosts like (cline, continue), you can write your own client to connect to the MCP server. The client should be able to send and receive messages using...
Retrieval-Augmented Generation (RAG) is a powerful approach that combines retrieval and generation to produce high-quality responses. However, the quality of the final response can...
If you're looking to experiment with various models easily, importing GGUF might be your go-to method. Here's how it works:
A collection of generative AI resources and project notes on schema mapping, feature engineering, PDF extraction, and agent frameworks.
Web crawling frameworks and a ScrapeGraphAI example that uses local Ollama models to extract structured information from a webpage.
Compare LangGraph and AutoGen across workflow structure, state management, customization, and use cases for LLM applications.
AutoGen is a framework for creating multi-agent AI applications that can act autonomously or work alongside humans.
This guide will walk you through setting up a local language model (LLM) using Ollama. Ollama is an open-source, lightweight language model that can be run locally on your machine...
Notes on processing unstructured PDF content and a demo architecture connecting a Gradio interface to models served by Ollama.
Notes on moving from Scala to PySpark, using Python dataclasses for data models, and creating DataFrames from Row objects.
Recently I successfully deploy my python wheel to Databricks Cluster. Here are some tips if you plan to deploy pyspark.
Recently I read a Blog that introducing how to use scala framework ZIO to build an application.
Reflex is a library to build full-stack web apps in pure Python.
I have enrolled in a private Snowflake Data Science Training. Let me list what I learned from it.
Now we can add our proxy config in our client, which will resolve connection issue.
We can use internal runpy to execute different moduls in our project.
Problem: How to introduce ml-based production/features to cross-functional teams.
Set up Minikube on macOS, deploy a sample application, and submit a Spark job to a local Kubernetes cluster.
Recently I'm working in Azure to implement ETL jobs. The main tool is ADF (Azure Data Factory). This post show some solutions to resolve issue in my work.
A Scala reference for working with window functions in Spark SQL.
Spark submit options and tuning notes covering executors, memory, parallelism, and resource allocation on YARN.
Data locality is how close data is to the code processing it. There are several levels of locality based on the data’s current location. In order from closest to farthest:
Airflow DAG examples for scheduling Python tasks, configuring retries and queues, and separating task implementations.
Recently reading a blog Structured Streaming in PySpark It's implemented in Databricks platform. Then I try to implement in my local Spark. Some tricky issue happened during my...
Vanilla gradient descent, aka batch gradient descent, computes the gradient of the cost function w.r.t. to the parameters θ