AI ENGINEERING
The Mandate for Leadership in AI Engineering
Over the next 12 to 24 months, the differentiator among engineers will shift from mastery of programming languages like Rust, Go, or Python, or the volume of code produced, to the...
From the archive
2025
Everything published in 2025.
LARGE LANGUAGE MODELS
LLM Interview Questions
Language models learn distributions over tokens; useful applications also depend on sound evaluation, efficient training and inference, and reliable access to evidence. These...
LARGE LANGUAGE MODELS
LLM Training Epoch
As large language models (LLMs) scale up, researchers have begun to notice a growing imbalance between model size and the availability of high-quality training tokens. The...
LARGE LANGUAGE MODELS
vllm throughput
vLLM pre-allocates a block of GPU memory for KV-cache based on the --gpu_memory_utilization parameter. By increasing this parameter, you free up more contiguous space for KV...
AI ENGINEERING
LangGraph Reflection
Reflection is related to agent self-improvement or reasoning feedback loops.
AI ENGINEERING
LangGraph Sample Project
A LangGraph sample project with separate research, analysis, and supervisor services, REST communication, and retry handling.
AI ENGINEERING
LangChain/LangGraph Q&A
LangGraph’s core philosophy is to orchestrate multi-step, stateful agent workflows using a directed graph (DAG or DG) structure. Each step is modeled as a node, while data and...
LARGE LANGUAGE MODELS
Training LLM From Zero
The goal of this project is to design, implement, and train a small-scale Large Language Model (LLM) from scratch, progressing through the full training lifecycle:
AI ENGINEERING
FastMCP MCP Server Hub
Currently, our different projects are using various MCP servers. To streamline and unify the process, we plan to implement a HUB MCP server that can handle multiple MCP Servers and...
AI ENGINEERING
LangChain Retry Logic
LLM call is not stable and may fail due to network issues or other reasons, therefore, retry logic is necessary.
AI ENGINEERING
MCP Transports
Compare stdio, SSE, and Streamable HTTP transports for MCP, with Python examples for configuring servers and clients.
AI ENGINEERING
Text to SQL (Smolagents)
A smolagents text-to-SQL example with query execution logs, answers from a receipts database, and tracing with Phoenix.
AI ENGINEERING
MCP Server & Client (SSE)
Beside the vscode extension hosts like (cline, continue), you can write your own client to connect to the MCP server. The client should be able to send and receive messages using...
AI ENGINEERING
RAG-Reranking
Retrieval-Augmented Generation (RAG) is a powerful approach that combines retrieval and generation to produce high-quality responses. However, the quality of the final response can...
LARGE LANGUAGE MODELS
Ollama Import GGUF Models
If you're looking to experiment with various models easily, importing GGUF might be your go-to method. Here's how it works:
AI ENGINEERING
GenAI Projects
A collection of generative AI resources and project notes on schema mapping, feature engineering, PDF extraction, and agent frameworks.
AI ENGINEERING
Crawling the Web with LLM
Web crawling frameworks and a ScrapeGraphAI example that uses local Ollama models to extract structured information from a webpage.
AI ENGINEERING
LangGraph VS AutoGen
Compare LangGraph and AutoGen across workflow structure, state management, customization, and use cases for LLM applications.
AI ENGINEERING
Autogen Intro and RAG Workflow
AutoGen is a framework for creating multi-agent AI applications that can act autonomously or work alongside humans.
LARGE LANGUAGE MODELS
Local LLM Setup
This guide will walk you through setting up a local language model (LLM) using Ollama. Ollama is an open-source, lightweight language model that can be run locally on your machine...