Static vs. Dynamic vs. Continuous Batching in LLM Inference
In this article, you will learn how static, dynamic, and continuous batching work in LLM inference, and why the differences between them matter at production…
Using a Transformer Model: From Training to Inference
This chapter is divided into four parts; they are: • Autoregressive Generation • Prefill and Decode • A Simple KV Cache • Memory Usage of the KV Cache A decoder-only transformer model predicts the next token from the tokens that came before it.
Decoding Strategies and Output Control
This chapter is divided into nine parts; they are: • Reading Logits from a Model • Greedy Decoding • Temperature Sampling • Top-$k$ Sampling • Nucleus Sampling • Repetition Penalties • Beam Search • Stop Conditions • Structured Output Constraints The model returns a vector of logits for every position in the input sequence.
The End-to-End Agentic AI Pipeline
In this article, you will learn the seven architectural components that separate a production-grade agentic AI system from a demo script, and how each one…
Ollama vs. LM Studio vs. llama.cpp: Which Local AI Runtime Should You Use in 2026?
In this article, you will learn how Ollama, LM Studio, and llama.cpp differ across the dimensions that matter most to practitioners, and how to choose…
From CUDA to MLX: How K-Search Brings Decades of Kernel Expertise to Apple Silicon
Figure 1: CUDA-to-MLX optimization translation map. CUDA optimization knowledge can be translated into architecture-native MLX strategies rather than copied instruction-for-instruction. We face a new epoch in computing. Hardware is changing rapidly — not just faster GPUs, but a growing range of chips from different vendors, each with its own architecture and often tailored to specific …
Read more “From CUDA to MLX: How K-Search Brings Decades of Kernel Expertise to Apple Silicon”
5 Architectural Patterns for Persistent Memory and State in AI Agents
Memory & State For AI Agents Building an AI agent can be tricky. Keeping it on track over a six-month deployment is incredibly hard. LLMs…
Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction
Overview of ABBEL compared to traditional recursive summarization. Beliefs replace the full interaction history as the agent’s working context, and belief grading improves performance by supervising the contents of each belief state.. As task horizons grow, LLM contexts can’t scale forever. Self-summarization enables concise, interpretable contexts, but at a significant performance cost, especially for human …
Read more “Teaching LLMs to Update Beliefs for Efficient Long-Horizon Interaction”
Stateful vs. Stateless Agent Design: Tradeoffs for Scalable Agentic Systems
In this article, you will learn how an agent’s approach to managing state — stateless or stateful — shapes both its implementation and the deployment…
An Introduction to Loop Engineering
It’s tempting to treat loop engineering as something invented in a single week in June, but the mechanics behind it are closer to five years old, and knowing the lineage is what separates a real understanding of the idea from just repeating the trend piece.