Retrieval vs. Memory in Agentic AI Systems
In this article, you will learn the conceptual and practical differences between retrieval and memory in agentic AI systems, and how to combine both effectively….
7 Async Patterns for Running Agents Concurrently in Python
In this article, you will learn seven async patterns for running AI agents concurrently in Python, what each pattern is suited for, and the production-level…
Prompt Caching vs. Fine-Tuning: A Cost and Latency Decision Framework
In this article, you will learn how prompt caching and fine-tuning differ as strategies for reducing cost and latency in agentic AI systems, and how…
Identifying Token Costs Hiding in Your Agentic Loop
But cutting your runtime token burn is just the first problem.
Designing AI Agents That Can Self-Correct
With the vocabulary and the failure modes in place, here’s the build.
7 Chunking Strategies That Decide Whether Your RAG Works
Day 100 in production isn’t really about chunking strategies anymore.
Measuring Performance of Transformer Inference
This chapter is divided into eight parts; they are: • Metrics for LLM Inference • Measuring a Single Request • Warmup and Synchronization • Measuring GPU Work with CUDA Events • Measuring Memory Usage • Measuring Concurrent Requests • Multiple GPUs and Multiple Machines • Cost per Token The most common inference metrics are: • …
Static vs. Dynamic vs. Continuous Batching in LLM Inference
In this article, you will learn how static, dynamic, and continuous batching work in LLM inference, and why the differences between them matter at production…
Using a Transformer Model: From Training to Inference
This chapter is divided into four parts; they are: • Autoregressive Generation • Prefill and Decode • A Simple KV Cache • Memory Usage of the KV Cache A decoder-only transformer model predicts the next token from the tokens that came before it.
Decoding Strategies and Output Control
This chapter is divided into nine parts; they are: • Reading Logits from a Model • Greedy Decoding • Temperature Sampling • Top-$k$ Sampling • Nucleus Sampling • Repetition Penalties • Beam Search • Stop Conditions • Structured Output Constraints The model returns a vector of logits for every position in the input sequence.