Categories: AI/ML Research

The Journey of a Token: What Really Happens Inside a Transformer

Large language models (LLMs) are based on the transformer architecture, a complex deep neural network whose input is a sequence of token embeddings.

Using a Transformer Model: From Training to Inference

This chapter is divided into four parts; they are: • Autoregressive Generation • Prefill and Decode • A Simple KV Cache • Memory Usage of the KV Cache A decoder-only transformer model predicts the next token from the tokens that came before it.

August 4, 2026

In "AI/ML Research"

Measuring Performance of Transformer Inference

This chapter is divided into eight parts; they are: • Metrics for LLM Inference • Measuring a Single Request • Warmup and Synchronization • Measuring GPU Work with CUDA Events • Measuring Memory Usage • Measuring Concurrent Requests • Multiple GPUs and Multiple Machines • Cost per Token The most…

August 5, 2026

In "AI/ML Research"

Creating a Llama or GPT Model for Next-Token Prediction

This article is divided into three parts; they are: • Understanding the Architecture of Llama or GPT Model • Creating a Llama or GPT Model for Pretraining • Variations in the Architecture The architecture of a Llama or GPT model is simply a stack of transformer blocks.

December 9, 2025

In "AI/ML Research"

AI Generated Robotic Content