Category: AI/ML Research

Embeddings — vector-based numerical representations of typically unstructured data like text — have been primarily popularized in the field of natural language processing (NLP).

Large language models like LLaMA, Mistral, and Qwen have billions of parameters that demand a lot of memory and compute power.

Editor’s note: This article is a part of our series on visualizing the foundations of machine learning.

Most ChatGPT users don’t know this, but when the model searches the web for current information or runs Python code to analyze data, it’s using tool calling.

This article is divided into four parts; they are: • The Reason for Fine-tuning a Model • Dataset for Fine-tuning • Fine-tuning Procedure • Other Fine-Tuning Techniques Once you train your decoder-only transformer model, you have a text generator.

The agentic AI field is moving from experimental prototypes to production-ready autonomous systems.

Editor’s note: This article is a part of our series on visualizing the foundations of machine learning.

This article is divided into five parts; they are: • An Example of Tensor Parallelism • Setting Up Tensor Parallelism • Preparing Model for Tensor Parallelism • Train a Model with Tensor Parallelism • Combining Tensor Parallelism with FSDP Tensor parallelism originated from the Megatron-LM paper.

This article is divided into five parts; they are: • Introduction to Fully Sharded Data Parallel • Preparing Model for FSDP Training • Training Loop with FSDP • Fine-Tuning FSDP Behavior • Checkpointing FSDP Models Sharding is a term originally used in database management systems, where it refers to dividing a database into smaller units, …

Read more “Train Your Large Model on Multiple GPUs with Fully Sharded Data Parallelism”

If you’ve built chatbots or worked with language models, you’re already familiar with how AI systems handle memory within a single conversation.