Categories: AI/ML Research

Preparing Data for BERT Training

This article is divided into four parts; they are: • Preparing Documents • Creating Sentence Pairs from Document • Masking Tokens • Saving the Training Data for Reuse Unlike decoder-only models, BERT’s pretraining is more complex.

Pretraining a Llama Model on Your Local GPU

This article is divided into three parts; they are: • Training a Tokenizer with Special Tokens • Preparing the Training Data • Running the Pretraining The model architecture you will use is the same as the one created in the

December 22, 2025

In "AI/ML Research"

Train Your Large Model on Multiple GPUs with Fully Sharded Data Parallelism

This article is divided into five parts; they are: • Introduction to Fully Sharded Data Parallel • Preparing Model for FSDP Training • Training Loop with FSDP • Fine-Tuning FSDP Behavior • Checkpointing FSDP Models Sharding is a term originally used in database management systems, where it refers to dividing…

December 31, 2025

In "AI/ML Research"

Building a Plain Seq2Seq Model for Language Translation

This post is divided into five parts; they are: • Preparing the Dataset for Training • Implementing the Seq2Seq Model with LSTM • Training the Seq2Seq Model • Using the Seq2Seq Model • Improving the Seq2Seq Model In

July 23, 2025

In "AI/ML Research"

AI Generated Robotic Content