Categories: AI/ML Research

Word Embeddings in Language Models

This post is divided into three parts; they are: • Understanding Word Embeddings • Using Pretrained Word Embeddings • Training Word2Vec with Gensim • Training Word2Vec with PyTorch • Embeddings in Transformer Models Word embeddings represent words as dense vectors in a continuous space, where semantically similar words are positioned close to each other.

Generating and Visualizing Context Vectors in Transformers

This post is divided into three parts; they are: • Understanding Context Vectors • Visualizing Context Vectors from Different Layers • Visualizing Attention Patterns Unlike traditional word embeddings (such as Word2Vec or GloVe), which assign a fixed vector to each word regardless of context, transformer models generate dynamic representations that…

April 15, 2025

In "AI/ML Research"

What exactly does word2vec learn?

November 8, 2025

In "AI/ML Research"

Word Embeddings for Tabular Data Feature Engineering

It would be difficult to argue that word embeddings — dense vector representations of words — have not dramatically revolutionized the field of natural language processing (NLP) by quantitatively capturing semantic relationships between words.

July 12, 2025

In "AI/ML Research"

AI Generated Robotic Content