Tag: AI/ML Techniques

Artificial intelligence (AI) is an umbrella computer science discipline focused on building software systems capable of mimicking human or animal intelligence capabilities to solve a task.

The intersection of traditional machine learning and modern representation learning is opening up new possibilities.

This post is divided into three parts; they are: • Low-Rank Approximation of Matrices • Multi-head Latent Attention (MLA) • PyTorch Implementation Multi-Head Attention (MHA) and Grouped-Query Attention (GQA) are the attention mechanisms used in almost all transformer models.

Pandas DataFrames are powerful and versatile data manipulation and analysis tools.

Ever felt like trying to find a needle in a haystack? That’s part of the process of building and optimizing machine learning models, particularly complex ones like ensembles and neural networks, where several hyperparameters need to be manually set by us before training them.

This post is divided into three parts; they are: • Why Attention is Needed • The Attention Operation • Multi-Head Attention (MHA) • Grouped-Query Attention (GQA) and Multi-Query Attention (MQA) Traditional neural networks struggle with long-range dependencies in sequences.

MLOps, or machine learning operations, is all about managing the end-to-end process of building, training, deploying, and maintaining machine learning models.

If you’ve been using large language models like GPT-4 or Claude, you’ve probably wondered how they can write actually usable code, explain complex topics, or even help you debug your morning coffee routine (just kidding!).

This post is divided into three parts; they are: • Interpolation and Extrapolation in Sinusoidal Encodings and RoPE • Interpolation in Learned Encodings • YaRN for Larger Context Window Sinusoidal encodings excel at extrapolation due to their use of continuous functions: $$ begin{aligned} PE(p, 2i) &= sinleft(frac{p}{10000^{2i/d}}right) \ PE(p, 2i+1) &= cosleft(frac{p}{10000^{2i/d}}right) end{aligned} $$ You …

Read more “Interpolation in Positional Encodings and Using YaRN for Larger Context Window”