Categories: AI/ML Research

A Gentle Introduction to Multi-Head Attention and Grouped-Query Attention

This post is divided into three parts; they are: • Why Attention is Needed • The Attention Operation • Multi-Head Attention (MHA) • Grouped-Query Attention (GQA) and Multi-Query Attention (MQA) Traditional neural networks struggle with long-range dependencies in sequences.

Advanced Techniques to Build Your RAG System

This post is divided into three parts; they are: • Query Expansion and Reformulation • Hybrid Retrieval: Dense and Sparse Methods • Multi-Stage Retrieval with Re-ranking One of the challenges in RAG systems is that the user's query might not match the terminology used in the knowledge base.

April 27, 2025

In "AI/ML Research"

5IDER: Unified Query Rewriting for Steering, Intent Carryover, Disfluencies, Entity Carryover and Repair

*=Equal Contributors Providing voice assistants the ability to navigate multi-turn conversations is a challenging problem. Handling multi-turn interactions requires the system to understand various conversational use-cases, such as steering, intent carryover, disfluencies, entity carryover, and repair. The complexity of this problem is compounded by the fact that these use-cases mix…

June 29, 2023

In "FAANG"