Categories: FAANG

Multimodal Large Language Models with Fusion Low Rank Adaptation for Device Directed Speech Detection

Although Large Language Models (LLMs) have shown promise for human-like conversations, they are primarily pre-trained on text data. Incorporating audio or video improves performance, but collecting large-scale multimodal data and pre-training multimodal LLMs is challenging. To this end, we propose a Fusion Low Rank Adaptation (FLoRA) technique that efficiently adapts a pre-trained unimodal LLM to consume new, previously unseen modalities via low rank adaptation. For device-directed speech detection, using FLoRA, the multimodal LLM achieves 22% relative reduction in equal error rate (EER) over…
AI Generated Robotic Content

Recent Posts

Combatting adversarial distillation

OpenAI’s layered approach to adversarial distillation combines model training, safeguards, detection, and enforcement to protect…

23 hours ago

7 Ways AI Agents Fail in Production (and How to Catch Each One)

Three events that look alike need separate names.

23 hours ago

Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth

We present Depth Pro 2, an approach to estimating dense metric depth for images and…

23 hours ago

New in AWS Quick: More capable flows, an updated web experience, and agents that work together

Turn one person’s process into your team’s process, get more done in the Quick web…

23 hours ago

Ultimate guide for fixing your team’s API keys

Most Gemini prototypes start the same way. You grab an API key from Google AI…

23 hours ago

Cops Are Pushing Extremely Corny ‘Grand Theft Auto 6’ Slop

The blockbuster game makes no secret of its contempt for law enforcement. Real police departments…

24 hours ago