Categories: FAANG

Leveraging Audio-Visual Data to Reduce the Multilingual Gap in Self-Supervised Speech Models

Self-supervised learning (SSL) has made significant advances in speech representation learning. Models like wav2vec 2.0 and HuBERT have achieved state-of-the-art results in tasks such as speech recognition, particularly in monolingual settings. However, multilingual SSL models tend to underperform their monolingual counterparts on each individual language, especially in multilingual scenarios with few languages such as the bilingual setting. In this work, we investigate a novel approach to reduce this performance gap by introducing limited visual grounding into bilingual speech SSL models. Our…

Unsupervised speech-to-speech translation from monolingual data

December 2, 2023

In "FAANG"

Unsupervised speech-to-speech translation from monolingual data

December 2, 2023

In "FAANG"

Can You Remove the Downstream Model for Speaker Recognition with Self-Supervised Speech Features?

Self-supervised features are typically used in place of filter-bank features in speaker verification models. However, these models were originally designed to ingest filter-banks as inputs, and thus, training them on self-supervised features assumes that both feature types require the same amount of learning for the task. In this work, we…

August 20, 2024

In "FAANG"

AI Generated Robotic Content