Categories: AI/ML Research

A Gentle Introduction to Model Distillation

In this article, you will learn what model distillation is, how it has evolved for large language models, and why it has become one of…

Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillation

Estimating dimensional emotions, such as activation, valence and dominance, from acoustic speech signals has been widely explored over the past few years. While accurate estimation of activation and dominance from speech seem to be possible, the same for valence remains challenging. Previous research has shown that the use of lexical…

October 12, 2022

In "FAANG"

Amazon Bedrock Model Distillation: Boost function calling accuracy while reducing cost and latency

May 1, 2025

In "FAANG"

Locking Pretrained Weights via Deep Low-Rank Residual Distillation

The quality of open-weight language models has dramatically improved in recent years. Sharing weights greatly facilitates model adoption by enabling their use across diverse hardware and software platforms. They also allow for more open research and testing, to the extent that users can use them as checkpoints, fine-tune them according…

August 7, 2026

In "FAANG"

AI Generated Robotic Content