| | Page: https://ernie-research.github.io/NAVA/ NAVA is a 6.3 B-parameter joint audio-video generator that synthesizes synchronized video and audio from a single prompt — including multi-speaker speech with reference-timbre control and image-conditioned continuations. Instead of post-hoc-aligned dual towers or fully unified tri-modal stacks, NAVA uses an Align-then-Fuse MMDiT: a dedicated alignment space first establishes audio-video correspondence, then context (text, speaker embeddings) is fused via cross-attention. On Verse-Bench it sets new SOTA on Sync-C / Sync-D / video quality / audio WER while using 2× to 5× fewer parameters than open-source baselines. submitted by /u/AgeNo5351 |
In this article, you will learn how to build a multi-modal RAG pipeline in Python…
When enterprise data sits behind a corporate firewall or in a private on-premises data center,…
Google Cloud has always been defined by choice, enterprise-grade governance, and market-leading price-performance. Key to…
Meta’s board feared that its CEO’s departure could disrupt the company’s AI ambitions. A new…
The start of a new battery materials library could pave the way for future AI…
Run this on a typical laptop and both systems will report latencies in the low…