Categories: FAANG

RT-2: New model translates vision and language into action

Introducing Robotic Transformer 2 (RT-2), a novel vision-language-action (VLA) model that learns from both web and robotics data, and translates this knowledge into generalised instructions for robotic control, while retaining web-scale capabilities. This work builds upon Robotic Transformer 1 (RT-1), a model trained on multi-task demonstrations which can learn combinations of tasks and objects seen in the robotic data. RT-2 shows improved generalisation capabilities and semantic and visual understanding, beyond the robotic data it was exposed to. This includes interpreting new commands and responding to user commands by performing rudimentary reasoning, such as reasoning about object categories or high-level descriptions.
AI Generated Robotic Content

Recent Posts

WAN 2.1 Vace makes the cut

100% Made with opensource tools: Flux, WAN2.1 Vace, MMAudio and DaVinci Resolve. submitted by /u/Race88…

20 hours ago

Combining XGBoost and Embeddings: Hybrid Semantic Boosted Trees?

The intersection of traditional machine learning and modern representation learning is opening up new possibilities.

20 hours ago

Gemini Robotics On-Device brings AI to local robotic devices

We’re introducing an efficient, on-device robotics model with general-purpose dexterity and fast task adaptation.

20 hours ago

Power Your LLM Training and Evaluation with the New SageMaker AI Generative AI Tools

Today we are excited to introduce the Text Ranking and Question and Answer UI templates…

20 hours ago

The secret to document intelligence: Box builds Enhanced Extract Agents using Google’s Agent-2-Agent framework

Box is one of the original information sharing and collaboration platforms of the digital era.…

20 hours ago

Stanford’s ChatEHR allows clinicians to query patient medical records using natural language, without compromising patient data

ChatEHR accelerates chart reviews for ER admissions, streamlines patient transfer summaries and synthesizes complex medical…

21 hours ago