Category Added in a WPeMatico Campaign
This paper was accepted at the Workshop on Foundation Models in the Wild at ICLR 2025. Visual understanding is inherently contextual – what we focus on in an image depends on the task at hand. For instance, given an image of a person holding a bouquet of flowers, we may focus on either the person …
Read more “FocalLens: Instruction Tuning Enables Zero-Shot Conditional Image Representations”
This post is co-written with Kim Nguyen and Shyam Banuprakash from Clario. Clario is a leading provider of endpoint data solutions to the clinical trials industry, generating high-quality clinical evidence for life sciences companies seeking to bring new therapies to patients. Since Clario’s founding more than 50 years ago, the company’s endpoint data solutions have …
At Apple, we believe privacy is a fundamental human right. And we believe in giving our users a great experience while protecting their privacy. For years, we’ve used techniques like differential privacy as part of our opt-in device analytics program. This lets us gain insights into how our products are used, so we can improve …
Read more “Understanding Aggregate Trends for Apple Intelligence Using Differential Privacy”
Large language models (LLMs) have raised the bar for human-computer interaction where the expectation from users is that they can communicate with their applications through natural language. Beyond simple language understanding, real-world applications require managing complex workflows, connecting to external data, and coordinating multiple AI capabilities. Imagine scheduling a doctor’s appointment where an AI agent …
Read more “Build multi-agent systems with LangGraph and Amazon Bedrock”
Many organizations rely on multiple third-party applications and services for different aspects of their operations, such as scheduling, HR management, financial data, customer relationship management (CRM) systems, and more. However, these systems often exist in silos, requiring users to manually navigate different interfaces, switch between environments, and perform repetitive tasks, which can be time-consuming and …
Read more “Building an AIOps chatbot with Amazon Q Business custom plugins”
Attending a tech conference like Google Cloud Next can feel like drinking from a firehose — all the news, all the sessions, and breakouts, all the learning and networking… But after a busy couple of days, watching the developer keynote makes it seem like there’s a method to the madness. A coherent picture starts to …
Read more “Next 25 developer keynote: From prompt, to agent, to work, to fun”
This research aims to comprehensively explore building a multimodal foundation model for egocentric video understanding. To achieve this goal, we work on three fronts. First, as there is a lack of QA data for egocentric video understanding, we automatically generate 7M high-quality QA samples for egocentric videos ranging from 30 seconds to one hour long …
Read more “MM-Ego: Towards Building Egocentric Multimodal LLMs”
Training a frontier model is highly compute-intensive, requiring a distributed system of hundreds, or thousands, of accelerated instances running for several weeks or months to complete a single job. For example, pre-training the Llama 3 70B model with 15 trillion training tokens took 6.5 million H100 GPU hours. On 256 Amazon EC2 P5 instances (p5.48xlarge, …
Read more “Reduce ML training costs with Amazon SageMaker HyperPod”
When it comes to AI, inference is where today’s generative AI models can solve real-world business problems. Google Kubernetes Engine (GKE) is seeing increasing adoption of gen AI inference. For example, customers like HubX run inference of image-based models to serve over 250k images/day to power gen AI experiences, and Snap runs AI inference on …
Read more “New GKE inference capabilities reduce costs, tail latency and increase throughput”
Building a generalist model for user interface (UI) understanding is challenging due to various foundational issues, such as platform diversity, resolution variation, and data limitation. In this paper, we introduce Ferret-UI 2, a multimodal large language model (MLLM) designed for universal UI understanding across a wide range of platforms, including iPhone, Android, iPad, Webpage, and …
Read more “Ferret-UI 2: Mastering Universal User Interface Understanding Across Platforms”