Categories: FAANG

Scalable Pre-training of Large Autoregressive Image Models

This paper introduces AIM, a collection of vision models pre-trained with an autoregressive objective. These models are inspired by their textual counterparts, i.e., Large Language Models (LLMs), and exhibit similar scaling properties. Specifically, we highlight two key findings: (1) the performance of the visual features scale with both the model capacity and the quantity of data, (2) the value of the objective function correlates with the performance of the model on downstream tasks. We illustrate the practical implication of these findings by pre-training a 7 billion parameter AIM on 2…
AI Generated Robotic Content

Recent Posts

Exploring Prediction Targets in Masked Pre-Training for Speech Foundation Models

Speech foundation models, such as HuBERT and its variants, are pre-trained on large amounts of…

5 hours ago

How GoDaddy built a category generation system at scale with batch inference for Amazon Bedrock

This post was co-written with Vishal Singh, Data Engineering Leader at Data & Analytics team…

5 hours ago

10 months to innovation: Definity’s leap to data agility with BigQuery and Vertex AI

At Definity, a leading Canadian P&C insurer with a history spanning over 150 years, we…

5 hours ago

Nvidia’s GTC keynote will emphasize AI over gaming

Don't expect to hear a lot about better framerates and raytracing at the Nvidia GTC…

6 hours ago

These Are the 10 DOGE Operatives Inside the Social Security Administration

The team working at the Social Security Administration appears to be among the largest DOGE…

6 hours ago

Exo 2: A new programming language for high-performance computing, with much less code

Many companies invest heavily in hiring talent to create the high-performance library code that underpins…

6 hours ago