Generative AI is now well beyond the hype and into the realm of practical application. But while organizations are eager to build enterprise-ready gen AI solutions on top of large language models (LLMs), they face challenges in managing, securing, and scaling these deployments, especially when it comes to APIs. As part of the platform team, …
Ibrahim Said Ahmad†, Antonios Anastasopoulos††††, Ondřej Bojar¶, Claudia Borg††, Marine Carpuat‡, Roldano Cattoni§, Mauro Cettolo§, William Chen‡‡, Qianqian Dong¶¶, Marcello Federico§§, Barry Haddow‡‡‡, Dávid Javorsky¶, Mateusz Krubiński¶, Tsz Kin Lam‡‡‡, Xutai Ma‡‡§, Prashant Mathur§§, Evgeny Matusov¶¶¶, Chandresh Kumar Maurya¶¶†, John P. McCrae†††, Kenton Murray†††, Satoshi Nakamura§§§, Matteo Negri§, Jan Niehues††¶, Xing Niu§§, Atul Kr. Ojha†††, …
There’s a growing demand from customers to incorporate generative AI into their businesses. Many use cases involve using pre-trained large language models (LLMs) through approaches like Retrieval Augmented Generation (RAG). However, for advanced, domain-specific tasks or those requiring specific formats, model customization techniques such as fine-tuning are sometimes necessary. Amazon Bedrock provides you with the …
Read more “Fine-tune LLMs with synthetic data for context-based Q&A using Amazon Bedrock”
Editor’s note: This article, originally published on March 13, 2023, has been updated. The mics were live and tape was rolling in the studio where the Miles Davis Quintet was recording dozens of tunes in 1956 for Prestige Records. When an engineer asked for the next song’s title, Davis shot back, “I’ll play it, and …
This blog post is co-written with George Orlin from Meta. Today, we are excited to announce that Meta’s Segment Anything Model (SAM) 2.1 vision segmentation model is publicly available through Amazon SageMaker JumpStart to deploy and run inference. Meta SAM 2.1 provides state-of-the-art video and image segmentation capabilities in a single model. This cutting-edge model …
Read more “Meta SAM 2.1 is now available in Amazon SageMaker JumpStart”
*Primary Contributors Attention is a key part of the transformer architecture. It is a sequence-to-sequence mapping that transforms each sequence element into a weighted sum of values. The weights are typically obtained as the softmax of dot products between keys and queries. Recent work has explored alternatives to softmax attention in transformers, such as ReLU …
Read more “Theory, Analysis, and Best Practices for Sigmoid Self-Attention”
This post is co-written with Gordon Campbell, Charles Guan, and Hendra Suryanto from RDC. The mission of Rich Data Co (RDC) is to broaden access to sustainable credit globally. Its software-as-a-service (SaaS) solution empowers leading banks and lenders with deep customer insights and AI-driven decision-making capabilities. Making credit decisions using AI can be challenging, requiring …
Read more “Transforming credit decisions using generative AI with Rich Data Co and AWS”
At Google Cloud, we strive to make it easy to deploy AI models onto our infrastructure. In this blog we explore how the Cross-Cloud Network solution supports your AI workloads. Managed and Unmanaged AI options Google Cloud provides both managed (Vertex AI) and do-it-yourself (DIY) approaches for running AI workloads. Vertex AI: A fully managed …
Every year, venomous snakes kill over 100,000 people and leave 300,000 more with devastating injuries — amputations, paralysis and permanent disabilities. The victims are often farmers, herders and children in rural communities across sub-Saharan Africa, South Asia and Latin America. For them, a snakebite isn’t just a medical crisis — it’s an economic catastrophe. Treatment …
As language models grow ever larger, so do their vocabularies. This has shifted the memory footprint of LLMs during training disproportionately to one single layer: the cross-entropy in the loss computation. Cross-entropy builds up a logit matrix with entries for each pair of input tokens and vocabulary items and, for small models, consumes an order …
Read more “Cut Your Losses in Large-Vocabulary Language Models”