Category: FAANG

In this post, we demonstrate how to use neural architecture search (NAS) based structural pruning to compress a fine-tuned BERT model to improve model performance and reduce inference times. Pre-trained language models (PLMs) are undergoing rapid commercial and enterprise adoption in the areas of productivity tools, customer service, search and recommendations, business process automation, and …

Read more “Reduce inference time for BERT models using neural architecture search and SageMaker Automated Model Tuning”

If you’ve joined a new team recently like I have, you’ve probably had a lot of questions. And answers to those questions may or may not be things you can find easily, and might rely heavily on the generosity, and spare time of your teammates. Let’s say you’re a DevRel engineer, working with Google Kubernetes …

Read more “Figuring out microservices running on your GKE cluster with help from Duet AI”

Posted by Jiefeng Chen, Student Researcher, and Jinsung Yoon, Research Scientist, Cloud AI Team In the fast-evolving landscape of artificial intelligence, large language models (LLMs) have revolutionized the way we interact with machines, pushing the boundaries of natural language understanding and generation to unprecedented heights. Yet, the leap into high-stakes decision-making applications remains a chasm …

Read more “Introducing ASPIRE for selective prediction in LLMs”

At Next ‘23, we launched AlloyDB AI, an integrated set of capabilities built into AlloyDB for building generative AI applications. One of those capabilities allows you to call a Vertex AI model directly from the database using SQL. AlloyDB is a fully managed PostgreSQL-compatible database that offers superior performance, availability and scale. In our performance …

Read more “AlloyDB AI powers gen AI applications with seamless Vertex AI integration”

Today, we’re excited to announce the availability of Llama 2 inference and fine-tuning support on AWS Trainium and AWS Inferentia instances in Amazon SageMaker JumpStart. Using AWS Trainium and Inferentia based instances, through SageMaker, can help users lower fine-tuning costs by up to 50%, and lower deployment costs by 4.7x, while lowering per token latency. …

Read more “Fine-tune and deploy Llama 2 models cost-effectively in Amazon SageMaker JumpStart with AWS Inferentia and AWS Trainium”