Categories: FAANG

Introducing built-in performance monitoring for Vertex AI Model Garden

Today, we’re announcing built-in performance monitoring and alerts for Gemini and other managed foundation models – right from Vertex AI’s homepage.

Monitoring the performance of generative AI models is crucial when building lightning-fast, reliable, and scalable applications. But understanding the performance of these models has historically had a steep learning curve: in the past, you had to learn where the metrics were stored and where you could find them in the Cloud Console. 

Now, these metrics are available right on Vertex AI’s home page, where you can easily find and understand the health of your models. Cloud Monitoring shows a built-in dashboard providing information about usage, latency, and error rates on your gen AI models. You can also quickly configure an alert if any requests have failed or been delayed.

aside_block
<ListValue: [StructValue([(‘title’, ‘$300 in free credit to try Google Cloud AI and ML’), (‘body’, <wagtail.rich_text.RichText object at 0x3e59b867a910>), (‘btn_text’, ‘Start building for free’), (‘href’, ‘http://console.cloud.google.com/freetrial?redirectPath=/vertex-ai/’), (‘image’, None)])]>

How it works

If you’re using Vertex AI foundation models, you can find overview metrics for your models on the Dashboard tab in Vertex AI, and click into an out-of-the-box dashboard in Cloud Monitoring to gain more information and customize the dashboard. Here, you will be better able to understand capacity constraints, predict costs, and troubleshoot errors. You can also quickly configure alerts to quickly inform you about failures and their causes.

View Model Observability in Vertex AI

Configure an alert

Let’s say you’re an SRE who is responsible for ensuring the uptime of your company’s new customer service chatbot. You want to find a dashboard that gives you a bird’s eye view of possible issues with the chatbot, whether they include slowness, errors, or unexpected usage volume. Instead of hunting for the right metrics and creating a dashboard that displays them, you can now go to the Vertex Dashboard page to view high level metrics, and click “Show all metrics” to view a detailed, opinionated dashboard with information about query rates, character and token throughput, latency, and errors. 

Then, let’s say that you notice that your model returned a 429 error for a number of your requests. This happens when the ML serving region associated with your model runs out of aggregate capacity across customers. You can remediate the issue by purchasing provisioned throughput, switching ML processing locations, or scheduling non-urgent requests for a less busy time using batch requests. You can also quickly turn on a recommended alert that will let you know if more than 1% of your requests return 429 errors ever again. 

Get started today

If you’re a user of managed gen AI models from Vertex AI Model Garden, check out the “Model Observability” tab in your project’s Vertex Dashboard page. Click “Show all metrics” to find the built-in dashboard. To configure recommended alerts related to your gen AI workloads, check out the Vertex AI Integration in Cloud Monitoring.

AI Generated Robotic Content

Recent Posts

Extra finger, mutated fingers, malformed, deformed hand,

submitted by /u/NetPlayer9 [link] [comments]

26 mins ago

Decision Trees Aren’t Just for Tabular Data

Versatile, interpretable, and effective for a variety of use cases, decision trees have been among…

26 mins ago

Netflix Tudum Architecture: from CQRS with Kafka to CQRS with RAW Hollow

By Eugene Yemelyanau, Jake GriceIntroductionTudum.com is Netflix’s official fan destination, enabling fans to dive deeper into…

26 mins ago

New capabilities in Amazon SageMaker AI continue to transform how organizations develop AI models

As AI models become increasingly sophisticated and specialized, the ability to quickly train and customize…

26 mins ago

$8.8 trillion protected: How one CISO went from ‘that’s BS’ to bulletproof in 90 days

Clearwater Analytics CISO Sam Evans dodged a bullet by blocking shadow AI from exposing data…

1 hour ago

The 7 Best Prime Day Action Camera Deals for Thrill Seekers (2025)

Action cameras are perfect for travel, social media vlogging, and careening around the lake on…

1 hour ago