Build, automate, and monitor BigQuery ML models with Vertex AI MLOps capabilities

Making machine learning (ML) models work in production is hard. It usually requires not only a deep understanding of ML, data engineering, and software engineering, but also a variety of tools and technologies.

BigQuery ML and Vertex AI make it easier to create, deploy, and manage machine learning models.

BigQuery ML provides a SQL interface for powerful machine learning capabilities. Vertex AI is a managed machine learning platform that offers a unified experience for managing the entire machine learning lifecycle, from data preparation to model deployment.

Together, BigQuery ML and Vertex AI can help you to overcome the challenges of creating, deploying, and managing machine learning models from end to end across tools.

Get started with an end-to-end guide

We’ve created a sample notebook to serve as your guide — you can access this notebook on GitHub. 

• Learn how to use BigQuery ML and Vertex AI. This notebook provides a step-by-step guide on how to use these services to create, deploy, and manage machine learning models.

• Get started with machine learning on BigQuery. BigQuery ML is one of the easiest ways to build and deploy machine learning models on BigQuery. 

• Learn about the different machine learning algorithms that are available in BigQuery ML. This notebook shows you how to use a variety of machine learning algorithms, including linear regression, logistic regression, and decision trees.

• Go deeper with more detailed resources. You’ll find links to documentation and support resources for BigQuery ML and Vertex AI.

Let’s take a look at each step of the workflow you’ll learn with this notebook.

1. Prepare the Data

The first step is to prepare the data for modeling:

• Import the necessary libraries. BigQuery ML automatically splits the data into training/test.

2. Train the Model

With the data prepared, you can train the model using the following steps:

• Create a BigQuery ML model.

• Train the model using the training set.

• Evaluate the model on the test set.

3. Register the Model to Vertex AI Model Registry

Once the model is trained, you can register it to Vertex AI Model Registry. This registration to Model Registry can be done directly from BigQuery ML, making it a very easy transition between tools. This will allow you to manage the model and deploy it to an endpoint for real-time prediction.

To register the model, you will use the following steps:

• Register the model to the Model Registry.

• Annotate the model.

4. Deploy the Model to an Endpoint

With a registered model, you can easily deploy it to an endpoint for real-time prediction. To do this, use the following steps:

• Create a Vertex AI Endpoint.

• Deploy the model to the Endpoint.

5. Make Predictions

Now that the model is deployed, you can make predictions with the following steps:

• Prepare a prediction request.

• Send the prediction request to the Endpoint.

• Get the prediction result.

With this sample notebook, you’ll learn how to use BigQuery ML and Vertex AI from end-to-end to make online predictions. We’re continuing to update these materials so please feel free to share any feedback via GitHub!