This is the third post in our blog series on Rubix (#1, #2), our effort to rebuild our cloud architecture around Kubernetes. Introduction The advent of containers and their orchestration platforms, most popularly Kubernetes (K8s), has familiarized engineers with the concept of ephemeral compute: once a workload has completed, the resources used to run it …
Read more “The Benefits of Running Kubernetes on Ephemeral Compute”
Enterprises across many industries are adopting artificial intelligence (AI) and machine learning (ML) at a rapid pace. Many factors fuel this accelerated adoption, including a need to realize value out of the massive amounts of data generated by multichannel customer interactions and the increasing stores of data from all facets of an enterprise’s operations. This …
Read more “ML Engineers: Partners for Scaling AI in Enterprises”
Editor’s note: In part one of this blog, Wayfair shared how it supports each of its 30 million active customers using machine learning (ML). Wayfair’s Vinay Narayana, Head of ML Engineering, Bas Geerdink, Lead ML Engineer, and Christian Rehm, Senior Machine Learning Engineer, take us on a deeper dive into the ways Wayfair’s data scientists …
Read more “How Wayfair is reaching MLOps excellence with Vertex AI”
From some angles, a lot of startup founders consider broadly similar questions, such as “should I use serverless?”, “how do I manage my data?”, or “do I have a use case for Web3?” But the deeper you probe, the more every startup’s rise becomes unique, from the early moments among founders, to the of hiring …
Read more “Founders and tech leaders share their experiences in “Startup Stories” podcast”
Today, to accelerate research in the bio-pharma space, from the creation of treatments for diseases to the production of new synthetic biomaterials, we are announcing a new Vertex AI solution that demonstrates how to use Vertex AI Pipelines to run DeepMind’s AlphaFold protein structure predictions at scale. Once a protein’s structure is determined and its …
Read more “Running AlphaFold batch inference with Vertex AI Pipelines”
Vertex AI Training delivers a serverless approach to simplify the ML model training experience for customers. As such, training data does not persist on the compute clusters by design. In the past, customers had only Cloud Storage (GCS) or BigQuery (BQ) as storage options. Now, you can also use NFS shares, such as Filestore, for …
Developers and data scientists are increasingly turning to Google Kubernetes Engine (GKE) to run demanding workloads like machine learning, visualization/rendering and high-performance computing, leveraging GKE’s support for NVIDIA GPUs. In the current economic climate, customers are under pressure to do more with fewer resources, and cost savings are top of mind. To help, in July, …
Read more “Sharing is caring: How NVIDIA GPU sharing on GKE saves you money”
This blog presents an open-source solution to heterogeneous graph sub-sampling at scale using Google Cloud Dataflow (Dataflow). Dataflow is Google’s publicly available, fully managed environment for running large scale Apache Beam compute pipelines. Dataflow provides monitoring and observability out of the box and is routinely used to scale production systems to easily handle extreme datasets. …
Read more “Scaling heterogeneous graph sampling for GNNs with Google Cloud Dataflow”
R is one of the most widely used programming languages for statistical computing and machine learning. Many data scientists love it, especially for the rich world of packages from tidyverse, an opinionated collection of R packages for data science. Besides the tidyverse, there are over 18,000 open-source packages on CRAN, the package repository for R. …
Read more “Use R to train and deploy machine learning models on Vertex AI”
Over the past few years, advances in training large language models (LLMs) have moved natural language processing (NLP) from a bleeding-edge technology that few companies could access, to a powerful component of many common applications. From chatbots to content moderation to categorization, a general rule for NLP is that the larger the model, the greater …
Read more “How Cohere is accelerating language model training with Google Cloud TPUs”