1eIWpHw7J 6qZCuaffD0lQQ

Trading a Cloud Identity for Your Own: Workload Attestation on Managed Compute

By Dhruv Pratap Introduction Organizations that have been around for a while usually run two identity systems side by side. One belongs to the cloud provider: IAM roles, instance profiles, execution roles. The other is your own, and it is the one your internal services actually check when they decide whether to answer a request. On infrastructure …

ML 21346 1

Scaling MoE reinforcement learning on Amazon EKS with EFA and DeepEP with 40% more throughput

When you post-train a Mixture-of-Experts (MoE) model with Reinforcement Learning from Human Feedback (RLHF) or Group Relative Policy Optimization (GRPO) at scale, three simultaneous challenges emerge. The first requires coordinating heterogeneous compute for rollout generation and policy training. Second, sustaining high-throughput communication across hundreds of accelerators. And third, dynamically orchestrating every subsystem to keep them …

1 Single Step RL Training Loopmax 1000x1000 1

Best practices guide for customizing Gemini models via Reinforcement Learning (RL)

Reinforcement learning (RL) has been a keystone of modern LLM post-training, but it demands large training clusters and access to model internals that external customers can’t have with proprietary models like Gemini. So here at Google Cloud, we packaged it into a managed RL fine-tuning service (RLFT service) — you bring prompts and a reward …

AI hardware design could filter out irrelevant visual data to reduce energy use

When we read a car’s license plate, we tend to focus on the numbers and letters, paying little attention to the surrounding buildings or sky. An artificial-intelligence (AI) system, however, may still process those surrounding details, consuming energy even when they contribute little to the task. As cameras capture ever more detailed images, this unnecessary …

Compressing Streaming Neural Audio Encoders via Latent-Space Distillation

System-wide Dictation on Apple devices runs entirely on-device, and the speech it transcribes reaches the foundation model through a tokenizer: an encoder that maps short windows of waveform onto the representation the language model reads. Because that model is sparsely activated under Instruction-Following Pruning, only a small subset of its experts occupies DRAM at any …