Tag: faang

Mixture-of-Experts (MoE) models enable sparse expert activation, meaning that only a subset of the model’s parameters is used during each inference. However, to translate this sparsity into practical performance, an expert caching mechanism is required. Previous works have proposed hardware-centric caching policies, but how these various caching policies interact with each other and different hardware …

Read more “SpecMD: A Comprehensive Study on Speculative Expert Prefetching”

Serving transformer language models with high throughput requires caching Key-Values (KVs) to avoid redundant computation during autoregressive generation. The memory footprint of KV caching is significant and heavily impacts serving costs. This work proposes to lessen these memory requirements. While recent work has largely addressed KV cache reduction via compression and eviction along the temporal …

Read more “Stochastic KV Routing: Enabling Adaptive Depth-Wise Cache Sharing”

Building AI agents that work well in a demo is one thing, but running them in production requires serious infrastructure.  At Google Cloud Next ’26, we introduced Gemini Enterprise Agent Platform to help developers build, deploy, scale, govern, and optimize  autonomous AI agents. From managing long-running state and enforcing security with the Agent Governance Stack, …

Read more “Five must-have guides to move agents into production with Gemini Enterprise Agent Platform”

Multi-tool-integrated reasoning enables LLM-empowered tool-use agents to solve complex tasks by interleaving natural-language reasoning with calls to external tools. However, training such agents using outcome-only rewards suffers from credit-assignment ambiguity, obscuring which intermediate steps (or tool-use decisions) lead to success or failure. In this paper, we propose PORTool, an importance-aware policy-optimization algorithm that reinforces agents’ …

Read more “PORTool: Importance-Aware Policy Optimization with Rewarded Tree for Multi-Tool-Integrated Reasoning”

This paper was accepted at the Fifth Workshop on Natural Language Generation, Evaluation, and Metrics at ACL 2026. Tool-calling agents are evaluated on tool selection, parameter accuracy, and scope recognition, yet LLM trajectory assessments remain inherently post-hoc. Disconnected from the active execution loop, such assessments identify errors that are usually addressed through prompt-tuning or retraining, …

Read more “Reinforced Agent: Inference-Time Feedback for Tool-Calling Agents”