This year, we’ve spent dozens of hours synthesizing hundreds of conversations with CXOs across leading organizations, trying to uncover their biggest thorns when it comes to building Multi-Agent Systems (MAS).
These conversations have revealed a clear pattern: MAS is helping enterprises re-think clunky legacy processes, but many CXOs are focused on automating those legacy processes rather than reimagining them. Plus, ethical risks are front and center – how do you balance innovation and ethical planning? How do CXOs take advantage of everything that’s available now, without uprooting their entire organization?
Today, we’ll explore some common missteps in the field, top questions executives have, and insights to move forward on adopting MAS today.
Quick recap: What’s the value of MAS?
MAS involves teams of coordinated AI agents working together to achieve multifaceted business goals. For example, when resolving complex customer issues, specialist agents (such as billing, usage, promotions) are managed by a coordinator agent. This orchestrator ensures that the overall resolution is driven by business logic and aligns with enterprise policies.
MAS is now transitioning from a conceptual promise to practical application. In contact centers, an orchestrator agent can analyze complex, multi-part customer queries and dynamically engage the right specialists, along with validation agents to ensure accuracy and compliance. This approach significantly improves first-contact resolution for intricate issues and increases call containment, thereby reducing the need to escalate to live agents.
Similar collaborative agent strategies are emerging across industries, such as supply chain optimization and complex research, which demonstrate MAS’s power to handle complexity through coordinated, intelligent action.
- aside_block
- <ListValue: [StructValue([(‘title’, ‘$300 in free credit to try Google Cloud AI and ML’), (‘body’, <wagtail.rich_text.RichText object at 0x3ebf9f883400>), (‘btn_text’, ‘Start building for free’), (‘href’, ‘http://console.cloud.google.com/freetrial?redirectPath=/vertex-ai/’), (‘image’, None)])]>
3 common missteps from the field
Misstep 1: Automating old processes instead of reimagining them
Applying MAS to automate existing processes severely limits its transformative potential. Real value comes from rethinking workflows to leverage MAS for dynamic and holistic-problem solving. A strong partnership between technical and business teams is essential to challenge the status quo. Customers are transitioning from bouncing customers between departments to answer complex queries, to empowering each department to answer questions more quickly, to ultimately consolidating everything into one MAS-driven department with oversight.
A key point to remember is that even though we are reimagining our current process, this doesn’t mean we need to do everything at once. If we want to increase the number of calls routed to a virtual agent, we should first identify the initial tranche of calls to address. Then, we can incrementally expand the types or topics the virtual agent can handle to ensure customer satisfaction and maintain overall support quality.
For example, this is how we sequentially move through key steps in a Multi-Agentic System program:
Misstep 2: Underestimating collaboration design effort
A critical error is under-resourcing the design of agent collaboration-particularly in defining roles, communication protocols , and conflict resolution strategies.
As MAS evolves, it’s increasingly important to know what, when, and why a specialist agent should be engaged. But how do you validate this orchestration logic? Through rigorous testing using ground truth evaluation and high-quality test data.
Customers that succeed in this area have a clear understanding of what “good” versus “bad” answers look like across different question types. These examples are critical in building agents that can determine which tools, other agents, services, verbosity, tonality, and format to use when providing a response.
Misstep 3: Delaying governance and ethical planning
Treating governance, ethics, and monitoring as afterthoughts invites significant risks, such as program delays, bias amplification, and critical policy gaps. The best way to achieve this with MAS is by embedding responsible AI principles, including establishing clear rules, audit trails and transparency. The old adage, “move slow to move fast,” becomes more relevant as we increase complexity.
For example, if bias monitoring is not considered until late in deployment, a virtual agent on an e-commerce platform might put too much weight on a customer’s zip code, displaying higher-priced products to those in wealthier areas and budget options to customers in lower-income zip codes. This could create an unfair shopping experience, where certain groups feel excluded or underserved, ultimately harming the brand’s reputation. As a result, there is rework, redesign and the need to rollback updates to go through the solution design and testing processes again, adding upwards of six months of additional work.
These concepts and the teams responsible for them must be incorporated from day 1 of a MAS project.
Top 3 questions from the field
Question 1: “Beyond cost savings, how do we measure ROI?”
We focus on tracking improved outcomes for complex tasks, enhancing customer experience, reducing manual risks, and driving new revenue streams. For instance, an analyst assistant can support a wealth manager by providing instant insights into complex financial data, identifying key trends, and generating customized reports. This propels the wealth manager to engage more meaningfully with clients, ask targeted follow-up questions, and ultimately build stronger relationships. As a result, MAS improves customer retention, increases wallet share, and minimizes the risk of misinterpreting critical financial information.
Question 2: “How do we balance human oversight with autonomous agents?”
MAS isn’t about replacing humans; rather, it’s about strategizing human skills where they have the most impact. Humans excel at navigating ambiguity, ethics, and novelty. In one real-world scenario, AI handles complex offers but escalates edge cases, such as price-matching a competitor’s promotion, to a human for final judgment. The key is ensuring that your use case and desired outcomes drive the solution. Not the other way around!
Question 3: “How can I predict outcomes and address ethical risks?”
Achieving successful outcomes in MAS requires thoughtful design, which starts with asking the right questions: What happens when a customer interacts with the system? What information is needed to answer their questions? Where should human oversight be applied, and how do we evaluate and monitor performance both in testing and production environments? To ensure reliability, we conduct a variety of tests with our customers, including load testing, accuracy and quality testing, red teaming, and user acceptance testing. This rigorous approach, combined with continuous monitoring, helps identify and correct unintended behaviors and ensures that the system performs as expected. Additionally, we proactively mitigate ethical risks such as bias amplification, unfairness, and accountability gaps by embedding rules, ensuring transparency and auditability, and assigning clear roles for both agents and humans.
This diagram depicts the MAS Ethical Lifecycle, showing the interconnected stages of Agent Design, Interaction and Coordination, Deployment and Operation, Human-AI Orchestration, and Continuous Improvement, all guided by fundamental ethical considerations.
Get started
Based on these field insights, consider prioritizing the following:
Develop a MAS strategy: Start small, think big
Prioritize governance, ethics and trust from day one
Foster a collaborative culture that puts your user first: IT and business unite
Google is here to support. Check out how Vertex AI supports building and managing MAS. Google Cloud Consulting brings an added layer of insights and expertise, helping organizations navigate the nuances of MAS adoption and harness its transformative power. Talk to a Google Cloud sales specialist today!