6 AI Agent Capabilities to Evaluate Before Enterprise Deployment

In our Engineering Energizers Q&A series, we highlight the engineering minds driving innovation across Salesforce. Today, we spotlight Shelby Heinecke, Director of AI Research, whose team tackled a high-stakes problem: how do you know an AI agent is truly ready for the enterprise before production exposes what your benchmarks missed? Getting the answer right is not enough. An enterprise agent must also choose the right tools, plan complex tasks, use context correctly, operate safely, and handle engineering work, while remaining accurate, reliable, and trustworthy.

Explore how Shelby’s team brought eight independently developed enterprise benchmarks under one framework, mapped their different metrics into six enterprise skill dimensions spanning more than 20 subdomains and 1,500 test tasks, and created a unified scoring system designed to expose weaknesses before they become real-world enterprise problems.

Why is proving an AI agent is truly ready for enterprise deployment such a difficult engineering challenge?

The nightmare scenario is that an agent passes your evaluations, goes into production, and only then exposes weaknesses your benchmarks never tested. Many benchmarks assess narrow capabilities, but enterprise agents must possess a broad range of skills. That means an agent can look ready in evaluation and still surprise you when it encounters real-world enterprise work.

The goal was to bring those capabilities together in a one-stop shop for assessing the LLMs powering enterprise agents. All of the benchmarks included in EGI-Bench were designed at Salesforce, so the collective experience with enterprise deployment is inherently baked into the evaluation.

Inside EGI Bench: How it actually works.

Why did the fragmented AI benchmark landscape make comprehensive LLM evaluation for enterprise agents so difficult?

There are many great AI benchmarks, with new ones appearing constantly. The challenge is figuring out which ones matter, understanding and configuring each benchmark, and then running them individually. For practitioners comparing multiple LLMs, that can mean understanding several evaluation systems before they can even assemble a comprehensive picture of enterprise performance.

The goal was to simplify that without sacrificing coverage. EGI-Bench spans more than 20 subdomains, more than 1,500 test tasks, and six enterprise skill dimensions. Practitioners can plug in the LLMs powering their agents and evaluate all six dimensions or select the ones most relevant to their use case.

How did you make deep AI agent evaluation practical when comprehensive benchmarks can take hours to run?

Comprehensive evaluation creates a tradeoff: the more models, tests, and enterprise dimensions you evaluate, the more time the process takes. Practitioners may also be comparing several candidate LLMs, making exhaustive benchmarking time-consuming before they even know where the meaningful differences are.

The team addressed that by creating both deep and quick modes. Deep mode runs the benchmarks fully for a comprehensive evaluation, while quick mode subsamples each benchmark to produce a faster directional signal. If quick mode reveals that a model is particularly weak in one area, practitioners can investigate that area more deeply rather than immediately running every available test.

What made combining eight AI benchmarks into a single enterprise scoring system so difficult?

Each of EGI-Bench’s eight benchmarks was already a deep standalone benchmark with its own metrics and areas of focus. Putting them in one repository was not enough. The measurements designed for different purposes needed to be reorganized around a common set of enterprise skills.

The team broke the benchmarks into their underlying sub-metrics and mapped those to the six enterprise dimensions. For example, parts of CRMArena measure task accuracy while others focus more on grounding. Because those metrics measure different things, their scores were also normalized so they could be meaningfully combined. That produced six EGI-Bench subscores, which were then weighted to create a single global EGI score. The repository provides default weights based on experience, but practitioners can change them when a particular capability, such as safety, planning, or grounding, matters more for their use case.

What challenges did testing 14 LLMs expose about choosing the right model for enterprise AI agents?

Three findings stood out. First, no single model was best across all six dimensions. Different models had different strengths, reinforcing that practitioners need to choose the right model for the work their agent actually performs. Second, multi-step planning remains challenging. Planning scores were on the lower end across the models evaluated, highlighting an area where the broader AI community still has room to push performance forward. Third, cost-efficient models are punching above their weight. Lower-cost models, including Haiku 4.5, GPT-5 mini, and Gemini 2.5 Flash, delivered increasingly competitive performance across enterprise tasks.

Together, those findings show why enterprise model selection benefits from looking beyond a single aggregate notion of which model is best.

How can developers adapt EGI-Bench to the priorities of their own enterprise AI agents?

EGI-Bench is available on GitHub so practitioners can use it themselves. They can connect the LLMs they want to evaluate, select the enterprise dimensions relevant to their use case, and examine how those models perform across the capabilities that matter to them.

The six subscores and customizable global EGI score are particularly useful because different applications have different priorities. One team might care more about grounding or safety, while another may emphasize planning, engineering, or tool actioning. EGI-Bench lets practitioners adjust the weights accordingly and evaluate models against their own priorities rather than assuming the same model or weighting is right for every enterprise workload.