Why AI Agents Get the Right Facts but the Wrong Answer—and How GraphRAG Helps

By Sivakumar Shanmugam and Indrojit Deb.

An AI agent rejects a valid customer return. You check its citation: the policy really does say 90 days. You check the purchase: it falls outside that window. Both facts are correct, but the customer qualifies for an exception the system never retrieved. Checking the evidence has made the wrong answer look more convincing.

Every fact needed for the correct decision exists in the system. But the product classification is in a catalog, the eligibility rule is in a wiki, and the customer’s membership is in the CRM. The retrieval-augmented generation (RAG) pipeline found the general policy without assembling the facts that change which policy applies.

That failure drove the GraphRAG work for Agentforce. GraphRAG combines RAG with a knowledge graph connecting entities and their relationships. How do those connections help an agent discover an exception that ordinary retrieval missed?

Why does RAG return the wrong answer from accurate sources?

To see what the citation leaves out, follow the purchase beyond the general policy. In the illustrative scenario, the customer purchased a WM-500 for $899. The product catalog classifies that model as a major appliance with a 120-day extended return window. The exceptions wiki adds a restriction: the extension applies only to purchases over $500 by Gold or Platinum members. The customer has Gold membership. Together, these facts establish eligibility that neither the general policy nor the purchase date reveals alone.

Why might retrieval miss them? The customer asked about a broken washing machine, not product classifications or membership thresholds. An intermediate fact can determine the answer without closely resembling the question. Engineers described this as the “flat bag of chunks” problem. Vector and keyword retrieval can return relevant passages while missing necessary connections across chunks and documents. Retrieving accurate evidence and retrieving enough evidence are different engineering problems.

For your system, distinguish a lookup from a conditional decision. A question about the stated return window can be answered from the policy itself. Determining whether a particular purchase qualifies for an exception requires connected facts. Establish which kind of question is failing before changing the architecture.

How does GraphRAG find facts the customer never mentioned?

Suppose the initial results contain the purchase record and the general policy. Neither establishes eligibility for the extension. How can the agent discover that product classification is the missing step? Start with WM-500. Its classification leads to Major Appliance, which leads to the extended return policy. That policy introduces two further checks: membership tier and purchase amount. Following the connections reveals what else the agent must retrieve. “Major Appliance” is not the answer the customer wants. It connects the purchased product to the policy that changes the answer. The agent needs to reach that fact even when it does not appear among the initial similarity matches.

The GraphRAG implementation extracts entities and relationships across source content into a knowledge graph. At retrieval time, the agent follows successive relationships to gather context beyond the initial matches. This is multi-hop retrieval: reaching necessary information through intermediate connections. The graph guides retrieval toward related facts, while text chunks supply supporting evidence and citations. In this example, the policy provides the eligibility conditions, and the connected records establish membership and purchase amount.

For your application, check whether retrieval reaches all the evidence required for the decision. Availability alone does not establish that the agent will interpret every condition correctly. Nor can traversal recover a condition the graph never captured.

What if your knowledge graph leaves out a business rule?

Imagine that your graph connects WM-500 to Major Appliance and Major Appliance to the extended policy, but contains no membership requirement. How would the agent distinguish an eligible Gold member from a customer who does not qualify? The represented relationships would not establish that distinction. Unlike leaving an existing connection out of retrieved context, this failure begins in the graph’s structure. Adding customer records cannot repair a blueprint that omits the eligibility rule.

Before populating the graph, its structure must express how the extended policy depends on membership tier and purchase amount. Actual records can then supply the values needed to evaluate those conditions. Our workflow separates these responsibilities into the TBox and ABox. The TBox defines entity types and relationships; the ABox contains instances such as this customer’s Gold membership, this WM-500 purchase, and its $899 price. AI proposes the blueprint from content, and a business user validates it before extraction populates the graph. That review must catch the absent membership condition before the omission affects answers across many purchases.

For your platform, walk a specific question through the proposed structure. Can it express the classification, policy, membership requirement, and price threshold? Once those relationships are represented, check whether they lead to the records containing the actual values.

How does GraphRAG connect unstructured content to structured records?

Suppose the graph captures the policy and price threshold correctly, but the purchase amount lives in a structured system. The agent knows which fact it needs. Does it know where to retrieve it? Without an explicit connection, it may need to inspect a catalog or metadata system, identify a likely matching entity, examine its schema, and retrieve the record. It must resolve the connection between a document-derived concept and structured data during the request.

The implementation supplies direct pointers from graph entities to structured information. When the agent moves from product information in the graph to the corresponding table, it follows an established connection instead of inferring the mapping from similar names. This addresses a different gap from discovering the exception. Relationships across documents identify which policy matters; pointers to structured records locate the facts needed to evaluate it. Representing the policy correctly does not automatically establish where the purchase information lives.

For your application, inspect whether that transition follows a known connection or requires repeated discovery. Explicit pointers reduce the need to rediscover mappings through additional reasoning and tool calls, although the speed improvement has not been quantified. Establish known connections during preparation rather than leaving their resolution to each request.

How can you locate the break in your RAG pipeline?

Return to the rejected washing machine return and compare the required evidence with what your agent received. The same wrong answer can expose three different gaps.

Check the retrieved context. Suppose the catalog contains the Major Appliance classification, but the agent receives only the purchase and general policy. Inspect whether retrieval can follow the classification to the extended policy instead of depending entirely on initial similarity matches.

Check the blueprint. Suppose traversal reaches the extended policy, but the graph omits its membership restriction. Review whether the source condition is represented before populating more records. Additional instances cannot supply a rule missing from the blueprint.

Check the record connections. Suppose the eligibility condition is represented, but the agent must search metadata to locate the purchase record. Inspect whether an explicit pointer connects the graph to the structured information needed to evaluate that condition.

These are diagnostic examples, not benchmark results. If all the required evidence reaches the agent, this missing-context diagnosis has not explained the failure. Retrieving evidence and using it correctly remain separate checks. Tomorrow, trace one failed question through the facts and conditions required for the correct answer. Compare that path with your graph’s structure, its record connections, and the context your agent received. Identify a missing connection or establish that the failure lies elsewhere before choosing a fix.

The most relevant passage can still produce the wrong answer when the decisive fact is a relationship away. The skill is finding where that relationship disappears, not simply retrieving more passages that look relevant.