Everything Agents

Lesson 9 of 14 · 7 min

Bug 2: the planner that lies

@outputs.X is what the planner says Apex returned. Vague schemas plus sticky context produce convincing fabrications.

The first bug was an ops problem: the agent user couldn't see the class. This second bug lives one layer deeper, inside the planner itself, and it is the deepest insight from this build. @outputs.X is what the planner says the action returned, not what Apex literally returned. The LLM can summarize, paraphrase, or fabricate a tool result, and that fabricated value becomes your variable's new value.

A real example we hit

User typed: "My email is info@salesforce.com" (not a known customer). Apex returned customer_id="", verified="false". But the trace showed:

verified_flag = "true"            # fabricated
customer_id   = "cust_100"        # alice's ID, the LLM had seen earlier

after_reasoning then saw verified_flag == "true" and flipped customer_verified = True. A non-customer was now verified.

The seam where hallucination entersApex returns one value. The planner substitutes a different value before set clauses fire. The variable ends up holding the planner's substitution, not the Apex byte stream.APEX RETURNEDcustomer_id = ""verified = "false"the truthPLANNER SUBSTITUTEScustomer_id = ""verified = "false"↓ replaced withcustomer_id = "cust_100"verified = "true"prior turns leak in(alice was verified earlier in the chat)VARIABLEScustomer_id ="cust_100"verified = "true"stored as if truethe seam where hallucination enters@outputs.X is what the planner says, not what Apex returned

Why this happens: three converging factors

  1. Sticky context. The LLM has alice's cust_100 in its conversation window from earlier turns and re-emits it.
  2. Vague output schema. The action's outputs: block had no description: for the fields, so the LLM's prior is "a customer ID probably looks like cust_100."
  3. No raw-output enforcement. Agent Script does not force @outputs.X to be the literal Apex return.