Everything Agents

Lesson 12 of 14 · 9 min

Where this fits in AI research

Neuro-symbolic Type-2 hybrid. ReAct + constrained generation. Not model-based RL, not Soar.

The previous eleven lessons were about building one agent well. This one zooms out. Guided determinism is not a new idea. It is a productized point in a well-studied design space. Naming the lineage helps you reason about trade-offs others have already documented, reach for the right research when easy fixes stop working, and avoid claiming inheritance from research that does not actually apply.

Where Agent Script sits in AI researchFour overlapping ellipses representing four research lineages: neuro-symbolic AI (Kautz, Garcez and Lamb), ReAct and tool use (Yao, Schick), constrained generation (Willard and Louf, Park), and cognitive architectures (CoALA / Sumers et al.). Agent Script sits at the central intersection where all four overlap. It is defined by the convergence, not downstream of it.Agent ScriptNeuro-symbolic AIKAUTZ, GARCEZ & LAMBReAct + tool useYAO, SCHICKConstrained generationWILLARD & LOUF, PARKCognitive architecturesCOALA / SUMERS ET AL.

For an industry write-up of the same idea from the team building Agentforce, see Agentforce's Agent Graph: toward guided determinism with hybrid reasoning on the Salesforce Engineering blog.

How it maps to the literature

Neuro-symbolic AI: the primary frame

The defining manifesto is Garcez & Lamb, Neurosymbolic AI: The 3rd Wave (arXiv:2012.05876, 2020). Their criterion for a neurosymbolic system is "sound reasoning under the constraint of the symbolic component while leveraging the learning capacity of the neural component." That is exactly what available when does to an otherwise unconstrained LLM.

Marcus 2020 (arXiv:2002.06177) argues neural systems need "explicit symbolic machinery for manipulating variables, types, and structured representations." Agent Script's typed variables, typed Invocable I/O, and explicit FSM topic transitions are a direct embodiment.

Model-based agents: not the right label

In the Russell & Norvig sense, a "model-based agent" maintains internal state. Agent Script's variables fit informally. But in the model-based RL sense (Ha & Schmidhuber, World Models, arXiv:1803.10122; Hafner et al., Dreamer, arXiv:1912.01603) you need a learned transition model. Agent Script has none. "Stateful agent" is fine. "Model-based agent" overclaims.

LLM tool use and ReAct: the proximate engineering lineage

  • Yao et al., ReAct (arXiv:2210.03629, ICLR 2023). The Thought-Action-Observation loop is the direct ancestor of planner-picks-action / receive-typed-return-value architectures.
  • Schick et al., Toolformer (arXiv:2302.04761, NeurIPS 2023). Established typed external function calls as a first-class LLM primitive.
  • Huang et al., Inner Monologue (arXiv:2207.05608, CoRL 2022). Closed-loop feedback from environment state into the LLM planner.

A defensible thesis: ReAct alone is not neurosymbolic. It is just LLM plus tools. Adding available when gates over the action space is what tips ReAct-style tool use into genuinely neurosymbolic territory.

Constrained generation: the closest formal analogue to available when

  • Willard & Louf, Efficient Guided Generation (arXiv:2307.09702, 2023). The Outlines paper. Frames constrained generation as finite-state-machine masking of the LLM's output distribution.
  • Park et al., Grammar-Aligned Decoding (arXiv:2405.21047, NeurIPS 2024). Formalizes the distribution-distortion problem when you constrain an LLM to a grammar.
  • Beurer-Kellner et al., Prompting Is Programming (LMQL) (arXiv:2212.06094, PLDI 2023). The closest DSL-shaped precedent.

Cognitive architectures: the synthesis frame

Sumers et al., Cognitive Architectures for Language Agents (CoALA) (arXiv:2309.02427, TMLR 2024). CoALA explicitly frames LLM agents as cognitive architectures with memory, action space, and decision procedure, drawing the line back to Soar (Laird, 2012) and ACT-R (Anderson, 2007). Agent Script maps cleanly:

  • Working memory ≈ Agent Script variables
  • External action space ≈ Invocable Apex actions
  • Decision procedure ≈ topic FSM + available when + LLM planner

References

  1. Kautz, H. "The Third AI Summer." AI Magazine 43(1), 2022. doi.org/10.1609/aaai.12000
  2. Garcez, A. d'A. & Lamb, L. C. "Neurosymbolic AI: The 3rd Wave." arXiv:2012.05876, 2020.
  3. Sarker, M. K. et al. "Neuro-Symbolic AI: Current Trends." arXiv:2105.05330, 2021.
  4. Marcus, G. "The Next Decade in AI." arXiv:2002.06177, 2020.
  5. Yao, S. et al. "ReAct." ICLR 2023. arXiv:2210.03629.
  6. Schick, T. et al. "Toolformer." NeurIPS 2023. arXiv:2302.04761.
  7. Huang, W. et al. "Inner Monologue." CoRL 2022. arXiv:2207.05608.
  8. Willard, B. T. & Louf, R. "Efficient Guided Generation." arXiv:2307.09702, 2023.
  9. Park, K. et al. "Grammar-Aligned Decoding." NeurIPS 2024. arXiv:2405.21047.
  10. Beurer-Kellner, L. et al. "LMQL." PLDI 2023. arXiv:2212.06094.
  11. Sumers, T. R. et al. "CoALA." TMLR 2024. arXiv:2309.02427.
  12. Russell, S. & Norvig, P. Artificial Intelligence: A Modern Approach, 4th ed., Pearson, 2020. aima.cs.berkeley.edu
  13. Salesforce Engineering. "Agentforce's Agent Graph: toward guided determinism with hybrid reasoning." engineering.salesforce.com