AI agents become difficult to maintain when business workflow and model uncertainty are written into the same code. Retries, branching, candidate generation, and verification loops accumulate around the steps that define what the application is meant to do.
Researchers from Asari AI, MIT CSAIL, and Caltech propose separating those concerns through Probabilistic Angelic Nondeterminism and a Python implementation called ENCOMPASS.
Developers write the readable “happy path” and mark unreliable model calls. A runtime engine then explores alternative execution paths using different search strategies without requiring the core workflow to be rewritten.
Branch Points Isolate Probabilistic Behaviour
ENCOMPASS uses a `branchpoint()` primitive to identify locations where a language-model call may produce several possible outcomes. At runtime, these points form a search tree.
The surrounding program remains in control. Code defines the sequence and invokes the model for bounded subtasks instead of asking the LLM to choose the entire workflow.
This architecture is more predictable and auditable than an LLM-in-control agent. A team can see which operations are fixed, where uncertainty enters, and what success criteria select among alternatives.
Without separation, adding best-of-N sampling wraps the application in loops, while beam search or refinement can require a state machine and major structural changes. High engineering cost discourages teams from testing better reliability strategies.
Search Strategy Can Change Without Rewriting Logic
The runtime can apply depth-first search, beam search, best-first search, or Monte Carlo tree search to the same program. Teams tune inference cost and accuracy independently from business logic.
Researchers demonstrated the approach on Java-to-Python repository migration. The workflow translated files, generated inputs, and validated output by execution. Fine-grained beam search at file and method level outperformed simpler sampling.
In another comparison, best-first search achieved performance similar to repeated self-reflection at lower task cost. Sophisticated search can therefore improve reliability more efficiently than adding more critique loops.
This flexibility supports risk tiers. A low-stakes internal tool may use a cheap greedy strategy, while customer-facing work applies more exhaustive search without maintaining a different workflow codebase.
Global strategy updates also simplify governance. If one search approach produces unacceptable errors, teams can replace or version it centrally instead of auditing every agent implementation.
Scoring and Side Effects Remain Hard
Search works only when the system can evaluate a path. Code translation has unit tests; creative writing, analysis, or subjective service outcomes lack equally reliable scoring functions.
Engineers must still decide where uncertainty exists and design meaningful verification. ENCOMPASS reduces control-flow complexity but does not design the agent or guarantee the evaluator is correct.
Branching also requires program state to be copied. External side effects such as database writes, payments, or API calls must not repeat every time the runtime explores another path. Operations need isolation, idempotency, or delayed commitment until a path is selected.
The framework is designed to work alongside libraries such as LangChain rather than replacing prompt, model, or tool layers. It occupies the control-flow layer where stochastic execution meets conventional software.
That position may make evaluation easier as well. Teams can compare search policies against the same program, record how much additional inference each policy consumes, and identify where alternative branches actually improve outcomes. The trade-off becomes observable: more search may raise success rates, but it also adds latency, cost, and a larger surface for side effects.
The proposal reflects a mature software-engineering principle: separate what a program does from how uncertain alternatives are searched. As inference-time compute grows, hardcoding that complexity into each application will create technical debt.
ENCOMPASS offers a promising model for durable agents. Business logic stays readable, search can evolve, and governance can reason about both separately. The result is not deterministic AI. It is a cleaner place to manage nondeterminism before it spreads through the entire codebase.
