Tool-Calling Design Patterns for Search-Grounded LLM Agents

A model that can call a search tool is not the same thing as an agent that can be trusted to use one well. Connecting an LLM to a search API lets the model reach the live web instead of relying only on what it learned in training. But reasoning and retrieval are separate skills, and a model that can call a tool without a pattern governing when and how to call it will do so without any real strategy. None of these failures come from a bad model or a bad search API. They come from the absence of structure around the two. The fix is architectural: treat the LLM as one reasoning component inside a larger system, not as an oracle that happens to have a search button bolted on. Which pattern governs that system, and why, is the question the rest of this piece works through.

How search results move through an agent trajectory

Every search-grounded agent runs the same four-phase loop regardless of the differences between patterns. A query goes out to a search API. The results that come back get extracted into structured facts. An LLM analyzes and synthesizes across those facts. And the agent generates a response with citations attached. This sequence, search, extraction, analysis, generation, is the shared mechanical skeleton that ReAct, Plan-and-Execute, Reflection, and every subagent pattern all build on top of.

Keeping search and generation as separate steps is what makes the whole system auditable. Collapse the two together and accuracy and provenance both become a single opaque LLM output: there's no way to tell whether a wrong answer came from bad retrieval or bad synthesis.

Extraction deserves its own scrutiny, because snippets alone often aren't enough for real analysis. Agents that need more than surface facts have to selectively fetch the full page and convert it into a clean format, typically Markdown, before it ever reaches the model. That step is required infrastructure for any task that goes past surface-level fact-checking, not a nice-to-have for squeezing out extra accuracy.

Multiple search calls start feeding into each other across a trajectory once an agent moves past single queries. Research out of the University of York, working with Qwen3-32B, found that this dependency structure is linearly decodable from the model's residual stream by layer 14, well above what a random-label control would predict. In plain terms, the model is internally tracking which earlier call's output a later call depends on, whether or not the agent's architecture ever makes that tracking explicit.

That finding has a direct implication for which pattern to build. Multi-hop search tasks, where later queries genuinely depend on what earlier ones turned up, are where explicit dependency management earns its keep. That distinction is the bridge into comparing the patterns themselves.

ReAct: the default pattern

ReAct, short for Reasoning and Acting, is the pattern most developers reach for first, and for good reason: it's the dominant execution mechanism behind search-grounded agents in production today. The agent alternates between a Thought step, an Action step where it issues a search call, and an Observation step where it reads what came back, looping through that cycle until it decides it has enough to answer the original question.

Three things make ReAct well suited to search grounding specifically. And when an observation contradicts something the agent assumed earlier, it can revise its next query on the spot, correcting its reasoning mid-trajectory instead of carrying a wrong assumption through to the final answer.

The honest cost of ReAct is that every loop requires a full LLM call, and the thought chains the pattern generates grow the context window with each additional turn. Latency and token cost scale with how complex the task is, predictably but unavoidably. That's not a flaw so much as the price of the pattern's flexibility.

ReAct earns the default slot when the task is open-ended and the retrieval path can't be known ahead of time, when the reasoning chain needs to be interpretable, or when the task is simple enough that a single sequential loop finishes in acceptable time. It's the wrong choice when a task actually decomposes into several independent sub-questions that could be retrieved in parallel, since ReAct's loop is sequential by nature and can't exploit that structure. It's also the wrong choice when the final answer needs to be checked against the retrieved content rather than simply synthesized from it, a job ReAct has no built-in mechanism to do.

Plan-and-Execute: when the retrieval path can be mapped before the first search call

Plan-and-Execute takes a different architectural bet. Instead of one agent reasoning and acting in the same loop, the work splits into a planner LLM that produces the full multi-step retrieval plan up front, and executor components that carry out individual search calls and report results back. Each role stays bounded.

That separation pays off in a few concrete ways. A smaller, cheaper model can run those steps while the expensive model is reserved for planning and final synthesis.

The dependency-graph finding from the previous section applies here almost literally. Where ReAct leaves the dependency structure implicit in the model's internal state, Plan-and-Execute externalizes it: the plan itself is the explicit graph, and the executors are simply operating on its nodes. That matches what the York research found the model is already representing internally, giving the system a dependency structure a developer can inspect and debug directly.

The constraint is just as important as the benefit. Ramp's use of this pattern is the clearest production example: tasks where the sub-questions are stable enough to plan before a single search call goes out, and where parallel execution meaningfully cuts latency compared to a sequential loop working through the same list one item at a time.

Reflection: verifying what search returned before committing to an answer

Neither ReAct nor Plan-and-Execute checks its own work. Reflection fills that gap by adding a critic step after the agent produces a candidate answer. The critic, which can be the same model under a different prompt or a separate model entirely, evaluates the draft answer against the sources that were actually pulled, looking for gaps, contradictions, or claims the retrieved content doesn't support. It either approves the answer or sends it back for another round of retrieval and synthesis.

This matters because the synthesis step itself is a point of failure that retrieval quality alone can't catch. Reflection targets the layer of error introduced by the model's own synthesis.

Financial and legal research workflows make this concrete. The requirement that every claim in a research output be reconciled with a specific retrieved source is, in effect, a production version of the Reflection pattern already baked into the task. An agent producing financial research isn't just expected to generate an answer. It has to tie each claim back to a document that supports it, because an unverifiable output has no place in a workflow where someone is going to act on it.

The tradeoff is latency. Reflection belongs in workflows where accuracy is the binding constraint and users can tolerate the extra time it takes to get there, not in low-stakes, high-volume interactions where speed matters more than a second pass.

Subagent patterns: how search tools scale when one agent is not enough

At some point, a single agent running a single search tool stops being able to keep up with the task in front of it. That point arrives when a task needs many retrievals running in parallel, when different sub-questions call for genuinely different domain knowledge, or when one agent's output needs to feed directly into another agent downstream. At that scale, the question changes from which pattern an agent should run internally to how retrieval and reasoning get distributed across multiple agents, each handling a bounded slice of the problem.

Four subagent coordination patterns cover this design space, organized by how much control the main agent keeps over a subagent's lifecycle. The Inline Tool pattern treats a subagent as a function call: the main agent calls a tool, that tool spawns a subagent, and the result comes back as a normal tool response. From the main agent's point of view, calling a subagent looks identical to calling any other tool. Within this pattern, subagents can run sequentially for tasks with ordered dependencies, in parallel for genuinely independent sub-questions, or in a loop that keeps iterating until a quality threshold is met.

Morgan Stanley's wealth management AI assistant and the broader category of anti-money-laundering and know-your-customer multi-agent systems both illustrate this argument in production terms. Complex compliance and advisory workflows don't resolve into one agent doing everything well. They decompose naturally into specialized agents, each grounded in a different retrieval domain, with an orchestrator coordinating the handoffs between them. That decomposition is the natural bridge into the next question: once a system has multiple patterns and multiple agents available, how does a team actually decide which to use where?

Tool-call schema design and production survival

The tool call itself is the one shared piece of infrastructure that every pattern above depends on. Every pattern depends on the agent emitting a structured payload, typically JSON, naming a function and its arguments, which the runtime then executes. If that schema is ambiguous, unversioned, or missing error handling, the pattern on top of it fails no matter how well the reasoning loop was designed.

A production-grade tool-calling schema needs several things in place at once: a canonical schema definition, whether that's JSON Schema, Pydantic v2, or zod; per-provider transformers that adapt that schema to each target runtime, since OpenAI's strict mode, Anthropic's input_schema, Bedrock's toolConfig, and Gemini's OpenAPI format all expect slightly different shapes; a standardized error envelope so failures come back in a predictable form; idempotency keys so a retry doesn't fire the same search twice; caps on parallel-call fan-out to keep cost under control; a semver-versioned tool registry with CI diff gates to catch breaking changes before they ship; and OpenTelemetry GenAI spans so the whole trajectory can be observed in production.

Measuring whether tool calls are working at all requires looking at three separate dimensions across the trajectory. Did the agent call the right tool in the first place, and were the arguments it passed in accurate? And did the tool return what was expected, with the agent actually using that output correctly afterward? Stopping at whether the final answer was right misses almost everything about trajectory quality along the way.

BFCL v4, released in July 2025, is built around a holistic agentic evaluation model covering three major agentic areas, which makes it the benchmark that actually matters for agent infrastructure teams, as opposed to general benchmarks like MMLU or HumanEval that measure factual recall and code generation rather than function selection and parameter filling across tool turns.

The dependency-graph finding from earlier carries a direct schema implication here. Since the model already tracks internally which earlier call's output a later call depends on, a schema that makes that provenance explicit, passing a call ID or a named output field rather than a raw string copied between calls, lines up with how the model is already representing the relationship. That alignment tends to reduce errors specifically in multi-hop retrieval chains, where a mistake in one call's input can quietly propagate into every call after it.

Choosing and combining patterns based on the constraints that govern your agent

Picking a pattern isn't a matter of ranking ReAct, Plan-and-Execute, Reflection, and subagent orchestration against each other in the abstract and crowning a winner. It comes down to where a given task sits on three constraints at once: whether the retrieval path is knowable in advance, what the acceptable bounds are on latency, cost, and accuracy in production, and whether the output needs citations, source reconciliation, or an audit trail to be usable.

A rough decision map follows from everything above. Open-ended, single-topic queries where the retrieval path isn't knowable up front, and where interpretability of the reasoning chain matters, point toward ReAct as the default loop. Multi-step tasks where the sub-questions are known before the first search call goes out, and where running retrieval in parallel would cut real latency, point toward Plan-and-Execute, with smaller models handling the individual executor steps. Any output that has to be reconciled against its sources before it reaches a human, financial research, legal analysis, compliance reporting, calls for a Reflection pass layered on top of whichever base pattern is running. And tasks that need domain-specialized retrieval spanning multiple knowledge areas, or whose output feeds directly into other agents, call for decomposition into subagents using the Inline Tool pattern, each one carrying its own search tools.

These patterns aren't mutually exclusive, and treating them that way undersells what's possible. A Plan-and-Execute orchestrator can spawn ReAct subagents to handle the open-ended sub-questions buried inside an otherwise well-structured plan. A ReAct loop can trigger a Reflection critic before it commits to a final answer. A multi-agent pipeline can mix patterns freely by agent role, running Plan-and-Execute at the orchestration layer while individual subagents run ReAct internally. The patterns are building blocks, not a menu with one correct item.

This depends on the infrastructure layer holding up its end: canonical schemas, provider-specific transformers, idempotency, observability, and benchmarks like BFCL v4 that measure tool selection and argument accuracy alongside the final answer. Choosing the right pattern and building the infrastructure that pattern depends on are two halves of the same architectural decision, and neither one holds up in production without the other.