What is tool use / function calling?

A language model can produce plausible text about a stock price. It cannot, by itself, look one up. Function calling is the mechanism that closes that gap: the model emits a structured request naming a specific function and its arguments, an external system runs that function, and the result comes back as input the model can read.

What the model actually emits

When a model decides a tool call is needed, it does not run anything. It generates a structured output, typically a JSON object, specifying the function name and the arguments to pass:

{"function": "get_stock_price", "arguments": {"ticker": "AAPL"}}

The application around the model intercepts that output, executes the function, and injects the result back as a new message in the conversation. The model then reads that result and continues generating.

For this to work, the available functions must be described in advance, usually through JSON Schema definitions in the system prompt or API call. The model selects from that declared set. It cannot call a function it has not been told about.

Language models do not have live access to external systems by default; function calling is the deliberate mechanism that extends them.

How the execution loop runs

The full cycle is a back-and-forth:

1. The model receives a request alongside the descriptions of available functions.
2. It decides whether to call a function or respond directly. If it calls, it outputs a structured request.
3. The orchestrating system executes the function and returns the result.
4. The model reads the result and either calls another function or produces a final response.

A single user request can trigger a sequence of calls, with each result informing the arguments of the next. Some implementations let the model emit multiple calls in a single step, receiving all results before continuing, which matters for latency when two lookups are independent.

Yao et al. formalised this interplay in the ReAct framework (ICLR 2023, arXiv:2210.03629), demonstrating that interleaving reasoning with tool calls produces outputs that are more verifiable and less prone to hallucination than either alone.

How models learn to call functions

The capability is trained in, not hard-coded. Schick et al. showed in Toolformer (NeurIPS 2023, arXiv:2302.04761) that a model can be taught to decide which tool to call and with what arguments through a self-supervised approach requiring minimal human annotation per tool.

More recent models are fine-tuned on large datasets of function-calling examples, often synthetically generated. The training teaches not just how to format a call but when to stay silent: a well-trained model sends a call only when the information is not already in its context.

Schema quality sets the ceiling

The non-obvious constraint is that the model selects from what it is told. A function description that is vague or ambiguous produces unreliable calls, not because the model reasoned poorly, but because it had inadequate information to reason from. The model cannot infer what a function does beyond what the description says.

For anyone building on tool use, writing precise function descriptions is the primary engineering task. The model’s behaviour on a given call is almost entirely downstream of that quality.

The Model Context Protocol addresses this at a different layer, standardising how functions are described and discovered across systems so the same server can be reached by any compliant application without custom integration work.

Sources

• Timo Schick et al. Toolformer: Language Models Can Teach Themselves to Use Tools. NeurIPS 2023.
• Shunyu Yao et al. ReAct: Synergizing Reasoning and Acting in Language Models. ICLR 2023.