Tool Calling

What Is Tool Calling?

Tool calling is a language model's ability to invoke external functions, such as running a command, querying an API, or reading a file, instead of only generating text. The model emits a structured request naming a tool and its arguments, the host application executes it, and the result feeds back into the model's next step.

Key Takeaways

• The model never executes anything itself. It produces a structured intent, typically JSON naming a function and arguments, and the surrounding application performs the actual call.
• Tool calling is the mechanism that turns a language model into an agent. The loop of call, observe result, decide next action is the engine underneath every agentic system. Adoption reflects that shift: LangChain's LangSmith platform data showed 21.9% of LLM traces involving tool calls in 2024, up from an average of 0.5% in 2023 [1].
• Tools are defined by schemas plus descriptions, and the descriptions are read by the model, so writing them is prompt engineering as much as API design.
• Every tool is attack surface. Whatever a tool can do, a manipulated model can do, which is why permissions and sandboxing sit alongside tool calling in production systems.

How It Works

The application declares available tools in the request, each with a name, a natural-language description, and a parameter schema, usually JSON Schema. During generation, the model can choose to respond with a tool call instead of prose: a structured block naming the tool and supplying arguments that fit the schema. Providers train models specifically for this, so emitting well-formed calls is a native capability of every major model line in 2026, under the interchangeable names tool use and function calling. The skill is benchmarked in its own right: UC Berkeley's Function Calling Leaderboard evaluates models on 2,000 question-function-answer pairs spanning Python, Java, SQL, REST APIs, and JavaScript across nine categories [2].

Execution belongs to the host. The application receives the call, runs the real function, and appends the result to the conversation as a tool result message. The model reads it and continues: answering the user, calling another tool, or chaining several calls in sequence. Modern models handle parallel calls, requesting several independent tools at once, and multi-step sequences where each call depends on the previous result. That loop, reason then act then observe, is the pattern formalized by ReAct-style agents and inherited by every coding agent since.

Two layers commonly sit on top. Model Context Protocol standardizes where tool definitions come from, letting third-party servers supply tools at runtime instead of hard-coding them into the application. And because argument generation is just constrained text generation, providers increasingly back it with strict schema enforcement, the same machinery behind structured outputs, so calls parse reliably instead of failing on malformed JSON.

Example

A user asks an engineering assistant, "why did checkout latency spike at 2pm?" The application has given the model three tools: query_metrics, search_logs, and list_deploys. The model first calls list_deploys with the service name and a time range, sees a deploy at 13:52, then calls query_metrics to confirm the spike started minutes later, then calls search_logs filtered to the new release and finds connection-pool exhaustion errors. Only after three call-and-observe rounds does it produce prose: the deploy shrank the pool size, here is the offending config change. No single completion could have answered that; the tool loop did.

What People Get Wrong

The persistent misconception is that the model runs the tools, which leads people to either overestimate the risk of the model itself or underestimate the responsibility of the application. The model only ever emits a request. Your code decides whether to execute it, with what credentials, in what environment, and with what human approval. That boundary is exactly where AI agent security is enforced: validation of arguments, least-privilege credentials, and confirmation gates for destructive actions all live in the thin layer between the model's request and the real function.

FAQ

Is function calling the same as tool calling? Yes. Function calling was OpenAI's original name for the feature in 2023, and tool calling became the broader industry term as capabilities expanded beyond simple functions. Documentation uses the terms interchangeably.

What is the relationship between LLM tool use and MCP? Tool use is the model-level capability; the Model Context Protocol is a distribution standard for tools. MCP servers advertise tool schemas to the application, which passes them to the model like any other tool definition. The model cannot tell the difference between a built-in tool and an MCP-provided one.

How many tools can a model handle? There is no hard limit, but selection accuracy degrades as tool counts grow, since every definition consumes context and similar tools blur together. Production systems curate small, well-described toolsets per task, or search a larger catalog and load only relevant definitions on demand.

Sources

1. LangChain. "State of AI 2024 report, share of LLM traces involving tool calls on LangSmith." https://www.langchain.com/blog/langchain-state-of-ai-2024. Accessed August 2026.
2. Berkeley Gorilla team, UC Berkeley. "Berkeley Function Calling Leaderboard dataset composition and evaluation categories." https://gorilla.cs.berkeley.edu/blogs/8_berkeley_function_calling_leaderboard.html. Accessed August 2026.