{"id":71,"date":"2026-09-16T23:04:52","date_gmt":"2026-09-16T15:04:52","guid":{"rendered":"https:\/\/wp.qoraapi.com\/ai-function-calling-tool-use\/"},"modified":"2026-09-20T03:53:07","modified_gmt":"2026-09-19T19:53:07","slug":"ai-function-calling-tool-use","status":"publish","type":"post","link":"https:\/\/qoraapi.com\/blog\/ai-function-calling-tool-use\/","title":{"rendered":"AI Function Calling Explained: Tools, JSON Schema, and the Tool-Use Loop"},"content":{"rendered":"

Function calling<\/strong> is what turns an AI API from a text generator into something that can take action in your system. The model does not actually run your code \u2014 it reads a JSON Schema description of the tools you offer, decides when one of them applies, returns a structured argument object that matches the schema, and lets your code do the real work. Your code runs the function, ships the result back into the next model turn, and the loop continues until the model decides the answer is complete.<\/p>\n

This guide walks through what function calling is in practice, the loop that makes it work, how to define tools with JSON Schema, how OpenAI, Claude, and Gemini differ in their conventions, and the failure modes that catch teams the first time they put tools in front of a model. The pattern is the same everywhere \u2014 it is the contract, not the SDK, that matters. Once you understand the loop, switching providers becomes a matter of changing a base_url<\/code>, which our guide to the OpenAI-compatible API<\/a> covers in detail.<\/p>\n

Why function calling matters<\/h2>\n

A model that can only generate text is a writer. A model that can call functions is an agent<\/em>. The difference is enormous: with tools, the model can fetch live information, query your database, run calculations, take actions in third-party services, and produce outputs that have real effects in the world. Without tools, everything the model knows is what was in its training data plus whatever you stuff into the prompt.<\/p>\n

Function calling is the bridge. The model proposes a structured call, your code executes it, and the result becomes part of the next prompt. The model stays in charge of deciding what<\/em> to do and when<\/em>; your code stays in charge of how<\/em>. That separation is what makes the pattern both safe and powerful.<\/p>\n

The five-step function-calling loop<\/h2>\n

Function calling is not a single request \u2014 it is a loop. Almost every provider implements the same five steps, and once you know them the API documentation becomes much easier to read:<\/p>\n

    \n
  1. Define<\/strong> \u2014 describe the functions your code can run, as a JSON Schema with name, description, and parameters.<\/li>\n
  2. Send<\/strong> \u2014 pass the schema alongside your prompt. The model reads the conversation and decides whether to call one or more of the tools.<\/li>\n
  3. Detect<\/strong> \u2014 if the model returned a tool call (rather than a final answer), parse it. The arguments are a JSON object that conforms to the schema you supplied.<\/li>\n
  4. Execute<\/strong> \u2014 run the function in your code. Treat this like any other user input: validate, sandbox, and log.<\/li>\n
  5. Return<\/strong> \u2014 ship the result back to the model in the next turn. The model uses it to produce either another tool call or a final answer.<\/li>\n<\/ol>\n

    The loop continues until the model stops producing tool calls and writes a final answer. In production systems it is common to cap the number of iterations to prevent runaway loops \u2014 usually between five and twenty rounds, depending on how expensive each step is.<\/p>\n

    Defining tools with JSON Schema<\/h2>\n

    The schema you give the model is its only window into what it can do. A good description reads like a function docstring that another developer would understand: what the tool does, when to use it, what each parameter means, and what the return shape looks like.<\/p>\n

    tools = [\n    {\n        \"type\": \"function\",\n        \"function\": {\n            \"name\": \"get_order_status\",\n            \"description\": (\n                \"Return the current fulfilment status of a customer order. \"\n                \"Use this whenever the user asks about shipping, delivery, \"\n                \"tracking, or whether an order has shipped.\"\n            ),\n            \"parameters\": {\n                \"type\": \"object\",\n                \"properties\": {\n                    \"order_id\": {\n                        \"type\": \"string\",\n                        \"description\": \"The order identifier, e.g. 'ORD-12345'.\",\n                    },\n                },\n                \"required\": [\"order_id\"],\n            },\n        },\n    },\n    {\n        \"type\": \"function\",\n        \"function\": {\n            \"name\": \"search_knowledge_base\",\n            \"description\": (\n                \"Search the public help-centre articles for a query and \"\n                \"return the three most relevant snippets.\"\n            ),\n            \"parameters\": {\n                \"type\": \"object\",\n                \"properties\": {\n                    \"query\": {\"type\": \"string\"},\n                    \"locale\": {\n                        \"type\": \"string\",\n                        \"enum\": [\"en\", \"fr\", \"de\"],\n                        \"default\": \"en\",\n                    },\n                },\n                \"required\": [\"query\"],\n            },\n        },\n    },\n]<\/code><\/pre>\n
    Two things matter more than the schema itself. First, the description is the instruction the model uses to choose this tool over another, so make it specific about when<\/em> to call it, not just what it returns. Second, keep the schema tight: a long list of overlapping tools confuses the model and degrades the quality of every call.<\/p>\n
    Detecting and executing the call<\/h2>\n
    When the model decides to act, it returns an assistant message containing one or more tool calls rather than a final answer. Your code reads the call, runs it, and feeds the result back in. The pattern below runs across every OpenAI-compatible provider with almost no changes:<\/p>\n
    import json\nfrom openai import OpenAI\n\nclient = OpenAI()  # base_url points at an OpenAI-compatible endpoint\n\ndef chat_with_tools(messages, tools, max_steps=8):\n    for _ in range(max_steps):\n        resp = client.chat.completions.create(\n            model=\"gpt-4o\",\n            messages=messages,\n            tools=tools,\n        )\n        msg = resp.choices[0].message\n\n        # Final answer: no tool calls, just text.\n        if not msg.tool_calls:\n            return msg.content\n\n        # Otherwise, append the assistant's tool-call message and run each.\n        messages.append(msg)\n\n        for call in msg.tool_calls:\n            args = json.loads(call.function.arguments)\n            result = dispatch(call.function.name, args)        # your code\n            messages.append({\n                \"role\": \"tool\",\n                \"tool_call_id\": call.id,\n                \"content\": json.dumps(result),\n            })\n    raise RuntimeError(\"tool loop did not converge\")\n\ndef dispatch(name, args):\n    \"\"\"Map a tool name to the function that actually runs in your system.\"\"\"\n    if name == \"get_order_status\":\n        return {\"status\": \"shipped\", \"tracking\": \"1Z999...\", \"eta\": \"2026-09-20\"}\n    if name == \"search_knowledge_base\":\n        return {\"snippets\": [\"...\"] * 3}\n    raise ValueError(f\"unknown tool: {name}\")<\/code><\/pre>\n
    The dispatch<\/code> function is the security boundary. Treat each argument as if it came from a user: validate the shape, escape strings, check authorisation, and never assume the model’s choice was correct. This is the place where prompt injection turns into real harm if you do not lock it down.<\/p>\n
    Parallel tool calls<\/h2>\n
    Most providers will return multiple<\/em> tool calls in a single assistant turn when those calls are independent \u2014 for example, fetching the weather for three cities at once. Calling them serially works but is slow; calling them concurrently with asyncio.gather<\/code> turns an N-step tool loop into one step from the model’s perspective:<\/p>\n
    async def run_calls(calls):\n    return await asyncio.gather(*(dispatch(c.function.name, c.function.arguments) for c in calls))<\/code><\/pre>\n
    Parallel calls are particularly useful for retrieval \u2014 fetching several documents, several user records, or several API endpoints at once \u2014 and they are the difference between a sluggish agent and a fast one.<\/p>\n
    Streaming and function calling<\/h2>\n
    Function calls are delivered as a single, structured object \u2014 you cannot stream the arguments one token at a time the way you stream text. What can<\/em> stream is everything around them: the model’s reasoning as it decides which tool to call, the final assistant text, and the tool results after execution. The practical pattern is to enable streaming and let the SDK accumulate the tool call once the stream completes:<\/p>\n
    stream = client.chat.completions.create(\n    model=\"gpt-4o\",\n    messages=messages,\n    tools=tools,\n    stream=True,\n)\n\nfor chunk in stream:\n    delta = chunk.choices[0].delta\n    if delta.content:\n        print(delta.content, end=\"\", flush=True)   # visible answer\n    # tool_calls arrive fully formed once the model is done thinking<\/code><\/pre>\n
    If perceived latency matters, this is where to focus; the tool call itself is fast once it returns, and showing the model “thinking” while it picks the right function keeps users engaged.<\/p>\n
    Cross-provider differences<\/h2>\n
    All three major providers implement the same loop, but they use different vocabulary and shapes. If you call more than one, an abstraction layer is worth the upfront cost:<\/p>\n
    \n\n\n\n\n\n\n\n\n
    Concept<\/th>\nOpenAI<\/th>\nAnthropic Claude<\/th>\nGoogle Gemini<\/th>\n<\/tr>\n<\/thead>\n
    Tool definition<\/td>\ntools[].function<\/code> with name<\/code>, description<\/code>, parameters<\/code> (JSON Schema)<\/td>\ntools[].name<\/code>, description<\/code>, input_schema<\/code><\/td>\ntools[].functionDeclarations<\/code> with name<\/code>, description<\/code>, parameters<\/code> (OpenAPI \/ JSON Schema)<\/td>\n<\/tr>\n
    Tool call surface<\/td>\nmessage.tool_calls[]<\/code> with function.name<\/code>, function.arguments<\/code><\/td>\ncontent[]<\/code> blocks of type tool_use<\/code><\/td>\nfunctionCall<\/code> on the candidate<\/td>\n<\/tr>\n
    Returning results<\/td>\nAppend a role:\"tool\"<\/code> message per call<\/td>\nAppend a role:\"user\"<\/code> turn with tool_result<\/code> blocks<\/td>\nSend a functionResponse<\/code> part in the next turn<\/td>\n<\/tr>\n
    Forcing a call<\/td>\ntool_choice: \"required\"<\/code><\/td>\ntool_choice: {\"type\": \"tool\", \"name\": \"...\"}<\/code><\/td>\ntool_config<\/code> with mode<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/figure>\n
    The JSON Schema for the tool itself is portable. The wrapper format is not. If you are routing through an OpenAI-compatible endpoint<\/a>, your code speaks OpenAI’s shape and the gateway translates to whichever upstream you choose \u2014 which is one of the practical benefits of consolidating on a single contract.<\/p>\n
    Common failure modes<\/h2>\n
    Function calling has its own set of recurring bugs. Most production incidents I have seen come from one of these:<\/p>\n