Tool calling

Tool calling is the standard OpenAI shape: you describe functions, the model returns a call, you run it, you send the result back, and it continues.

Define the tools

{
  "model": "cog-fast",
  "messages": [{ "role": "user", "content": "What changed in this host's scheduled tasks?" }],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "read_scheduled_task",
        "description": "Return the XML of a scheduled task by name.",
        "parameters": {
          "type": "object",
          "properties": { "name": { "type": "string" } },
          "required": ["name"]
        }
      }
    }
  ]
}

The description is the prompt. A vague description is the most common reason a capable model picks the wrong function, and no amount of retrying fixes it.

Run the loop

1. Send the messages and the tool definitions.
2. If the reply has finish_reason: "tool_calls", run each call.
3. Append the assistant message, then one tool message per result, each carrying the tool_call_id of the call it answers.
4. Send again. Repeat until finish_reason is stop.

Keep the assistant's tool-call message in the history exactly as it arrived. Rebuilding it by hand and dropping a field is the second most common cause of a loop that works twice and then fails.

Several calls in one turn

The model may return more than one call. Run them, and answer all of them in the same follow-up request. Answering three of four leaves the conversation in a state the model cannot continue from.

Validate before you execute

A model-generated function call is untrusted input. Treat the arguments as you would treat a form submission from the internet: check the types, check the ranges, and never interpolate them into a shell command or a filesystem path without validation.

Choosing the model

cog-fast is the right default for a loop that runs many turns, because latency compounds and the cost of a stalled run is higher than the cost of a slightly worse decision. cog-pro earns its place when the loop is short and the decision is hard.

Client support

Most clients here implement tool calling themselves and need no configuration. Two exceptions worth knowing:

• Roo Code uses native tool calling only, with no XML fallback, so it fails rather than degrades if a call comes back malformed.
• Continue.dev needs capabilities: [tool_use] declared explicitly, because its capability detection cannot infer it from an unfamiliar model name.

Both are noted on their pages in Tools.