Tool Calling

Tool calling lets a model request that an application run a function. You describe the available functions in the request. The model returns a function name and arguments. The application validates the request, runs the function when appropriate, and sends the result back to the model.

Compatible with OpenAI Tool Calling

Tuna uses the industry-standard tool calling format from the OpenAI Chat Completions API. The tools list, tool_choice, the tool_calls in replies, and the tool messages you send back all have the same fields as OpenAI's. Code, SDKs, and agent frameworks written for OpenAI tool calling work with Tuna after you set Tuna's base URL and a Tuna model ID. See OpenAI Compatibility for the rest of the API.

How It Works

1. Describe Your Tools

   Send a tools list with the request. Each tool has a name, a description of what it does and when to use it, and its parameters as a JSON Schema.

   tools = [{
       "type": "function",
       "function": {
           "name": "get_weather",
           "description": "Get the current weather in a city.",
           "parameters": {
               "type": "object",
               "properties": {
                   "city": {"type": "string", "description": "The city name, like Boston"},
               },
               "required": ["city"],
           },
       },
   }]
   messages = [{"role": "user", "content": "What's the weather in Boston?"}]
   
   response = client.chat.completions.create(
       model="tuna-us/example-lab/example-model-id",
       messages=messages,
       tools=tools,
   )

2. Read the Tool Calls

   When the model wants a tool, the reply has tool_calls instead of text, and finish_reason is tool_calls. The arguments are a JSON string.

   {
     "role": "assistant",
     "content": null,
     "tool_calls": [
       {
         "id": "call_abc123",
         "type": "function",
         "function": {"name": "get_weather", "arguments": "{\"city\": \"Boston\"}"}
       }
     ]
   }

3. Send the Results Back

   Add the model's message to the conversation. Then run each tool and add one tool message per call, with the call's id in tool_call_id.

   import json
   
   message = response.choices[0].message
   messages.append(message)
   for call in message.tool_calls or []:
       args = json.loads(call.function.arguments)
       result = get_weather(**args)  # Your own function
       messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})

4. Get the Final Reply

   Send the conversation again, with the same tools. The model reads the results and answers, or asks for more tool calls. An agent repeats steps 2 to 4 until finish_reason is stop.

   response = client.chat.completions.create(
       model="tuna-us/example-lab/example-model-id",
       messages=messages,
       tools=tools,
   )
   print(response.choices[0].message.content)

Choosing When Tools Are Used

tool_choice controls whether the model calls a tool:

A model can also ask for several tool calls in one reply, for example to check the weather in two cities. Each call has its own id. Run all of them and send one tool message for each.

Streaming Tool Calls

When streaming, tool calls arrive in pieces in delta.tool_calls. The first piece of each call has its id and function.name. Later pieces add to function.arguments. Each piece has an index that says which call it belongs to, so join the pieces by index. The SDKs' stream helpers do this for you.

Tips

• Write clear descriptions. The model picks tools and fills in arguments from the names and descriptions. Say what each tool does, when to use it, and what each parameter means.
• Check the arguments. The model writes arguments itself. They can be invalid JSON, miss a required field, or have a value you don't expect. Validate them before you run anything.
• Limit what tools can do. Treat a tool call like input from a user. For example, a tool that reads files should only reach the files it needs, and a tool that spends money or deletes data should ask a person first.
• Only send the tools a request needs. A shorter list makes it easier for the model to pick the right tool, and uses fewer input tokens.
• Tools cost input tokens. Tool definitions are added to the prompt, so they are billed as input tokens on every request that includes them. They stay the same between requests, so they are usually cached input.

Tool calling works on models that were trained for it, which includes most recent open-weight models.