Inference Overview
Tuna's Inference API provides access to models hosted and run by Tuna. This type of service is commonly called serverless inference: you send requests without having to deploy models or manage servers. Usage is billed per token.
You can use the OpenAI SDKs with Tuna's supported endpoints. See OpenAI compatibility for client setup and supported features.
Base URL
API requests use this base URL:
https://api-us.tuna.ai/v1Requests require an API key in the Authorization header, except when listing models.
Endpoints
| Endpoint | What it does |
|---|---|
POST /v1/chat/completions |
Sends a conversation to a model and returns the model's reply. |
GET /v1/models |
Lists the models Tuna serves, with their ids and prices. |
Model IDs
Tuna's model ids start with tuna-us/, like tuna-us/moonshotai/kimi-k3-ultrafast. The prefix provides a regional routing guarantee for GPU inference and KV caches in US data centers. Code examples in these docs use the placeholder tuna-us/example-lab/example-model-id in place of a real id. See List Model Endpoints.
What the API Can Do
| Page | What it covers |
|---|---|
| Streaming | Get the reply in pieces while the model writes it. |
| Tool calling | Let the model ask your code to run functions. |
| Reasoning effort | Choose a reasoning effort level to balance response quality, latency, and token usage. |
| Image input | Send images to models that support image input. |
| Input caching | Pay the lower cached input price for a repeated prompt start. |
| Advanced model control | Set max_tokens, temperature, top_p, stop, and the penalties. |
| Structured outputs | Make replies match a JSON Schema. Coming soon. |
| Assistant prefill | Write the start of the model's reply. Coming soon. |
Pricing
Serverless requests are billed per token, at the model's input, cached input, and output prices. Charges are paid from your organization's balance of credits. See Billing.
Privacy
The Inference API has Zero Data Retention (ZDR). Tuna does not store prompts or replies. See Privacy and ZDR.
Errors and Limits
All the API keys in an organization share one rate limit. A failed request returns an HTTP status and an error message. See Errors for what each status means, and Retries for which ones to retry.