Structured Outputs (Coming Soon)

Structured outputs will constrain a model's response to the JSON Schema you provide. Check that finish_reason is stop before parsing the response, and validate the returned values for your application.

Why They Are Useful

Models are good at following format instructions, but they still make mistakes. A reply might wrap the JSON in extra text, leave out a field, or add a trailing comma. Code that parses replies then needs checks and retries.

Common uses include:

• Extracting data. Pull names, dates, and amounts from emails or documents into fixed fields.
• Classifying. Make the model pick one label from a fixed list, with enum in the schema.
• Passing data between steps. An agent or pipeline can hand the reply straight to the next piece of code.
• Filling in a UI. Render the reply as a form, table, or card without guessing at its shape.

How They Will Work

Structured outputs will use the response_format parameter from the OpenAI API:

{
  "model": "tuna-us/example-lab/example-model-id",
  "messages": [{"role": "user", "content": "Jane Doe, 34, lives in Boston."}],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "person",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "name": {"type": "string"},
          "age": {"type": "integer"},
          "city": {"type": "string"}
        },
        "required": ["name", "age", "city"],
        "additionalProperties": false
      }
    }
  }
}

The reply's content would then be JSON like {"name": "Jane Doe", "age": 34, "city": "Boston"}.

This works by constrained decoding. As the model writes each token, any token that would break the schema is ruled out, so the reply can only follow the schema. A reply can still be cut off by max_tokens, so check that finish_reason is stop.

A simpler JSON mode, {"type": "json_object"}, will make the reply valid JSON of any shape.

Until Then

• Ask for JSON in the prompt. Describe the fields you want, show an example, and parse the reply. Validate it and retry if it doesn't match.
• Use tool calling. Describe a tool whose parameters are your schema, and set tool_choice to that tool. The model's arguments then follow your schema closely, though you should still validate them.