Assistant Prefill (Coming Soon)

With assistant prefill, you write the first part of the model's reply, and the model continues from where your text ends. It is a simple way to steer a reply, because the model treats your text as something it already wrote.

How It Will Work

End the messages list with an assistant message. Its content is the start of the reply:

{
  "model": "tuna-us/example-lab/example-model-id",
  "messages": [
    {"role": "user", "content": "List three fish that live in the Atlantic, as JSON."},
    {"role": "assistant", "content": "{\"fish\": ["}
  ]
}

The reply will hold only the new text that continues the prefill, such as "Atlantic cod", "Bluefin tuna", "Haddock"]}. To get the full reply, join your prefill and the new text.

What It Is Useful For

• Setting the format. Start the reply with {, a code fence, or a heading, and the model continues in that format.
• Skipping the preamble. Start the reply with the answer itself, so the model doesn't open with "Sure, here is...".
• Beginning a plan. Supply the first step of a plan as an assistant prefill. The model will generate text that continues from that prefix.
• Continuing a partial reply. In a new request, include the original conversation and end it with the partial assistant reply. The response will contain only the continuation. Resent input is billed at the normal or cached-input rate, depending on cache reuse. Newly generated tokens are billed as output.