Responses

Create OpenAI Responses API requests against Baseten Model APIs, including streaming, function tools, and structured outputs.

POST /v1/responses

Create a response

cURL

curl --request POST \
  --url https://inference.baseten.co/v1/responses \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "<string>",
  "input": "<string>",
  "instructions": "<string>",
  "max_output_tokens": 2,
  "temperature": 1,
  "top_p": 1,
  "stream": false,
  "tools": [
    {
      "type": "function",
      "name": "<string>",
      "parameters": {},
      "description": "<string>"
    }
  ],
  "parallel_tool_calls": true,
  "text": {
    "format": {
      "type": "text"
    }
  },
  "reasoning": {},
  "metadata": {},
  "truncation": "disabled"
}
'

200

{
  "id": "<string>",
  "object": "response",
  "created_at": 123,
  "status": "completed",
  "model": "<string>",
  "output": [
    {
      "type": "message",
      "id": "<string>",
      "role": "assistant",
      "content": [
        {
          "type": "output_text",
          "text": "<string>",
          "annotations": [
            {}
          ]
        }
      ],
      "status": "<string>"
    }
  ],
  "completed_at": 123,
  "usage": {
    "input_tokens": 123,
    "input_tokens_details": {
      "cached_tokens": 123,
      "cache_write_tokens": 123
    },
    "output_tokens": 123,
    "output_tokens_details": {
      "reasoning_tokens": 123
    },
    "total_tokens": 123
  },
  "incomplete_details": {
    "reason": "<string>"
  },
  "error": {},
  "metadata": {},
  "instructions": "<string>",
  "max_output_tokens": 123,
  "parallel_tool_calls": true,
  "temperature": 123,
  "top_p": 123,
  "tool_choice": "<string>",
  "tools": [
    {}
  ],
  "truncation": "<string>",
  "service_tier": "<string>",
  "text": {},
  "reasoning": {}
}

Download the OpenAPI schema for code generation and client libraries.

Model APIs accept requests in the OpenAI Responses API format at https://inference.baseten.co/v1/responses. Use it for the OpenAI Agents SDK and other clients built on the Responses API. Requests are stateless: send the full conversation as input on every call.

​

Call with the OpenAI SDK

Point the OpenAI client at Baseten and call responses.create:

Python

To call a model with the Responses API:

responses.py

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://inference.baseten.co/v1",
    api_key=os.environ["BASETEN_API_KEY"],
)

response = client.responses.create(
    model="zai-org/GLM-5.3",
    instructions="Answer in three words.",
    input="What is the capital of France?",
)

print(response.output_text)

JavaScript

To call a model with the Responses API:

responses.mjs

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://inference.baseten.co/v1",
  apiKey: process.env.BASETEN_API_KEY,
});

const response = await client.responses.create({
  model: "zai-org/GLM-5.3",
  instructions: "Answer in three words.",
  input: "What is the capital of France?",
});

console.log(response.output_text);

cURL

To call a model with the Responses API:

Request

curl https://inference.baseten.co/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $BASETEN_API_KEY" \
  -d '{
    "model": "zai-org/GLM-5.3",
    "instructions": "Answer in three words.",
    "input": "What is the capital of France?"
  }'

The remaining Python examples on this page reuse client from this section. The SDK convenience property output_text collects text from output items; the raw JSON response contains output, not an output_text field. Check status before treating that text as a complete answer.

​

Use with the OpenAI Agents SDK

Configure the Agents SDK to use the Responses API, then pass the model as an OpenAIResponsesModel. Agent model names are parsed as <provider>/<model> strings, so a Baseten slug such as zai-org/GLM-5.3 fails with Unknown prefix: zai-org if you pass it as a string. Build the model object instead:

agent.py

import asyncio
import os

from agents import Agent, Runner, set_default_openai_api, set_default_openai_client
from agents.models.openai_responses import OpenAIResponsesModel
from openai import AsyncOpenAI

client = AsyncOpenAI(
    base_url="https://inference.baseten.co/v1",
    api_key=os.environ["BASETEN_API_KEY"],
)

set_default_openai_client(client)
set_default_openai_api("responses")

MODEL = OpenAIResponsesModel(model="zai-org/GLM-5.3", openai_client=client)

agent = Agent(
    name="Assistant",
    model=MODEL,
    instructions="Answer in three words.",
)

result = asyncio.run(Runner.run(agent, "What is the capital of France?"))
print(result.final_output)

Tool calls, multi-turn runs, and streaming work through Runner.run and Runner.run_streamed.

​

Requests are stateless

Baseten stores no response and exposes no retrieval endpoint, so a request can’t refer to an earlier response. Keep the original input, append all response output items, then append the next user turn. Resend instructions on each request when you use them.

To continue a conversation:

continue.py

messages = [{"role": "user", "content": "Name three countries in South America."}]
first = client.responses.create(model="zai-org/GLM-5.3", input=messages)
if first.status != "completed":
    raise RuntimeError(f"Response did not complete: {first.status}")

messages.extend(first.output)
messages.append({"role": "user", "content": "Which of those has the largest population?"})
second = client.responses.create(model="zai-org/GLM-5.3", input=messages)
if second.status != "completed":
    raise RuntimeError(f"Response did not complete: {second.status}")
print(second.output_text)

​

Stream responses

Set stream to receive server-sent events. Text arrives as response.output_text.delta. The terminal response can be response.completed, response.incomplete, or response.failed; an error event or a connection that ends before a terminal response also needs error handling.

For a complete request and output, including terminal-status and interrupted-stream checks, see Stream responses and choose Responses.

​

Call function tools

Declare function tools in tools. Each function_call output item contains name, JSON-encoded arguments, and a call_id. Execute every call, preserve all output items, and append a function_call_output with the matching call_id for each result. Keep the earlier input turns too.

A response can contain no calls or several parallel calls. Check its status and any refusal before treating a response without calls as final text. The model may request more tools after receiving results, so continue until it returns a complete answer.

For the complete request, tool results, and continuation loop, see Function calling and choose Responses.

​

Return structured output

Set text.format to a JSON schema to constrain the message text:

structured.py

response = client.responses.create(
    model="zai-org/GLM-5.3",
    input="Classify this review: the fans are loud but the screen is gorgeous.",
    text={
        "format": {
            "type": "json_schema",
            "name": "sentiment",
            "schema": {
                "type": "object",
                "properties": {
                    "sentiment": {"type": "string", "enum": ["positive", "negative", "mixed"]},
                    "aspects": {"type": "array", "items": {"type": "string"}},
                },
                "required": ["sentiment", "aspects"],
                "additionalProperties": False,
            },
            "strict": True,
        }
    },
)

print(response.output_text)

​

Unsupported fields

Baseten serves Responses statelessly, so fields that depend on stored state aren’t supported. The endpoint rejects requests that set previous_response_id, conversation, background, or prompt with HTTP 501; send the full prompt and prior turns as input instead.

The endpoint accepts store, but there is no GET /v1/responses/{id} to retrieve a stored response. Store the response yourself if you need to refer to it later.

Authorizations

Authorization · string · header · required

A Baseten API key.

Body

application/json

model · string · required

Model slug, such as zai-org/GLM-5.3.

input · string object[] · required

A prompt string, or the full conversation as input items. Include prior user messages, all response output items, and a function_call_output for each application function call.

instructions · string

System-level instructions for the request.

max_output_tokens · integer

Upper bound on generated tokens, including reasoning tokens.

Required range: x >= 1

temperature · number<float> · default: 1

Sampling temperature, from 0 to 2.

Required range: 0 <= x <= 2

top_p · number<float> · default: 1

Nucleus sampling threshold, from 0 to 1.

Required range: 0 <= x <= 1

stream · boolean · default: false

Stream server-sent events instead of returning one JSON object.

tools · object[]

Function tools the model can call.

Show child attributes

tool_choice · enum<string> object

Whether the model calls a tool. auto lets the model decide, required forces a call, none forbids one.

Available options: auto, required, none

parallel_tool_calls · boolean · default: true

Lets the model emit more than one function call in a response.

text · object

Output format for the message text.

Show child attributes

reasoning · object

Reasoning settings for models that expose them.

Show child attributes

metadata · object

Up to 16 key-value pairs, echoed back on the response.

Show child attributes

truncation · enum<string> · default: disabled

auto or disabled.

Available options: auto, disabled

Response

200

application/json

A response, which can be completed, incomplete, or failed; or a server-sent event stream when stream is true. Inspect the outcome before using the output.

id · string · required

Response identifier, prefixed resp_.

object · enum<string> · required

Available options: response

created_at · integer · required

status · enum<string> · required

Available options: completed, incomplete, failed, in_progress

model · string · required

output · object[] · required

Output items in the order the model produced them.

Option 1

Option 2

Option 3

Show child attributes

completed_at · integer | null

usage · object | null

Show child attributes

incomplete_details · object | null

Show child attributes

error · object | null

metadata · object

Show child attributes

instructions · string | null

max_output_tokens · integer | null

parallel_tool_calls · boolean

temperature · number<float>

top_p · number<float>

tool_choice · string

tools · object[]

truncation · string

service_tier · string

text · object

reasoning · object | null