Overview

Call models on Baseten through Model APIs for managed LLMs or deployed model endpoints for custom models and chains.

Baseten exposes two types of HTTPS inference endpoints. Model APIs provide hosted LLMs through a shared OpenAI-compatible endpoint. Deployed model endpoints serve models and Chains on dedicated subdomains.

Model APIs

Model APIs run supported models such as DeepSeek, GLM, and Kimi on infrastructure managed by Baseten. You choose a model slug and pay per million tokens. All models support tool calling, structured outputs, and JSON mode; some also support reasoning, vision, or audio.

Model APIs implement the OpenAI Chat Completions format. To update an OpenAI client, change its base URL, API key, and model slug. Requests use this endpoint:

https://inference.baseten.co/v1/chat/completions

The Chat Completions reference documents request and response schemas. See the Model APIs guide for supported models, features, and examples.

Deployed model endpoints

Baseten assigns each deployed model or Chain a dedicated subdomain. Use a deployed endpoint for models outside the Model APIs catalog, fine-tuned weights, custom serving logic, or multi-step Chains. You control the hardware, autoscaling settings, and serving engine.

Each endpoint URL identifies the model or Chain in the hostname and the target in the path.

For models:

https://model-{model_id}.api.baseten.co/production/{endpoint}
https://model-{model_id}.api.baseten.co/environments/{env_name}/{endpoint}
https://model-{model_id}.api.baseten.co/development/{endpoint}
https://model-{model_id}.api.baseten.co/deployment/{deployment_id}/{endpoint}

For chains:

https://chain-{chain_id}.api.baseten.co/production/{endpoint}
https://chain-{chain_id}.api.baseten.co/environments/{env_name}/{endpoint}
https://chain-{chain_id}.api.baseten.co/development/{endpoint}
https://chain-{chain_id}.api.baseten.co/deployment/{deployment_id}/{endpoint}

• model_id: the model’s alphanumeric ID, found in your model dashboard.
• chain_id: the chain’s alphanumeric ID, found in your chain dashboard.
• env_name: the environment name, such as staging.
• deployment_id: the deployment’s alphanumeric ID.
• endpoint: the API action, such as predict.

Regional environments place the environment name in the hostname instead of the URL path:

https://model-{model_id}-{env_name}.api.baseten.co/{endpoint}
https://chain-{chain_id}-{env_name}.api.baseten.co/{endpoint}

For long-running tasks, the inference API supports asynchronous inference with priority queuing.

Use the inference API OpenAPI specification with code generators, SDK tools, and API clients.

Predict endpoints

All predict endpoints accept a JSON request body that is forwarded directly to the model’s predict function (for models) or chain entrypoint (for chains).

Models

Method	Endpoint	Description
POST	/production/predict	Call the production environment.
POST	/environments/{env_name}/predict	Call a named environment.
POST	/development/predict	Call the development deployment.
POST	/deployment/{deployment_id}/predict	Call a specific deployment.
POST	/production/async_predict	Async predict on production.
POST	/environments/{env_name}/async_predict	Async predict on a named environment.
POST	/development/async_predict	Async predict on the development deployment.
POST	/deployment/{deployment_id}/async_predict	Async predict on a specific deployment.

Chains

Method	Endpoint	Description
POST	/production/run_remote	Call the production environment.
POST	/environments/{env_name}/run_remote	Call a named environment.
POST	/development/run_remote	Call the development deployment.
POST	/deployment/{deployment_id}/run_remote	Call a specific deployment.
POST	/production/async_run_remote	Async call on production.
POST	/environments/{env_name}/async_run_remote	Async call on a named environment.
POST	/development/async_run_remote	Async call on the development deployment.
POST	/deployment/{deployment_id}/async_run_remote	Async call on a specific deployment.

Regional

Regional endpoints use bare paths on regional hostnames (model-{model_id}-{env_name}.api.baseten.co).

Method	Endpoint	Description
POST	/predict	Synchronous predict.
POST	/run_remote	Synchronous chain call.
POST	/async_predict	Async predict.
POST	/async_run_remote	Async chain call.

Status endpoints

Method	Endpoint	Description
GET	/async_request/{request_id}	Get the status of an async request.
DELETE	/async_request/{request_id}	Cancel a queued async request.
GET	/production/async_queue_status	Queue status for production.
GET	/environments/{env_name}/async_queue_status	Queue status for a named environment.
GET	/development/async_queue_status	Queue status for the development deployment.
GET	/deployment/{deployment_id}/async_queue_status	Queue status for a specific deployment.
GET	/async_queue_status	Queue status (regional).

Wake endpoints

Method	Endpoint	Description
POST	/production/wake	Wake the production environment.
POST	/environments/{env_name}/wake	Wake a named environment.
POST	/development/wake	Wake the development deployment.
POST	/deployment/{deployment_id}/wake	Wake a specific deployment.
POST	/wake	Wake (regional).

Timeouts

Baseten applies server-side timeouts to inference and wake operations. A prediction timeout begins after routing forwards the request for inference. A synchronous request that exceeds this timeout returns a 504.

Surface	Default timeout
Sync prediction execution (/predict, /run_remote)	1200 seconds (20 minutes)
Async prediction attempt (/async_predict, /async_run_remote)	3600 seconds (60 minutes)
Wake (/wake)	600 seconds (10 minutes)

The async prediction timeout applies to each inference attempt, not to the submission request. Baseten returns 201 after placing the request in the queue, and the async retry policy can start another attempt after a retryable timeout. A synchronous request can also wait for capacity for up to the separate 1200-second default parking timeout before inference begins. For the complete sequence, see Request lifecycle.

Timeouts aren’t user-configurable. Set client timeouts based on your model’s expected response time and any expected wait for capacity. For more information, see Configure HTTP clients. For how to interpret and respond to a 504, see Inference errors.

Request size

The ingress proxy limits each request body to 100 MB. Larger requests return 413 Request Entity Too Large before reaching the model or Chain. The limit includes the JSON envelope and any base64-encoded media and cannot be changed.

For large audio, video, or batched media, upload the file to S3, GCS, Azure Blob, or other object storage. Send a presigned URL in the request instead of embedding the file. The model can fetch the file directly from storage.

For how to interpret and respond to a 413, including the separate async payload limit, see Inference errors.