Skip to main content
Baseten is a platform for model inference and training. Call hosted models through OpenAI-compatible and Anthropic-compatible APIs, deploy your own models on dedicated infrastructure, or train models and serve the resulting checkpoints. Agents run their code in sandboxes on the same platform. Baseten manages model containers, GPU capacity across clouds and regions, scaling, and observability. Its inference engines optimize supported model architectures for production traffic.

Quickstart: Make your first inference call

Call a hosted model through Model APIs with an API key and the OpenAI SDK.

Choose how to start

Call a hosted model

Use Model APIs to call supported language models without deploying them.

Deploy your model

Deploy an open-source model on dedicated GPUs with the Baseten CLI.

Train a model

Fine-tune with Loops or run your own training code with Training Jobs.
To point a coding agent at Model APIs, see Use models in coding agents. To let an agent manage your workspace, see Manage Baseten with an agent. For a dedicated speech endpoint, transcribe speech in batch or real time with a model from the Model Library.

Model APIs

Model APIs expose hosted endpoints for a curated set of models. Language models support the OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages APIs, so you can use your existing client SDK. Decision models classify, route, and score inputs. Choose Model APIs when a supported model fits your application and you want to start with an API call. Baseten manages the model, engine, hardware, and scaling.

Dedicated Inference

Deploy an open-source model, private checkpoint, or custom model on dedicated infrastructure. Define the model with a configuration file, a Python model class, or a custom Docker server, then deploy it with the Baseten CLI. See Develop a model on Baseten for each approach. For supported architectures, a config.yaml can define the model, hardware, and inference engine without custom serving code. Use a Python model class or a custom Docker server when you need custom preprocessing, postprocessing, dependencies, or server behavior. Dedicated deployments give you control over the model, hardware, autoscaling settings, and release lifecycle. They also support stable environments for development, staging, and production. To run a pipeline of several models or steps as one deployment, build it with Chains.

Deploy a model

Deploy Gemma on dedicated hardware, then call its endpoint.

How Baseten works

Follow the build, routing, autoscaling, cold-start, and promotion lifecycle.

Frontier Gateway

Use Frontier Gateway to offer an inference API to your customers. Manage their API keys and usage limits, and use billing events to charge for model usage. The gateway can route requests to models on Baseten, supported third-party providers, or your own OpenAI-compatible servers.

Inference engines

Baseten maintains inference engines for common model architectures:
  • Engine-Builder-LLM compiles dense text-generation models with TensorRT-LLM.
  • BIS-LLM (Baseten Inference Stack) serves mixture-of-experts and large dense models with token-based autoscaling and KV-aware routing.
  • Baseten Embeddings Inference (BEI) serves embedding, reranking, and classification models.
Select an engine in config.yaml, or run a server such as vLLM or SGLang in a custom Docker container. To pick an engine for your model, see Choose an engine.

Training

Train a model with Loops or with Training Jobs:
  • Loops provides dedicated trainers and samplers for supervised fine-tuning and reinforcement learning on supported base models.
  • Training Jobs runs your framework and training code in your own container on dedicated GPU clusters.
Both paths store checkpoints that you can deploy to the Baseten inference stack. See Training on Baseten to compare them. If you already run a reinforcement learning trainer, use RL Rollouts to scale rollout generation on Baseten.

Sandboxes

Sandboxes give agents an isolated environment to run code and work with files. Create one for each user or project in your agentic application, and run the agent’s tool calls there. An idle sandbox goes on standby and resumes on the next request.

Production operations

Every dedicated deployment includes logs, metrics, autoscaling, and environment promotion. Export metrics to Prometheus, Grafana Cloud, Datadog, or New Relic, and export OpenTelemetry traces from deployments that use the Truss server. Store secrets in the workspace instead of in source control. Multi-cloud Capacity Management provisions GPUs across cloud providers and regions. Hosting options support private networking, workload isolation, infrastructure ownership, and specific data regions. Baseten is SOC 2 Type II and HIPAA compliant. See Security at Baseten for details. Use regional environments to constrain replica placement and inference routing for architectures with data-residency requirements, or regional deployments to select a region for a single deployment.

Baseten for Model Labs

Use the Distribution Platform to sell your model through Baseten. Baseten manages the customer contract and billing. To compare this approach with offering your own API through Frontier Gateway, see Baseten for Model Labs. Access is enabled per workspace. To get access, talk to us.

Next steps

Manage Baseten with an agent

Install the Baseten skill and MCP servers so your agent can manage your Baseten workspace and search these docs.

Examples

Follow end-to-end guides for deploying and optimizing models.

API reference

Browse the inference, management, training, and gateway APIs.