Quickstart: Make your first inference call
Call a hosted model through Model APIs with an API key and the OpenAI SDK.
Choose how to start
Call a hosted model
Use Model APIs to call supported language models without deploying them.
Deploy your model
Deploy an open-source model on dedicated GPUs with the Baseten CLI.
Train a model
Fine-tune with Loops or run your own training code with Training Jobs.
Model APIs
Model APIs expose hosted endpoints for a curated set of models. Language models support the OpenAI Chat Completions, OpenAI Responses, and Anthropic Messages APIs, so you can use your existing client SDK. Decision models classify, route, and score inputs. Choose Model APIs when a supported model fits your application and you want to start with an API call. Baseten manages the model, engine, hardware, and scaling.Dedicated Inference
Deploy an open-source model, private checkpoint, or custom model on dedicated infrastructure. Define the model with a configuration file, a Python model class, or a custom Docker server, then deploy it with the Baseten CLI. See Develop a model on Baseten for each approach. For supported architectures, aconfig.yaml can define the model, hardware, and inference engine without custom serving code. Use a Python model class or a custom Docker server when you need custom preprocessing, postprocessing, dependencies, or server behavior.
Dedicated deployments give you control over the model, hardware, autoscaling settings, and release lifecycle. They also support stable environments for development, staging, and production.
To run a pipeline of several models or steps as one deployment, build it with Chains.
Deploy a model
Deploy Gemma on dedicated hardware, then call its endpoint.
How Baseten works
Follow the build, routing, autoscaling, cold-start, and promotion lifecycle.
Frontier Gateway
Use Frontier Gateway to offer an inference API to your customers. Manage their API keys and usage limits, and use billing events to charge for model usage. The gateway can route requests to models on Baseten, supported third-party providers, or your own OpenAI-compatible servers.Inference engines
Baseten maintains inference engines for common model architectures:- Engine-Builder-LLM compiles dense text-generation models with TensorRT-LLM.
- BIS-LLM (Baseten Inference Stack) serves mixture-of-experts and large dense models with token-based autoscaling and KV-aware routing.
- Baseten Embeddings Inference (BEI) serves embedding, reranking, and classification models.
config.yaml, or run a server such as vLLM or SGLang in a custom Docker container. To pick an engine for your model, see Choose an engine.
Training
Train a model with Loops or with Training Jobs:- Loops provides dedicated trainers and samplers for supervised fine-tuning and reinforcement learning on supported base models.
- Training Jobs runs your framework and training code in your own container on dedicated GPU clusters.
Sandboxes
Sandboxes give agents an isolated environment to run code and work with files. Create one for each user or project in your agentic application, and run the agent’s tool calls there. An idle sandbox goes on standby and resumes on the next request.Production operations
Every dedicated deployment includes logs, metrics, autoscaling, and environment promotion. Export metrics to Prometheus, Grafana Cloud, Datadog, or New Relic, and export OpenTelemetry traces from deployments that use the Truss server. Store secrets in the workspace instead of in source control. Multi-cloud Capacity Management provisions GPUs across cloud providers and regions. Hosting options support private networking, workload isolation, infrastructure ownership, and specific data regions. Baseten is SOC 2 Type II and HIPAA compliant. See Security at Baseten for details. Use regional environments to constrain replica placement and inference routing for architectures with data-residency requirements, or regional deployments to select a region for a single deployment.Baseten for Model Labs
Use the Distribution Platform to sell your model through Baseten. Baseten manages the customer contract and billing. To compare this approach with offering your own API through Frontier Gateway, see Baseten for Model Labs. Access is enabled per workspace. To get access, talk to us.Next steps
Manage Baseten with an agent
Install the Baseten skill and MCP servers so your agent can manage your Baseten workspace and search these docs.
Examples
Follow end-to-end guides for deploying and optimizing models.
API reference
Browse the inference, management, training, and gateway APIs.