Serverless Loops

Train and sample without managing deployments, and pay for the tokens you use.

In the serverless mode, you create a run and call the same training and sampling operations as in any Loops run. Baseten manages the infrastructure, and you pay for the tokens you train and sample. Runs use shared capacity, so you don’t reserve GPUs before you start.

The serverless mode supports the models marked Serverless and dedicated in Supported models, at the sequence length listed there.

​

Start a serverless run

Runs use the serverless mode by default when serverless supports the model and the run’s LoRA rank and sequence length. To require the serverless mode, set scope to serverless:

from baseten.loops import ServiceClient

training_client = ServiceClient().create_lora_training_client(
    base_model="zai-org/GLM-5.3-Flash",
    scope="serverless",
)

With scope="serverless", creating the run fails instead of falling back to the dedicated mode when serverless can’t take the run. See choose a mode for the environment variable and REST equivalents.

To check which models serverless supports, call Get server capabilities. Each model in the response has a serverless field.

​

When a run can’t use serverless

With the default scope, a run uses the dedicated mode instead when:

• Serverless doesn’t support the model.
• The run asks for a LoRA rank or sequence length that serverless doesn’t support for the model.
• The run reuses another run’s deployments.

Replica counts and the availability_model capacity type apply only in the dedicated mode. To scale out a run with them, set scope to deployment.

​

When capacity is full

Serverless runs share a capacity pool that Baseten manages for you. When that pool is full, creating a training run or standalone sampler returns a 429, and the SDK attempts the request up to ten times before raising LoopsCapacityError.

To resolve it, you can:

• Wait for capacity. Other runs finish and free their share, and the SDK’s retries cover the first few minutes.
• Free capacity in your organization. When the response reports details.limit of org_cap, your organization is at its limit of concurrent runs and samplers. Deactivate a run you no longer need.
• Request more access. The Loops request form is how you get more concurrent runs, a larger model, or longer sequences.
• Give the run its own GPUs. Set scope to deployment to run in the dedicated mode, which reserves a trainer and sampler for the run.

If none of those fit, or you need capacity planned for a launch or a long training campaign, work with us through the request form or your Baseten representative rather than waiting on retries.

​

Billing

Serverless runs bill for prefill, cached prefill, sample, and train tokens at the per-model rates listed under Serverless Loops token pricing. These rates are separate from Model API token rates for the same base model. Time between training operations, such as time spent reading results or changing a reward function, doesn’t bill.

​

Usage and logs

baseten loops usage reports dedicated GPU allocations. It doesn’t report serverless token usage or spend.

To inspect dedicated GPU allocations:

Run the following command with an authenticated Baseten CLI:

Command

baseten loops usage

When allocations hold GPUs, the output also includes a table of runs, GPU types, and statuses. For more information, see baseten loops usage.

Deployment logs cover dedicated runs. For more information about errors from serverless training and sampling calls, see Loops troubleshooting.

​

Finish a run

An idle serverless run incurs no token charges. Saved checkpoints remain available to download, deploy, or start a new run from.

To end a run and its paired sampler:

Save the checkpoints you need, then run the following command. Replace <run_id> with TrainingClient.run_id, and confirm when prompted:

baseten loops run deactivate --run-id "<run_id>"

The CLI confirms success with Deactivated Loops run <run_id>.

For more information, see Deactivate a run.