> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Train a coding agent on Terminal-Bench

> Train GLM-5.3-Flash with asynchronous reinforcement learning as a multi-turn bash agent on Terminal-Bench 2.1 tasks in Blaxel sandboxes.

This example trains [GLM-5.3-Flash](https://docs.baseten.co/loops/supported-models.md) with reinforcement learning (RL) as a coding agent. The agent works through [Terminal-Bench 2.1](https://github.com/harbor-framework/terminal-bench-2-1) tasks with one `bash` tool, each task in its own [Blaxel](https://blaxel.ai) sandbox. Each task's own tests grade the result.

The tasks are [Harbor](https://github.com/laude-institute/harbor) tasks, so Harbor builds each environment and runs each task's verifier. Start with the [tool-using agent example](https://docs.baseten.co/examples/rl-tool-agent.md) if you haven't run RL with the Loops cookbook before. This run is much larger, and some tasks take hours.

## Prerequisites

* A Baseten workspace with Loops access to `zai-org/GLM-5.3-Flash`, and its [API key](https://docs.baseten.co/organization/api-keys.md). GLM-5.3-Flash trains in either [availability mode](https://docs.baseten.co/loops/availability-modes/overview.md).
* A Blaxel workspace and API key, with capacity for about 50 concurrent sandboxes at 8 GB of memory each.
* [uv](https://docs.astral.sh/uv/) and Python 3.12 or later.

## Install the cookbook

Install the [Loops cookbook](https://github.com/basetenlabs/loops-cookbook/tree/main) from [PyPI](https://pypi.org/project/baseten-loops-cookbook/) into a [uv](https://docs.astral.sh/uv/) project. The cookbook needs Python 3.12 or later, installs the [`baseten-loops`](https://pypi.org/project/baseten-loops/) SDK, and provides the `loops` command. `loops` runs both recipes; each training config names its recipe with `recipe = "rl"` or `recipe = "sft"`. Create a project first if you don't have one:

```sh theme={"system"}
uv init --python 3.12 loops-project
cd loops-project
uv add baseten-loops-cookbook
```

The package includes the cookbook's examples. Run one using its bundled config path, such as `examples/counting/train.toml`, from your project's directory. To change an example, copy its directory into your project at the same path. A config in your project wins over the installed one, and `loops check` prints the directory of the config it used.

Create a `.env` file in your project's directory and set `BASETEN_API_KEY` to a [workspace API key](https://docs.baseten.co/organization/api-keys.md). The commands load `.env` from the training config's directory or the directory you run them from. Never commit `.env`.

A run uses the serverless [availability mode](https://docs.baseten.co/loops/availability-modes/overview.md) when serverless supports the model and the run's LoRA rank and sequence length, and the dedicated mode otherwise. To choose a mode, add `BASETEN_LOOPS_SCOPE=serverless` or `BASETEN_LOOPS_SCOPE=deployment` to `.env`.

Install Harbor with its Blaxel environment, and add `BL_WORKSPACE` and `BL_API_KEY` to `.env`:

```sh theme={"system"}
uv add "baseten-loops-cookbook[harbor-blaxel]"
```

The config mirrors metrics to Weights & Biases. To use it, install the extra with the following command, then add `WANDB_API_KEY` to `.env`:

```sh theme={"system"}
uv add "baseten-loops-cookbook[harbor-blaxel,wandb]"
```

To turn it off, start the run with the following command:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train.toml --output.wandb_project ""
```

Every setting in the config has a command-line flag named by its section and key, so you can change the example without copying it. `uv run loops run --help` lists the flags.

## Read the agent

`examples/terminal_bench/terminal_agent.py` defines `TerminalAgent`, a Harbor agent that calls the policy directly. Its system prompt describes a software engineer in a sandbox. Its one tool runs a bash command in the task's environment and returns the exit code, stdout, and stderr, each truncated to `max_output_bytes`. The agent loops until the model replies without a tool call or reaches `max_turns`.

`TerminalAgent` discards a reply the token limit cut and samples the turn again, up to three times, so a cut generation never trains. After three cut attempts, the episode ends as `truncated` and is still graded. The [tool-using agent](https://docs.baseten.co/examples/rl-tool-agent.md) makes the opposite choice: it ends and grades the episode, and the cut generation trains with the rest.

`examples/terminal_bench/harbor.yaml` pins the Terminal-Bench 2.1 dataset to one commit, selects the cookbook's Blaxel environment, and passes the agent's limits:

```yaml examples/terminal_bench/harbor.yaml wrap theme={"system"}
datasets:
  - repo: https://github.com/harbor-framework/terminal-bench-2-1@7131e4375048a0e408a8fb404b5f499d726b695b
    path: tasks
agents:
  - import_path: terminal_agent:TerminalAgent
    kwargs:
      max_turns: 64
      command_timeout: 120
      max_output_bytes: 16384
```

The job must use `loops_cookbook.harbor.blaxel:BlaxelEnvironment` rather than Harbor's own Blaxel environment. The cookbook's class records every sandbox in the run's resource journal, so `loops cleanup` can release them after a crash.

## Read the training config

`examples/terminal_bench/train.toml` trains asynchronously: the sampler collects rollouts for the next update while the trainer applies the current one. The settings that shape the run:

```toml examples/terminal_bench/train.toml theme={"system"}
[model]
base_model = "zai-org/GLM-5.3-Flash"
renderer = "loops_cookbook.llm.renderer_extensions.glm53:renderer"   # required for this model
max_seq_len = 262144
lora_rank = 32
renderer_chat_template_kwargs = { reasoning_effort = "low", clear_thinking = false }

[training]
mode = "async"
max_steps = 50
learning_rate = 0.00001
adam_epsilon = 0.00000001
router_replay = false
group_size = 8
max_group_losses = 2
groups_per_step = 4
max_consecutive_unusable_groups = 100
max_steps_off_policy = 3
max_policy_delta_in_trajectory = 3

[compute]
spot_job = true
sampler_replica_affinity = true
```

* Each update trains 4 groups of 8 attempts, for 50 updates.
* `max_steps_off_policy = 3` lets a trained rollout come from a policy up to three updates behind the trainer. Async runs trade that staleness for throughput. Read `freshness/lag_mean` to see the lag the run actually trained on.
* `max_group_losses = 2` lets a group train when up to 2 of its 8 sandboxes fail.
* `spot_job` and `sampler_replica_affinity` are account-specific opt-ins. Spot capacity can be reclaimed mid-run, which stops the run and nothing restarts it. Sticky sessions need sampler affinity routing enabled for your Loops workspace; sampling requests fail without it.

To run without them, start the run with the following command:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train.toml \
  --compute.spot_job false \
  --compute.sampler_replica_affinity false
```

`spot_job` requests spot capacity for deployments; it doesn't choose between the serverless and dedicated modes.

## Scale out the run

GLM-5.3-Flash uses the serverless mode by default when your workspace has access, and needs no capacity request. A long asynchronous run that needs more training and rollout throughput belongs in the dedicated mode, where you set the trainer and sampler replica counts.

To use the dedicated mode, add the following line to `.env`:

```sh .env theme={"system"}
BASETEN_LOOPS_SCOPE=deployment
```

Then raise the replica counts when you start the run, for example as follows. `trainer_replicas` adds data-parallel trainer replicas for update throughput, and `sampler_replicas` adds sampler replicas for rollout throughput. Each trainer replica is a full copy of the model's GPU allocation, and the deployment bills for GPU time, including idle time:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train.toml \
  --compute.trainer_replicas 2 \
  --compute.sampler_replicas 4
```

To plan capacity for a large run in the dedicated mode, [request more access](https://docs.baseten.co/loops/overview.md#request-more-access). [Scale the trainer](https://docs.baseten.co/loops/availability-modes/dedicated.md#scale-the-trainer) explains how batches split across replicas.

## Run the example

To check the configuration, run the following command. It downloads the dataset, validates the job and agent options, and launches nothing. It warns about any task that declares more disk than the sandbox provides:

```sh theme={"system"}
uv run loops check examples/terminal_bench/train.toml
```

To train, run the following command. With serverless Loops, Baseten runs the trainer and sampler and bills for the tokens the run trains and samples, as long as serverless supports the model, the LoRA (low-rank adaptation) rank, and the sequence length. Otherwise, the run uses dedicated capacity, which bills for GPU time. The run also starts one Blaxel sandbox per running attempt: 48 at a time, plus up to 5 started ahead so a freed slot is filled at once. Sandboxes bill while they run:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train.toml
```

Tasks vary from minutes to hours. The config allows each rollout up to 6 hours (`rollout_timeout_seconds = 21600`) and reports pipeline progress every 60 seconds, so a slow phase is distinguishable from a stall. The run writes `run.json` and `steps.jsonl` to its run directory.

## Read the results

Monitor `reward/mean`, `discard/groups/no_variance`, and `rollout/*` in W\&B or in the run directory's `steps.jsonl`. Terminal-Bench tasks vary widely in difficulty, so expect many `no_variance` groups from tasks the starting model always fails. For help reading the metrics, see [Check whether an RL run is learning](https://docs.baseten.co/loops/rl-run-health.md).

To tell whether the trained agent is a better coding agent, score a saved checkpoint on tasks it didn't train on; training reward measures only the tasks the run trained on. For how to run that evaluation, see [Evaluate a model](https://docs.baseten.co/post-training/evaluate.md).

## Add a loss on command output

`examples/terminal_bench/train_echo.toml` runs the same job with [ECHO](https://arxiv.org/abs/2605.24517), a custom loss defined in `echo_loss.py`. It adds a next-token loss on the output the agent's commands print, weighted by `observation_weight`, to the importance-sampling loss on the agent's own tokens. The module's docstring describes where it differs from the paper.

To install the loss's extra, run the following command:

```sh theme={"system"}
uv add "baseten-loops-cookbook[harbor-blaxel,custom-loss]"
```

To check the configuration, run the following command:

```sh theme={"system"}
uv run loops check examples/terminal_bench/train_echo.toml
```

To start the run:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train_echo.toml
```

## Resume a stopped run

If the run stops before its last update, continue it with `--resume`. The command finds the run of this config that trained most recently, and continues from the last checkpoint it saved:

```sh theme={"system"}
uv run loops run examples/terminal_bench/train.toml --resume
```

`max_steps` counts the updates of the whole run, so the resumed run makes only the updates that are left. To check the resumed run first, run `uv run loops check examples/terminal_bench/train.toml --resume`. If no run has trained, or the run saved no checkpoint, the command exits with code 2 and starts nothing.

## Shut down the run

`loops run` leaves the Loops run active. On serverless, deactivation is optional because idle runs incur no token charges. On dedicated capacity, [deactivate the run](https://docs.baseten.co/loops/deactivate-a-run.md) when you finish to stop GPU billing without waiting for automatic shutdown. To end the run and its paired sampler in either mode, run the following command with your run directory:

```sh theme={"system"}
uv run loops deactivate "runs/terminal-bench/<run directory>"
```

The command prints the deactivated run ID. Saved checkpoints remain available.

## Release sandboxes after a crash

Sandboxes bill until they're deleted. If the run crashed, release its sandboxes with the following command:

```sh theme={"system"}
uv run loops cleanup examples/terminal_bench/train.toml \
  --state-dir "runs/terminal-bench/<run directory>/state"
```

## Deploy the checkpoint

The run's sampler checkpoint is ready to serve. The deployment loads the adapter on the base weights, downloads the base weights with the `hf_access_token` in [workspace secrets](https://docs.baseten.co/organization/secrets.md), and bills for its GPU while it's live. With the [Baseten CLI](https://docs.baseten.co/reference/cli/baseten/overview.md#install) installed, list the run's checkpoints and deploy the one you want:

```sh theme={"system"}
baseten loops checkpoint list --run-id "<run_id>"
baseten loops checkpoint deploy --checkpoint-id "<checkpoint_id>"
```

The deploy command prompts for a model name, GPU type, GPU count, and the Hugging Face secret name. [Deploy a checkpoint](https://docs.baseten.co/loops/deploy-checkpoints.md) covers the REST API and dashboard paths.

## Next steps

* [Design RL environments and rewards](https://docs.baseten.co/loops/rl-environments.md): Graders, task sizing, and exploits.
* [Check whether an RL run is learning](https://docs.baseten.co/loops/rl-run-health.md): Metrics for long, asynchronous runs.
* [Deploy a checkpoint](https://docs.baseten.co/loops/deploy-checkpoints.md): Serve the trained adapter.