ServiceClient is the entry point for every session. It calls the Baseten control plane to create a session, then provisions trainers and samplers within that session on demand. It returns the TrainingClient and SamplingClient you drive for the rest of a run.
Provision a trainer and get a TrainingClient:
ServiceClient
Construct a
ServiceClient and create a new TrainerSession on the Baseten control plane. All arguments are keyword-only. api_key defaults to the BASETEN_API_KEY environment variable, and base_url to LOOPS_BASE_URL (falling back to https://api.baseten.co).Pass team to create the session, trainers, and samplers inside a specific Baseten team instead of your organization’s default team. Give the team’s name, not its ID. ServiceClient resolves the name against your available teams when you construct it, and raises ValueError listing the teams you can use when none match. ServiceClient reads the LOOPS_TEAM environment variable when you don’t pass the kwarg. See Choose a team.availability_model sets the capacity type for every trainer and standalone sampler this client creates. Paired samplers inherit their run’s capacity type. Pass an AvailabilityModel value or its string ("dedicated" or "spot"). If you omit it, ServiceClient reads LOOPS_AVAILABILITY_MODEL, then defaults to dedicated. The constructor argument takes precedence over the environment variable. Reuse applies only to a warm trainer with the same capacity type; otherwise Loops provisions a new trainer. See Spot capacity.Pass reuse_from_run_id to attach create_lora_training_client calls to a prior run’s warm trainer and paired sampler instead of provisioning fresh. ServiceClient reads the LOOPS_REUSE_FROM_RUN_ID environment variable when you don’t pass the kwarg; the kwarg wins when you set both.reuse_from_session_id (environment variable LOOPS_REUSE_FROM_SESSION_ID) is the deprecated session-based equivalent; when both are set, the run ID wins and the session ID is ignored. Standalone samplers from create_sampling_client reuse through the session ID only.The named run or session must belong to the same team. Reuse is best-effort: if the prior trainer or sampler is stopped, failed, or unhealthy, the backend provisions a fresh one and the call still succeeds. See Reuse infrastructure across sessions.ServiceClient
Bind to already-running local trainer and sampler processes without contacting the control plane. Pass
trainer_url and sampler_url as the base URLs of local server processes. Useful for end-to-end testing.TrainingClient
Provision a trainer for the given Hugging Face
base_model and return a connected TrainingClient. Blocks until the trainer’s HTTP server is up, so you can use the returned client immediately. By default the trainer comes up without a sampler; one is provisioned the first time you request a sampling client from the returned TrainingClient. Pass a WandbConfig instance to stream training metrics to a Weights & Biases run.replicas sets the number of data-parallel copies of the trainer to run. More copies raise throughput (larger effective batch, faster weight updates) at proportionally more GPUs; they don’t change model size or sequence length. Must be at least 1. Values above 1 provision fresh capacity and bypass run and session reuse. See Scale the trainer for how batches shard across replicas.max_seq_len sets the longest sequence in tokens the run must handle. Omit it and the backend picks the default for the model and the accelerators available to you. The run’s paired sampler serves at this length too.sampler_timeout sets the read timeout for a single sample() call on the run’s paired sampler. Omit it and a generation runs as long as it needs. Pass a number of seconds to bound it, and the sampler client raises SampleTimeoutError when a generation exceeds it.name sets the run’s display name. Omit it and the run takes the base model name.with_sampler provisions the run’s paired sampler up front so it boots alongside the trainer, instead of waiting for your first sampling request. Use it when you know you’ll sample, so you don’t pay the sampler cold start mid-loop. Leave it off for train-only workloads so no sampler GPU is provisioned. If the sampler create fails, the run continues and the sampler falls back to being created on first use.sampler_num_replicas sets the replica count for the run’s paired sampler, applied as both its minimum and maximum. Must be at least 1. The count takes effect when the sampler is created: eagerly with with_sampler, or at the first sampling request. Omit it and the backend default applies. Use it to scale sampling throughput for high-QPS rollout workloads, the same way replicas scales the trainer.TrainingClient
Return a
TrainingClient initialized with the weights saved at path. The new run uses the checkpoint’s base model and LoRA configuration. The optimizer starts fresh. Use this to resume from a saved checkpoint when you don’t need the prior optimizer state. replicas works as in create_lora_training_client.TrainingClient
Return a
TrainingClient that resumes from path with the optimizer state and step count intact. The new run uses the checkpoint’s base model and LoRA configuration. Use this to continue a run exactly where it left off. replicas works as in create_lora_training_client.SamplingClient
Provision a standalone sampler and return a connected
SamplingClient. A standalone sampler has no trainer or run behind it and receives no weight syncs. Pass exactly one of base_model or model_path (a bt://loops:<run_id>/sampler_weights/<name> checkpoint URI to serve); passing both or neither raises ValueError. Blocks until the sampler is ACTIVE.max_seq_len sets the longest prompt in tokens the sampler must handle. Omit it and the backend picks the default for the model and the accelerators available to you.timeout sets the read timeout for a single sample() call on the returned client. Omit it and a generation runs as long as it needs. Pass a number of seconds to bound it, and sample() raises SampleTimeoutError when a generation exceeds it.num_replicas sets the replica count for the standalone sampler, applied as both its minimum and maximum. Must be at least 1. Omit it and the backend default applies. Use it to scale sampling throughput, the same way replicas scales a trainer.To sample from a live run’s latest weights, use TrainingClient.save_weights_and_get_sampling_client instead. ServiceClient.local clients don’t support model_path.ServerCapabilities
Return the control plane’s view of supported base models and the GPU classes it can provision them on. Useful for confirming a base model is available before calling
create_lora_training_client. Returns ServerCapabilities.list[Checkpoint]
List checkpoints saved by the run identified by
run_id. Calls the list checkpoints API, not the trainer directly. Returns a list of Checkpoint.CheckpointFilesResponse
Return presigned URLs for every file in an S3-backed checkpoint folder. Pass either
checkpoint_id (a Checkpoint.id) or name with run_id; target (a CheckpointTarget) picks one when a name was saved for both a trainer and a sampler checkpoint. The Loops stack writes checkpoints as unzipped directories rather than archives, so this returns a file list instead of a single archive URL. Volume-backed checkpoints can’t be fetched by URL; use download_checkpoint() instead. Wraps the get checkpoint files API.CheckpointDownload
Download a checkpoint to a local directory, whether its files live in S3 or a Baseten volume. Pass either
checkpoint_id (a Checkpoint.id) or name with run_id; target (a CheckpointTarget) picks one when a name was saved for both a trainer and a sampler checkpoint.dest_dir must be empty for an S3-backed checkpoint. A volume-backed checkpoint lands in a subdirectory under dest_dir, and that landing directory must be empty. The returned CheckpointDownload reports the actual destination, file count, and bytes written.str
Property. The session ID assigned by the control plane. Available after construction.