Training inputs
A single training example: a
ModelInput paired with a dict of TensorData loss function inputs. routed_experts_ref optionally carries an ExternalRef pointing at captured expert-routing data for the sequence.A tokenized prompt, represented as a list of
ModelInputChunk objects. Construct with ModelInput.from_ints(token_ids) for the common case.A discriminated union of
EncodedTextChunk (a list of token IDs), ImageChunk (a base64-encoded image with an expected token count), and ImageAssetPointerChunk.A pointer to an out-of-band image asset, present for Tinker import parity only. Baseten samplers don’t resolve out-of-band assets, so constructing one raises
ValueError. Pass images inline as ImageChunk instead. See Image inputs must be inline.A serializable tensor with a flat data list, a dtype string, and a shape. Convert to and from
torch.Tensor with TensorData.to_torch() and TensorData.from_torch(tensor).A pointer to a payload held outside the request body, carrying a
uri and a version. The uri is a logical bt:// name that each side resolves against its own storage root, not an absolute location. version determines how to read the bytes. Used by Datum.routed_experts_ref and SampledSequence.routed_experts.Configuration
Controls for text generation:
temperature, top_p, top_k, max_tokens, seed, and stop.Optimizer hyperparameters:
learning_rate, beta1, beta2, eps, weight_decay, and grad_clip_norm.grad_clip_norm defaults to 1.0. Set it to 0.0 to disable gradient clipping.A
StrEnum that sets trainer capacity. AvailabilityModel.dedicated uses capacity that Baseten doesn’t preempt. AvailabilityModel.spot uses idle capacity that Baseten can reclaim. Pass it to ServiceClient(availability_model=...) or set LOOPS_AVAILABILITY_MODEL. See Spot capacity.Optional Weights & Biases settings (
project and an optional run name) passed to create_lora_training_client to stream training metrics.String literal aliases.
LossFnType names the loss functions in the SDK’s typed surface: "cross_entropy", "importance_sampling", "ppo", "cispo", "dro". The trainer also accepts "dpo" and "dppo"; see Loss functions for the full set. StopReason is "stop" or "length", the value of SampledSequence.stop_reason.Results and handles
The full response from
sample(): a list of SampledSequence objects in sequences, the policy_version the sampler replica was running, and prompt_logprobs / topk_prompt_logprobs populated when the matching sample() flags are set.A single generated sequence: a list of output token IDs, optional per-token log-probabilities, a stop reason, and optional decoded
text. routed_experts carries an ExternalRef to captured expert-routing data covering the whole sequence, or None when routing wasn’t captured.Metadata for a saved checkpoint, populated by
list_checkpoints().A paginated list of presigned file URLs for a checkpoint, populated by
get_checkpoint_archive_url().One entry in a
CheckpointFilesResponse.presigned_urls list: a presigned URL plus relative_file_name, node_rank, size_bytes, and last_modified metadata.Returned by
ServiceClient.get_server_capabilities(); describe which base models the control plane can provision and on which GPU classes.A handle to a long-running training operation. Call
.result() or .result(timeout=seconds) to block until the operation completes and return the result. The forward and forward_backward methods return a ForwardBackwardFuture with the same .result() contract.The result returned by
forward() and forward_backward(). It contains the aggregate loss, a metrics mapping, and one loss_fn_outputs entry per input datum.Each entry in loss_fn_outputs contains a logprobs TensorData value for the target tokens. When the request sets top_k_logprobs=K, each entry also contains topk_token_ids and topk_logprobs values with shape [sequence_length, K].Response payloads returned by the matching
TrainingClient and SamplingClient methods.