Manage your model

Deploy, validate, publish, update, and retire a model on the Distribution Platform.

Publishing makes a validated deployment available through the Distribution Platform. Choose a Dedicated listing when each customer needs an isolated deployment pinned to a published version; choose a Model API when customers should call a shared endpoint.

Customers can deploy a published Dedicated listing with the REST API, including into a specific team.

Deploy your model

To deploy your model:

Run baseten model push --output json. The JSON output returns the model and deployment IDs needed in later steps.

Request

baseten model push --output json

For more information, see Deploy to production.

Validate your model

To validate the deployed model:

Target the returned model and deployment IDs, and use representative input for your model. Save the model-defined response body for inspection and print the HTTP status.

Request

curl --request POST \
  --url https://model-abc123.api.baseten.co/deployment/def456/predict \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{"prompt": "Hello, world!"}' \
  --output response.json \
  --write-out '%{http_code}\n'

Inspect response.json and confirm that the output matches your expectations. For more information, see Call your model.

For a Model API, promote the exact deployment that you validated to production before you create the endpoint.

To promote the validated Model API deployment to production:

Set --model-id to the model ID returned by baseten model push, abc123, and set --deployment-id to the validated deployment ID, def456. The command promotes to production by default.

Request

baseten model deployment promote --model-id abc123 --deployment-id def456

For more information, see Promote to an environment.

For a Dedicated listing, choose a stable listing ID and version tag. For either channel, prepare the price, billing unit, license, supplemental terms, and the customers you intend to make the model available to.

If your model bills by units the Distribution Platform inference gateway cannot meter, such as characters, audio seconds, or generated images, integrate usage reporting before publishing. Do the same for protocols, such as WebSockets, where the Distribution Platform inference gateway cannot observe the billable units.

Publish your model

Publish a Dedicated model

Create a private, closed-source listing, publish the validated model version, and then make that version live for adoption.

To create a private, closed-source listing:

Set display_name to the customer-facing name, user_defined_id to a stable listing ID, is_public to false so the Dedicated listing is not publicly accessible, and closed_source to true to protect the Truss and weights. closed_source controls artifact access, not customer availability.

Request

curl --request POST \
  --url https://api.baseten.co/v1/library_listings \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "display_name": "Acme Text Model",
    "user_defined_id": "acme-text-model",
    "is_public": false,
    "closed_source": true
  }'

For more information, see Create a library listing.

To publish a new Dedicated listing version:

Set oracle_version_id to the deployed model version, version_tag to the identifier for the published version, and allow_truss_download to false to prevent downloads. allow_truss_download controls artifact access, not customer availability.

Request

curl --request POST \
  --url https://api.baseten.co/v1/library_listings/acme-text-model/versions \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "oracle_version_id": "def456",
    "version_tag": "1.0.0",
    "allow_truss_download": false
  }'

For more information, see Create a listing version.

To make the version live:

Use the listing ID and version_tag in the path to select the published version, then set is_live to true to make it live for customer adoption.

Request

curl --request PATCH \
  --url https://api.baseten.co/v1/library_listings/acme-text-model/versions/1.0.0 \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "is_live": true
  }'

For more information, see Update a listing version.

Publish a Model API

The Distribution Platform uses the Gateway endpoint management resource to configure the model target for a Model API. This shared control-plane resource doesn’t make customer inference a Frontier Gateway call. Customers call the Model API with workspace API keys and Model API inference semantics, not Frontier Gateway federated keys.

Create an endpoint that maps the Model API slug to your validated model in production. The response structure follows Create an endpoint. This example uses a generic slug and model ID, and the response omits environment_name because production targets don’t echo it.

To create the Model API endpoint:

Set slug to the endpoint’s routing identifier. In the OpenAI-compatible example below, customers send this value in the model field. Add one targets entry with provider set to BASETEN, the deployed model_id, and environment_name set to production to route requests to that deployment.

Request

curl --request POST \
  --url https://api.baseten.co/v1/gateway/endpoints \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "slug": "acme/acme-text-model",
    "targets": [
      {
        "provider": "BASETEN",
        "model_id": "abc123",
        "environment_name": "production"
      }
    ]
  }'

For more information, see Create an endpoint.

Complete the Baseten-assisted setup

After you create the listing or endpoint, Baseten enables your organization for distribution, configures the billing setup and customer pricing, attaches the license and supplemental terms, and configures Model API availability. A lab can start a Model API with an allowlist of selected customer workspaces, then make it available to all eligible Baseten customers after validation. Baseten configures both the allowlist and broader availability during assisted setup.

Send your Baseten contact the listing ID or endpoint ID, distribution channel, price and unit, terms, and intended customer availability. If you don’t have a Baseten contact, talk to us. Baseten confirms when setup is complete and, for a Model API, provides the customer activation URL.

After Baseten completes setup, give one test customer access. Customer access is outside the five lifecycle stages.

Test your published model

After Baseten completes setup and the test customer activates the Model API, give this request to a developer in that customer’s workspace. They must set BASETEN_API_KEY to an API key from that workspace. This is an OpenAI-compatible example. Custom Model APIs use the request and response schema that the lab defines.

To test an OpenAI-compatible Model API end to end:

Set model to the published slug and put the chat input in messages so the shared endpoint routes the request to your model.

Request

curl --request POST \
  --url https://inference.baseten.co/v1/chat/completions \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "model": "acme/acme-text-model",
    "messages": [
      {"role": "user", "content": "Hello, world!"}
    ]
  }'

For Model API request fields and responses, see the Chat Completions API reference. For more information about Dedicated request bodies, see Call your model.

After the end-to-end test succeeds, roll out access to the remaining selected customers. Customer access is outside the five lifecycle stages.

Update your published model

Update a Dedicated model

To publish a new Dedicated version:

First, publish the new deployed model version to the listing with a new version tag.

Replace acme-text-model with your listing ID, pqr678 with the deployed model version ID, and 2.0.0 with the new version tag.

Request

curl --request POST \
  --url https://api.baseten.co/v1/library_listings/acme-text-model/versions \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "oracle_version_id": "pqr678",
    "version_tag": "2.0.0",
    "allow_truss_download": false
  }'

For more information, see Create a listing version.

Then make version 2.0.0 live for adoption. This demotes the previous live version.

Replace acme-text-model with your listing ID and 2.0.0 with the version tag to make live.

Request

curl --request PATCH \
  --url https://api.baseten.co/v1/library_listings/acme-text-model/versions/2.0.0 \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{"is_live": true}'

For more information, see Update a listing version.

Each customer deployment stays pinned to the listing version that the customer adopted. Making a newer listing version live changes the version available for new adoption, but it does not update existing customer deployments. A customer adopts the newer version when they are ready; the older deployment continues to run until the customer replaces or retires it.

Update a Model API

When you deploy a new version of the same Baseten model, validate the new deployment and promote it to production. The endpoint continues to target the same model ID and production environment, so you don’t need to update it.

To update the deployment for the same Baseten model:

Set --model-id to the existing Baseten model ID, abc123, and set --deployment-id to its new validated deployment ID, mno345. The command promotes to production by default.

Request

baseten model deployment promote --model-id abc123 --deployment-id mno345

For more information, see Promote to an environment.

When you switch the endpoint to a different Baseten model, promote that model’s validated deployment before you replace the endpoint target.

To promote a deployment for a different Baseten model:

Set --model-id to the new Baseten model ID, ghi789, and set --deployment-id to its validated deployment ID, jkl012. The command promotes to production by default.

Request

baseten model deployment promote --model-id ghi789 --deployment-id jkl012

For more information, see Promote to an environment.

To point a Model API to a different Baseten model:

Replace the endpoint’s complete target list with one Baseten target for the new model in production. Set abc123hash to your endpoint ID and ghi789 to the new model ID that you promoted.

Request

curl --request PATCH \
  --url https://api.baseten.co/v1/gateway/endpoints/abc123hash \
  --header "Authorization: Bearer $BASETEN_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{
    "targets": [
      {
        "provider": "BASETEN",
        "model_id": "ghi789",
        "environment_name": "production"
      }
    ]
  }'

For more information, see Update an endpoint.

Retire your model

Deletion is destructive and does not create a deprecated or disabled state.

Retire a Dedicated model

To delete a Dedicated listing:

Replace acme-text-model with the listing ID to delete.

Deleting the listing stops new adoption and removes its catalog resources. It does not terminate customer deployments already created from an adopted version. Those deployments continue until each customer replaces or retires them.

Request

curl --request DELETE \
  --url https://api.baseten.co/v1/library_listings/acme-text-model \
  --header "Authorization: Bearer $BASETEN_API_KEY"

The API removes the listing and all associated versions, including the live version. For more information, see Delete a library listing.

Retire a Model API

To delete a Model API endpoint:

Replace abc123hash with the endpoint ID to delete.

Request

curl --request DELETE \
  --url https://api.baseten.co/v1/gateway/endpoints/abc123hash \
  --header "Authorization: Bearer $BASETEN_API_KEY"

The API stops routing the slug and frees it for reuse. For more information, see Delete an endpoint.

Next steps

• Give customers access: Understand customer activation and limits.
• Insights: Track customers, models, and proceeds.