> ## Documentation Index
> Fetch the complete documentation index at: https://docs.baseten.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Zembed-1

> ZeroEntropy's Zembed-1 is a 4B-parameter text embedding model built for high-quality semantic search and retrieval.

<div className="capability-pills">
  <a href="/examples/models/capabilities/embedding" className="capability-pill">Embeddings</a>
</div>

ZeroEntropy's Zembed-1 is a 4B-parameter text embedding model built for high-quality semantic search and retrieval. It produces native 2,560-dimensional embeddings and supports smaller dimensions through learned projection layers rather than standard Matryoshka truncation.

## Setup

Install the Baseten CLI and sign in, then install the OpenAI SDK.

<Columns cols={2}>
  <Column>
    **Install and sign in to Baseten**

    <Tabs>
      <Tab title="macOS or Linux">
        ```bash Terminal theme={"system"}
        brew tap basetenlabs/baseten
        brew trust --formula basetenlabs/baseten/baseten
        brew install baseten
        ```
      </Tab>

      <Tab title="Windows">
        Download and extract the Windows archive for the latest Baseten CLI release:

        ```powershell Terminal theme={"system"}
        $release = Invoke-RestMethod https://api.github.com/repos/basetenlabs/baseten-cli/releases/latest
        $asset = $release.assets | Where-Object { $_.name -like "*_windows_amd64.zip" }
        Invoke-WebRequest $asset.browser_download_url -OutFile baseten.zip
        Expand-Archive -Force baseten.zip .
        ```

        Move `baseten.exe` to a directory on your `PATH`.
      </Tab>
    </Tabs>

    For other platforms or a specific version, see the [Baseten CLI install reference](/reference/cli/baseten/overview#install).

    ```sh theme={"system"}
    baseten auth login
    ```
  </Column>

  <Column>
    **Install the OpenAI SDK**

    ```sh theme={"system"}
    uv pip install openai
    ```
  </Column>
</Columns>

To install without Homebrew, see the [Baseten CLI install reference](/reference/cli/baseten/overview#install).

This preset serves Zembed-1 Embedding 4B on a single H100 through Hugging Face Text Embeddings Inference, optimized for batch embedding throughput with a high concurrency ceiling.

<CardGroup cols={4}>
  <Card title="Hardware" icon="microchip">H100</Card>
  <Card title="Engine" icon="server">Text Embeddings Inference cuda-1.9</Card>
  <Card title="Context" icon="ruler-horizontal">40K</Card>
  <Card title="Concurrency" icon="layer-group">64</Card>
</CardGroup>

## Write the config

Create and move into the project directory:

```sh theme={"system"}
mkdir zembed-1-embedding-4b-throughput && cd zembed-1-embedding-4b-throughput
```

Then create a file named `config.yaml` and paste the following:

```yaml config.yaml theme={"system"}
# Registry copy of the config serving the ZeroEntropy lab listing zeroentropy/zembed-1
# (Hugging Face Text Embeddings Inference). Values below are the deployed ones. Two things
# differ in form, not effect: model_name follows the repo convention, and the checkpoint is
# pinned to cf13c81f (the Apache-2.0 relicense) instead of the deployed 10378878; every
# model file is byte-identical between the two revisions, only LICENSE/README changed.
model_name: model:zembed-1-embedding-4b preset:throughput
model_metadata:
  repo_id: zeroentropy/zembed-1-embedding
  example_model_input:
    inputs:
      - What is backpropagation?
    prompt_name: query
    dimensions: 1280
base_image:
  image: ghcr.io/huggingface/text-embeddings-inference:cuda-1.9
  python_executable_path: /usr/bin/true
environment_variables:
  HF_HUB_DISABLE_XET: "1"
build_commands:
  - mkdir -p /data /tmp/hf-cache && chmod -R 777 /data /tmp/hf-cache
weights:
  - source: hf://zeroentropy/zembed-1-embedding@cf13c81f3274394053d166740294f7eea4586f7a
    mount_location: /data/zembed-1-embedding
docker_server:
  start_command: >-
    text-embeddings-router-90
    --model-id /data/zembed-1-embedding
    --served-model-name zembed-1
    --hostname 0.0.0.0
    --port 3000
    --dtype float16
    --max-concurrent-requests 4096
    --max-client-batch-size 128
    --max-batch-tokens 32768
    --auto-truncate false
    --payload-limit 8388608
    --huggingface-hub-cache /tmp/hf-cache
    --default-prompt-name document
    --json-output
  server_port: 3000
  predict_endpoint: /embed
  readiness_endpoint: /health
  liveness_endpoint: /health
resources:
  accelerator: H100
  instance_type: H100
  use_gpu: true
runtime:
  predict_concurrency: 64
  streaming_read_timeout: 300
```

This config tells Baseten to serve `zeroentropy/zembed-1-embedding` on a single H100 with `ghcr.io/huggingface/text-embeddings-inference:cuda-1.9` running the text-embeddings-router in FP16. The `weights:` block pins the checkpoint to the Apache-2.0 relicense revision, mirrors it to the Baseten Delivery Network at deploy time, and mounts it at `/data/zembed-1-embedding`, so the router loads from local disk and never calls Hugging Face at runtime. The router accepts batches of up to 8192 tokens and returns 2,560-dimensional embeddings.

## Flags

The `start_command` passes these flags to the engine. Each one controls a runtime or serving behavior:

| Flag | Value | What it does |
| - | - | - |
| `--model-id` | `/data/zembed-1-embedding` | Model to serve, either a Hugging Face model ID or a local directory holding the model files. |
| `--hostname` | `0.0.0.0` | IP address the server listens on. The `0.0.0.0` default accepts requests from outside the container. |
| `--dtype` | `float16` | Weight precision loaded at runtime. **float16:** FP16 weights. |
| `--max-concurrent-requests` | `4096` | Maximum concurrent requests the deployment accepts. Past it, the server rejects requests instead of queueing them. |
| `--max-client-batch-size` | `128` | Maximum number of inputs a client can send in a single request. |
| `--max-batch-tokens` | `32768` | Maximum total tokens in a batch, the main control on how much of the GPU one batch can use. |
| `--auto-truncate` | `false` | Whether to truncate inputs longer than the model's maximum sequence length instead of rejecting them. **false:** Reject over-length inputs with an error instead of truncating them. |
| `--payload-limit` | `8388608` | Request payload size limit in bytes. The TEI default is 2 MB. |
| `--huggingface-hub-cache` | `/tmp/hf-cache` | Location of the Hugging Face Hub cache, which you can point at a mounted disk. |
| `--default-prompt-name` | `document` | Prompt from the model's sentence-transformers `prompts` configuration that the server prepends by default. No prompt is applied when it is unset. |
| `--json-output` | (no value) | Emit server logs as JSON for telemetry pipelines. |

## Deploy

Push the config to Baseten with the Baseten CLI, or with the Truss CLI if you prefer it:

<CodeGroup>
  ```sh Baseten CLI theme={"system"}
  baseten model push
  ```

  ```sh Truss CLI theme={"system"}
  uvx truss push
  ```
</CodeGroup>

You should see output similar to:

```output theme={"system"}
Pushing model "zembed-1-embedding-4b-throughput"...
Uploading model...
Uploaded model in 0s
✨ Model zembed-1-embedding-4b-throughput was successfully pushed ✨

  Model:       zembed-1-embedding-4b-throughput (abc1d2ef)
  Deployment:  xyz123
  Environment: production

🪵 View logs:
   deployment:   baseten model deployment logs --model-id abc1d2ef --deployment-id xyz123
   environment:  baseten model environment logs --model-id abc1d2ef --environment production  (once deployed)
   app:          https://app.baseten.co/models/abc1d2ef/logs/xyz123

🚀 Invoke your model:
   URL:  https://model-abc1d2ef.api.baseten.co/deployment/xyz123/predict
   CLI:  baseten model predict --model-id abc1d2ef
```

`baseten model push` prints your **model ID** (`abc1d2ef` in the example). The examples below use it wherever you see `{model_id}`, and read your API key from the `BASETEN_API_KEY` environment variable.

## Call the model

Your deployment serves an embeddings API at `/embed`.

Now call your deployment to generate embeddings:

<Tabs>
  <Tab title="Python">
    ```python main.py theme={"system"}
    import os
    from openai import OpenAI

    client = OpenAI(
        api_key=os.environ["BASETEN_API_KEY"],
        base_url="https://model-{model_id}.api.baseten.co/environments/production/sync/v1",
    )

    response = client.embeddings.create(
        model="zembed-1",
        input=[
            "Baseten is a fast inference provider.",
            "Embeddings power semantic search and RAG.",
        ],
    )

    for item in response.data:
        print(len(item.embedding), item.embedding[:4])
    ```
  </Tab>

  <Tab title="cURL">
    ```sh theme={"system"}
    curl -s https://model-{model_id}.api.baseten.co/environments/production/sync/v1/embeddings \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer $BASETEN_API_KEY" \
      -d '{
        "model": "zembed-1",
        "input": [
          "Baseten is a fast inference provider.",
          "Embeddings power semantic search and RAG."
        ]
      }'
    ```
  </Tab>
</Tabs>

Zembed-1 is instruction-aware. Pass `prompt_name: query` for search queries and
`prompt_name: document` for indexed text. The router falls back to the document
prompt when the field is absent, so set `prompt_name: query` on the query side.

For smaller vectors, pass the `dimensions` parameter instead of truncating the
output yourself. Zembed-1 uses learned projection layers, not Matryoshka
truncation, so naive truncation degrades quality.

For higher throughput, use the [Baseten Performance Client](https://www.baseten.co/blog/your-client-code-matters-10x-higher-embedding-throughput-with-python-and-rust/), which batches and pipelines requests automatically.

## Next steps

<CardGroup cols={2}>
  <Card title="Call your model" icon="code" href="/inference/calling-your-model">
    Endpoint anatomy, authentication, and sync versus async inference
  </Card>

  <Card title="Autoscaling" icon="arrow-up-right-dots" href="/deployment/autoscaling/overview">
    Scale replicas with traffic, including scale to zero
  </Card>
</CardGroup>