Rate and usage limits

Per-group, per-model token and request limits, two inheritance modes, and how Frontier Gateway computes the effective limits the runtime enforces.

In Frontier Gateway, rate and usage limits live on the group, not on individual API keys. Every key minted under a group inherits the group’s effective limits, so rotating a customer’s credentials doesn’t change what they can spend. Rate limits cap short-window throughput (per second or per minute), and usage limits cap total consumption per daily window. Both are scoped to a single (group, model slug) pair, so a group can carry separate limits for every model its keys are allowed to call.

You configure both kinds of limit by passing them inside models[].rate_limits and models[].usage_limits when you call POST /v1/gateway/groups or PATCH /v1/gateway/groups/{group_id}. Workspace API keys and the shared Model APIs product use a different limit model; for the comparison, see Frontier Gateway versus Model APIs limits.

Rate limits

A rate limit caps short-window throughput. You attach one or more rate limits to each model slug on a group.

Field	Values	Description
type	TOKEN, REQUEST	Whether the limit counts tokens (prompt plus completion) or requests.
unit	SECOND, MINUTE	The window the threshold applies to.
threshold	Integer >= 1	The maximum count allowed per window.

You can set both a TOKEN and a REQUEST rate limit on the same model slug, but you can’t set two rate limits with the same type.

{
  "metadata": { "external_entity_id": "cust_42" },
  "models": [
    {
      "slug": "your-org/your-model",
      "rate_limits": [
        { "type": "TOKEN", "unit": "MINUTE", "threshold": 1000000 },
        { "type": "REQUEST", "unit": "MINUTE", "threshold": 100 }
      ]
    }
  ],
  "hierarchy": { "limit_enforcement": "INDEPENDENT", "parent_group_id": null }
}

In this example, the group can spend up to one million prompt-plus-completion tokens per minute on your-org/your-model, and up to 100 requests per minute against the same model. Both ceilings are enforced; whichever the caller hits first triggers a 429 Too Many Requests response.

Usage limits

A usage limit caps how much a group can spend in a daily window. Usage limits are optional. You can attach a usage limit to any model slug the group is allowed to call.

Field	Values	Description
type	TOKEN, REQUEST	Whether the limit counts tokens or requests.
unit	DAY	The window the threshold applies to. Daily is the only supported window.
threshold	Integer >= 1	The maximum count allowed per daily window.

Both TOKEN and REQUEST are supported as the type for a usage limit:

{
  "models": [
    {
      "slug": "your-org/your-model",
      "usage_limits": [
        { "type": "TOKEN", "unit": "DAY", "threshold": 10000000 },
        { "type": "REQUEST", "unit": "DAY", "threshold": 5000 }
      ]
    }
  ]
}

In this example, the group can spend up to ten million tokens per day and up to 5,000 requests per day on your-org/your-model. Whichever ceiling the caller hits first triggers a 429 Too Many Requests response for the rest of the daily window.

Per-model scope

Limits are scoped per (group, model slug) pair. A group can be authorized for multiple model slugs, and each slug carries its own independent rate-limit and usage-limit buckets. Spending tokens against one model doesn’t draw down another model’s budget on the same group.

{
  "models": [
    {
      "slug": "your-org/your-model",
      "rate_limits": [
        { "type": "TOKEN", "unit": "MINUTE", "threshold": 1000000 }
      ],
      "usage_limits": [
        { "type": "TOKEN", "unit": "DAY", "threshold": 10000000 }
      ]
    },
    {
      "slug": "your-org/your-other-model",
      "rate_limits": [
        { "type": "REQUEST", "unit": "SECOND", "threshold": 20 }
      ]
    }
  ]
}

In this example, your-org/your-model carries a per-minute token rate limit and a daily token usage limit, while your-org/your-other-model carries only a per-second request rate limit. The two slugs are independent.

Inheritance modes

Every group declares an enforcement mode at creation by setting hierarchy.limit_enforcement to one of two values: INDEPENDENT or CASCADING. The mode controls how a child group’s usage interacts with its ancestors. The mode is fixed for the whole hierarchy: children must declare the same mode as their parent, and the field is immutable after creation. Hierarchies are capped at five levels deep.

Independent mode

In an independent hierarchy, a child group inherits any limit its ancestors set when the child omits it, but the child’s usage is metered separately from its ancestors. A child can override an inherited threshold upward or downward. A sibling’s traffic never draws down another sibling’s budget.

Think of an independent hierarchy as a template. The parent group establishes default limits, and children opt out of them by declaring their own. Consumption is bucketed per group, with no cross-group accounting.

Worked example. A root group free-tier has:

{ "type": "TOKEN", "unit": "MINUTE", "threshold": 100000000 }

A child group john under free-tier declares no limits. The runtime enforces 100M TPM on john, sourced from free-tier. If you later raise free-tier’s threshold to 150M TPM, john automatically gets 150M TPM too.

A sibling child group sally under free-tier declares its own ceiling:

{ "type": "TOKEN", "unit": "MINUTE", "threshold": 120000000 }

The runtime enforces 120M TPM on sally, sourced from sally. john’s traffic doesn’t draw down sally’s budget, and sally’s traffic doesn’t draw down john’s.

Cascading mode

In a cascading hierarchy, a child group’s usage counts against every ancestor at the same time. A request that fits the child’s own limit can still be rejected if an ancestor is exhausted.

Think of a cascading hierarchy as a shared pool. An ancestor establishes a hard cap on the subtree’s total consumption, and children divide it. Siblings can compete for the same pool: one sibling spending heavily reduces what’s available to the others.

Children in a cascading hierarchy can’t declare a threshold higher than any ancestor’s threshold for the same (slug, type, unit) tuple. Frontier Gateway enforces this at write time, on both create and update. Any of the following requests fails with 400 Bad Request: "Child group exceeds parent group limit.":

• Creating a child whose declared threshold exceeds an ancestor’s threshold for the same (slug, type, unit).
• Raising a descendant’s threshold past an ancestor’s via PATCH.
• Lowering an ancestor’s threshold via PATCH below the highest existing descendant threshold.

To raise a subtree’s ceiling, raise the ancestor first, then the descendants. To lower an ancestor below a descendant, lower the descendant first. Each direction is rejected if you do it out of order.

Worked example. A root group org has:

{ "type": "TOKEN", "unit": "MINUTE", "threshold": 100000000 }

Two children finance and engineering under org each declare:

{ "type": "TOKEN", "unit": "MINUTE", "threshold": 70000000 }

Each child’s effective_models shows 70M TPM sourced from itself, but the runtime also enforces the 100M TPM ceiling sourced from org against the combined traffic of finance and engineering. If finance consumes 70M in a given minute, engineering has only 30M of headroom left in that minute, regardless of its own declared 70M ceiling. The 70M + 70M over-provisioning is allowed at create time because each individual child threshold (70M) stays at or below the parent’s (100M); only a single child threshold that exceeded the parent’s would be rejected.

Effective limits and inheritance

Every group response carries two parallel blocks:

• models: the configuration you wrote on this specific group, as if you were reading the row alone.
• effective_models: the limits the runtime enforces on this group after walking the hierarchy. Each limit carries a source_group field pointing to the group (this one or an ancestor) the limit is anchored to.

In an independent hierarchy, effective_models resolves each (slug, type, unit) tuple by taking the closest ancestor (including self) that declared it.

In a cascading hierarchy, effective_models lists every distinct ancestor limit the request is subject to. Read it as the full set of ceilings that gate this group’s traffic.

effective_models is read-only. To change what a group enforces, update the models block on the group itself (or on an ancestor) with PATCH /v1/gateway/groups/{group_id}.

Enforcement and reset

When a request from one of a group’s keys exceeds any limit on the request’s effective_models for the requested model slug, the platform rejects the request with 429 Too Many Requests. The 429 fires for the first limit hit: if a group has a TOKEN/MINUTE rate limit and a REQUEST/DAY usage limit, either can trigger rejection. In a cascading hierarchy, the limit hit can be one anchored on an ancestor rather than the calling group’s own configuration.

Daily usage windows reset at midnight UTC. After reset, a group’s consumption for each DAY limit returns to zero and the group can spend up to the threshold again over the next 24 hours.

Rate-limit windows (per second, per minute) are short rolling windows enforced inline on every request and don’t have a reset timestamp you need to track.

Frontier Gateway versus Model APIs limits

Frontier Gateway and the shared Model APIs product use different limit models:

• Frontier Gateway limits are per group, per model slug, with an inheritance mode picked at the root. You configure TOKEN/REQUEST rate limits (SECOND or MINUTE) and optional TOKEN/REQUEST usage limits (DAY) on the group, and every key minted under the group inherits the group’s effective config.
• Model APIs limits are account-tier RPM/TPM ceilings that apply to your workspace API key as a whole, regardless of which Model APIs model you’re calling.

For more information on Model APIs limits, see Rate limits and budgets.

Next steps

• Manage groups and API keys: Configure limits when you create or update a group, and rotate keys without changing them.