Rate Limits

Rate limits protect the service and ensure fair usage. Limits are defined per plan and vary by model.

Limits by Plan

Limits vary by both plan and model. The table below shows representative figures across the three plans:

Metric	Essential	Professional	Agentic
Models available	42	44	44
Representative RPM †	300	1,000	3,000

† Example RPM for a representative T-Cloud model — exact limits vary by model. Professional and Agentic add Premium-tier models (e.g. Claude Opus) on top of the Standard catalog available in Essential.

Example: Per-model limits across plans

To illustrate how limits scale, here are the limits for two representative models:

GPT-OSS 120B (T-Cloud, Germany):

Plan	RPM	Input TPM	Output TPM
Essential	300	300,000	150,000
Professional	600	600,000	300,000
Agentic	1,000	2,000,000	1,000,000

GPT-5.2 (Azure, EU):

Plan	RPM	Input TPM	Output TPM
Essential	30,000	3,000,000	1,500,000
Professional	60,000	6,000,000	3,000,000
Agentic	100,000	10,000,000	5,000,000

Note

Preview models (the Test catalog, e.g. Gemma 4) use dynamic rate limits rather than fixed RPM/TPM. Telekom may adjust or restrict preview-model usage at any time without prior notice.

For the full per-model breakdown for your plan, see the Plans & Pricing page or download the Service Description PDF.

How Rate Limits Work

• TPM (Tokens per Minute) — Maximum number of input tokens processed per minute
• RPM (Requests per Minute) — Maximum number of API requests per minute
• Limits are applied per API key
• Both input and output tokens count toward TPM limits

Rate Limit Response Headers

Responses from Azure-hosted models (GPT, o-series) include headers that help you track usage against limits:

Note

Rate limit headers are currently returned by Azure-hosted models (e.g., gpt-4.1, o3-mini). Open-source models hosted on T-Cloud (e.g., Llama-3.3-70B-Instruct) do not return these headers.

Header	Description
x-ratelimit-limit-requests	Maximum requests allowed per minute
x-ratelimit-limit-tokens	Maximum tokens allowed per minute
x-ratelimit-remaining-requests	Requests remaining in the current window
x-ratelimit-remaining-tokens	Tokens remaining in the current window
x-ratelimit-reset-requests	Time until the request limit resets
x-ratelimit-reset-tokens	Time until the token limit resets

Reading Headers

Python

import os
import httpx

# Use httpx directly to inspect response headers
response = httpx.post(
    "https://llm-server.llmhub.t-systems.net/v2/chat/completions",
    headers={"Authorization": f"Bearer {os.getenv('OPENAI_API_KEY')}"},
    json={
        "model": "gpt-4.1",
        "messages": [{"role": "user", "content": "Hello"}],
        "max_tokens": 10,
    },
)

print(f"Requests remaining: {response.headers.get('x-ratelimit-remaining-requests')}")
print(f"Tokens remaining:   {response.headers.get('x-ratelimit-remaining-tokens')}")
print(f"Resets in:          {response.headers.get('x-ratelimit-reset-requests')}")

curl

curl -i -X POST "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4.1",
    "messages": [{"role": "user", "content": "Hello"}],
    "max_tokens": 10
  }'

# The -i flag shows response headers including:
# x-ratelimit-remaining-requests: 19
# x-ratelimit-remaining-tokens: 18740
# x-ratelimit-reset-requests: 3s

Handling Rate Limits

When you exceed a rate limit, the API returns a 429 Too Many Requests error. Best practices:

1. Monitor response headers — Check x-ratelimit-remaining-* headers to avoid hitting limits
2. Implement exponential backoff — Wait longer between retries
3. Batch requests — Combine multiple small requests into fewer larger ones
4. Cache responses — Avoid repeating identical requests
5. Use the Queue API — For batch workloads, use asynchronous requests to spread load

import time

from openai import OpenAI, RateLimitError

client = OpenAI()

def safe_completion(messages, max_retries=3):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(
                model="Llama-3.3-70B-Instruct",
                messages=messages,
            )
        except RateLimitError:
            wait_time = 2 ** attempt
            print(f"Rate limited. Waiting {wait_time}s...")
            time.sleep(wait_time)
    raise Exception("Max retries exceeded")

Need Higher Limits?

• Upgrade your plan — Higher tiers have significantly higher TPM and RPM limits
• Dedicated instances — For enterprise workloads, contact us for dedicated GPU resources with custom rate limits
• Contact: T-Cloud Marketplace or reach out to the AIFS team at ai@t-systems.com