API transition

We are upgrading the LLM Serving Service to provide lower latency, higher throughput, and a more standardized API experience.

The New API introduces consistent response structures, structured error handling, and additional diagnostic information, making integrations easier to maintain and troubleshoot.

To ensure a smooth transition, the Legacy API remains available during the migration period. This gives applications time to adapt while continuing to use the existing API behavior.

This section explains the transition timeline, required changes, and compatibility considerations for applications moving from the Legacy API to the New API.

Transition overview

The transition takes place in three phases. Exact dates will be communicated separately.

Endpoint	Phase 1 (2 weeks)	Phase 2 (2 months)	Phase 3 (permanent)
Existing `llm-server.llmhub.t-systems.net`	Legacy API format	New API format	Discontinued
Legacy `llm-server.legacy.ai.t-systems.net`	Legacy API format	Legacy API format	Discontinued
New `llm-server.ai.t-systems.net`	Not available yet	New API format	New API format

How the transition works

Phase 1 — Preparation. The existing endpoint continues to use the Legacy API format. Applications that depend on the current response or error structure can switch to the dedicated Legacy endpoint, https://llm-server.legacy.ai.t-systems.net/v2. This endpoint is already available and preserves the existing API behavior until the end of Phase 2.

Phase 2 — Migration. The existing endpoint switches to the New API format. The dedicated Legacy endpoint remains available for applications that still require the previous format. The new permanent endpoint becomes available: https://llm-server.ai.t-systems.net/v1.

Phase 3 — New API. The existing and Legacy endpoints are discontinued. All applications should use the permanent New API endpoint, https://llm-server.ai.t-systems.net/v1.

What you need to do

In Phase 1, if needed: Does your application depend on the current response or error format? Then switch your base URL to https://llm-server.legacy.ai.t-systems.net/v2. Your application keeps working as it does today until the end of Phase 2. The Legacy endpoint is already available, so you can make this switch before Phase 1 begins.
By the end of Phase 2, all customers: Move to the New API endpoint https://llm-server.ai.t-systems.net/v1 and the New API format.

The New API uses /v1 paths, for example /v1/chat/completions instead of /v2/chat/completions. When moving to the New API, change the base URL to https://llm-server.ai.t-systems.net/v1 and review your model identifiers and response handling.

Getting started

Most applications can migrate by updating their endpoint while keeping their existing authentication mechanism and standard request structure.

Connection settings

Setting	Legacy API	New API
Recommended base URL	`https://llm-server.legacy.ai.t-systems.net/v2`	`https://llm-server.ai.t-systems.net/v1`
Authentication	API key	API key
Supported keys	Existing and new API keys	Existing and new API keys
Model identifiers	Existing model names	Existing names supported; provider prefixes recommended
Standard API paths	`/v2/...`	`/v1/...`

API key compatibility

Existing API keys remain supported during the transition.

Endpoint	Existing API keys	New API keys
Existing endpoint	Supported	Not supported
Legacy endpoint	Supported	Supported
New endpoint	Supported	Supported

With the introduction of the new management UI, you will also be able to create and manage new API keys through a self-service interface.

Existing API keys will be deprecated in favor of newly issued keys. Deprecation does not immediately invalidate existing keys. Any future retirement will be communicated separately.

Applications using existing keys do not need to replace them solely to access the New API.

Model identifiers

The New API supports both unprefixed and provider-prefixed model identifiers. For example, the following model name can be used:

{
  "model": "gpt-6.1-sol"
}

For explicit provider selection, use the provider/model-name format:

{
  "model": "azure/gpt-6.1-sol"
}

The prefix for each model is listed in /v1/models.

Provider prefixes are recommended but not required. Using the provider/model-name format makes routing more predictable by explicitly selecting the intended provider.

Model discovery

If your application retrieves model information dynamically, review the model-list response structure: the Legacy API serves it at GET /v2/models, the New API at GET /v1/models. The New API can expose additional model capabilities and pricing information, while some fields from the Legacy API are no longer available in the same format.

Aspect	Legacy API	New API
Model identifier	`gpt-6.1-sol`	`azure/gpt-6.1-sol`
Model ownership	`owned_by`, e.g. `T-Systems International`	May be empty or absent
Model type	`meta_data.model_type`, e.g. `LLM`	Not included
Top-level `object`	`"list"`	May be empty or absent
Model entry `object`	`"model"`	May be `"model"` or absent
Context length	`meta_data.max_sequence_length`	`context_length`
Output token limit	`meta_data.max_output_length`	`max_output_tokens`
Additional information	Deployment and provider metadata	May include `alias`, `max_input_tokens`, and `pricing`

What to check for models

If your application retrieves models dynamically, check whether it depends on:

meta_data.model_type to distinguish LLM and embedding models.
owned_by or model-level object fields.
The top-level object: "list" value.
Unprefixed model identifiers in model-list results.

Applications using strict response validation should account for these differences.

For available models, supported capabilities, and configuration details, see Models and the Model comparison.

Model datasheets

New datasheet endpoints under /v1/datasheet will soon be available. They describe each model’s capabilities and parameter specifications: which endpoints a model supports, which parameters it accepts, and their allowed values and defaults.

The following example shows how a model is specified:

{
  "name": "example-model",
  "max_input_tokens": 200000,
  "max_output_tokens": 64000,
  "supported_endpoints": ["/v1/chat/completions", "/v1/responses"],
  "supports_reasoning": true,
  "model_parameters": [
    {
      "id": "temperature",
      "type": "number",
      "default": 1,
      "range": { "min": 0, "max": 2 }
    },
    {
      "id": "reasoning_effort",
      "type": "select",
      "default": "medium",
      "options": ["low", "medium", "high"]
    }
  ]
}

Requests

The New API (https://llm-server.ai.t-systems.net/v1) supports standard chat completion requests, including commonly used parameters such as messages, temperature, top_p, tools, and response_format, where supported by the selected model.

Most applications can continue using their existing request structure with only minor adjustments.

curl -X POST "https://llm-server.legacy.ai.t-systems.net/v2/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-6.1-sol",
    "messages": [
      {"role": "user", "content": "Hello!"}
    ]
  }'

curl -X POST "https://llm-server.ai.t-systems.net/v1/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "azure/gpt-6.1-sol",
    "messages": [
      {"role": "user", "content": "Hello!"}
    ]
  }'

The New API example uses an explicit provider prefix (azure/) for predictable routing; the prefix for each model is listed in /v1/models. An unprefixed model name such as gpt-6.1-sol can also be used when the model is available through automatic provider resolution.

What changes in requests

Aspect	Legacy API	New API
Standard chat parameters	Supported	Supported, subject to model capabilities
Model identifiers	Unprefixed model names	Unprefixed and provider-prefixed names
Provider selection	Managed by the existing routing configuration	Explicit selection supported through provider prefixes
Tool calling	Supported	Supported through standard `tools` and `tool_calls`
Structured output	Supported when available for the model	Supported when available for the model
Provider-specific parameters	Generally forwarded to the provider	May be filtered or ignored unless supported
Request validation	Gateway and provider validation	Primarily provider-dependent; some parameters may be adjusted or ignored

What to check for requests

For standard integrations: Update the endpoint. Existing request structures and unprefixed model names can continue to work, provided the model is available.

For predictable model routing: Use provider-prefixed model identifiers.

For advanced integrations: If your application uses provider-specific parameters, custom request fields, or model-specific features, verify that the selected model supports them through the New API.

Request validation depends on the model provider and API parameter handling. Some unsupported or out-of-range parameters may be adjusted or silently ignored instead of being rejected.

For detailed model capabilities and supported parameters, see Models and the Model comparison.

Responses

The New API (https://llm-server.ai.t-systems.net/v1) introduces a standardized response structure across different model providers. This improves consistency while preserving the core fields used by standard chat completion integrations: choices, message, finish_reason, and usage remain available.

Example: successful response

Legacy API
New API

{
  "id": "chatcmpl-example",
  "object": "chat.completion",
  "model": "gpt-6.1-sol",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello!"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 3,
    "total_tokens": 13
  }
}

{
  "id": "chatcmpl-example",
  "object": "chat.completion",
  "model": "gpt-6.1-sol",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello!"
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 3,
    "total_tokens": 13
  },
  "extra_fields": {}
}

The value of response.model may also differ from the requested model identifier for some providers.

What changes in responses

Aspect	Legacy API	New API
Response structure	Provider-oriented response format	Standardized chat completion format
Standard fields	`choices`, `message`, `usage`, etc.	Same core fields
Token usage	Original provider usage information	Usage details may vary by provider; zero-valued optional fields may be omitted
Optional fields	May include explicit `null` or empty values	Some optional fields may be omitted
Response `model`	Typically the requested model identifier	May contain the provider-reported model name, which can differ from the requested identifier
Provider-specific metadata	Generally preserved	May be normalized or omitted
Additional information	Provider-specific fields	Optional diagnostic metadata, including latency information

Token usage details

Standard token usage fields remain available, but additional usage details depend on the selected model and provider. For example, some providers return detailed completion token breakdowns, while others expose different usage fields or omit optional details entirely.

Fields such as completion_tokens_details, reasoning_tokens, or provider-specific token categories should therefore be treated as optional. Zero-valued usage details may also be omitted rather than returned explicitly as 0.

Applications that track token consumption should account for these differences when processing usage information.

Additional response metadata

The New API may include additional diagnostic information through the extra_fields object. Depending on the request and model, this may include:

Latency information — helps understand request processing time.
Routing information — provides additional details about how the request was processed.
Provider response metadata — additional information when available.

These fields are optional and should not be treated as part of the required standard response structure.

What to check for responses

For standard integrations: Applications that primarily use choices, message, finish_reason, and the common token usage fields generally require minimal changes to response parsing.

For model identification: Review code that assumes response.model always matches the requested model name.

For strict response validation: Allow optional fields to be absent and support additional response metadata.

For token accounting: Review provider-specific usage fields and do not assume that every provider returns identical token breakdowns.

For provider-specific integrations: Review any code that depends on nonstandard response fields.

Streaming

The New API (https://llm-server.ai.t-systems.net/v1) continues to support Server-Sent Events (SSE). Most streaming behavior remains compatible, but token usage information is returned differently. For streaming basics, see Streaming.

What changes in streaming

Behavior	Legacy API	New API
Streaming protocol	SSE	SSE
Token usage	Separate final usage chunk	Included in the completion’s final chunk
Usage by default	Included without an explicit request	Included without an explicit request
Completion marker	`[DONE]`	`[DONE]`
Errors during streaming	Stream may terminate without an explicit error event	Structured errors may be included in the stream

Token usage in the stream

Legacy usage
New API final

{
  "choices": [],
  "usage": {
    "total_tokens": 150
  }
}

{
  "choices": [
    {
      "index": 0,
      "delta": {},
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "total_tokens": 150
  }
}

Both APIs include token usage in successful streaming responses by default, even when stream_options.include_usage is not explicitly requested. The main difference is where this information appears in the stream.

What to check for streaming

Applications that process SSE responses directly should read token usage from whichever chunk contains the usage field, rather than relying on a separate chunk with an empty choices array.

Do not assume that streaming usage details will have an identical structure across all model providers.

Error handling

The New API (https://llm-server.ai.t-systems.net/v1) introduces structured error responses, making it easier for applications to identify errors and handle them consistently. Standard HTTP status codes continue to indicate request outcomes, while certain error scenarios now have more specific status codes. For the general error reference, see Error codes.

What changes in error handling

Scenario	Legacy API	New API
Missing API key	403	401
Invalid API key	401	401
Blocked or deactivated key	401	403
Model not found	404	400 for model resolution errors
Model access denied	404	403
Rate limit exceeded	429	429
Budget exceeded	429	402
Invalid request	400, sometimes plain text	400, structured JSON
Provider connection failure	500	502
Provider timeout	500	504

Example: error response

The Legacy API typically returns provider error information as a string inside detail. The New API returns structured error information.

Legacy API (HTTP 400)
New API (HTTP 400)

{
  "detail": "Error: 400 - Bad Request - {\"error\":{\"message\":\"Invalid request parameter\",\"type\":\"invalid_request_error\"}}"
}

{
  "error": {
    "message": "Invalid request parameter",
    "type": "invalid_request_error",
    "code": "invalid_value",
    "param": "messages[0].role"
  }
}

Structured error fields allow applications to access error details without parsing an embedded provider error message.

What to check for error handling

For standard HTTP error handling: Applications can continue relying on HTTP response status codes.

For custom error handling: Review code that depends on specific status codes, error-message text, or the Legacy API’s detail field.

Use the actual HTTP response status as the primary indicator, and treat error fields such as type, code, and param as optional.

Rate limiting and retries

The New API distinguishes between provider rate limits, service-level rate limits, and budget restrictions.

Scenario	HTTP status	Retry information
Provider rate limit	429	Provider-supplied retry information may be available
Service-level rate limit	429	No `Retry-After` header; reset information may appear in the error message
Budget exceeded	402	No `Retry-After` header; budget information may appear in the error message

When a provider supplies a Retry-After header and it is forwarded in the response, applications should respect it. For service-level rate limits, implement your own backoff strategy; see Rate limits.

Budget-related errors should be handled separately and should not trigger automatic retries.

Response validation

Validate the response Content-Type before attempting to parse the response body. Some unsupported API routes or methods may return HTML rather than JSON.

A successful HTTP status alone does not guarantee that the response contains the expected API structure.

Troubleshooting

When reporting an issue, include the request timestamp, endpoint, HTTP status, and any available request or trace identifier. This helps support investigate requests more efficiently.

Summary

The New API provides a more scalable and standardized foundation for LLM integrations while maintaining compatibility with commonly used chat completion features. Before migrating, review your model identifiers, request parameters, response parsing, token usage handling, streaming behavior, and error handling.

All applications should move to the permanent New API endpoint by the end of Phase 2. For questions or migration support, contact AI@t-systems.com.