Provider-specific fields

Some providers attach opaque state to their responses that must be sent back verbatim on the next turn. The gateway round-trips that state through the client, with no server-side storage. Two kinds exist today:

• Tool-call-bound state — Gemini thought signatures. Gemini 3 rejects a follow-up request whose replayed function calls lack their thoughtSignature (400 INVALID_ARGUMENT).
• Message-bound state — OpenAI encrypted reasoning (reasoning items with encrypted_content), which improves multi-turn tool-use quality when replayed; Gemini signatures on text responses, which Google recommends sending back to keep reasoning continuous across turns; and Anthropic's signed thinking blocks.

Carriers by surface

Each kind of state rides on a carrier that standard clients send back unchanged, where the surface has one:

provider_specific_fields is a LiteLLM-style extension that standard clients (Vercel AI SDK, the OpenAI SDKs, the Agents SDK) drop, so state that only travels there is lost for them. Use /v1/responses for multi-turn conversations with reasoning models.

Thought signatures (tool-call-bound)

When an upstream tool call carries a thought signature, the gateway embeds it in the public tool call id, LiteLLM-compatible:

<id>__thought__<signature>

The suffixed id appears as tool_calls[].id on /v1/chat/completions, as function_call.call_id on /v1/responses, and as tool_use.id on /v1/messages (non-stream and streaming). Because every standard client echoes tool call ids verbatim — even clients that drop unknown fields, like the Vercel AI SDK — the signature survives the round trip without the client understanding anything.

The signature is also mirrored on the rendered tool call for rich clients:

{
  "id": "call_1__thought__EjQKMg...",
  "type": "function",
  "function": { "name": "get_weather", "arguments": "{}" },
  "extra_content": { "google": { "thought_signature": "EjQKMg..." } },
  "provider_specific_fields": { "thought_signature": "EjQKMg..." }
}

Message/response level additionally aggregates provider_specific_fields.thought_signatures: [...].

Inbound precedence

On replay the gateway accepts the signature in any of three forms, merged in this order (later sources win per key):

1. The __thought__ id suffix (on tool_calls[].id, function_call.call_id or function_call.id, tool_use.id).
2. provider_specific_fields.thought_signature on the tool call / tool_use block.
3. extra_content.google.thought_signature (OpenAI-shaped surfaces only).

The suffix is stripped before anything reaches an upstream: adapters always see clean ids, and tool-result references (tool_call_id, function_call_output.call_id, tool_result.tool_use_id) are stripped symmetrically so call/result matching keeps working.

Notes

• Signatures are standard base64 (A-Za-z0-9+/=), which cannot contain _ — the separator is unambiguous.
• Ids can exceed 64 characters. The gateway never truncates this lossless carrier; clients that truncate tool call ids would lose the signature. LiteLLM uses the same scheme in production.
• With parallel function calls Gemini may sign only the first call; only signed calls get a suffix.
• For migrated or already-persisted Gemini 3 history where every carrier is missing, the Google adapter applies Google's documented skip_thought_signature_validator sentinel to the first function call in that step. Exact signatures are always preferred; the sentinel is only a compatibility recovery path at the provider boundary.

OpenAI encrypted reasoning (message-bound)

For reasoning-capable OpenAI models the gateway requests include: ["reasoning.encrypted_content"] upstream and always calls the upstream with store: false (OpenAI only returns encrypted reasoning for unstored responses; the gateway never uses persisted upstream storage). A live Responses WebSocket may use OpenAI's connection-local previous-response cache, but public and upstream response ids remain separate.

Per surface:

• /v1/responses — encrypted reasoning is rendered as native reasoning output items, with encrypted_content when the request includes reasoning.encrypted_content, as on OpenAI. Clients that replay output items — the Vercel AI SDK does — round-trip it automatically; on input, reasoning items with encrypted_content are replayed upstream before the function calls they precede.
• /v1/chat/completions — carried as message-level provider_specific_fields: { "openai": { "reasoning": [{ "id", "encrypted_content", "summary" }] } } on the assistant message (and streaming delta). Standard clients drop this field, so encrypted reasoning does not round-trip for them on this surface — same limitation as LiteLLM.
• /v1/messages — carried in a gateway redacted_thinking block; see On /v1/messages.

Gemini signatures on text responses (message-bound)

Gemini can also sign a response that has no function call. The signature arrives on the last part, often an empty text part at the end of the stream:

[
  { "text": "**Defining the goal**…", "thought": true },
  { "text": "Hello!" },
  { "text": "", "thoughtSignature": "EsAKCr0KAWkU…" }
]

On /v1/responses, when the request includes reasoning.encrypted_content, the gateway seals those parts into one more reasoning item, placed after the last item of its turn:

{ "type": "reasoning", "id": "rs_…", "summary": [], "encrypted_content": "bfrs_1.eyJ2Ijo…" }

• Sealed, not stored. The value is encrypted with the gateway's encryption keyring (AES-256-GCM): clients cannot read or alter it, and nothing is kept server-side. It appears even when Gemini returned no visible thought summary, and the summary item, if any, is unchanged.
• Replayed by standard clients. Clients that send output items back — the Vercel AI SDK with store: false, the OpenAI Agents SDK — round-trip it without knowing what it is.
• Restored onto its own message. The gateway opens the item at the edge and restores the parts onto the assistant message it was issued with: the message item whose id it names or, when the client drops ids, the assistant message right before it. It is never forwarded upstream; when routing falls back to another provider, that provider ignores it.
• Edited messages lose it. If the message's text no longer matches the signed parts, the gateway sends the client's text without signatures.
• No expiry, no errors. Conversations last for months, so items do not expire. One that cannot be opened — its key was removed from ENCRYPTION_KEYRING, it was tampered with, or its version is unknown — is dropped with a warning and the request continues without it. Drops are counted in ai_gateway_message_state_discarded_total by reason.
• The message item keeps provider_specific_fields.google.content_parts for LiteLLM-style clients. When both arrive, the reasoning item wins.

Anthropic thinking blocks (message-bound)

Claude returns signed thinking and redacted_thinking blocks. Anthropic requires the thinking of a tool-use turn to come back with the tool results, and on Claude Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5 it binds every block to all the content before it: an assistant turn must come back exactly as returned, block order included, or a later request fails with 400 ("The block is bound to a different conversation").

On /v1/responses, when the request includes reasoning.encrypted_content, the gateway seals the turn's whole content — thinking, text and tool_use blocks, in order — into the same kind of reasoning item, after the last item of the turn. A turn that only calls tools gets one too: it names the turn's first call_id instead of a message item.

• Replayed as returned. On replay the gateway sends the turn to Anthropic as one assistant message with the original blocks, as long as its text and tool calls are unchanged. A turn the client edited is rebuilt from the client's items, thinking first, as before.
• No gaps. Anthropic accepts thinking trimmed from the start of a conversation, not a gap in the middle. When a carrier cannot be opened, the gateway also drops the thinking of every turn before it.
• The message item mirrors provider_specific_fields.anthropic.thinking_blocks and content_blocks for LiteLLM-style clients, and /v1/chat/completions carries the same fields. A turn that only calls tools has no message item to mirror them on.

On /v1/messages

Anthropic clients send every content block back verbatim, redacted_thinking included, so that is the carrier here.

• Other providers' state. When Gemini signs a text response or OpenAI returns encrypted reasoning, the response ends with one more block, { "type": "redacted_thinking", "data": "bfrs_1.…" }, sealed like the /v1/responses item. The gateway opens it when the client sends the message back and restores the state onto that same message; Claude never receives it, and a turn holding one can still be served by any provider. Gemini state whose text was edited is dropped.
• Claude's own blocks stay native, in the order Claude returned them, including thinking blocks with no visible text, whose signature carries the reasoning. The gateway sends an assistant turn back to Anthropic exactly as the client holds it, as long as its text and tool calls still match.

Storage

None of this uses the database. The response_states table only persists responses the client explicitly stores (store: true on /v1/responses) for previous_response_id / item_reference chaining.