Prompt caching

Reuse stable prompt prefixes across multi-turn Prism requests.

Prism automatically caches eligible prompt prefixes. Repeated system prompts, tool definitions, repository context, and conversation history can reuse work when their leading tokens stay identical.

Keep the prefix stable

Put stable content first and frequently changing content last:

1. System instructions
2. Tool definitions
3. Repository or reference context
4. Conversation history
5. The newest user message

Changing one token near the beginning of the prompt prevents reuse for the content that follows it.

Set a conversation key

For Chat Completions, send the same prompt_cache_key on every turn in one conversation:

const response = await client.chat.completions.create({
  model: "prism-glm53",
  prompt_cache_key: "conversation_7f3a2c",
  cache_ttl: "1h",
  messages,
  tools,
});

Use an opaque, stable identifier. Do not put email addresses, API keys, or other secrets in the cache key. prompt_cache_key accepts up to 512 bytes.

cache_ttl accepts 5m, 30m, 1h, 6h, or 24h. Invalid values return 400.

You can also send the identifier as a header:

x-session-id: conversation_7f3a2c

When both are present, x-session-id takes precedence over prompt_cache_key. Use one mechanism consistently for a conversation. A cache key provides affinity and is not an idempotency key.

Preserve byte-stable inputs

• Serialize tool schemas deterministically.
• Keep tool order stable between turns.
• Avoid timestamps and request IDs in system prompts.
• Append new conversation turns instead of rebuilding older messages.
• Reuse the same model within a conversation.
• Keep large, shared context before the newest user input.

Multi-tenant safety

Generate a different key for each tenant and conversation. Never reuse one customer’s key for another customer, even when their prompts look similar.

The cache is an inference optimization. Your application remains responsible for conversation storage, retention, and access control.