Prompt caching

There is nothing to enable. When a model serves part of your input from its prompt cache, those tokens are reported as cached (usage.prompt_tokens_details.cached_tokens on the OpenAI wire, usage.cache_read_input_tokens on the Anthropic wire) and billed at the model's cache-read rate from the price page. A model with no cache-write rate charges nothing for writing to its cache.

Conversation affinity#

A prompt cache is only warm where the earlier turns of the same conversation were served. Sator keeps a conversation together by deriving a key from the request itself — the model, the first system message and the first non-system message — so a multi-turn chat that keeps its opening messages stable is routed consistently and its cache hits keep landing, with no work on your side.

Keying it yourself#

If your requests share a prefix in a way the derived key cannot see — several conversations that begin identically, a batch job, sharded workers — name the conversation yourself:

curl https://sator-api.princep.org/v1/chat/completions \
  -H "Authorization: Bearer $SATOR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "deepseek-v4-flash", "prompt_cache_key": "ticket-4821", "messages": [{"role": "user", "content": "…"}]}'

• The key is a routing label, not content: up to 256 characters, otherwise 400 invalid_cache_key naming the field or header it came in. Hash anything longer yourself.
• Your key wins over the derived one when both apply.
• Keys are scoped to your account before use, so two customers who both call a session "session-1" cannot collide with or steer each other.
• On the OpenAI wire the scoped key is also passed to the model as its own prompt_cache_key hint, so routing and caching agree. On the Anthropic wire the key routes the conversation and nothing is added to your request body.

What to expect#

Cache hits depend on the model and on your prompt staying byte-identical up to the point of divergence: put the stable material (system prompt, tool definitions, documents) first and the changing material last. Whether a given turn hit the cache is in the response's usage block, and what it cost is in the dashboard.