Invalidation: Everything That Breaks a Cache

23.1 The Hierarchy #

Caching follows the rendering order—tools → system → messages—and a change at any level invalidates that level and everything after it.¹² The table is request-level: it describes what changes between two API calls. A harness sitting in front of the API can route the same user-facing action differently, and §23.1’s closing paragraphs cover the two cases where it does.

The top row is the expensive case, and it is a claim about the definitions in the submitted prefix rather than about servers. Changing the set of tool definitions in the request rewrites the block that renders first, which invalidates everything after it. Whether an MCP server change reaches that row depends on how its tools load (§17.1). Where tool search defers them—the default on supported models—a server connecting, disconnecting or changing its tool list appends content and leaves the cached prefix intact.⁴⁸ Where definitions load into the prefix instead, adding one invalidates the cache and so does deliberately removing one: that is the case wherever tool search is unavailable or disabled, which the vendor documents for pre-4.5-generation models on Google Cloud’s Agent Platform, for a custom ANTHROPIC_BASE_URL gateway, and for an Azure-hosted Foundry deployment that rejects tool search.⁴⁸

Even on that eager path the trigger is narrower than “a server changed.” A stdio server whose process simply exits leaves the definitions and the cache alone; a call to one of its tools returns an error rather than running. A remote server that reconnects on its own normally does the same, with one documented exception that proves the rule above: a request sent while it is reconnecting can add the WaitForMcpServers tool if the conversation has not listed it yet, and that addition invalidates the cache once before the tool stays listed for the remainder of the session.⁴⁸ The server’s own definitions never changed—the set did, which is the thing the cache is keyed on. Otherwise it is connecting, adding tools, or removing one on purpose that costs you. And editing the MCP config file changes nothing by itself; the cost arrives at the restart that connects or disconnects something. This is why “settle your tool set before you start” is a practice rather than fussiness: it costs nothing, and it is what keeps the eager case from surprising you.

Two rows carry exceptions, and both are the same lesson as the reconnection case: the cache is keyed on what the request actually contains, so how a control reaches the request decides what it costs.

Effort. Changing the request-level output_config.effort value always invalidates message blocks, and the model-specific columns above apply to tools and system. Two paths avoid that. Setting effort explicitly to the model’s default is equivalent to omitting it and does not invalidate at all. And on models supporting per-message effort, an effort change carried in a role: "system" message inside messages leaves the cached prefix intact.¹² Claude Code uses that second path where it can: on Opus 5.5 and Fable 5.1 with an API key or a Claude subscription, /effort keeps the cache and applies without a confirmation prompt; on other models it warns you first because the next request reads the whole history uncached. The exemption does not extend to Bedrock, Google Cloud’s Agent Platform or a Claude apps gateway, nor where experimental betas are disabled or a HIPAA configuration applies—and on Fable 5.1 it arrived in v2.1.260, so an older installation still pays.⁴⁸

Fast mode. At the API, switching between speed: "fast" and standard speed invalidates system and message caches.¹² A CLI toggle is not the same event. Claude Code’s fast-mode header is part of the cache key, so the first request sent with fast mode on reads the whole history uncached, at fast-mode rates—which is why enabling it early in a session costs less than deep into a long one. That cost lands once per conversation: afterward the header stays and only the speed value varies, which is not part of the key, so turning fast mode off, falling back to standard speed after a rate limit, and turning it on again all keep the cache. /clear and /compact reset that state, since they rebuild the cache anyway.⁴⁸

Neither exception makes the rows above wrong, and neither generalizes: a thinking-mode or budget change is not covered by either, and a direct API speed change is not cache-safe because a CLI toggle can be.

23.2 The Six Habits #

All of them follow from one idea: caching is a prefix match, so what matters is what changes and when. They are listed separately because each has a distinct trigger.

23.3 The Silent Ones #

There are four causes that produce neither an error nor an obvious symptom.

Non-deterministic JSON key ordering. Some languages randomize map iteration order during serialization. If your tool arguments or tool results serialize with different key order between requests, the bytes differ and the cache misses. Anthropic’s troubleshooting guidance names Swift and Go specifically.¹²

# Force stable ordering anywhere you serialize into a prompt.
import json
payload = json.dumps(data, sort_keys=True, separators=(",", ":"))

// C#: System.Text.Json preserves property declaration order for POCOs,
// but Dictionary<string,T> ordering is not guaranteed across runtimes.
// Sort explicitly when the result enters a prompt.
var ordered = new SortedDictionary<string, object>(data);
var payload = JsonSerializer.Serialize(ordered);

A timestamp in the prefix. Any now() above the breakpoint guarantees a miss on every request. Move it below.

A gateway stripping cache markers. If you route through a proxy or gateway, it may drop cache_control markers while returning success—in which case your entire conversation bills as uncached input on every turn with no error anywhere. Verify by checking that cache_read_input_tokens is nonzero after the second request.

Falling below the minimum. Shortening a prefix past the model’s floor silently disables caching (§22.5).

23.4 A Diagnostic Procedure #

When hit rate drops and the cause is unclear, work through the following: