Verified Prompt Caching Pricing Matrix & Discounts (2026)
Data verified against official documentation from OpenAI, Anthropic, and Google Cloud.
| Model Name | Standard Input / 1M | Cached Input / 1M | Discount % | Min Cache Tokens | Official Docs |
|---|---|---|---|---|---|
| Claude 3.5 Sonnet | $3.00 | $0.30 | 90% off | 1,024 tokens | Anthropic Docs ↗ |
| GPT-4o | $2.50 | $1.25 | 50% off | 1,024 tokens | OpenAI Docs ↗ |
| Gemini 2.5 Flash | $0.075 | $0.01875 | 75% off | 32,768 tokens | Google AI Docs ↗ |
The Mechanics of LLM Prompt Caching
In traditional transformer inference, every token in a request requires computing Key-Value (KV) attention matrices across all previous tokens. When applications send large system instructions, API schemas, or reference documents on every turn, the LLM cluster is forced to re-compute identical attention weights repeatedly.
Prompt caching allows foundation model providers to store the pre-computed KV cache directly in high-bandwidth GPU memory (HBM). When a new API call begins with the exact same prefix tokens, the inference engine resumes execution from the cached state, bypassing quadratic attention computation and slashing latency by up to 80%.
Architectural Rules for Maximizing Cache Hit Rates
Static Prefix Ordering
Always place completely static content (system instructions, tool schemas, few-shot examples) at the top of the context window. Dynamic values (current timestamp, user question) must always be placed at the very end.
Mind the Minimum Thresholds
Most providers do not cache short prompts. OpenAI and Anthropic both require a minimum of 1,024 prefix tokens. If your system prompt is only 400 tokens, you will pay full price unless you combine it with reference context.
Frequently Asked Questions
How does prompt caching save money?
When you send repetitive prompt prefixes (like system instructions or documents), providers keep the parsed KV cache in GPU memory. Re-queries that match the exact prefix avoid recomputation and are billed at a 50% to 90% discount.
What minimum token count is required for prompt caching?
Anthropic requires at least 1,024 tokens for Claude 3.5/3.7 to trigger prompt caching. OpenAI automatically caches prompts over 1,024 tokens in 128-token increments.
What happens if a single character in the prefix changes?
Prompt caching relies on exact sequential prefix matching. If you alter a single character near the beginning of the prompt (such as injecting dynamic timestamps or user IDs at the top), the entire subsequent cache is invalidated.