Why cache accounting needs its own audit
A small museum uses the Responses API to review draft grant narratives against a stable rubric. The rubric and approved policy excerpts are repeated across many requests, while each applicant question is different. Prompt caching can reuse a matching prefix and reduce some input-processing costs, but a cached prompt is not a free prompt. A team that records only total input tokens can miss which tokens were read from an existing cache, which were written to it, and which were processed normally. The practical goal is a reproducible cost ledger, not an invented claim that repeated prompts always produce cache hits. Use the actual response usage object for every completed request and retain the model and billing configuration alongside it.
Read the three input-token categories correctly
Start with `response.usage.input_tokens`, then inspect `usage.input_tokens_details`. Current Responses documentation distinguishes `cached_tokens` (tokens read from a qualifying cache), `cache_write_tokens` (tokens written to cache), and their optional lifetime breakdown. Input tokens that were neither reads nor writes are ordinary input. Do not add `cached_tokens` and `cache_write_tokens` to `input_tokens` to invent a larger total: these are components of the same input count. Likewise, do not use `total_tokens` as an input-only number, because it includes output tokens. Export the counters as integers, and label missing fields as unsupported or unavailable instead of silently replacing them with measured zeros.
Map the request to a stable grant-review prefix
For a hypothetical pilot, put stable instructions, the authorized rubric, fixed tool definitions, and a dated policy edition before the changing applicant question. If a new rubric revision or tool definition changes the prompt prefix, the previous cache entry may no longer match. Keep confidential applicant identifiers and variable facts after the stable section, and never shuffle the order of tool schemas merely to increase a dashboard metric. The OpenAI prompt-caching guide explains that matching applies to the rendered prefix, not just an apparently similar natural-language instruction. Cache behavior depends on the selected model, eligible boundaries, and retention policy; a long shared prefix is necessary in many cases but is not by itself proof of a hit.
Build a request-by-request usage ledger
Capture a local request identifier, timestamp, selected model, input token count, cached read count, cache write count, output token count, and the relevant usage breakdown returned by the API. Keep an internal reference to the rubric version so unexpected changes can be investigated. Record processing mode and cache lifetime settings only when you know they actually applied to that request. If usage is absent because a request failed, mark the record incomplete; do not assume it consumed no tokens or that billing will be zero. In a streaming integration, wait for the completed response usage data rather than estimating the final totals from intermediate text chunks. The ledger should avoid copying applicant passages or other sensitive prompt bodies into cost telemetry.
Do the cost arithmetic without double-counting writes
Consider an illustrative record with 12,000 input tokens: 4,000 cached reads, 7,000 cache writes, and 1,000 ordinary input tokens. If the write details report 3,000 tokens for a 30-minute lifetime and 4,000 for a 12-hour lifetime, those two figures partition the 7,000 writes. The input cost is the sum of ordinary input at the applicable ordinary rate, cached reads at the cached-input rate, and each write category at its applicable write rate. Use the actual model pricing page at billing time; prices and supported caching controls can change. This is an arithmetic example, not a benchmark or a claim about a live LabFin account. Keep output tokens and any tool charges on separate ledger lines.
Compare reads, writes, and ordinary processing
Three requests with similar total input tokens can have different bills. The first request may establish a cache entry and produce a larger write count. A later request may reuse much of the prefix as cached reads. A request after a rubric change, or after a relevant cache window has elapsed, may process more tokens without reusing the earlier entry. These are conceptual states, not guaranteed trajectories. In the reporting dashboard, compare the actual read share `cached_tokens / input_tokens` across a set of completed requests, while retaining absolute volume: an apparently high hit rate can still coexist with rising overall expenditure when traffic increases. A cache hit also says nothing about factual accuracy or whether the grant-review recommendation is fair.
Account for model-specific cache retention
Do not copy retention settings from one model generation to another. The current prompt-caching guide describes `prompt_cache_options.ttl` and breakpoint controls for supported GPT-5.6-and-later models, while earlier models use the older `prompt_cache_retention` parameter with different values. A minimum cache lifetime does not guarantee that a later request will reuse tensors or improve latency. In particular, extending the lifetime can change the write price, so a longer retention choice is not automatically a cost saving. Test compatibility against the deployed model and SDK, audit request errors, and document the chosen retention scheme in the ledger. For restricted grant records, involve the organization’s data-retention owner rather than treating a cache TTL as a substitute for a privacy policy.
Check a cache result without claiming an evaluation
A reproducible internal check would hold the rubric and tool list constant while changing only a non-sensitive question at the end. Inspect completed usage objects across multiple eligible requests and note whether `cached_tokens` rises when a matching prefix is reused. Then intentionally change the rubric version and see how the counters behave. Record all results and model settings, including requests that were not cache hits. This suggested procedure is not a test we have executed, and no expected percentage should be represented as a measured LabFin result. Never publish real applicant prompts merely to demonstrate a successful cache hit. A cache counter is a cost signal; the validity of a policy recommendation still needs human review against original grant documents.
Release checklist for a defensible budget report
Before reporting savings, reconcile request-level counters with the official organization usage and billing records for the same time window, model, and billing mode. Check that read and write categories fit within the input total, that write subtotals add up, that output is counted separately, and that failed or incomplete requests have an explicit status. Preserve the rubric edition and audit period, not the confidential applicant text. If a model exposes fewer usage fields, label the gap plainly and avoid interpolating a precise cost. Finally, do not equate caching with security isolation, compliance approval, deterministic answers, or correctness. A useful grant-review budget report says what the API measured, what the billing record confirms, and what remains uncertain.
How We Research
We consulted OpenAI’s official Prompt caching guide, the Responses API response schema, and OpenAI’s pricing page. We compared the named usage fields, cache read/write semantics, model-dependent retention controls, and the guidance on reconciling token categories. The museum grant scenario and numerical example are illustrative. We did not access a production organization, run a benchmark, confirm an actual cache hit, or measure cost savings. Deployment behavior, model support, and prices should be verified again when a real system is implemented.
Frequently Asked Questions
No. The cached-read counter is a breakdown of input usage. Adding it to input_tokens would double-count those tokens.
Not necessarily. A request can contain both cached reads and writes, depending on which eligible prefixes were reused and which new portions were written.
They represent categories of input tokens billed at their applicable cache-write rates. Count those tokens once, not once as ordinary input and again as writes.
No. Matching prefixes, model compatibility, eligible breakpoints, traffic patterns, and cache lifetime all affect actual reuse. Inspect returned usage counters.
No. Cache usage describes processing economics, not accuracy, fairness, evidence quality, policy compliance, or human authorization.