A historical society has a long digitized committee minute book and wants to extract decisions by date. Sending every page to a model without checking size can lead to rejected requests, unnecessary expense, or a workflow too large to audit. Claude’s token counting endpoint can estimate input tokens for a structured message before generation. It is not a content quality check and does not establish that restricted archival records may be uploaded. This tutorial uses a fictional book and a proposed preflight procedure, not a token count performed with a live API key.
Decide what may be sent before counting tokens
Inventory the minute-book’s access restrictions, personal information, and digital rights. Some pages may contain donor information or confidential correspondence. A technically countable document is not automatically permissible to send to an external API. Identify an authorized source set, apply appropriate redaction or exclusion, and record the version to be analyzed. Keep the archival master unchanged and prepare a separate working copy with only necessary pages. Recheck the provider’s data policies for the intended account and endpoint.
Define the desired output narrowly, such as a table of motion date, decision, and exact source reference. Do not ask for a sweeping narrative that merges old and revised minutes without a trace. A preflight checklist should ensure each page or text block has a stable source ID. If source quality is poor, fix OCR ambiguities upstream. Token counting can reveal how large a request is, but it cannot tell whether a scanned vote tally was read correctly.
Build the actual message shape for preflight
The Claude Platform documentation exposes POST /v1/messages/count_tokens. It accepts a request shaped like the intended Messages API call, including the model, messages, system prompt, and supported tool specifications. Supply the same structure you plan to send for generation so the estimate includes operational overhead. A count of only the raw minute-book text would omit instructions, message framing, and any tool schema that will be included later. Keep model IDs explicit when comparing strategies.
For a PDF or image payload, the docs show base64 source content for token counting. They also describe cases where certain inputs accepted by message generation are rejected by the count endpoint: server tools, MCP connectors, and image or document blocks using URL or file sources. Your preflight code must detect these unsupported cases and route them to a separate planning path. Do not replace a rejected payload with an invented zero-token estimate.
Interpret input_tokens as an estimate, not an invoice
A successful count response includes input_tokens, representing the anticipated total of input tokens for the structured message. Anthropic warns that the count is an estimate and may differ slightly from usage in a generated request. Some model families also use different tokenizers, so recount against the exact intended model instead of copying an earlier count. Keep the count timestamp and model in the job log. Do not imply that token count alone predicts dollars spent.
Actual cost depends on current prices, cached input treatment, output length, and possibly model-specific pricing or tooling. The token counting endpoint does not use prompt caching to produce a cached-token billing estimate. It also says counting itself is free but subject to its own rate limits. Record the preflight quantity as a planning number, and reconcile completed work with actual message usage rather than reporting the estimate as a measured bill.
Design the size comparison for a real decision
Use a schematic of three request designs: a full book with instructions, selected pages with a provenance index, and too aggressively trimmed excerpts that omit decisions’ context. None of the visual bars represents live token measurements. The correct choice balances context, permission, quality, and capacity; it is not always the smallest request. A safe scope must retain the preceding and following language necessary to understand the motion and any amendment or dissent.
If a request is too large, split by dated agenda unit and keep stable page identifiers. Provide cross-page context where one meeting refers back to a decision in an earlier volume. De-duplicate repetitive boilerplate cautiously: a note that appears identical may have a different effect when amended later. Preserve a log showing which pages were included in which request. Review coverage so a short excerpt cannot silently stand in for the whole archival record.
Handle model windows and unsupported server tools
A count below a stated context window does not mean every available token can be consumed by inputs; generation output and other model or tool limitations also matter. Consult the context-window page for the model you intend to call. The token count endpoint cannot accurately preflight requests containing the unsupported server-tool configurations listed in its documentation. For those workflows, use the actual Messages response usage and a conservative, auditable budget instead of presenting an approximate number as exact.
Model changes can alter counts even with unchanged source text. If a batch job migrates to a different model, rerun preflight and recalculate the request grouping. Add a test where the request is accepted for counting but rejected for a separate message constraint. Handle failures without deleting source references or consuming a partial result as a finished record. The safe design treats token budget errors as workflow events that need clear retry and review rules.
Include human evidence review after generation
The token counting endpoint performs no research and returns no summary of the archive. After a permitted generation request, an archivist still has to compare extracted decisions with the scanned or transcribed page and record the correct citation. If a date is missing or a motion is ambiguous, leave it unresolved. A generated table that fits comfortably within the context limit can still contain fabricated attributions or merge two votes. Add reviewer status to each extracted row, not just the batch as a whole.
For privacy, avoid logging full minute text in cost dashboards. Keep request identifiers, source revision, planned page range, model, estimated count, actual usage, and exception status under the organization’s retention rules. Where a source’s access terms change, pause any reuse of derived text. Make the extracted records traceable so archivists can correct them without rerunning the entire volume, and keep generated summaries distinct from authorized transcriptions.
Test count discrepancies and decide when to split
Write synthetic messages representing a short excerpt, a long plain-text section, a PDF payload, an attached image, a tool schema, and a prohibited URL-sourced document block. Verify the endpoint’s allowed structures and error behavior against current documentation before production deployment. Store preflight results as estimates and compare with actual usage only when authorized live calls are later made. The comparison figure illustrates decision logic rather than fabricated token totals.
Before launching, verify active model IDs, rate limits, API prices, retention requirements, and context-window rules. Document the split policy, page-coverage checks, and who reviews extracted dates. The aim is to avoid avoidable request failures while maintaining an honest connection to the archive and its permissions. A successful preflight says the proposed message is size-plausible; only a completed generation and independent source review can produce a trustworthy historical summary.
How We Research
For this November 2026 source review, we read these named official references: Claude Platform Docs: Token counting; Claude Platform Docs: Context windows. We compared documented inputs, outputs, limitations, supported formats, and workflow prerequisites against the specific scenario. The images and examples are original conceptual illustrations, not screenshots, measurements, or real outputs. We did not create a vendor account, submit actual community artwork, member forms, seed catalogs, recordings, or restricted archives, or run a live API performance test. We do not claim accuracy rates, latency, tested prices, legal consent, or verified translation, and we do not infer those from the drawings. Before operational use, confirm current feature availability, rate limits, regional and data-handling requirements, pricing, and organization policy. Where permissions, linguistic accuracy, historical quotations, or authorizations matter, a qualified person must review the original evidence.
Frequently Asked Questions
No. It estimates input tokens for a proposed request; it does not generate text or evaluate source accuracy.
Yes for supported structured requests using base64 sources; the documented endpoint rejects image and document URL or file sources.
Not necessarily. Anthropic calls it an estimate, and actual generation usage, cached input treatment, and output affect billing.
No. The documentation says token counting does not use caching logic even when cache_control blocks are provided.
Yes. Permissions, privacy, source quality, tool compatibility, and output budget require independent checks.