Start with a sentence that needs evidence, not a score
Imagine a curator asks a language assistant for a 60-word label describing a donated brass instrument. The draft reads smoothly but mentions a maker and date that still need accession-record checks. A model may strongly favor a particular token sequence even when the underlying statement is unsupported. Output token probabilities can help an editor understand variations in a draft, but they do not establish provenance. The practical goal of this workflow is to inspect generated wording and then review claims against the museum collection database. The instrument, label, and audit described here are illustrative; they are not a tested publishing case.
Find the documented Responses include option
The OpenAI Responses API reference lists message.output_text.logprobs among values that can be requested through the include array. A response output message may contain output_text objects with text, token, bytes, logprob, and top_logprobs data. This is additional output detail, not a separate source retrieval operation. First confirm the chosen model and response configuration support the requested diagnostic field; optional fields may be absent, and callers should handle that case explicitly. Do not invent missing values by estimating from surface text. Keep a model/version identifier and the actual request settings with your internal review record.
Distinguish tokens from words and sentences
A token is not necessarily a complete English word: punctuation and word fragments may be separate units. The documented logprob value refers to a selected token in the output sequence, while top_logprobs describes alternative token candidates at that position. These numbers are conditional on model context and previous tokens. They are not calibrated probabilities that a whole exhibit label is historically correct. Editors should inspect the particular phrase and its source references; adding the numbers together and calling the result a fact-confidence score is unjustified without a validated evaluation method.
Create a constrained editorial test case
Use an approved, nonsensitive example label and keep a versioned copy of the source record. Ask the model to produce a brief draft in a specified reading level, and request the supported logprob include field only if the integration allows it. A reviewer might highlight the maker name, year, and material description in the returned text. Treat the returned token list as an annotation of generated language rather than instructions to automatically change claims. Capture one request ID, the rendered text, and the exact source excerpt used for checking; avoid logging visitor or donor identifiers just to make a debugging chart.
Read alternatives without overreacting to a low value
An illustrative chart may show several candidate tokens around a phrase such as “made in” versus “attributed to.” This can alert an editor to wording that deserves clarification, but a low likelihood can also result from rare names or specialist terminology that is correct. Conversely, a high likelihood may simply reflect a common phrase. Review the candidates in the context of the full sentence, not as isolated indicators of error. Neither the highest-ranked alternative nor the originally selected token should replace the museum record without a person confirming the specific claim.
Compare the diagnostics with an independent evidence trail
Build a simple two-column review note. On one side, record the draft claim and the relevant token span; on the other, list the accession card, conservation report, or verified bibliography that supports or contradicts it. For example, if a generated label assigns a year but the object record only provides a decade, the safe revision should reflect the documented uncertainty regardless of token likelihood. Keep a link to the authoritative source version. This comparison is an editorial procedure, not a measure of model quality, and it does not require exposing confidential collection files to a public analytics service.
Avoid a false numerical confidence badge
It is tempting to convert log probabilities into a green or red confidence indicator for a public museum website. Resist that shortcut. Token likelihood values are context-sensitive and may not compare meaningfully across prompts, model families, or output formats. If a project wants uncertainty labels, define them through source quality and editorial review rules, then test them against independently labeled examples. An accessible exhibit page should communicate provenance in plain language instead of displaying unexplained decimals. The token diagnostic belongs in a restricted drafting tool, not in the published historical claim.
Handle refusals, tool output, and missing token detail
A response output array may contain several item types; do not assume the first item is a completed assistant text message. Check item type, message status, and content type before reading logprobs. Tool calls, refusals, and other content do not necessarily have the same token annotations as ordinary output_text content. A missing field is a supported operational outcome to record, not evidence of zero uncertainty or zero likelihood. When a response is incomplete or has a refusal, stop the label approval path. The editor can still use established catalog sources to prepare a manual replacement.
Keep diagnostic logs proportional to the editorial purpose
Token-by-token records can reproduce large parts of a draft, including private internal notes if the prompt includes them. Restrict access, set a retention period, and avoid duplicating entire outputs in unrelated monitoring systems. Keep the fields actually needed to reproduce the investigation: draft version, model identifier, whether logprobs were present, the questionable phrase, and the source documents checked. Delete temporary copies according to the institution’s policy. An audit trail should show why a claim was accepted or corrected, not become an additional ungoverned database of museum information.
Know when to leave this feature turned off
For everyday copyediting, grammar checks, or public visitor chat, token diagnostics often add complexity without answering the real question. Turn them on only when the investigation has a defined purpose and the selected configuration supports them. A museum with limited engineering resources may get a better result from a citation checklist, versioned accession data, and a required human sign-off. The final outcome is a label whose claims are defensible independently of the model. A neatly shaped probability curve is not a substitute for the curator’s evidence.
How We Research
Our named primary official documentation sources were: OpenAI API Reference — Create a model response; OpenAI API Reference — Get a model response. We checked the published include option, output_text logprob object fields, alternative-token structure, and the fact that these are optional response details. We did not call an API, measure token values, or evaluate any real museum text. Check the current documentation, account permissions and model or API version before implementation. The figures are original editorial diagrams rather than live vendor screenshots or measured results.
Frequently Asked Questions
No. It describes token likelihood in model context, not verified factual correctness. Check the underlying record.
The Responses reference documents include: [“message.output_text.logprobs”]. Confirm your chosen model and endpoint actually return the optional data.
No. Tokens may be word pieces or punctuation, so interpret positions and byte sequences carefully.
Handle the field as optional. Record missing diagnostics and continue with source-based editorial review rather than inventing scores.
Not without independent validation; raw token likelihood is not a trustworthy public fact-confidence label.