Why a batch status is not an evidence guarantee
A museum librarian uploads a folder of grant conditions, rights policies and application guidelines to an OpenAI vector store. The file_search workflow starts answering questions about eligibility, but one critical PDF never finished indexing. A fluent answer might omit a funding restriction even though the other files are searchable. The vector store file-batch API exposes processing state and file counts; it does not certify that the chosen PDF collection was complete, current or legally controlling. Build the retrieval release around an external source manifest, per-file processing checks and human review of important evidence. Keep the original documents outside the index as authoritative records and regard generated summaries as secondary, reviewable outputs.
Build a frozen manifest before ingestion
List every intended file with a local document identifier, exact edition, approved source URL or archive path, owner, hash and expected file ID after upload. Put grant deadlines and restricted eligibility clauses in a separate high-priority checklist. Do not rely on counting files in a temporary directory; the directory may contain old scans, duplicate PDFs or an unauthorized revision. For a collection updated during the week, freeze the specific edition being indexed and treat later uploads as a new, separately reviewed release. The manifest should explain which files are intentionally excluded, because absence can otherwise appear identical to a failed upload. Remove patron and donor information from examples used for test queries.
Create and poll batches with bounded scope
The API supports attaching multiple file IDs to a vector store through a file batch, with a documented maximum of 2,000 files in one batch. The request can apply shared file attributes and chunking configuration, or individual file options through the files array. A successful create response normally means the batch exists, not that indexing has finished. Persist the vector store ID, batch ID and exact source manifest together. Poll the batch status instead of guessing from an elapsed delay. Prepare for in_progress, completed, cancelled and failed states, and capture the full set of file_counts rather than looking only at a single green status label.
Reconcile counts and individual failures
The file_counts object separates in_progress, completed, failed, cancelled and total files. Compare those numbers against the frozen manifest before enabling new queries. Use the batch’s list-files endpoint, including its status filter and pagination, to inspect individual records. Failed vector store files can expose a last_error code, such as unsupported_file or invalid_file, and an explanatory message. Distinguish an unsupported document that needs conversion from a transient service error that may justify a controlled retry. Never silently discard a bad PDF and call the rest of the grant collection complete. Preserve the error and route the document to a named reviewer.
Compare missing, partial and approved evidence
The diagram presents three conceptual states. In the first, an expected eligibility PDF has no indexed counterpart, so the research set is incomplete. In the second, most files show completed but one high-impact document is still in progress or failed; successful retrieval from other documents could create false confidence. In the third, every intended source has a reconciled file ID, acceptable status, verified edition and successful representative retrieval check. The last state is an operational review gate, not an assertion that the model will always cite correctly. A green batch status cannot substitute for inspecting whether the correct passages were actually extracted and are findable.
Validate retrieval beyond ingestion success
Prepare targeted test questions from the frozen source checklist: a grant deadline, an exception clause, a required signature and an explicitly prohibited funding use. For each, record the expected source document and the exact section where the answer should be grounded. Run controlled file_search queries and review the returned citations and text against original PDF pages. If the workflow cannot retrieve a key exception, adjust preprocessing, file quality or source coverage before releasing the assistant. A retrieval test should include deliberate traps, such as two similarly named annual grant notices, to catch wrong-edition citations. These are proposed tests, not claims of measured OpenAI retrieval quality.
Handle retries without creating a misleading corpus
Before retrying a failed file, decide whether the source PDF is malformed, encrypted, truncated or merely subject to a transient processing error. Correct the document at the source and record the new hash; never quietly replace one edition while keeping an old audit label. Avoid attaching the same document repeatedly and then trusting whichever version ranks first. Record the cleanup or replacement action in the manifest, including the relationship between previous and current file IDs. A batch may finish while files from earlier batches remain in the same vector store; therefore check the full active corpus rather than assuming the latest batch is the only material searchable.
Guard editorial and privacy boundaries
Not all files suitable for a staff archive are suitable for a public question-answering index. Separate internal finance drafts, private donor correspondence and unpublished agreements from approved public grant conditions before ingestion. A metadata label can help retrieval filters, but access controls must also be enforced at the application and document levels. Do not promise zero retention based on batch status or on the presence of a file attribute. Review current OpenAI data-control documentation, your account settings and your organization’s records policy. If the citation points to a restricted or outdated document, block that answer even when the ingestion process succeeded.
Use a release gate that someone can audit
Give the reviewer a compact release report containing the frozen manifest, current batch counts, unresolved file errors, corpus inventory and sample source-citation results. Require an explicit decision for any missing or failed high-impact document, with a visible limitation statement if the assistant must operate without that topic. Keep the prior approved index available until a new one is verified; do not assume switching a front end to the newest store is harmless. After deployment, monitor new uploads, failed ingestion and outdated document editions as separate risks. The service provides useful processing telemetry; completeness, authorization and factual accuracy remain responsibilities of the team using it.
How We Research
Our research consulted these named official documents: OpenAI: Create vector store file batch; OpenAI: Retrieve vector store file batch; OpenAI: List vector store files in a batch. We checked tool behavior, version-specific options, and stated limitations against an illustrative museum scenario. The examples are suggested implementation and editorial review procedures, not production tests. We did not call the services with live patron records, benchmark latency or accuracy, measure storage savings, or obtain third-party consent. All images are original explanatory drawings rather than screenshots or claimed measured results. Verify provider documentation, service availability, access policies and current organization rules before adoption; keep final publication approval with authorized staff.
Frequently Asked Questions
No. Batch completion describes file processing, not source completeness, retrieval recall, current policy or factual accuracy.
List files in the batch and filter for failed records, then inspect each file status and last_error.
Yes, subject to the documented maximum of 2,000 files per batch; use smaller audited groups when that helps review.
Avoid blind retries. Verify the edition, format, error type and whether a replacement would duplicate existing indexed content.
A missing or unverified high-impact source, incorrect citations, unauthorized documents or unresolved eligibility clauses should block publication.