Why batch order matters more than throughput
A fictional museum needs semantic search over hundreds of short exhibit descriptions. Sending multiple labels in one embeddings request is convenient, but convenience introduces a reconciliation obligation: every vector must be assigned to the accession record that produced its text. An accession identifier is a database key, not a semantic concept that the model understands or verifies. Even a correctly returned numerical vector can become operationally wrong when it is attached to the next row in a spreadsheet. The useful unit of work is therefore a stable record-to-input mapping, not merely a successful HTTP response. This guide describes a design review, not a measured import benchmark.
Freeze a versioned manifest before making the request
Prepare a local manifest containing a stable accession ID, text revision, canonicalized input string and an ordinal input position for every row. For example, a fictional artifact A-014 may occupy input position zero, while B-021 occupies position one. Treat the position as temporary within that exact request; the accession ID remains authoritative across requests. Store a digest of the normalized text, the selected model and intended dimensions so an editor can tell whether the source changed between batches. Do not include private donor notes or restricted conservation details merely because a larger batch is more efficient. Data minimization still applies to each input.
Use the documented array input and respect limits
The Create embeddings endpoint accepts an array of strings as input, not only a single string. The API reference also documents per-input token limits, an aggregate token limit across the request, and a maximum number of array elements. Validate the present limits against the documentation and tokenizer for the exact model before submission; do not infer limits from file size or character count. Reject empty strings and record them as skipped source rows rather than quietly removing them after assigning positions. A skipped row changes every subsequent ordinal if the manifest is not regenerated. Partition long archives into smaller deterministic batches with their own identifiers and record counts.
Map response data.index back to the manifest
The response has a data array of embedding objects, and each object includes an integer index. Use that returned index to look up the corresponding item in the exact request manifest. Then attach the embedding to that item’s accession ID and text revision. Do not rely solely on the order in which a JSON parser iterates objects or on the hope that a later asynchronous worker preserves source rows. Validate that each index is an integer in the permitted range, appears only once, and corresponds to an expected row. Count the results and record the response model and usage separately. Reject rather than silently repair any ambiguous mapping.
Keep vector configuration consistent inside the index
Even a perfectly matched ID is not enough if vector settings differ. Save the model identifier, the configured dimensions where applicable, and the embedding encoding format with the ingestion run. OpenAI documents the dimensions option for text-embedding-3 models and float or Base64 output encodings; these choices affect what your vector store should accept. Verify decoded values and dimensionality before a database write. Never append a vector of one dimension to an index configured for another, and do not treat numeric similarity as proof that the correct record is being returned. Retrieval must still join against the authoritative accession catalog.
Compare the unsafe and reconciled import paths
In the unsafe workflow, a worker removes one invalid label, calls embeddings on the shortened list, then writes returned vectors alongside the original spreadsheet rows by position. Every record after that omission can shift. In the reconciled workflow, the worker builds a new manifest after validation, calls the endpoint, and uses each returned data.index against the frozen manifest. A simple review table can show requested position, accession ID, returned index, text revision and write status. This example illustrates a failure mode; it does not assert that the API naturally reorders responses or that such a mix-up was observed during testing.
Retry only unresolved work with an idempotent ledger
Network timeouts create another subtle hazard. The client may not know whether an API call completed, and a blind retry may produce a second billable request. Keep a ledger keyed by a local batch ID and source revision, recording request submission, response receipt, mapping validation and storage commit separately. If storage succeeded but acknowledgment failed, check the vector store before inserting again. Retry only rows whose final database state is unresolved, and build a new manifest for the retry subset. Avoid presenting duplicate vectors as distinct catalog evidence. Token usage data can inform cost accounting, but is not a receipt for a database write.
Run a human-checkable sample before broad ingestion
Pick a small set of fictional descriptions with similar wording but distinct accession IDs, such as two ceramic bowls with different dates. Produce an expected mapping table independently of the embeddings response. Verify the exact vector count, unique indices and database keys, then retrieve several IDs through the search interface and inspect the original source records. A similarity search can return imperfect rankings even when the mapping is correct; the QA objective here is identity integrity, not ranking quality. If a result points to the wrong accession, stop the release and trace its request manifest before adjusting relevance settings.
Document boundaries and release criteria
Sign off only when every accepted source row has one traceable mapping or a documented exception. Keep the request manifest, model configuration, validation log and non-sensitive sample results together. Do not log raw restricted texts or API keys in a public build report. Reconcile reruns by accession ID and text revision rather than ordinal alone. The release checklist should distinguish service acceptance, vector generation, index storage, retrieval identity and curatorial approval as five different milestones. Passing one milestone does not imply that the others occurred, and a high similarity score never substitutes for accession provenance.
How We Research
For this guide we reviewed official, named primary sources on November 10, 2026: OpenAI Create embeddings API reference; OpenAI vector embeddings guide. We cross-checked parameter names, response structures and documented limits against the described use case, then designed a hypothetical, non-sensitive museum workflow. The diagrams explain possible paths rather than represent live tests, screenshots, measured performance or a verified compliance review. Pricing and service behavior can change; readers should confirm the current reference, test with fixtures and obtain organizational approval before any production change.
Frequently Asked Questions
Yes. Its input parameter supports an array of strings. Your application still needs to respect current token and element limits, and must preserve each source record’s identity.
Use the documented embedding data.index and the frozen manifest for that exact request. Confirm indices are unique and complete before associating vectors with accession IDs.
Rebuild the input-position manifest after filtering. Otherwise, one removed row can shift later positions and create incorrect source-to-vector associations.
No. Similarity measures relationship between vectors, not database provenance. Check the stored accession ID and original catalog record independently.
No. API generation and vector database commit are separate operations. Keep an idempotent ingestion ledger and verify the destination state.