Amazon Comprehend Custom Entity Recognition addresses a specific design risk when AI-assisted workflows meet real records and real people. This guide works through a hypothetical community organization example, showing what the documented feature does, what it does not prove, and where human review belongs. The examples use invented names and identifiers; we did not conduct product tests, inspect customer data, or measure results. The four diagrams are schematic teaching images, not vendor screenshots or records of a successful deployment.
Define the difference between a code and a lookalike
Imagine an archive containing donor letters that reference collection COLL-17 and storage shelf C0LL-17, where the second string uses a zero. An entity extractor may highlight both as plausible catalog references, but only the first is an approved collection identifier.
The operational objective is to prepare a human review queue of candidate collection codes, not to rename stored records automatically. Preserve the scanned letter, transcription, page or line location, and catalog authority list. All identifiers in this example are fictitious and the images are explanatory drawings.
Choose custom entity recognition for a narrow label
Amazon Comprehend custom entity recognition learns organization-defined labels such as COLLECTION_CODE rather than necessarily providing the standard built-in entities. The AWS documentation notes that custom recognizers identify only entity types included in their training; built-in types such as PERSON or DATE are not implicitly bundled into a custom model. Name each entity for the information your application truly needs. Avoid conflating location codes, document numbers, and collection IDs under a broad “reference” label, because the review team needs to understand why a span was marked.
Build a source-preserving annotation guide
Write an annotation guide with positive examples of a collection code, negative examples of shelf labels, and tricky cases such as OCR zero-versus-letter confusion. Define the exact character span that belongs to each label.
For plain-text annotation CSV, AWS documents File, optionally Line, Begin offset, End offset, and Type columns depending on the document layout. Offsets belong to the supplied text, so editing whitespace after annotation can misalign labels. Keep both raw OCR and normalized text, with a reproducible mapping back to the source image.
Select training data without accidental leakage
Use authorized letters from several years and writers, and separate evaluation documents from training documents. If repeated copies of one letter appear in both sets, apparent quality can be misleading. AWS supports training from annotations or entity lists; those are distinct approaches with different requirements.
An entity list combines example values and a corpus where they are expected to appear. The documentation recommends sufficient matches per entity type and warns against common terms that incorrectly match ordinary text. Do not claim a model was trained merely because a CSV was generated.
Route candidates through a human catalog lookup
The conceptual process diagram shows scanned letter, text extraction or annotation, custom recognizer, and approved collection lookup. Recognition produces candidate spans; it does not prove the code is valid, current, or attributed to the right object.
Validate candidates against a controlled registry, with exact-character comparison and reviewer-visible source coordinates. If the candidate is not present or conflicts with a shelf reference, mark it unresolved. Never create a catalog accession solely from a statistical entity suggestion, especially when an OCR error can alter one character.
Compare a confirmed code with a visual near miss
The comparison illustrates COLL-17 versus C0LL-17 in large letters so the letter O and digit zero are clearly different. A risky pipeline normalizes both to one collection and publishes a mistaken provenance claim.
The reviewed pipeline flags the uncertain span and consults the source letter and catalog authority before linking it. These examples are intentionally synthetic; there is no measured precision rate or live AWS output shown. Test variations in capitalization, punctuation, damaged scans, and handwritten corrections so the workflow catches more than a single toy example.
Interpret confidence without treating it as identity
A model confidence value helps rank candidates for review, but does not make a source text genuine or a collection code authoritative. Examine false positives for shelf tags and false negatives for legitimate codes in a held-out set.
Note whether errors cluster around scans with ink bleed, small fonts, or unusual document structure. When a document lacks an approved registry match, avoid publishing the code as validated. Depending on volume and risk, a simple deterministic lookup plus human review may be better than maintaining a trained model at all.
Protect donor privacy and document permissions
Donor correspondence can contain names, addresses, and sensitive acquisition notes unrelated to collection-code extraction. Minimize the text submitted for training and inference, check document rights, and restrict output visibility by user role.
Do not return a scanned personal letter to a public search user merely because one code was detected. Store original document identifiers and annotation provenance under the archive’s retention policy. If a letter is withdrawn or corrected, update the search index and dependent links as part of an established catalog change-control process.
Audit span coordinates and publication boundaries
The review illustration identifies four checkpoints: span accuracy, source-page context, authority-registry match, and staff approval. Ask reviewers to inspect both matching and mismatching cases, and require a way to report an incorrect link after publication.
A near-match should produce an explicit uncertainty flag rather than a silently corrected identifier. Preserve reviewer notes with stable IDs so future model updates can be compared against the same known examples. Nothing about the diagram claims an actual custom recognizer was created or evaluated.
Decide whether model maintenance is justified
Training, inference, permissions, region support, and possible costs need current AWS verification. Source patterns may change when the archive adds collections or alters its naming scheme, requiring new examples and reevaluation.
AWS documentation also distinguishes supported annotation formats and language limitations for some workflows; confirm those requirements for your actual input type. Publish a clear editorial rule: extracted codes are candidates until approved by the catalog team. This guide uses fictional letters and did not run a Comprehend training or inference job.
How We Research
For this November 2026 article, we reviewed vendor-maintained documentation: AWS: Custom entity recognition, AWS: Training custom entity recognizers, AWS: Entity lists for custom recognition, AWS: Plain-text annotation CSV. We compared documented capabilities, settings, and explicit limitations with the fictional scenario, then checked where source revisions, missing evidence, or human permissions could change the answer. We did not use a paid production account, benchmark models, process real visitor or donor information, or obtain approval from a vendor. Example diagrams and names are educational. Before implementing this workflow, recheck the provider change log, model availability, region, price, privacy controls, and current API reference; obtain independent review for any safety, legal, or accessibility decision.
Frequently Asked Questions
No. It identifies the custom types included in training; built-in entity types require their own supported processing or training.
No. Entity lists and annotations are different supported approaches with different input constraints and quality considerations.
No. Compare extracted strings against the authoritative catalog and original document.
To identify false positives and misses on documents the training workflow has not already seen.
Not by default. In accession codes those characters can distinguish different identifiers, so preserve the original and request review.