A fictional museum is building a touch-screen guide that explains a gallery route with synthetic narration and a simple illustrated speaking face. When the audio says ‘use the east entrance,’ the avatar should move its mouth in a plausible rhythm without distorting the route or hiding the warning. Azure Speech can emit viseme events and audio offsets alongside supported speech synthesis. The events help control an animation; they do not prove that the spoken words are correct or that the avatar represents a real person with permission.
Start with an approved script and an authorized likeness
Write and approve the exact guide script before choosing a face rig. A small change from ‘east entrance’ to ‘east exit’ can have operational consequences even if the phoneme timing still looks good. Mark safety warnings, pronunciation of exhibit names, accessible route descriptions, and any place where visitors must contact staff. The approved source should be versioned outside the animation project. Do not assume a generated voice or moving face can independently establish the correctness of museum directions.
An illustrated, non-identifiable character may avoid the likeness permissions associated with a realistic volunteer avatar, but teams still need appropriate rights to images, speech recordings, and voice models. If a real person is represented, obtain clear permission covering the intended channel and duration. Set an accessible alternative such as readable text or staff assistance. Some visitors prefer not to watch moving faces, and animation should never be the only place an essential warning appears.
Understand visemes, phonemes, and offsets
Microsoft defines a viseme as a visible mouth shape associated with a group of speech sounds. It documents 22 viseme identifiers including silence, along with audio offsets that locate the start of events in synthesized speech. Several phonemes may share one viseme; the mapping is not one-to-one. Therefore viseme events should be treated as animation signals rather than a complete transcript or an exact word-alignment file. They are produced for supported voices, locales, and output modes, not automatically for every voice or language.
Keep the synthesized audio file and its accompanying event sequence as one immutable version. Store the sample rate, voice configuration, script revision, and offset conversion used by the rendering engine. Audio offsets may be given in ticks of 100 nanoseconds, so converting them to milliseconds requires deliberate unit handling. A millisecond-versus-tick mistake can place mouth poses far from the audible speech. Display both the waveform and event marks during staff review rather than guessing from visual smoothness alone.
Choose the appropriate facial representation
The Azure Speech documentation describes viseme IDs, SVG animation support for specified locales, and blend shapes for supported use cases. A 2D museum illustration may need only a few drawn poses, while a 3D character might use a blend-shape system. SVG availability is not universal; Microsoft’s page notes that SVG support is limited to en-US, and supported locales for other outputs must be checked. Do not promise a particular voice, language, or face-rig output without confirming current compatibility.
Design simple mouth artwork that remains readable on a dim kiosk. Keep facial expressions neutral when delivering a safety instruction, and do not animate gestures that could imply the wrong direction. Define an explicit fallback pose when an event is missing. The avatar should pause or return to neutral rather than cycling through arbitrary lips. Avoid representing one viseme ID as an exact spelling: visible mouth shapes collapse multiple speech sounds into the same drawing.
Build a time-aligned storyboard
The process graphic shows an approved sentence, generated speech, a sequence of event ticks, and rendered mouth poses. It is a conceptual timeline, not captured Azure output. The speech signal continues between individual event marks; an animator may interpolate poses, but interpolation does not create new speech evidence. Anchor animation to audio playback time rather than to the variable speed of a browser drawing loop, and verify what happens when the viewer pauses, resumes, or scrubs.
For example, the fictional guide says, ‘Please use the east entrance.’ Place the audible sentence, displayed caption, and visual mouth sequence on the same reviewed time axis. Check the word ‘east’ carefully because a wrong direction is a safety and accessibility issue, regardless of how convincing the face looks. The timeline illustration uses simplified waveform bumps and face poses to show synchronization without inventing live event values or vendor-generated screenshots.
Compare lagging and aligned animation
In a reviewer comparison, one mouth begins moving after the audio has already started, another holds a wide-open pose through a silence, and a third changes poses near the relevant speech events. These are intentionally exaggerated cartoon examples, not service measurements. The correct outcome is not perfect photorealism; it is an animation that neither distracts from nor contradicts the spoken and captioned instructions. A seemingly aligned mouth does not guarantee accurate wording or good pronunciation.
Inspect the clip at ordinary speed, with subtitles visible, and while navigating rapidly between sections. Test under reduced-motion settings and with the animated face hidden. If comprehension falls when the animation is disabled, the interface is too dependent on it. Staff can approve an acceptable low-motion or text-only fallback even if the animated version looks more polished. Treat motion quality and content accuracy as different acceptance decisions.
Check captions, warnings, and playback accessibility
Provide text captions that reflect the approved spoken script, including the full emergency instruction. Viseme identifiers are not captions and cannot reliably reconstruct words. Audio volume, pacing, pronunciation, text contrast, keyboard access, and non-visual navigation need their own review. If the guide names a restricted door or a temporary accessible route, require a current staff-approved source rather than a sentence generated from old exhibit material.
When narration is re-synthesized, regenerate or review captions and viseme timing together. Do not retain old event offsets against a new audio file because even a small voice or speed change can shift the phrase. Test kiosk pause and interruption behavior so a resumed mouth does not appear to speak while the speaker is silent. Provide a visible option to turn animation off, and do not infer that a talking avatar is automatically more accessible for everyone.
Release a stable asset set with rollback
Keep a release manifest linking the approved script, voice settings, audio file, caption file, viseme sequence, illustrated face rig, and effective museum notice revision. Review the rendered sequence after export in its actual playback context. A staging preview on a developer laptop may behave differently from the deployed kiosk due to video frame rate, network buffering, or audio autoplay restrictions. The release reviewer should verify that the route warning is audible and readable in the final build.
If source guidance changes, withdraw the affected clip until the approved script, regenerated audio, and corresponding face events have been reviewed as a new bundle. Preserve the previous approved version for controlled rollback only while its factual guidance remains valid. The outcome is a documented, optional animation layered on top of accurate and accessible content, not a substitute for staff-confirmed directions or live visitor assistance.
How We Research
For this November 2026 editorial review, we consulted official vendor materials: Microsoft Learn: Get facial position with viseme; Microsoft Learn: Text to speech overview. We checked the named feature, input/output scope, documented limitations, and the points where software results must be reviewed against an authoritative record. The project is a hypothetical instructional design; the graphics, source cards, time marks, sample identifiers, and decision states are original explanatory illustrations, not captured screenshots or measured service results. We did not upload real archival data, conduct a production API evaluation, obtain permissions on behalf of an institution, or measure an accuracy or latency result. Before deployment, verify current feature availability, regions, quotas, prices, data protection terms, licensing, and the latest developer documentation. A qualified human must review permissions, factual accuracy, accessibility, and any publication or high-impact changes.
Frequently Asked Questions
No. Several phonemes share the same visible mouth pose. Viseme events are animation cues, not a verified transcript.
No. Microsoft documents SVG support specifically for en-US; check current voice and locale support for other event formats.
Microsoft documents offsets in ticks of 100 nanoseconds for viseme events. Convert deliberately to match the playback engine.
Do not assume so. Regenerate and review audio, event timing, and captions as one versioned asset set.
No. It is an optional presentation layer; captions, readable text, navigation, and staff help remain necessary.