Tuning TTS output with ActivePrompts

The first step for using Vocalizer for prompt concatenation is to define the set of recordings, then enter them into Nuance Vocalizer Studio to create an ActivePrompt database. Each prompt needs the following:

• Logical prompt name, which the run-time engine transforms to a recording file name by appending a recording file suffix (such as .wav) and then using it as a URI relative to the ActivePrompt database. For example, if the ActivePrompt database http://myserver/apdb_rp_tom_alphanum.dat contains a prompt named alphanum/f.alpha0 and the database specifies a file suffix of .wav, the recording file must be http://myserver/alphanum/f.alpha0.wav.
• Input text matched by the prompt. Vocalizer does its matching using a normalized form of the input text (converts it to lowercase, normalizes spaces, expands abbreviations, and so on) so there is flexibility for differences between the run-time and prompt text. It is best to think of this as word-by-word matching after expanding dynamic types like dates, times, and numbers to their word sequence (such as 110 to “one hundred ten”).
• Boundary conditions, one for each side of the input text. This can be one of: sentence boundary, phrase boundary, sentence or phrase boundary, a specific punctuation symbol, phrase internal, or a wildcard (anything).

Some Vocalizer voices include predefined ActivePrompt databases and recordings for a variety of dynamic types, along with recording scripts. When possible, it is best to rely on those ActivePrompt databases (optionally re-recording them with the application voice talent) rather than re-creating those databases from scratch.

For other languages or voices, carefully consider the set of recordings required to speak the application's static and dynamic content. For the static content, consider each of the carrier phrases, which are typically listed in user interface documents and straightforward to define. Dynamic content is a bit more challenging: it requires knowing the language-specific output word sequences, then determining what variations to record for better sounding output.

For example, a basic recording set for digits playback could be one recording for each of the numbers 0 through 9, using wildcard boundary conditions. While that would produce understandable output, it would not sound natural. Much better output could be obtained by recording three variations of each number 0 through 9: one for phrase initial contexts (left boundary condition of sentence or phrase), one for phrase-medial contexts (wildcard boundary conditions), and one for phrase-final contexts (right boundary condition of sentence or phrase). Even better output could be obtained by recording digit pairs in those three contexts, so that a digit sequence like “0 2 3 7” after a carrier phrase would be played with one phrase-internal recording for “0 2”, then one phrase final recording for “3 7”.

Of course, this involves cost-versus-benefit decisions and may require experimentation to determine the lowest cost solution with a target quality level. The Nuance predefined ActivePrompt databases use both of these techniques and come with recording scripts, so even for new languages and types they provide a good reference point for making these decisions.

For flexibility, it is best to create a separate ActivePrompt database for each dynamic type, so applications can selectively enable them, such as one for alphanumeric sequences and another for dates, otherwise they may conflict with each other. They can use the same prompt names and recordings, which is desirable to improve run-time Internet fetch cache performance.

For additional flexibility, Vocalizer’s ActivePrompt run-time engine supports fallback if a recording is missing. This fallback builds a sophisticated ActivePrompt database that has features like multiple prompt variations, but later only recording a more basic prompt set. Vocalizer is of course also a text-to-speech engine, so if it fails to find any match it will automatically fall back to using text-to-speech output (except for recording only custom voices as described below).

The list of ActivePrompts used at run-time is available within the Vocalizer call log, a log that reports information for application tuning and capacity planning purposes. This is often helpful during ActivePrompt development and testing.

Once the ActivePrompt databases are defined, the next step is obtaining a set of recordings. When using a Nuance predefined ActivePrompt database, this can mean using the Nuance recordings as-is or re-recording them. Custom ActivePrompt databases will always need to be recorded.

As described above, Vocalizer supports fallback if a recording is missing, allowing a smaller set of base recordings to be made for a specific deployment rather than always having to record the full set of recordings specified by the ActivePrompt database.

Vocalizer supports inserting headerless, WAV format, AU format, or NIST SPHERE format audio files that contain mulaw, alaw, or linear 16-bit PCM samples, and the recording’s sampling rate must match the current Vocalizer voice.

Vocalizer makes it easy to define custom voices for recording-only playback. This is done by simply choosing or creating a set of ActivePrompt databases, choosing or creating a set of recordings, choosing a custom voice name, and running a Vocalizer tool to add the new voice. These voices are referred to as “CPR-only” (concatenated prompt recording only) voices, and can be used with alternative Vocalizer CPR-only licenses instead of requiring full text-to-speech licenses.

The new voice must be based on an existing installed Vocalizer voice, such as defining a custom Maureen voice based on the Nuance Tom voice. The custom voice re-uses the Vocalizer data of the template voice for input text processing.

Custom recording-only voices do not support fallback to text-to-speech. Instead, if any portion of the input text cannot be satisfied by the current recordings, Vocalizer logs an error and the current speak request fails. The logged error specifies the portions of the input text that cannot be satisfied using the normalized input text: it is quite readable, but may not exactly match the original input text, and it is romanized for Asian languages. If text-to-speech fallback is required, use a Nuance provided voice and just provide ActivePrompt databases and recordings for the application voice talent.

Applications use ActivePrompts by loading them into the system (see Load ActivePrompt databases) and then referencing the ActivePrompts. Those references can be explicit references using the ActivePrompt names (VoiceXML <prompt> or the native <ESC>\prompt\ control sequence), or they can be implicit references where the Vocalizer engine automatically searches the ActivePrompt database for each synthesis request, substituting ActivePrompts whenever the normalized input text matches an ActivePrompt’s normalized input text and boundary constraints.

Implicit ActivePrompt references can be further controlled by configuring each ActivePrompt database for either fully automatic mode (ActivePrompt database is always consulted) or normal mode (ActivePrompt database is consulted only when explicitly enabled by SSML ssft-domaintype or the native <ESC>\domain\ control sequence).

For implicit ActivePrompt matching when using Vocalizer 5.5 or later voice packs, Vocalizer can optionally match against both the original input text (original orthography) and normalized input text instead of the default of matching just against the normalized input text. This option can be useful for migrating applications across major Vocalizer language updates, because those updates may change the normalization rules, resulting in failed matches for older ActivePrompt databases that rely on the affected rules.

However, it is better to just recompile and retest ActivePrompt databases before deploying language updates rather than relying on matching against the original input text. Original input text matches are less powerful for information like numbers, dates, and times that involve heavy text normalization, and that matching adds additional CPU and ActivePrompt database storage overhead. Matching against the original input text is controlled by the Nuance Vocalizer Studio settings when creating ActivePrompt databases.

For dynamic content like alphanumeric sequences, dates, times, cardinal numbers, and telephone numbers, it is best to use implicit ActivePrompt references using the normal mode. This mode selectively enables the proper database for each dynamic data type, avoiding conflicts. The input text then becomes a SSML ssft-domaintype or native <ESC>\domain\ control sequence to enable the desired types, then the carrier phrase and text to speak. For example, the following SSML example speaks a pre-recorded carrier phrase (part_code_intro.wav) with the dynamic portion wrapped within a <say-as> element that explicitly specifies the spell:alphanumeric type:

<s ssft-domaintype="spell:alphanumeric">

<audio src="part_code_intro.wav">The part code is</audio>

<say-as interpret-as="spell:alphanumeric">8jihpey3wy</say-as></s>

Note: Some XML-based application development environments block the use of Nuance SSML extensions like ssft-domaintype. For those environments, set the escape_sequence parameter in the Vocalizer configuration file so you can use a sequence like "\!" instead of the <ESC> character (the <ESC> character is not allowed in XML documents), then use the native <ESC>\domain\ control sequence such as "\!\domain=spell:alphanumeric\". See Defining an alternative escape sequence.

This XML can of course be generated in many alternative ways. For example, in a VoiceXML environment, the VoiceXML expr attribute can be used to specify an ECMAScript variable that contains the dynamic portion, and an ActivePrompt can be used for the carrier phrase:

<s ssft-domaintype="spell:alphanumeric">The part code is

<say-as interpret-as="spell:alphanumeric"><value expr="partcode"/>

</say-as></s>

For all dynamic content, it is important to make sure the input format is compatible with Vocalizer. For common data types that are supported by Vocalizer, this is best done by wrapping the dynamic portion within the matching Vocalizer SSML <say-as> or the native <ESC>\tn\ control sequence, checking the Vocalizer Language Supplement to ensure the input format is compatible.

<?xml version="1.0" encoding="ISO-8859-1"?>  <!-- XML version -->
<?xml-stylesheet type="text/xsl" href="ttsconfig.xsl"?>
<ttsconfig version="5.7.0" xmlns="http://www.nuance.com/nvn57/ttsconfig">
  <activeprompt_dbs> 
    <activeprompt_db content-type="application/x-vocalizer-activeprompt-db;mode=automatic">
        file:///C:/lex/apdb_tp_serena_bet5f22_unknown.dat
    </activeprompt_db> 
  </activeprompt_dbs>
</ttsconfig> 

<?xml version="1.0" encoding="ISO-8859-1"?>  <!-- XML version -->
<?xml-stylesheet type="text/xsl" href="ttsconfig.xsl"?>
<ttsconfig version="5.7.0" xmlns="http://www.nuance.com/nvn57/ttsconfig">
  <activeprompt_dbs> 
    <activeprompt_db content-type="application/x-vocalizer-activeprompt-db;mode=automatic">
        C:/lex/apdb2_tp_serena_bet5f22_unknown.dat
    </activeprompt_db> 
  </activeprompt_dbs>
</ttsconfig>