Vocalizer SSML support

Vocalizer uses the Apache Xerces-C XML parser with the ICU (International Components for Unicode) library for supporting a wide variety of character encodings. Use the UTF-16 character set for optimal performance. The next best choice is UTF-8.

To learn about supported character sets, visit the IANA website at http://www.iana.org/assignments/character-sets.

SSML requires specifying the document language using xml:lang, an IETF language code.

By default, Vocalizer does strict validation of input SSML documents against compiled-in versions of the SSML schema documents with Vocalizer specific enhancements (VOCALIZER_SDK\doc\synthesis.xsd and synthesis-core.xsd within the Vocalizer installation). This is the most robust setting because it ensures the application and Vocalizer engine properly cooperate, rather than forcing Vocalizer to try and handle bad SSML input that may lead to strange TTS output. However, if required, use the ssml_validation parameter in the Vocalizer configuration file to adjust the validation mode.

When validation errors occur, Vocalizer logs errors or warnings to the normal error log destinations (a file, a system log, and/or an application callback depending on the configuration settings), then returns a TTS_E_SSML_PARSE_ERROR error. When this occurs, review the error logs for details about the SSML document errors.

When SSML <audio> is used to insert an audio recording, and the fetch fails or the audio format is not supported, Vocalizer proceeds with using the <audio> element’s content as fallback as described in the SSML 1.0 Recommendation. However, the default Vocalizer handling of certain error cases is different than what the SSML 1.0 Recommendation states.

If the <audio> fallback exists and can be spoken, the speak request succeeds, but Vocalizer logs a single warning message to indicate the audio URI that could not be spoken and note that its fallback was used. The only use of this error message (error ID 1422) is to report the case of a failed <audio> insertion that has successful fallback. You can disable the error message if you like (given that the speak request succeeds). (See Error and diagnostic logging.)

If the <audio> fallback does not exist or cannot be spoken (such as another <audio> element without fallback that could not be spoken), Vocalizer logs a series of errors that indicate the failure details, including the audio URI. Then if the ssml_validation parameter in the Vocalizer configuration parameter is set to "strict" (the default), the speak request fails with a TTS_E_SSML_PARSE_ERROR error code, with no audio delivered to the application. If the ssml_validation parameter is set to any other value, the speak request proceeds, merely omitting the <audio> element.

The ssml_validation "strict" setting gives the most robust behavior, making sure the listener doesn’t hear confusing or misleading audio for the case of a failed <audio> element with no or failed fallback. However, Vocalizer’s behavior in "strict" validation mode is stricter then the SSML 1.0 Recommendation: the specification requests that the TTS engine notifies the application of the error while proceeding with the speak request (Vocalizer’s behavior in "warn" and "none" validation mode). However, most Vocalizer environments require the speak request to completely fail in order to apply an appropriate fallback strategy (such as transferring a call to a human). Thus Vocalizer’s default behavior of "strict" makes the overall solution more robust.

If this is undesirable, change ssml_validation to "warn," or simply ensure all SSML requests contain text-to-speech fallback for all <audio> elements, even if it is simply a very small pause by using a <break strength="x-weak"/> element.

Vocalizer supports native control sequences within SSML documents. However, the default <ESC> character used to initiate a native control sequence, "\x1B", is not allowed in XML documents as per the W3C XML 1.0 specification. The escape_sequence Vocalizer configuration file parameter can be used to specify an alternate sequence that permits using native control sequences within SSML documents.

However, beware that mixing native control sequences with SSML markup can lead to unexpected behavior, because Vocalizer internally handles SSML by expanding it to native control sequences as well. The Vocalizer SSML parser is “blind” to any native control sequences within the SSML document, so if a native control sequence is used that conflicts with the SSML parser expansions, unexpected behavior may occur. For example, if <ESC>\voice\ is used to switch the voice, then SSML <voice> is used to change the voice again, at the end of the SSML <voice> element the voice is restored to the previous language as determined by the SSML parser state, not the voice selected by the native <ESC>\voice\ control sequence. For this reason, Nuance recommends the following:

• Give preference to SSML markup in SSML documents. Only use native control sequences when there is no SSML equivalent.
• If you need native control sequences in a document, use them throughout the document (by eliminating all SSML markup if possible).
• If you mix SSML markup and native control sequences in the same document, carefully test for possible conflicts or unexpected interactions.

The default value is 100, and the scale is amplitude linear. Although SSML specifies a range of 0–100, Vocalizer extends the range to 200 with the values above 100 reached via relative changes or the symbolic values "loud" and "x-loud".

The table below describes the mapping between the SSML volume scale and the Nuance native volume scale (where the volume value is an integer in the range 0 to 100 which can be set via the native <ESC>\vol=x\ markup).

The formula for converting the SSML volume value (Vssml) to the amplification factor A is very simple:

A = Vssml / 100 

The formula for converting a non-zero amplification factor A to the corresponding Vocalizer volume value Xvocalizer is:

Xvocalizer = Round((20 * log10(A) / 0.30) + 80) 

The formula for converting a non-zero Vocalizer value Xvocalizer to the dB value Y is:

Y (dB) = (Xvocalizer - 80) * 0.3 dB 

The SSML symbolic values are mapped as follows:

Vocalizer fully supports SSML rate markup. Use the following tables/rules to map SSML markup to equivalent Vocalizer native markup.

The default value is 1.00.

SSML descriptive and number values change the rate against the voice default. All other rate changes are relative against the (XML) parent element.

Here are the formulas used to convert an SSML <prosody rate=" Xssml"> value into a Vocalizer <ESC>\rate=Yvocalizer\ value. Do note that rate changes are relative to the parent element: you must keep track of all changes to your SSML value before converting. For example, to change from 50 to 25 is -50%; but to restore the original value from 25 to 50 is +100%.

For increasing the rate (Xssml > 1.0)

Yvocalizer = Round(100 * Xssml) 

When decreasing the rate (Xssml < 1.0)

Yvocalizer = Round(50 - (1 - Xssml) * 100) 

A <break> element is a pause of a certain duration. Attributes:

• time: Optional. Takes precedence over strength. Duration in seconds or milliseconds. For example, 3000ms or 3s.

• strength: Optional. Has one of the following symbolic values:

  Symbolic value for strength	Duration in ms of the pause	Corresponding native markup value
  x-weak	100	<ESC>\pause=100\
  weak	200	<ESC>\pause=200\
  medium (default)	400	<ESC>\pause=400\
  strong	700	<ESC>\pause=700\
  x-strong	1200	<ESC>\pause=1200\
  none	0	<ESC>\eos=0\

In the Nuance implementation, <break strength="xxx"/> maps directly to a Vocalizer <ESC>\pause=x\ tag with one exception: the SSML <break strength="none"/> element maps to a Vocalizer <ESC>\eos=0\ tag.

While the W3C SSML 1.0 Recommendation specifies interpret-as, format, and detail attributes, it does not define any specific say-as types. Vocalizer extends the recommendation by adding a significant number of built-in SSML say-as types. In addition, you can override those types (or add entirely new types) with the rulesets feature.

Vocalizer does this by mapping each say-as element to a native <ESC>\tn\ control sequence by building a string that consists of the interpret-as, format, and detail attributes with colon separators, using an empty string for unspecified attributes, and then stripping trailing colons off the end of the final string. Vocalizer then uses the following precedence for handling the say-as type:

• Look for a ruleset with a matching "type" attribute (which may include wildcards).
• Look for a matching Vocalizer front-end <ESC>\tn\ type.
• Trim off the "detail" attribute, then look for a matching Vocalizer front-end <ESC>\tn\ type. If found, it logs a warning, but then proceeds with that revised type.
• Trim off the "format" and "detail" attributes, then look for a matching Vocalizer front-end <ESC>\tn\ type. If found, it logs a warning, but then proceeds with that revised type.
• Log a warning, then handle the text as if it was plain text (as if the <say-as> wrapper was discarded).

For details on the supported SSML say-as and <ESC>\tn\ types, see each voice Language Supplement for an exact list of the types supported for your language and for language-specific implementation details that apply for each type.

Nuance supports loading user dictionaries, rulesets, and ActivePrompt databases through the SSML <lexicon> element.

The "type" attribute is optional. If specified this overrides the MIME content type returned by the web server (for http access) or extension mapping rules (for local file access). Valid values are:

• application/edct-bin-dictionary for a Vocalizer binary format user dictionary
• application/x-vocalizer-rettt+text for a ruleset
• application/x-vocalizer-activeprompt-db for an ActivePrompt database, optionally with ":mode=automatic" appended to override its default matching mode to automatic

When using a web server, remember to add entries to your web server’s MIME content type mapping table to associate the file extensions with the correct MIME content types (see Configuring MIME content types). If you do not, your system has no defaults and the lexicon load fails unless the "type" attribute is specified. (Other approaches for setting the MIME content type may also be possible, depending on the web server used.) For local file access, Vocalizer uses the inet_extension_rules configuration file parameter to map file extensions to the MIME content type.

Vocalizer parses all <lexicon> elements and loads tuning data before starting text to speech conversion. This tuning data is unloaded when the last sample buffer is generated, or when the TTS process is stopped, so <lexicon> elements only affect the current synthesis request.

As always, check the SSML specification for additional information.