Compiling grammars

At runtime, applications can load grammars that are in either uncompiled (.grxml) or precompiled (.gram) formats. Recognizer determines whether a grammar is precompiled, and acts accordingly by loading the grammar directly if it’s precompiled, or compiling and loading the grammar if it isn’t.

Each format offers different advantages and disadvantages:

Uncompiled grammars can be changed dynamically in the moments before loading (see Dynamic-link grammars). However, the additional runtime processing can be disadvantageous: large grammars can overload an already busy runtime CPU, and slow application response when the grammar is first activated—resulting in latency (slow recognition).
Precompiled grammars load faster. However, a precompiled grammar is static and cannot be changed in the moments before loading, which makes it unsuitable in situations where a dynamic grammar is required.

When a grammar is compiled, it uses dictionaries to look up word pronunciations. This means that the appropriate system and user dictionaries must be set before compilation.

While it is not necessary to precompile before runtime loading, you must perform test compilations during grammar development to reveal errors.

The compiler utility

The compiler utility is called sgc, and is installed in the following location:

%SWISRSDK%\bin\sgc.exe

Where %SWISRSDK% is the path to the directory where Recognizer was installed.

The sgc utility is invoked by Recognizer whenever it’s necessary to compile a GrXML file into a usable grammar. You can also use sgc in a command line to precompile a grammar into a binary (.gram) grammar file.

The command line for the compiler is simple:

sgc [option] mygrammar.grxml

Where option represents one or more compilation options for the sgc utility, and mygrammar.grxml is the pathname to the grammar file you want to compile. The output is a binary file with the same name as the original GrXML file, but ending in a .gram extension (in this example, mygrammar.gram).

You can compile more than one grammar at a time by putting several file paths in a space-separated list:

sgc [option] InputFile1 InputFile2

Various compilation options are available for the compiler tool: for example, there are options for optimization and various statistical models. These options are described fully in the description of the sgc utility. The options that you are most likely to use for a simple compilation are:

Option	Description
-batch filename	Specifies a batch file with instructions to compile several files.
-baseline path	Specifies the starting location for all relative file path references.
-language lang	Specifies the language to use in the compilation.
-optimize n	Sets the optimization level for the compilation (see Grammar optimization).
-out filename	Specifies the filename for the compiled output.
-no_logo	Suppresses the version info when using a script.
-no_script_verify	Does not check ECMAScript when compiling the grammar.

Compilation time and resources, and compiler output

The sgc tool displays status messages as it compiles. Here is sample output:

sgc: Compiling grammar 'mygrammar.grxml'.

sgc: Start compile.

sgc: Compile time was 1361.96 msec

sgc: ECMA scripts passed verification.

sgc: Dumping compiled grammar to 'mygrammar.gram'.

The compile time includes overhead from loading dictionaries and text-to-pronunciation rules (which are used to generate pronunciations automatically for words not found in the dictionaries). You can isolate the compile time from the overhead by double-compiling the grammar: if you use a comma to submit the same grammar twice, the first compilation will include overhead while the second will not. For example:

sgc a.grxml a.grxml

sgc: Compiling grammar 'a.grxml'.

sgc: Start compile.

sgc: Compile time was 2463.06 msec

sgc: ECMA scripts passed verification.

sgc: Dumping compiled grammar to 'a.gram'.

sgc: Compiling grammar 'a.grxml'.

sgc: Start compile.

sgc: Compile time was 1562.88 msec

sgc: ECMA scripts passed verification.

sgc: Dumping compiled grammar to 'a.gram'.

Diagnostic logging during compilation

When using sgc, you can turn on diagnostic logging tags to write information about the grammars being compiled:

The diagnostic log uses a tagmap file (defaultTagmap.txt) to enable various logging features. In particular, the SWIdictPronWarningTag tag can be useful; it sends a warning message to the diagnostic log when a pronunciation is not found in a dictionary and Recognizer has automatically generated the pronunciation instead.
To learn about logging tags, read the contents of Recognizer’s tagmap file.
The diagnostic log is primarily a tool for voice platform integrators. While the PIM describes the tagmap file and the creation of the log, it does not dictate how to provide access to the log for application developers. For access information, consult the documentation provided with the voice platform.

Compilation memory requirements

The amount of memory required to compile a grammar is mostly determined by the number of words it contains. However, the complexity of the grammar is also a significant factor.

For example, a grammar of 40,000 words requires about 3 KB per word to compile and a total of approximately 120 MB of memory. This is not the size of the compiled grammar; just the amount of memory required during the compilation process.

Compilation fails when an extremely large grammar exceeds the available memory. Here are ways to avoid the failure:

Try different optimization levels. For example, optimize 12.
Split the large grammar into two or more smaller grammars. Then compile and reference the grammars separately.
On Windows 2003, boot the system with a "/3GB" setting in the boot.ini file. This allows processes to use 50% more memory.
Download and install the 64-bit version of Recognizer for the Linux operating system. On Linux, you extend memory usage up to 4 GB.

Resource usage

Quick and accurate recognition is the primary objective of any grammar. However, although CPU usage and storage space are of secondary or even tertiary importance, they can still be a concern.

The factors that determine a grammar’s resource usage are discussed in Grammar performance. If you experience problems with CPU usage and/or storage space, refer to this section for advice on resolving them.

Grammar optimization

The optional -optimize option balances the CPU cycles required for compilation with the cycles needed for recognition:

> sgc -optimize 9 direct.grxml

Above, the integer values for optimization are 0 to 12 (but not 10). The value of 10 is reserved for internal use, so you must not use it.

Here is a summary of optimization levels. Use it as a general guideline for the trade-off between compilation time and runtime recognition time. In situations where the trade-off is important, verify the costs for specific grammars:

Level	General behavior
0-3	Compilation is faster. Runtime recognition is slower.
4-6	Compilation is slower. Runtime recognition is faster. Level 6 is used automatically for dynamic compilations.
7-9	Compilation is slower. Runtime recognition is possibly faster. If another optimization level is not specified in the command line, level 9 is used by default.
10	Reserved. Do no use.
11	Compilation is faster. Runtime recognition is slower.
12	For very large grammars, and especially SLMs.

The value 9 is set as the default on the assumption that fast runtime recognition will be the highest priority for most developers, so a long compilation time is not a problem. If you prefer another default, see swirec_optimization for guidance on setting optimization.