Test the grammars

It is crucial to test your grammars. A grammar document is code: an untested grammar is likely to have bugs, just like any other piece of untested software.

The testing process can reveal syntax errors, uncover omissions in the grammar, alert you to latency problems (long pauses in recognition), and evaluate resource usage. Include these tests in your grammar test plan:

• Coverage test: This test verifies that your grammar is able to parse a set of test phrases. You can maintain a set of phrases—and interpretations—that you will require your grammar to parse after any change is introduced or any compilation property is modified.
• Interpretation test: This test checks that slots are filled accurately and that all grammars return the correct values—that is, that your grammar delivers the right interpretations for all phrases in the test set.
• Over-generation test: This test checks for nonsense phrases that the grammar should not parse. This test verifies that your grammar will not accept unwanted sentences. It also bolsters recognition speed and accuracy, helping you to reduce your grammar size by removing unnecessary coverage.
• Ambiguity test: This test identifies phrases parsed by your grammar that may have multiple interpretations.
• Pronunciation test: This test is used to detect words with unknown pronunciations. It also exposes misspellings in your grammars.

The Recognizer package includes testing utilities for coded grammars:

• The parseTool program tests a grammar interactively, with sub-tools that let you conduct tests from your keyboard, trace ECMAScript operations, and check your grammar for overgeneration and nonsense phrases.
• The test_parser tool allows you to perform interpretation tests on one or several grammars by comparing the correct key/value pairs that are passed to Recognizer with those actually generated.
• The dicttest program is a command line tool that lets you check pronunciations of individual words or lists of words inside a file.
• The acc_test tool tests recognition accuracy of grammars according to the instructions contained in a script, which must be prepared ahead of time in a proprietary format.

Once your grammar is deployed, you can also gather actual call data for statistical purposes. You can use these actual results to refine the grammar further by using natural language techniques.