Understanding grammar caching

System administrators of runtime host systems can tune performance by manipulating Recognizer’s caching behavior.

Because applications can specify a source grammar multiple times, Recognizer compiles grammars before putting them into the disk and memory caches so they can be reused without recompilation. Administrators can change default behaviors in several ways, for example:

  • Reconfigure cache parameters. For example, changing the cache size.
  • Configure grammar properties. For example, locking a grammar into the cache.

Grammar operations such as fetching, compiling, and loading can consume significant resources. To minimize the costs, Recognizer has several caches. The following diagram shows a high-level view of the caching mechanisms:

  • Inet cache—The VXIinet implementation fetches grammars from application or grammar servers. The inet cache stores source grammars after they are fetched. The HTTP protocol determines whether this cache is used, and how long a grammar is stored. The VoiceXML application controls the inet cache with standard VoiceXML properties, and the browser translates the properties through standard MRCP headers. See Controlling grammar caches.
  • Memory cache—This cache stores in-memory form grammars after converting them from compiled binary form. A grammar in the memory cache requires no load time, and it is immediately available for recognition. The memory cache is especially important for large grammars (more than 5000 words) which might take a comparatively long time to load in binary form from the disk cache.
  • Disk cache—The disk cache stores compiled binary grammars to avoid redundant fetching and compilation. The disk cache normally serves as a backup for the memory cache; it is checked only when a grammar is not found in the memory cache. The disk cache is particularly valuable because the cost of remote fetching and compilation of a grammar is usually higher than loading a precompiled binary from disk. (As the number of words in a grammar increases, the cost of remote fetching and compilation increases more quickly than the cost of loading it from a local disk.)

Caches are independent of sessions

Recognizer shares grammars among sessions whenever possible:

  • If a session loads a grammar that was already loaded by another session, Recognizer shares the loaded instance of the grammar instead of reloading it unnecessarily.
  • If a session frees a grammar that is still loaded by another session, Recognizer does not remove the grammar from cache unnecessarily. (There are instances when Recognizer must remove a grammar from cache even though it is still loaded by a session. These situations are included in the discussions below.)
  • Once locked into cache, a grammar remains locked until all sessions using the grammar have freed it.

How Recognizer chooses between cached or reloaded grammars

When the application loads a grammar, Recognizer first determines whether the load operation is necessary. If the grammar is not yet loaded, Recognizer loads it. If the grammar is already loaded, Recognizer might reload it or do nothing, a decision that depends on the following conditions:

  • If the grammar is a local file, has it changed since the previous load?
  • If the grammar was fetched from a webserver via HTTP, has the expiration time been reached? Is the validation information provided by the webserver up-to-date? (If the webserver provides no validation information, Recognizer assumes the grammar is not valid for subsequent loads, and reloads it. For example, the webserver may provide “Expires”, but no “E-Tag” or “Last-Modified” information.)
  • Is a reload required by a configuration parameter or a file property setting?
  • Has the previously loaded grammar been flushed from the caches? This happens if the grammar is purged as part of a cache cleanup operation.

The following diagram shows the progression of a grammar from external source form to a fully loaded form using Recognizer’s caches:

  1. The application requests a grammar.
  2. If the grammar is already in the memory cache, Recognizer validates the age and other grammar properties and continues.
  3. If the grammar is not in the memory cache, but is in disk cache, Recognizer validates and loads the grammar into memory.
  4. If the grammar is not in the disk cache, Recognizer gets it from the inet cache, compiles if necessary, and loads into the disk and memory caches.
  5. If the grammar is not in the inet cache, Recognizer fetches, stores, compiles, and copies to the caches.