Training gRPC API

The Training gRPC API contains methods for compiling wordsets.

The Training service is available to Mix users who create resources using https://mix.nuance.com  and access them using the asr.api.nuance.com ASR endpoint. It is not available in other geographies.

Proto file structure

ASRaaS provides protocol buffer (.proto) files to define Nuance’s gRPC wordset training service. These files allow you to compile and manage large wordsets for use with your ASRaaS applications. See gRPC setup to download these files.

  • training.proto contains the methods and messages for working with large wordsets.
  • The RPC files contain status and error messages.

You may use these proto files in conjunction with the other ASRaaS proto files described in the Recognizer API.

└── nuance
    ├── asr
    │   ├── v1
    │   │   ├── recognizer.proto
    │   │   ├── resource.proto
    │   │   └── result.proto
    │   └── v1beta1
    │       └── training.proto
    └── rpc (RPC message files)

The proto files define a Training service with several RPC methods for creating and managing compiled wordsets. Details about each component are referenced by name within the proto file.

  Fields in wordset training request  

For the RPC fields, see RPC gRPC messages.

Job status vs. request status

This API includes two types of status:

  • Job status: The condition of the job that is compiling the wordset. Its values are set in JobStatus and can be JOB_STATUS_QUEUED, JOB_STATUS_PROCESSING, JOB_STATUS_COMPLETE, or JOB_STATUS_FAILED.

  • Request status:The condition of the gRPC request. Its values are set in nuance.rpc.Status and can be OK, INVALID_REQUEST, NOT_FOUND, ALREADY_EXISTS, and so on.

This report on an ongoing job combines job status and request status:

2021-04-05 16:41:28,369 INFO : Received response: job_status_update {
  job_id: "c21b0be0-964e-11eb-9e4a-5fb8e278d1ad"
  status: JOB_STATUS_PROCESSING
}
request_status {
  status_code: OK
  http_trans_code: 200
}

2021-04-05 16:41:28,896 INFO : new server stream count 2
2021-04-05 16:41:28,896 INFO : Received response: job_status_update {
  job_id: "c21b0be0-964e-11eb-9e4a-5fb8e278d1ad"
  status: JOB_STATUS_COMPLETE
}
request_status {
  status_code: OK
  http_trans_code: 200
}

Training service

The Training service offers five RPC methods to compile and manage wordsets. Each method consists of a request and a response message.

caption
Method Request and response Description
CompileWordset AndWatch CompileWordsetRequest
WatchJobStatusResponse stream
Submit and watch for job completion (server streaming).
GetWordset Metadata GetWordsetMetadataRequest
GetWordsetMetadataResponse
Get a compiled wordset’s metadata (unary).
DeleteWordset DeleteWordsetRequest
DeleteWordsetResponse
Delete the compiled wordset (unary).

For examples of using all these methods, see Sample training client.

CompileWordsetAndWatch

This RPC method submits a request to compile a wordset and returns streaming messages from the server until the job completes. It consists of CompileWordsetRequest and WatchJobStatusResponse. The response is a server stream of job progress notifications, which stays alive until the end of the compilation job, followed by a final job status.

CompileWordsetAndWatch consists of request and response

The CompileWordsetAndWatch method consists of:

CompileWordsetRequest
  wordset
  companion_artifact_reference (ResourceReference)
  target_artifact_reference (ResourceReference)
  metadata
  client_data
WatchJobStatusResponse
  job_status_update (JobStatusUpdate)
  request_status (nuance.rpc.Status)

  CompileWordsetAndWatch process flow  
  Python example for CompileWordsetAndWatch  
  Input file  

CompileWordsetRequest

Request to compile a wordset. The request message has a maximum size of 4 MB. See Sample training client for examples.

caption
Field Type Description
wordset string Mandatory. Inline wordset JSON resource.
companion_artifact_reference ResourceReference Mandatory. URN reference to the domain LM that the wordset extends, to use during compilation.
target_artifact_reference ResourceReference Mandatory. URN reference for the compiled wordset being generated.
metadata map<string,string> Client-supplied key,value pairs to associate with the wordset being compiled.
client_data map<string,string> Client-supplied key,value pairs to inject into the logs.

ResourceReference

Definition of an external resource: either a domain LM containing an entity that the wordset extends, or a compiled wordset. See also Wordsets.

caption
Field Type Description
uri string Mandatory. Location of the resource as a URN reference. See below.
headers map<string,string> Optional field for internal use.

The format of the URN reference depends on the message and resource. In these examples, the language is eng-USA.

  • DLM in companion_artifact_reference. The DLM must exist in the Mix environment under the specified context tag.
    urn:nuance-mix:tag:model/context_tag/mix.asr?=language=eng-USA

  • Wordset in target_artifact_reference, with an application-level wordset. The context tag is either an existing tag used for DLMs or wordsets, or a new tag that will be created. The wordset name is a new name for the compiled wordset being created:
    urn:nuance-mix:tag:wordset:lang/context_tag/wordset_name/eng-USA/mix.asr

  • Or a user wordset:
    urn:nuance-mix:tag:wordset:lang/context_tag/wordset_name/eng-USA/mix.asr?=user_id=<user_id>

WatchJobStatusResponse

Server stream information about a compilation job in progress.

Response to CompileWordsetRequest. This response is streamed from the server, giving information about the compilation job as it progresses.

caption
Field Type Description
job_status_update JobStatusUpdate Immediate job status.
request_status nuance.rpc.Status Nuance RPC status of the request.

WatchJobStatusResponse returns these notifications:

  • Job ID.

  • Multiple responses with JOB_STATUS_QUEUED or ~PROCESSING. The same status may be returned multiple times. Repeated notifications also keep the process alive.

  • Final job status (JOB_STATUS_COMPLETE or ~FAILED) with error messages when appropriate.

  • Final request status, either OK for successful requests, or INVALID_REQUEST and similar for others. This status refers to the request itself, not the wordset compile job that was created. See Job status vs. request status.

Invalid requests do not create a job, so the notification stream consists only of the request status.

JobStatusUpdate

Job status update to a request.

caption
Field Type Description
job_id string The job ID, a unique identifier.
status JobStatus Job status.
messages JobMessage Repeated. Error details on job.

JobStatus

Job status as a keyword. See Job status vs. request status.

caption
Name Number Description
JOB_STATUS_UNKNOWN 0 Job status not specified or unknown.
JOB_STATUS_QUEUED 1 Job is queued.
JOB_STATUS_PROCESSING 2 Job is processing.
JOB_STATUS_COMPLETE 3 Job is complete.
JOB_STATUS_FAILED 4 Job has failed.

JobMessage

Message about the job in progress.

caption
Field Type Description
code int32 Message code.
message string Job message.
data map<string, string> Additional key,value pairs.

GetWordsetMetadata

This RPC method requests and returns information about a compiled wordset. This method consists of GetWordsetMetadataRequest and ~Response.

Training messages

The GetWordsetMetadata method consists of:

GetWordsetMetadataRequest
  artifact_reference (ResourceReference)
GetWordsetMetadataResponse
  metadata
  request_status (nuance.rpc.Status)

  GetWordsetMetadata process flow  
  Python example for GetWordsetMetadata  
  Input file  

GetWordsetMetadataRequest

Request for information about a compiled wordset.

caption
Field Type Description
artifact_reference ResourceReference Mandatory. Reference to the compiled wordset artifact.

GetWordsetMetadataResponse

Response information about a compiled wordset.

caption
Field Type Description
metadata map<string,string> Default and client-supplied key,value pairs.
request_status nuance.rpc.Status Nuance RPC status of fetching the metadata.

GetWordsetMetadataResponse does not return the JSON content of the wordset. It provides two types of metadata:

  • Custom metadata, optionally supplied by the client as metadata in CompileWordsetRequest.

  • Default metadata (reserved keys):

    x_nuance_companion_checksum_sha256: The companion DLM, SHA256 hash in hex format.

    x_nuance_wordset_content_checksum_sha256: The source wordset content, SHA256 hash in hex format.

    x_nuance_compiled_wordset_checksum_sha256: The compiled wordset, SHA256 hash in hex format.

    x_nuance_compiled_wordset_last_update: Date and time of last update as ISO 8601 UTC date.

DeleteWordset

This RPC method removes a wordset from storage. It consists of DeleteWordsetRequest and ~Response.

Training messages

The DeleteWordset method consists of:

DeleteWordsetRequest
  artifact_reference (ResourceReference)
DeleteWordsetResponse
  job_status_update
  job_status

  DeleteWordset process flow  
  Python example for DeleteWordset  
  Input file  

DeleteWordsetRequest

Request to delete a compiled wordset.

caption
Field Type Description
artifact_reference ResourceReference Mandatory. Reference to the compiled wordset artifact.

DeleteWordsetResponse

Response to the delete request.

caption
Field Type Description
request_status nuance.rpc.Status Nuance RPC status of deleting the wordset.

Scalar value types

The data types in the proto files are mapped to equivalent types in the generated client stub files.

Scalar data types
Proto Notes C++ Java Python
double double double float
float float float float
int32 Uses variable-length encoding. Inefficient for encoding negative numbers. If your field is likely to have negative values, use sint32 instead. int32 int int
int64 Uses variable-length encoding. Inefficient for encoding negative numbers. If your field is likely to have negative values, use sint64 instead. int64 long int/long
uint32 Uses variable-length encoding. uint32 int int/long
uint64 Uses variable-length encoding. uint64 long int/long
sint32 Uses variable-length encoding. Signed int value. These encode negative numbers more efficiently than regular int32s. int32 int int
sint64 Uses variable-length encoding. Signed int value. These encode negative numbers more efficiently than regular int64s. int64 long int/long
fixed32 Always four bytes. More efficient than uint32 if values are often greater than 2^28. uint32 int int
fixed64 Always eight bytes. More efficient than uint64 if values are often greater than 2^56. uint64 long int/long
sfixed32 Always four bytes. int32 int int
sfixed64 Always eight bytes. int64 long int/long
bool bool boolean boolean
string A string must always contain UTF-8 encoded or 7-bit ASCII text. string String str/unicode
bytes May contain any arbitrary sequence of bytes. string ByteString str