Generating call analysis reports

The Call Volume report charts the number of calls and average call duration over time for a particular customer and application (if specified). It also provides a details table showing a breakdown of the information.

1. Click Call Volume Report in the left frame. A chart appears summarizing the total number of calls for all call log files in the database.
2. Change the Group By: field and click Refresh, or scroll down and specify additional filters to select data that interests you.
3. Generate the report, either immediately or for a later time.
4. If you choose to generate the report immediately, the report appears in a new window, showing a Call Volume chart, an Average Call Duration chart, and a Details table with the busiest-hours rows highlighted. For example (printer-friendly version in PDF format):

Example

Let’s say you’re interested in seeing the amount of call traffic for each hour during the midnight to 6 a.m. shift. You want to generate a report every Monday morning that charts the data for the previous week. This scenario assumes that you have scheduled to import call log files as a recurring task every night at 2 a.m. (Scheduling the data import is beyond the scope of this example. See Scheduling and managing tasks for another scenario where this is shown.)

Customize the Call Volume report as follows:

1. For the date range, select Past and enter 7 for the number of days. The report will be generated on data starting from the current day and back.
   1. Specifying a “past” number of days ensures that the report is always generated on new data.
   2. Keep all days of the week checked.
   3. For the time interval, specify from 12 AM to 6 AM.
   4. Group by hour.
   5. Accept the remaining defaults.
2. Click Schedule Report. The Task Recurrence page appears:
   1. Since you want the report to show Monday through Sunday, select the next Sunday for the start date. Make sure to specify a time after the time you’ve scheduled for the daily data import, say 4 a.m.
   2. Select a weekly recurrence.
   3. Click Schedule. The Task Manager page appears showing your scheduled report. Once the scheduled start time expires, the report executes, and the next scheduled occurrence automatically appears.

      From now on, a weekly report will be generated every Sunday, reflecting data accumulated over the past seven days.

The Voice Site Hit report shows how often a dialog state was accessed over time for a particular company and application (if specified). This report is useful for analyzing web-site call traffic among applications.

Generating this report requires that you code grammar labels in your application. A grammar label is a unique identifier for the grammar associated with each dialog state. See your Nuance speech products documentation to learn about coding these labels.

1. Click Voice Site Hit Report in the left frame.
2. Specify the filters to select data that interests you. For example, change the Number of Dialog States to Show value, if desired.
3. Generate the report, either immediately or for a later time.
4. If you choose to generate the report immediately, the report appears in a new window, showing:
   • A bar chart with the number of hits for the number of dialog states specified.
   • A details table identifying the specific grammar label accessed and number of hits for each.

     For example, a report might look like this:

     

5. Click any grammar label. A new chart and details table appear showing a breakdown of the number of hits for that particular dialog state and when they occurred.
6. You can change the Group By field on this page, if desired. For example, the report grouped by day might look like this:

The Hotspot report shows the status of the recognition operation for selected dialog states in a VoiceXML application. For example, you can see the number of recognitions, rejections, aborts, and so on.

Generating this report requires that you provide grammar labels in your application code. A grammar label is a unique (and more meaningful) identifier for the grammar associated with each dialog state. The default is simply the grammar name, but you can add any number of grammar labels in your application.

1. Click Hotspot Report in the left frame.
2. Specify the filters to select data that interests you.
3. Click Next. A list of dialog states appears. The dialog-state name identifies the specific grammar label accessed. For example:

   

4. Select the dialog states that you want to analyze.
5. Generate the report, either immediately or for a later time.

   Note: Nuance recommends scheduling the report for off-peak hours when system use is low. Generating these reports on demand isn’t advisable because they can be performance-intensive and place demands on the system.

   1. If you choose to generate the report immediately, the report appears in a new window, showing a Hotspot activity chart (percentage of recognitions and rejections for each dialog state), followed by a Details table of information for each dialog state. For example:

      

      The possible values in the Details table:

      • Recognition—A successful recognition.
      • Rejection—An unsuccessful recognition.
      • Too Much Speech—Recognition aborted due to a too much speech timeout. Not applicable.
      • Speech Too Early—The recognition aborted due to speech spoken too early. Not applicable.
      • Interpretation—Not applicable.
      • Hang up—The recognition aborted because the caller hung up.
      • Aborted—The recognition aborted.
      • Exception—The recognition aborted due to an exception.
      • Left overs—Some information was recorded after the call completed. Not applicable.
      • Recognizer Too Slow Timeout—The recognition aborted due to a slow response from the Recognizer. This includes when recognition is aborted because no speech was detected.
      • Enrollment—A successful voice-enrollment record. Not applicable.
      • Other—Value does not fall into any other category.

Note: This report is only relevant users of the Nuance Voice browser service, and only when your VoiceXML application uses the <nuance:taskbegin> and <nuance:taskend> tags to define the start and end of tasks. For details, see the Nuance Voice Platform for Speech Suite 11 documentation.

The Task Completion report shows the number of successful, failed, unknown, and incomplete task results for a particular task. For successes and failures, the report provides reasons and details why. To generate this report, tasks must be coded in your VoiceXML applications.

A task is a unit of work, for example, accessing a bank account, selling an item, obtaining a telephone listing. Each task has a beginning and an end, a name, reasons with results (success, failure, unknown), and where appropriate, details about specific reasons.

From an application perspective:

• Attempted tasks encompass everything—completed and incomplete tasks results
• Completed tasks are those that reached the end of the task and include successes, failures, and unknowns:
  • Successful task results are those reasons labeled success in the application
  • Failed task results are those reasons labeled failure in the application
  • Unknown task results are reasons that don’t fall into either category
• Incomplete task results are those that never reached the end of the task

For example, say the task is to access your bank account. The application asks for your name and PIN. If you provide the correct information, you’ll get into your account (successful task result, reason is “valid PIN”). If you give the wrong PIN, the application denies you access (failed task result, reason is “invalid PIN”, detail is “Maximum attempts exceeded”). If a system error occurs, for example, the system dropped the call, then the application never reaches the end of the task. This is identified as an incomplete task result.

This report also charts the ratio of successfully completed task results to total task attempts over time. These results are reasons that you have decided count as successful. By default, the application determines these for you by displaying all reasons labeled success. However, you make the final decision and can choose other reasons to filter on when generating the report.

Finally, the report calculates a return on investment (ROI). By entering an ROI multiplier, you can generate the total benefit to your company for successfully completed tasks over a given time period.

The following instructions use a sample directory-assistance application. The procedure is the same regardless of the application you’re running.

1. Click Task Completion Report in the left frame. The Task Completion Report page appears where you specify filters to select data that interests you:

   

   Where:

   • Date Range: Enter the number of days or specify a particular date and time interval. The value represents when the calls were recorded.

     The Today field provides a shortcut for specifying a block of time, for example, the past 30 or 90 days.

     The Past day(s) field lets you specify your own block of time, for example, 2 days or 14 days. The number of days includes the current day. For example, Past 1 day would be today.

     Note: It is possible that you may get less than one day’s worth of data in your report, depending on the what time it runs. To avoid this situation, schedule the report to run at 2 or 3 a.m. and select Past 2 day(s). You’ll get all of yesterday’s data, plus a little bit of the early morning.

     The From and To fields let you specify larger time intervals. Select the option, then click the calendar icon to select a date and time. Note that the Management Station detects the date of the first and last call sessions stored in the database and displays these as the default values.

   • Tenant(s): This filter appears if a tenant ID field is included in the call log file (meaning it was coded in the application). This filter lets you create a report based on a particular tenant. In a hosted environment, a tenant is a client of a service provider for whom the application has been customized

     Choose a particular tenant from the list. Choose (all) to include all tenants, otherwise, accept the (default) to not filter by tenant.

   • Task: Choose the task name from the list. These values were coded in the application.
   • Other Reasons: By default, this list shows those reasons labeled failure and unknown in the application for the chosen task. Choose the reasons you want counted as successfully completed in the Task Completion Ratio chart and add them to Success Reasons.
   • Success Reasons: By default, this list shows those reasons labeled success in the application for the chosen task. Any reason added here will be counted as successfully completed tasks in the Task Completion Ratio chart. Remove any reason you don’t want counted in the calculation.

     As you can see, these two fields, Other Reasons and Success Reasons, provide flexibility in what is counted as successfully completed. You can let the application decide (default) or you can decide by picking your own list of reasons.

   • Group By: Choose a time interval for displaying the task completion results along the x-axis of the chart.
   • Task Completion Percentage (optional): Specify a number indicating a desired success rate. This is an arbitrary value determined by you. The Management Station displays this value as a threshold on the chart to which you can visually compare the actual results.
   • Report Host(s): You can generate Task Completion reports on call log files imported from specific hosts. Choose the host(s) you want to report on. You can also filter on specific clusters or choose all to select all hosts and clusters. What you select appears in the report heading.
   • ROI Multiplier (optional): Specify a number to calculate the ROI. Enter the number as a string without the $ sign, for example, 1.50 or 3.49. This number multiplied by the total number of what you’ve chosen as successfully completed task reasons gives the ROI, graphed as the total dollar benefit over the specified time interval.

     For example, the accounting department in a call center has determined that each automated call (without the intervention of a live agent) saves the company $5.00 per call. You would enter 5.00 in this field. If 10,000 calls were successfully completed over a one-week period, then the ROI for the company is $5.00 x 10,000 for a total of $50,000.

2. Generate the report, either immediately or for a later time.
3. If you choose to generate the report immediately, the report appears in a new window, showing:
   • Task Summary pie chart potentially divided into four task-status regions:

     Success	Task reasons labeled success in the application
     Failure	Task reasons labeled failure in the application
     Unknown	Task reasons not falling into either category
     Incomplete	Failed to reach task end

   • Task Summary table showing statistics like total calls received, total task attempts, the number of successes, failures, unknowns, and incompletes. Note that total number of attempts could be greater than the total calls received. This happens when a caller attempts the same task in the same call.
   • Other tables showing a breakdown of the information, for example, the reasons and details for each task status.
   • ROI Tracking Chart: A line chart showing the total benefit in dollar amounts for successful tasks. This chart only appears if you specified an ROI Multiplier.
   • Task Completion Ratio: A line chart showing the ratio of successfully completed to attempted task results. The success rate threshold appears if you specified a Task Completion Percentage. Completed task results are those reasons you included in the Success Reasons field on the filters page. Attempted task results are all successes, failures, unknowns, and incompletes.
   • Details: A table showing a breakdown of the information in the ratio chart.
4. Hold the mouse cursor over a region in the pie chart, or place it on any of the points in the pie or line charts to see more information, for example, a total, a timestamp, or a ratio.

   Part of a sample Task Completion report might look like this:

   

The Task Duration report shows the average and maximum duration over a specified period of time for specific tasks. To generate this report, tasks must be coded in your VoiceXML applications. See Generating a Task Completion report.

1. Click Task Duration Report in the left frame.
2. Specify the filters to select data that interests you. You can select a single task, multiple tasks, or all tasks.
3. Generate the report, either immediately or for a later time.
4. If you choose to generate the report immediately, the report appears in a new window, showing for each selected task:
   • The total number of times that the task completed, that is, how often it reached the end of the task, and include successes, failures, and unknowns.

     A 0 value indicates an incomplete task, that is, at no time did it ever reach the end of the task.

   • The average amount of time, in seconds, it took for the task to complete, that is, reach task end. Any value less than 500 milliseconds is treated as 0; any value greater than 500 milliseconds is treated as 1. Incomplete tasks appear as NA.
   • The maximum amount of time, in seconds, it took for the task to complete. Any value less than 500 milliseconds is treated as 0; any value greater than 500 milliseconds is treated as 1. Incomplete tasks appear as NA.