Kafka consumer API
This section describes the Kafka consumer API.
Version: V1
This document covers v1 of the Mix Kafka REST Consumer API.
URL
https://log.api.nuance.com
Partitions
Get Partitions
Retrieve the list of available partitions for a topic.
Responses
200 response example
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partitions": [
0
]
}
Get Partition Offsets
GET /partitions/{partition}/offsets
Retrieve the first and last available offsets for a partition.
Parameters
Name |
In |
Type |
Description |
partition |
path |
number |
Partition number |
Responses
200 response example
{
"beginning_offset": 441,
"end_offset": 641
}
Consumers and Groups
Create Consumer
Create a new consumer instance in a consumer group. The consumer group name must follow the pattern appID-{appId/topic}-clientName-{clientName}-{two-digit number}
, for example, appID-DEMO-OMNICHANNEL-APP-DEV-clientName-default-01
The consumer name must follow the pattern consumer-{UUIDv4}
, for example, consumer-9872ec64-50d9-4dcd-9f4f-e8b5099a7f4f
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
body |
body |
createConsumers |
Object that provides the parameters used when creating a consumer |
Body parameter
{
"auto.offset.reset": "latest",
"consumer.request.timeout.ms": "1000",
"fetch.min.bytes": "-1",
"auto.commit.enable": "false"
}
Responses
Status |
Meaning |
Description |
Schema |
204 |
No Content |
Consumer instance created successfully |
None |
400 |
Bad Request |
Bad Request, topic does not exist |
errorResponse |
400 response example
{
"error_code": 401,
"message": "The request could not be authorized"
}
Delete Consumer
Delete the specified consumer instance from a consumer group.
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
Responses
Status |
Meaning |
Description |
Schema |
204 |
No Content |
Consumer instance deleted successfully |
None |
Subscriptions
Subscribe
POST /consumers/subscription
Subscribe a consumer to a topic to get dynamically assigned partitions. If a subscription already exists, it will be replaced.
Note:
Use either the Subscribe endpoint (to assign partitions automatically) or Assign Partitions endpoint (to manually assign partitions).
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
body |
body |
subscription |
Object that provides the topic to subscribe to |
Body parameter
{
"topics": [
"DEMO-OMNICHANNEL-APP-DEV"
]
}
Responses
Status |
Meaning |
Description |
Schema |
204 |
No Content |
Consumer successfully subscribed to the topic specified |
None |
Unsubscribe
DELETE/consumers/subscription
Remove the existing topic subscription for a consumer.
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
Responses
Status |
Meaning |
Description |
Schema |
204 |
No Content |
Consumer successfully unsubscribed from the topic specified |
None |
Get Subscription
GET /consumers/subscription
Retrieve the topic to which the consumer is currently subscribed.
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
Responses
Status |
Meaning |
Description |
Schema |
204 |
No Content |
Current subscribed topic |
subscription |
204 example response
{
"topics": [
"DEMO-OMNICHANNEL-APP-DEV"
]
}
Records
Get Records
Fetch data for the topic to which the specified consumer is subscribed. This request requires the accept: application/json
header.
Notes:
- An inactivity timeout is configured for customer instances, so they will be deleted if not active for a certain period.
Therefore, Nuance recommends that you call the Get Records endpoint at a regular interval to keep the consumer instance alive. See the documentation for more information.
- In some cases, this endpoint returns an empty array even if there are event logs available. For example, this can happen if an internal fetch timeout occurs. Nuance recommends that you try this endpoint again before determining that all event logs have been fetched.
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
Accept |
header |
string |
Required accept: application/json header |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
Responses
Status |
Meaning |
Description |
Schema |
200 |
OK |
Records fetched by the consumer instance, in JSON format. |
jsonConsumerRecords |
200 response example
[
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"key": {
"service": "DLGaaS",
"id": "3a4b511c-c06b-497f-ba0a-24c1f95a7e60"
},
"value": {},
"partition": 0,
"offset": 210
}
]
Assignments
Assign Partitions
POST /consumers/assignments
Manually assign a list of partitions to this consumer. This request requires both the content-type: application/json
header and the accept: application/json
.
Note:
Use either the Subscribe endpoint (to assign partitions automatically) or Assign Partitions endpoint (to manually assign partitions).
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
Accept |
header |
string |
Required accept: application/json header |
Content-Type |
header |
string |
Required content-type: application/json header |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
body |
body |
partitionsAssignment |
Object that provides the partitions to assign to a consumer |
Body parameter
{
"partitions": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0
}
]
}
Responses
Status |
Meaning |
Description |
Schema |
204 |
No Content |
Partition successfully assigned to the consumer |
None |
Get Assigned Partitions
GET /consumers/assignments
Retrieve the list of partitions currently manually assigned to this consumer. This request requires the accept: application/json
header.
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
Accept |
header |
string |
Required accept: application/json header |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
Responses
Status |
Meaning |
Description |
Schema |
200 |
OK |
List of partitions assigned to the specified consumer |
partitionsAssignment |
200 response example
{
"partitions": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0
}
]
}
Offsets
Get Committed Offsets
200 response example
{
"offsets": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0,
"offset": 412,
"metadata": "string"
}
]
}
POST /consumers/committed/offsets
Retrieve the last committed offsets for the partitions specified.
Body parameter
{
"partitions": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0
}
]
}
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
body |
body |
partitions |
Object that provides the partitions for which to retrieve offsets |
Responses
Status |
Meaning |
Description |
Schema |
200 |
OK |
Last committed offsets for the partitions specified |
committedOffsets |
Commit Offsets
Commit a list of offsets for the consumer. When the POST body is empty, this endpoint commits all the records that have been fetched by the consumer instance.
Body parameter
{
"offsets": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0,
"offset": 412
}
]
}
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
body |
body |
offsets |
Object that provides the offsets to commit |
Responses
Status |
Meaning |
Description |
Schema |
204 |
No Content |
Committed offsets for the partitions specified |
None |
Positions
Override Offsets
POST /consumers/positions
Overrides the offsets that the consumer will use when fetching the next set of records.
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
body |
body |
offsets |
Object that provides the offsets to use when fetching the next set of records |
Body parameter
{
"offsets": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0,
"offset": 412
}
]
}
Responses
Status |
Meaning |
Description |
Schema |
204 |
No Content |
Committed offsets for the partitions specified. |
None |
Seek First Offset
POST /consumers/positions/beginning
Seek to the first offset for each of the partitions specified.
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
body |
body |
partitions |
Object that specifies the partitions for which to seek the first offsets |
Body parameter
{
"partitions": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0
}
]
}
Responses
Status |
Meaning |
Description |
Schema |
204 |
No Content |
The current offset is set to the first available offset |
None |
Seek Last Offset
POST /consumers/positions/end
Seek to the last offset for each of the partitions specified.
Parameters
Name |
In |
Type |
Description |
consumer-group |
header |
string |
Consumer group name appID-{appId/topic}-clientName-{clientName}-{two-digit number} |
consumer-name |
header |
string |
Consumer name consumer-{UUIDv4} |
body |
body |
partitions |
Object that specifies the partitions for which to see the last offsets |
Body parameter
{
"partitions": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0
}
]
}
Responses
Status |
Meaning |
Description |
Schema |
204 |
No Content |
The current offset is set to the last available offset |
None |
Schemas
createConsumers
{
"auto.offset.reset": "latest",
"consumer.request.timeout.ms": "1000",
"fetch.min.bytes": "-1",
"auto.commit.enable": "false"
}
Object that provides the parameters used when creating a consumer
Properties
Name |
Type |
Description |
auto.offset.reset |
string |
Offset reset policy for the consumer group. Determines the initial position to use when the group is created. Valid values are earliest (earliest available offset), latest (latest available offset), and none (current offset if the consumer group is present). If there is no current offset associated with the consumer group, the option none will throw an error. This property applies only if no offset is committed. When an offset is committed, auto.offset.reset is not considered. Default is latest . |
consumer.request.timeout.ms |
string |
The maximum amount of time to wait for messages for a request if the maximum number of messages (as per fetch.min.bytes ) has not yet been reached. Default is 1000 . |
fetch.min.bytes |
string |
Minimum number of bytes in message keys and values that must be returned by a single request before the timeout specified in consumer.request.timeout.ms expires. A value of -1 disables this functionality. Default is -1 . |
auto.commit.enable |
string |
If set to true, the committed offset will be automatically updated when the record is fetched using the Get Records endpoint. Default is false . |
Enumerated Values
Property |
Value |
auto.offset.reset |
earliest |
auto.offset.reset |
latest |
auto.offset.reset |
none |
auto.commit.enable |
true |
auto.commit.enable |
false |
committedOffsets
{
"offsets": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0,
"offset": 412,
"metadata": "string"
}
]
}
Last committed offsets for the partitions specified
Properties
Name |
Type |
Description |
offsets |
[any] |
Array of committed offsets |
» topic |
string |
Name of the topic. In Mix, this is the AppID. |
» partition |
number |
Number of the partition |
» offset |
number |
Committed offset |
» metadata |
string |
Currently not used |
offsets
{
"offsets": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0,
"offset": 412
}
]
}
Object that defines offsets
Properties
Name |
Type |
Description |
offsets |
[any] |
Array of offsets |
» topic |
string |
Name of the topic. In Mix, this is the AppID. |
» partition |
number |
Number of the partition |
» offset |
number |
Offset number |
partitions
{
"partitions": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0
}
]
}
Object that defines an array of partitions
Properties
Name |
Type |
Description |
partitions |
[any] |
partitions |
» topic |
string |
Name of the topic. In Mix, this is the AppID. |
» partition |
number |
Number of the partition |
topicPartitions
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partitions": [
0
]
}
Object that defines a topic and its available partitions
Properties
Name |
Type |
Description |
topic |
string |
Name of the topic. In Mix, this is the AppID. |
partitions |
[number] |
Array of partition numbers |
errorResponse
{
"error_code": 401,
"message": "The request could not be authorized"
}
Object that defines an error response
Properties
Name |
Type |
Description |
error_code |
string |
Error code |
message |
string |
Error message |
jsonConsumerRecords
[
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"key": {
"service": "DLGaaS",
"id": "3a4b511c-c06b-497f-ba0a-24c1f95a7e60"
},
"value": {},
"partition": 0,
"offset": 210
}
]
Records fetched by the consumer
Properties
Name |
Type |
Description |
Records fetched by the consumer |
[object] |
Records fetched by the consumer |
topic |
string |
Name of the topic. In Mix, this is the AppID. |
key |
object |
Key for the message |
» service |
any |
Name of the runtime service that generated this message |
» id |
any |
Randomly-generated UUID that uniquely identifies the message |
value |
object |
Contents of the Kafka message |
partition |
number |
Partition number for this topic |
offset |
number |
Offset for this record |
subscription
{
"topics": [
"DEMO-OMNICHANNEL-APP-DEV"
]
}
Object that describes a topic subscription
Properties
Name |
Type |
Description |
topics |
[string] |
Topic name. In Mix, this is the AppID. |
partitionsAssignment
{
"partitions": [
{
"topic": "DEMO-OMNICHANNEL-APP-DEV",
"partition": 0
}
]
}
Object that describes the assigned partitions
Properties
Name |
Type |
Description |
partitions |
[object] |
Array of assigned partitions |
» topic |
string |
Topic name. In Mix, this is the AppID. |
» partition |
number |
Partition number |
partitionsOffsets
{
"beginning_offset": 441,
"end_offset": 641
}
Object that provides the first and last available offsets in a partition
Properties
Name |
Type |
Description |
beginning_offset |
number |
First available offset in the partition specified |
end_offset |
number |
Last available offset in the partition specified |