Kafka consumer API

This section describes the Kafka consumer API.

Version: V1

This document covers v1 of the Mix Kafka REST Consumer API.

URL

https://log.api.nuance.com

Partitions

Get Partitions

GET /partitions

Retrieve the list of available partitions for a topic.

Responses

Status	Meaning	Description	Schema
200	OK	List of available partitions	topicPartitions
404	Not Found	Topic does not exist	errorResponse

200 response example

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "partitions": [
    0
  ]
}

Get Partition Offsets

GET /partitions/{partition}/offsets

Retrieve the first and last available offsets for a partition.

Parameters

Name	In	Type	Description
partition	path	number	Partition number

Responses

Status	Meaning	Description	Schema
200	OK	Partition offset details	partitionsOffsets

200 response example

{
  "beginning_offset": 441,
  "end_offset": 641
}

Consumers and Groups

Create Consumer

POST /consumers

Create a new consumer instance in a consumer group.

The consumer group name must follow the pattern appID-{appId/topic}-clientName-{clientName}-{two-digit number}, for example, appID-DEMO-OMNICHANNEL-APP-DEV-clientName-default-01

The consumer name must follow the pattern consumer-{UUIDv4}, for example, consumer-9872ec64-50d9-4dcd-9f4f-e8b5099a7f4f

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`
body	body	createConsumers	Object that provides the parameters used when creating a consumer

Body parameter

{
  "auto.offset.reset": "latest",
  "consumer.request.timeout.ms": "1000",
  "fetch.min.bytes": "-1",
  "auto.commit.enable": "false"
}

Responses

Status	Meaning	Description	Schema
204	No Content	Consumer instance created successfully	None
400	Bad Request	Bad Request, topic does not exist	errorResponse

400 response example

{
  "error_code": 401,
  "message": "The request could not be authorized"
}

Delete Consumer

DELETE /consumers

Delete the specified consumer instance from a consumer group.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`

Responses

Status	Meaning	Description	Schema
204	No Content	Consumer instance deleted successfully	None

Subscriptions

POST /consumers/subscription

Subscribe a consumer to a topic to get dynamically assigned partitions. If a subscription already exists, it will be replaced.

Note:

Use either the Subscribe endpoint (to assign partitions automatically) or Assign Partitions endpoint (to manually assign partitions).

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`
body	body	subscription	Object that provides the topic to subscribe to

Body parameter

{
  "topics": [
    "DEMO-OMNICHANNEL-APP-DEV"
  ]
}

Responses

Status	Meaning	Description	Schema
204	No Content	Consumer successfully subscribed to the topic specified	None

Unsubscribe

DELETE/consumers/subscription

Remove the existing topic subscription for a consumer.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`

Responses

Status	Meaning	Description	Schema
204	No Content	Consumer successfully unsubscribed from the topic specified	None

Get Subscription

GET /consumers/subscription

Retrieve the topic to which the consumer is currently subscribed.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`

Responses

Status	Meaning	Description	Schema
204	No Content	Current subscribed topic	subscription

204 example response

{
  "topics": [
    "DEMO-OMNICHANNEL-APP-DEV"
  ]
}

Records

Get Records

GET /consumers/records

Fetch data for the topic to which the specified consumer is subscribed. This request requires the accept: application/json header.

Notes:

An inactivity timeout is configured for customer instances, so they will be deleted if not active for a certain period. Therefore, Nuance recommends that you call the Get Records endpoint at a regular interval to keep the consumer instance alive. See the documentation for more information.

In some cases, this endpoint returns an empty array even if there are event logs available. For example, this can happen if an internal fetch timeout occurs. Nuance recommends that you try this endpoint again before determining that all event logs have been fetched.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
Accept	header	string	Required `accept: application/json` header
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`

Responses

Status	Meaning	Description	Schema
200	OK	Records fetched by the consumer instance, in JSON format.	jsonConsumerRecords

200 response example

[
  {
    "topic": "DEMO-OMNICHANNEL-APP-DEV",
    "key": {
      "service": "DLGaaS",
      "id": "3a4b511c-c06b-497f-ba0a-24c1f95a7e60"
    },
    "value": {},
    "partition": 0,
    "offset": 210
  }
]

Assignments

Assign Partitions

POST /consumers/assignments

Manually assign a list of partitions to this consumer. This request requires both the content-type: application/json header and the accept: application/json.

Note:

Use either the Subscribe endpoint (to assign partitions automatically) or Assign Partitions endpoint (to manually assign partitions).

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
Accept	header	string	Required `accept: application/json` header
Content-Type	header	string	Required `content-type: application/json` header
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`
body	body	partitionsAssignment	Object that provides the partitions to assign to a consumer

Body parameter

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Responses

Status	Meaning	Description	Schema
204	No Content	Partition successfully assigned to the consumer	None

Get Assigned Partitions

GET /consumers/assignments

Retrieve the list of partitions currently manually assigned to this consumer. This request requires the accept: application/json header.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
Accept	header	string	Required `accept: application/json` header
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`

Responses

Status	Meaning	Description	Schema
200	OK	List of partitions assigned to the specified consumer	partitionsAssignment

200 response example

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Offsets

Get Committed Offsets

200 response example

{
  "offsets": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0,
      "offset": 412,
      "metadata": "string"
    }
  ]
}

POST /consumers/committed/offsets

Retrieve the last committed offsets for the partitions specified.

Body parameter

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`
body	body	partitions	Object that provides the partitions for which to retrieve offsets

Responses

Status	Meaning	Description	Schema
200	OK	Last committed offsets for the partitions specified	committedOffsets

Commit Offsets

POST /consumers/offsets

Commit a list of offsets for the consumer. When the POST body is empty, this endpoint commits all the records that have been fetched by the consumer instance.

Body parameter

{
  "offsets": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0,
      "offset": 412
    }
  ]
}

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`
body	body	offsets	Object that provides the offsets to commit

Responses

Status	Meaning	Description	Schema
204	No Content	Committed offsets for the partitions specified	None

Positions

Override Offsets

POST /consumers/positions

Overrides the offsets that the consumer will use when fetching the next set of records.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`
body	body	offsets	Object that provides the offsets to use when fetching the next set of records

Body parameter

{
  "offsets": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0,
      "offset": 412
    }
  ]
}

Responses

Status	Meaning	Description	Schema
204	No Content	Committed offsets for the partitions specified.	None

Seek First Offset

POST /consumers/positions/beginning

Seek to the first offset for each of the partitions specified.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`
body	body	partitions	Object that specifies the partitions for which to seek the first offsets

Body parameter

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Responses

Status	Meaning	Description	Schema
204	No Content	The current offset is set to the first available offset	None

Seek Last Offset

POST /consumers/positions/end

Seek to the last offset for each of the partitions specified.

Parameters

Name	In	Type	Description
consumer-group	header	string	Consumer group name `appID-{appId/topic}-clientName-{clientName}-{two-digit number}`
consumer-name	header	string	Consumer name `consumer-{UUIDv4}`
body	body	partitions	Object that specifies the partitions for which to see the last offsets

Body parameter

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Responses

Status	Meaning	Description	Schema
204	No Content	The current offset is set to the last available offset	None

Schemas

createConsumers

{
  "auto.offset.reset": "latest",
  "consumer.request.timeout.ms": "1000",
  "fetch.min.bytes": "-1",
  "auto.commit.enable": "false"
}

Object that provides the parameters used when creating a consumer

Properties

Name	Type	Description
auto.offset.reset	string	Offset reset policy for the consumer group. Determines the initial position to use when the group is created. Valid values are `earliest` (earliest available offset), `latest` (latest available offset), and `none` (current offset if the consumer group is present). If there is no current offset associated with the consumer group, the option `none` will throw an error. This property applies only if no offset is committed. When an offset is committed, `auto.offset.reset` is not considered. Default is `latest`.
consumer.request.timeout.ms	string	The maximum amount of time to wait for messages for a request if the maximum number of messages (as per `fetch.min.bytes`) has not yet been reached. Default is `1000`.
fetch.min.bytes	string	Minimum number of bytes in message keys and values that must be returned by a single request before the timeout specified in `consumer.request.timeout.ms` expires. A value of -1 disables this functionality. Default is `-1`.
auto.commit.enable	string	If set to true, the committed offset will be automatically updated when the record is fetched using the `Get Records` endpoint. Default is `false`.

Enumerated Values

Property	Value
auto.offset.reset	earliest
auto.offset.reset	latest
auto.offset.reset	none
auto.commit.enable	true
auto.commit.enable	false

committedOffsets

{
  "offsets": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0,
      "offset": 412,
      "metadata": "string"
    }
  ]
}

Last committed offsets for the partitions specified

Properties

Name	Type	Description
offsets	[any]	Array of committed offsets
» topic	string	Name of the topic. In Mix, this is the AppID.
» partition	number	Number of the partition
» offset	number	Committed offset
» metadata	string	Currently not used

offsets

{
  "offsets": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0,
      "offset": 412
    }
  ]
}

Object that defines offsets

Properties

Name	Type	Description
offsets	[any]	Array of offsets
» topic	string	Name of the topic. In Mix, this is the AppID.
» partition	number	Number of the partition
» offset	number	Offset number

partitions

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Object that defines an array of partitions

Properties

Name	Type	Description
partitions	[any]	partitions
» topic	string	Name of the topic. In Mix, this is the AppID.
» partition	number	Number of the partition

topicPartitions

{
  "topic": "DEMO-OMNICHANNEL-APP-DEV",
  "partitions": [
    0
  ]
}

Object that defines a topic and its available partitions

Properties

Name	Type	Description
topic	string	Name of the topic. In Mix, this is the AppID.
partitions	[number]	Array of partition numbers

errorResponse

{
  "error_code": 401,
  "message": "The request could not be authorized"
}

Object that defines an error response

Properties

Name	Type	Description
error_code	string	Error code
message	string	Error message

jsonConsumerRecords

[
  {
    "topic": "DEMO-OMNICHANNEL-APP-DEV",
    "key": {
      "service": "DLGaaS",
      "id": "3a4b511c-c06b-497f-ba0a-24c1f95a7e60"
    },
    "value": {},
    "partition": 0,
    "offset": 210
  }
]

Records fetched by the consumer

Properties

Name	Type	Description
Records fetched by the consumer	[object]	Records fetched by the consumer
topic	string	Name of the topic. In Mix, this is the AppID.
key	object	Key for the message
» service	any	Name of the runtime service that generated this message
» id	any	Randomly-generated UUID that uniquely identifies the message
value	object	Contents of the Kafka message
partition	number	Partition number for this topic
offset	number	Offset for this record

subscription

{
  "topics": [
    "DEMO-OMNICHANNEL-APP-DEV"
  ]
}

Object that describes a topic subscription

Properties

Name	Type	Description
topics	[string]	Topic name. In Mix, this is the AppID.

partitionsAssignment

{
  "partitions": [
    {
      "topic": "DEMO-OMNICHANNEL-APP-DEV",
      "partition": 0
    }
  ]
}

Object that describes the assigned partitions

Properties

Name	Type	Description
partitions	[object]	Array of assigned partitions
» topic	string	Topic name. In Mix, this is the AppID.
» partition	number	Partition number

partitionsOffsets

{
  "beginning_offset": 441,
  "end_offset": 641
}

Object that provides the first and last available offsets in a partition

Properties

Name	Type	Description
beginning_offset	number	First available offset in the partition specified
end_offset	number	Last available offset in the partition specified

Feedback

Was this page helpful?

Glad to hear it! Please tell us how we can improve.

Sorry to hear that. Please tell us how we can improve.