Confluent Kafka Rest

In this tutorial, I will show you how to produce and consume messages to and from Confluent Kafka Rest. I can directly post messages using Postman or curl command.

Please go through link and this will give you complete idea around Use Cases and Architectures for HTTP and REST APIs with Apache Kafka.

I’ve not covered confluent-kafka setup that can be easily available on web.

confluent local services start
The local commands are intended for a single-node development environment only,
NOT for production usage. https://docs.confluent.io/current/cli/index.html

Using CONFLUENT_CURRENT: /var/folders/kn/4wr9__651l37hckxvnnwt4hh0000gn/T/confluent.162571
Starting ZooKeeper
ZooKeeper is [UP]
Starting Kafka
Kafka is [UP]
Starting Schema Registry
Schema Registry is [UP]
Starting Kafka REST
Kafka REST is [UP]
Starting Connect
Connect is [UP]
Starting ksqlDB Server
ksqlDB Server is [UP]
Starting Control Center
Control Center is [UP]

Create Topic

kafka-topics --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --create --topic kafka-rest--demo-topic

Using CURL command to send message:

curl --location --request POST 'http://localhost:8082/topics/kafka-rest-demo-topic' \
--header 'Content-Type: application/vnd.kafka.json.v2+json' \
--data-raw '{
"records"
: [
{
"value": {
"John": "Message sent to Confluent Kafka for Kafka Rest"
}
}
]
}'

Consumer —

kafka-console-consumer --bootstrap-server localhost:9092 --topic kafka-rest-demo-topic --from-beginning

— — — -

GET All Topics: http://localhost:8082/topics

GET Specific Topic details: http://localhost:8082/topics/kafka-rest-demo-topic

{
"name": "kafka-rest-demo-topic",
"configs": {
"compression.type": "producer",
"leader.replication.throttled.replicas": "",
"min.insync.replicas": "1",
"message.downconversion.enable": "true",
"segment.jitter.ms": "0",
"cleanup.policy": "delete",
"flush.ms": "9223372036854775807",
"follower.replication.throttled.replicas": "",
"segment.bytes": "1073741824",
"retention.ms": "604800000",
"flush.messages": "9223372036854775807",
"message.format.version": "3.0-IV1",
"max.compaction.lag.ms": "9223372036854775807",
"file.delete.delay.ms": "60000",
"max.message.bytes": "1048588",
"min.compaction.lag.ms": "0",
"message.timestamp.type": "CreateTime",
"preallocate": "false",
"index.interval.bytes": "4096",
"min.cleanable.dirty.ratio": "0.5",
"unclean.leader.election.enable": "false",
"retention.bytes": "-1",
"delete.retention.ms": "86400000",
"segment.ms": "604800000",
"message.timestamp.difference.max.ms": "9223372036854775807",
"segment.index.bytes": "10485760"
},
"partitions": [
{
"partition": 0,
"leader": 1,
"replicas": [
{
"broker": 1,
"leader": true,
"in_sync": true
}
]
}
]
}

References:

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store