Confluent Kafka Rest

In this tutorial, I will show you how to produce and consume messages to and from Confluent Kafka Rest. I can directly post messages using Postman or curl command.

Please go through link and this will give you complete idea around Use Cases and Architectures for HTTP and REST APIs with Apache Kafka.

HTTP and REST APIs with Apache Kafka - Use Cases and Architectures

I’ve not covered confluent-kafka setup that can be easily available on web.

confluent local services start
The local commands are intended for a single-node development environment only,
    NOT for production usage. https://docs.confluent.io/current/cli/index.html

    Using CONFLUENT_CURRENT: /var/folders/kn/4wr9__651l37hckxvnnwt4hh0000gn/T/confluent.162571
    Starting ZooKeeper
    ZooKeeper is [UP]
    Starting Kafka
    Kafka is [UP]
    Starting Schema Registry
    Schema Registry is [UP]
    Starting Kafka REST
    Kafka REST is [UP]
    Starting Connect
    Connect is [UP]
    Starting ksqlDB Server
    ksqlDB Server is [UP]
    Starting Control Center
    Control Center is [UP]

Create Topic

kafka-topics --bootstrap-server localhost:9092 --replication-factor 1 --partitions 1 --create --topic kafka-rest--demo-topic

Using CURL command to send message:

curl --location --request POST 'http://localhost:8082/topics/kafka-rest-demo-topic' \
        --header 'Content-Type: application/vnd.kafka.json.v2+json' \
        --data-raw '{
        "records": [
        {
        "value": {
        "John": "Message sent to Confluent Kafka for Kafka Rest"
        }
        }
        ]
        }'

Consumer —

kafka-console-consumer --bootstrap-server localhost:9092 --topic kafka-rest-demo-topic --from-beginning

— — — -

GET All Topics: http://localhost:8082/topics

GET Specific Topic details: http://localhost:8082/topics/kafka-rest-demo-topic

{
        "name": "kafka-rest-demo-topic",
        "configs": {
        "compression.type": "producer",
        "leader.replication.throttled.replicas": "",
        "min.insync.replicas": "1",
        "message.downconversion.enable": "true",
        "segment.jitter.ms": "0",
        "cleanup.policy": "delete",
        "flush.ms": "9223372036854775807",
        "follower.replication.throttled.replicas": "",
        "segment.bytes": "1073741824",
        "retention.ms": "604800000",
        "flush.messages": "9223372036854775807",
        "message.format.version": "3.0-IV1",
        "max.compaction.lag.ms": "9223372036854775807",
        "file.delete.delay.ms": "60000",
        "max.message.bytes": "1048588",
        "min.compaction.lag.ms": "0",
        "message.timestamp.type": "CreateTime",
        "preallocate": "false",
        "index.interval.bytes": "4096",
        "min.cleanable.dirty.ratio": "0.5",
        "unclean.leader.election.enable": "false",
        "retention.bytes": "-1",
        "delete.retention.ms": "86400000",
        "segment.ms": "604800000",
        "message.timestamp.difference.max.ms": "9223372036854775807",
        "segment.index.bytes": "10485760"
        },
        "partitions": [
        {
        "partition": 0,
        "leader": 1,
        "replicas": [
        {
        "broker": 1,
        "leader": true,
        "in_sync": true
        }
        ]
        }
        ]
}

References:

Messing around with Apache Kafka using the Confluent Cloud free trial

REST Proxy Quick Start | Confluent Documentation

GitHub - simplesteph/kafka-rest: Confluent REST Proxy for Kafka