Categories
DevOps

How to extract Kafka topic parameters using command-line utilities

Use command-line utilities to extract Kafka topic parameters.

Create sample topic.

$ ./bin/kafka-topics.sh --bootstrap-server localhost:9092 --create --topic logs --partitions 10 --replication-factor 3 --config retention.ms=50000 --config segment.bytes=1073741824
Created topic logs.

Display topic parameters.

$ ./bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic logs 
Topic: logs     TopicId: v23wE8t9SVyJCEeOStPj-w PartitionCount: 10      ReplicationFactor: 3    Configs: segment.bytes=1073741824,retention.ms=50000
        Topic: logs     Partition: 0    Leader: 5       Replicas: 5,3,4 Isr: 5,3,4
        Topic: logs     Partition: 1    Leader: 4       Replicas: 4,5,1 Isr: 4,5,1
        Topic: logs     Partition: 2    Leader: 1       Replicas: 1,4,2 Isr: 1,4,2
        Topic: logs     Partition: 3    Leader: 2       Replicas: 2,1,3 Isr: 2,1,3
        Topic: logs     Partition: 4    Leader: 3       Replicas: 3,2,5 Isr: 3,2,5
        Topic: logs     Partition: 5    Leader: 5       Replicas: 5,4,1 Isr: 5,4,1
        Topic: logs     Partition: 6    Leader: 4       Replicas: 4,1,2 Isr: 4,1,2
        Topic: logs     Partition: 7    Leader: 1       Replicas: 1,2,3 Isr: 1,2,3
        Topic: logs     Partition: 8    Leader: 2       Replicas: 2,3,5 Isr: 2,3,5
        Topic: logs     Partition: 9    Leader: 3       Replicas: 3,5,4 Isr: 3,5,4

Display topic parameters in a machine-friendly way.

$ ./bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic logs | head --lines 1 | tr '\t' '\n'
Topic: logs
TopicId: v23wE8t9SVyJCEeOStPj-w
PartitionCount: 10
ReplicationFactor: 3
Configs: segment.bytes=1073741824,retention.ms=50000

Display PartitionCount

$ ./bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic logs | head --lines 1 | tr '\t' '\n' | awk -F: '$1=="PartitionCount" { print $2 }' | tr -d ' '
10

Display ReplicationFactor.

$ ./bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic logs | head --lines 1 | tr '\t' '\n' | awk -F: '$1=="ReplicationFactor" { print $2 }' | tr -d ' '
3

Display a topic configuration overrides.

$ ./bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic logs | head --lines 1 | tr '\t' '\n' | awk -F: '$1=="Configs" { print $2 }' | tr ',' '\n' | sed 's/^ //'
segment.bytes=1073741824
retention.ms=50000

This is splendid!