Generate Kafka cluster ID for Kafka Raft metadata mode.
Kafka cluster ID is expected to contain 16 bytes of a base64-encoded UUID.
You can generate cluster ID using official utilities.
$ cd /opt/kafka/kafka/
proc filesystem, which comes in handy when you want to dynamically generate multiple clusters without locally extracting Kafka archive to generate UUIDs.
$ cat /proc/sys/kernel/random/uuid | tr -d '-' | base64 | cut -b 1-22
There is also
uuidgen program, which gives you more control over the process as you can generate random-based or time-based UUIDs.
$ apt info uuid-runtime
Package: uuid-runtime Version: 2.36.1-7 Priority: optional Section: utils Source: util-linux Maintainer: util-linux packagers
Installed-Size: 224 kB Pre-Depends: libuuid1 (>= 2.25-5~), init-system-helpers (>= 1.54~) Depends: adduser, libc6 (>= 2.17), libsmartcols1 (>= 2.27~rc1), libsystemd0, libuuid1 (>= 2.31.1) Homepage: http://www.kernel.org/pub/linux/utils/util-linux/ Tag: role::program Download-Size: 101 kB APT-Manual-Installed: yes APT-Sources: http://deb.debian.org/debian bullseye/main amd64 Packages Description: runtime components for the Universally Unique ID library The libuuid library generates and parses 128-bit Universally Unique IDs (UUIDs). A UUID is an identifier that is unique within the space of all such identifiers across both space and time. It can be used for multiple purposes, from tagging objects with an extremely short lifetime to reliably identifying very persistent objects across a network. . See RFC 4122 for more information. . This package contains the uuidgen program and the uuidd daemon. . The uuidd daemon is used to generate UUIDs, especially time-based UUIDs, in a secure and guaranteed-unique fashion, even in the face of large numbers of threads trying to grab UUIDs running on different CPUs. It is used by libuuid as well as the uuidgen program.
$ sudo apt-get install uuid-runtime
Inspect manual page.
$ man uuidgen
[...] -r, --random Generate a random-based UUID. This method creates a UUID consisting mostly of random bits. It requires that the operating system have a high quality random number generator, such as /dev/random. -t, --time Generate a time-based UUID. This method creates a UUID based on the system clock plus the system's ethernet hardware address, if present. [...]
Generate time-based UUID, it does not meet our requirements at this time as it is too long.
$ uuidgen --time
Shorten it a bit to meet our requirements.
$ uuidgen --time | tr -d '-' | base64 | cut -b 1-22
Now it can be used to format storage.
$ sudo -u kafka /opt/kafka/kafka/bin/kafka-storage.sh format -t N2Q0ZTNlMWFhMTgyNDhiZj -c /opt/kafka/kafka/config/kraft/server.properties
$ sudo -u kafka cat /opt/kafka/kafka/kafka_data/meta.properties
# #Sat Jun 19 14:06:49 UTC 2021 node.id=1 version=1 cluster.id=N2Q0ZTNlMWFhMTgyNDhiZj