Alter memory allocation pool for Kafka and ZooKeeper.
GC Logging for both of these services is enabled using KAFKA_HEAP_OPTS
variable.
Kafka service.
$ cat bin/kafka-server-start.sh
#!/bin/bash # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. if [ $# -lt 1 ]; then echo "USAGE: $0 [-daemon] server.properties [--override property=value]*" exit 1 fi base_dir=$(dirname $0) if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties" fi if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G" fi EXTRA_ARGS=${EXTRA_ARGS-'-name kafkaServer -loggc'} COMMAND=$1 case $COMMAND in -daemon) EXTRA_ARGS="-daemon "$EXTRA_ARGS shift ;; *) ;; esac exec $base_dir/kafka-run-class.sh $EXTRA_ARGS kafka.Kafka "$@"
ZooKeeper service.
$ cat bin/zookeeper-server-start.sh
#!/bin/bash # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. if [ $# -lt 1 ]; then echo "USAGE: $0 [-daemon] zookeeper.properties" exit 1 fi base_dir=$(dirname $0) if [ "x$KAFKA_LOG4J_OPTS" = "x" ]; then export KAFKA_LOG4J_OPTS="-Dlog4j.configuration=file:$base_dir/../config/log4j.properties" fi if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then export KAFKA_HEAP_OPTS="-Xmx512M -Xms512M" fi EXTRA_ARGS=${EXTRA_ARGS-'-name zookeeper -loggc'} COMMAND=$1 case $COMMAND in -daemon) EXTRA_ARGS="-daemon "$EXTRA_ARGS shift ;; *) ;; esac exec $base_dir/kafka-run-class.sh $EXTRA_ARGS org.apache.zookeeper.server.quorum.QuorumPeerMain "$@"
Alter KAFKA_HEAP_OPTS
for the Kafka service variable to alter memory allocation pool.
$ cat /etc/systemd/system/kafka.service
[Unit] Description=Kafka Service After=network-online.target Requires=network-online.target [Service] Environment="KAFKA_HEAP_OPTS=Xmx32G -Xms32G" Type=simple User=kafka Group=kafka WorkingDirectory=/opt/kafka Restart=on-failure ExecStart=/opt/kafka/kafka/bin/kafka-server-start.sh /opt/kafka/kafka/config/server.properties ExecStop=/opt/kafka/kafka/bin/kafka-server-stop.sh /opt/kafka/kafka/config/server.properties [Install] WantedBy=multi-user.targe
Alter KAFKA_HEAP_OPTS
for the ZooKeeper service variable to alter memory allocation pool.
$ cat /etc/systemd/system/zookeeper.service
[Unit] Description=ZooKeeper Service After=network-online.target Requires=network-online.target [Service] Environment="KAFKA_HEAP_OPTS=Xmx1G -Xms1G" Type=simple User=kafka Group=kafka WorkingDirectory=/opt/kafka Restart=on-failure ExecStart=/opt/kafka/kafka/bin/zookeeper-server-start.sh /opt/kafka/kafka/config/zookeeper.properties ExecStop=/opt/kafka/kafka/bin/zookeeper-server-stop.sh /opt/kafka/kafka/config/zookeeper.properties [Install] WantedBy=multi-user.target
It’s as simple as that.