Categories
DevOps

How to remove old ZooKeeper snapshots and transaction logs

How to remove old ZooKeeper snapshots and transaction logs.

Inspect ZooKeeper snapshots and transaction logs.

$ ls data/version-2/
acceptedEpoch  log.00000001  log.00000003  log.100000001  log.80000000c  snapshot.0          snapshot.500000000  snapshot.700000000  snapshot.900000000
currentEpoch   log.00000002  log.00000004  log.800000001  log.c00000001  snapshot.100000000  snapshot.600000000  snapshot.800000000  version-2

Use built-in shell script to keep last 3 snapshots/transaction logs.

$ bin/zkCleanup.sh -n 3
/usr/bin/java
2021-07-30 22:26:54,732 [myid:] - INFO  [main:FileTxnSnapLog@124] - zookeeper.snapshot.trust.empty : false
2021-07-30 22:26:54,743 [myid:] - INFO  [main:SnapStream@61] - zookeeper.snapshot.compression.method = CHECKED
2021-07-30 22:26:54,755 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:25:19 PM       /opt/zookeeper/zookeeper/data/version-2/log.00000003
Removing file: Jul 30, 2021, 10:25:19 PM        /opt/zookeeper/zookeeper/data/version-2/log.00000003
2021-07-30 22:26:54,756 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:25:19 PM       /opt/zookeeper/zookeeper/data/version-2/log.00000001
Removing file: Jul 30, 2021, 10:25:19 PM        /opt/zookeeper/zookeeper/data/version-2/log.00000001
2021-07-30 22:26:54,765 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:25:19 PM       /opt/zookeeper/zookeeper/data/version-2/log.00000002
Removing file: Jul 30, 2021, 10:25:19 PM        /opt/zookeeper/zookeeper/data/version-2/log.00000002
2021-07-30 22:26:54,766 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:25:19 PM       /opt/zookeeper/zookeeper/data/version-2/log.00000004
Removing file: Jul 30, 2021, 10:25:19 PM        /opt/zookeeper/zookeeper/data/version-2/log.00000004
2021-07-30 22:26:54,787 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:25:19 PM       /opt/zookeeper/zookeeper/data/version-2/snapshot.0
Removing file: Jul 30, 2021, 10:25:19 PM        /opt/zookeeper/zookeeper/data/version-2/snapshot.0
2021-07-30 22:26:54,789 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:25:19 PM       /opt/zookeeper/zookeeper/data/version-2/snapshot.600000000
Removing file: Jul 30, 2021, 10:25:19 PM        /opt/zookeeper/zookeeper/data/version-2/snapshot.600000000
2021-07-30 22:26:54,790 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:25:19 PM       /opt/zookeeper/zookeeper/data/version-2/snapshot.500000000
Removing file: Jul 30, 2021, 10:25:19 PM        /opt/zookeeper/zookeeper/data/version-2/snapshot.500000000
2021-07-30 22:26:54,791 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:25:19 PM       /opt/zookeeper/zookeeper/data/version-2/snapshot.100000000
Removing file: Jul 30, 2021, 10:25:19 PM        /opt/zookeeper/zookeeper/data/version-2/snapshot.100000000

This operation is equivalent to a direct call.

$ java -cp lib/zookeeper-3.7.0.jar:lib/log4j-1.2.17.jar:lib/zookeeper-jute-3.7.0.jar:lib/slf4j-api-1.7.30.jar:lib/slf4j-log4j12-1.7.30.jar:lib/snappy-java-1.1.7.7.jar:conf org.apache.zookeeper.server.PurgeTxnLog /opt/zookeeper/zookeeper/data -n 3 
2021-07-30 22:29:00,611 [myid:] - INFO  [main:FileTxnSnapLog@124] - zookeeper.snapshot.trust.empty : false
2021-07-30 22:29:00,619 [myid:] - INFO  [main:SnapStream@61] - zookeeper.snapshot.compression.method = CHECKED
2021-07-30 22:29:00,630 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:28:15 PM       /opt/zookeeper/zookeeper/data/version-2/log.00000003
Removing file: Jul 30, 2021, 10:28:15 PM        /opt/zookeeper/zookeeper/data/version-2/log.00000003
2021-07-30 22:29:00,633 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:28:15 PM       /opt/zookeeper/zookeeper/data/version-2/log.00000001
Removing file: Jul 30, 2021, 10:28:15 PM        /opt/zookeeper/zookeeper/data/version-2/log.00000001
2021-07-30 22:29:00,636 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:28:15 PM       /opt/zookeeper/zookeeper/data/version-2/log.00000002
Removing file: Jul 30, 2021, 10:28:15 PM        /opt/zookeeper/zookeeper/data/version-2/log.00000002
2021-07-30 22:29:00,638 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:28:15 PM       /opt/zookeeper/zookeeper/data/version-2/log.00000004
Removing file: Jul 30, 2021, 10:28:15 PM        /opt/zookeeper/zookeeper/data/version-2/log.00000004
2021-07-30 22:29:00,639 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:28:15 PM       /opt/zookeeper/zookeeper/data/version-2/snapshot.0
Removing file: Jul 30, 2021, 10:28:15 PM        /opt/zookeeper/zookeeper/data/version-2/snapshot.0
2021-07-30 22:29:00,645 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:28:15 PM       /opt/zookeeper/zookeeper/data/version-2/snapshot.600000000
Removing file: Jul 30, 2021, 10:28:15 PM        /opt/zookeeper/zookeeper/data/version-2/snapshot.600000000
2021-07-30 22:29:00,646 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:28:15 PM       /opt/zookeeper/zookeeper/data/version-2/snapshot.500000000
Removing file: Jul 30, 2021, 10:28:15 PM        /opt/zookeeper/zookeeper/data/version-2/snapshot.500000000
2021-07-30 22:29:00,647 [myid:] - INFO  [main:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:28:15 PM       /opt/zookeeper/zookeeper/data/version-2/snapshot.100000000
Removing file: Jul 30, 2021, 10:28:15 PM        /opt/zookeeper/zookeeper/data/version-2/snapshot.100000000

Alternatively, set the autopurge.snapRetainCount to the number of recent snapshots you want to keep and the autopurge.purgeInterval to the purge interval in hours.

autopurge.snapRetainCount=3
autopurge.purgeInterval=1

Restart ZooKeeper service and sleep well.

$ grep PurgeTask logs/zookeeper-zookeeper-server-zookeeper1.out
2021-07-30 22:33:14,521 [myid:1] - INFO  [PurgeTask:DatadirCleanupManager$PurgeTask@139] - Purge task started.
2021-07-30 22:33:14,543 [myid:1] - INFO  [PurgeTask:FileTxnSnapLog@124] - zookeeper.snapshot.trust.empty : false
2021-07-30 22:33:14,567 [myid:1] - INFO  [PurgeTask:SnapStream@61] - zookeeper.snapshot.compression.method = CHECKED
2021-07-30 22:33:14,580 [myid:1] - INFO  [PurgeTask:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:29:35 PM /opt/zookeeper/zookeeper/data/version-2/log.00000003
2021-07-30 22:33:14,581 [myid:1] - INFO  [PurgeTask:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:29:35 PM /opt/zookeeper/zookeeper/data/version-2/log.00000001
2021-07-30 22:33:14,581 [myid:1] - INFO  [PurgeTask:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:29:35 PM /opt/zookeeper/zookeeper/data/version-2/log.00000002
2021-07-30 22:33:14,582 [myid:1] - INFO  [PurgeTask:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:29:35 PM /opt/zookeeper/zookeeper/data/version-2/log.00000004
2021-07-30 22:33:14,582 [myid:1] - INFO  [PurgeTask:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:29:35 PM /opt/zookeeper/zookeeper/data/version-2/snapshot.0
2021-07-30 22:33:14,583 [myid:1] - INFO  [PurgeTask:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:29:35 PM /opt/zookeeper/zookeeper/data/version-2/snapshot.600000000
2021-07-30 22:33:14,583 [myid:1] - INFO  [PurgeTask:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:29:35 PM /opt/zookeeper/zookeeper/data/version-2/snapshot.500000000
2021-07-30 22:33:14,583 [myid:1] - INFO  [PurgeTask:PurgeTxnLog@157] - Removing file: Jul 30, 2021, 10:29:35 PM /opt/zookeeper/zookeeper/data/version-2/snapshot.100000000
2021-07-30 22:33:14,583 [myid:1] - INFO  [PurgeTask:DatadirCleanupManager$PurgeTask@145] - Purge task completed.