Categories
DevOps

How to update reconfigurable properties on Hadoop node

Update reconfigurable properties on Hadoop node without restart.

Display reconfigurable properties on namenode.

$ sudo -u hadoop -i hdfs dfsadmin -reconfig namenode namenode.example.org:9000 properties
Node [namenode.example.org:9000] Reconfigurable properties:
dfs.heartbeat.interval
dfs.namenode.heartbeat.recheck-interval
dfs.namenode.replication.max-streams
dfs.namenode.replication.max-streams-hard-limit
dfs.namenode.replication.work.multiplier.per.iteration
dfs.storage.policy.satisfier.mode
fs.protected.directories
hadoop.caller.context.enabled
ipc.9000.backoff.enable

Display reconfigurable properties on datanode.

$ sudo -u hadoop -i hdfs dfsadmin -reconfig datanode datanode3.example.org:9867 properties
Node [datanode3.example.org:9867] Reconfigurable properties:
dfs.datanode.data.dir
dfs.datanode.balance.max.concurrent.moves

Reconfigure datanode after configuration update.

$ sudo -u hadoop -i hdfs dfsadmin -reconfig datanode datanode3.example.org:9867 start 
Started reconfiguration task on node [datanode3.example.org:9867].

Inspect changes.

$ sudo -u hadoop -i hdfs dfsadmin -reconfig datanode datanode3.example.org:9867 status
Reconfiguring status for node [datanode3.example.org:9867]: started at Tue Jul 20 18:41:06 UTC 2021 and finished at Tue Jul 20 18:41:06 UTC 2021.
SUCCESS: Changed property dfs.datanode.data.dir
        From: "/opt/hadoop/local_data/datanode"
        To: "/opt/hadoop/local_data/datanode,/opt/hadoop/local_data/disk2,/opt/hadoop/local_data/disk3"