The Hadoop cluster enters safe mode during the name node startup till the basic indicators are met and later in case of emergency, which means that the cluster enters read-only mode.
The client will immediately notice that the name node is in safe mode.
$ hdfs dfs -copyFromLocal -f /var/log/syslog /logs/backup/
copyFromLocal: Cannot create file/logs/backup/syslog._COPYING_. Name node is in safe mode.
Inspect log file on the name node.
$ tail -100 /opt/hadoop/hadoop-3.2.2/logs/hadoop-hadoop-namenode-namenode.log
2021-05-18 22:53:04,014 WARN org.apache.hadoop.hdfs.server.namenode.NameNodeResourceChecker: Space available on volume '/dev/mapper/cerberus--ssd-vm--100--disk--0' is 0, which is below the configured reserved amount 104857600 2021-05-18 22:53:04,025 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: NameNode low on available disk space. Entering safe mode.
Focus on resolving the issue before any other action.
Determine current safe mode status.
$ hdfs dfsadmin -safemode get
Safe mode is ON
Disable safe mode.
$ hdfs dfsadmin -safemode leave
Safe mode is OFF
Additional notes
There are other useful options like enter to deliberately enter safe mode, wait which is useful inside a shell script as it just waits till safe mode is off.
$ hdfs dfsadmin -safemode enter|leave|get|wait|forceExit
Inspect mentioned earlier dfs.namenode.resource.du.reserved option.
Also dfs.namenode.safemode.threshold-pct, dfs.namenode.safemode.min.datanodes and dfs.namenode.safemode.extension options which are important during name node startup.
2021-05-18 23:35:12,479 INFO org.apache.hadoop.hdfs.StateChange: STATE* Safe mode ON, in safe mode extension. The reported blocks 5836 has reached the threshold 0.9990 of total blocks 5836. The number of live datanodes 3 has reached the minimum number 3. In safe mode extension. Safe mode will be turned off automatically in 9 seconds.
