Use an HDFS filesystem checking to display Hadoop block locations.
Execute an HDFS filesystem checking utility to display file details including block locations.
$ hdfs fsck /sample_file -files -blocks -replicaDetails
Connecting to namenode via http://namenode.example.org:9870/fsck?ugi=hadoop&files=1&blocks=1&replicadetails=1&path=%2Fsample_file FSCK started by hadoop (auth:SIMPLE) from /192.168.8.176 for path /sample_file at Mon Jun 28 19:55:52 UTC 2021 /sample_file 805306368 bytes, replicated: replication=3, 6 block(s): OK 0. BP-2108220822-127.0.1.1-1620683988326:blk_1073752713_11889 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE)] 1. BP-2108220822-127.0.1.1-1620683988326:blk_1073752714_11890 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE)] 2. BP-2108220822-127.0.1.1-1620683988326:blk_1073752715_11891 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE)] 3. BP-2108220822-127.0.1.1-1620683988326:blk_1073752716_11892 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE)] 4. BP-2108220822-127.0.1.1-1620683988326:blk_1073752717_11893 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE)] 5. BP-2108220822-127.0.1.1-1620683988326:blk_1073752718_11894 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE)] Status: HEALTHY Number of data-nodes: 3 Number of racks: 1 Total dirs: 0 Total symlinks: 0 Replicated Blocks: Total size: 805306368 B Total files: 1 Total blocks (validated): 6 (avg. block size 134217728 B) Minimally replicated blocks: 6 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 0 (0.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 2 Average block replication: 3.0 Missing blocks: 0 Corrupt blocks: 0 Missing replicas: 0 (0.0 %) Erasure Coded Block Groups: Total size: 0 B Total files: 0 Total block groups (validated): 0 Minimally erasure-coded block groups: 0 Over-erasure-coded block groups: 0 Under-erasure-coded block groups: 0 Unsatisfactory placement block groups: 0 Average block group size: 0.0 Missing block groups: 0 Corrupt block groups: 0 Missing internal blocks: 0 FSCK ended at Mon Jun 28 19:55:52 UTC 2021 in 1 milliseconds The filesystem under path '/sample_file' is HEALTHY
Sample file with under-replicated blocks.
Connecting to namenode via http://namenode.example.org:9870/fsck?ugi=hadoop&files=1&blocks=1&replicadetails=1&path=%2Fsample_file FSCK started by hadoop (auth:SIMPLE) from /192.168.8.176 for path /sample_file at Mon Jun 28 19:56:30 UTC 2021 /sample_file 805306368 bytes, replicated: replication=5, 6 block(s): Under replicated BP-2108220822-127.0.1.1-1620683988326:blk_1073752713_11889. Target Replicas is 5 but found 3 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s). Under replicated BP-2108220822-127.0.1.1-1620683988326:blk_1073752714_11890. Target Replicas is 5 but found 3 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s). Under replicated BP-2108220822-127.0.1.1-1620683988326:blk_1073752715_11891. Target Replicas is 5 but found 3 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s). Under replicated BP-2108220822-127.0.1.1-1620683988326:blk_1073752716_11892. Target Replicas is 5 but found 3 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s). Under replicated BP-2108220822-127.0.1.1-1620683988326:blk_1073752717_11893. Target Replicas is 5 but found 3 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s). Under replicated BP-2108220822-127.0.1.1-1620683988326:blk_1073752718_11894. Target Replicas is 5 but found 3 live replica(s), 0 decommissioned replica(s), 0 decommissioning replica(s). 0. BP-2108220822-127.0.1.1-1620683988326:blk_1073752713_11889 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE)] 1. BP-2108220822-127.0.1.1-1620683988326:blk_1073752714_11890 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE)] 2. BP-2108220822-127.0.1.1-1620683988326:blk_1073752715_11891 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE)] 3. BP-2108220822-127.0.1.1-1620683988326:blk_1073752716_11892 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE)] 4. BP-2108220822-127.0.1.1-1620683988326:blk_1073752717_11893 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE)] 5. BP-2108220822-127.0.1.1-1620683988326:blk_1073752718_11894 len=134217728 Live_repl=3 [DatanodeInfoWithStorage[192.168.8.175:9866,DS-1afc9b31-2387-4267-80bb-8be36458479a,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.173:9866,DS-66764d91-1c27-4d08-8370-1d03d7fed867,DISK](LIVE), DatanodeInfoWithStorage[192.168.8.174:9866,DS-4c9ae720-729b-4cd7-b791-6f20dd786011,DISK](LIVE)] Status: HEALTHY Number of data-nodes: 3 Number of racks: 1 Total dirs: 0 Total symlinks: 0 Replicated Blocks: Total size: 805306368 B Total files: 1 Total blocks (validated): 6 (avg. block size 134217728 B) Minimally replicated blocks: 6 (100.0 %) Over-replicated blocks: 0 (0.0 %) Under-replicated blocks: 6 (100.0 %) Mis-replicated blocks: 0 (0.0 %) Default replication factor: 2 Average block replication: 3.0 Missing blocks: 0 Corrupt blocks: 0 Missing replicas: 12 (40.0 %) Erasure Coded Block Groups: Total size: 0 B Total files: 0 Total block groups (validated): 0 Minimally erasure-coded block groups: 0 Over-erasure-coded block groups: 0 Under-erasure-coded block groups: 0 Unsatisfactory placement block groups: 0 Average block group size: 0.0 Missing block groups: 0 Corrupt block groups: 0 Missing internal blocks: 0 FSCK ended at Mon Jun 28 19:56:30 UTC 2021 in 1 milliseconds The filesystem under path '/sample_file' is HEALTHY
Beware, these parameters are not available in older Hadoop releases.
