HDFS Cheat Sheet
List of commands which I use day to day basis to perform my task in HDFS. Checkout below hdfs cheat sheet for reference.
In below list I have covered Copy, Move, Check file/directory size, Change ownership of file/group, delete the files and directories etc.
HDFS Cheat Sheet
| hdfs dfs -ls / | list of all files and directories in given path |
| hdfs dfs -ls -h /hdfsproject/path | list of all files/directories in human readable format |
| hdfs dfs -ls -d /hdfsproject /path1 | list of all directories of /hdfs/path1 as plain file |
| hdfs dfs -mkdir /hdfsproject /path2 | To create new directory inside hdfs folder |
| hdfs dfs -mv /hdfsproject/path1 /hdfsproject/ | Move directory path1 inside /hdfsproject |
| hdfs dfs -cp /hdfsproject/path1 /hdfsproject/ | Copy directory path1 in /hdfsproject directory |
| hdfs dfs -du -h /hdfsproject/file | To display the file size |
| hdfs dfs -du -s /hdfsproject/path1 | To display list of all files in given HDFS path |
| hdfs dfs -df /hdfsproject | Shows the capacity, free and used space of the filesystem. |
| hdfs dfs -chmod 755 /hdfsproject/file1 | Changes permissions of the file. |
| hdfs dfs -chmod -R 755 /hadoop | Changes permissions of the files recursively |
| hdfs dfs -chown hduser:hdgrp /hadoop | Change file group ownership |
| hdfs dfs -chown -R hduser:hdgrp /hadoop | Change file group ownership recursively |
| hdfs dfs -chgrp ubuntu /hadoop | Change group ownership |
| hdfs dfs -chgrp -R ubuntu /hadoop | Change group ownership recursively |
| hdfs dfs -put /home/ubuntu/sample /hadoop | To copies the file from local file system to HDFS. |
| hdfs dfs -put -f /home/ubuntu/sample /hadoop | To copies the file from local file system to HDFS, overwrite it if already exist |
| hdfs dfs -copyFromLocal /home/techieroop/sample /hdfsproject/ | To copy the file from local filesystem to hdfs path |
| hdfs dfs -copyToLocal /hdfsproject/file /home/techieroop/ | To copy the file from HDFS path to local file system |
| hdfs dfs -rm -f /hdfsproject | If the file does not exist, do not display a diagnostic message or modify the exit status to reflect an error. |
| hdfs dfs -rm -skipTrash /hdfsproject | Delete the directory recursively |
| hdfs dfs -rm -skipTrash /hdfsproject | Delete the directory recursively and skip the trash |
(Visited 231 times, 17 visits today)
