HADOOP - HDFS OPERATIONS

Starting HDFS

To format the configured HDFS file system, execute the following command in namenode HDFS server,

$ hadoop namenode -format

Start the distributed file system. After formatting the HDFS, The following command will start the namenode as well as the data nodes as cluster.

$ start-dfs.sh

Listing Files in HDFS

After loading the information in the server, we can find the list of files in a directory, status of a file, using ‘ls’. Given below is the syntax of ls that you can pass to a directory or a filename as an argument.

$ $HADOOP_HOME/bin/hadoop fs -ls <args>

Inserting Data into HDFS

Imagine we have data in the file called file.txt in the local file system which we intend to save in the Hadoop file system (HDFS). Just follow the steps...

Step 1

-MaKe a DIRrectory in HDFS if you want your file in a new directory

$ $HADOOP_HOME/bin/hadoop fs -mkdir /user/input

Step 2

"-put" your file there.

$ $HADOOP_HOME/bin/hadoop fs -put /home/file.txt /user/input

Step 3

Check your file while taking a look at the "-LiSt "

$ $HADOOP_HOME/bin/hadoop fs -ls /user/input

Retrieving Data from HDFS

After being able to load the data, now you also need to know how to view the data in those files.

Just view the data in the file... call a "-cat "

-cat can be used to get our job done.

$ $HADOOP_HOME/bin/hadoop fs -cat /user/output/outfile

No no no .. i want the file into my local...

Don't worry.. just "-get" it from hadoop fs

$ $HADOOP_HOME/bin/hadoop fs -get /user/output/ /home/hadoop_tp/

Shutting Down the HDFS

Aaaaah!! Done ?? Wrap up by shutting down the HDFS.

$ stop-dfs.sh