سناریوی کار با هادوپ

Hadoop Workshop 1- Login (username: root, password:1)

2- Start terminal

3- Run: Start-all.sh

4- Check if the services are runing:

a. Namenode

b. Datanode

c. Jobtracker

d. Tasktracker

e. Secondary Namenode

5- run PiEstimator program with different arguments and compare the results:

a. hadoop jar /root/NetBeansProjects/PiEstimator/dist/PiEstimator.jar PiEstimator #Map #sample

6- creat a directory into the HDFS:

a. Hadoop fs -mkdir /input

7- creat a text file and write something in it!

8- upload the text file into the HDFS (in the /input directory):

a. hadoop fs -copyFromLocal /root/Desktop/text.txt /input

9- Run the wordCount program:

a. hadoop jar /root/NetBeansProjects/wordcount/dist/wordcount.jar wordcount /input /out

10- you can watch the Jobtracker while processing: http://master:50030

11- after finishing the process, you can see the results using a browser: http://master:50070

12- download the results from HDFS: Hadoop fs -copyToLocal /root/Desktop/text.txt /input

13- you can add some more text files into the /input directory and run the program again.

Some useful commands:

• to remove a directory from HDFS:

o hadoop fs –rmr /directory

• to see the list of files in a directory:

o hadoop fs -ls URI

• to see the content of a file:

o hadoop fs -cat URI [URI …]

• to see the last 1 kilobyte of a file:

o hadoop fs -tail URI

• to see the version of Hadoop:

o hadoop version

• to copy files in HDFS:

o hadoop fs -cp URI

• to move a file across the HDFS:

o hadoop fs -cp URI