سناریوی کار با هادوپ
Hadoop Workshop 1- Login (username: root, password:1)
2- Open the terminal
3- Run: Start-all.sh
4- Check if the services are running:
a. Namenode b. Datanode c. Jobtracker d. Tasktracker e. Secondary Namenode
5- run PiEstimator program with different arguments and compare the results:
a. hadoop jar /root/NetBeansProjects/PiEstimator/dist/PiEstimator.jar PiEstimator #Map #sample
6- creat a directory into the HDFS:
a. Hadoop fs -mkdir /input
7- creat a text file and write something in it!
8- upload the text file into the HDFS (in the /input directory):
a. hadoop fs -copyFromLocal /root/Desktop/text.txt /input
9- Run the wordCount program:
a. hadoop jar /root/NetBeansProjects/wordcount/dist/wordcount.jar wordcount /input /out
10- you can watch the Jobtracker while processing: http://master:50030
11- after finishing the process, you can see the results using a browser: http://master:50070
12- download the results from HDFS: Hadoop fs -copyToLocal /root/Desktop/text.txt /input
13- you can add some more text files into the /input directory and run the program again.
Some useful commands:
- to remove a directory from HDFS:
o hadoop fs –rmr /directory
- to see the list of files in a directory:
o hadoop fs -ls URI
- to see the content of a file:
o hadoop fs -cat URI [URI …]
- to see the last 1 kilobyte of a file:
o hadoop fs -tail URI
- to see the version of Hadoop:
o hadoop version
- to copy files in HDFS:
o hadoop fs -cp URI DestinationDirectory
- to move a file across the HDFS:
o hadoop fs -cp URI DestinationDirectory
By: Abolfazl Mohammadi Seif