سناریوی کار با هادوپ: تفاوت میان نسخهها
پرش به ناوبری
پرش به جستجو
(صفحهای جدید حاوی «Hadoop Workshop 1- Login (username: root, password:1) 2- Start terminal 3- Run: Start-all.sh 4- Check if the services are runing: a. Namen...» ایجاد کرد) |
بدون خلاصۀ ویرایش |
||
(۱۰ نسخهٔ میانی ویرایش شده توسط ۳ کاربر نشان داده نشد) | |||
خط ۱: | خط ۱: | ||
Hadoop Workshop | <div dir="ltr"> | ||
==Hadoop Workshop== | |||
1- Login (username: root, password:1) | 1- Login (username: root, password:1) | ||
2- | |||
3- | 2- Open the terminal | ||
4- Check if the services are | |||
a. Namenode | 3- Start Hadoop: | ||
b. Datanode | |||
c. Jobtracker | a. start-all.sh | ||
d. Tasktracker | |||
e. Secondary Namenode | 4- Check if the services are running: | ||
a.Namenode | |||
b.Datanode | |||
c.Jobtracker | |||
d.Tasktracker | |||
e.Secondary Namenode | |||
5- run PiEstimator program with different arguments and compare the results: | 5- run PiEstimator program with different arguments and compare the results: | ||
a. hadoop jar /root/NetBeansProjects/PiEstimator/dist/PiEstimator.jar PiEstimator #Map #sample | |||
6- | a. hadoop jar /root/NetBeansProjects/PiEstimator/dist/PiEstimator.jar PiEstimator #Map #sample | ||
a. Hadoop fs -mkdir /input | |||
7- | 6- create a directory into the HDFS: | ||
a. Hadoop fs -mkdir /input | |||
7- create a text file and write something in it! | |||
8- upload the text file into the HDFS (in the /input directory): | 8- upload the text file into the HDFS (in the /input directory): | ||
a. hadoop fs -copyFromLocal /root/Desktop/text.txt /input | |||
a. hadoop fs -copyFromLocal /root/Desktop/text.txt /input | |||
9- Run the wordCount program: | 9- Run the wordCount program: | ||
a. hadoop jar /root/NetBeansProjects/wordcount/dist/wordcount.jar wordcount /input /out | |||
a. hadoop jar /root/NetBeansProjects/wordcount/dist/wordcount.jar wordcount /input /out | |||
10- you can watch the Jobtracker while processing: http://master:50030 | 10- you can watch the Jobtracker while processing: http://master:50030 | ||
11- after finishing the process, you can see the results using a browser: http://master:50070 | 11- after finishing the process, you can see the results using a browser: http://master:50070 | ||
12- download the results from HDFS: Hadoop fs -copyToLocal /root/Desktop | |||
12- download the results from HDFS: | |||
a. Hadoop fs -copyToLocal /out /root/Desktop | |||
13- you can add some more text files into the /input directory and run the program again. | 13- you can add some more text files into the /input directory and run the program again. | ||
Some useful commands: | |||
'''Some useful commands:''' | |||
o hadoop fs –rmr /directory | |||
* to remove a directory from HDFS: | |||
o hadoop fs -ls URI | |||
o hadoop fs –rmr /directory | |||
o hadoop fs -cat URI [URI …] | |||
* to see the list of files in a directory: | |||
o hadoop fs -tail URI | |||
o hadoop fs -ls URI | |||
o hadoop version | |||
* to see the content of a file: | |||
o hadoop fs -cp URI | |||
o hadoop fs -cat URI [URI …] | |||
o hadoop fs - | |||
* to see the last 1 kilobyte of a file: | |||
o hadoop fs -tail URI | |||
* to see the version of Hadoop: | |||
o hadoop version | |||
* to copy files in HDFS: | |||
o hadoop fs -cp URI DestinationDirectory | |||
* to move a file across the HDFS: | |||
o hadoop fs -mv URI DestinationDirectory | |||
'''By: Abolfazl Mohammadi Seif''' | |||
</div> |
نسخهٔ کنونی تا ۴ مارس ۲۰۱۵، ساعت ۱۲:۰۶
Hadoop Workshop
1- Login (username: root, password:1)
2- Open the terminal
3- Start Hadoop:
a. start-all.sh
4- Check if the services are running:
a.Namenode b.Datanode c.Jobtracker d.Tasktracker e.Secondary Namenode
5- run PiEstimator program with different arguments and compare the results:
a. hadoop jar /root/NetBeansProjects/PiEstimator/dist/PiEstimator.jar PiEstimator #Map #sample
6- create a directory into the HDFS:
a. Hadoop fs -mkdir /input
7- create a text file and write something in it!
8- upload the text file into the HDFS (in the /input directory):
a. hadoop fs -copyFromLocal /root/Desktop/text.txt /input
9- Run the wordCount program:
a. hadoop jar /root/NetBeansProjects/wordcount/dist/wordcount.jar wordcount /input /out
10- you can watch the Jobtracker while processing: http://master:50030
11- after finishing the process, you can see the results using a browser: http://master:50070
12- download the results from HDFS:
a. Hadoop fs -copyToLocal /out /root/Desktop
13- you can add some more text files into the /input directory and run the program again.
Some useful commands:
- to remove a directory from HDFS:
o hadoop fs –rmr /directory
- to see the list of files in a directory:
o hadoop fs -ls URI
- to see the content of a file:
o hadoop fs -cat URI [URI …]
- to see the last 1 kilobyte of a file:
o hadoop fs -tail URI
- to see the version of Hadoop:
o hadoop version
- to copy files in HDFS:
o hadoop fs -cp URI DestinationDirectory
- to move a file across the HDFS:
o hadoop fs -mv URI DestinationDirectory
By: Abolfazl Mohammadi Seif