سناریوی کار با هادوپ: تفاوت میان نسخهها
پرش به ناوبری
پرش به جستجو
بدون خلاصۀ ویرایش |
بدون خلاصۀ ویرایش |
||
| (۴ نسخهٔ میانی ویرایش شده توسط ۲ کاربر نشان داده نشد) | |||
| خط ۱: | خط ۱: | ||
<div dir="ltr"> | <div dir="ltr"> | ||
Hadoop Workshop | ==Hadoop Workshop== | ||
1- Login (username: root, password:1) | 1- Login (username: root, password:1) | ||
| خط ۷: | خط ۷: | ||
2- Open the terminal | 2- Open the terminal | ||
3- | 3- Start Hadoop: | ||
a. start-all.sh | |||
4- Check if the services are running: | 4- Check if the services are running: | ||
a. Namenode | a.Namenode | ||
b. Datanode | b.Datanode | ||
c. Jobtracker | c.Jobtracker | ||
d. Tasktracker | d.Tasktracker | ||
e. Secondary Namenode | e.Secondary Namenode | ||
5- run PiEstimator program with different arguments and compare the results: | 5- run PiEstimator program with different arguments and compare the results: | ||
| خط ۲۱: | خط ۲۳: | ||
a. hadoop jar /root/NetBeansProjects/PiEstimator/dist/PiEstimator.jar PiEstimator #Map #sample | a. hadoop jar /root/NetBeansProjects/PiEstimator/dist/PiEstimator.jar PiEstimator #Map #sample | ||
6- | 6- create a directory into the HDFS: | ||
a. Hadoop fs -mkdir /input | a. Hadoop fs -mkdir /input | ||
7- | 7- create a text file and write something in it! | ||
8- upload the text file into the HDFS (in the /input directory): | 8- upload the text file into the HDFS (in the /input directory): | ||
| خط ۳۹: | خط ۴۱: | ||
11- after finishing the process, you can see the results using a browser: http://master:50070 | 11- after finishing the process, you can see the results using a browser: http://master:50070 | ||
12- download the results from HDFS: Hadoop fs -copyToLocal /root/Desktop | 12- download the results from HDFS: | ||
a. Hadoop fs -copyToLocal /out /root/Desktop | |||
13- you can add some more text files into the /input directory and run the program again. | 13- you can add some more text files into the /input directory and run the program again. | ||
Some useful commands: | '''Some useful commands:''' | ||
* to remove a directory from HDFS: | * to remove a directory from HDFS: | ||
| خط ۷۲: | خط ۷۶: | ||
* to move a file across the HDFS: | * to move a file across the HDFS: | ||
o hadoop fs - | o hadoop fs -mv URI DestinationDirectory | ||
'''By: Abolfazl Mohammadi Seif''' | |||
</div> | </div> | ||
نسخهٔ کنونی تا ۴ مارس ۲۰۱۵، ساعت ۱۲:۰۶
Hadoop Workshop
1- Login (username: root, password:1)
2- Open the terminal
3- Start Hadoop:
a. start-all.sh
4- Check if the services are running:
a.Namenode b.Datanode c.Jobtracker d.Tasktracker e.Secondary Namenode
5- run PiEstimator program with different arguments and compare the results:
a. hadoop jar /root/NetBeansProjects/PiEstimator/dist/PiEstimator.jar PiEstimator #Map #sample
6- create a directory into the HDFS:
a. Hadoop fs -mkdir /input
7- create a text file and write something in it!
8- upload the text file into the HDFS (in the /input directory):
a. hadoop fs -copyFromLocal /root/Desktop/text.txt /input
9- Run the wordCount program:
a. hadoop jar /root/NetBeansProjects/wordcount/dist/wordcount.jar wordcount /input /out
10- you can watch the Jobtracker while processing: http://master:50030
11- after finishing the process, you can see the results using a browser: http://master:50070
12- download the results from HDFS:
a. Hadoop fs -copyToLocal /out /root/Desktop
13- you can add some more text files into the /input directory and run the program again.
Some useful commands:
- to remove a directory from HDFS:
o hadoop fs –rmr /directory
- to see the list of files in a directory:
o hadoop fs -ls URI
- to see the content of a file:
o hadoop fs -cat URI [URI …]
- to see the last 1 kilobyte of a file:
o hadoop fs -tail URI
- to see the version of Hadoop:
o hadoop version
- to copy files in HDFS:
o hadoop fs -cp URI DestinationDirectory
- to move a file across the HDFS:
o hadoop fs -mv URI DestinationDirectory
By: Abolfazl Mohammadi Seif