سناریوی کار با هادوپ

از OCCC Wiki
نسخهٔ تاریخ ‏۴ مارس ۲۰۱۵، ساعت ۱۱:۰۶ توسط Abolfazl (بحث | مشارکت‌ها)
(تفاوت) → نسخهٔ قدیمی‌تر | نمایش نسخهٔ فعلی (تفاوت) | نسخهٔ جدیدتر ← (تفاوت)
پرش به ناوبری پرش به جستجو

Hadoop Workshop

1- Login (username: root, password:1)

2- Open the terminal

3- Start Hadoop:

a.	start-all.sh

4- Check if the services are running:

a.Namenode
b.Datanode
c.Jobtracker
d.Tasktracker
e.Secondary Namenode

5- run PiEstimator program with different arguments and compare the results:

a.	hadoop jar /root/NetBeansProjects/PiEstimator/dist/PiEstimator.jar PiEstimator #Map #sample

6- create a directory into the HDFS:

a.	Hadoop fs -mkdir   /input

7- create a text file and write something in it!

8- upload the text file into the HDFS (in the /input directory):

a.	hadoop fs -copyFromLocal /root/Desktop/text.txt   /input

9- Run the wordCount program:

a.	hadoop  jar  /root/NetBeansProjects/wordcount/dist/wordcount.jar  wordcount  /input   /out

10- you can watch the Jobtracker while processing: http://master:50030

11- after finishing the process, you can see the results using a browser: http://master:50070

12- download the results from HDFS:

a.	Hadoop fs -copyToLocal /out /root/Desktop

13- you can add some more text files into the /input directory and run the program again.


Some useful commands:

  • to remove a directory from HDFS:
o	hadoop fs –rmr    /directory
  • to see the list of files in a directory:
o	hadoop fs -ls URI
  • to see the content of a file:
o	hadoop fs -cat URI [URI …]
  • to see the last 1 kilobyte of a file:
o	hadoop fs -tail URI
  • to see the version of Hadoop:
o	hadoop version
  • to copy files in HDFS:
o	hadoop fs -cp URI  DestinationDirectory
  • to move a file across the HDFS:
o	hadoop fs -mv URI  DestinationDirectory


By: Abolfazl Mohammadi Seif