FATAL org.apache.hadoop.mapred.JobTracker: ENOENT: No such file or directory
at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method)
at org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:699)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:654)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:509)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:344)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:189)
at org.apache.hadoop.mapred.CompletedJobStatusStore.(CompletedJobStatusStore.java:81)
at org.apache.hadoop.mapred.JobTracker.initialize(JobTracker.java:2051)
at org.apache.hadoop.mapred.JobTracker.offerService(JobTracker.java:2328)
at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:4792)
I realized the most important error is the bold one, which is related to logging as specified in mapred-site.xml
mapred.job.tracker.persist.jobstatus.dir file:////var/log/hadoop/mr/jobstatus
After the the correct folder permissions are set, it should look like following
$ ll /var/log/hadoop/
drwxr-xr-x. 2 hdfs hadoop 4096 Jul 22 22:45 hdfs
drwxr-xr-x. 4 mapred hadoop 4096 Jul 22 23:10 mapred
drwxr-xr-x. 2 mapred hadoop 4096 Jul 22 20:14 mr
After all the services are running, I can list them by#jps
22896 DataNode
26189 TaskTracker
22225 NameNode
26502 Jps
26042 JobTracker
And I can list the listening ports of the services by# lsof -i -P | egrep -v "ESTABLISHED" | grep java
java 22225 hdfs 66u IPv4 573305 0t0 TCP localhost:8020 (LISTEN)
java 22225 hdfs 79u IPv4 573311 0t0 TCP localhost:50070 (LISTEN)
java 22896 hdfs 63u IPv4 575844 0t0 TCP *:50010 (LISTEN)
java 22896 hdfs 65u IPv4 576654 0t0 TCP *:50075 (LISTEN)
java 22896 hdfs 73u IPv4 575856 0t0 TCP *:50020 (LISTEN)
java 26042 mapred 64u IPv4 589571 0t0 TCP localhost:9000 (LISTEN)
java 26042 mapred 75u IPv4 589575 0t0 TCP localhost:50030 (LISTEN)
java 26189 mapred 61u IPv4 590005 0t0 TCP localhost:40660 (LISTEN)
java 26189 mapred 79u IPv4 590177 0t0 TCP *:50060 (LISTEN)
No comments:
Post a Comment