You get below Error in your Mapreduce job and there are directories which need permission fix.
The directories are
1. <HADDOP_LOG>/userlogs to have right permission for mapred
2. mr/userlogs - These are all the directories which are mentioned as mapreduce local directories in the configuration files.
Once you fix the permission such that user who is starting mapreduce job - tasktrackers. It will get fixed automatically on next start of job.
2018-03-03 15:02:40,011 WARN org.apache.hadoop.mapred.TaskTracker: Exception while localization ENOENT: No such file or directory
at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method)
at org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:701)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:656)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:514)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:349)
at org.apache.hadoop.mapred.JobLocalizer.initializeJobLogDir(JobLocalizer.java:240)
at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:205)
at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1336)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1311)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
at java.lang.Thread.run(Thread.java:748)
2018-03-03 15:02:40,012 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:mapred cause:ENOENT: No such file or directory
2018-03-03 15:02:40,013 WARN org.apache.hadoop.mapred.TaskTracker: Error initializing attempt_201803031350_0001_m_000019_3:
ENOENT: No such file or directory
The directories are
1. <HADDOP_LOG>/userlogs to have right permission for mapred
2. mr/userlogs - These are all the directories which are mentioned as mapreduce local directories in the configuration files.
Once you fix the permission such that user who is starting mapreduce job - tasktrackers. It will get fixed automatically on next start of job.
2018-03-03 15:02:40,011 WARN org.apache.hadoop.mapred.TaskTracker: Exception while localization ENOENT: No such file or directory
at org.apache.hadoop.io.nativeio.NativeIO.chmod(Native Method)
at org.apache.hadoop.fs.FileUtil.execSetPermission(FileUtil.java:701)
at org.apache.hadoop.fs.FileUtil.setPermission(FileUtil.java:656)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:514)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:349)
at org.apache.hadoop.mapred.JobLocalizer.initializeJobLogDir(JobLocalizer.java:240)
at org.apache.hadoop.mapred.DefaultTaskController.initializeJob(DefaultTaskController.java:205)
at org.apache.hadoop.mapred.TaskTracker$4.run(TaskTracker.java:1336)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.TaskTracker.initializeJob(TaskTracker.java:1311)
at org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:1226)
at org.apache.hadoop.mapred.TaskTracker$5.run(TaskTracker.java:2603)
at java.lang.Thread.run(Thread.java:748)
2018-03-03 15:02:40,012 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:mapred cause:ENOENT: No such file or directory
2018-03-03 15:02:40,013 WARN org.apache.hadoop.mapred.TaskTracker: Error initializing attempt_201803031350_0001_m_000019_3:
ENOENT: No such file or directory
No comments:
Write comments