Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException
2014-08-24 15:26
495 查看
新搭建了个集群,搭建完了,运行
hadoop jar /opt/hadoop-2.4.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar pi 10 100
测试一下,却爆出下面的错误:
2014-08-24 02:24:02,456 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Application application_1408860487557_0002 failed 2 times due to AM Container for appattempt_1408860487557_0002_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
2014-08-24 02:24:02,457 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1408860487557_0002 State change from FINAL_SAVING to FAILED
2014-08-24 02:24:02,457 WARN org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1408860487557_0002 failed 2 times due to AM Container for appattempt_1408860487557_0002_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
想到yarn-site.xml中使用了一个以前没配置过的参数:
<property>
<name>yarn.application.classpath</name>
<value>
$HADOOP_CONF_DIR,
$HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,
$HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,
$HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,
$HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*
</value>
</property>
注释掉这个配置,重新运行MR任务,不报错了,毋庸置疑就是路径的问题了。先记录下来,以后再找具体原因。
hadoop jar /opt/hadoop-2.4.1/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar pi 10 100
测试一下,却爆出下面的错误:
2014-08-24 02:24:02,456 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Application application_1408860487557_0002 failed 2 times due to AM Container for appattempt_1408860487557_0002_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
2014-08-24 02:24:02,457 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1408860487557_0002 State change from FINAL_SAVING to FAILED
2014-08-24 02:24:02,457 WARN org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=mapred OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1408860487557_0002 failed 2 times due to AM Container for appattempt_1408860487557_0002_000002 exited with exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException:
org.apache.hadoop.util.Shell$ExitCodeException:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
想到yarn-site.xml中使用了一个以前没配置过的参数:
<property>
<name>yarn.application.classpath</name>
<value>
$HADOOP_CONF_DIR,
$HADOOP_COMMON_HOME/*,$HADOOP_COMMON_HOME/lib/*,
$HADOOP_HDFS_HOME/*,$HADOOP_HDFS_HOME/lib/*,
$HADOOP_MAPRED_HOME/*,$HADOOP_MAPRED_HOME/lib/*,
$HADOOP_YARN_HOME/*,$HADOOP_YARN_HOME/lib/*
</value>
</property>
注释掉这个配置,重新运行MR任务,不报错了,毋庸置疑就是路径的问题了。先记录下来,以后再找具体原因。
相关文章推荐
- Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException
- Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException
- 从window中的eclipse中提交jar包到yarn框架运行,出现Exception from container-launch: org.apache.hadoop.util.Shell$
- org.apache.hadoop.util.Shell$ExitCodeException
- 在hadoop集群上运行mapreduce程序时报错“org.apache.hadoop.util.Shell$ExitCodeException:***not found”
- 【解决】org.apache.hadoop.util.Shell$ExitCodeException: /bin/bash: line 0: fg: no job control
- spark on yarn 报 org.apache.hadoop.util.Shell$ExitCodeException: 问题
- hadoop错误提示 exitCode: 1 due to: Exception from container-launch.
- 关于使用hive删除表时出现的return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException
- hadoop错误提示 exitCode: 1 due to: Exception from container-launch.
- java.lang.ClassNotFoundException: org.apache.hadoop.util.PlatformName
- hive Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException
- Hive学习笔记 --- return code 1 from org.apache.hadoop.hive
- Exit code from container executor initialization is : 24 ExitCodeException exitCode=24: Configuration file ../etc/hadoop/container-executor.cfg not found.
- 关于org.apache.hadoop.yarn.exceptions.YarnException: Unauthorized request to start container
- Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/util/Apps Hadoop2.6.0编程问题与解决
- Exceptioninthread"main"java.lang.NoClassDefFoundError:org/apache/hadoop/util/ProgramDriver
- Windows环境下eclipse提交到远程wordcount程序报错 at org.apache.hadoop.util.Shell.runCommand(Shell.java:545)
- hbase无法启动Regionserver:ClassNotFoundException: org.apache.hadoop.util.PlatformName问题解决
- WARN No appenders could be found for logger (org.apache.hadoop.util.Shell).