您的位置:首页 > 编程语言 > Java开发

java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries

2015-12-30 16:44 537 查看
       错误如下:
[DefaultQuartzScheduler_Worker-5] ERROR org.apache.hadoop.util.Shell - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:355)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:370)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:363)
at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:79)
at org.apache.hadoop.yarn.conf.YarnConfiguration.<clinit>(YarnConfiguration.java:590)
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:88)
at org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.serviceStart(YarnClientImpl.java:174)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at com.bigdata.bi.unicorn.core.common.yarn.GetYarnResourceTools.getAllResource(GetYarnResourceTools.java:43)
at com.bigdata.bi.unicorn.core.common.yarn.GetYarnResourceTools.getResourceUsedRatio(GetYarnResourceTools.java:58)
at com.bigdata.bi.unicorn.core.schedule.executor.WorkFlowExector.hasResourceByRatio(WorkFlowExector.java:80)
at com.bigdata.bi.unicorn.core.schedule.executor.WorkFlowExector.hasResource(WorkFlowExector.java:64)
at com.bigdata.bi.unicorn.core.schedule.executor.WorkFlowExector.executeWorkFlow(WorkFlowExector.java:40)
at com.bigdata.bi.unicorn.core.quartz.schedule.WorkFlowScheduleJob.execute(WorkFlowScheduleJob.java:39)
at org.quartz.core.JobRunShell.run(JobRunShell.java:203)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:520)
[DefaultQuartzScheduler_Worker-5] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at /10.0.30.101:8032


查看hadoop源码发现里有这么一段:

public static final String getQualifiedBinPath(String executable)
throws IOException {
// construct hadoop bin path to the specified executable
String fullExeName = HADOOP_HOME_DIR + File.separator + "bin"
+ File.separator + executable;

File exeFile = new File(fullExeName);
if (!exeFile.exists()) {
throw new IOException("Could not locate executable " + fullExeName
+ " in the Hadoop binaries.");
}

return exeFile.getCanonicalPath();
}

private static String HADOOP_HOME_DIR = checkHadoopHome();
private static String checkHadoopHome() {

// first check the Dflag hadoop.home.dir with JVM scope
String home = System.getProperty("hadoop.home.dir");

// fall back to the system/user-global env variable
if (home == null) {
home = System.getenv("HADOOP_HOME");
}
...
}
很明显应该是HADOOP_HOME的问题。如果HADOOP_HOME为空,必然fullExeName为null\bin\winutils.exe。解决方法很简单,配置环境变量,不想重启电脑可以在程序里加上:

?
注:E:\\Program Files\\hadoop-2.7.0是我本机解压的hadoop的路径。

稍后再执行,你可能还是会出现同样的错误,这个时候你可能会要怪我了。其实一开始我是拒绝的,因为你进入你的hadoop-x.x.x/bin目录下看,你会发现你压根就没有winutils.exe。

?。
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: