您的位置:首页 > 运维架构

Could not locate executable null\bin\winutils.exe in the Hadoop binaries

2017-08-31 10:46 603 查看
在本地调试hadoop和spark程序时一直报一个错误Could not locate executable null\bin\winutils.exe in the Hadoop binaries:,看起来像是HADOOP_HOME没有配置,但我明明配置了啊,检查了好几遍也没发现问题,跟进取可以看到在hadoop.util.Shell中有一个checkHadoopHome的方法,
private static String checkHadoopHome() {

// first check the Dflag hadoop.home.dir with JVM scope
String home = System.getProperty("hadoop.home.dir");

// fall back to the system/user-global env variable
if (home == null) {
home = System.getenv("HADOOP_HOME");
}

try {
// couldn't find either setting for hadoop's home directory
if (home == null) {
throw new IOException("HADOOP_HOME or hadoop.home.dir are not set.");
}

if (home.startsWith("\"") && home.endsWith("\"")) {
home = home.substring(1, home.length()-1);
}

// check that the home setting is actually a directory that exists
File homedir = new File(home);
if (!homedir.isAbsolute() || !homedir.exists() || !homedir.isDirectory()) {
throw new IOException("Hadoop home directory " + homedir
+ " does not exist, is not a directory, or is not an absolute path.");
}

home = homedir.getCanonicalPath();

} catch (IOException ioe) {
if (LOG.isDebugEnabled()) {
LOG.debug("Failed to detect a valid hadoop home directory", ioe);
}
home = null;
}

return home;
}
可以看到它首先是从hadoop.home.dir这个系统变量中获取,无奈只好在运行时通过System.setProperty("hadoop.home.dir", "E:/asWorkSpace/hadoop")来进行手动配置,后来实在想不明白,就一步步调试了一下,发现是在if (!homedir.isAbsolute()|| !homedir.exists() || !homedir.isDirectory())这一步出了问题,”E:/asWorkSpace/hadoop“即不是绝对路径,也不存在,这就搞不懂了,没办法只好将HADOOP_HOME配置成“E://asWorkSpace//hadoop”的形式,问题解决了,不用每次手动设置了,但是为啥HADOOP_HOME的路径设置和JAVA_HOME的格式不一样,我还是没搞懂,不知道还有没有人碰到过这个奇怪的问题。
                                            
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: 
相关文章推荐