解决Hadoop0.21.0 HDFS/MapReduce编译错误(impossible to resolve dependencies)
2011-11-17 17:03
483 查看
转自:http://www.iteye.com/topic/1070647
最近老大安排,要跟踪HDFS ConcatenateFile的前提条件;于是尝试用ant/ivy编译了一下hdfs,提示ivy找不到hadoop-common组件;于是又编译了一下mapreduce,提示ivy找不到hadoop-comm/hadoop-hdfs;编译hadoop-common,提示编译成功;在编译mapreduce错误如下:
ivy-resolve-common:
[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve]
module not found: org.apache.hadoop#hadoop-common;0.21.0
[ivy:resolve]
==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-common;0.21.0!hadoop-common.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.jar
[ivy:resolve]
==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-common;0.21.0!hadoop-common.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.jar
[ivy:resolve]
module not found: org.apache.hadoop#hadoop-common-test;0.21.0
[ivy:resolve]
==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-common-test;0.21.0!hadoop-common-test.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.jar
[ivy:resolve]
==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-common-test;0.21.0!hadoop-common-test.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.jar
[ivy:resolve]
module not found: org.apache.hadoop#hadoop-hdfs;0.21.0
[ivy:resolve]
==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-hdfs;0.21.0!hadoop-hdfs.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.jar
[ivy:resolve]
==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-hdfs;0.21.0!hadoop-hdfs.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.jar
[ivy:resolve]
::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
:: UNRESOLVED DEPENDENCIES ::
[ivy:resolve]
::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
:: org.apache.hadoop#hadoop-common;0.21.0: not found
[ivy:resolve]
:: org.apache.hadoop#hadoop-common-test;0.21.0: not found
[ivy:resolve]
:: org.apache.hadoop#hadoop-hdfs;0.21.0: not found
[ivy:resolve]
::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
BUILD FAILED
/opt/hadoop-0.21.0/mapred/build.xml:1861: impossible to resolve dependencies:
resolve failed - see output for details
通过Google,发现原来这是Apache-Hadoop的BUG:
https://issues.apache.org/jira/browse/HDFS-1519(hdfs ivy配置bug)
主要原因是ivy配置问题:
hdfs的ivy在配置关联类库hadoop-common时候位置指定错误,具体修改如下:
hdfs/ivy.xml
<dependency org="org.apache.hadoop" name="hadoop-common"
-----rev="${hadoop-common.version}"
+++
rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
hdfs/src/contrib/hdfsproxy/ivy.xml
<dependency org="org.apache.hadoop"
name="hadoop-common"
----
rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
<dependency org="org.apache.hadoop"
name="hadoop-common-test"
--- rev="${hadoop-common.version}"
+++
rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
hdfs/src/contrib/thriftfs/ivy.xml
<dependency org="org.apache.hadoop"
name="hadoop-common"
--- rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
[b]mapreduce的ivy在配置关联类库hadoop-common/hadoop-dfs时候位置指定错误,修改方法与hdfs类似:[/b]
1.修改mapreduce project里的ivy.xml(多个)
在eclipse里search file(ivy.xml):
(1)rev="${hadoop-common.version}" replace rev="${hadoop-common.version}-SNAPSHOT"
(2)rev="${hadoop-hdfs.version}" replace rev="${hadoop-hdfs.version}-SNAPSHOT"
2.修改buid.xml
<unzip
-- -- src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}.jar"
+++ src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}-SNAPSHOT.jar"
dest="${build.dir}">
最近老大安排,要跟踪HDFS ConcatenateFile的前提条件;于是尝试用ant/ivy编译了一下hdfs,提示ivy找不到hadoop-common组件;于是又编译了一下mapreduce,提示ivy找不到hadoop-comm/hadoop-hdfs;编译hadoop-common,提示编译成功;在编译mapreduce错误如下:
ivy-resolve-common:
[ivy:resolve]
[ivy:resolve] :: problems summary ::
[ivy:resolve] :::: WARNINGS
[ivy:resolve]
module not found: org.apache.hadoop#hadoop-common;0.21.0
[ivy:resolve]
==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-common;0.21.0!hadoop-common.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.jar
[ivy:resolve]
==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-common;0.21.0!hadoop-common.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/0.21.0/hadoop-common-0.21.0.jar
[ivy:resolve]
module not found: org.apache.hadoop#hadoop-common-test;0.21.0
[ivy:resolve]
==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-common-test;0.21.0!hadoop-common-test.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.jar
[ivy:resolve]
==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-common-test;0.21.0!hadoop-common-test.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common-test/0.21.0/hadoop-common-test-0.21.0.jar
[ivy:resolve]
module not found: org.apache.hadoop#hadoop-hdfs;0.21.0
[ivy:resolve]
==== apache-snapshot: tried
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-hdfs;0.21.0!hadoop-hdfs.jar:
[ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.jar
[ivy:resolve]
==== maven2: tried
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.pom
[ivy:resolve]
-- artifact org.apache.hadoop#hadoop-hdfs;0.21.0!hadoop-hdfs.jar:
[ivy:resolve] http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/0.21.0/hadoop-hdfs-0.21.0.jar
[ivy:resolve]
::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
:: UNRESOLVED DEPENDENCIES ::
[ivy:resolve]
::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
:: org.apache.hadoop#hadoop-common;0.21.0: not found
[ivy:resolve]
:: org.apache.hadoop#hadoop-common-test;0.21.0: not found
[ivy:resolve]
:: org.apache.hadoop#hadoop-hdfs;0.21.0: not found
[ivy:resolve]
::::::::::::::::::::::::::::::::::::::::::::::
[ivy:resolve]
[ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS
BUILD FAILED
/opt/hadoop-0.21.0/mapred/build.xml:1861: impossible to resolve dependencies:
resolve failed - see output for details
通过Google,发现原来这是Apache-Hadoop的BUG:
https://issues.apache.org/jira/browse/HDFS-1519(hdfs ivy配置bug)
主要原因是ivy配置问题:
hdfs的ivy在配置关联类库hadoop-common时候位置指定错误,具体修改如下:
hdfs/ivy.xml
<dependency org="org.apache.hadoop" name="hadoop-common"
-----rev="${hadoop-common.version}"
+++
rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
hdfs/src/contrib/hdfsproxy/ivy.xml
<dependency org="org.apache.hadoop"
name="hadoop-common"
----
rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
<dependency org="org.apache.hadoop"
name="hadoop-common-test"
--- rev="${hadoop-common.version}"
+++
rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
hdfs/src/contrib/thriftfs/ivy.xml
<dependency org="org.apache.hadoop"
name="hadoop-common"
--- rev="${hadoop-common.version}"
+++ rev="${hadoop-common.version}-SNAPSHOT"
conf="common->default"/>
[b]mapreduce的ivy在配置关联类库hadoop-common/hadoop-dfs时候位置指定错误,修改方法与hdfs类似:[/b]
1.修改mapreduce project里的ivy.xml(多个)
在eclipse里search file(ivy.xml):
(1)rev="${hadoop-common.version}" replace rev="${hadoop-common.version}-SNAPSHOT"
(2)rev="${hadoop-hdfs.version}" replace rev="${hadoop-hdfs.version}-SNAPSHOT"
2.修改buid.xml
<unzip
-- -- src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}.jar"
+++ src="${common.ivy.lib.dir}/hadoop-hdfs-${hadoop-hdfs.version}-SNAPSHOT.jar"
dest="${build.dir}">
相关文章推荐
- 解决Hadoop0.21.0 HDFS/MapReduce编译错误(ivy-resolve-common does not find hadoop-commo)
- Hadoop HA HDFS启动错误之org.apache.hadoop.ipc.Client: Retrying connect to server问题解决
- 解决 android studio 编译错误 failed to resolve "com.google.zxing"
- MapReduce——LongWritable cannot be cast to org.apache.hadoop.io.Text 错误原因
- Unable to resolve superclass of错误的解决方法
- 解决ionic编译android支持问题> Could not resolve all dependencies for configuration ':_debugCompile'. > Co
- Ubuntu / Debian: sudo 出现 unable to resolve host 错误解决办法
- hadoop2.2.0编译--错误解决办法
- Hadoop出现 Wrong FS: hdfs://......错误的解决方法
- 解决android studio 编译新项目出现的Error:Failed to open zip file. Gradle's dependency cache may be corrupt 的错误
- iOS开发中Xcode编译错误 linker command failed with exit code 1 (use -v to see invocation) 的解决办法
- 编译WU FTPD出现"undefined reference to crypt"错误解决方法
- PHP编译时错误解决:Don't know how to define struct flock on this system, set --enable-opcache=no
- linux内核编译错误 No rule to make target menuconfig解决方法
- sqlite3 交叉编译 undefined reference to `fdatasync' 错误解决
- Unable to instantiate org.apache.hadoop.hive.metastore.HiveMetaStoreClient错误解决方法
- Android Studio创建新module时出现: "Failed to resolve: junit:junit:4.12" 编译错误
- GCC编译uboot出现(.ARM.exidx+0x0): undefined reference to `__aeabi_unwind_cpp_pr0'错误的解决的方法
- Ubuntu "sudo: unable to resolve host 主机名称"错误解决方法
- sudo不起作用,出现unable to resolve host错误的解决办法