解决 java.lang.NoSuchMethodError: org.apache.hadoop.fs.FSOutputSummer.(Ljava/util/zip/Checksum;II)V
2015-11-03 11:59
621 查看
环境: linux部署hdfs2.6.0集群,windows部署开发环境
为了学习mahout,使用网上的HdfsDAO读写远程hdfs时,ls和mkdir测试通过,但上传文件报错:
参考了: http://softdevblogs.com/ 上的相关帖子
发现是 原有的pom中依赖的hadoop2.6因为与mahout 0.11.0依赖的2.4.1版而冲突,实际使用了2.4.1与线上服务器不符
排除掉mahout 0.11.0间接依赖的hadoop设置,只用自己直接设置的hadoop版本号
如:
为了学习mahout,使用网上的HdfsDAO读写远程hdfs时,ls和mkdir测试通过,但上传文件报错:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.fs.FSOutputSummer.<init>(Ljava/util/zip/Checksum;II)V at org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:1553) at org.apache.hadoop.hdfs.DFSOutputStream.<init>(DFSOutputStream.java:1582) at org.apache.hadoop.hdfs.DFSOutputStream.newStreamForCreate(DFSOutputStream.java:1614) at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1465) at org.apache.hadoop.hdfs.DFSClient.create(DFSClient.java:1390) at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:394) at org.apache.hadoop.hdfs.DistributedFileSystem$6.doCall(DistributedFileSystem.java:390) at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81) at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:390) at org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:334) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:909) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:890) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:787) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:776) at org.playground.HadoopDFSFileReadWrite.main(HadoopDFSFileReadWrite.java:37) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:497) at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)解决方案:
参考了: http://softdevblogs.com/ 上的相关帖子
发现是 原有的pom中依赖的hadoop2.6因为与mahout 0.11.0依赖的2.4.1版而冲突,实际使用了2.4.1与线上服务器不符
排除掉mahout 0.11.0间接依赖的hadoop设置,只用自己直接设置的hadoop版本号
如:
<!-- mahout --> <dependency> <groupId>org.apache.mahout</groupId> <artifactId>mahout-examples</artifactId> <version>0.11.0</version> <exclusions> <exclusion> <artifactId>jdk.tools</artifactId> <groupId>jdk.tools</groupId> </exclusion> <exclusion> <artifactId>hadoop-client</artifactId> <groupId>org.apache.hadoop</groupId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.mahout</groupId> <artifactId>mahout-math</artifactId> <version>0.11.0</version> </dependency> <dependency> <groupId>org.apache.mahout</groupId> <artifactId>mahout-hdfs</artifactId> <version>0.11.0</version> <exclusions> <exclusion> <artifactId>hadoop-client</artifactId> <groupId>org.apache.hadoop</groupId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> <version>2.6.0</version> <exclusions> <exclusion> <groupId>jdk.tools</groupId> <artifactId>jdk.tools</artifactId> </exclusion> <exclusion> <artifactId>jasper-compiler</artifactId> <groupId>tomcat</groupId> </exclusion> <exclusion> <artifactId>jasper-runtime</artifactId> <groupId>tomcat</groupId> </exclusion> <exclusion> <artifactId>servlet-api</artifactId> <groupId>javax.servlet</groupId> </exclusion> <exclusion> <artifactId>jsp-api</artifactId> <groupId>javax.servlet.jsp</groupId> </exclusion> </exclusions> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-hdfs</artifactId> <version>2.6.0</version> <exclusions> <exclusion> <artifactId>jdk.tools</artifactId> <groupId>jdk.tools</groupId> </exclusion> </exclusions> </dependency>
相关文章推荐
- APR 池
- Apache 防盗链设置
- 玩转apache之日志【转帖】
- Apache mahout中HMM(隐马尔可夫)算法的测试
- apache kafka源代码分析工程搭建(IDEA)
- PHP基础——PHP、MySQL、Apache配置
- Linux系统Apache SSL安装与配置
- CentOS7配置Apache多站点VirtualHost
- php的开发的apache的配置及伪静态的应用
- Apache2.2 + php-5.4.45-Win32-VC9-x86 配置
- Apache Server 2.2 多站点域名转向配置文件
- 性能测试-ApacheBench
- Apache Kafka源码分析 - PartitionStateMachine
- Linux(Centos)Apache安装配置过程中可能出现的错误
- [实践证明整理系列]CentOS 6中简单架设Apache James邮件服务器
- Ubuntu 14.04 + Apache2 + Django setup
- Apache ActiveMQ 启动失败解决方法
- 【xampp】修改apache的admin按钮默认路径
- Apache开启伪静态
- Introducing The Newly Redesigned Apache HAWQ [作者:常雷]