让mahout0.7在hadoop2.0环境下运行
2015-07-27 15:47
169 查看
由于官方打的jar包是用1.0环境下的,所以要想在2.0下运行,必须重新打包,重新打包前修改几个pom文件就可以,现已经测试通过:
1、根pom文件修改如下:
2、在mahout-core、mahout-examples、mahout-integration工程的pom文件中增加如下代码
3、在根目录下执行 mvn package -Phadoop2 -DskipTests 重新生成jar包
1、根pom文件修改如下:
<hadoop.version>2.5.0</hadoop.version> <!-- 将原来的1.0改为2.0版本--> <!-- 并注释掉hadoop1.0的依赖--> <!--dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-core</artifactId> <version>${hadoop.version}</version> <exclusions> <exclusion> <groupId>net.sf.kosmosfs</groupId> <artifactId>kfs</artifactId> </exclusion> <exclusion> <groupId>org.mortbay.jetty</groupId> <artifactId>jetty</artifactId> </exclusion> <exclusion> <groupId>org.mortbay.jetty</groupId> <artifactId>jetty-util</artifactId> </exclusion> <exclusion> <groupId>hsqldb</groupId> <artifactId>hsqldb</artifactId> </exclusion> <exclusion> <groupId>junit</groupId> <artifactId>junit</artifactId> </exclusion> <exclusion> <groupId>oro</groupId> <artifactId>oro</artifactId> </exclusion> <exclusion> <groupId>org.mortbay.jetty</groupId> <artifactId>jsp-2.1</artifactId> </exclusion> <exclusion> <groupId>org.mortbay.jetty</groupId> <artifactId>jsp-api-2.1</artifactId> </exclusion> <exclusion> <groupId>org.mortbay.jetty</groupId> <artifactId>servlet-api-2.5</artifactId> </exclusion> <exclusion> <groupId>tomcat</groupId> <artifactId>jasper-runtime</artifactId> </exclusion> <exclusion> <groupId>tomcat</groupId> <artifactId>jasper-compiler</artifactId> </exclusion> <exclusion> <groupId>xmlenc</groupId> <artifactId>xmlenc</artifactId> </exclusion> <exclusion> <groupId>net.java.dev.jets3t</groupId> <artifactId>jets3t</artifactId> </exclusion> <exclusion> <groupId>org.eclipse.jdt</groupId> <artifactId>core</artifactId> </exclusion> </exclusions> </dependency-->
2、在mahout-core、mahout-examples、mahout-integration工程的pom文件中增加如下代码
<profiles> <profile> <id>hadoop1</id> <!--activation> <property> <name>!hadoop.version</name> </property> </activation--> <dependencies> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-core</artifactId> <version>1.0.0</version> <scope>compile</scope> <exclusions> <exclusion> <groupId>net.sf.kosmosfs</groupId> <artifactId>kfs</artifactId> </exclusion> <exclusion> <groupId>org.mortbay.jetty</groupId> <artifactId>jetty</artifactId> </exclusion> <exclusion> <groupId>org.mortbay.jetty</groupId> <artifactId>jetty-util</artifactId> </exclusion> <exclusion> <groupId>hsqldb</groupId> <artifactId>hsqldb</artifactId> </exclusion> <exclusion> <groupId>junit</groupId> <artifactId>junit</artifactId> </exclusion> <exclusion> <groupId>oro</groupId> <artifactId>oro</artifactId> </exclusion> <exclusion> <groupId>org.mortbay.jetty</groupId> <artifactId>jsp-2.1</artifactId> </exclusion> <exclusion> <groupId>org.mortbay.jetty</groupId> <artifactId>jsp-api-2.1</artifactId> </exclusion> <exclusion> <groupId>org.mortbay.jetty</groupId> <artifactId>servlet-api-2.5</artifactId> </exclusion> <exclusion> <groupId>tomcat</groupId> <artifactId>jasper-runtime</artifactId> </exclusion> <exclusion> <groupId>tomcat</groupId> <artifactId>jasper-compiler</artifactId> </exclusion> <exclusion> <groupId>xmlenc</groupId> <artifactId>xmlenc</artifactId> </exclusion> <exclusion> <groupId>net.java.dev.jets3t</groupId> <artifactId>jets3t</artifactId> </exclusion> <exclusion> <groupId>org.eclipse.jdt</groupId> <artifactId>core</artifactId> </exclusion> </exclusions> </dependency> </dependencies> </profile> <profile> <id>hadoop2</id> <activation> <property> <name>hadoop.version</name> </property> </activation> <dependencies> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-common</artifactId> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-client-common</artifactId> </dependency> <dependency> <groupId>org.apache.hadoop</groupId> <artifactId>hadoop-mapreduce-client-core</artifactId> </dependency> </dependencies> </profile> </profiles>
3、在根目录下执行 mvn package -Phadoop2 -DskipTests 重新生成jar包
相关文章推荐
- tomcat结合nginx使用小结
- linux CR2 to JPG
- linux 精灵函数
- opencv简介
- 实战Nginx与PHP(FastCGI)的安装、配置与优化
- 从下方弹出的PopupWindow示例 附带一些方法介绍
- FLAG_ACTIVITY_CLEAR_TOP的使用
- 基于Storm的Nginx log实时监控系统
- (转)Linux软链接与硬链接
- hadoop之MapReduce调用R的一次失败的总结~(续四)
- linux 常用命令
- linux 基础命令
- shell脚本首行的解释和用法
- haproxy+keepalived实现高可用负载均衡
- linux下napi学习
- xshell不能输入中文,显示为??
- 关于tomcat端口的修改,以及同时部署多个应用,自己遇到的问题以及解决办法
- LVS Nginx HAProxy 优缺点
- 明明白白AOP(傻瓜也会心领神会)
- Linux学习1_Linux命令及英文全称