【hadoop 2.6】hadoop 2.6源码编译过程,redhat 5.8操作系统进行编译【附:软件下载】
2015-02-04 16:53
691 查看
大家在官网下载hadoop2.6安装完使用的时候,总是在控制台有这样一句
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
因为官网给的32位系统编译的版本,在64位的操作系统上使用就需要自己编译源码了
./configure
make
make check
make install
mvn
package -Pdist,native -DskipTests -Dtar
[INFO] Apache Hadoop Project POM ......................... SUCCESS [16.054s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [4.956s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.536s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [18.575s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [5.347s]
[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [4.062s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [6.434s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [4.823s]
[INFO] Apache Hadoop Common .............................. SUCCESS [2:44.417s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [9.645s]
[INFO] Apache Hadoop KMS ................................. SUCCESS [19.681s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.074s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [5:05.240s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [28.723s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [15.884s]
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [6.125s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.072s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.068s]
[INFO] hadoop-yarn-api ................................... SUCCESS [4:41.183s]
[INFO] hadoop-yarn-common ................................ SUCCESS [43.377s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.073s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [17.120s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [23.088s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [4.509s]
[INFO] hadoop-yarn-server-applicationhistoryservice ...... SUCCESS [9.256s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [25.804s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [8.690s]
[INFO] hadoop-yarn-client ................................ SUCCESS [10.050s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.062s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [4.423s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [2.732s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.064s]
[INFO] hadoop-yarn-registry .............................. SUCCESS [8.125s]
[INFO] hadoop-yarn-project ............................... SUCCESS [7.215s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.114s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [27.948s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [31.602s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [6.624s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [12.136s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [12.574s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [6.447s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [2.662s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [8.593s]
[INFO] hadoop-mapreduce .................................. SUCCESS [6.852s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [7.445s]
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [11.579s]
[INFO] Apache Hadoop Archives ............................ SUCCESS [3.206s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [8.972s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [6.194s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [4.075s]
[INFO] Apache Hadoop Ant Tasks ........................... SUCCESS [3.546s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [4.469s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [9.900s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [8.320s]
[INFO] Apache Hadoop Amazon Web Services support ......... SUCCESS [6.285s]
[INFO] Apache Hadoop Client .............................. SUCCESS [10.249s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.183s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [6.699s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [14.929s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.058s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [48.591s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 22:18.595s
[INFO] Finished at: Wed Feb 04 16:35:19 CST 2015
[INFO] Final Memory: 101M/251M
[INFO] ------------------------------------------------------------------------
将以上编译好的文件夹内的所有内容替换掉你原来的32位的hadoop的native文件
现在你可以安心舒服的进行文件系统操作了,再也不会看见讨厌的:
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
maven
ant
findbugs 3.0.0 protobuf 2.5.0 hadoop 2.6
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
因为官网给的32位系统编译的版本,在64位的操作系统上使用就需要自己编译源码了
下面介绍下我的编译过程:
这里是下载了maven,ant,findbugs,分别解压后,配置环境变量
这是hadoop 2.6的源码,解压即可
下面继续安装需要的软件:1、安装gcc,gcc-c++
2、安装make,cmake
3、安装openssl-devel,ncurses-devel
4、最后开始安装protoc buf
压缩包解压后./configure
make
make check
make install
5、编译
mvn clean install -DskipTestsmvn
package -Pdist,native -DskipTests -Dtar
6、编译成功
[INFO] Apache Hadoop Main ................................ SUCCESS [7.430s][INFO] Apache Hadoop Project POM ......................... SUCCESS [16.054s]
[INFO] Apache Hadoop Annotations ......................... SUCCESS [4.956s]
[INFO] Apache Hadoop Assemblies .......................... SUCCESS [0.536s]
[INFO] Apache Hadoop Project Dist POM .................... SUCCESS [18.575s]
[INFO] Apache Hadoop Maven Plugins ....................... SUCCESS [5.347s]
[INFO] Apache Hadoop MiniKDC ............................. SUCCESS [4.062s]
[INFO] Apache Hadoop Auth ................................ SUCCESS [6.434s]
[INFO] Apache Hadoop Auth Examples ....................... SUCCESS [4.823s]
[INFO] Apache Hadoop Common .............................. SUCCESS [2:44.417s]
[INFO] Apache Hadoop NFS ................................. SUCCESS [9.645s]
[INFO] Apache Hadoop KMS ................................. SUCCESS [19.681s]
[INFO] Apache Hadoop Common Project ...................... SUCCESS [0.074s]
[INFO] Apache Hadoop HDFS ................................ SUCCESS [5:05.240s]
[INFO] Apache Hadoop HttpFS .............................. SUCCESS [28.723s]
[INFO] Apache Hadoop HDFS BookKeeper Journal ............. SUCCESS [15.884s]
[INFO] Apache Hadoop HDFS-NFS ............................ SUCCESS [6.125s]
[INFO] Apache Hadoop HDFS Project ........................ SUCCESS [0.072s]
[INFO] hadoop-yarn ....................................... SUCCESS [0.068s]
[INFO] hadoop-yarn-api ................................... SUCCESS [4:41.183s]
[INFO] hadoop-yarn-common ................................ SUCCESS [43.377s]
[INFO] hadoop-yarn-server ................................ SUCCESS [0.073s]
[INFO] hadoop-yarn-server-common ......................... SUCCESS [17.120s]
[INFO] hadoop-yarn-server-nodemanager .................... SUCCESS [23.088s]
[INFO] hadoop-yarn-server-web-proxy ...................... SUCCESS [4.509s]
[INFO] hadoop-yarn-server-applicationhistoryservice ...... SUCCESS [9.256s]
[INFO] hadoop-yarn-server-resourcemanager ................ SUCCESS [25.804s]
[INFO] hadoop-yarn-server-tests .......................... SUCCESS [8.690s]
[INFO] hadoop-yarn-client ................................ SUCCESS [10.050s]
[INFO] hadoop-yarn-applications .......................... SUCCESS [0.062s]
[INFO] hadoop-yarn-applications-distributedshell ......... SUCCESS [4.423s]
[INFO] hadoop-yarn-applications-unmanaged-am-launcher .... SUCCESS [2.732s]
[INFO] hadoop-yarn-site .................................. SUCCESS [0.064s]
[INFO] hadoop-yarn-registry .............................. SUCCESS [8.125s]
[INFO] hadoop-yarn-project ............................... SUCCESS [7.215s]
[INFO] hadoop-mapreduce-client ........................... SUCCESS [0.114s]
[INFO] hadoop-mapreduce-client-core ...................... SUCCESS [27.948s]
[INFO] hadoop-mapreduce-client-common .................... SUCCESS [31.602s]
[INFO] hadoop-mapreduce-client-shuffle ................... SUCCESS [6.624s]
[INFO] hadoop-mapreduce-client-app ....................... SUCCESS [12.136s]
[INFO] hadoop-mapreduce-client-hs ........................ SUCCESS [12.574s]
[INFO] hadoop-mapreduce-client-jobclient ................. SUCCESS [6.447s]
[INFO] hadoop-mapreduce-client-hs-plugins ................ SUCCESS [2.662s]
[INFO] Apache Hadoop MapReduce Examples .................. SUCCESS [8.593s]
[INFO] hadoop-mapreduce .................................. SUCCESS [6.852s]
[INFO] Apache Hadoop MapReduce Streaming ................. SUCCESS [7.445s]
[INFO] Apache Hadoop Distributed Copy .................... SUCCESS [11.579s]
[INFO] Apache Hadoop Archives ............................ SUCCESS [3.206s]
[INFO] Apache Hadoop Rumen ............................... SUCCESS [8.972s]
[INFO] Apache Hadoop Gridmix ............................. SUCCESS [6.194s]
[INFO] Apache Hadoop Data Join ........................... SUCCESS [4.075s]
[INFO] Apache Hadoop Ant Tasks ........................... SUCCESS [3.546s]
[INFO] Apache Hadoop Extras .............................. SUCCESS [4.469s]
[INFO] Apache Hadoop Pipes ............................... SUCCESS [9.900s]
[INFO] Apache Hadoop OpenStack support ................... SUCCESS [8.320s]
[INFO] Apache Hadoop Amazon Web Services support ......... SUCCESS [6.285s]
[INFO] Apache Hadoop Client .............................. SUCCESS [10.249s]
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [0.183s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [6.699s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [14.929s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [0.058s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [48.591s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 22:18.595s
[INFO] Finished at: Wed Feb 04 16:35:19 CST 2015
[INFO] Final Memory: 101M/251M
[INFO] ------------------------------------------------------------------------
7、替换native文件夹
/home/hadoop/hadoop-2.6.0-src/hadoop-dist/target/hadoop-2.6.0/lib/native将以上编译好的文件夹内的所有内容替换掉你原来的32位的hadoop的native文件
现在你可以安心舒服的进行文件系统操作了,再也不会看见讨厌的:
WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
附录:
软件下载:maven
ant
findbugs 3.0.0 protobuf 2.5.0 hadoop 2.6
相关文章推荐
- Ubuntu12.04下在Android4.0.4源码下载及其编译过程
- Android源码从下载到编译全过程
- redhat5.8+pam+mysql+vsftp(提供软件包可以到 百度云盘下载 账号 pankuo1@sina.cn 密码pk492940446)
- Android源码下载与编译全过程
- Ubuntu12.04下在Android4.0.4源码下载及其编译过程
- Linux操作系统下如何编译安装源码包软件
- TextMate2 最新版下载及源码编译过程
- Ubuntu12.04下在Android4.0.4源码下载及其编译过程
- Hadoop 源码下载,编译,install,导入eclipse
- FrameLayout measure过程源码Log全解析之一:源码下载及编译
- Ubuntu下载编译Android源码全过程
- Linux操作系统下如何编译安装源码包软件
- Linux操作系统下如何编译安装源码包软件
- openWRT自学---自己编译的第一个 backfire10.03 版本的过程记录 --- 实际是由于下载了错误的backfire源码包导致的
- 在进行嵌入式开发之前,首先要建立一个交叉编译环境,这是一套编译器、连接器和libc库等组成的开发环境。文章通过一个具体的例子说明了这些嵌入式交叉编译开发工具的制作过程。 随着消费类电子产品的大量开发和应用和Linux操作系统的不断健壮和强大,嵌入式系统越来
- android4.2源码下载编译过程
- Linux操作系统下如何编译安装源码包软件
- android 源码下载编译全过程必读
- Android源码下载和编译过程
- 用Eclipse对Hadoop文件进行压缩和解压缩编译过程中遇到的问题