Hadoop集群搭建

Hadoop集群搭建

一、Linux系统配置
1、在所有的机器上都建立相同的目录，也可以就建立相同的用户，以该用户的home路径来做hadoop的安装路径。例如在所有的机器上都建立了名为fan的用户：/home/fan。
etc/hosts文件末尾添加IP-主机映射关系

2、rysnc

vi/etc/rysnc.conf

rsync--daemon

客户端：

rsync -avzP  mjiang@venus::hadoop  hadoop-1.0.0/etc/hadoop/

2、ssh 免密码登录
/article/10262722.html
附录1：
#!/bin/bash

filename=/home/mjiang/.ssh/authorized_keys

slaves=(mars jupiter saturn)

for i in ${slaves[@]}

do

echo $i

scp $filename mjiang@$i:$filename

done

3、安装hadoop，借助rysnc
tar zxvf hadoop-1.0.0-bin.tar.gz

4、安装jdk，借助rysnc
su tar zxvf jdk-7u1-linux-i586.tar.gz -C /usr/share/

5、配置文件修改
http://hadoop.apache.org/common/docs/r0.19.2/cn/cluster_setup.html

vi hadoop-1.0.0/etc/hadoop/hadoop-env.sh
export JAVA_HOME=/usr/share/jdk1.7.0_01
chmod+x hadoop-1.0.0/sbin/*

6、开始
./sbin/stop-all.sh
stoppingjobtracker
mars: stoppingtasktracker
jupiter:stopping tasktracker
saturn:stopping tasktracker
no namenodeto stop


（如果出现了这个那么就会出现

mjiang@venus~/hadoop-1.0.0 $ hadoop fs -ls /

12/02/1516:31:32 INFO ipc.Client: Retrying connect to server:venus/211.83.105.12:9000. Already tried 0 time(s).）
mars: stoppingdatanode
jupiter:stopping datanode
saturn:stopping datanode
venus:stopping secondarynamenode
hadoopnamenode -format
12/02/1516:39:13 INFO common.Storage: Storage directory/tmp/hadoop-mjiang/dfs/name has been successfully formatted.

./sbin/start-dfs.sh
..12/02/1516:39:13 INFO common.Storage: Storage directory/tmp/hadoop-mjiang/dfs/name
has been successfully formatted....
./sbin/start-mapred.sh

在客户端：
mjiang@mars~ $ hadoop fs -mkdir /user/mjiang
mkdir: cannotcreate directory /user/mjiang: File exists
－－－是同步的