每天一点hadoop第六篇（搭建伪分布）

hadoop2.6.0伪分布搭建

搭建环境:centos6.5(64位虚拟机)

工具：编译好的hadoop2.6.0 JDK1.7（1.6编译不通过）

搭建过程：

1，设置静态ip:vi /etc/sysconfig/network-scripts/ifcfg-eth0

修改： BOOTPROTO=”static”

IPADDR=”192.168.253.22”

NETMASK=”255.255.255.0”

GATWAY=”192.168.253.22”

2.设置主机名：vi /etc/sysconfig/network

3.设置防火墙：

service iptables stop

chkconfig iptables off

service iptables status

chkconfig iptables –list

重启网卡：service network restart

4.安装JDK：

tar -zxvf jdk-7u79-linux-x64.tar.gz

设置环境变量：

export JAVA_HOME = /usr/java/jdk

export PATH = PATH:PATH:JAVA_HOME/bin

重启环境变量：source /etc/profile

java -version

5.设置ssh免登陆：

进入到要的陆客户的 ~/.ssh目录下

执行：ssh-keygen –t rsa(四个回车)

cat~/.ssh/id_rsa.pub >>~/.ssh/authorized_keys

6.解压:hadoop2.6.0

tar -zxvf hadoop2.6.0-tar.gz

7.修改环境变量：

vi /etc/profile

export HADOOP_INSTALL=/usr/opt/hadoop

export PATH=${HADOOP_INSTALL}/bin:${HADOOP_INSTALL}/sbin${PATH}

export HADOOP_MAPRED_HOME=${HADOOP_INSTALL}

export HADOOP_COMMON_HOME=${HADOOP_INSTALL}

export HADOOP_HDFS_HOME=${HADOOP_INSTALL}

export YARN_HOME=${HADOOP_INSTALLL}

export HADOOP_COMMON_LIB_NATIVE_DIR=${HADOOP_INSTALL}/lib/natvie

export HADOOP_OPTS="-Djava.library.path=${HADOOP_INSTALL}/lib:${HADOOP_INSTALL}/lib/native"

8.修改配置文件：

1.修改：hadoop-env.sh

vi hadoop-env.sh (/usr/hadoop/hadoop/etc/hadoop)

export JAVA_HOME=/usr/java/jdk

2.修改：core –site.xml

<configuration>

<property>

<name>hadoop.tmp.dir</name>

<value>/usr/opt/hadoop/tmp</value>  <!--一定要配置  系统默认的缓存文件夹在每次重启后会被删除-->

</property>

<property>

<name>fs.default.name</name>

<value>hdfs://master:9000</value>

</property>

</configuration>

3.修改：hdfs-site.xml

<configuration>

<property>

<name>dfs.replication</name>

<value>1</value>

</property>

<property>

<name>dfs.namenode.name.dir</name>

<value>/usr/opt/hadoop/dfs/name</value>

</property>

<property>

<name>dfs.datanode.data.dir</name>

<value>/usr/opt/hadoop/dfs/data</value>

</property>

<property>                 <！--这个属性节点是为了防止后面eclipse存在拒绝读写设置的 -->

<name>dfs.permissions</name>

<value>false</value>

</property>

</configuration>

4.修改：vi yarn-site.xml

`

mapreduce.framework.name

yarn

><property>
><name>yarn.nodemanager.aux-services</name>
><value>mapreduce_shuffle</value>
></property>
></configuration>`
>5.修改：vi mapred-site.xml
>`<property>
><name>mapred.job.tracker</name>
><value>master:9001</value>
></property>`

9.格式化:bin/hdfs namenode-format 或者hadoop namenode –format

10.启动:./sbin/start-dfs.sh ./sbin/start-yarn.sh

11.查看JPS



网页：http://192.168.253.22:50070 (HDFS管理界面)


http://192.168.253.22:8088 （MR管理界面）


参考：http://www.iyunv.com/thread-38384-1-1.html

http://www.centoscn.com/hadoop/2015/0118/4525.html