hadoop 2.6配置记录
2015-12-01 13:46
232 查看
本地hadoop配置
1)core-site.xml
3) mapred-site.xml
4) yarn-site.xml
5)slaves
6) hadoop-env.sh和yarn-env.sh
7) 配置完执行
记得配置PATH环境变量
8)查看状态
http://ejiajie-thinkpad-t450s:50070 http://ejiajie-thinkpad-t450s:8088
环境:ubuntu
1)core-site.xml
<?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>hadoop.tmp.dir</name> <value>/home/hadoop/tmp</value> <description>A base for other temporary directories</description> </property> <property> <name>fs.defaultFS</name> <value>hdfs://nameNode:10000</value> </property> <property> <name>io.file.buffer.size</name> <value>4096</value> </property> </configuration>2)hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>dfs.nameservices</name> <value>hadoop-xxxxx</value> </property> <property> <name>dfs.namenode.secondary.http-address</name> <value>nameNode:50090</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>file:///home/hadoop/dfs/name</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>file:///home/hadoop/dfs/data</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.webhdfs.enabled</name> <value>true</value> </property> </configuration>
3) mapred-site.xml
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> <property> <name>mapreduce.jobtracker.http.address</name> <value>nameNode:50030</value> </property> <property> <name>mapreduce.jobhistory.address</name> <value>nameNode:10020</value> </property> <property> <name>mapreduce.jobhistory.webapp.address</name> <value>nameNode:19888</value> </property> </configuration>
4) yarn-site.xml
<?xml version="1.0"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <configuration> <!-- Site specific YARN configuration properties --> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.resourcemanager.address</name> <value>nameNode:8032</value> </property> <property> <name>yarn.resourcemanager.scheduler.address</name> <value>nameNode:8030</value> </property> <property> <name>yarn.resourcemanager.resource-tracker.address</name> <value>nameNode:8031</value> </property> <property> <name>yarn.resourcemanager.admin.address</name> <value>nameNode:8033</value> </property> <property> <name>yarn.resourcemanager.webapp.address</name> <value>nameNode:8088</value> </property> </configuration>
5)slaves
dataNode
6) hadoop-env.sh和yarn-env.sh
... export JAVA_HOME=/path/to/jdk ...
7) 配置完执行
# hadoop namenode -format
记得配置PATH环境变量
8)查看状态
http://ejiajie-thinkpad-t450s:50070 http://ejiajie-thinkpad-t450s:8088
环境:ubuntu
相关文章推荐
- centos 7 从零开始
- JDK-7u4(rpm)+Tomcat-7.0+JavaCenterHome
- linux下定时执行sql语句
- 企业总机技术架构
- 源码编译安装Nginx
- linux上mysql远程备份和自动导入到指定数据库的shell脚本
- hadoop 2.2 错误总结
- 每天一个linux命令(54):sftp命令
- linux下安装vncserver
- 每天一个linux命令(53):wget命令
- 架构-gradle
- Apache安装与配置
- 充电五分钟通话两小时 VOOC闪充助OPPO
- CentOS下编译安装gcc 4.9.0
- 在Mac上配置Apache+PHP环境
- Tomcat6 只允许指定域名访问,禁用IP地址访问,防止恶意解析
- Linux之kc.cfg文件参数详解
- OpenDJ LDAP
- 【linux】 Linux cp 实现强行覆盖
- 给你的Linux装一个"D盘"