【hadoop】14、hadoop2.5的mapreduce的 配置
2015-06-30 17:26
417 查看
配置mapreduce
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> </configuration> 加上这个 <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>
然后在yarn-site.xml里面进行配置
<configuration> <!-- Site specific YARN configuration properties --> <property> <name>yarn.resourcemanager.hostname</name> <!-- 在哪里运行相应的程序 --> <value>hadoop1</value> </property> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> </configuration>
启动
前面启动了hdfs之后
在启动yarn
sbin/start-yarn.sh
Hadoop1上
Node1
Node2
Node3
项目
开始我们的eclipse中的mapreduce的项目运行
首先建立一个java项目
然后导入相应的包
Common下面的
Mapreduce下的
Yarn下面的额
Hdfs下面的
我们的需求文档
1949-10-01 14:21:02 34℃
1949-10-02 14:01:02 36℃
1950-01-01 11:21:02 32℃
1950-10-01 12:21:02 37℃
1951-12-01 12:21:02 23℃
1950-10-02 12:21:02 41℃
1950-10-03 12:21:02 27℃
1951-07-01 12:21:02 45℃
1951-07-02 12:21:02 46℃
1949-10-01 14:21:02 34℃
1949-10-01 14:21:02 34℃
1949-10-01 14:21:02 34℃
1949-10-01 14:21:02 34℃
相关文章推荐
- linux apache vhosts 配置(2.4以上版本)
- VM虚拟机Linux克隆后网卡的操作
- Hadoop2.6.0环境搭建
- Hadoop2.6.0环境搭建
- linux下JDK的卸载
- linux是32位还是64位的识别
- 【hadoop】13、hadoop2.5搭建集群
- CentOS下的JDK安装(.tar.gz文件)
- 浅谈大型web系统架构
- shell脚本调试中打开set选项
- 一个优美的架构需要考虑的几个问题
- CentOS下的JDK安装(.rpm文件)
- linux光标为黑色的问题
- shell文件读取及字符串分隔符
- Tomcat中的webapps中的web应用的文件结构
- 给网站编辑MM写的简易代码情书
- 【Hadoop】12、运行hadoop的时候报错
- 在Linux下安装JITwatch
- perl the special operator
- Nginx 反向代理、负载均衡、页面缓存、URL重写及读写分离详解