Hadoop学习笔记(一)之示例程序:计算每年的最高温度MaxTemperature
2014-10-20 15:07
323 查看
转载自:http://www.cnblogs.com/beanmoon/archive/2012/12/07/2804183.html
链接保留 手打完再填坑
链接保留 手打完再填坑
package testMapReduce; import java.io.IOException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.IntWritable; import org.apache.hadoop.io.LongWritable; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce.Reducer; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; public class TestTemperature { public static class dataMapper extends Mapper<LongWritable,Text,Text,IntWritable>{ private static final int MISSING=9999; public void map(LongWritable key,Text value,Context context)throws IOException,InterruptedException{ String line=value.toString(); String year=line.substring(15,19); int airTemperature; if(line.charAt(87) == '+'){ airTemperature= Integer.parseInt(line.substring(88, 92)); }else{ airTemperature=Integer.parseInt(line.substring(87,92)); } String quality=line.substring(92,93); if(airTemperature!=MISSING&&quality.matches("[01459]")){ context.write(new Text(year), new IntWritable(airTemperature)); } } } public static class dataReducer extends Reducer<Text,IntWritable,Text,IntWritable>{ public void reduce(Text key,Iterable<IntWritable> values,Context context)throws IOException,InterruptedException{ int maxValue=Integer.MIN_VALUE; for(IntWritable value:values) { maxValue= Math.max(maxValue, value.get()); } context.write(key, new IntWritable(maxValue)); } } public static void main(String args[])throws Exception{ Configuration conf=new Configuration(); if(args.length!=2){ System.err.println("missing in or out"); System.exit(2); } Job job=new Job(conf,"TestTemperature"); job.setJarByClass(TestTemperature.class); job.setMapperClass(dataMapper.class); job.setReducerClass(dataReducer.class); job.setCombinerClass(dataReducer.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(IntWritable.class); FileInputFormat.addInputPath(job, new Path(args[0])); FileOutputFormat.setOutputPath(job,new Path(args[1])); System.exit(job.waitForCompletion(true)?0:1); } }
相关文章推荐
- 【hadoop】Hadoop学习笔记(一)之示例程序:计算每年的最高温度MaxTemperature
- Hadoop学习笔记(一)之示例程序:计算每年的最高温度MaxTemperature
- Hadoop学习笔记(一)之示例程序:计算每年的最高温度MaxTemperature
- 学习笔记(一)之示例程序:计算每年的最高温度MaxTemperature
- hadoop学习笔记-5-最高气温示例MaxTemperature
- hadoop编程入门学习笔记-2 通过示例程序理解hadoop
- HADOOP的学习笔记 (第五期) hadoop示例代码分析 .
- Hadoop学习笔记2--第一个Mapreduce程序
- C# Hadoop学习笔记(七)—C#的云计算框架借鉴(下)
- 云计算学习笔记---Hadoop简介,hadoop实现原理,NoSQL介绍...与传统关系型数据库对应关系,云计算面临的挑战
- Hadoop学习笔记二----运行程序停止不动
- 云计算学习笔记---异常处理---hadoop问题处理ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.lang.NullPoin
- C# Hadoop学习笔记(六)—C#的云计算框架借鉴(上)
- 文件数据云计算学习笔记---Hadoop HDFS和MapReduce 架构浅析
- java tutorial6, netbeans7 示例程序学习笔记
- 计算每年的最高温度MaxTemperature
- PJSIP学习笔记——从simple_pjsua.c示例程序了解PJSUA-LIB的基本使用流程
- j2me学习笔记【1】——helloworld程序示例
- hadoop学习笔记-3-运行wordcount示例
- 计算广告学习笔记2.4 合约广告系统-Hadoop