mapreduce编程:求平均值
2015-02-12 16:08
281 查看
求平均值的程序:
输入:
输出:
package my.hadoopstudy; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.Text; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce.Reducer; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; import org.apache.hadoop.util.GenericOptionsParser; import java.io.IOException; public class AvgCompute { public static class MyMapper extends Mapper<Object, Text, Text, Text>{ public void map(Object key, Text value, Context context) throws IOException, InterruptedException { String[] e = value.toString().split(" ",2); context.write(new Text(e[0]),new Text(e[1])); } } public static class MyReducer extends Reducer<Text,Text,Text,Text> { public void reduce(Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException { float sum = 0; float count=0; for (Text val : values) { sum += Float.parseFloat(val.toString()); count +=1.0f; } sum=sum/count; context.write(key,new Text(String.valueOf(sum))); } } public static void main(String[] args) throws Exception { Configuration conf = new Configuration(); String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs(); if (otherArgs.length < 2) { System.err.println("Usage: EventCount <in> <out>"); System.exit(2); } Job job = Job.getInstance(conf, "avg compute"); job.setJarByClass(AvgCompute.class); job.setMapperClass(MyMapper.class); job.setCombinerClass(MyReducer.class); job.setReducerClass(MyReducer.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(Text.class); FileInputFormat.addInputPath(job, new Path(otherArgs[0])); FileOutputFormat.setOutputPath(job, new Path(otherArgs[1])); System.exit(job.waitForCompletion(true) ? 0 : 1); } }
输入:
输出:
相关文章推荐
- mapreduce编程实例(3)-求平均值
- MapReduce编程基础(二)——数值概要(计算最大值、最小值、平均值)
- MapReduce 编程 系列五 MapReduce 主要过程梳理
- 利用MapReduce的java编程接口完成数据的统计
- (五)MapReduce编程实例
- 2、随机生成一个三维数组,编程求深度的平均值,保存在一个二维数组中。
- Hadoop ->> MapReduce编程模型
- MapReduce(四) 典型编程场景(二)
- mapreduce编程模型学习--1
- Hadoop系列之三:函数式编程语言和MapReduce
- MapReduce 编程之 倒排索引
- MapReduce编程实战之“调试”
- MapReduce编程模型:用MapReduce进行大数据分析
- mapreduce编程实例(2)-求最大值和最小值
- MapReduce 2.0编程实践(涉及多语言编程)
- MapReduce编程实现txt文件中的内容导入HBase
- mapreduce编程--(准备篇)
- Hbase编程入门之MapReduce
- MapReduce编程基础(二)——数值概要(计算中位数、标准差)[内存优化]
- Hadoop MapReduce编程 API入门系列之分区和合并(十四)