Hadoop自定义Counter
2013-09-10 22:22
232 查看
1.通过enum自定义Counter
public static num LOG_PROCESSOR_COUNTER {
BAD_RECORDS
};
2.在Mapper或者Reducer中操作Counter
context.getCounter(LOG_PROCESSOR_COUNTER.BAD_RECORDS).
increment(1);
3.在Job完成后可以进行最终统计结果的输出
Job job = new Job(getConf(), "log-analysis");
……
Counters counters = job.getCounters();
Counter badRecordsCounter = counters.findCounter(
LOG_PROCESSOR_COUNTER.BAD_RECORDS);
System.out.println("# of Bad Records:"+
badRecordsCounter.getValue());
public static num LOG_PROCESSOR_COUNTER {
BAD_RECORDS
};
2.在Mapper或者Reducer中操作Counter
context.getCounter(LOG_PROCESSOR_COUNTER.BAD_RECORDS).
increment(1);
3.在Job完成后可以进行最终统计结果的输出
Job job = new Job(getConf(), "log-analysis");
……
Counters counters = job.getCounters();
Counter badRecordsCounter = counters.findCounter(
LOG_PROCESSOR_COUNTER.BAD_RECORDS);
System.out.println("# of Bad Records:"+
badRecordsCounter.getValue());
相关文章推荐
- [Hadoop] - Mapreduce自定义Counter
- hadoop自定义counter简单java Demo
- Hadoop自定义计数器Counter
- Hadoop自定义Counter
- hadoop counter介绍和自定义counter
- Hadoop Map Reduce 限制counter的默认数量120
- Hadoop学习笔记—8.Combiner与自定义Combiner
- Hadoop实战-Flume之自定义Source(十八)
- Hadoop & Hadoop Streaming 自定义输出格式
- Hadoop Pig、Hive 自定义输入输出分隔符
- [转]自定义hadoop map/reduce输入文件切割InputFormat
- Hadoop自定义类型处理手机上网日志(笔记5)
- hadoop2 中log4j日志详解及自定义
- Hadoop入门案例(四)全排序之自定义分区 字符串(单词)排序
- Hadoop Oozie 自定义安装和启动
- Hadoop中MapReduce中的counter的意义
- 自定义hadoop的InputFormat
- hadoop自定义文件的输入格式
- Hadoop(11) 自定义Writable
- hadoop的自定义数据类型和与关系型数据库交互