Hadoop自定义Counter
2013-09-11 20:00
204 查看
1.通过enum自定义Counter
public static num LOG_PROCESSOR_COUNTER {
BAD_RECORDS
};
2.在Mapper或者Reducer中操作Counter
context.getCounter(LOG_PROCESSOR_COUNTER.BAD_RECORDS).
increment(1);
3.在Job完成后可以进行最终统计结果的输出
Job job = new Job(getConf(), "log-analysis");
……
Counters counters = job.getCounters();
Counter badRecordsCounter = counters.findCounter(
LOG_PROCESSOR_COUNTER.BAD_RECORDS);
System.out.println("# of Bad Records:"+
badRecordsCounter.getValue());
public static num LOG_PROCESSOR_COUNTER {
BAD_RECORDS
};
2.在Mapper或者Reducer中操作Counter
context.getCounter(LOG_PROCESSOR_COUNTER.BAD_RECORDS).
increment(1);
3.在Job完成后可以进行最终统计结果的输出
Job job = new Job(getConf(), "log-analysis");
……
Counters counters = job.getCounters();
Counter badRecordsCounter = counters.findCounter(
LOG_PROCESSOR_COUNTER.BAD_RECORDS);
System.out.println("# of Bad Records:"+
badRecordsCounter.getValue());
相关文章推荐
- hadoop counter介绍和自定义counter
- Hadoop自定义Counter
- [Hadoop] - Mapreduce自定义Counter
- hadoop自定义counter简单java Demo
- Hadoop自定义计数器Counter
- hadoop学习;自定义Input/OutputFormat;类引用mapreduce.mapper;三种模式
- 利用hadoop命令rcc生成Record 一种简单的方式实现自定义的writable对象
- hadoop自定义数据类型
- Hadoop 自定义InputFormat实现自定义Split
- hadoop 添加自定义日志
- Hadoop自定义数据类型和输入格式
- hadoop自定义输入格式(InputFormat)
- hadoop 几个默认counter的含义及收集原理
- 使用Hive时提示org.apache.hadoop.metrics.jvm.EventCounter is deprecated警告的解决方案
- c#实现自定义window performance counter
- 在hadoop作业中自定义分区和归约
- hadoop 自定义inputformat和outputformat
- Hadoop中自定义计数器
- Hadoop中自定义计数器
- 在Hadoop的streaming中使用自定义的inputformat和outputformat