mapreduce的二次排序(字符型)
2016-10-11 10:15
429 查看
import java.io.DataInput; import java.io.DataOutput; import java.io.IOException; import org.apache.hadoop.io.WritableComparable; public class StringPair implements WritableComparable { private String name; private String addr; public void write(DataOutput out) throws IOException { out.writeUTF(name); out.writeUTF(addr); } public void readFields(DataInput in) throws IOException { this.name = in.readUTF(); this.addr = in.readUTF(); } public int compareTo(StringPair o) { if (!this.name.equals(o.name)) { return (int) this.name.compareTo(o.name); } else { return (int) this.addr.compareTo(o.addr); } } public String getName() { return name; } public void setName(String name) { this.name = name; } public String getAddr() { return addr; } public void setAddr(String addr) { this.addr = addr; } } package test; import java.io.IOException; import org.apache.hadoop.conf.Configuration; import org.apache.hadoop.fs.FileSystem; import org.apache.hadoop.fs.Path; import org.apache.hadoop.io.Text; import org.apache.hadoop.io.WritableComparable; import org.apache.hadoop.io.WritableComparator; import org.apache.hadoop.mapreduce.Job; import org.apache.hadoop.mapreduce.Mapper; import org.apache.hadoop.mapreduce.Partitioner; import org.apache.hadoop.mapreduce.Reducer; import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; public class SecondSort { public static class MyPatitioner extends Partitioner{ @Override public int getPartition(StringPair key, Text value, int numPartitions) { return (key.getName().hashCode() & Integer.MAX_VALUE) % numPartitions; } } public static class MyGroup extends WritableComparator{ public MyGroup(){ // 将要比较的数据类型发给WritableComparator父类,因为欧文继承来自WritableComparator // 的原因就是想使用WritableComparator中已经定义的一些特性 super(StringPair.class,true) ; } @SuppressWarnings("rawtypes") @Override public int compare(WritableComparable a, WritableComparable b) { StringPair o1 = (StringPair)a ; StringPair o2 = (StringPair)b ; String first = o1.getName() ; String second = o2.getName() ; return first.compareTo(second) ; } } public static class SSmap extends Mapper
相关文章推荐
- MapReduce 二次排序详解
- MapReduce处理二次排序(分区-排序-分组)
- mapreduce的二次排序 SecondarySort
- MapReduce中的二次排序
- mapreduce的二次排序 SecondarySort
- mapreduce的二次排序 SecondarySort
- Hadoop学习笔记: MapReduce二次排序
- mapreduce二次排序案例
- MapReduce的排序和二次排序
- Hadoop Mapreduce分区、分组、二次排序过程详解
- Hadoop MapReduce 深入理解!二次排序案例!
- Hadoop Mapreduce分区、分组、连接以及辅助排序(也叫二次排序)过程详解
- MapReduce-三次排序-曾经想不通的二次排序
- mapreduce的二次排序 SecondarySort
- Mapreduce的排序、全排序以及二次排序
- mapreduce(三):自定义二次排序流程实例详解
- 十一、理解MapReduce的二次排序功能,包括自定义数据类型、分区、分组、排序
- MapReduce中的二次排序
- MapReduce的二次排序
- Hadoop1.x MapReduce 实现二次排序 实现WritableComparable接口