Hadoop 之 WritableComparable 接口
2015-07-10 21:30
375 查看
Writable接口:
它是一个实现了序列化协议的序列化对象。在Hadoop中定义一个结构化对象都要实现Writable接口,使得该结构化对象可以序列化为字节流,字节流也可以反序列化为结构化对象。
WritableComparable接口
WritableComparable接口是可序列化并且可比较的接口。MapReduce中所有的key值类型都必须实现这个接口,既然是可序列化的那就必须得实现readFiels()和write()这两个序列化和反序列化函数,既然也是可比较的那就必须得实现compareTo()函数,该函数即是比较和排序规则的实现。这样MR中的key值就既能可序列化又是可比较的。
一个实现了该接口的对象的列子:
A Writable which is also Comparable.
WritableComparables can be compared to each other, typically via Comparators. Any type which is to be used as a key in the Hadoop Map-Reduce framework should implement this interface.
Example:
它是一个实现了序列化协议的序列化对象。在Hadoop中定义一个结构化对象都要实现Writable接口,使得该结构化对象可以序列化为字节流,字节流也可以反序列化为结构化对象。
WritableComparable接口
WritableComparable接口是可序列化并且可比较的接口。MapReduce中所有的key值类型都必须实现这个接口,既然是可序列化的那就必须得实现readFiels()和write()这两个序列化和反序列化函数,既然也是可比较的那就必须得实现compareTo()函数,该函数即是比较和排序规则的实现。这样MR中的key值就既能可序列化又是可比较的。
一个实现了该接口的对象的列子:
public interface WritableComparable<T> extends Writable,Comparable<T>
A Writable which is also Comparable.
WritableComparables can be compared to each other, typically via Comparators. Any type which is to be used as a key in the Hadoop Map-Reduce framework should implement this interface.
Example:
static class NewK2 implements WritableComparable<NewK2>{ Long first; Long second; public NewK2(){} public NewK2(long first, long second){ this.first = first; this.second = second; } @Override public void readFields(DataInput in) throws IOException { this.first = in.readLong(); this.second = in.readLong(); } @Override public void write(DataOutput out) throws IOException { out.writeLong(first); out.writeLong(second); } /** * 当k2进行排序时,会调用该方法. * 当第一列不同时,升序;当第一列相同时,第二列升序 */ @Override public int compareTo(NewK2 o) { final long minus = this.first - o.first; if(minus !=0){ return (int)minus; } return (int)(this.second - o.second); } @Override public int hashCode() { return this.first.hashCode()+this.second.hashCode(); } @Override public boolean equals(Object obj) { if(!(obj instanceof NewK2)){ return false; } NewK2 oK2 = (NewK2)obj; return (this.first==oK2.first) &&(this.second==oK2.second); } }
相关文章推荐
- Linux下突然不识别无线网卡
- Ubuntu下整合PHP和nginx
- 开发第一个OpenDaylight中的OSGI bundle
- Eclipse正常启动tomcat但无法访问主页
- Apache 2.4虚拟主机配置
- Bash 快捷键
- 深入理解Linux内核-内存寻址
- [Solved] MATLAB Linux install error: /tmp/mathworks_15227/java/jre/glnxa64/jre/bin/java: Permission
- runLoop和runtime的分析
- 奔跑中的2015——运维在路上
- openrefactory/c笔记 (一)
- Linux : chkconfig
- JDK+MyEclipse+Tomcat的配置(修改Tomcat 6.x的端口)
- centos7 lnmp 一键安装脚本
- Linux的SOCKET编程详解
- no installed service named "apache2"
- Nginx源代码分析之I/O模型细节(七)
- 警告: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property ...
- linux用户权限管理, chmod, ln
- 配置openssh服务器(学习整理)