python 调用HBase的简单实例
2016-12-18 12:09
573 查看
新来的一个工程师不懂HBase,java不熟,python还行,我建议他那可以考虑用HBase的thrift调用,完成目前的工作。
首先,安装thrift
下载thrift,这里,我用的是thrift-0.7.0-dev.tar.gz 这个版本
tar xzf thrift-0.7.0-dev.tar.gz cd thrift-0.7.0-dev sudo ./configure --with-cpp=no --with-ruby=no sudo make sudo make install
然后,到HBase的源码包里,找到
src/main/resources/org/apache/hadoop/hbase/thrift/
执行
thrift --gen py Hbase.thrift
mv gen-py/hbase/ /usr/lib/python2.4/site-packages/ (根据python版本可能有不同)
我这里写了些调用的脚本,供大家参考
from unittest import TestCase, main from thrift import Thrift from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol from hbase import Hbase from hbase.ttypes import ColumnDescriptor, Mutation, BatchMutation class HBaseTester: def __init__(self, netloc, port, table="staftesttable"): self.tableName = table self.transport = TTransport.TBufferedTransport( TSocket.TSocket(netloc, port)) self.protocol = TBinaryProtocol.TBinaryProtocol(self.transport) self.client = Hbase.Client(self.protocol) self.transport.open() tables = self.client.getTableNames() if self.tableName not in tables: self.__createTable() def __del__(self): self.transport.close() def __createTable(self): name = ColumnDescriptor(name='name') foo = ColumnDescriptor(name='foo') self.client.createTable(self.tableName, [name,foo]) def put(self,key,name,foo): name = Mutation(column="name:v", value=name) foo = Mutation(column="foo:v",value=foo) self.client.mutateRow(self.tablename,key,[name,foo]) def scanner(self,column): scanner = client.scannerOpen(self.tablename,"",[column]) r = client.scannerGet(scanner) result= [] while r: print r[0] result.append(r[0]) r = client.scannerGet(scanner) print "Scanner finished" return result class TestHBaseTester(TestCase): def setUp(self): self.writer = HBaseTester("localhost", 9090) def tearDown(self): name = self.writer.tableName client = self.writer.client client.disableTable(name) client.deleteTable(name) def testCreate(self): tableName = self.writer.tableName client = self.writer.client self.assertTrue(self.writer.tableName in client.getTableNames()) columns =['name:','foo:'] for i in client.getColumnDescriptors(tableName): self.assertTrue(i in columns) def testPut(self): self.writer.put("r1","n1","f1") self.writer.put("r2","n2","f2") self.writer.put("r3","n3","") self.writer.scanner("name:") if __name__ == "__main__": main()
以上这篇python 调用HBase的简单实例就是小编分享给大家的全部内容了,希望能给大家一个参考,也希望大家多多支持脚本之家。
您可能感兴趣的文章:
相关文章推荐
- Python动态类型的学习---引用的理解
- Python3写爬虫(四)多线程实现数据爬取
- 垃圾邮件过滤器 python简单实现
- 下载并遍历 names.txt 文件,输出长度最长的回文人名。
- install and upgrade scrapy
- Scrapy的架构介绍
- Centos6 编译安装Python
- 使用Python生成Excel格式的图片
- 让Python文件也可以当bat文件运行
- [Python]推算数独
- Python中zip()函数用法举例
- Python中map()函数浅析
- Python将excel导入到mysql中
- Facebook's New Real-time Messaging System: HBase to Store 135+ Billion Messages a Month
- Python在CAM软件Genesis2000中的应用
- 使用Shiboken为C++和Qt库创建Python绑定
- FREEBASIC 编译可被python调用的dll函数示例