Python 使用thrift连接hbase || 远程连接hbase
2015-10-13 15:53
1216 查看
1.首先下载Python & thrift &hbase
有两种安装thrift的方式:1.下载的thrift-0.9.3.tar.gz >> 解压 tar xzvf thrift-0.9.3.tar.gz >> ./configure >> make >> make install >>thrift -version 验证2.解压之后进入lib/py/ >> 执行Python setup.py install 安装 。这种方式不需要软连接到python的site-packages/ >> python >> import thrift 验证。
2.解压hbase:
tar xzvf hbase-1.1.2.tar.gz >> cd hbase-thrift >> thrift -gen py /root/zhutong/hbase-1.1.2/hbase-thrift/src/main/resources/org/apache/hadoop/hbase/thrift 会生成一个 gen-py的文件 >> cd gen-py >> cp -R hbase/usr/local/python2.7/lib/python2.7/site-packages/3.启动thrift:
单机模式启动:首先启动hbase >>cd bin >> ./start-hbase.sh >> 启动thrift ./hbase-daemon.sh start thrift # 默认的端口是9090配置hbase的rootdir
<property> <name>hbase.rootdir</name> <value>/root/zhutong/hbase_data</value> </property>
环境配置好了 现在准备测试:
编写test.py 文件
#!/usr/bin/python #-*- coding:utf-8 -*- from thrift import Thrift from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol from thrift.protocol import TCompactProtocol from hbase import Hbase from hbase.ttypes import * #192.168.110.78:8020 transport1 = TSocket.TSocket('localhost',9090 ) transport = TTransport.TBufferedTransport(transport1) protocol = TCompactProtocol.TCompactProtocol(transport); client = Hbase.Client(protocol) transport.open() contents=ColumnDescriptor(name='cf:',maxVersions=1) tablename='ta' client.createTable(tablename,[contents]) print client.getTableNames()
异常:如果报错找不到thrift模块 ,查看Python安装目录site-packages下没有thrift*.egg 文件 ,使用easy-install 或者第一步中第二种方式安装。
如果提示主机名错误异常;请前去/etc/hosts文件中添加映射关系。
成功之后再对hbase进行操作;
#!/usr/bin/python #insert data from thrift import Thrift from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol from hbase import Hbase from hbase.ttypes import * transport = TSocket.TSocket('localhost', 9090) transport = TTransport.TBufferedTransport(transport) protocol = TBinaryProtocol.TBinaryProtocol(transport) client = Hbase.Client(protocol) transport.open() for i in range(10): row = 'row-key1'+str(i) mutations = [Mutation(column="cf:a", value=str(i))] client.mutateRow('ta', row, mutations, None)
#!/usr/bin/python from thrift import Thrift from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol from hbase import Hbase from hbase.ttypes import * transport = TSocket.TSocket('localhost', 9090) transport = TTransport.TBufferedTransport(transport) protocol = TBinaryProtocol.TBinaryProtocol(transport) client = Hbase.Client(protocol) transport.open() tableName = 'ta' rowKey = 'row-key1' result = client.getRow(tableName, rowKey, None) print result print type(result) for r in result: print 'the row is ' , r.row print 'the values is ' , r.columns.get('cf:a').value
#!/usr/bin/python # get some data from thrift import Thrift from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol from hbase import Hbase from hbase.ttypes import * transport = TSocket.TSocket('localhost', 9090) transport = TTransport.TBufferedTransport(transport) protocol = TBinaryProtocol.TBinaryProtocol(transport) client = Hbase.Client(protocol) transport.open() scan = TScan() tableName = 'ta' id = client.scannerOpenWithScan(tableName, scan, None) result2 = client.scannerGetList(id, 10) print result2 print type(result2) for i in result2: print i.row
4.远程连接hbase
修改Python文件的IP地址和端口进行连接会出现如下的错误:或者:
这是因为远程访问的hbase也是需要thrift来进行访问的,所以需要在访问的hbase服务器上启动thrift:
nohup hbase thrift -p 9999 start & 启动 >> ps aux | grep thrift 验证
然后再执行 : 端口号为你设置的端口号;
相关文章推荐
- python自动化执行脚本
- 搭建Python开发环境
- python学习002
- 举例讲解Python中的身份运算符的使用方法
- poker 游戏实现 (python)
- python module getopt usage
- python学习001
- python装饰器学习笔记
- 如何用Python输出漂亮的xml文件
- 使用python + tornado 做项目demo演示模板
- Python中的字典与成员运算符初步探究
- [python] 多线程编程
- 详解Python中的元组与逻辑运算符
- 项目部署 python fabric
- python - PyDev统一编码
- python02-new环境搭建
- 5.Python标准库_存储对象 (pickle包,cPickle包)
- windows ipython 出现ImportError: No module named XXXX错误
- Note of "Learn Python the HardWay"
- Python递归查找文件(os.walk, os.path, fnmatch)