Ubuntu下Python访问Hive——thrift安装及测试

1、下载thrift：

wget http://mirror.bjtu.edu.cn/apache//thrift/0.8.0/thrift-0.9.0.tar.gz

2、安装依赖：

sudo apt-get install build-essential

sudo apt-get install bison flex

sudo apt-get install libboost-dev python-dev

sudo apt-get install autoconf automake libtool pkg-config

sudo apt-get install git

3、安装thrift：

tar -xzvf thrift-0.9.0.tar.gz

cd thrift-0.9.0

sudo ./configure --prefix=/host/java/thrift

sudo make && sudo make install

4、启动Hive Thrift Server (>>hive --service hiveserver)

要先启动hadoop 设置hdfs安全模式为leave

>>start-all.sh

>>hadoop dfsadmin -safemode leave

启动时报错……没关系，运行python时候才会再有反应

Starting Hive Thrift Server
13/07/29 13:24:13 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.

解决办法：

因为在0.10里面，hive.metastore.local参数已经取消，都使用hive.metastore.uris参数，如果hive.metastore.uris为空，就默认为local模式。

把hive.metastore.local参数删掉就好了。

5、测试代码编写如下：

testHive.py:

#!/usr/bin/env python
import sys
sys.path.append('/usr/local/hadoop/hive/lib/py')#'/usr/local/hadoop/hive是你的hive安装路径
from hive_service import ThriftHive
from hive_service.ttypes import HiveServerException
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

def hiveExe(sql):

try:
transport = TSocket.TSocket('127.0.0.1', 10000)
transport = TTransport.TBufferedTransport(transport)
protocol = TBinaryProtocol.TBinaryProtocol(transport)
client = ThriftHive.Client(protocol)
transport.open()

client.execute(sql)

print "The return value is : "
print client.fetchAll()
print "............"
transport.close()
except Thrift.TException, tx:
print '%s' % (tx.message)

if __name__ == '__main__':
hiveExe("select * from testParttion")#t_afan_test是你的表名
6、执行
>>python testHive.py
7、结果

The return value is:
['[12,23,23,34]\t["what","are","this"]','[34,45,34,23,12]\t["who","am","i","are"]']