Kerberos HBase集群Bulk Load权限问题

我们在使用HBase Bulkload工具进行数据导入时，最后一步会调用LoadIncrementalHFiles的doBulkLoad方法完成HFile move到regionserver的region目录下，但是对于启用Kerberos的HBase集群，就会涉及到严格的权限问题了。

如果你执行时遇到以下报错，那请继续往下看

16/06/02 23:38:11 WARN ipc.CoprocessorRpcChannel: Call failed on IOException
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=10, exceptions:
Thu Jun 02 23:30:11 CST 2016, org.apache.hadoop.hbase.client.RpcRetryingCaller@7bc68525, org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.exceptions.UnknownProtocolException): org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered coprocessor service found for name SecureBulkLoadService in region XXX,f0000000,1464875879413.ee09bfa250de9a508a09b9573b8fc293.
at org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:5579)
at org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3416)
at org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3398)
at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29591)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
at java.lang.Thread.run(Thread.java:744)

HBase对Secure Bulkload提供的解决方法：开启Regionserver SecureBulkLoadEndpoint Coprocessor

具体操作步骤比较简单

1、增加配置到hbase-site.xml

<property>
<name>hbase.bulkload.staging.dir</name>
<value>/hbase/hbase-staging</value>
</property>
<property>
<name>hbase.coprocessor.region.classes</name>
<value>org.apache.hadoop.hbase.security.token.TokenProvider,
org.apache.hadoop.hbase.security.access.AccessController,org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint</value>
</property>

2、设置staging dir权限

hdfs dfs -chown hbase:hbase /hbase/hbase-staging
hdfs dfs -chmod 711 /hbase/hbase-staging

3、重启Regionserver，完毕

现在我们来解析下该coprocessor的原理

1、在HDFS创建属于hbase的staging目录${hbase.bulkload.staging.dir}，默认是/tmp/hbase-staging

2、用户调用Bulkload并输出HFile，如输出到/user/foo/data

3、coprocessor调用hbase创建"secret staging directory"目录：/hbase/hbase-staging/averylongandrandomdirectoryname，权限为(-rwxrwxrwx, 777)

4、用户将/user/foo/data设置为(-rwxrwxrwx, 777)，然后将数据move到上面的random directory目录