Hadoop RPC protocol description--转

原文地址：https://spotify.github.io/snakebite/hadoop_rpc.html

Snakebite currently implements the following protocol in snakebite.channel.SocketRpcChannel to communicate with the NameNode.

Connection

The Hadoop RPC protocol works as described below. On connection, headers are sent to setup a session. After that, multiple requests can be sent within the session.

Function	Type	Default
Header	bytes	“hrpc”
Version	uint8	7
Auth method	uint8	80 (Auth method SIMPLE)
Serialization type	uint8	0 (protobuf)
IpcConnectionContextProto length	uint32
IpcConnectionContextProto	bytes

Sending messages

When sending a message, the following is sent to the sever:

Function	Type
Length of the next two parts	uint32
RpcPayloadHeaderProto length	varint
RpcPayloadHeaderProto	protobuf serialized message
HadoopRpcRequestProto length	varint
HadoopRpcRequestProto	protobuf serialized message

varint is a Protocol Buffer variable int.

Note

The Java protobuf implementation uses writeToDelimited to prepend the message with their lenght, but the python implementation doesn’t implement such a method (yet).

Next to an rpcKind (snakebites default is RPC_PROTOCOL_BUFFER), an rpcOp (snakebites default isRPC_FINAL_PAYLOAD), the RpcPayloadHeaderProto message defines a callId that is added in the RPC response (described below).

The HadoopRpcRequestProto contains a methodName field that defines what server method is called and a has a property request that contains the serialized actual request message.

Receiving messages

After a message is sent, the response can be read in the following way:

Function	Type
Length of the RpcResponseHeaderProto	varint
RpcResponseHeaderProto	bytes
Length of the RPC response	uint32
Serialized RPC response	bytes

The RpcResponseHeaderProto contains the callId of the request and a status field. The status can beSUCCESS, ERROR or FAILURE. In case SUCCESS the rest of response is a complete protobuf response.

In case of ERROR, the response looks like follows:

Function	Type
Length of the RpcResponseHeaderProto	varint
RpcResponseHeaderProto	bytes
Length of the RPC response	uint32
Length of the Exeption class name	uint32
Exception class name	utf-8 string
Length of the stack trace	uint32
Stack trace	utf-8 string