byte[]与String转换引起的protobuf反序列化抛异常问题

使用protobuf时，将string写入ssdb然后再读出来反序列化protobuf message的时候报错：

com.google.protobuf.InvalidProtocolBufferException: While parsing a protocol message, the input ended unexpectedly in the middle of a field.  This could mean either than the input has been truncated or that an embedded message misreported its own length.
at com.google.protobuf.InvalidProtocolBufferException.truncatedMessage(InvalidProtocolBufferException.java:70)
at com.google.protobuf.CodedInputStream.readRawBytes(CodedInputStream.java:789)
at com.google.protobuf.CodedInputStream.readBytes(CodedInputStream.java:329)
at com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:484)
at com.google.protobuf.UnknownFieldSet$Builder.mergeFrom(UnknownFieldSet.java:461)
at com.google.protobuf.UnknownFieldSet$Builder.mergeFrom(UnknownFieldSet.java:579)
at com.google.protobuf.UnknownFieldSet$Builder.mergeFrom(UnknownFieldSet.java:280)
at com.google.protobuf.CodedInputStream.readGroup(CodedInputStream.java:240)
at com.google.protobuf.UnknownFieldSet$Builder.mergeFieldFrom(UnknownFieldSet.java:488)
at com.google.protobuf.GeneratedMessage.parseUnknownField(GeneratedMessage.java:193)

代码如下：

Msg.Builder msg1 =  Msg.newBuilder();
//set value
msg1.setId(11);
byte[] data = msg1.build().toByteArray();
//写入ssdb
ssdb.set(key,data);
//立刻get出来
String str = ssdb.get(key);

Msg.Builder msg2 = Msg.newBuilder();
try {
msg2.mergeFrom(str.getBytes());
} catch (InvalidProtocolBufferException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return;
}
logger.info("msg===={}",msg2);

很诡异，只是把数据写入再读出来怎么会抛

InvalidProtocolBufferException

异常。。。。

加个日志定位，把写入前后的字符串长度打印出来：

Msg.Builder msg1 =  Msg.newBuilder();
//set value
msg1.setId(11);
byte[] data = msg1.build().toByteArray();

//写入前的字符串长度
System.out.println("==============before length="+data.length);

//写入ssdb
ssdb.set(key,data);
//立刻get出来
String str = ssdb.get(key);

//读出来的字符串长度
System.out.println("==============after length="+str.length());

Msg.Builder msg2 = Msg.newBuilder();
try {
msg2.mergeFrom(str.getBytes());
} catch (InvalidProtocolBufferException e) {
// TODO Auto-generated catch block
e.printStackTrace();
return;
}
logger.info("msg===={}",msg2);

执行结果：

==============before length=9977
==============after length=9963

纳尼，前后的长度居然不一样！！！！

通过各种折腾发现是string和byte转换的问题，因为protobuf序列化后是二进制的数据流，不能转换成string类型，String类型是带编码的，如UTF-8

所以一个保存二进制数据的byte[]转为string，再转回byte[]就会和原来不一样。

解决办法1:使用iso8859-1编码

public static void testByte2String() {
byte bytes[] = new byte[] { 50, 0, -1, 28, -24 };
String string = new String(bytes);
byte[] ret = string.getBytes();

String isoString;
byte[] isoret = null;
try {
isoString = new String(bytes, "ISO-8859-1");
isoret = isoString.getBytes("ISO-8859-1");
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
}

System.out.println(System.getProperty("file.encoding"));
System.out.println(ret);
System.out.println(isoret);
}

解决办法2：使用base64编码传输

对于二进制数据最好的办法是采用base64编码成可打印字符串进行传输

参考：

- http://stackoverflow.com/questions/16270994/difference-between-string-length-and-string-getbytes-length

- http://blog.csdn.net/hjxgood/article/details/20057989