hama程序出现Messages must never be behind the vertex in ID问题解决
2013-11-05 11:05
459 查看
在运行hama程序时,在partition完成后,经常出现java.lang.IllegalArgumentException: Messages must never be behind the vertex in
ID! Current Message ID: 100003 vs. 100004。异常
看了一下hama源码,如下:
/**
* Iterating utility that ensures following things: <br/>
* - if vertex is active, but the given message does not match the vertexID,
* return null. <br/>
* - if vertex is inactive, but received a message that matches the ID, build
* an iterator that can be iterated until the next vertex has been reached
* (not buffer in memory) and set the vertex active <br/>
* - if vertex is active, and the given message does match the vertexID,
* return an iterator that can be iterated until the next vertex has been
* reached. <br/>
* - if vertex is inactive, and received no message, return null.
*/
@SuppressWarnings("unchecked")
private VertexMessageIterable<V, M> iterate(GraphJobMessage currentMessage,
V firstMessageId, Vertex<V, E, M> vertex,
BSPPeer<Writable, Writable, Writable, Writable, GraphJobMessage> peer) {
int comparision = firstMessageId.compareTo(vertex.getVertexID());
if (conf.getBoolean("hama.check.missing.vertex", true)) {
if (comparision < 0) {
throw new IllegalArgumentException(
"Messages must never be behind the vertex in ID! Current Message ID: "
+ firstMessageId + " vs. " + vertex.getVertexID());
}
} else {
while (comparision < 0) {
VertexMessageIterable<V, M> messageIterable = new VertexMessageIterable<V, M>(currentMessage,
firstMessageId, peer);
currentMessage = messageIterable.getOverflowMessage();
firstMessageId = (V)currentMessage.getVertexId();
comparision = firstMessageId.compareTo(vertex.getVertexID());
}
}
if (comparision == 0) {
// vertex id matches with the vertex, return an iterator with newest
// message
return new VertexMessageIterable<V, M>(currentMessage,
vertex.getVertexID(), peer);
} else {
// return null
return null;
}
}
问题出在传递的消息没有找到对应的节点,是自己输入的文件格式有问题。Hama中如果使用Vertex类,则输入的默认格式是(点、边<起点
终点>)。与mapreduce类似,mapper类为文件的每一行调用一个map()函数,Vertex类为每一个点调用一个compute()函数。该点名作为一个peer的名字。所以在边的输入文件里的<起点
终点>也要在点的位置出现,如果没有出现就会出现上面的错误。举个例子:如果输入文件如下就会出错(边只写终点):
1 2 3 17
2 1
3 1
图形结构如下
![](http://img.blog.csdn.net/20131105110330640?watermark/2/text/aHR0cDovL2Jsb2cuY3Nkbi5uZXQvcGFyYWRpc2VfZmNo/font/5a6L5L2T/fontsize/400/fill/I0JBQkFCMA==/dissolve/70/gravity/Center)
17在边的位置出现了,但在点的位置没有出现。消息传递不到17的点,所以抛出异常。输入文件改为
1 2 3 17
2 1
3 1
17 1
就可以正常运行了。
ID! Current Message ID: 100003 vs. 100004。异常
看了一下hama源码,如下:
/**
* Iterating utility that ensures following things: <br/>
* - if vertex is active, but the given message does not match the vertexID,
* return null. <br/>
* - if vertex is inactive, but received a message that matches the ID, build
* an iterator that can be iterated until the next vertex has been reached
* (not buffer in memory) and set the vertex active <br/>
* - if vertex is active, and the given message does match the vertexID,
* return an iterator that can be iterated until the next vertex has been
* reached. <br/>
* - if vertex is inactive, and received no message, return null.
*/
@SuppressWarnings("unchecked")
private VertexMessageIterable<V, M> iterate(GraphJobMessage currentMessage,
V firstMessageId, Vertex<V, E, M> vertex,
BSPPeer<Writable, Writable, Writable, Writable, GraphJobMessage> peer) {
int comparision = firstMessageId.compareTo(vertex.getVertexID());
if (conf.getBoolean("hama.check.missing.vertex", true)) {
if (comparision < 0) {
throw new IllegalArgumentException(
"Messages must never be behind the vertex in ID! Current Message ID: "
+ firstMessageId + " vs. " + vertex.getVertexID());
}
} else {
while (comparision < 0) {
VertexMessageIterable<V, M> messageIterable = new VertexMessageIterable<V, M>(currentMessage,
firstMessageId, peer);
currentMessage = messageIterable.getOverflowMessage();
firstMessageId = (V)currentMessage.getVertexId();
comparision = firstMessageId.compareTo(vertex.getVertexID());
}
}
if (comparision == 0) {
// vertex id matches with the vertex, return an iterator with newest
// message
return new VertexMessageIterable<V, M>(currentMessage,
vertex.getVertexID(), peer);
} else {
// return null
return null;
}
}
问题出在传递的消息没有找到对应的节点,是自己输入的文件格式有问题。Hama中如果使用Vertex类,则输入的默认格式是(点、边<起点
终点>)。与mapreduce类似,mapper类为文件的每一行调用一个map()函数,Vertex类为每一个点调用一个compute()函数。该点名作为一个peer的名字。所以在边的输入文件里的<起点
终点>也要在点的位置出现,如果没有出现就会出现上面的错误。举个例子:如果输入文件如下就会出错(边只写终点):
1 2 3 17
2 1
3 1
图形结构如下
17在边的位置出现了,但在点的位置没有出现。消息传递不到17的点,所以抛出异常。输入文件改为
1 2 3 17
2 1
3 1
17 1
就可以正常运行了。
相关文章推荐
- 【转】断电或强制退出后VMware虚拟机出现The virtual machine appears to be in use无法启动问题解决
- The description for Event ID ( 27745 ) in Source ( Windows SharePoint Services 3 ) cannot be found问题解决
- 断电或强制退出后VMware虚拟机出现The virtual machine appears to be in use无法启动问题解决
- MPI编译C++程序出现#error "SEEK_SET is #defined but must not be for the C++ binding of MPI" 的解决方法
- ISAPI程序出现“The specified module could not be found”问题解决方法
- 解决问题:The icon file must be 57x57 pixels, in .png format (-19014)
- MPI编译C++程序出现#error "SEEK_SET is #defined but must not be for the C++ binding of MPI" 的解决方法
- 断电或强制退出后VMware虚拟机出现The virtual machine appears to be in use无法启动问题解决
- 问题解决: Attribute value request.getParameter("id") is quoted with " which must be escaped when used within the value
- Hbase运行时出现的问题(There could be a mismatch with the one configured in the master.)及其解决方法:
- mac上 go-delve 安装出现The specified item could not be found in the keychain 解决方法
- 在Eclipse中编写servlet时出现"The import javax.servlet cannot be resolved" 问题解决办法
- android4.1 JELLY_BEAN:All WebView methods must be called on the same thread[问题已解决]
- In-App Purch - Your first In-App Purchase(s) must be submitted with a new app version...问题解决方案
- 导入项目出现a cycle was detected in the build path of project问题解决
- matlab运行出现错误For floats all values in I must be smaller than 1.解决方法
- 解决struts2中UI标签出现的问题: The Struts dispatcher cannot be found
- 有关Constructor call must be the first statement in a constructor的问题
- WPF部署问题 解决:The application requires that the assembly...be installed in the GAC
- Nested exception: The entity name must immediately follow the '&' in the entity reference.问题的解决