您的位置:首页 > Web前端 > Node.js

Hadoop NameNode启动之FSImage、edits重置(三)

2013-08-23 14:24 751 查看
   Namenode启动过程中,在FSImage载入后,会对FSImage和edits重置,相当于创建一组新的镜像文件和日志文件,但前提是前面的载入都已经成功了,这个操作紧跟fsImage.recoverTransitionRead之后,具体调用流程如下:
Namenode.main ->NameNode.createNameNode -> NameNode.initialize -> FSNameSystem.initialize-> FSDirecotry.loadFSImage -> fsImage.saveNamespace
这个函数内部流程很清晰,会经历如下步骤:
       1、关闭edits日志文件
       2、将当前current目录重命名为lastcheckpoint.tmp
       3、保存新的image到current目录
       4、在current目录中创建edits
       5、将第二步中lastcheckpoint.tmp重命名为previous.checkpoint
       6、打开fsiamge和edits文件

void  saveNamespace(boolean renewCheckpointTime) throws IOException {
//关闭edits日志文件
editLog.close();
if(renewCheckpointTime)
this.checkpointTime = FSNamesystem.now();

//将当前current目录重命名为lastcheckpoint.tmp
for(Iterator<StorageDirectory> it = dirIterator(); it.hasNext();) {
StorageDirectory sd = it.next();
try {
//开始重命名操作,实际调用File.renameTo函数
moveCurrent(sd);
} catch(IOException ie) {
LOG.error("Unable to move current for " + sd.getRoot(), ie);
removeStorageDir(sd.getRoot());
}
}

// save images into current
for (Iterator<StorageDirectory> it =dirIterator(NameNodeDirType.IMAGE);it.hasNext();){
StorageDirectory sd = it.next();
try {
//在这个函数中会创建current目录,保存新的fsimage、创建新的edits,
//注意此时edits是初始化状态,只写入了一个LAYOUT_VERSION
//创建VERSION fstime文件
saveCurrent(sd);
} catch(IOException ie) {
LOG.error("Unable to save image for " + sd.getRoot(), ie);
removeStorageDir(sd.getRoot());
}
}

// -NOTE-
// If NN has image-only and edits-onlystorage directories and
// fails here
// the image will have the latest namespace state.
// During startup the image-only directories will recoverby discarding
// lastcheckpoint.tmp, while
// the edits-only directories will recover by fallingback
// to the old state contained in theirlastcheckpoint.tmp.
// The edits directories should be discarded duringstartup because their
// checkpointTime is older than that of imagedirectories.

// recreate edits in current
for (Iterator<StorageDirectory> it =dirIterator(NameNodeDirType.EDITS);it.hasNext();) {
StorageDirectory sd = it.next();
try {
saveCurrent(sd);
} catch(IOException ie) {
LOG.error("Unable to save edits for " + sd.getRoot(), ie);
removeStorageDir(sd.getRoot());
}
}
//将第二步中lastcheckpoint.tmp重命名为previous.checkpoint

for(Iterator<StorageDirectory> it = dirIterator(); it.hasNext();) {
StorageDirectory sd = it.next();
try {
moveLastCheckpoint(sd);
} catch(IOException ie) {
LOG.error("Unable to move last checkpoint for" + sd.getRoot(),ie);
removeStorageDir(sd.getRoot());
}
}
if(!editLog.isOpen()) editLog.open();
ckptState = CheckpointStates.UPLOAD_DONE;
}
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: