Hadoop NameNode启动之FSImage、edits重置(三)
2013-08-23 14:24
751 查看
Namenode启动过程中,在FSImage载入后,会对FSImage和edits重置,相当于创建一组新的镜像文件和日志文件,但前提是前面的载入都已经成功了,这个操作紧跟fsImage.recoverTransitionRead之后,具体调用流程如下:
Namenode.main ->NameNode.createNameNode -> NameNode.initialize -> FSNameSystem.initialize-> FSDirecotry.loadFSImage -> fsImage.saveNamespace
这个函数内部流程很清晰,会经历如下步骤:
1、关闭edits日志文件
2、将当前current目录重命名为lastcheckpoint.tmp
3、保存新的image到current目录
4、在current目录中创建edits
5、将第二步中lastcheckpoint.tmp重命名为previous.checkpoint
6、打开fsiamge和edits文件
Namenode.main ->NameNode.createNameNode -> NameNode.initialize -> FSNameSystem.initialize-> FSDirecotry.loadFSImage -> fsImage.saveNamespace
这个函数内部流程很清晰,会经历如下步骤:
1、关闭edits日志文件
2、将当前current目录重命名为lastcheckpoint.tmp
3、保存新的image到current目录
4、在current目录中创建edits
5、将第二步中lastcheckpoint.tmp重命名为previous.checkpoint
6、打开fsiamge和edits文件
void saveNamespace(boolean renewCheckpointTime) throws IOException { //关闭edits日志文件 editLog.close(); if(renewCheckpointTime) this.checkpointTime = FSNamesystem.now(); //将当前current目录重命名为lastcheckpoint.tmp for(Iterator<StorageDirectory> it = dirIterator(); it.hasNext();) { StorageDirectory sd = it.next(); try { //开始重命名操作,实际调用File.renameTo函数 moveCurrent(sd); } catch(IOException ie) { LOG.error("Unable to move current for " + sd.getRoot(), ie); removeStorageDir(sd.getRoot()); } } // save images into current for (Iterator<StorageDirectory> it =dirIterator(NameNodeDirType.IMAGE);it.hasNext();){ StorageDirectory sd = it.next(); try { //在这个函数中会创建current目录,保存新的fsimage、创建新的edits, //注意此时edits是初始化状态,只写入了一个LAYOUT_VERSION //创建VERSION fstime文件 saveCurrent(sd); } catch(IOException ie) { LOG.error("Unable to save image for " + sd.getRoot(), ie); removeStorageDir(sd.getRoot()); } } // -NOTE- // If NN has image-only and edits-onlystorage directories and // fails here // the image will have the latest namespace state. // During startup the image-only directories will recoverby discarding // lastcheckpoint.tmp, while // the edits-only directories will recover by fallingback // to the old state contained in theirlastcheckpoint.tmp. // The edits directories should be discarded duringstartup because their // checkpointTime is older than that of imagedirectories. // recreate edits in current for (Iterator<StorageDirectory> it =dirIterator(NameNodeDirType.EDITS);it.hasNext();) { StorageDirectory sd = it.next(); try { saveCurrent(sd); } catch(IOException ie) { LOG.error("Unable to save edits for " + sd.getRoot(), ie); removeStorageDir(sd.getRoot()); } } //将第二步中lastcheckpoint.tmp重命名为previous.checkpoint for(Iterator<StorageDirectory> it = dirIterator(); it.hasNext();) { StorageDirectory sd = it.next(); try { moveLastCheckpoint(sd); } catch(IOException ie) { LOG.error("Unable to move last checkpoint for" + sd.getRoot(),ie); removeStorageDir(sd.getRoot()); } } if(!editLog.isOpen()) editLog.open(); ckptState = CheckpointStates.UPLOAD_DONE; }
相关文章推荐
- Hadoop NameNode启动之载入FSImage(二)
- Hadoop学习;測试;启动bug;secondary合并edits到fsimage
- hadoop之fsimage和edits工作机制和元数据namenode宕机恢复
- Hadoop NameNode启动之载入FSImage(一)
- Hadoop学习;测试;启动bug;secondary合并edits到fsimage
- hadoop1.2.1 伪分布式搭建后,无法启动namenode(jps中没有namenode),localhost:50070无法访问的解决。
- Hadoop NameNode和SecondaryNameNode通常能在一台机器上启动吗?
- hadoop 无法启动 namenode
- hadoop中namenode无法启动
- hadoop集群namenode启动不了问题
- hadoop namenode启动过程详细剖析及瓶颈分析
- hadoop中启动namenode等出现的一些问题
- [Hadoop异常处理] Namenode和Datanode都正常启动,但是web页面不显示
- 【总结】Hadoop文件系统元数据fsimage和编辑日志edits
- Hadoop HA——namenode无法启动问题解决
- hadoop namenode启动失败
- Hadoop中namenode无法启动
- hadoop中master能够启动datanode,但是datanode无法连接namenode 报 17/11/16 03:49:13 WARN ipc.Client: Failed to conn
- 【hadoop】DataNode,NameNode,JobTracker,TaskTracker用jps查看无法启动解决办法
- 异常:hadoop启动时namenode节点找不到以至于启动不起来