HadoopSourceAnalyse --- Nodemanager Container FSM
2013-05-07 17:57
531 查看
overview
上图中还有一个状态变换,从killing 到 Exited_with_failure,这个变换由事件:CONTAINER_EXITED_WITH_FAILURE 触发,执行:ExitedWithFailureTransition;
INIT_CONTAINER And RESOURCE_LOCALIZED Handle
Container 收到该事件通知后,会首先偿试启动所有的和当前App 相关的 AuxServices,for (Map.Entry<String,ByteBuffer> service : csd.entrySet()) { container.dispatcher.getEventHandler().handle( new AuxServicesEvent(AuxServicesEventType.APPLICATION_INIT, ctxt.getUser(), container.container.getId() .getApplicationAttemptId().getApplicationId(), service.getKey().toString(), service.getValue())); }
然年请求所有与当前用户及App相关的 localization resource,然后进入localizing 状态,等待系统加载Resource,当ResourceLocalizer成功加载Resource之后,会向Container发送:RESOURCE_LOCALIZED 事件:
for (ContainerId container : rsrc.ref) { rsrc.dispatcher.getEventHandler().handle( new ContainerResourceLocalizedEvent( container, rsrc.rsrc, rsrc.localPath)); }当所有的Resource都成功加载之后,Container向 ContainersLauncher 发送一个LAUNCH_CONTAINER事件并进入Localized状态, ContainersLauncher将用一个新的线程 去launche container:
Application app = context.getApplications().get( containerId.getApplicationAttemptId().getApplicationId()); ContainerLaunch launch = new ContainerLaunch(getConfig(), dispatcher, exec, app, event.getContainer(), dirsHandler); running.put(containerId, new RunningContainer(containerLauncher.submit(launch), launch));
在新在线程中,先会替换所有的变量(主要是log相关),并用给定的命令生成一份本地执行的脚本文件,奖后向container 发送 LAUNCH_CONTAINER 事件,然后开始执行该脚本,线程会一直等到脚 本执行结束,或被Kill掉,最后返回执行结果。
try { // /////////// Write out the container-script in the nmPrivate space. List<Path> appDirs = new ArrayList<Path>(localDirs.size()); for (String localDir : localDirs) { Path usersdir = new Path(localDir, ContainerLocalizer.USERCACHE); Path userdir = new Path(usersdir, user); Path appsdir = new Path(userdir, ContainerLocalizer.APPCACHE); appDirs.add(new Path(appsdir, appIdStr)); } containerScriptOutStream = lfs.create(nmPrivateContainerScriptPath, EnumSet.of(CREATE, OVERWRITE)); // Set the token location too. environment.put( ApplicationConstants.CONTAINER_TOKEN_FILE_ENV_NAME, new Path(containerWorkDir, FINAL_CONTAINER_TOKENS_FILE).toUri().getPath()); // Sanitize the container's environment sanitizeEnv(environment, containerWorkDir, appDirs, localResources); // Write out the environment writeLaunchEnv(containerScriptOutStream, environment, localResources, launchContext.getCommands()); // /////////// End of writing out container-script // /////////// Write out the container-tokens in the nmPrivate space. tokensOutStream = lfs.create(nmPrivateTokensPath, EnumSet.of(CREATE, OVERWRITE)); Credentials creds = container.getCredentials(); creds.writeTokenStorageToStream(tokensOutStream); // /////////// End of writing out container-tokens } finally { IOUtils.cleanup(LOG, containerScriptOutStream, tokensOutStream); }
发送 LAUNCH_CONAINER,并启动脚 本:
// LaunchContainer is a blocking call. We are here almost means the // container is launched, so send out the event. dispatcher.getEventHandler().handle(new ContainerEvent( containerID, ContainerEventType.CONTAINER_LAUNCHED)); // Check if the container is signalled to be killed. if (!shouldLaunchContainer.compareAndSet(false, true)) { LOG.info("Container " + containerIdStr + " not launched as " + "cleanup already called"); ret = ExitCode.TERMINATED.getExitCode(); } else { exec.activateContainer(containerID, pidFilePath); ret = exec.launchContainer(container, nmPrivateContainerScriptPath, nmPrivateTokensPath, user, appIdStr, containerWorkDir, localDirs, logDirs); }
LAUNCH_CONTAINER Handle
收到该事件之后,通产ContainerMonitor 监 视该container的执行。For other complete event just ignore here;
相关文章推荐
- HadoopSourceAnalyse --- Nodemanager Container FSM
- HadoopSourceAnalyse --- Nodemanager Application FSM
- HadoopSourceAnalyse --- Nodemanager Application FSM
- HadoopSourceAnalyse --- Nodemanager Container request handler
- HadoopSourceAnalyse --- Nodemanager Container request handler
- HadoopSourceAnalyse --- Mapreduce ApplicationMaster Task FSM
- HadoopSourceAnalyse --- Mapreduce ApplicationMaster Job FSM
- HadoopSourceAnalyse --- Mapreduce ApplicationMaster Task FSM
- HadoopSourceAnalyse---RMAppAttempt FSM
- HadoopSourceAnalyse --- Mapreduce ApplicationMaster Job FSM
- HadoopSourceAnalyse --- Mapreduce ApplicationMaster TaskAttempt FSM
- HadoopSourceAnalyse --- Mapreduce ApplicationMaster TaskAttempt FSM
- Hadoop 2.0 中 NameNode/ResourceManager HA 总结
- Hadoop 2.0 中 NameNode/ResourceManager HA 总结
- hadoop2.0安装中遇到的错误:mapreduce.shuffle set in yarn.nodemanager.aux-services is invalid
- Hadoop2.7.1配置NameNode+ResourceManager高可用原理分析
- Hadoop 故障问题 org.apache.hadoop.hdfs.server.namenode.DecommissionManager: Interrupted Monitor
- Hadoop - YARN NodeManager 剖析
- Hadoop2.x NodeManager启动之服务初始化
- 研磨Hadoop源码(三)-yarn-RMContainerTokenSecretManager