jobhistory中的job信息获取与分析
2015-10-22 20:19
239 查看
YARN源码学习(六)-----JobHistory中的job信息获取与分析
前言
继续延续上一篇文章的主题,2个字,监控,分布式系统要想做到足够大,足够强,足够稳定,首先需要做好的就是其中的监控.现在开源的分布式系统很多,YARN就是其中一种,比较值得庆幸的一点是,Yarn已经在Ganglia做了很多指标的监控分析.比如namenode rpc请求数,datanode写入字节数,读字节数,jvm相关的gc次数等等.但是看似这些指标非常的完美,其实不然,为什么这么说呢,因为粒度太粗,比如说下面这个场景,我想分析集群中特点节点机器上哪个task异常,导致拖垮整个集群的运作效率.这个时候,显然分析Ganglia上的粗粒度监控指标就不能解决这样的场景问题了吧.不过还好,Yarn提供了这样的额外服务,叫做JobHistory,他也是一项独立的服务.
什么是JobHistory
什么是JobHistory,jobHistory翻译成中文就是作业历史,就是作业历史记录.就是保存了集群运行过的历史Job信息数据.下面是一张此服务的Web UI视图:
可以很清楚的看到了上面执行过的job记录.因为是我测试是跑的几个word-count程序,所以信息比较少.当然每个job记录的链接还能往里继续点,里面保存了更加详细的task的运行信息,包括map数,reduce数,开始结束时间等等,如下图
JobHistory上所展示的数据是非常多的,但是唯一感到不足的是,JobHistory的展示效果太过单一,每个Job的数据结果都是独立展现的,并没有一个汇总的页面,不便于比较分析.所以一个比较大胆的想法就诞生了,我们是不是可以拿到Job的信息记录,存入自己的db,然后自己做分析呢.OK,想法固然不错,但是还是得从源码中进行分析,首先要明白这些数据到底存在哪.
JobHistory作业数据存储
下面来描述一下我是如何分析发现JobHistory作业数据的存储源的.首先定位到JobHistory这个大类.<span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">/**
* Loads and manages the Job history cache.
*/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">class</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobHistory</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">extends</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">AbstractService</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">implements</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryContext</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">private</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">static</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">final</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Log</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> LOG </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">LogFactory</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getLog</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobHistory</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">class</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">static</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">final</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Pattern</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> CONF_FILENAME_REGEX </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Pattern</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">compile</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"("</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobID</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">JOBID_REGEX </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">")_conf.xml(?:\\.[0-9]+\\.old)?"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">static</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">final</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">String</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> OLD_SUFFIX </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">".old"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">// Time interval for the move thread.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">private</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">long</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> moveThreadInterval</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">private</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Configuration</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">private</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">ScheduledThreadPoolExecutor</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> scheduledExecutor </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">null</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">//注意下面这2个类的名称,显然与存储信息相关</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">private</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryStorage</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> storage </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">null</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">private</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryFileManager</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> hsManager </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">null</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">ScheduledFuture</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><?></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> futureHistoryCleaner </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">null</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">...</span>从这里就可以看出来,JobHistory也是一项服务.关注到上面的倒数3行有与存储相关的类,我们可以重点关注这2个变量.然后扫描JobHistory的内部方法,你应该会发现有下面这样的方法
<span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">@Override</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Map</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> getAllJobs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">return</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> storage</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getAllPartialJobs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>这个方法的任务就是获取所有的job信息存入map中,然后继续跟踪这行代码,看看他的具体实现.但是得要先明白storage是什么类.在服务初始化方法中,会存在初始构造的过程<span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">@Override</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">protected</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">void</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> serviceInit</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Configuration</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throws</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Exception</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
LOG</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">info</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"JobHistory Init"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.....</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
hsManager </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> createHistoryFileManager</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
hsManager</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">init</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">try</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
hsManager</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">initExisting</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">catch</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throw</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">YarnRuntimeException</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"Failed to intialize existing directories"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
storage </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> createHistoryStorage</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">if</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">storage </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">instanceof</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Service</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">((</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Service</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> storage</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">).</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">init</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
storage</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">setHistoryFileManager</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hsManager</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">super</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">serviceInit</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>在这里可以看到,historyStorage的使用需要hsManager的协助.在构造historyStorage的构造操作中,是执行了下面的方法<span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">protected</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryStorage</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> createHistoryStorage</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">return</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">ReflectionUtils</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">newInstance</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getClass</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JHAdminConfig</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">MR_HISTORY_STORAGE</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">CachedHistoryStorage</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">class</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryStorage</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">class</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">),</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>说明具体实现子类是CacheHistoryStorage类,getJob的方法是下面的方法实现<span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">@Override</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Map</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> getAllPartialJobs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
LOG</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">debug</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"Called getAllPartialJobs()"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">SortedMap</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> result </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">TreeMap</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">>();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">try</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">for</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryFileInfo</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> mi </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> hsManager</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getAllFileInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">())</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">if</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mi </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">!=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">null</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobId</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> id </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> mi</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getJobId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
result</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">put</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">id</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">PartialJob</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mi</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getJobIndexInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(),</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> id</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">));</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">catch</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
LOG</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">warn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"Error trying to scan for all FileInfos"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throw</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">YarnRuntimeException</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">return</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> result</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>这里果然用到了hsManager,所以可以得出结论,job信息是从historyInfo信息中得来.而在hsManager的getAllPartialJobs是从下面这个方法来的<span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Collection</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryFileInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> getAllFileInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throws</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
scanIntermediateDirectory</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">return</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> jobListCache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">values</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>他的初始化方法在下面的方法中实现<span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">/**
* Populates index data structures. Should only be called at initialization
* times.
*/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">@SuppressWarnings</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"unchecked"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">void</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> initExisting</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throws</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
LOG</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">info</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"Initializing Existing Jobs..."</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> timestampedDirList </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> findTimestampedDirectories</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">// Sort first just so insertion is in a consistent order</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Collections</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">sort</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">timestampedDirList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">for</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fs </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> timestampedDirList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">// TODO Could verify the correct format for these directories.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
addDirectoryToSerialNumberIndex</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">for</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">int</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> i</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> timestampedDirList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">size</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
i </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">>=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">0</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">&&</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">!</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">jobListCache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">isFull</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> i</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">--)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fs </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> timestampedDirList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">get</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">i</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
addDirectoryToJobListCache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>第二行扫描目录的方法就是发现JobHistory的存储目录,然后获取FileStatus对象.下面是对第一个方法的具体介绍<span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">/**
* Finds all history directories with a timestamp component by scanning the
* filesystem. Used when the JobHistory server is started.
*
* @return list of history directories
*/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">protected</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> findTimestampedDirectories</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throws</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fsList </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobHistoryUtils</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">localGlobber</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">doneDirFc</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
doneDirPrefixPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> DONE_BEFORE_SERIAL_TAIL</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">return</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fsList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>doneDirPrefixPath就是存储目录,他是从配置而来的.<span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">/**
* Gets the configured directory prefix for Done history files.
* @param conf the configuration object
* @return the done history directory
*/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">static</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">String</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> getConfiguredHistoryServerDoneDirPrefix</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Configuration</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">String</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> doneDirPrefix </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">get</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JHAdminConfig</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">MR_HISTORY_DONE_DIR</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">if</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">doneDirPrefix </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">==</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">null</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
doneDirPrefix </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">get</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">MRJobConfig</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">MR_AM_STAGING_DIR</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">MRJobConfig</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">DEFAULT_MR_AM_STAGING_DIR</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"/history/done"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">return</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> ensurePathInDefaultFileSystem</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">doneDirPrefix</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>配置中的路径加上前缀"history/done",配置是下面这个<span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"><property></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"><name></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapreduce.jobhistory.done-dir</span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"></name></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"><value></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">${yarn.app.mapreduce.am.staging-dir}/history/done</span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"></value></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"><source></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapred-default.xml</span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"></source></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"></property></span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"><property></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"><name></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn.app.mapreduce.am.staging-dir</span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"></name></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"><value></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">/tmp/hadoop-yarn/staging</span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"></value></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"><source></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapred-default.xml</span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"></source></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="tag" style="box-sizing: border-box; color: rgb(0, 0, 136);"></property></span>
因此我找到我的配置最终地址为/tmp/hadoop-yarn/staging/history/done,然后马上用hadoop fs -ls 目标目录观察一下保存job信息的文件,
<span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">bin</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop fs </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">ls </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tmp</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">staging</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">history</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">done</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span>
<span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">drwxrwx</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">---</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> root supergroup </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">0</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">13</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">47</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tmp</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">staging</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">history</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">done</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">000000</span>
不过这还是目录,继续ls命令
<span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Found</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">8</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> items </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">rwxrwx</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">---</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> root supergroup </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">33711</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">11</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">05</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tmp</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">staging</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">history</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">done</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">000000</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">job_1442921980247_0001</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442977423178</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">root</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">word</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">count</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442977507137</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">SUCCEEDED</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">root</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">default</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442977472789.jhist</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">rwxrwx</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">---</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> root supergroup </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">115932</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">11</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">05</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tmp</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">staging</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">history</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">done</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">000000</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">job_1442921980247_0001_conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">xml </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">rwxrwx</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">---</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> root supergroup </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">33707</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">11</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">18</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tmp</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">staging</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">history</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">done</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">000000</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">job_1442978197910_0001</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442978284737</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">root</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">word</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">count</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442978334462</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">SUCCEEDED</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">root</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">default</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442978306980.jhist</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">rwxrwx</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">---</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> root supergroup </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">115933</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">11</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">18</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tmp</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">staging</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">history</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">done</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">000000</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">job_1442978197910_0001_conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">xml </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">rwxrwx</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">---</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> root supergroup </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">33703</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">13</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">32</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tmp</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">staging</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">history</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">done</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">000000</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">job_1442986230207_0001</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442986273588</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">root</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">word</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">count</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442986329305</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">SUCCEEDED</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">root</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">default</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442986297304.jhist</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">rwxrwx</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">---</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> root supergroup </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">115933</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">13</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">32</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tmp</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">staging</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">history</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">done</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">000000</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">job_1442986230207_0001_conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">xml </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">rwxrwx</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">---</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> root supergroup </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">33720</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">13</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">46</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tmp</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">staging</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">history</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">done</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">000000</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">job_1442987051344_0001</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442987116527</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">root</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">word</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">count</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442987193624</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">SUCCEEDED</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">root</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">default</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442987152826.jhist</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">rwxrwx</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">---</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> root supergroup </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">115933</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">13</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">46</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tmp</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">-</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">yarn</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">staging</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">history</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">done</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">2015</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">09</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">23</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">000000</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">job_1442987051344_0001_conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">xml</span>这下就看到了目录下保存的是.jhis文件和xml配置文件,然后重点关注.jhist文件如何保存job信息,很显然每个.jhist文件对应1个Job.用cat命令查看
<span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"type"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"JOB_FINISHED"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"event"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:{</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"org.apache.hadoop.mapreduce.jobhistory.JobFinished"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:{</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"jobid"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"job_1442921980247_0001"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"finishTime"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1442977507137</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"finishedMaps"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"finishedReduces"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">1</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"failedMaps"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">0</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"failedReduces"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">0</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"totalCounters"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:{</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"name"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"TOTAL_COUNTERS"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"groups"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:[{</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"name"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"org.apache.hadoop.mapreduce.FileSystemCounter"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"displayName"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"File System Counters"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"counts"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:[{</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"name"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"FILE_BYTES_READ"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"displayName"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"FILE: Number of bytes read"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"value"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">10992</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>可以看到,里面用json字符串的格式保存了很多counter信息,而这些信息就是JobHistory上面所显示的内容.
JobHistory文件信息获取
OK,上一步骤了解了存储文件的存储位置后,我们面临的问题就是如何取出来,最好转化为对象的形式进行值的获取.非常幸运的是在HistoryFileManager中,恰好有对HistoryFileInfo到Job的转换方法<span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">/**
* Parse a job from the JobHistoryFile, if the underlying file is not going
* to be deleted.
*
* @return the Job or null if the underlying file was deleted.
* @throws IOException
* if there is an error trying to read the file.
*/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">synchronized</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Job</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> loadJob</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throws</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">return</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">CompletedJob</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> jobIndexInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getJobId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(),</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> historyFile</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">false</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> jobIndexInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getUser</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(),</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">this</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> aclsMgr</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>而且还能控制是否要加载task的数据信息.我对照JobHistory此方面的代码,对其进行模仿,写了一个抓取程序.主工具代码如下<span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">package</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapreduce</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">v2</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tool</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> java</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">io</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileNotFoundException</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> java</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">io</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> java</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">util</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">ArrayList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> java</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">util</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> java</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">util</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Map</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">conf</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Configuration</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileContext</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">PathFilter</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">RemoteIterator</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">UnsupportedFileSystemException</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapreduce</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">v2</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">api</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">records</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">TaskId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapreduce</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">v2</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">app</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapreduce</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">v2</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">app</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Task</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapreduce</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">v2</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">tool</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryFileInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapreduce</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">v2</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">jobhistory</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileNameIndexUtils</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapreduce</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">v2</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">jobhistory</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobHistoryUtils</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> org</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">apache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hadoop</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">mapreduce</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">v2</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">jobhistory</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobIndexInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">import</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> com</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">google</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">common</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">annotations</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">VisibleForTesting</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">class</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HSTool</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">private</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">static</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">String</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> DONE_BEFORE_SERIAL_TAIL </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobHistoryUtils</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">doneSubdirsBeforeSerialTail</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">String</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> jobHistoryPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Path</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> doneDirPrefixPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileContext</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> doneDirFc</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">ArrayList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryFileInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> historyFileInfos</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HSTool</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">String</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> jobHistoryPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">this</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">jobHistoryPath </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> jobHistoryPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">this</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">historyFileInfos </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">ArrayList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryFileInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">>();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">public</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">void</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> getHistoryData</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">String</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> doneDirPrefix </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> jobHistoryPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">try</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
doneDirPrefixPath </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileContext</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getFileContext</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Configuration</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">())</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">makeQualified</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">doneDirPrefix</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">));</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
doneDirFc </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileContext</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getFileContext</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">doneDirPrefixPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">toUri</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
doneDirFc</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">setUMask</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobHistoryUtils</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">HISTORY_DONE_DIR_UMASK</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">catch</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">UnsupportedFileSystemException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">// TODO Auto-generated catch block</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">printStackTrace</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">catch</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IllegalArgumentException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">// TODO Auto-generated catch block</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">printStackTrace</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
fileStatus </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">null</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">try</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
fileStatus </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> findTimestampedDirectories</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">catch</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">// TODO Auto-generated catch block</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">printStackTrace</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">if</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fileStatus </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">==</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">null</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">System</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">out</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">println</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"fileStatus is null"</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">else</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">System</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">out</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">println</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"dir fileStatus size is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">size</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">for</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fs </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">System</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">out</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">println</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"child path name is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">().</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getName</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">try</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
addDirectoryToJobListCache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">catch</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">// TODO Auto-generated catch block</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">printStackTrace</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">System</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">out</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">println</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"history fileInfo size is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">this</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">historyFileInfos</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">size</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">for</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryFileInfo</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> hfi </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">this</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">historyFileInfos</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">System</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">out</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">println</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"file jobId is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> hfi</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getJobId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
parseCompleteJob</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hfi</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">true</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">/**
* Finds all history directories with a timestamp component by scanning the
* filesystem. Used when the JobHistory server is started.
*
* @return list of history directories
*/</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">private</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> findTimestampedDirectories</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throws</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fsList </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobHistoryUtils</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">localGlobber</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">doneDirFc</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
doneDirPrefixPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> DONE_BEFORE_SERIAL_TAIL</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">return</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fsList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">private</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">void</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> addDirectoryToJobListCache</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Path</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throws</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> historyFileList </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> scanDirectoryForHistoryFiles</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
doneDirFc</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">for</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fs </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> historyFileList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobIndexInfo</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> jobIndexInfo </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileNameIndexUtils</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getIndexInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">().</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getName</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">String</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> confFileName </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobHistoryUtils</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getIntermediateConfFileName</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">jobIndexInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getJobId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">String</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> summaryFileName </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobHistoryUtils</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getIntermediateSummaryFileName</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">jobIndexInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getJobId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryFileInfo</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fileInfo </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryFileInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(),</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">().</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getParent</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(),</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> confFileName</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">),</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
fs</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">().</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getParent</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(),</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> summaryFileName</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">),</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
jobIndexInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">true</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
historyFileInfos</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">add</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fileInfo</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">protected</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> scanDirectoryForHistoryFiles</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Path</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileContext</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fc</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throws</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">return</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> scanDirectory</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fc</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">JobHistoryUtils</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getHistoryFileFilter</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="lit" style="box-sizing: border-box; color: rgb(0, 102, 102);">@VisibleForTesting</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">protected</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">static</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> scanDirectory</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Path</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileContext</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fc</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">PathFilter</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> pathFilter</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">throws</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
path </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fc</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">makeQualified</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">List</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> jhStatusList </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">new</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">ArrayList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">>();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">try</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">RemoteIterator</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fileStatusIter </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fc</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">listStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">while</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fileStatusIter</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">hasNext</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">())</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileStatus</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fileStatus </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fileStatusIter</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">next</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Path</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> filePath </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getPath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">if</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">isFile</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">&&</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> pathFilter</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">accept</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">filePath</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">))</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
jhStatusList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">add</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">fileStatus</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">catch</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">FileNotFoundException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> fe</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">System</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">out</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">println</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"Error while scanning directory "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> path</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">return</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> jhStatusList</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">private</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">void</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> parseCompleteJob</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">HistoryFileInfo</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> hfi</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">boolean</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> loadTask</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Job</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Task</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> task</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Map</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">TaskId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Task</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> taskInfos</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
job </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">null</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">;</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">try</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
job </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> hfi</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">loadJob</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">loadTask</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">);</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">catch</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">IOException</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">)</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="com" style="box-sizing: border-box; color: rgb(136, 0, 0);">// TODO Auto-generated catch block</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
e</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">printStackTrace</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">System</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">out</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">println</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"job info : job user is"</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getUserName</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">", map num is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getTotalMaps</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">", job name is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getName</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">", start time is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getReport</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">().</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getStartTime</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">", finish time is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getReport</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">().</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getFinishTime</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
taskInfos </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> job</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getTasks</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">System</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">out</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">println</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"job task total num is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> taskInfos</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">size</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">for</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Map</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Entry</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);"><</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">TaskId</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">,</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">Task</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">></span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> entry </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">:</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> taskInfos</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">entrySet</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">())</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">{</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
task </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">=</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> entry</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getValue</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">();</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="typ" style="box-sizing: border-box; color: rgb(102, 0, 102);">System</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="kwd" style="box-sizing: border-box; color: rgb(0, 0, 136);">out</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">println</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">(</span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"task id is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> task</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getID</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">()</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="str" style="box-sizing: border-box; color: rgb(0, 136, 0);">"task start time is "</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> </span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">+</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);"> task</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">.</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getReport</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">().</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">getStartTime</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">());</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span><span class="pln" style="box-sizing: border-box; color: rgb(0, 0, 0);">
</span><span class="pun" style="box-sizing: border-box; color: rgb(102, 102, 0);">}</span>有了这把利器,相信会帮助大家更精准的发现Yarn集群中的问题.
转载地址：http://itfish.net/article/48735.html
内容来自用户分享和网络整理，不保证内容的准确性，如有侵权内容，可联系管理员处理 
相关文章推荐
- 兄弟连Linux学习笔记之权限管理
- python os模块常用方法
- NFS服务配置
- opencv2-第五章-图像处理
- SSI框架学习
- libevent源码深度剖析
- linux下线程的同步和互斥
- 空引用，未将对象引用设置到对象的实例
- codeforces #326 div 2 A. Duff and Meat(水）
- 状态压缩 + 暴力 HDOJ 4770 Lights Against Dudely
- 利用51单片机 串口通信实现字符串发送
- 年月日地址
- FMDB取出所有数据
- java使用正则表达式过滤掉字符串中的html标签
- Eclipse 中报Unhandled event loop exception Java heap space(如果是JS导致的)解决方案
- HDU - 2955 Robberies 01背包
- 控制器的创建方式
- Arrays
- js 函数的方法 函数调用模式 this => that arguments
- Spring MVC 静态资源的访问