输出、状态-hadoop源码TaskAttemptID TaskTrackerAction JobTracker，FileOutputCommitter相关-by小雨

首先声明，我是一个菜鸟。一下文章中出现技术误导情况盖不负责

1,TaskAttemptID代表task attempt，一个task attempt就是一个map/reduce task 的一个例实taskid，而个每TaskAttemptID由两部分构成：TaskID+task序列号

eg:

attempt_200707121733_0003_m_000005_0

代表2007年07月12日17点33分启动的第0003号作业（job）的第00005号map任务的第0号task attempt

2,TaskTrackerAction的类型有，LaunchTaskAction()，KillTaskAction()，KillJobAction()，ReinitTrackerAction()，CommitTaskAction();

/**
* Ennumeration of various 'actions' that the {@link JobTracker}
* directs the {@link TaskTracker} to perform periodically.
*
*/
public static enum ActionType {
/** Launch a new task. */
LAUNCH_TASK,

/** Kill a task. */
KILL_TASK,

/** Kill any tasks of this job and cleanup. */
KILL_JOB,

/** Reinitialize the tasktracker. */
REINIT_TRACKER,

/** Ask a task to save its output. */
COMMIT_TASK
};

/**
* A factory-method to create objects of given {@link ActionType}.
* @param actionType the {@link ActionType} of object to create.
* @return an object of {@link ActionType}.
*/
public static TaskTrackerAction createAction(ActionType actionType) {
TaskTrackerAction action = null;

switch (actionType) {
case LAUNCH_TASK:
{
action = new LaunchTaskAction();
}
break;
case KILL_TASK:
{
action = new KillTaskAction();
}
break;
case KILL_JOB:
{
action = new KillJobAction();
}
break;
case REINIT_TRACKER:
{
action = new ReinitTrackerAction();
}
break;
case COMMIT_TASK:
{
action = new CommitTaskAction();
}
break;
}

return action;
}

3,JobTracker中相干：

offerService()

|--taskScheduler.start();

|--recoveryManager.recover();

通过RecoveryManager的recover()法方恢复Job

|-- this.expireTrackersThread.start();

检测已失效的TaskTracker点节。

// Used to expire TaskTrackers that have gone down

|-- this.retireJobsThread.start();

除清那些已成完很长时光仍然存在队列中的job

|--expireLaunchingTaskThread.start();

停止那些在超时时光内未告报进度的Tasks。

A thread to timeout tasks that have been assigned to task trackers,
* but that haven't reported back yet.

|--completedJobsStoreThread.start();

|--this.interTrackerServer.start();/ start the inter-tracker server once the jt is ready

/**
* Run forever
*/
public void offerService() throws InterruptedException, IOException {
// Prepare for recovery. This is done irrespective of the status of restart
// flag.
while (true) {
try {
recoveryManager.updateRestartCount();
break;
} catch (IOException ioe) {
LOG.warn("Failed to initialize recovery manager. ", ioe);
// wait for some time
Thread.sleep(FS_ACCESS_RETRY_PERIOD);
LOG.warn("Retrying...");
}
}

taskScheduler.start();

// Start the recovery after starting the scheduler
try {
recoveryManager.recover();
} catch (Throwable t) {
LOG.warn("Recovery manager crashed! Ignoring.", t);
}

this.expireTrackersThread = new Thread(this.expireTrackers,
"expireTrackers");
this.expireTrackersThread.start();
this.retireJobsThread = new Thread(this.retireJobs, "retireJobs");
this.retireJobsThread.start();
expireLaunchingTaskThread.start();

if (completedJobStatusStore.isActive()) {
completedJobsStoreThread = new Thread(completedJobStatusStore,
"completedjobsStore-housekeeper");
completedJobsStoreThread.start();
}

// start the inter-tracker server once the jt is ready
this.interTrackerServer.start();

synchronized (this) {
state = State.RUNNING;
}

4, Job的五种状态

public static final int RUNNING = 1;
public static final int SUCCEEDED = 2;
public static final int FAILED = 3;
public static final int PREP = 4;

public static final int KILLED = 5;

5，jobStatus信息:

this.jobid = jobid;
this.setupProgress = setupProgress;
this.mapProgress = mapProgress;
this.reduceProgress = reduceProgress;
this.cleanupProgress = cleanupProgress;
this.runState = runState;

6，FileOutputCommitter相干

setupJob：Hadoop初始化时设置job的输出；

commitJob：当job成完时，除清job的输出，这个法方在馈反来回的job状态为SUCCEEDED时用调；

cleanupJob：job结束后除清job的输出；

// do the clean up of temporary directory

abortJob：当job的返回状态是FAILED或KILLED时，行执该数函，用于终止作业的输出；

setupTask：设置task的输出；

// FileOutputCommitter's setupTask doesn't do anything. Because the
// temporary task directory is created on demand when the
// task is writing.

needsTaskCommit：检测task否是须要交提；

commitTask：将task的输出移到作业的输出目录；

// Move the task outputs to their final place

// Delete the temporary task-specific output directory

abortTask：取消task的输出；

文章结束给大家分享下程序员的一些笑话语录：

IBM和波音777

　　波音777是有史以来第一架完全在电脑虚拟现实中设计制造的飞机，所用的设备完全由IBM公司所提供。试飞前，波音公司的总裁非常热情的邀请IBM的技术主管去参加试飞，可那位主管却说道:“啊，非常荣幸，可惜那天是我妻子的生日，So..”..

　　波音公司的总载一听就生气了:“胆小鬼，我还没告诉你试飞的日期呢!”