HBase源码解析(二) HMaster主要类成员解析

本文基于HBase-0.94.1分析HMaster的主要类成员.

HMaster是HBase主/从集群架构中的中央节点。通常一个HBase集群存在多个HMaster节点,其中一个为Active Master,其余为Backup Master.

HMaster的主要类成员如下:

1.ZooKeeper侦听

这些类都继承自ZookeeperListener.

/******************************ZooKeeperListener and ZooKeeperWatcher*********************************************/
// Our zk client.
private ZooKeeperWatcher zooKeeper;
// Manager and zk listener for master election//继承ZooKeeperListener
private ActiveMasterManager activeMasterManager;
// Region server tracker			// 继承ZooKeeperListener
private RegionServerTracker regionServerTracker;
// Draining region server tracker//继承ZooKeeperListener
private DrainingServerTracker drainingServerTracker;
/** manager of assignment nodes in zookeeper*/
AssignmentManager assignmentManager;

1.1 ZooKeeperWatcher

ZooKeeperWatcher是HBase里唯一一个实现Watcher接口的类，其他需要感知ZooKeeper的内部类都要通过ZooKeeperWatcher.registerListener(ZooKeeperListener listener)方法向ZooKeeperWatcher实例注册。

zooKeeper是一个ZooKeeperWatcher对象，它在HMaster构造函数中初始化; 它是一个单一的ZooKeeper Watcher ，每个HMaster，RegionServer,Client都会实例化一个ZooKeeperWatcher。

1.2 ZooKeeperListener

ZooKeeperListener是HBase用来侦听Zookeeper事件的基类。

(1)一个进程的ZookeeperWatcher将执行ZookeeperListener父类的某个方法。为了从watcher收到事件，每个listener必须通过ZookeeperWatcher.registerListener注册自己.

(2)ZooKeeperListener的子类需要重写自己感兴趣的方法 (注意,wather在调用listeners里的方法时将会被阻塞，所以listener里的方法不要long-running)

ActiveMasterManager

该类用来处理master端所有与master选举相关的事情.

(1).侦听并响应关于master znode的zk通知(ZK Notifications),包括nodeCreated和nodeDeleted.

(2).包括一个阻塞方法:持有backup masters，等待active master挂掉.

(3).这个class在HMaster里被初始化,HMaster调用blockUntilBecomingActiveMaster()以阻塞等待成为active master.

RegionServerTracker

该类用来通过ZK跟踪所有OnLine状态的RegionServer

DrainingServerTracker

该类用来通过ZK跟踪所有处于上线/下线流水状态的RegionServer.

AssignmentManager

assignmentManager 负责管理region的分配工作.

2. RPC Server

HMaster通过HBase RPC机制，将自己封装成一个RPC Server,对外提供RPC调用服务.

// RPC server for the HMaster
private final RpcServer rpcServer;

/**
* This servers address.
*/
private final InetSocketAddress isa;

3.HBase文件系统: MasterFileSystem

MasterFileSystem 抽象了HMaster与低层文件系统交互所需的一系列接口

它在HMaster.finishInitialization()方法中被初始化

// file system manager for the master FS operations//抽象了HMaster与底层文件系统交互所需的一系列操作
private MasterFileSystem fileSystemManager

private void finishInitialization(MonitoredTask status, boolean masterRecovery)
throws IOException, InterruptedException, KeeperException {
//将是否为active master标识置true
isActiveMaster = true;

/*我们已经是active master了，开始初始化相关组件.
...
this.masterActiveTime = System.currentTimeMillis();
// TODO: Do this using Dependency Injection, using PicoContainer, Guice or Spring.
// 1.创建HBase文件系统 MasterFileSystem.(抽象了HMaster与低层文件系统交互所需的一系列接口)
this.fileSystemManager = new MasterFileSystem(this, this, metrics, masterRecovery);
...
}

4. RegionServer和Catalog管理

ServerManager

ServerManager管理所有的regionServer信息。

实际上，RegionServerTracker,DrainingServerTracker,AssignmentManager 初始化时需要serverManager作为构造函数的参数.

/******************************ServerManager****************************************************/
/** server manager to deal with region server info*/
private ServerManager serverManager;

RegionServerTracker,DrainingServerTracker,AssignmentManager在initializeZKBasedSystemTracker()时进行初始化.

private void initializeZKBasedSystemTrackers() throws IOException,
InterruptedException, KeeperException {
this.catalogTracker = new CatalogTracker(this.zooKeeper, this.conf, this);
this.catalogTracker.start();

this.balancer = LoadBalancerFactory.getLoadBalancer(conf);
this.assignmentManager = new AssignmentManager(this, serverManager,
this.catalogTracker, this.balancer, this.executorService);
zooKeeper.registerListenerFirst(assignmentManager);

this.regionServerTracker = new RegionServerTracker(zooKeeper, this,
this.serverManager);
this.regionServerTracker.start();

this.drainingServerTracker = new DrainingServerTracker(zooKeeper, this,
this.serverManager);
this.drainingServerTracker.start();

CatalogTracker

catalogTracker 跟踪"目录表"(-ROOT-和.META.表)的可用性

ClusterStatusTracker

clusterStatusTracker跟踪集群在zookeeper上的配置(Tracker on cluster settings up in zookeeper). ClusterStatusTracker和ClusterStatus不同，后者只是一个存储 cluster当前视图的镜像的数据结构，而ClusterStatusTracker是用于跟踪集群在zookeeper上配置的属性信息.

// manager of catalog regions
private CatalogTracker catalogTracker;
// Cluster status zk tracker and local setter
private ClusterStatusTracker clusterStatusTracker;

5.HBase 执行器服务:ExecutorService

ExecutorService是一个通用的执行器服务类,这个组件抽象了一个threadPool，一个队列，和一个Runnable线程(Handler角色)

/******************************ExecutorService****************************************************/
// Instance of the hbase executor service.
ExecutorService executorService;

该变量在finishInitialization方法中初始化

private void finishInitialization(MonitoredTask status, boolean masterRecovery)
throws IOException, InterruptedException, KeeperException {
......
if (!masterRecovery) {
//初始化ExecutorService和serverManager
this.executorService = new ExecutorService(getServerName().toString());//维护一个threadPool和队列
this.serverManager = new ServerManager(this, this);//管理所有的regionserver
}

6.负载均衡:LoadBalancer

LoadBalancer的职责是维护regionServers之间的负载均衡; balancerChore线程做一些Balancer相关的清理工作.

用户可以通过实现LoadBalancer接口来定制自己的负载均衡策略. 默认情况下，HBase采用的是org.apache.hadoop.hbase.master.DefaultLoadBalancer类做负载均衡.

/******************************LoadBalancer, BalancerChore****************************************/
private LoadBalancer balancer;
private Thread balancerChore;

loadBalancer是在initializeZKBasedSystemTrackers()方法中调用LoadBalancerFactory.getLoadBalancer(conf)初始化的.

private void initializeZKBasedSystemTrackers() throws IOException,
InterruptedException, KeeperException {
this.balancer = LoadBalancerFactory.getLoadBalancer(conf);
...
}

从LoadBalancerFactory代码可以看出, LoadBalancer由参数HBASE_MASTER_LOADBALANCER_CLASS ( hbase.master.loadbalancer.class)指定，默认值是DefaultLoadBalancer.class

public class LoadBalancerFactory {

/**
* Create a loadblanacer from the given conf.
* @param conf
* @return A {@link LoadBalancer}
*/
public static LoadBalancer getLoadBalancer(Configuration conf) {

// Create the balancer
Class<? extends LoadBalancer> balancerKlass = conf.getClass(
HConstants.HBASE_MASTER_LOADBALANCER_CLASS,
DefaultLoadBalancer.class, LoadBalancer.class);
return ReflectionUtils.newInstance(balancerKlass, conf);

}
}

7. Table描述符管理：TableDescriptor

TableDescriptor 接口描述了用来管理Table描述符的一系列操作,FSTableDescriptor是该接口的实现类。

/******************************TableDescriptors***************************************************/
private TableDescriptors tableDescriptors;

在HMaster.finishInitialization方法中初始化

private void finishInitialization(MonitoredTask status, boolean masterRecovery)
throws IOException, InterruptedException, KeeperException {
....
// 2.初始化tableDescriptors，从文件系统读取HTable的描述信息.
this.tableDescriptors =
new FSTableDescriptors(this.fileSystemManager.getFileSystem(),
this.fileSystemManager.getRootDir());

FTableDescriptor的构造函数如下: FSTableDescriptors根据fs和rootdir可以读取、更改所有的table描述符

public FSTableDescriptors(final FileSystem fs, final Path rootdir,
final boolean fsreadOnly) {
super();
this.fs = fs;
this.rootdir = rootdir;
this.fsreadonly = fsreadOnly;
}

以FSTableDescriptor。FSTableDescriptor获取rootdir下面的Path(这些Path经过FileUtils.getTableDirs(fs,rootdir)获得，经过了过滤) . 将Path的Name(也即文件名)作为tableName.

@Override
public Map<String, HTableDescriptor> getAll()
throws IOException {
Map<String, HTableDescriptor> htds = new TreeMap<String, HTableDescriptor>();
//获取所有的tableName
List<Path> tableDirs = FSUtils.getTableDirs(fs, rootdir);
for (Path d: tableDirs) {
HTableDescriptor htd = null;
try {
htd = get(d.getName());
} catch (FileNotFoundException fnfe) {
} if (htd == null)
continue;
htds.put(d.getName(), htd);
} return htds;
}

8. HTable镜像管理:SnapshotManager

SnapshotManager管理snapshots的生成和装载(taking and restoring)过程.

// monitor for snapshot of hbase tables
private SnapshotManager snapshotManager;  //HBase tables的镜像管理者

9.协同处理器管理: CoprocessorProtocol Map

/******************************protocolHandlers 管理coprocessor protocol的注册*********************/
// Registered master protocol handlers
private ClassToInstanceMap<CoprocessorProtocol>
protocolHandlers = MutableClassToInstanceMap.create();

10.健康检查和垃圾清理: 各种Chore线程

/** The health check chore. */
private HealthCheckChore healthCheckChore;//健康检查
/******************************垃圾回收:CatalogJanitor,LogCleaner,HFileCleaner*********************/
private CatalogJanitor catalogJanitorChore;  //定时扫描-META-，对无用的region进行垃圾回收
private LogCleaner logCleaner;
private HFileCleaner hfileCleaner;

11. 其他

//todo