HBase Tools and Utilities

hadoop jar /usr/local/fqlhadoop/hbase-1.1.2/lib/hbase-server-1.1.2.jar与

hbase org.apache.hadoop.hbase.mapreduce.Driver等效。

hadoop /usr/local/fqlhadoop/hbase-1.1.2/lib/hbase-server-1.1.2.jar importtsv

hbase org.apache.hadoop.hbase.mapreduce.Driver importtsv

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv



####129. HBase Tools and Utilities BEGIN####

#1

hbase org.apache.hadoop.hbase.mapreduce.Driver

##1.1

hbase org.apache.hadoop.hbase.mapreduce.Driver verifyrep与hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication等效

hbase org.apache.hadoop.hbase.mapreduce.replication.VerifyReplication --starttime=1475251200000 --endtime=1475424000000 1 hbase_test:fql_hbase_replication_ms

hbase org.apache.hadoop.hbase.mapreduce.Driver verifyrep --starttime=1475251200000 --endtime=1475424000000 1 hbase_test:fql_hbase_replication_ms

#1.2

hbase org.apache.hadoop.hbase.mapreduce.Driver importtsv

hbase org.apache.hadoop.hbase.mapreduce.ImportTsv

#2 hbase wal

wal             Write-ahead-log analyzer

WAL Tools

hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog

##get a textual dump of a WAL file

hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --dump /hbase/WALs/11.hadoop.com,16020,1474628670626/11.hadoop.com%2C16020%2C1474628670626.default.1475741264910

##force a split of a log file directory

hbase org.apache.hadoop.hbase.regionserver.wal.FSHLog --split /hbase/WALs/5.hadoop.com,16020,1475304063995

##print the contents of a WAL

hbase wal /hbase/WALs/11.hadoop.com,16020,1474628670626/11.hadoop.com%2C16020%2C1474628670626.default.1475741613348

hbase wal --printvals /hbase/WALs/11.hadoop.com,16020,1474628670626/11.hadoop.com%2C16020%2C1474628670626.default.1475741613348

hbase wal --json --printvals /hbase/WALs/11.hadoop.com,16020,1474628670626/11.hadoop.com%2C16020%2C1474628670626.default.1475742164406

hbase org.apache.hadoop.hbase.wal.WALPrettyPrinter

 

旧版

hbase hlog 

hbase org.apache.hadoop.hbase.regionserver.wal.HLogPrettyPrinter

#3 hbase hbck

hbck            Run the hbase 'fsck' tool

hbase hbck

hbase hbck hbase org.apache.hadoop.hbase.util.HBaseFsck

hbase hbck -details

hbase hbck -metaonly

hbase hbck -fixAssignments

hbase hbck -fix   

hbase hbck -repairHoles

#4 verify the compressor is available

hbase org.apache.hadoop.hbase.util.CompressionTest /compressionTest snappy

hbase org.apache.hadoop.hbase.util.CompressionTest /compressionTest lzo

hbase org.apache.hadoop.hbase.util.CompressionTest /compressionTest gz

#5 WALPlayer

hbase org.apache.hadoop.hbase.mapreduce.WALPlayer

#6.1 RowCounter 

hbase org.apache.hadoop.hbase.mapreduce.RowCounter -Dhbase.client.scanner.caching=100 PH_MONITOR.T_MONITOR_INTERFACE_LOG

#6.2 RowCounter 

hbase org.apache.hadoop.hbase.mapreduce.CellCounter PH_MONITOR.T_MONITOR_FSOF_PROVIDER_SERVICE /test53 |

<property>

  <name>mapreduce.job.counters.limit</name>

  <!--<value>120</value>-->

  <value>20000</value>

  <description>Limit on the number of counters allowed per job. The default value is 200.</description>

</property>

#7

hbase org.apache.hadoop.hbase.regionserver.CompactionTool

#8

hbase clean

#9 hbase mapredcp

hbase org.apache.hadoop.hbase.util.MapreduceDependencyClasspathTool

#10 hbase pe

hbase org.apache.hadoop.hbase.PerformanceEvaluation

#11 hbase ltt

hbase ltt

hbase org.apache.hadoop.hbase.util.LoadTestTool

其他

hbase org.apache.flume.tools.GetJavaProperty

hbase org.apache.hadoop.hbase.util.HBaseConfTool hbase.rootdir

####129. HBase Tools and Utilities END####

130. Region Management

130.2. Merge

hbase org.apache.hadoop.hbase.util.Merge <tablename> <region1> <region2>

69.7.4. StoreFile (HFile)

hbase hfile

hbase org.apache.hadoop.hbase.io.hfile.HFile

hfile           Store file analyzer

HBase中表ns_monitor:t_monitor_memory的Region分区与HBase在HDFS中的路径/hbase/data/ns_monitor/t_monitor_memory/相对应。

#to view the content of the file by HDFS file

hbase hfile -v -f /hbase/data/ns_monitor/t_monitor_memory/e15236a1387c7148dfde23022884cde3/monitor_memory/e1f274ed5c6c44f0984ac95de30fa880

hbase hfile -b -v -f /hbase/data/ns_monitor/t_monitor_memory/e15236a1387c7148dfde23022884cde3/monitor_memory/e1f274ed5c6c44f0984ac95de30fa880

hbase hfile -m -v -f /hbase/data/ns_monitor/t_monitor_memory/e15236a1387c7148dfde23022884cde3/monitor_memory/e1f274ed5c6c44f0984ac95de30fa880

hbase hfile -p -v -f /hbase/data/ns_monitor/t_monitor_memory/e15236a1387c7148dfde23022884cde3/monitor_memory/e1f274ed5c6c44f0984ac95de30fa880

#to view the content of the file by HBase region name

hbase hfile -p -v -r ns_monitor:t_monitor_memory,,1463673066515.07ded346fc2ee8ee07fdcd04eb644243.

hbase hfile -s -v -r ns_monitor:t_monitor_memory,,1463673066515.07ded346fc2ee8ee07fdcd04eb644243.

#to view the content by the RowKey of a record

hbase hfile -p -v -w "2001-01-01|192.168.16.123|mem_cache"

输出Scanned kv count -> 0

-b,--printblocks         Print block index meta data

-e,--printkey            Print keys

-f <arg>                 File to scan. Pass full-path

-h,--printblockheaders   Print block headers for each block.

(useful)-m,--printmeta           Print meta data of file

(useful)-p,--printkv             Print key/value pairs

 

-r,--region <arg>        Region to scan. Pass region name

-s,--stats               Print statistics

-v,--verbose             Verbose output

-w <arg>                 Seek to this row and print all the kvs for this row only

 

69.7.7. Compaction

A StoreFile is a facade of HFile. In terms of compaction, use of StoreFile seems to have prevailed in the past.

A Store is the same thing as a ColumnFamily. StoreFiles are related to a Store, or ColumnFamily.