分布式日志收集系统- Cloudera Flume 命令介绍

这里介绍的flume版本为flume-0.9.4-cdh3u3,下载地址: https://ccp.cloudera.com/display/SUPPORT/CDH3+Downloadable+Tarballs
下载解压后到 FLUME_HOME/bin目录:

[hadoop@localhost bin]$ sh flume
usage: flume command [args...]
commands include:
dump            Takes a specified source and dumps to console
source          Takes a specified source and dumps to console
node            Start a Flume node/agent (with watchdog)
master          Start a Flume Master server (with watchdog)
version         Dump flume build version information
node_nowatch    Start a flume node/agent (no watchdog)
master_nowatch  Start a Flume Master server (no watchdog)
class <class>   Run specified fully qualified class using Flume environment (no watchdog)
ex: flume com.cloudera.flume.agent.FlumeNode
classpath       Dump the classpath used by the java executables
shell           Start the flume shell
killmaster      Kill a running master
dumplog         Takes a specified WAL/DFO log file and dumps to console
sink            Start a one-shot flume node with console source and specified sink

dump:

将指定的source dump到console

在当前目录下创建文件 test, 输入一行记录"hello flume!", 执行以下命令

[hadoop@localhost bin]$ sh flume source 'text("test")'

2012-03-27 05:51:14,859 [logicalNode dump-9] INFO debug.ConsoleEventSink: ConsoleEventSink( debug ) opened
localhost [INFO Tue Mar 27 05:51:14 PDT 2012] hello flume!
2012-03-27 05:51:14,865 [logicalNode dump-9] INFO debug.TextFileSource: File test closed

source:

source命令等同于dump

node:

启动flume node节点,同时启动watchdog进程(watchdog进程会监控node节点进程，一旦发现node节点进程挂掉，会立即重启node节点进程)

master:

启动flume master节点,同时启动watchdog进程

node_nowatch:

启动flume node节点,但不启动watchdog进程

master_nowatch:

启动flume master节点, 但不启动watchdog进程

sink:

指定一个sink, 将console作为source

[hadoop@localhost bin]$ sh flume sink 'text("test","raw")'

在console中输入:hello flume^_^

查看test文件

[hadoop@localhost bin]$ cat test
hello flume^_^
[hadoop@localhost bin]$

shell:

进入flume shell，需要指定master地址及端口，连接上后可以运行flume集群管理命令, 查看帮助命令如下:

[flume (disconnected)] help
I know about these commands:

connect host[:adminport=35873[:reportport=45678]]
exec    (requires a command and arguments)
exec config node 'source' 'sink'
exec decommission logicalnode
exec map physicalnode logicalnode
exec multiconfig 'spec'
exec noop [delaymillis (no arg means no wait)]
exec refresh 'spec'
exec refreshAll
exec setChokeLimit physicalnode chokeid limit
exec spawn physicalnode logicalnode (synonym for exec map. deprecated.)
exec unmap physicalnode logicalnode
exec unmapAll
getconfigs
getmappings [physical node]
getnodestatus
getreports
help
quit
source load a file and execute flume shell commands in it
submit    (requires a command and arguments)
submit config node 'source' 'sink'
submit decommission logicalnode
submit map physicalnode logicalnode
submit multiconfig 'spec'
submit noop
submit refresh 'spec'
submit refreshAll
submit spawn physicalnode logicalnode (synonym for submit map. deprecated.)
submit unmap physicalnode logicalnode
submit unmapAll
wait  [maxmillis (0==infinite) [, cmdid]]
waitForNodesActive  [maxmillis (0==infinite) [, node[, ...]]]
waitForNodesDone  [maxmillis (0==infinite) [, node[, ...]]]
[flume (disconnected)]