Flume 1.4.0 User Guide
2013-08-02 10:35
316 查看
Apache Flume is a distributed, reliable, and available system for efficiently collecting, aggregating and moving large amounts of log data from many different sources to a centralized data store.
A Flume event is defined as a unit of data flow having a byte payload and an optional set of string attributes. A Flume agent is a (JVM) process that hosts the components through which events flow from an external source to the next destination (hop).
An agent is started using a shell script called flume-ng which is located in the bin directory of the Flume distribution. You need to specify the agent name, the config directory, and the config file on the command line:
Here, we give an example configuration file, describing a single-node Flume deployment. This configuration lets a user generate events and subsequently logs them to the console.
A Flume event is defined as a unit of data flow having a byte payload and an optional set of string attributes. A Flume agent is a (JVM) process that hosts the components through which events flow from an external source to the next destination (hop).
An agent is started using a shell script called flume-ng which is located in the bin directory of the Flume distribution. You need to specify the agent name, the config directory, and the config file on the command line:
$ bin/flume-ng agent -n $agent_name -c conf -f conf/flume-conf.properties.template
Here, we give an example configuration file, describing a single-node Flume deployment. This configuration lets a user generate events and subsequently logs them to the console.
# example.conf: A single-node Flume configuration # Name the components on this agent a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = netcat a1.sources.r1.bind = localhost a1.sources.r1.port = 44444 # Describe the sink a1.sinks.k1.type = logger # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1
相关文章推荐
- Flume 1.7.0 User Guide
- Flume 1.5.0.1 User Guide
- 【Hadoop】Flume官方文档翻译——Flume 1.7.0 User Guide (unreleased version)(一)
- 【Hadoop】Flume官方文档翻译——Flume 1.7.0 User Guide (unreleased version)中一些知识点
- flume User Guide翻译-logging raw data / plugin
- Flume 学习笔记 一 : Flume User Guide
- 【Hadoop】Flume官方文档翻译——Flume 1.7.0 User Guide (unreleased version)(二)
- Markdown Editor's user guide
- BIOS MCSDK 2.0 User Guide - Acronyms and Definitions
- Groovy User Guide
- PhysX3 User Guide 06 - Callbacks and Customization
- Havok_2014-1-0_Pc_Xs_User_Guide(2.2.1-创建一个Physics2012对象)
- Gradle User Guide(3)——使用命令行
- Flume-ng-1.4.0安装及运行遇到问题总结
- Kibana User Guide [4.2] » Introduction
- Kibana User Guide [4.2] » Kibana Plugins
- OpenNI UserGuide
- Kibana User Guide [4.2] » Using Kibana in a Production Environment
- MINA 2.0 User Guide
- Gradle Plugin User Guide - Android