1.2 Use Cases中 Stream Processing官网剖析(博主推荐)
2017-04-26 08:45
435 查看
不多说,直接上干货!
一切来源于官网
![](https://images2015.cnblogs.com/blog/855959/201704/855959-20170426084431225-1060692913.png)
Many users of Kafka process data in processing pipelines consisting of multiple stages, where raw input data is consumed from Kafka topics and then aggregated, enriched, or otherwise transformed into new topics for further consumption or follow-up processing. For example, a processing pipeline for recommending news articles might crawl article content from RSS feeds and publish it to an "articles" topic; further processing might normalize or deduplicate this content and published the cleansed article content to a new topic; a final processing stage might attempt to recommend this content to users. Such processing pipelines create graphs of real-time data flows based on the individual topics. Starting in 0.10.0.0, a light-weight but powerful stream processing library called Kafka Streams is available in Apache Kafka to perform such data processing as described above. Apart from Kafka Streams, alternative open source stream processing tools include Apache Storm and Apache Samza.
一切来源于官网
http://kafka.apache.org/documentation/
![](https://images2015.cnblogs.com/blog/855959/201704/855959-20170426084431225-1060692913.png)
Stream Processing
流处理
Many users of Kafka process data in processing pipelines consisting of multiple stages, where raw input data is consumed from Kafka topics and then aggregated, enriched, or otherwise transformed into new topics for further consumption or follow-up processing. For example, a processing pipeline for recommending news articles might crawl article content from RSS feeds and publish it to an "articles" topic; further processing might normalize or deduplicate this content and published the cleansed article content to a new topic; a final processing stage might attempt to recommend this content to users. Such processing pipelines create graphs of real-time data flows based on the individual topics. Starting in 0.10.0.0, a light-weight but powerful stream processing library called Kafka Streams is available in Apache Kafka to perform such data processing as described above. Apart from Kafka Streams, alternative open source stream processing tools include Apache Storm and Apache Samza.
kafka消息处理包含多个阶段。其中原始输入数据是从kafka主题消费的,然后汇总,丰富,或者以其他的方式处理转化为新主题, 例如,一个推荐新闻文章,文章内容可能从“articles”主题获取; 然后进一步处理内容,得到一个处理后的新内容,最后推荐给用户。 这种处理是基于单个主题的实时数据流。从0.10.0.0开始,轻量,但功能强大的流处理,就进行这样的数据处理了。 除了Kafka Streams,还有Apache Storm和Apache Samza可选择。
相关文章推荐
- 1.2 Use Cases中 Messaging官网剖析(博主推荐)
- 1.2 Use Cases中 Website Activity Tracking官网剖析(博主推荐)
- 1.2 Use Cases中 Metrics官网剖析(博主推荐)
- 1.2 Use Cases中 Event Sourcing官网剖析(博主推荐)
- 1.2 Use Cases中 Commit Log官网剖析(博主推荐)
- 1.2 Use Cases中 Log Aggregation官网剖析(博主推荐)
- Event Serializers官网剖析(博主推荐)
- 1.1 Introduction中 Guarantees官网剖析(博主推荐)
- 1.3 Quick Start中 Step 5: Start a consumer官网剖析(博主推荐)
- Flume Interceptors官网剖析(博主推荐)
- 1.1 Introduction中 Kafka as a Messaging System官网剖析(博主推荐)
- 1.1 Introduction中 Kafka as a Storage System官网剖析(博主推荐)
- 1.3 Quick Start中 Step 6: Setting up a multi-broker cluster官网剖析(博主推荐)
- 1.1 Introduction中 Kafka for Stream Processing官网剖析(博主推荐)
- 1.3 Quick Start中 Step 7: Use Kafka Connect to import/export data官网剖析(博主推荐)
- 1.1 Introduction中 Putting the Pieces Together官网剖析(博主推荐)
- Flume Source官网剖析(博主推荐)
- 1.3 Quick Start中 Step 8: Use Kafka Streams to process data官网剖析(博主推荐)
- 2.3 Streams API 官网剖析(博主推荐)
- 1.4 Ecosystem官网剖析(博主推荐)