graylog2 架构--转载
2015-05-08 09:22
232 查看
原文地址:http://docs.graylog.org/en/latest/pages/architecture.html
graylog-server nodes should have a focus on CPU power.
Elasticsearch nodes should have as much RAM as possible and the fastest disks you can get. Everything depends on I/O speed here.
MongoDB is only being used to store configuration and the dead letter messages, and can be sized fairly small.
graylog-web-interface nodes are mostly waiting for HTTP answers of the rest of the system and can also be rather small.
graylog-radio nodes act as workers. They don’t know each other and you can shut them down at any point in time without changing the cluster state at all.
Also keep in mind that messages are only stored in Elasticsearch. If you have data loss on Elasticsearch, the messages are gone - except if you have created backups of the indices.
MongoDB is only storing meta information and will be abstracted with a general database layer in future versions. This will allow you to use other databases like MySQL instead.
If you are running a setup with Graylog Radio we recommend to shut down the Graylog Radio architecture including AMQP or Kafka brokers completely and directly send messages to thegraylog-server nodes. If you have been using Graylog Radio for load balancing, you should now put a classic load balancer in front of your graylog-server nodes.
This approach has been proven to work great in large high-throughput setups of several of our large scale customers and immensely reduced complexity of their setups.
The Kafka and AMQP inputs are still supported and can be used to build a custom setup using message brokers, if you want to keep using that. A reason for this might be that Graylog is not the only subscriber to the messages on the bus. However we would recommend to use Graylog forwarders to either write to a message bus after processing or write to other systems directly.
Architectural considerations
There are a few rules of thumb when scaling resources for Graylog:graylog-server nodes should have a focus on CPU power.
Elasticsearch nodes should have as much RAM as possible and the fastest disks you can get. Everything depends on I/O speed here.
MongoDB is only being used to store configuration and the dead letter messages, and can be sized fairly small.
graylog-web-interface nodes are mostly waiting for HTTP answers of the rest of the system and can also be rather small.
graylog-radio nodes act as workers. They don’t know each other and you can shut them down at any point in time without changing the cluster state at all.
Also keep in mind that messages are only stored in Elasticsearch. If you have data loss on Elasticsearch, the messages are gone - except if you have created backups of the indices.
MongoDB is only storing meta information and will be abstracted with a general database layer in future versions. This will allow you to use other databases like MySQL instead.
Minimum setup
This is a minimum Graylog setup that can be used for smaller, non-critical, or test setups. None of the components is redundant but it is easy and quick to setup.Bigger production setup
This is a setup for bigger production environments. It has several graylog-server nodes behind a load balancer that share the processing load. The load balancer can ping the graylog-server nodes via REST/HTTP to check if they are alive and take dead nodes out of the cluster.Highly available setup with Graylog Radio
Beginning with Graylog 1.0 on we do no longer recommend running Graylog Radio because we are now using a high-performant message journal (from the Apache Kafka project) in every graylog-server instance which is spooling all incoming messages to disk immediately and is able to buffer load spikes just at least as good as Graylog Radio was, but with less dependencies and maintenance overhead.If you are running a setup with Graylog Radio we recommend to shut down the Graylog Radio architecture including AMQP or Kafka brokers completely and directly send messages to thegraylog-server nodes. If you have been using Graylog Radio for load balancing, you should now put a classic load balancer in front of your graylog-server nodes.
This approach has been proven to work great in large high-throughput setups of several of our large scale customers and immensely reduced complexity of their setups.
The Kafka and AMQP inputs are still supported and can be used to build a custom setup using message brokers, if you want to keep using that. A reason for this might be that Graylog is not the only subscriber to the messages on the bus. However we would recommend to use Graylog forwarders to either write to a message bus after processing or write to other systems directly.
相关文章推荐
- 基于AngularJS的企业软件前端架构[转载]
- 转载:大型网站架构分析系列技术文档合集
- 转载:假如我来架构12306网站(二) - 浅谈系统需求调研
- 新浪微博技术架构分析-转载
- (转)新浪微博技术架构分析-转载
- Android GPS架构分析(转载二)
- ASP.NET MVC:模块化/插件式架构实现(转载)
- 转载 解密蓝牙mesh系列 | 第四篇【蓝牙mesh架构】【地址(Address)】【消息(Message)】【消息安全】【消息交换】
- 转载自思特沃克(马伟) ----RESTful架构风格下的4大常见安全问题
- asp.net mvc(模式)和三层架构(BLL、DAL、Model)的联系与区别 转载自:http://blog.csdn.net/luoyeyu1989/article/details/8275866
- 【转载】oracle 体系架构图
- 大型网站架构演变和知识体系(转载)
- 架构:Hexagonal Architecture Guidelines for Rails(转载)
- 基于gridview的三层结构的代码演示 (二 ) 三层架构的实例演示 (原创,如需转载请联系作者)
- [转载]微服务实战(一):微服务架构的优势与不足
- Swift:高级架构、流水线深度、内存延迟 转载
- 浅谈Web网站架构演变过程[转载]
- 大型高并发高负载网站的系统架构-Web开发(转载)
- 【转载,忘记源出处了】大型网站架构演变和知识体系
- 热点推荐:秒杀系统架构分析与实战--转载