logstash介绍 - 2.简单安装以及使用

说明

  本文使用的elasticsearch logstash 都是6.1.2版本，基于centos7环境上进行测试验证。

   本文测试节点的IP地址为： 192.168.5.60 。

   本文不做elasticsearch logstash安装详细说明。

   文中最后附下载地址链接。

一、简单安装 

java环境安装

# java -version
openjdk version "1.8.0_161"

从文中最后下载链接下载下来的安装包

elasticsearch-6.1.2.rpm
logstash-6.1.2.rpm

安装

rpm -ivh elasticsearch-6.1.2.rpm
rpm -ivh logstash-6.1.2.rpm

修改 elasticsearch.yml 配置中的network.host，并关闭防火墙

# cat /etc/elasticsearch/elasticsearch.yml |grep network.host
network.host: 192.168.5.60
# systemctl stop irewalld.service

启动elasticsearch

systemctl enable elasticsearch.service
systemctl start elasticsearch.service

检测elasticsarch状态

curl '192.168.5.60:9200/_cat/health?v'

二、logstash监听本地文件

配置

# cat /etc/logstash/conf.d/log2.conf
input {
file {
path => ["/var/log/lyh/messages"]
type => "system"
start_position => "beginning"
}
}
filter {

}

output {
stdout {}
}

执行logstash 

# cd /usr/share/logstash/bin/
# ./logstash -f /etc/logstash/conf.d/log2.conf  --path.settings /etc/logstash

往 /var/log/lyh/messages 插入日志

echo "Jan 23 08:51:59 localhost kernel: LYH 111" >> /var/log/lyh/messages
echo "Jan 24 08:41:58 localhost systemd: Starting Session 36 of user root.  " >> /var/log/lyh/messages

查看信息

可以在执行 ./logstash -f /etc/logstash/conf.d/log2.conf  --path.settings /etc/logstash 的界面看到打印日志

2018-01-24T01:10:00.202Z 0.0.0.0 Jan 23 08:51:59 localhost kernel: LYH 111
....

三、logstash作为syslog-server监听syslog日志信息

配置

# cat /etc/logstash/conf.d/log3.conf
input {
tcp {
port => 514
type => syslog
}
udp {
port => 514
type => syslog
}
}

filter {

}

output {
stdout {}
}

启动logstash

# cd /usr/share/logstash/bin/
# ./logstash -f /etc/logstash/conf.d/log3.conf  --path.settings /etc/logstash

模拟一条syslog的日志

# logger -T -P 514 -n 127.0.0.1 'hello world '

查看信息

可以在执行 ./logstash -f /etc/logstash/conf.d/log2.conf  --path.settings /etc/logstash 的界面看到打印日志

2018-01-24T06:22:55.969Z 127.0.0.1 <5>Jan 24 14:22:55 root: hello world

四、logstash作为syslog-server监听syslog日志信息，并将日志对接到elasticsearch

配置

# cat /etc/logstash/conf.d/log4.conf
input {
tcp {
port => 514
type => syslog
}
udp {
port => 514
type => syslog
}
}

filter {

}

output {
elasticsearch {
action => "index"
hosts  => "192.168.5.60:9200"
index  => "lyh-test"
}
stdout {}
}

启动logstash

# cd /usr/share/logstash/bin/
# ./logstash -f /etc/logstash/conf.d/log4.conf  --path.settings /etc/logstash

模拟一条syslog的日志 

# logger -T -P 514 -n 127.0.0.1 'hello world '

查看信息

可以在执行 ./logstash -f /etc/logstash/conf.d/log2.conf  --path.settings /etc/logstash 的界面看到打印日志

2018-01-24T06:22:55.969Z 127.0.0.1 <5>Jan 24 14:22:55 root: hello world

获取elasticsearch的索引

增加了一条lyh-test的的索引

# curl -X GET 192.168.5.60:9200/_cat/indices?v
health status index    uuid                   pri rep docs.count docs.deleted store.size pri.store.size
yellow open   lyh-test nWx7hdNqQOStFbEVXd8tYQ   5   1          5            0     27.7kb         27.7kb

获取该索引里面的数据

# curl -X GET -H 'Content-type: application/json' 192.168.5.60:9200/lyh-test/_search -d '{
"query": {
"match_all": {}
}
}'

# 通过size from sort进行分页排序查找
# curl -X GET -H 'Content-type: application/json' http://192.168.5.60:9200/ssp-attacklog--*/_search?size=10\&from=1\&pretty -d '{
"query": {"match_all": {}},
"sort": {
"happentime": {"order": "desc"}
}
}'

五、下载地址

logstash: https://www.elastic.co/downloads/logstash
elasticsearch: https://www.elastic.co/downloads/elasticsearch
kibana: https://www.elastic.co/downloads/kibana
更多logstash filter相关的可以参见官网
https://www.elastic.co/guide/en/logstash/current/config-examples.html