ELK实战之Tomcat的json日志收集
2017-12-28 10:13
901 查看
一、安装tomcat
1、下载tomcat
[root@linux-node2 ~]# wget http://apache.fayea.com/tomcat/tomcat-8/v8.5.20/bin/apache-tomcat-8.5.20.tar.gz [root@linux-node2 ~]# tar -zxf apache-tomcat-8.5.24.tar.gz [root@linux-node2 ~]# mv apache-tomcat-8.5.24 /usr/local/tomcat
2、修改tomcat日志格式
[root@linux-node2 ~]# cd /usr/local/tomcat/conf [root@linux-node2 conf ]# cp server.xml{,.bak} [root@linux-node2 conf ]# vim server.xml <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs" prefix="tomcat_access_log" suffix=".log" pattern="{"clientip":"%h","ClientUser":"%l","authenticated":"%u","AccessTime":"%t","method":"%r","status":"%s","SendBytes":"%b","Query?string":"%q","partner":"%{Referer}i","AgentVersion":"%{User-Agent}i"}"/>
3、增加tomcat网页
[root@linux-node2 conf ]# cd ../webapps/ [root@linux-node2 webapps ]# mkdir webdir && cd webdir [root@linux-node2 webdir ]# echo "<h1>welcome to use tomcat</h1>" > index.html [root@linux-node2 conf ]# ../bin/catalina.sh start [root@linux-node2 conf ]# netstat -tulnp |grep 8080 tcp6 0 0 :::8080 :::* LISTEN 2362/java
4、压测页面,生成tomcat的访问日志
[root@linux-node2 conf ]# ab -n1000 -c100 http://192.168.56.12:8080/webdir/index.html [root@linux-node2 ~]# tailf /usr/local/tomcat/logs/tomcat_access_log.2017-12-28.log {"clientip":"192.168.56.12","ClientUser":"-","authenticated":"-","AccessTime":"[28/Dec/2017:09:52:57 +0800]","method":"GET /webdir2/index.html HTTP/1.0","status":"200","SendBytes":"32","Query?string":"","partner":"-","AgentVersion":"ApacheBench/2.3"} {"clientip":"192.168.56.12","ClientUser":"-","authenticated":"-","AccessTime":"[28/Dec/2017:09:52:57 +0800]","method":"GET /webdir2/index.html HTTP/1.0","status":"200","SendBytes":"32","Query?string":"","partner":"-","AgentVersion":"ApacheBench/2.3"} {"clientip":"192.168.56.12","ClientUser":"-","authenticated":"-","AccessTime":"[28/Dec/2017:09:52:57 +0800]","method":"GET /webdir2/index.html HTTP/1.0","status":"200","SendBytes":"32","Query?string":"","partner":"-","AgentVersion":"ApacheBench/2.3"} {"clientip":"192.168.56.12","ClientUser":"-","authenticated":"-","AccessTime":"[28/Dec/2017:09:52:57 +0800]","method":"GET /webdir2/index.html HTTP/1.0","status":"200","SendBytes":"32","Query?string":"","partner":"-","AgentVersion":"ApacheBench/2.3"} {"clientip":"192.168.56.12","ClientUser":"-","authenticated":"-","AccessTime":"[28/Dec/2017:09:52:57 +0800]","method":"GET /webdir2/index.html HTTP/1.0","status":"200","SendBytes":"32","Query?string":"","partner":"-","AgentVersion":"ApacheBench/2.3"} {"clientip":"192.168.56.12","ClientUser":"-","authenticated":"-","AccessTime":"[28/Dec/2017:09:52:57 +0800]","method":"GET /webdir2/index.html HTTP/1.0","status":"200","SendBytes":"32","Query?string":"","partner":"-","AgentVersion":"ApacheBench/2.3"} {"clientip":"192.168.56.12","ClientUser":"-","authenticated":"-","AccessTime":"[28/Dec/2017:09:52:57 +0800]","method":"GET /webdir2/index.html HTTP/1.0","status":"200","SendBytes":"32","Query?string":"","partner":"-","AgentVersion":"ApacheBench/2.3"} {"clientip":"192.168.56.12","ClientUser":"-","authenticated":"-","AccessTime":"[28/Dec/2017:09:52:56 +0800]","method":"GET /webdir2/index.html HTTP/1.0","status":"200","SendBytes":"32","Query?string":"","partner":"-","AgentVersion":"ApacheBench/2.3"} {"clientip":"192.168.56.12","ClientUser":"-","authenticated":"-","AccessTime":"[28/Dec/2017:09:52:57 +0800]","method":"GET /webdir2/index.html HTTP/1.0","status":"200","SendBytes":"32","Query?string":"","partner":"-","AgentVersion":"ApacheBench/2.3"} {"clientip":"192.168.56.12","ClientUser":"-","authenticated":"-","AccessTime":"[28/Dec/2017:09:52:57 +0800]","method":"GET /webdir2/index.html HTTP/1.0","status":"200","SendBytes":"32","Query?string":"","partner":"-","AgentVersion":"ApacheBench/2.3"}
二、配置logstash
1、配置logstash
[root@linux-node2 ~]# vim /etc/logstash/conf.d/tomcat_accesslog.conf input { file { path => "/usr/local/tomcat/logs/tomcat_access_log.*.log" type => "tomcat-accesslog" start_position => "beginning" stat_interval => "2" } } output { elasticsearch { hosts => ["192.168.56.11:9200"] index => "logstash-tomcat5612-accesslog-%{+YYYY.MM.dd}" } file { path => "/tmp/logstash-tomcat5612-accesslog-%{+YYYY.MM.dd}" } }
2、检查logstash的配置语法并重启logstash
[root@linux-node2 tomcat]# /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/tomcat_accesslog.conf -t OpenJDK 64-Bit Server VM warning: If the number of processors is expected to increase from one, then you should configure the number of parallel GC threads appropriately using -XX:ParallelGCThreads=N WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console Configuration OK [root@linux-node2 tomcat]# systemctl restart logstash [root@linux-node2 tomcat]# ps -ef |grep logstash logstash 2527 1 98 09:33 ? 00:00:28 /bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+DisableExplicitGC -Djava.awt.headless=true -Dfile.encoding=UTF-8 -XX:+HeapDumpOnOutOfMemoryError -Xmx1g -Xms256m -Xss2048k -Djffi.boot.library.path=/usr/share/logstash/vendor/jruby/lib/jni -Xbootclasspath/a:/usr/share/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/usr/share/logstash/vendor/jruby -Djruby.lib=/usr/share/logstash/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh org.jruby.Main /usr/share/logstash/lib/bootstrap/environment.rb logstash/runner.rb --path.settings /etc/logstash root 2572 15939 0 09:34 pts/3 00:00:00 grep --color=auto logstash
3、elasticsearch的head插件查看
数据浏览:
注:此处如果无法出现tomcat的数据索引,查看一下是否有权限访问日志。
[root@linux-node2 logs]# ll /usr/local/tomcat/logs/ -d drwxr-x--- 2 root root 4096 12月 28 09:29 /usr/local/tomcat/logs/ [root@linux-node2 logs]# chmod 755 /usr/local/tomcat/logs [root@linux-node2 logs]# ll /usr/local/tomcat/logs/ 总用量 512 -rw-r----- 1 root root 7140 12月 28 09:29 catalina.2017-12-28.log -rw-r----- 1 root root 7140 12月 28 09:29 catalina.out -rw-r----- 1 root root 0 12月 28 09:29 host-manager.2017-12-28.log -rw-r----- 1 root root 284 12月 28 09:29 localhost.2017-12-28.log -rw-r----- 1 root root 0 12月 28 09:29 manager.2017-12-28.log -rw-r----- 1 root root 502039 12月 28 09:47 tomcat_access_log.2017-12-28.log [root@linux-node2 logs]# chmod 644 /usr/local/tomcat/logs/
4、添加到Kibana
相关文章推荐
- ELK实战之收集Nginx的json格式日志
- 收集、分析线上日志数据实战——ELK
- ELK系列~Nxlog日志收集加转发(解决log4日志换行导致json转换失败问题)
- ELK+kafka收集 Nginx与tomcat日志
- 收集、分析线上日志数据实战——ELK
- ELK实战-Logstash:收集rsyslog日志
- ELK实战之Kibana部署及message日志收集
- ELK实战之通过TCP收集日志
- elk部署配置,收集nginx和tomcat日志
- ELK实战之java日志收集
- ELK实战之logstash收集日志写入redis
- ELK实战之使用filebeat代替logstash收集日志
- ELK实战二:日志收集利器rsyslog
- ELK实战篇--logstash日志收集eslaticsearch和kibana
- ELK收集nginx日志并用高德地图展示出IP
- logback配置示例-包含生成json日志,与ELK整合
- 云计算Docker全面项目实战(Maven+Jenkins、日志管理ELK、WordPress博客镜像)
- Python脚本收集腾讯云CDN日志,并入ELK日志分析
- 收集Tomcat异常日志并发送邮件
- Syslog-ng+Rsyslog收集日志:收集Tomcat日志(三)