Awstats流量监控nginx日志全过程

Awstats流量监控
首先下载awstats7.0
http://cdnetworks-kr-2.dl.sourceforge.net/project/awstats/AWStats/7.0/awstats-7.0.tar.gz
安装，路径选择网站根目录建立awstats
我的路径/usr/local/www/php/code/awstats
把安装包考到linux下（位置自己定）
1．找到awstats-7.0.tar.gztar –zxvf awstats-7.0.tar.gz #解压安装包2．mv awstats-7.0/usr/local/www/php/code/awstats/awstats
#移动到网站根目录
3．cd /usr/local/www/php/code/aestats
perl awstats_configure.plperl awstats_configure.pl ☆安装实例Perlawstats_configure.pl----- AWStats awstats_configure 1.0 (build 1.9) (c) Laurent Destailleur -----This tool will help you to configure AWStats to analyze statistics forone web server. You can try to use it to let it do all that is possiblein AWStats setup, however following the step by step manual setupdocumentation (docs/index.html) is often a better idea. Above all if:- You are not an administrator user,- You want to analyze downloaded log files without web server,- You want to analyze mail or ftp log files instead of web log files,- You need to analyze load balanced servers log files,- You want to 'understand' all possible ways to use AWStats...Read the AWStats documentation (docs/index.html).-----> Running OS detected: Linux, BSD or UnixWarning: AWStats standard directory on Linux OS is '/usr/local/awstats'.If you want to use standard directory, you should first move all contentof AWStats distribution from current directory:/data0/htdocs/winner/awstatsto standard directory:/usr/local/awstatsAnd then, run configure.pl from this location.Do you want to continue setup from this NON standard directory [yN] ? y开始安装awstats-----> Check for web server installEnter full config file path of your Web server.Example: /etc/httpd/httpd.confExample: /usr/local/apache2/conf/httpd.confExample: c:\Program files\apache group\apache\conf\httpd.confConfig file path ('none' to skip web server setup):> none确定web服务配置文件，因为应用nginx所以不定义Your web server config file(s) could not be found.You will need to setup your web server manually to declare AWStatsscript as a CGI, if you want to build reports dynamically.See AWStats setup documentation (file docs/index.html)-----> Update model config file '/data0/htdocs/winner/awstats/wwwroot/cgi-bin/awstats.model.conf'File awstats.model.conf updated.-----> Need to create a new config file ?Do you want me to build a new AWStats config/profilefile (required if first install) [y/N] ? y建立一个awstats的配置文件-----> Define config file name to createWhat is the name of your web site or profile analysis ?Example: www.mysite.comExample: demoYour web site, virtual server or profile name:> 113.105.93.6输入你的域名-----> Define config file pathIn which directory do you plan to store your config file(s) ?Default: /etc/awstatsDirectory path to store config file(s) (Enter for default):> 确定配置文件路径-----> Create config file '/etc/awstats/awstats.113.105.93.6.conf'Config file /etc/awstats/awstats.113.105.93.6.conf created.-----> Add update process inside a schedulerSorry, configure.pl does not support automatic add to cron yet.You can do it manually by adding the following command to your cron:/data0/htdocs/winner/awstats/wwwroot/cgi-bin/awstats.pl -update -config=113.105.93.6Or if you have several config files and prefer having only one command:/data0/htdocs/winner/awstats/tools/awstats_updateall.pl nowPress ENTER to continue... 回车A SIMPLE config file has been created: /etc/awstats/awstats.113.105.93.6.confYou should have a look inside to check and change manually main parameters.You can then manually update your statistics for '113.105.93.6' with command:> perl awstats.pl -update -config=113.105.93.6You can also build static report pages for '113.105.93.6' with command:> perl awstats.pl -output=pagetype -config=113.105.93.6Press ENTER to finish...#启动awstats安装向导
A．第一个选项选择none因为默认支持apache，nginx选择none（确定配置文件路径）B．然后选Y创建一个新的配置文件C．这里输入自己的域名D．第三个选项是配置文件路径，直接回车使用默认路径E．回车F．回车（完成）4．修改awstats配置文件
Vi/etc/awstats/awstat.域名.conf
找到LogFile="/var/log/httpd/mylog.log"
咱们自己修改成自己的日志位置
LogFile="/usr/local/www/weblogs/nginx_%YYYY-24%MM-24%DD-24.log"
取到当前日期的前一天的日期
5．修改nginx.conf的日志文件格式
Vi/usr/local/nginx/conf/nginx.conf
log_formatmain'$remote_addr - $remote_user [$time_local] "request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log/usr/local/www/weblogs/www.test.com.access.logaccess;
6.好，现在差不多了，切割nginx日志
编写个sh吧

LOGS_PATH=/usr/local/www/weblogs #定义日志路径
YESTERDAY=$(date -d "yesterday" +%Y%m%d)取一个昨天的日期
mv${LOGS_PATH}/www.purcotton.com.access.log ${LOGS_PATH}/nginx_access/access_${YESTERDAY}.log #移动新日志到指定位置并命名为昨天的日期
kill -USR1 $(cat /usr/local/nginx/logs/nginx.pid) #使nginx切换至新日志文件，这个不需要太多的时间瞬间就搞定
执行此脚本
7.初始化生成静态页面（这个操作会耗时比较大，它会一条一条的去检查这些日志）
/安装路径/ awstats/wwwroot/cgi-bin/awstats.pl -update -config=113.105.93.6
☆安装实例
执行后的输出↓我的log大小在30个G多，具体熬了多久就不知道了，第二天一看就好了估计几个小时吧
Create/Update database for config "/etc/awstats/awstats.113.105.93.68.conf" by AWStats version 7.0 (build 1.971)
From data in log file "/data1/logs/nginx_access/access_20110425.log"...
Phase 1 : First bypass old records, searching new record...
Searching new records from beginning of log file...
Phase 2 : Now process new records (Flush history on disk after 20000 hosts)...
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Flush history file on disk (unique hosts reach flush limit of 20000)
Jumped lines in file: 0
Parsed lines in file: 131668109
Found 2 dropped records,
Found 0 comments,
Found 0 blank records,
Found 170823 corrupted records,
Found 0 old records,
Found 131497284 new qualified records.
Ok 已经生成
8.更新数据的代码
/usr/local/www/php/code/liuliang/awstats/tools/awstats_buildstaticpages.pl
-update
-config=113.105.93.6
-dir=/usr/local/www/php/code/liuliang/awstats/log/
-lang=cn -awstatsprog=/usr/local/www/php/code/liuliang/awstats/wwwroot/cgi-bin/awstats.pl
9.到此通过绝对路径可以访问到，
加密方法如下通过apache的htpasswd工具生成一个密钥包
这个工具的路径自己查找
/usr/bin/htpasswd-cadmin.pass admin
生成一个admin.pass的文件
修改nginx.conf
location ~^/log/ {#重定向好做加密
root /usr/local/www/php/code/liuliang/awstats;#定向的目录路径
index index.html;#支持的文件类型
access_log off;#关掉日志
error_log off;
auth_basic "admin";#加密
auth_basic_user_file /tools/admin.pass;#密钥路径（位置放哪里写哪里）
}
location ~^/icon/ {#用到的图片路径
root /usr/local/www/php/code/liuliang/awstats/wwwroot;#
index index.html;
access_log off;
error_log off;
#charser gd2312;
}
这样，在登录http://www.name.com/log/awstats.www.name.com.html就需要输入密码进行验证了
Awstats安装到此结束