Nginx LOGGING AND MONITORING

https://www.nginx.com/resources/admin-guide/logging-and-monitoring/

This section describes how to configure logging of errors and processed requests, as well as how to use the runtime monitoring service in NGINX Open Source and NGINX Plus.

In This Section

Setting
up the Error Log

Setting
up the Access Log

Logging
to Syslog

Live
Activity Monitoring

Setting Up the Error Log

NGINX writes information about encountered issues of different severity levels to the error log. The

error_log

 directive
sets up logging to a particular file, 

stderr

,
or 

syslog

 and specifies the minimal
severity level of messages to log. By default, the error log is located at logs/error.log (the absolute
path depends on the operating system and installation), and messages from all severity levels above the one specified are logged.

The configuration below changes the minimal severity level of error messages to log from 

error

 to 

warn

:

error_log logs/error.log warn;

In this case, messages of 

warn

, 

error

 

crit

, 

alert

,
and 

emerg

 levels are logged.

The default setting of the error log works globally. To override it, place the 

error_log

 directive
in the main (top-level) configuration context. Settings in the main context are always inherited by other configuration levels. The 

error_log

 directive
can be also specified at the 

http

, 

stream

, 

server

 and 

location

 levels
and overrides the setting inherited from the higher levels. In case of an error, the message is written to only one error log, the one closest to the level where the error has occurred. However, if several 

error_log

directives
are specified on the same level, the message are written to all specified logs.

Note: The ability to specify multiple 

error_log

 directives
on the same configuration level was added in open source NGINX version 1.5.2.

Setting Up the Access Log

NGINX writes information about client requests in the access log right after the request is processed. By default, the access log is located at logs/access.log,
and the information is written to the log in the predefined combined format. To override the default setting,
use the 

log_format

 directive
to change the format of logged messages, as well as the 

access_log

 directive
to specify the location of the log and its format. The log format is defined using variables.

The following examples define the log format that extends the predefined combined format with the value
indicating the ratio of gzip compression of the response. The format is then applied to a virtual server that enables compression.

http {
log_format compression '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" "$gzip_ratio"';

server {
gzip on;
access_log /spool/logs/nginx-access.log compression;
...
}
}

Another example of the log format enables tracking different time values between NGINX and an upstream server that may help to diagnose a problem if your website experience slowdowns. You can use the following variables to log the indicated time values:

$upstream_connect_time

 – The
time spent on establishing a connection with an upstream server

$upstream_header_time

 – The
time between establishing a connection and receiving the first byte of the response header from the upstream server

$upstream_response_time

 – The
time between establishing a connection and receiving the last byte of the response body from the upstream server

$request_time

 – The
total time spent processing a request

All time values are measured in seconds with millisecond resolution.

http {
log_format upstream_time '$remote_addr - $remote_user [$time_local] '
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent"'
'rt=$request_time uct="$upstream_connect_time" uht="$upstream_header_time" urt="$upstream_response_time"';

server {
access_log /spool/logs/nginx-access.log upstream_time;
...
}
}

Here are some rules how to read the resulting time values:

When a request is processed through several servers, the variable contains several values separated by commas

When there is an internal redirect from one upstream group to another, the values are separated by semicolons

When a request is unable to reach an upstream server or a full header cannot be received, the variable contains “0” (zero)

In case of internal error while connecting to an upstream or when a reply is taken from the cache, the variable contains “-” (hyphen)

Logging can be optimized by enabling the buffer for log messages and the cache of descriptors of frequently used log files whose names contain variables. To enable buffering use the 

buffer

 parameter
of the 

access_log

 directive
to specify the size of the buffer. The buffered messages are then written to the log file when the next log message does not fit into the buffer as well as in some other cases.

To enable caching of log file descriptors, use the 

open_log_file_cache

 directive.

Similar to the 

error_log

 directive,
the 

access_log

 directive defined
on a particular configuration level overrides the settings from the previous levels. When processing of a request is completed, the message is written to the log that is configured on the current level, or inherited from the previous levels. If one level defines
multiple access logs, the message is written to all of them.

Enabling Conditional Logging

Conditional logging allows excluding trivial or unimportant log entries from the access log. In NGINX, conditional logging is enabled by the 

if

 parameter
to the 

access_log

 directive.

This example excludes requests with HTTP status codes 

2xx

 (Success)
and 

3xx

 (Redirection):

map $status $loggable {
~^[23]  0;
default 1;
}

access_log /path/to/access.log combined if=$loggable;

Logging to Syslog

The 

syslog

 utility is a standard
for computer message logging and allows collecting log messages from different devices on a single syslog server. In NGINX, logging to syslog is configured with the 

syslog:

prefix
in 

error_log

 and 

access_log

 directives.

Syslog messages can be sent to a 

server=

 which
can be a domain name, an IP address, or a UNIX-domain socket path. A domain name or IP address can be specified with a port to override the default port, 514. A UNIX-domain socket path can be specified after the 

unix:

 prefix:

error_log server=unix:/var/log/nginx.sock debug;
access_log syslog:server=[2001:db8::1]:1234,facility=local7,tag=nginx,severity=info;

In the example, NGINX error log messages are written to a UNIX domain socket at the 

debug

 logging
level, and the access log is written to a syslog server with an IPv6 address and port 1234.

The 

facility=

 parameter specifies
the type of program that is logging the message. The default value is

local7

.
Other possible values are: 

auth

, 

authpriv

, 

daemon

, 

cron

, 

ftp

, 

lpr

, 

kern

, 

mail

, 

news

, 

syslog

, 

user

, 

uucp

,

local0
... local7

.

The 

tag=

 parameter applies a custom
tag to syslog messages (

nginx

 in
our example).

The 

severity=

 parameter sets the
severity level of syslog messages for access log. Possible values in order of increasing severity are: 

debug

, 

info

, 

notice

, 

warn

, 

error

 (default), 

crit

, 

alert

,
and 

emerg

. Messages are logged
at the specified level and all more severe levels. In our example, the severity level 

error

 also
enables 

crit

, 

alert

,
and 

emerg

 levels to be logged.

Live Activity Monitoring

NGINX Plus provides a real-time live activity monitoring interface that shows key load and performance metrics of your HTTP and TCP upstream
servers as well as other data including:

NGINX version, uptime, and identification information

Cumulative and current numbers of connections and requests

Request and response counts for each status_zone

Request and response counts per server in each dynamically
configured group of servers, plus health-check and uptime statistics

Statistics per server, including its current state and total values (total failures, etc.)

Instrumentation for each named cache
zone

The complete list of metrics is available here.

The statistics can be viewed from the status.html page provided in the NGINX Plus package. The page polls
status information and displays it in a simple table:





Live
load-balancing status from NGINX Plus

And using a simple RESTful JSON interface, it’s easy to connect these stats to live dashboards and third-party monitoring tools.

Enabling Live Activity Monitoring

To enable live activity monitoring and the JSON interface, you must specify the location that checks the exact match of the URI with /status and
contains the 

status

 directive.
To enable the status.html page, you must specify one more location that sets it:

server {
listen 127.0.0.1;
root /usr/share/nginx/html;

location /status {
status;
}

location = /status.html {
}
}

With this configuration, accessing http://127.0.0.1/status.html in a browser on the NGINX host displays
the status.html page located at /usr/share/nginx/html.

Using the RESTful JSON Interface

NGINX Plus stats can be easlily fed to third-party applications, such as performance dashboards. This can be done with the JSON interface.

If you request /status (or whichever URI matches the 

location

 directive),
NGINX Plus returns a JSON document containing the current activity data.

The status information of any element in the JSON document can be requested with a slash-separated URL:

http://127.0.0.1/status http://127.0.0.1/status/nginx_version http://127.0.0.1/status/caches/cache_backend http://127.0.0.1/status/upstreams http://127.0.0.1/status/upstreams/backend http://127.0.0.1/status/upstreams/backend/peers/1 http://127.0.0.1/status/upstreams/backend/peers/1/weight