Monitoring and Troubleshooting Using Event Logs

原文: http://www.windowsnetworking.com/articles_tutorials/Monitoring-Troubleshooting-Event-Logs.html
This article reviews best practices for working with Windows
event logs including how to interpret event messages, how to configure event
logs, how to search and filter events, how to view events on remote systems, and
how to use EventCombMT.exe and other tools to monitor events on multiple
systems.

The event logs on Windows systems are helpful for both troubleshooting when
things go wrong and monitoring performance and behavior. An event log is a file
that contains events, which are entries to the log that notify the user of some
occurrence relating to the operating system or applications running on the
system. An event includes information about the type of occurrence, the date and
time when it occurred, the computer where it happened and the user who was
logged on at the time, and other information such as event ID, the event
category, and the source of the event. Events may also include further detailed
information concerning the event and possibly a link to where more information
can be found. Figure 1 below illustrates an example of an event from the DNS
Server event log on a Windows Server 2003 domain controller:



Figure
1:
Example of an event

Finding More Information About an Event

If an event contains a link and you click on it, a dialog box opens warning
you that information about the event will be sent to Microsoft to see if they
have more information available concerning the event:



Figure
2:
Sending event information to Microsoft.

Clicking Yes opens the Help and Support Center and checks to see if there is
any more information about the event that may be helpful. Figure 3 shows a
typical response:



Figure
3:
Additional help concerning the event.

How many times have you been frustrated by the lack of helpful information
available this way concerning some obscure event? In the example above, the
additional help provided is that “this error could be caused by either a high
load on the domain controller or the failure of other domain controller
services” and the suggested remedy is to “restart the DNS Server service” and
check the event log for anything else that happened at the same time and could
be a clue. In other words, its like the old mantra “when all else fails, try
rebooting.” Where can you find more help?

Altair Technologies
maintains a
helpful site called EventID.net
where
users can search for additional information about obscure Windows events to help
you interpret them. This site is community-based, meaning that users post their
comments concerning events to create a community database that can then be
searched by others. If you search EventID.net for information about the above
event (source = DNS, event ID = 4004) the following is displayed:



Figure
4:
Searching EventID.net for more information about event ID 4004 for
DNS.

The really useful feature is under Details, where you can click the link
“Comments and links for event id 4004 from source DNS” to see comments posted by
other users:



Figure
5:
Comments on event ID 4004 for DNS posted by users of
EventID.net

The last comment is particularly useful as it indicates MS is aware of why
this event occurs and suggests it can usually be safely ignored. Help and
Support never told us that!

Configuring Event Logs

One of the first things you should do after you install a new Windows system
is configure the event logs on that system. This is particularly important for
servers where event logs can provide critical information to help you
troubleshoot when things go wrong. Before we look at how to configure event
logs, we need some background information on the different logs available, and
Table 1 provides this below:

Event log	Log file	Function	Availability
Application log	AppEvent.evt	Records events as determined by each software vendor	All Windows systems
Security log	SecEvent.evt	Records events based on how audit policy is configured	All Windows systems
System log	SysEvent.evt	Records events for Windows operating system components	All Windows systems
Directory Service log	NTDS.evt	Records events for Active Directory	Domain controllers only
DNS Server log	DnsEvent.evt	Records events for DNS servers and name resolution	DNS servers only
File Replication Service log	NtFrs.evt	Records events for domain controller replication	Domain controllers only

Table
1:
Summary of Windows event logs
By default all event logs are:

Stored in the %Windir%/system32/config folder

Have a maximum size of 16 MB (Windows Server 2003) or 512 KB (Windows
2000/XP)

Overwrite events more than 7 days old



Figure
6:
Default configuration of DNS Server event log on a Windows Server
2003 DNS server.

Before you put your new Windows server into production, you should decide if
these default settings are appropriate. Suggested best practices for configuring
event logs on servers include the following:

Increase the size of each event log to at least 50 MB. Since a typical event
is about half a kilobyte in size, this means you’ll be able to store 100,000
events in each log. Note that the maximum supported size of each event log is
about 300 MB. If your system drive has insufficient space for your event logs,
you can move them to a separate volume by editing the subkey for each log under
the HKLM/SYSTEM/CurrentControlSet/Services/Eventlog using Registry Editor, see
Microsoft
Knowledge Base article 315417
for more information.

Change the overwrite behavior for the Security log to Do Not Overwrite
Events if your enterprise is a high security environment. That way if the
Security log fills up the system will shut down to ensure that no events in the
Security log are lost. If you do this, make sure you also archive and then clear
your Security log regularly to prevent such a shutdown from occurring
unexpectedly.

Change the overwrite behavior for the other event logs to Overwrite Events
As Needed so that no overwriting occurs until the entire log becomes full.
Again, be sure to regularly archive and clear your event logs to prevent the log
from filling up and losing events because of overwrites.

If you have a number of computers and are running Active Directory on your
network, you can also use Group Policy to configure event log settings. These
settings are found under Computer Configuration/Windows Settings/Security
Settings/Event Log in Group Policy Object Editor:



Figure
7:
Group Policy settings for configuring event logs.

Searching and Filtering Events

While scrolling through the Event console lets you easily examine the most
recent events that have been logged on your system, this quickly becomes
impractical on busy systems where event logs are tens of megabytes in size. If
you’re looking for instances of a particular kind of event however, you can use
the Find and Filter options to speed things up.

Say you want to find all instances of Event ID 4004 in the DNS Server log as
shown previously in Figure 1 above. To use the Find feature to accomplish this,
right-click on the DNS Server log and select View --> Find, then fill in the
Event ID and log name in the Find box:



Figure
8:
Finding instances of Event ID 4004 in the DNS Server
log.

Click the Find Next button and the first instances of this event is displayed
in Event Viewer:



Figure
9:
An instance of Event ID 4004 displayed in Event
Viewer.

Then click Find Next to display the next instance of this event, and so on.

The frustrating thing about this approach is that the Find interface is not
built directly into the Event Viewer window. So let’s try a different approach
and use Filter instead. Right-click the DNS Server log again and select View
--> Filter, then fill in the Event ID in the Filter tab of the DNS Server
Properties sheet:



Figure
10:
Filtering the DNS Server log for Event ID 4004.

Click OK and Event Viewer and the only events displayed in the DNS Server log
are those having Event ID 4004:



Figure
11:
All instances of Event ID 4004 are displayed.

From this information we could conclude that this was only a transient
problem that happened a couple of weeks ago when we rebooted the DNS server.

Viewing Events on Remote Systems

Event Viewer also lets you connect to remote systems to view their event
logs. The procedure is simple: right-click on the root (top) node in the console
tree of Event Viewer and select Connect To Another Computer:



Figure
12:
Connecting to a remote computer to view its event
logs.

Then either type the name (NetBIOS or FQDN) of the remote computer or click
Browse to find it in Active Directory. Click OK to connect:



Figure
13:
Can’t connect to a remote computer to view event
logs.

Oops, can’t connect! And the error message is cryptic. What went wrong?
Typically this error message either indicates one of the following:

You are not logged on with an account that has local Administrator access to
the remote machine (a Domain Admins account should work).

The Remote Registry service is not running or has been disabled on the
remote machine.

Correct the situation and you should be able to connect to the remote machine
and view its event logs.

Using EventCombMT.exe

In a previous
article
on WindowsSecurity.com we looked at the Account Lockout and
Management Tools (ALTools.exe) download from Microsoft. One of these tools is
EventCombMT.exe, which can be used to consolidate event logs from multiple
computers into a single location for analysis. To use this tool double-click on
EventCombMT.exe in the folder where you installed it, then specify the domain,
servers, and kinds of events you want to find. For example, say you want to find
all W32Time events on two servers (TEST230 and TEST235) in the testtwo.local
domain:



Figure
14:
Using EventCombMT to search for W32Time events on two
servers.

Click Search and when the operation is finished a folder will open up
displaying the results files generated:



Figure
15:
Results files generated by our EventCombMT search.

Double-clicking on one of the two server files displays a comma-delimited
list of W32Time events for that server:



Figure
16:
Comma-delimited list of W32Time events on server
TEST230.

You could then import these files into Excel to consolidate them for further
analysis. EventCombMT also has a number of built-in queries you can use for
common tasks like searching for locked-out accounts:



Figure
17:
Searching for locked-out accounts using
EventCombMT.

Other Event Monitoring Tools

EventCombMT.exe is useful but it isn’t very friendly to use. If you have a
lot of computers whose event logs you want to monitor, you’re better off
purchasing a commercial-quality tool for this purpose. We’ll end this article by
mentioning two such tools:

Microsoft Operations Manager (MOM)

MOM is a Windows Server System product from Microsoft that lets you monitor
events, health, and performance of computers on your network in real-time,
consolidate such information in a central repository, and generate graphical
web-based reports. MOM 2000 is showing its age however and will soon to be
replaced by MOM 2005, which has a new Operator Console, greater security,
enhanced rules, and improved reports. MOM 2005 also supports agentless
monitoring, internationalization, and 64-bit agent support. For more information
see this link
on
Microsoft’s web site.

GFI LANguard SELM

GFI LANguard Security Event Log Monitor
(SELM)
is a tool from GFI
that lets you
manage event logs on remote machines, consolidate event logs from multiple
machines into a single repository, and view, report and filter events
network-wide with easy and simplicity. You can also create your own custom
alerts based on event ID, contents, and event condition so you can monitor
specify issues across your network. SELM even lets you analyze event details,
something MOM won’t let you do. GFI products are excellent--I speak from
personal experience here--so this is one solution you should consider when
looking for tools to monitor and troubleshoot event logs across your network.