Submit a ticketCall us

Welcome to the NEW Success Center. Search all resources (documentation, videos, training, knowledge base articles) or browse resources by product. If you are unable to find what you are looking for, please contact us at customersuccess@solarwinds.com

 

 

 

 

Home > Success Center > Log & Event Manager (LEM) > Troubleshoot excessive rule activity, duplicate emails, email templates

Troubleshoot excessive rule activity, duplicate emails, email templates

Table of contents
Created by David Clark, last modified by MindTouch on Jun 23, 2016

Views: 975 Votes: 3 Revisions: 12

Overview

This article describes how to address excessive rule activity and duplicate emails from LEM and how to determine the email template used.

Environment

All LEM versions

Steps

  1. Perform an nDepth search for the time frame that the email was sent. Allow five minutes before and after the email was sent.
  2. Go to Explore > nDepth.
  3. Click Events (yellow triangle).
  4. Type InternalRuleFired in the search field.
  5. Click and drag the results to the top and change the type frame.
  6. In nDepth search, click Refine Fields (blue funnel) > InferenceRule.
  7. Check each rule fired in the ExtraneousInfo line to determine only the rules that fired the emails.
  8. Go to Build > Rules.
  9. Type the name of the first rule.
  10. Verify the email template used in the Action section.
  11. Go to Build > Groups.
  12. Click Type  and choose Email Template.
  13. Locate the name of the email template that matches the rule. 
  14. Perform the same steps for the second rule that fire.

 

Your rules should be firing less than a 1,000 times a day.  For rules that are busy, such as for UDP, TCP, ICMP, the default is 10 events in 10 seconds.  This is not many events at all, if you had about 10 computers this would be fine but if you have several hundred or more than a thousand workstations and you are logging every build up and teardown for UDP and TCP events these can get very busy rules. 

  1. Under Build > Rules, find the exact rule that fired.
  2. For each rule, change the number of events to 100.
  3. If the rules are still firing every second:
    1. Go to Monitor > Overview > Rule Activity.
    2. Increase the number by 50 but stop at 200. The rule may be firing on a trusted IP address that is known. If so, modify the rule by adding the IP address to the Correlation box:
      1. Go to Build > Rules.
      2. Double-click the rule.
      3. Click Events.
      4. Type the event in the search box.
      5. Click and drag DetectionIP to the Correlations box.
      6. Type the trusted IP address. This will stop the rule from firing on the that IP address.

 

Another issue with rules firing so often can be if you turned on Suspicious DNS rule but never added in your Approved DNS Servers. To do this, perforn the following:

  1. Go to Build > Groups.
  2. In the search field, type DNS.
  3. Click the gear icon beside Approved DNS Servers and select Edit.
  4. Click + on the bottom left to add the DNS servers.
  5. Type a name/hostname in the name box type.
  6. Type the exact IP address of your DNS server in the Data box.
  7. Click Save.
  8. Perform the steps for every DNS server.

 

Sometimes you will have rules fire for users that do something and sometimes you have rules fire when a computer or machine name changes something from Group Policy.  To prevent rules from firing when a machine makes a change:

  1. Go to Build > Rules.
  2. Double-click on the rule you want to modify.
  3. Click Event Groups > Auditable Domain Events.
  4. Under fields, click and drag DetectionIP to the Correlations box.
  5. Type *$* in the blank box.

 

 

Note:  You can also click on the specific InternalRule fired either from the Monitor Filter for Rule Activity or from the nDepth query that you did above and on the top right hand-side, click on Explore above the time frame like Last 10 minutes and select Event.  What this shows you is the events that caused the rule to fire. Also click on the event and Event Details at the top.

 

NOTE: If you have logs replicating between your Domain Controllers, the LEM is seeing logs from both servers.  Either stop replicating logs between the servers or increase the number of events in the rule.

 

 

 

Last modified
20:23, 22 Jun 2016

Tags

Classifications

Public