Troubleshoot excessive rule activity, duplicate emails, and email templates in SEM

This article describes how to address excessive rule activity, duplicate emails from Security Event Manager and how to determine the email template used.

Perform an Historical Events & Reports search for the time frame that the email was sent. Allow five minutes before and after the email was sent:

1. Go to Historical Events & Reports
2. Dropdown Refine fields and select Events.
3. Type InternalRuleFired in the search field.
4. Click and drag the results to the top and change the type frame.
5. In Historical Events & Reports "search in results field"  type "email"
6. Check each rule fired in the ExtraneousInfo line to determine only the rules that fired the emails.
7. Go to the Rules Tab.
8. Search the name of the first rule.
9. Verify the email template used in the Action section.
10. Go to Configure > Email templates.
11. Locate the name of the email template that matches the rule. 
12. Perform the same steps for the second rule that fire.

Your rules should be firing less than a 200-300 times a day.  For rules that are busy, such as for UDP, TCP, ICMP, the default is 10 events in 10 seconds.  This is not many events at all, if you had about 10 computers this would be fine but if you have several hundred or more than a thousand workstations and you are logging every build up and teardown for UDP and TCP events these can get very busy rules. 

1. Under Rules, find the exact rule that fired.
2. For each rule, change the number of events to 100.
3. If the rules are still firing every second:
   1. Go to Live Events > Overview > Rule Activity.
   2. Increase the number by 50 but stop at 200. The rule may be firing on a trusted IP address that is known. If so, modify the rule by adding the IP address to the Correlation box:
      1. Go to Rules.
      2. Double-click the rule.
      3. Click Events.
      4. Type the event in the search box.
      5. Click and drag DetectionIP to the Correlations box.
      6. Type the trusted IP address. This will stop the rule from firing on the that IP address.

Another issue with rules firing so often can be if you turned on Suspicious DNS rule but never added in your Approved DNS Servers. To do this, perforn the following:

1. Go to Configure > User Defined Groups.
2. In the search field, type DNS.
3. Edit the Approved DNS Servers.
4. Click Add element to add the DNS servers.
5. Type a name/hostname in the name box type.
6. Type the exact IP address of your DNS server in the Value box.
7. Click Add.
8. Perform the steps for every DNS server.

Sometimes you will have rules fire for users that do something and sometimes you have rules fire when a computer or machine name changes something from Group Policy.  To prevent rules from firing when a machine makes a change:

1. Go to Rules.
2. Select the rule you want to modify and click edit.
3. Click Event Groups > Auditable Domain Events.
4. Under fields, click and drag DestinationAccount  to the "Rule is true when" area.  Add it as an "AND" corrolation.
5. Ensure it is set to "is not equal to " *$*

Duplicate Email Active Response Connectors

SEM supports having multiple Email Active Response connectors setup. This would allow you to have rules that fire off alert emails to different servers based on need (e.g., an internal relay for sensitive alerts, and a normal server for urgent alerts.) Depending on your environment this could cause you to receive duplicate emails from SEM.

1. Go to Configure > Manager Connectors
2. Verify that that is only one Email Active Response connector
3. Remove any duplicate Email Active Response connectors.

Note: If you have logs replicating between your Domain Controllers, the SEM is seeing logs from both servers. Either stop replicating logs between the servers or increase the number of events in the rule.