Security Compliance

The reason why SEM needs memory and CPU resource reservations

This article explains why SEM needs memory and CPU resources set in the form of reservations.

First published date

11/29/2018 10:36 PM

Last published date

5/17/2022 4:31 AM

Overview

The reasons for the SEM's requirement for CPU and Memory reservations are detailed out below.

Product section

Security Event Manager

Cause

N/A

Resolution

SEM requires resource reservations in the form of CPU and RAM, just as other appliances like routers and firewalls have 'dedicated' resources. 

SEM handles traffic in real time, receiving and normalizing data, firing rules when a specific type of data is received, displaying data in the console, pushing data into the database, allowing nDepth searches for that data, and allowing the Reports application to retrieve data from the database.

A lot of the communication that makes this happen is Java based, and because of the way SEM utilizes memory, it needs more RAM than other devices, and cannot allow other VMs to take the memory in use by SEM. In addition, SEM commonly receives a constant stream of log data from multiple sources, so there is very little recovery time for SEM to 'catch up'. SEM is very unique in how it operates, along with the different types of data received, so reservations of memory and CPU are critical.

 

By the nature of virtualization, all VM administrators want to share resources and not dedicate resources. Unfortunately, because of what SEM does, memory and CPU reservations are critical to keeping SEM running correctly.  And SEM is unique enough, that viewing the performance measures in vSphere or Hyper-V does not always give a clear indication of whether SEM needs more or fewer reservations. The need for more reservations is based primarily on the volume of traffic received, but also on configurations set in SEM, or the type of data received.

While a shortage of RAM can cause excessive swapping and balooning, or out of memory errors, it's still possible to experience instability, performance problems, or even file system corruption. In vSphere, always set the CPU limits and RAM limits to match the reservations. In Hyper-V, always use 100%/100%/100% or the slider bars set to maximum so that the the reservations match the allocations/limits.

The following chart is a guideline for setting reservations, but it is very dependent upon the type of traffic and the configurations set in SEM. The number of CPU's can vary, depending upon the type, date, and technologies used by the manufacturer. In addition, the storage speed would need to be sufficient to handle the traffic. VM administrators can help in this area, and SolarWinds support can run some speed tests to verify the speed as well.
 

 < 15       million  Events-Per-Day 8GB          Ram 2-CPU  @2Ghz 
 15-35     million  Events-Per-Day 10-16GB   Ram   4-CPU  @2Ghz  
 35-60     million  Events-Per-Day 20-32GB   Ram 6-CPU  @2Ghz
 60-90     million  Events-Per-Day   32-48GB   Ram 8-CPU  @2Ghz
 90-120   million Events-Per-Day  48-64GB   Ram 10-CPU @2Ghz
 120-150 million Events-Per-Day  64-96GB   Ram 12-CPU @2Ghz
 150-200 million Events-Per-Day  96-128GB Ram 14-CPU @2Ghz

 

These links are essential to understanding SEM:

 

Additional Links:

  • Full-memory reservation eliminates ballooning or hypervisor swapping, which leads to more predictable performance with no latency overheads.  For more information, click here. (© 2013 VMware, Inc., available at http://www.vmware.com/, obtained on May 4, 2016.)

  • Set the memory reservation value in the VMware Infrastructure Client to the size of memory for the virtual machine. Find more information here.   (© 2013 VMware, Inc., available at http://www.vmware.com/, obtained on May 4, 2016.)

  • Without memory reservation, vSphere may reclaim memory from the VM when the host free memory gets scarce. Find more information here. (© 2013 VMware, Inc., available at http://www.vmware.com/, obtained on May 4, 2016.)

  • If a performance-critical virtual machine needs a guaranteed memory allocation, the reservation needs to be specified carefully because it may impact other virtual machines. To find more inforation, visit http://www.vmware.com/content/dam/di...management.pdf. (© 2013 VMware, Inc., available at http://www.vmware.com/, obtained on May 4, 2016.)

Using the VM performance metrics to define what is needed for the SEM will not help keep the SEM running. Without reservations, the virtual appliance will have stability issues and performance problems, and expect the SEM to go down periodically.