The number of days' worth of live data that the LEM database will store varies for every implementation. The information below should help you determine this number for your environment, while also promoting a more detailed understanding of how the database works in general.
All LEM versions
What the LEM Database Stores
By default, the LEM appliance allocates up to 230 GB of the 250 GB configured for the LEM virtual appliance. This partition consists of three data stores:
Alert database (Alert store) - Default normalized data
The Alert store consists of all of the normalized Events/Alerts collected by the LEM Manager and LEM Agents. Data in this store is compressed at a ratio of 40:1 to 60:1, which equates to an average compression rate of about 95-98%. LEM reports and nDepth queries this store for Alert data whenever they are run.
RAW database (Original log store) - Optional original data storage if enable for auditing requirements
The original log store is an optional store for original, or RAW, log messages, which is searchable using Log Message queries in nDepth. The data in this store can come from LEM Agents or other devices that are logging to the LEM appliance. You can define whether data is sent to this store at the connector level, so connector configurations define devices/logs that are sent to the RAW database.
Temporary Syslog data (Syslog store) or SNMP data log storage from network devices logging directly to the LEM
TheSyslog store consists of all Syslog/SNMP log data that is sent to the LEM appliance. The LEM appliance reads and processes the data in real time, and then sends it to the Alert database for long-term storage. The LEM appliance stores the original data into files for 50 days in its original format, just in case you need to review it. The LEM rotates and compresses the data in the Syslog store daily, but can be changed to accommodate high volume devices that may exceed size limits for zipping files. You can configure Syslog to rotate the Syslog store hourly, and keep 1 to 100 of these log files.
Where to Find the Numbers
There are three primary sources for statistics related to how your LEM database is being used: the Disk Usage summary in the CMC, the Database Maintenance Report, and the Log Storage Maintenance Report.
Disk Usage Summary
When you initially log into your LEM virtual appliance using the vSphere "console" view or an SSH client such as PuTTY, the LEM appliance automatically generates a Disk Usage summary. You can also generate an ad hoc Disk Usage summary by running the diskusage command from the cmc::acm# (cmc > appliance) prompt. The two lines to note here are:
Database Maintenance Report
Run the Database Maintenance Report in LEM Reports to see a snapshot of your current database utilization. For the sake of this discussion, note the following sections:
Note: The Other Files figure in the Database Maintenance Report consists primarily of the data in the Syslog store noted above.
Log Storage Maintenance Report
Run the Log Storage Maintenance Report in LEM Reports to get detailed information about the original log store. If you have not enabled your LEM appliance and tools to store original log messages, this report will be blank by default, unless the RAW database was enabled and connectors configured to send data to this database.
Alternate Storage Methods
Depending on the needs of your environment, you might want to utilize one or more of the alternate storage methods listed below. For more details or assistance with any of these methods, please open a ticket with Support.
ArchiveConfigcommand to configure the Alert database backups. The first time the database is backed up, the entire Alert database is placed onto a network share. Subsequent backups are incremental/differential backups.
LogMArchiveConfigcommand to configure the RAW database backups. The first time the database is backed up, the entire RAW database is placed onto a network share. Subsequent backups are incremental/differential backups.
LogBackupConfigcommand to configure the Syslog backups.
limitsyslog) to adjust the storage space used.