Disaster Recovery Planning in SEM

10/30/2018 6:45 PM

10/30/2018 6:45 PM

This document describes how to plan or test the disaster recoverability of Security Event Manager (formerly Log & Event Manager).

Sometimes an auditor demands to validate the ability to either restore the database (in part or whole).

Security Event Manager

Restore the database 

Notes: 

• Read through entire procedure before following each step.
• An auditor could require proof that you can recover data, or ability to restore beyond the 'required' 6 months direct online access to logs.
• Procedure will not impact the current running SEM, which is collecting and reporting events/alerts.
• Procedure assumes database backup is available from a network share (archiveconfig).
• Procedure assumes a single SEM appliance deployment. (Contact support if SEM is a multi-server deployment.)

 

1. Access the Solarwinds customer portal to download an 'evaluation SEM'. You want to download the same SEM version as your current online running SEM. An evaluation SEM is the same image, just not licensed for production, so functionality will stop 30 days after deployment..
2. Deploy the new SEM as outlined in the Installation Guide, except you will not be installing the license.
3. Notes about the default SEM deployment (this is only for testing and running reports):
   • Sets (reserves) 8GB of RAM  --> does not need to match the production SEM
   • Sets (reserves) 2-CPU's at 2.0Ghz  --> does not need to match the production SEM
   • VM will reserve 250GB of storage space  --> only need to size for what is restored  depends upon the amount of the database to be restored (number of partitions)
   • Sets the default hostname as "swi-sem"  -->  verify unique name, different from production SEM. use CMC 'hostname' command to change.
   • Sets the SEM to obtain DHCP address  -->  use CMC 'netconfig' command to change.
   • Sets "password" as the default CMC user password  -->  use CMC 'password' command to change. Change size/RAM/CPU while shutdown, reboot the SEM (from CMC/putty) after making network changes
4. The QuickStart Guide has you open a browser to access the SEM, and install the license. Do not do this part. When the first event comes into the SEM, SEM will create the the database.
5. Although you can run the "import" command from the CMC/SSH access, avoid doing this. This will put this test SEM online with the same hostname and IP address as the production SEM. The "import" command is used to recover configurations on the production SEM that were accidentally deleted or changed.
6. Verify that you have access to the backups on a network share.
7. Contact Solarwinds support to perform either the actual restore of the database.

Procedure to recover the entire SEM 

Notes: 

• Read through entire procedure before following each step.
• Procedure assumes the production SEM is offline or missing from the network or virtual environment.
• Procedure assumes the database backup is available from a network share (archiveconfig).
• Procedure assumes the configuration backup is available from a network share (backupconfig).

 

1. Very important: Access the Solarwinds customer portal to download the same SEM version as the production SEM.
2. Deploy the new SEM as outlined in the Installation Guide.
3. Before starting up this 'new' replacement SEM, be sure to set the resources to be reserved, just like the production SEM.
   • Reserve the same amount of RAM  --> somewhere between 8 to 256GB of RAM
   • Reserve the same number of CPU (at 2.0Ghz)  --> somewhere between 2 to 16 CPU (core/processors)
   • Increase the VMware (or Hyper-V)   -->  restoring the entire database
4. After starting the SEM, contact Solarwinds support (or Customer Service) to have the license reset.
5. Open a browser to the SEM, and install the license per the Quick Start.

   (Browser access is available on ports 8443 and 8080. Once the 2 step activation is performed, which includes running the "activate" command under CMC/SSH access, port 8080 will be blocked. Port 8080 can be re-opened by using the "togglehttp" command.)

6. Open a Vsphere-console (or putty/SSH to SEM), enter the "appliance" menu, enter the "import" command, and point to the network share where the configuration backups reside (backupconfig command configurations). Then reboot the SEM from the CMC user access.
7. Although you can run the "import" command from the CMC/SSH access, avoid doing this. This will put this test SEM online with the same hostname and IP address as the production SEM. The "import" command is used to recover configurations on the production SEM that were accidentally deleted or changed.
8. Reboot the SEM from the CMC command.
9. Verify that you have access to the database backups on a network share.
10. Contact SolarWinds support to perform the actual restore of the database.