Recover unbootable SEM without root access

• In some instances, during the boot process of your SEM or LEM appliance, you may come across a PXE boot error. This occurs when the SEM OS Partition (hard disk1) has been unintentionally deleted from the VMWare V-Center console. Once the VM is powered on, you will encounter the following PXE error message

PXE-M0F: Exiting Intel PXE ROM. 
"Operating System not found"

• This article can also be used for SEM/LEM data migration where both the VMs are on the same virtualization host. Likely VMware to VMware or Hyper-V to Hyper-V. 

• While resizing the Log/Data partition, user accidentally deletes the SEM(LEM) OS disk/partition (hard disk1) from VMWare V-Center console. Below message is displayed while booting the SEM appliance. 
• Data migration of SEM/LEM, involving virtual machines (VMs) hosted on the same virtualization platform

Scenario 1: When OS partition is deleted/Corrupted and SEM is unbootable (Disaster Recovery Secenario)

1. Shutdown the old SEM VM appliance and change it's IP address.
2. Preparing new SEM:
   • Deploy a new SEM VM with default deployment template size on the same datastore where the old SEM is present. You can follow any of the available deployment methods mentioned in this KB - Refer: Install SEM on the hypervisor and the cloud
   • Boot into the new SEM VM and perform a graceful shutdown via CMC console
     • cmc::appliance > shutdown [Default CMC credentials: username: cmc password: password]
   • Edit new SEM's VM Settings and delete/detach the Log/Data partition (hard disk2). 
3. Move/Migrate/Copy Data partition of Old SEM:
   • Browse to VMWare datastore and move the (hard disk2 and all relevant disk files) of the old SEM VM to the newly deployed SEM folder. 
4. Modify Settings of new SEM VM:
   • On V-Center console, select the new SEM VM > Go to settings > Attach the moved "hard disk 2" (logs/data partition) to your old SEM VM to the new SEM VM as Hard Disk 2 by selecting the relevant .vmdk file(s) 
   • Proceed further and save changes.
5. Booting into new SEM:
   • Power On or Start your new SEM VM and let it boot.
   • During the boot process, it will detect the attached 2nd drive and continue.
   • (Or) SEM will perform any pending "resizefs" action in case you performed a SEM Resize on hard disk 2 and the action was pending.
6. Configure Network Settings:
   • On new SEM. reconfigure the same network settings that you had for your old SEM (IP address, subnet and DNS servers) - Configure network settings
   • Log into the Web console to validate 
7. Migrate License:
   • Contact SolarWinds Customer service and get the SEM license reset done - This will take 20 to 30 minutes
   • Apply the license to new SEM.

• Once logged into the SEM Web Console, you will come across "Error loading Event details issue". To fix this, you need to contact SolarWinds SEM Technical support by raising a new case on the customer portal here
• Missing Configurations like (rules/custom filters,etc) - You can use SEM backupconfig from previous backups (conditioned, the backup was off the same version of SEM). 

Scenario 2: SEM Data Migration from VM to VM (VMware). 

1. Backup source SEMs configurations: 
   • Run backupconfig command as per Configure Backups on your SEM Appliance
2. Prepare destination new SEM: 
   • Deploy a new SEM VM - Install SEM on the hypervisor and the cloud 
     • Note: Deploy the same version of as that of the source/old SEM.
   • Post deployment, allocate temp IP > Validate if the web console is accessible or not
3. Match Source SEM configuration:
   • Shutdown the new SEM
   • Edit the new VM settings and perform a SEM Resize in-terms of CPUs/RAM/Disk to match the configurations of your source/old SEM.  
   • Power up and let it boot. 
   • Restore backed up backupconfig on new SEM via cmc>appliance>import
4. Move-Migrate Logs/data partition (hard disk2):
   • From Scenario 1, repeat Steps 3 till Step 7.   

​​​Note:

• Since the OS partition was deleted, all user profile settings and configurations, including rules, custom filters, and other configurations, will be lost.
• If you have a configuration backup stored on a network share, you can easily import it through the CMC console using the "cmc::appliance > import" command. However, if you don't have a backup, you will need to manually reconfigure connectors, rules, agent profiles, and other settings.

​​