Submit a ticketCall us

Looking to compare latest NPM features with previous versions of NPM?
The NPM new feature summary offers a comparison of new features and improvements offered with this release.

 

Home > Success Center > Failover Engine (FoE) > Upgrade SolarWinds products in Orion Failover Engine V2V environment

Upgrade SolarWinds products in Orion Failover Engine V2V environment

Created by Lori Krell, last modified by Lori Krell on Jul 22, 2016

Views: 39 Votes: 1 Revisions: 7

Updated: 6-6-2016

This guide walks you through upgrading SolarWinds products in a Virtual to Virtual (V2V) Orion Failover Engine (FoE) environment. We include checklists to complete preflight checks of Primary and Secondary servers, prep your environment, upgrade products, gotchas, and troubleshooting steps.

We recommend reviewing the entire guide to fully understand the process of your SolarWinds product upgrades with FoE. This information does not include how to install or upgrade FOE as part of the process.

Understanding FOE Terms

We use the following terms throughout this guide, important to understanding FoE:

  • Primary server: The physical or virtual server that initially hosts your SolarWinds installation.
  • Secondary server: The physical or virtual server that is configured to provide continuous monitoring in the event of a failover.
  • Active server: The physical or virtual server that actively provides monitoring services. FoE designates the active server as the currently monitoring server. If a failover occurs, FoE designates the available passive server as active, switching the active to passive.
  • Passive server: The physical or virtual server available to provide monitoring services if a failover occurs. FoE designates the passive server at installation and in the case of a failover. When a failover occurs, the active and passive servers switch designations. Only the active server actively monitors.
  • Principal interface: Network interface or card (NIC) utilized for the monitoring services' traffic to pass through, also called Public interface.
  • Channel interface: Network interface or card (NIC) utilized for the internal FoE traffic to pass through, for example replication, instruction to switchover.
  • LAN HA: This is the High Availability configuration for FoE.
  • WAN DR: This is the Disaster Recovery configuration for FoE.

Preflight Checklist

This preflight checklist details a number of important steps to help plan and prep for your upgrade with FoE. You will also have a preflight checklist for your SolarWinds product upgrade.

You may need the help of a Server Admin if you are new to working with NICs, backups, cloning, and deploying cloned systems.

Review product upgrade information

Review the SolarWinds Upgrade Guide and release notes for SolarWinds products. You may need to update your current FoE server hardware or software to support changes in SolarWinds platform requirements and supported OS/SQL Server versions.

Verify installed applications

Verify the current version of each Orion application installed on your servers. Having an updated inventory of the environment prior to upgrades are applied gives you a clear path for planning upgrades and troubleshooting any issues.

Planning Tip: To determine any important changes or gotchas between product versions, review the release notes.

Review SolarWinds Product system requirements

Make sure your environment meets the required hardware and software needs for your SolarWinds product upgrades. You can verify the requirements for Orion products and multi-module environment guidelines.

  • Any hardware or software changes and updates to your servers must match. Primary and Secondary server hardware and software must be near exact matches for cloning and failover.
  • If you perform migrations during an upgrade, keep the SolarWinds Orion database installed on a dedicated server. Do not migrate the database onto the Primary or Secondary server.
Windows update process As a best practice, follow the upgrade process for Windows updates on your servers. You should check for and complete any Windows updates prior to and after Orion product upgrades. Updates in Orion products can require additional Windows updates.
Gather credentials Gather local administrator rights and credentials to install SolarWinds product upgrades, servers (VM / physical), access to cloning technologies, and SQL Server Management Studio.
Verify NIC prioritization Check your NIC prioritization. If you completed Microsoft Updates prior to upgrading products, NIC prioritization may have changed.

Notify your company

Prepare for downtime

When installing upgrades, you will need to shutdown FoE. Be prepared and schedule your upgrade during off-peak hours. Send a message to your company of the upgrade schedule and maintenance window. If you need additional help, contact and allocate specific staff to be available.

When using cloning software as part of your upgrade process, consider the following preflight checks:

Check backup or snapshot tools are accessible

Make sure you have full access to the tools needed to clone your servers:

  • P2P: Windows Server Backup or other third party clonging tool
  • V2V: VM snapshot tools like VMWare Vsphere 

When you create backups or snapshots, you need to push this data to an external drive or USB. Do not keep the backup on the backup target server.

Prep a secondary drive for clones You need a secondary drive such as a USB, external drive, or shared drive for saving clones and snapshots. You should not create a backup or snapshot directly on the same drive.

Upgrade Method

You have two methods for upgrading SolarWinds products in an FoE environment. For this guide, we use the cloning method.

  • Upgrade one server fully and clone (recommended): You can upgrade the secondary entire server of all updates, new product versions, and configurations. You can then create a snapshot for VM or backup of your physical server to clone and deploy to your primary. If you do not want to clone a completed server, you must upgrade every product version on the secondary then the primary server in a sequence.
  • Upgrade one product and version at a time per server: Upgrade one version at a time on the Primary and Secondary servers. The versions must match between the main and additional poller, or you will receive a Database Configuration Failure Error.

Warning: Do not upgrade one server fully then upgrade the second server fully, or you will receive a Database Configuration Failure Error. If you want to upgrade one server completely, clone to upgrade the other server.

Prep your servers

Before upgrading products, make sure to prepare your servers with the following information.

Upgrade Tip! For this upgrade, we provide instructions for fully upgrading the Primary server, cloning Primary, and deploying the clone to the Secondary server. This can save time and reduces issues with product versions matching while you upgrade products on multiple servers.

1. Take a VM Snapshot of the Primary server

Backup the SolarWinds Orion database

Creating a snapshot before upgrading products allows you to restore the Primary server should the installation cause unrecoverable issues. Use the tools that come with your VM.

If you use VMWare Vsphere, launch and do the following:

  1. Click on Hosts and Clusters on the left-pane and browse to the Primary Server.
  2. Right-click on the VM to bring up the menu. Select Snapshot > Take Snapshot…
  3. Give the snapshot a name and description and click OK. Wait until the snapshot completes.

Back up your SolarWinds SQL database or create a snapshot of your database server. If you need help, please check your vendor's site for documentation and instructions.

Upgrades will make changes to the database. If you restore to a previous product version, you need to restore the original database matching the server product versions.

2. Backup the SolarWinds Orion SQL Database

Back up your SolarWinds Orion SQL database. If you need help, please check your vendor's site for documentation and instructions. If you have your database on a VM, create a snapshot of your VM.

Backing up your database is required before upgrading Solarwinds because it may change the DB and will need to be restored if the Primary Server needs to be restored.

3. Exclude folders and files from antivirus In order to ensure the best performance on your server host and ensure syncing completes, you should exclude specific file paths and directories from anti-virus software scans on the primary and secondary servers. See the FoE interoperability with antivirus scanners article and this general antivirus exclusion article for details.
4. Gather IP Information of the Secondary server

Gather the Network Information from the Secondary server. Open a CMD prompt and run the following commands: ipconfig /all and route print.

Take note of the following information:

  • Host Name
  • Adapter name, IP address, Mask, Gateway
  • Adapter name, IP address, Mask. There is no gateway for adapter.
  • Persistent Routes – use a route print command to gather this info.
5. Shutdown FoE and take the Secondary server offline
  1. On the Secondary/Passive Server, right-click on the FoE icon File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png in the System Tray and select Shutdown Solarwinds Failover Engine. Wait until FoE notifies that it is shutdown.
  2. On the Primary/Active Server, right-click on the FoE icon File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-2.png in the System Tray and select Shutdown Solarwinds Failover Engine > Leave protected applications running. This will take some time. Wait until FoE notifies that it is shutdown.
  3. On the Secondary Server, disable all Ethernet adapters.

Upgrade your products

With your servers prepared, you can begin upgrading your SolarWinds products.

We recommend using the SolarWinds Upgrade Guide as needed when upgrading your products.

1. Upgrade your products

Upgrade all products on the Primary server.

2. Reboot the Primary server if prompted

If you are prompted to reboot your SolarWinds server, complete the following steps:

  1. Reboot your SolarWinds server.
  2. After the reboot, FoE will start and all SolarWinds Orion services will be stopped. Shutdown Orion Failover Engine.
  3. Complete the SolarWinds Configuration Wizard.
3. Test your upgraded products

Keep FoE shut down and test your upgraded SolarWinds products:

All SolarWinds products are working as expected.

All Orion Services are running.

Web services are running.

Results:

You should open the products and verify the versions displayed in the footer of the web console. Try current and new features with your system to check performance and expected functionality. If you run into issues, check the troubleshooting tips.

Troubleshooting 

If an upgrade does not complete successfully:

  1. Determine the cause of the upgrade failure. You can research issues through our Success Center.
  2. If the issue can be resolved, resolve the issue and fully complete upgrades to the latest versions.
  3. If the issue cannot be resolved, you can restore your primary SolarWinds server to its previous state by restoring the clone.
  4. If the DB was updated during the installation, it will need to be restored.

Clone the upgraded Primary server to Secondary

These instructions detail how to clone the upgrade Primary server to the Secondary server.

1. Clone the Primary VM

After testing your products, create a clone of the Primary Server using your VM tool. It is important that the clone has the same Hostname and SID.

The clone must not be on the network. You can disable the NIC from the VM side.

Wait until the cloning completes. This step may take over an hour or longer, depending on the amount of products installed and size of server.

2. Deploy clone to Secondary VM

 

Instructions include WAN and LAN.

Deploy to WAN

For WAN environments, deploy the clone of Primary onto the Secondary server. Continue testing FOE when the servers are fully synced.

  1. Log into the cloned server. Shut down Orion services and FoE.
  2. Properly adjust the IP information of the NICs. Use the information you gathered on the Secondary server during server prep.
  3. Adjust the persistent routes to match the information you gathered:
    1. Open the CMD prompt.
    2. To add persistent route type: route add [destination_IP] mask [netmask] [gateway_IP] metric [metric] IF [interface] –p

      Example: route add 10.6.236.0 mask 255.255.255.0 10.32.236.254 metric 1 IF 16 –p

    3. To delete a persistent route: route delete [dest_IP]

      Or if there is more than one route with that same destination IP: route delete [destination_IP] mask [netmask] [gateway_IP]

      If the proper default route is missing, this is because the gateway is not configured on the public NIC.

  4. Run the FoE Configuration Wizard and ensure that this new server is set to the Secondary role.
    1. Click Start > All Programs > SolarWinds > SolarWinds Failover Engine > Configure Server.
    2. Click the Machine tab.
    3. Set the Physical Hardware Identity to Secondary. Set the Active Server to Primary. Click Finish.
  5. Enable the NICs through the server. If they are disabled in Windows, enable them.
  6. Verify that you can ping the Primary server on the channel IP address.
  7. Verify the network settings and IP information are correct.
  8. Reboot the new cloned Secondary Server.
  9. As the Secondary server is booting, enable FoE on the Primary Server.
  10. When the Secondary server comes back up, FoE should start automatically. Verify FoE syncs between the Primary and Secondary servers.

 

Deploy to LAN

For LAN environments, deploy the clone of Primary onto the Secondary server. Continue testing FOE when the servers are fully synced.

  1. Log into the cloned server. Shut down Orion services and FoE.
  2. Properly adjust the IP information of the NICs. Use the information you gathered on the Secondary server during server prep.
  3. Run the FoE Configuration Wizard and ensure that this new server is set to the Secondary role.
    1. Click Start > All Programs > SolarWinds > SolarWinds Failover Engine > Configure Server.
    2. Click the Machine tab.
    3. Set the Physical Hardware Identity to Secondary. Set the Active Server to Primary. Click Finish.
  4. Enable the NICs through the server. If they are disabled in Windows, enable them.
  5. Verify that you can ping the Primary server on the channel IP address.
  6. Verify the network settings and IP information are correct.
  7. Reboot the new cloned Secondary Server.
  8. As the Secondary server is booting, enable FoE on the Primary Server.
  9. When the Secondary server comes back up, FoE should start automatically. Verify FoE syncs between the Primary and Secondary servers.
3. Test FoE Failover
  1. Open the SolarWinds Failover Manager: Right-click on the FoE icon and select Manage Server…
  2. Verify that you are connected to the localhost.
  3. On the Server tab > Summary page, click on the Secondary server icon. Click Make Active…
  4. FoE will now automatically go through the process of making the Secondary Server Active.
  5. Verify that the DNS has been updated with the Secondary Server’s public IP address. Flushing the DNS may be required or need to wait for DNS to timeout (10 minutes).
  6. Make sure the Solarwinds website is up. May take time to refresh.
  7. Verify no nodes are now down. This signifies that the node IP address cannot reach the Secondary Server through icmp. Check policies and routes.
  8. To fail back, select the Primary server icon in Failover Manager. Click Make Active…
  9. Verify that the DNS has been updated with the Secondary Server’s public IP address. Flushing the DNS may be required or need to wait for DNS to timeout (10 minutes).
  10. Verify the Solarwinds website is up. The site may take additional time to refresh.
4. Services set to Manual The Configuration Wizard may have reset SolarWinds services to the Statup Type of Automatic. Make sure all SolarWinds services are set to Manual in the Microsoft Management Console. FoE should manage all automated actions for turning services on and off.

After the upgrade

We recommend cleaning up your systems and creating a snapshot of the Primary and Secondary servers. If you continue to make changes that may cause issues with your server, you can roll back to these server backups.

Snapshot your completed servers

After your servers are running correctly and configured correctly, we highly recommend creating backup or snapshot copies to an external drive or USB. You can use this backup as a rollback if you continue making changes to the servers for testing features.

 

Verify SolarWinds HA disabled

When upgrading to an Orion Platform product 2016.2 or later, verify the SolarWinds HighAvailability service is disabled. Orion Platform products such as NPM 12.0.1 and later and SAM 6.3 and later include a new SolarWinds High Availability feature. For FOE environments, this service must be disabled. For details, see this article.

 

Last modified
08:42, 22 Jul 2016

Tags

Classifications

Public