Submit a ticketCall us

Systems Monitoring for Dummies
Our new eBook will teach you the fundamentals and help you create monitors and alerts that are effective, meaningful, and actionable. Monitoring is more than a checkbox on your to-do list. This free eBook will give you practical advice to help you succeed in all aspects of monitoring – discovery, alerting, remediation, and troubleshooting. Don’t miss out on this indispensable resource for newbies, experienced IT pros, and everyone in between. Register Now.

Home > Success Center > Network Performance Monitor (NPM) > Upgrade SolarWinds products with Orion Failover Engine installed - P2P Environment - Manual Upgrade

Upgrade SolarWinds products with Orion Failover Engine installed - P2P Environment - Manual Upgrade

Created by Lori Krell_ret, last modified by Lori Krell_ret on Aug 22, 2016

Views: 42 Votes: 0 Revisions: 9

Updated: June 21, 2016

This guide walks you through upgrading SolarWinds products in a Physical to Physical (P2P) Orion Failover Engine (FoE) environment with manual upgrading steps. We include checklists to complete preflight checks of Primary and Secondary servers, prep your environment, upgrade products, gotchas, and troubleshooting steps.

We strongly recommend reviewing the entire guide to fully understand the process of your SolarWinds product upgrades with FoE. This information does not include how to install or upgrade FoE as part of the process.

These instructions details how to upgrade one product. You can repeat to upgrade multiple products. You must carefully review this information before installing any Orion products.

The overall upgrade process will take time and requires precise steps completed in order to ensure your products, database schemas, and configurations match across all of your servers simultaneously:

  • You will upgrade one product version at a time on all servers. This means installing a new version on the Secondary server, then the Primary server, then every additional polling engine, and every web server before continuing to the next product version upgrade.
  • After every upgrade, you must run the Configuration Wizard. If you skip the Configuration Wizard, you will encounter database schema and out of sync product issues.

Understanding FoE Terms

We use the following terms throughout this guide, important to understanding FoE:

  • Primary server: The physical or virtual server that initially hosts your SolarWinds installation.
  • Secondary server: The physical or virtual server that is configured to provide continuous monitoring in the event of a failover.
  • Active server: The physical or virtual server that actively provides monitoring services. FoE designates the active server as the currently monitoring server. If a failover occurs, FoE designates the available passive server as active, switching the active to passive.
  • Passive server: The physical or virtual server available to provide monitoring services if a failover occurs. FoE designates the passive server at installation and in the case of a failover. When a failover occurs, the active and passive servers switch designations. Only the active server actively monitors.
  • Principal interface: Network interface or card (NIC) utilized for the monitoring services' traffic to pass through, also called Public interface.
  • Channel interface: Network interface or card (NIC) utilized for the internal FoE traffic to pass through, for example replication, instruction to switchover.
  • LAN HA: This is the High Availability configuration for FoE.
  • WAN DR: This is the Disaster Recovery configuration for FoE.

Preflight Checklist

This preflight checklist details a number of important steps to help plan and prep for your upgrade with FoE. You will also have a preflight checklist for your SolarWinds product upgrade.

You may need the help of a Server Admin if you are new to working with NICs, backups, cloning, and deploying cloned systems.

Review product upgrade information

Review the SolarWinds Upgrade Guide and release notes for SolarWinds products. You may need to update your current FoE server hardware or software to support changes in SolarWinds platform requirements and supported OS/SQL Server versions.

Verify installed applications

Verify the current version of each Orion application installed on your servers. Having an updated inventory of the environment prior to upgrades are applied gives you a clear path for planning upgrades and troubleshooting any issues.

Planning Tip: To determine any important changes or gotchas between product versions, review the release notes.

Review SolarWinds Product system requirements

Make sure your environment meets the required hardware and software needs for your SolarWinds product upgrades. You can verify the requirements for Orion products and multi-module environment guidelines.

  • Any hardware or software changes and updates to your servers must match. Primary and Secondary server hardware and software must be near exact matches for cloning and failover.
  • If you perform migrations during an upgrade, keep the SolarWinds Orion database installed on a dedicated server. Do not migrate the database onto the Primary or Secondary server.
Windows update process As a best practice, follow the upgrade process for Windows updates on your servers. You should check for and complete any Windows updates prior to and after Orion product upgrades. Updates in Orion products can require additional Windows updates.
Gather credentials Gather local administrator rights and credentials to install SolarWinds product upgrades, servers (VM / physical), access to cloning technologies, and SQL Server Management Studio.
Verify NIC prioritization Check your NIC prioritization. If you completed Microsoft Updates prior to upgrading products, NIC prioritization may have changed.

Notify your company

Prepare for downtime

When installing upgrades, you will need to shutdown FoE. Be prepared and schedule your upgrade during off-peak hours. Send a message to your company of the upgrade schedule and maintenance window. If you need additional help, contact and allocate specific staff to be available.

Prepare your servers

Before upgrading products, you may need to prepare your servers.  If you are upgrading multiple products, get an upgrade path from the Product Upgrade Advisor.

1. Backup the SolarWinds Orion SQL Database

Back up your SolarWinds Orion SQL database. If you need help, please check your vendor's site for documentation and instructions.

If you have your database on a VM, create a snapshot of your VM.

Backing up your database is required before upgrading Solarwinds. If you need to restore back to the previous versions due to issues, you will need the original database for the Primary server.

2. Exclude folders and files from antivirus In order to ensure the best performance on your server host and ensure syncing completes, you should exclude specific file paths and directories from anti-virus software scans on the primary and secondary servers. See the FoE interoperability with antivirus scanners article and this general antivirus exclusion article for details.
3. Shutdown FoE and take the Secondary server offline
  1. On the Secondary/Passive Server, right-click on the FoE icon File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png in the System Tray and select Shutdown Solarwinds Failover Engine. Wait until FoE notifies that it is shutdown.
  2. On the Primary/Active Server, right-click on the FoE icon File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-2.png in the System Tray and select Shutdown Solarwinds Failover Engine > Leave protected applications running. This will take some time. Wait until FoE notifies that it is shutdown.
  3. On the Secondary Server, disable all Ethernet adapters.

Upgrade a module with FOE protecting only the Main Poller

This method details how to upgrade if you have a Primary and Secondary server pair only for the main poller.

Important! You should only upgrade one module or product at a time!  You must follow these instructions carefully and exactly to ensure your products are upgraded without sync issues or database configuration failures.

A. Make Main Poller Secondary server active and upgrade

  1. Using an account with administrative privileges, log on to your Primary server for the Main Poller and open Orion Failover Engine from the FoE icon File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png  in the System Tray.
  2. Select your Secondary (passive) server, and click Make Active.

    There may be a brief delay before the secondary server becomes active.

  3. Using an account with administrative privileges, log on to your Secondary server. Shutdown Orion Failover Engine on the Secondary (active) server, leaving all protected applications running.
  4. Shutdown Orion Failover Engine on the Primary (passive) server.

    This is required to prevent the servers from attempting to communicate before the upgrade process is complete. f the Secondary (active) server requires a reboot, the FOE service starts automatically after the reboot  completes. Having the Orion Failover Engine shutdown on the Primary (passive) server is a safe measure to help avoid issues.

  5. Install the product upgrade on the Secondary server. Run the Configuration Wizard.
  6. If prompted, reboot your SolarWinds server.
  7. Verify that the installation was completed successfully before upgrading the Primary (passive) server.

    See the next section "E. Test your product upgrades" for troubleshooting if you encounter errors.

  8. Log on to your Secondary server for the Main Poller and open Orion Failover Engine from the FoE icon File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png  in the System Tray. Select your Secondary (active) server, and click Make Passive.
B. Make Main Poller Primary server active and upgrade the product
  1. Using an account with administrative privileges, log on to your Main Poller Primary (passive) server.
  2. Disable the Neverfail Packet Filter Driver on the primary server's public NIC.
    1. Open the Properties window for your Public Network Adapter.

      Typically, you can open the properties window by clicking Start > Control Panel > Network.

    2. Clear Neverfail Packet Filter Driver and click OK.

  3. Install the product upgrade on the Main Poller Primary server. Run the Configuration Wizard.
  4. If prompted, reboot your SolarWinds server.
  5. Verify that your upgraded SolarWinds installation is operational on all servers.

    See  "E. Test your product upgrades" for troubleshooting if you encounter errors.

  6. Enable the Neverfail Packet Filter Driver on the Primary server’s public NIC.

    1. Open the Properties window for your Public Network Adapter.

      Typically, you can open the properties window by clicking Start > Control Panel > Network.

    2. Check Neverfail Packet Filter Driver and click OK.

  7. Start the Orion Failover Engine on your Secondary server, and then click Start Replication.

    See the next section "E. Test your product upgrades" for troubleshooting if you encounter errors.

C. Upgrade each Additional Polling Engine
  1. Log on to each of your additional polling engines.
  2. Install the product upgrade on every additional polling engine. Run the Configuration Wizard.
  3. If prompted, reboot your SolarWinds server.
  4. Verify that your upgraded SolarWinds installation is operational on all servers.

    See "E. Test your product upgrades" for troubleshooting if you encounter errors.

  D. Upgrade each Additional Web Server
  1. Log on to each of your additional web servers.
  2. Install the product upgrade on every additional web server. Run the Configuration Wizard.
  3. If prompted, reboot your SolarWinds server.
  4. Verify that your upgraded SolarWinds installation is operational on all servers.

    See "E. Test your product upgrades" for troubleshooting if you encounter errors.

E. Test your product upgrades

If an upgrade does not complete successfully, determine the cause of the upgrade failure or issue. Use the Success Center to search the error message or issue.

If the issue can be resolved, resolve the issue and complete the upgrade procedures.

 

If the issue cannot be resolved and the upgrade was only attempted on the Secondary (active) server, do the following:

After the reboot, SolarWinds Orion Failover Engine will start, and all SolarWinds Orion services will be stopped.

  1. Shutdown Orion Failover Engine from the FoE icon  File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png in the System Tray on the Secondary (active) server.
  2. Launch the SolarWinds Orion Failover Engine Configure Server Wizard from the Secondary (active) server.
  3. Click to open the Machine tab.
  4. Change the role to Primary/Active.
  5. Start Orion Failover Engine from the FoE icon  File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png in the System Tray on the Secondary (active) server.
  6. Wait for the Secondary server to change its role to Passive.
  7. Restore the Orion database using the backup that was taken before the upgrade was attempted.
  8. Launch the SolarWinds Orion Failover Engine Configure Server Wizard from the Primary (passive) server. 
  9. Click to open the Machine tab.
  10. Change the role to Primary/Active.
  11. Start Orion Failover Engine from the FoE icon  File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png in the System Tray”
  12. Wait for the Primary server to change its role to active. 
  13. Verify the Orion platform and products are up and running. 
  14. Important! Consider the following next steps:
    • If the Orion platform and products are up and running on the Primary (active) server, please open an Urgent case from your customer portal and call SolarWinds support if you are able to begin troubleshooting immediately. 
    • If you are not able to get Orion up and running during an Upgrade Failure, please open a System Down case from your customer portal and call SolarWinds Support immediately.
F. Test FoE Failover
  1. Open the SolarWinds Failover Manager: Right-click on the FoE icon and select Manage Server…
  2. Verify that you are connected to the localhost.
  3. On the Server tab > Summary page, click on the Secondary server icon. Click Make Active…
  4. FoE will now automatically go through the process of making the Secondary Server Active.
  5. Verify that the DNS has been updated with the Secondary Server’s public IP address. Flushing the DNS may be required or need to wait for DNS to timeout (10 minutes).
  6. Make sure the Solarwinds website is up. May take time to refresh.
  7. Verify no nodes are now down. This signifies that the node IP address cannot reach the Secondary Server through icmp. Check policies and routes.
  8. To fail back, select the Primary server icon in Failover Manager. Click Make Active…
  9. Verify that the DNS has been updated with the Secondary Server’s public IP address. Flushing the DNS may be required or need to wait for DNS to timeout (10 minutes).
  10. Verify the Solarwinds website is up. The site may take additional time to refresh.
G. Services set to Manual The Configuration Wizard may have reset SolarWinds services to the Startup Type of Automatic. Make sure all SolarWinds services are set to Manual in the Microsoft Management Console. FoE should manage all automated actions for turning services on and off.

Upgrade a module with FOE protecting Main Poller, APE, and AWS

This method details how to upgrade if you have a Primary and Secondary server pair for the main poller, additional polling engine, and additional web server.

Important! You should only upgrade one module or product at a time! You must follow these instructions carefully and exactly to ensure your products are upgraded without sync issues or database configuration failures.

 

A. Make Main Poller Secondary server active and upgrade

 

Make Main Poller Primary server active and upgrade the product

Make Main Poller Secondary server active and upgrade

  1. Using an account with administrative privileges, log on to your Main Poller Primary server and open Orion Failover Engine from the FoE icon File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png  in the System Tray.
  2. Select your Main Poller Secondary (passive) server, and click Make Active.

    There may be a brief delay before the secondary server becomes active.

  3. Using an account with administrative privileges, log on to your Main Poller Secondary server. Shutdown Orion Failover Engine on the Secondary (active) server, leaving all protected applications running.
  4. Shutdown Orion Failover Engine on the Primary (passive) server.

    This is required to prevent the servers from attempting to communicate before the upgrade process is complete. f the Secondary (active) server requires a reboot, the FOE service starts automatically after the reboot  completes. Having the Orion Failover Engine shutdown on the Primary (passive) server is a safe measure to help avoid issues.

  5. Install the product upgrade on the Secondary server. Run the Configuration Wizard.
  6. If prompted, reboot your SolarWinds server.
  7. Verify that the installation was completed successfully before upgrading the Main Poller Primary (passive) server.

    See the next section "D. Test your product upgrades" for troubleshooting if you encounter errors.

 

Make Main Poller Primary server active and upgrade the product

  1. Using an account with administrative privileges, log on to your Main Poller Primary (passive) server.
  2. Disable the Neverfail Packet Filter Driver on the primary server's public NIC.
    1. Open the Properties window for your Public Network Adapter.

      Typically, you can open the properties window by clicking Start > Control Panel > Network.

    2. Clear Neverfail Packet Filter Driver and click OK.

  3. Install the product upgrade on the Main Poller Primary server. Run the Configuration Wizard.
  4. If prompted, reboot your SolarWinds server.
  5. Verify that your upgraded SolarWinds installation is operational on all servers.

    See the next section "D. Test your product upgrades" for troubleshooting if you encounter errors.

  6. Enable the Neverfail Packet Filter Driver on the Primary server’s public NIC.

    1. Open the Properties window for your Public Network Adapter.

      Typically, you can open the properties window by clicking Start > Control Panel > Network.

    2. Check Neverfail Packet Filter Driver and click OK.

  7. Start the Orion Failover Engine on your secondary server, and then click Start Replication.

    See the next section "D. Test your product upgrades" for troubleshooting if you encounter errors.

 

B. Make APE Secondary server active and upgrade

 

Make APE Primary server active and upgrade the product

Make Additional Polling Engine Secondary server active and upgrade

  1. Using an account with administrative privileges, log on to your APE Primary server and open Orion Failover Engine from the FoE icon File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png  in the System Tray.
  2. Select your APE Secondary (passive) server, and click Make Active.

    There may be a brief delay before the secondary server becomes active.

  3. Using an account with administrative privileges, log on to your APE Secondary server. Shutdown Orion Failover Engine on the APE Secondary (active) server, leaving all protected applications running.
  4. Shutdown Orion Failover Engine on the APE Primary (passive) server.

    This is required to prevent the servers from attempting to communicate before the upgrade process is complete. f the Secondary (active) server requires a reboot, the FOE service starts automatically after the reboot  completes. Having the Orion Failover Engine shutdown on the Primary (passive) server is a safe measure to help avoid issues.

  5. Install the product upgrade on the APE Secondary server. Run the Configuration Wizard.
  6. If prompted, reboot your SolarWinds server.
  7. Verify that the installation was completed successfully before upgrading the APE Primary (passive) server.

    See the next section "D. Test your product upgrades" for troubleshooting if you encounter errors.

 

Make Additional Polling Engine Primary server active and upgrade the product

  1. Using an account with administrative privileges, log on to your APE Primary (passive) server.
  2. Disable the Neverfail Packet Filter Driver on the primary server's public NIC.
    1. Open the Properties window for your Public Network Adapter.

      Typically, you can open the properties window by clicking Start > Control Panel > Network.

    2. Clear Neverfail Packet Filter Driver and click OK.

  3. Install the product upgrade on the APE Primary server. Run the Configuration Wizard.
  4. If prompted, reboot your SolarWinds server.
  5. Verify that your upgraded SolarWinds installation is operational on all servers.

    See the next section "D. Test your product upgrades"  for troubleshooting if you encounter errors.

  6. Enable the Neverfail Packet Filter Driver on the Primary server’s public NIC.

    1. Open the Properties window for your Public Network Adapter.

      Typically, you can open the properties window by clicking Start > Control Panel > Network.

    2. Check Neverfail Packet Filter Driver and click OK.

  7. Start the Orion Failover Engine on your APE Secondary server, and then click Start Replication.

    See the next section "D. Test your product upgrades" for troubleshooting if you encounter errors.

 

C. Make AWS Secondary server active and upgrade

 

Make AWS Primary server active and upgrade the product

Make Additional Web Server Secondary server active and upgrade

  1. Using an account with administrative privileges, log on to your AWS Primary server and open Orion Failover Engine from the FoE icon File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png  in the System Tray.
  2. Select your AWS Secondary (passive) server, and click Make Active.

    There may be a brief delay before the secondary server becomes active.

  3. Using an account with administrative privileges, log on to your AWS Secondary server. Shutdown Orion Failover Engine on the AWS Secondary (active) server, leaving all protected applications running.
  4. Shutdown Orion Failover Engine on the AWS Primary (passive) server.

    This is required to prevent the servers from attempting to communicate before the upgrade process is complete. If the Secondary (active) server requires a reboot, the FOE service starts automatically after the reboot  completes. Having the Orion Failover Engine shutdown on the Primary (passive) server is a safe measure to help avoid issues.

  5. Install the product upgrade on the AWS Secondary server. Run the Configuration Wizard.
  6. If prompted, reboot your SolarWinds server.
  7. Verify that the installation was completed successfully before upgrading the AWS  Primary (passive) server.

    See the next section "D. Test your product upgrades" for troubleshooting if you encounter errors.

 

Make Additional Web Server Primary server active and upgrade the product

  1. Using an account with administrative privileges, log on to your AWS Primary (passive) server.
  2. Disable the Neverfail Packet Filter Driver on the primary server's public NIC.
    1. Open the Properties window for your Public Network Adapter.

      Typically, you can open the properties window by clicking Start > Control Panel > Network.

    2. Clear Neverfail Packet Filter Driver and click OK.

  3. Install the product upgrade on the AWS Primary server. Run the Configuration Wizard.
  4. If prompted, reboot your SolarWinds server.
  5. Verify that your upgraded SolarWinds installation is operational on all servers.

    See the next section "D. Test your product upgrades" for troubleshooting if you encounter errors.

  6. Enable the Neverfail Packet Filter Driver on the Primary server’s public NIC.

    1. Open the Properties window for your Public Network Adapter.

      Typically, you can open the properties window by clicking Start > Control Panel > Network.

    2. Check Neverfail Packet Filter Driver and click OK.

  7. Start the Orion Failover Engine on your AWS Secondary server, and then click Start Replication.

    See the next section "D. Test your product upgrades" for troubleshooting if you encounter errors.

D. Test your product upgrades

If an upgrade does not complete successfully, determine the cause of the upgrade failure or issue. Use the Success Center to search the error message or issue.

If the issue can be resolved, resolve the issue and complete the upgrade procedures.

 

If the issue cannot be resolved and the upgrade was only attempted on the Secondary (active) server, do the following:

After the reboot, SolarWinds Orion Failover Engine will start, and all SolarWinds Orion services will be stopped.

  1. Shutdown Orion Failover Engine from the FoE icon  File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png in the System Tray on the Secondary (active) server.
  2. Launch the SolarWinds Orion Failover Engine Configure Server Wizard from the Secondary (active) server.
  3. Click to open the Machine tab.
  4. Change the role to Primary/Active.
  5. Start Orion Failover Engine from the FoE icon  File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png in the System Tray on the Secondary (active) server.
  6. Wait for the Secondary server to change its role to Passive.
  7. Restore the Orion database using the backup that was taken before the upgrade was attempted.
  8. Launch the SolarWinds Orion Failover Engine Configure Server Wizard from the Primary (passive) server. 
  9. Click to open the Machine tab.
  10. Change the role to Primary/Active.
  11. Start Orion Failover Engine from the FoE icon  File:Success_Center/New_Articles/Release_Notes_CHM_Import_Area/Upgrade-Mindtouch-CHM/010/foe-icon-1.png in the System Tray”
  12. Wait for the Primary server to change its role to active. 
  13. Verify the Orion platform and products are up and running. 
  14. Important! Consider the following next steps:
    • If the Orion platform and products are up and running on the Primary (active) server, please open an Urgent case from your customer portal and call SolarWinds support if you are able to begin troubleshooting immediately. 
    • If you are not able to get Orion up and running during an Upgrade Failure, please open a System Down case from your customer portal and call SolarWinds Support immediately.
E. Test FoE Failover
  1. Open the SolarWinds Failover Manager: Right-click on the FoE icon and select Manage Server…
  2. Verify that you are connected to the localhost.
  3. On the Server tab > Summary page, click on the Secondary server icon. Click Make Active…
  4. FoE will now automatically go through the process of making the Secondary Server Active.
  5. Verify that the DNS has been updated with the Secondary Server’s public IP address. Flushing the DNS may be required or need to wait for DNS to timeout (10 minutes).
  6. Make sure the Solarwinds website is up. May take time to refresh.
  7. Verify no nodes are now down. This signifies that the node IP address cannot reach the Secondary Server through icmp. Check policies and routes.
  8. To fail back, select the Primary server icon in Failover Manager. Click Make Active…
  9. Verify that the DNS has been updated with the Secondary Server’s public IP address. Flushing the DNS may be required or need to wait for DNS to timeout (10 minutes).
  10. Verify the Solarwinds website is up. The site may take additional time to refresh.
F. Services set to Manual The Configuration Wizard may have reset SolarWinds services to the Startup Type of Automatic. Make sure all SolarWinds services are set to Manual in the Microsoft Management Console. FoE should manage all automated actions for turning services on and off.

After the upgrade

We recommend cleaning up your systems and creating a backup of the Primary and Secondary servers. If you continue to make changes that may cause issues with your server, you can roll back to these server backups.

Backup your completed servers

After your servers are running and configured correctly, we highly recommend creating a backup clone of your servers to an external drive or USB. You can use this backup as a rollback if you continue making changes to the servers for testing features.

 

Verify SolarWinds HA disabled

When upgrading to an Orion Platform product 2016.2 or later, verify the SolarWinds HighAvailability service is disabled. Orion Platform products such as NPM 12.0.1 and later and SAM 6.3 and later include a new SolarWinds High Availability feature. For FOE environments, this service must be disabled. For details, see this article.

 

Last modified

Tags

Classifications

Public