Configuration Wizard Fails: Failed to setup RabbitMQ for Orion

Configuring message bus FAILED
Services configuration failed:
 • Failed to setup RabbitMQ for Orion. See log for details.

ERROR TaskAdapter - Error running task Orion.MessageBus.MessageBusConfigurator Configuring message bus
SolarWinds.ConfigurationWizard.Common.ConfigurationException: Failed to setup RabbitMQ for Orion. See log for details. ---> System.Exception: An error occurred in 'rabbitmqctl.bat'. Error: unable to perform an operation on node 'rabbit@ServerName'. Please see diagnostics information and suggestions below.

Most common reasons for this are:

 * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues)
 * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server)
 * Target node is not running
 * System permissions are incorrectly set on some key locations on the Orion Server.

In addition to the diagnostics info below:

 * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more
 * Consult server logs on node rabbit@ServerName
 * If target node is configured to use long node names, don't forget to use --longnames with CLI tools

DIAGNOSTICS
===========

attempted to contact: [rabbit@ServerName]

rabbit@ServerName:
  * connected to epmd (port 4369) on ServerName
  * epmd reports: node 'rabbit' not running at all
                  no other nodes on ServerName
  * suggestion: start the node

Current node details:
 * node name: 'rabbitmqcli-55896-rabbit@ServerName'
 * effective user's home directory: C:\ProgramData\SolarWinds\Orion\RabbitMQ
 * Erlang cookie hash: F0voAQw5eIUHFbnIycsB+A==

ERROR TaskAdapter - Error running task Orion.MessageBus.MessageBusConfigurator Configuring message bus
SolarWinds.ConfigurationWizard.Common.ConfigurationException: Failed to setup RabbitMQ for Orion. See log for details. ---> System.ArgumentException: The provided notBefore value is later than the notAfter value.
   at System.Security.Cryptography.X509Certificates.CertificateRequest.CreateSelfSigned(DateTimeOffset notBefore, DateTimeOffset notAfter)
   at SolarWinds.Orion.Security.Certificates.CertificateCreator.CreateRootCertificate(String subjectName, DateTime expirationDayUtc)
   at SolarWinds.ConfigurationWizard.Plugin.Orion.MessageBus.MessageBusConfigurator.<>c__DisplayClass23_0.<GenerateAndSaveCertificates>b__0()
   at SolarWinds.ConfigurationWizard.Plugin.Orion.MessageBus.MessageBusConfigurator.GetOrCreateCredential[TCredential](String credentialOwner, String credentialName, Func`1 factory)

1. Pending Windows Updates / UAC issues.
2. Bad installation/upgrade of RabbitMQ during Orion upgrade.
3. System permissions are incorrectly set on some key locations on the Orion Server.

Resolution 1:

1. Check if UAC is enabled, if yes disable and reboot.
2. Check for any pending windows update on the Server > Install them if found and reboot.
3. Re-run the Configuration Wizard.

Resolution 2:
If the messages found match this article, you will need to remove RabbitMQ from the Main Polling Engine and install it properly. Please refer to Full RabbitMQ Reset in Orion Platform products for complete steps.

Resolution 3:
Run the Orion Permission Checker. Please refer to Check permissions using the Orion Permission Checker. Click repair in the Orion Permission Check and make sure that the user has read/write access to RabbitMQ. Re-run the configuration wizard.
 

Resolution 4:
Anti Virus. Check erl.exe in Task Manager\Details if the process is not being blocked. Kindly refer to the sample screenshot below.


(Screenshot property of © 2019 Microsoft)

Resolution 5:

• Ran RabbitMQ Manually and will see an error for The code execution cannot proceed because VCRUNTIME140.dll was not found. Reinstalling the program may fix this problem.

(Screenshot property of © 2019 Microsoft)

1. RDP to the Orion Server.
2. Go to the path C:\Windows\System32.
3. Check for the following files if present:

• VCRUNTIME140.DLL.
• VCRUNTIME140_1.DLL
• VCRUNTIME140_clr0400.DLL
  • You may also refer to the article VCRUNTIME140.DLL.

The content posted herein are provided as a suggestion or recommendation to you for your internal use. The information set forth herein may come from third-party website or customers. SolarWinds is not liable for any downtime or any issue that may occur if you perform the following suggestions on the link provided. Your organization should internally review and assess to what extent, if any, such custom scripts or recommendations will be incorporated into your environment.

4. If the files are missing, if the customer has a working server that has the same OS, you may ask the customer to copy the file and move it to the Orion Server.
5. Once done, reboot the server.
6. Re-run the Configuration Wizard and  the Task Manager will now show erl;.exe.
7. The Configuration Wizard will now be complete with no errors for Configuring message bus FAILED. Failed to setup RabbitMQ for Orion. See log for details.

Resolution 6 for Error 3:

1. Run the command below via powershell to check the windows system locale of the server.
If the result is Thai, the year could be 2566

(Screenshot property of © 2019 Microsoft)

2. If this setting is set to thai:
     a. Run this command in powershell:

Set-WinSystemLocale en-US

     b. Reboot server 
     c. Change server's date and time settings' Regional format to English United States
     c. Upgrade to 2023.4 or higher 

 Resolution 7:

a. RDP to the Main SolarWinds Platform Server.

b. Go to the C:\ProgramData\SolarWinds\Orion\RabbitMQ.

c. Create a backup of the DB folder and placed it on the desktop.

d. Delete DB folder inside C:\ProgramData\SolarWinds\Orion\RabbitMQ.

e. Run the Configuration Wizard as an administrator and ensure it completes.