Network Management
Connection Refused for manually installed Sever Initiated Agents
This article explains the cause and resolution for the "Connection Refused" error encountered when connecting to manually installed Server Initiated (passive) agents from the SolarWinds web console.
First published date
Last published date
Overview
After manually installing a Server Initiated (passive) agent, attempts to connect to the agent from the SolarWinds web console may fail with a "Connection Refused" error.
C:\ProgramData\SolarWinds\Logs\Agent\SolarWinds.Agent.Service.exe.###.log
23/07/07 10:38:18.140 PID: 2436 TID: 3256 [VERB] http::server3::server::start_accept - waiting for connections, secure[true]23/07/07 10:38:18.140 PID: 2436 TID: 3256 [VERB] http::server3::connection_nonssl::connection_nonssl - called, persistent connection23/07/07 10:38:18.140 PID: 2436 TID: 3256 [VERB] http::server3::connection_stream::create - called: connection_stream_nonssl23/07/07 10:38:18.140 PID: 2436 TID: 3256 [VERB] http::server3::connection_ssl::connection_ssl - called23/07/07 10:38:18.140 PID: 2436 TID: 3256 [VERB] http::server3::connection_stream::create - called: connection_stream_ssl23/07/07 10:38:18.145 PID: 2436 TID: 4820 [ERROR] http::server3::connection_ssl::handle_handshake - handshake failed, error [peer did not return a certificate (SSL routines)]23/07/07 10:38:18.145 PID: 2436 TID: 4820 [VERB] http::server3::connection_ssl::stop - called23/07/07 10:38:18.145 PID: 2436 TID: 4820 [VERB] http::server3::connection_timer::cancel_read_timer - read [deadline] timer aborted [0] operations.23/07/07 10:38:18.145 PID: 2436 TID: 4820 [VERB] http::server3::connection_timer::cancel_write_timer - read [deadline] timer aborted [0] operations.23/07/07 10:38:18.145 PID: 2436 TID: 4820 [VERB] http::server3::connection_nonssl::shutdown - socket shutdown succeeded :The operation completed successfully23/07/07 10:38:18.145 PID: 2436 TID: 4820 [VERB] http::server3::connection_ssl::stop - socket close succeeded23/07/07 10:38:18.145 PID: 2436 TID: 4820 [VERB] http::server3::connection_nonssl::stop - called23/07/07 10:38:18.145 PID: 2436 TID: 4820 [VERB] http::server3::connection_timer::cancel_read_timer - read [deadline] timer aborted [0] operations.23/07/07 10:38:18.145 PID: 2436 TID: 4820 [VERB] http::server3::connection_timer::cancel_write_timer - read [deadline] timer aborted [0] operations.23/07/07 10:38:18.145 PID: 2436 TID: 4820 [ERROR] http::server3::connection_nonssl::shutdown - error during socket shutdown :The file handle supplied is not valid23/07/07 10:38:18.145 PID: 2436 TID: 4820 [VERB] http::server3::connection_nonssl::stop - socket close succeeded23/07/07 10:38:18.147 PID: 2436 TID: 3256 [VERB] http::server3::server::start_accept - waiting for connections, secure[true]23/07/07 10:38:18.147 PID: 2436 TID: 3256 [VERB] http::server3::connection_nonssl::connection_nonssl - called, persistent connection23/07/07 10:38:18.147 PID: 2436 TID: 3256 [VERB] http::server3::connection_stream::create - called: connection_stream_nonssl23/07/07 10:38:18.147 PID: 2436 TID: 3256 [VERB] http::server3::connection_ssl::connection_ssl - called23/07/07 10:38:18.147 PID: 2436 TID: 3256 [VERB] http::server3::connection_stream::create - called: connection_stream_ssl
C:\ProgramData\SolarWinds\Logs\AgentManagement\AgentManagement.Service.log
2023-07-07 12:09:02,582 [93] INFO SolarWindsAMSProxy.Communications - SolarWindsAMSProxy::Communications::WebProxy2ProxyInfo_t - No Url is specified for proxy. Disabling proxy.2023-07-07 12:09:02,745 [93] WARN SolarWindsAMSProxy.Communications - `anonymous-namespace'::SimpleClientPersistentConnection::Disconnect - TcpClient is empty2023-07-07 12:09:02,749 [93] INFO SolarWindsAMSProxy.Communications - `anonymous-namespace'::SimpleClientPersistentConnection::CreateTcpClient - created new TcpClient for [1.1.1.1:17790]2023-07-07 12:09:02,762 [93] WARN SolarWindsAMSProxy.Communications - se_exception_initializer::add - Exception - error code [0x80004005], context [File [c:\buildagent\work\e3e8e11d5d332205\src\agent\src\platform-agent\src\solarwinds.ams.proxy\synchttpexchangeclient.cpp], line [923]], message: System.Security.Authentication.AuthenticationException: A call to SSPI failed, see inner exception. ---> System.ComponentModel.Win32Exception: The message received was unexpected or badly formatted --- End of inner exception stack trace --- at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, Exception exception) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslState.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest, Boolean renegotiation) at System.Net.Security.SslState.ProcessAuthentication(LazyAsyncResult lazyResult) at ?A0x05bcbdb6.SimpleClientPersistentConnection.CreatePersistentConnection()2023-07-07 12:09:02,763 [93] INFO SolarWindsAMSProxy.Communications - `anonymous-namespace'::SimpleClientPersistentConnection::SimpleClientPersistentConnection - created connection with host [10.140.210.124], port [17790]2023-07-07 12:09:02,764 [93] ERROR SolarWindsAMSProxy.Communications - `anonymous-namespace'::SimpleClientPersistentConnectionManager::GetConnection - failed to connect to: url [https://1.1.1.1:17790/solarwinds/agent/v1]2023-07-07 12:09:02,766 [93] WARN SolarWindsAMSProxy.Communications - se_exception_initializer::Handle - w32_exception caught: Error [0x65b], [Connection failed.], File: c:\buildagent\work\e3e8e11d5d332205\src\agent\src\platform-agent\src\solarwinds.ams.proxy\synchttpexchangeclient.cpp, Line: 13092023-07-07 12:09:02,767 [93] ERROR SolarWindsAMSProxy.Communications - `anonymous-namespace'::SimpleClientPersistentConnectionManager::GetConnection - failed with exception, status [0x65b] : w32_exception caught: Error [0x65b], [Connection failed.], File: c:\buildagent\work\e3e8e11d5d332205\src\agent\src\platform-agent\src\solarwinds.ams.proxy\synchttpexchangeclient.cpp, Line: 13092023-07-07 12:09:02,818 [93] WARN SolarWindsAMSProxy.Communications - se_exception_initializer::Handle - w32_exception caught: Error [0x80004005], [Disconnected from Passive Agent], File: c:\buildagent\work\e3e8e11d5d332205\src\agent\src\platform-agent\src\solarwinds.ams.proxy\synchttpexchangeclient.cpp, Line: 1181
Product section
Cause
This issue is caused by a legacy SolarWinds-Orion certificate that uses SHA1 1024-bit encryption. This is incompatible with the newer version of OpenSSL that the agents are using for SSL connections. This causes the connection to fail when attempting to connect to the agent from the web console.
Resolution
Resolution 1:
The best long-term solution is to update the SolarWinds-Orion certificate to SHA 256 2048-bit using this article:
Resolution 2:
If possible, you can push the agent installation from the web console instead, which bypasses this limitation.
Resolution 3:
If an immediate solution is needed, you can make this configuration change to the agent machine. This change would need to be done on every new agent that is deployed. Use Resolution 1 for a permanent fix. After making this change, you should be able to connect to the agent normally.
Edit the agent configuration file located at:
Windows
C:\Program Files (x86)\SolarWinds\Agent\SolarWinds.Agent.Service.exe.cfg
Linux
/opt/SolarWinds/Agent/bin/swiagent.cfg
Find the line:
<openSSLCipherList/>
And change it to:
<openSSLCipherList>TLSv1.2+FIPS:kRSA+FIPS:!eNULL:!aNULL@SECLEVEL=0</openSSLCipherList>
Save the change.
Example: