NCM few nodes failing downloading config files (Connection Refused ) / (Connection TimeOut)/ (Cannot Log into Device : bad password.) Error what should i check?

• Device configuration 
• L2/L3 connectivity 
• Device Template
• Incorrect protocol selected in NCM Connection Profile  (OR SSH or Telnet is blocked from NCM  poller failed to SSH or Telnet to this device ) 
• Incorrect password in NCM Connection Profile
• Use keyboard interactive authentication YES
• F5s takes a long time to save the UCS file locally
• The file transfer between the device and the TFTP/SCP server takes a long time (Cisco WLC)
• Others 

Pick one single node and work along to make sure the node in question actually have no issues with connectivity.

(Please make sure you work on the correct polling Engine where the node is assigned for polling if you have multiple pollers)

Checking NCM Profile
Orion web console > Go to this node  "target effected node" > Edit Node > NCM Properties check the Connection Profile "Test" if this successful ?

Checking NCM Profile with SSH Auto

Also please on Orion web console > Go to this node  "target effected node" > Edit Node > NCM Properties check the Connection Profile

Select SSHAuto >

"Test" if this successful ?

 

Checking NCM Profile
Orion web console > Go to this node  "target effected node" > Edit Node > NCM Properties check the Connection Profile "Test"

Checking NCM Profile with SSH Auto

Also please on Orion web console > Go to this node  "target effected node" > Edit Node > NCM Properties check the Connection Profile

Select SSHAuto >

"Test"


Incorrect password in NCM Connection Profile
In case you are using any connection profile, it is possible connection profile contains old password and you are tyring new password in putty . 
Try to type credentials directly for node instead of using profile.
Please double check that these credentials are typed correctly

Incorrect protocol selected in NCM Connection Profile 
Please check if you have selected correct protocol for the device in Connection Profile . 
For Example : You have selected SSH and device only supported for Telnet . 
Double check and make sure correct supported protocol is selected in NCM Connection Profile 


Use keyboard interactive authentication
Please also try set ‘Use keyboard interactive authentication” option to “Yes” on edit node NCM page and try ones more. Even in case NCM Credential test failed try to downlload the device configuration .
 

If you have connection failure (Make sure you RDP  on the the same Poller where the node is assigned )

Connect with the device using PUTTY and try to list the device configuration

Still failed ? 

In this case we have to check the Session Trace for the Node.

Enable Session Trace

1. Open the Orion Web Console.
2. Go to:Settings > All Settings > Product Specific Settings > CLI Settings
3. Enable Session Tracing check box.
4. Go to the trace log location:
   C:\ProgramData\SolarWinds\Logs\Orion\CLI\Session-Trace\NCM
5. Delete the trace files.

Now Run the below Test and check the Session Trace log file for the Error (Error will be listed on the bottom of the log file)

Go to the Session Trace File C:\ProgramData\SolarWinds\Logs\Orion\CLI\Session-Trace\NCM

Open this in notepad use the following KB 

What is a Session Trace? A Session Trace logs all communication between NCM and the target device, once enabled. This will work only for console communications - where NCM is actively logging into the device to run commands, in order to download, upload or change a configuration. If NCM is manipulating the configuration via SNMP, then you will not see a session trace for that communication.

NCM How to Read a Session Trace

If you are unable to understand the Error why its failing -  Open Support Ticket and provide us the Session Trace file.

If you already have case opened please send this Session Trace file through case email 

Please do not forget to ZIP the Session Trace file 

Open Support Ticket

Tips and Tricks on opening a Support Ticket with SolarWinds