Submit a ticketCall us

Get a crash course on Network Monitoring delivered right to your inbox
This free 7-day email course provides a primer to the philosophy, theory, and fundamental concepts involved in IT monitoring. Lessons will explain not only how to perform various monitoring tasks, but why and when you should use them. Sign up now.

Home > Success Center > Network Configuration Manager (NCM) > NCM Jobs remain running and never complete

NCM Jobs remain running and never complete

Created by Bill Corgey, last modified by Bill Corgey on Mar 21, 2017

Views: 418 Votes: 2 Revisions: 11

Overview

This article provides brief information and steps to resolve the issue when a job appears completed.

The History reflects the job did not complete. 

 

Environment

NCM version 7.2 or later

NCM 7.4.1 (Hotfix 2 released)

 

Cause 

Any of the following can cause the issue:

  • Logging -  Logging has been enabled for scheduled jobs inventory, session tracing, security and left on for an extended period of time.  As these files grow in size, it takes longer and longer to access and write to these files.  Eventually there is enough overhead, the job itself fails and is not able to clear itself.
  • Download Config Jobs - There may be devices with very large config files such as Wireless LAN Controllers and Fortinet devices.  In some cases there are many of them and can accumulate and decrease resources and the job eventually fails.
  • If the job is an inventory job hanging - This can be due to the Routing tables. Items are enabled for the job.  If there are routers with large BGP or other large route tables, this can over task the application.
  • May fail in the event an email is set up for the job, and there is an error.  Commonly the email errors would be because an additional poller has been allowed to send emails.

 

Resolution

 

  • Logging:
    Turn off all logging and session tracing if not troubleshooting an issue. 
  • Download Config Jobs:
    Generally the large config files present an issue when transferring using SSH or telnet,  Using TFTP or SCP if possible will generally take less processing and decrease the time it takes to download.  In some cases breaking down a single job running XX number of nodes into multiple jobs may also help.  But for WLCs for sure using TFTP is optimal  For the Cisco WLC and the Cisco WISM, See Cisco Wireless Lan Controller/WLC device template for more information on Device Templates. 
  • For Inventory:
    Modify the job and remove the items for both route table options.
  • ​Test connection to the email server from each poller:
    If there is an issue with connection this could be a root cause.  Verify the server is approved or verify the email setting in NCM setting and the job are correct.  Verify reply to, from and to emails are valid and are not restricted by the email server.
    Note: In order to stop the job, it may be necessary to disable the job, wait a few moments and re-enable it.
  • At times a job may become corrupt, at this point it would be necessary to recreate the job.
  • Check the Collector logs located under C:\ProgramData\Solarwinds\Collector\Logs, if the Collector is having issues, run a Repair on Collector.

Resolution for the bug in NCM 7.4.1

 

External Link: http://downloads.solarwinds.com/solarwinds/Release/HotFix/NCM-v7.4.1-HotFix2.zip 

Readme: 

SolarWinds Network Configuration Manager (NCM) 7.4.1 Hotfix 2

Hotfix 2 addresses the following issue:
* Download config job may crash if config does not contain standard characters.


Notes:
* This fix is applicable for NCM 7.4.1 only.
___________________

Install SolarWinds NCM 7.4.1 Hotfix 2

1. Stop all Orion services.
2. Back up the original file: c:\Program Files (x86)\SolarWinds\Orion\NCM\SWConfigTransfer8.dll.
3. Replace c:\Program Files (x86)\SolarWinds\Orion\NCM\SWConfigTransfer8.dll with the binary in the HotFix 2 zip package.
4. Start Orion services.
5. Repeat these steps on all polling engines.

Uninstall SolarWinds NCM 7.4.1 Hotfix 2

Use the following steps if you want to rollback the Hotfix.

1. Stop all Orion services.
3. Replace c:\Program Files (x86)\SolarWinds\Orion\NCM\SWConfigTransfer8.dll with the original SWConfigTransfer8.dll.
4. Start Orion services.
5. Repeat these steps on all polling engine

 

 

 

 

Last modified
08:03, 21 Mar 2017

Tags

Classifications

Public