Unknown Applications and Unexpected Website Behavior: Over Polling SAM Component Monitors

This article has a high rate of resolution to such statements:
"My system has been working for many months, why all of a sudden is there an issue?"
"My polling rate has not exceeded 100%, how could I be affected by over polling SAM components?"

Over polling SAM component monitors are a common cause of many different issues. Overloading the polling engine(s) can cause a wide variety of issues, many of the symptoms fall under the categories of unexpected website behavior and inconsistent polling for monitored objects (nodes/agents/applications/etc).

All known over polling symptoms:

• Application issues
  • Applications are unknown
  • Applications are not polling
  • Applications are stuck in initial poll in progress
  • Applications display unable to start polling
  • Gaps in metrics
  • Not able to assign new applications to nodes
  • Not able to delete applications from nodes
• Node issues
  • Nodes are not polling
  • Nodes not polling at expected frequency (10min)
  • Agents are failing to poll status [icmp & agent native status pollers]
• Other polled object issues
  • Volumes not polling at expected frequencies
  • Interfaces not polling at expected frequencies
• Website issues
  • Inconsistent Website behavior
  • Web pages are slow to load
  • Web pages do not load or time out (receive: Unexpected Website Error)
  • Report data is incorrect or outdated
  • Gaps in various statistic charts
  • Alerts not firing
• System issues
  • SQLite collector queues filling up

Please note the following for all scripts contained herein:

-- Scripts are not supported under any SolarWinds support program or service.
-- Scripts are provided AS IS without warranty of any kind. SolarWinds further
-- disclaims all warranties including, without limitation, any implied warranties
-- of merchantability or of fitness for a particular purpose. The risk arising
-- out of the use or performance of the scripts and documentation stays with you.
-- In no event shall SolarWinds or anyone else involved in the creation,
-- production, or delivery of the scripts be liable for any damages whatsoever
-- (including, without limitation, damages for loss of business profits, business
-- interruption, loss of business information, or other pecuniary loss) arising
-- out of the use of or inability to use the scripts or documentation.

Managing Monitored Objects in the SolarWinds Platform

Ensure your system is up to the requirements of the Scalability Engine Guidelines for SolarWinds products. Over polling network elements or SolarWinds Platform agents have been known to have the same or similar effects as over polling SAM components.

Managing SAM Component Monitors

Reduce the number of components polled by each polling engine. Polling engines are designed to poll 40,000 components per polling engine. To check the monitored component count across all polling engines, open the web console and go to 

SELECT COUNT(DISTINCT NodeID) FROM APM_Application

1. Go to Settings > Network Discovery
2. Select one of your discovery jobs and click edit. 
3. Go to the monitoring settings tab.
4. If you have the option: "Automatically monitor based on my defined monitoring settings" selected then click "Define Monitoring Settings" (if you have manual discovery option selected then please disregard these steps).
5. In the window that pops up go to the Applications tab. 
6. It is recommended to uncheck the AppInsight Applications from automatic monitoring. Otherwise it could apply unnecessary monitors across your environment.

SELECT t.Name AS TemplateName, a.Name AS AppName, Count(ccs.ApplicationID) AS [Component Count], 
n.Caption AS [Node Name] 
FROM [dbo].[APM_CurrentComponentStatus] ccs
JOIN [APM_Application] a ON ccs.ApplicationID = a.ID
JOIN [Nodesdata] n ON a.NodeID = n.NodeID
JOIN [APM_ApplicationTemplate] t ON a.TemplateID = t.ID
GROUP BY t.Name, a.Name, n.Caption
ORDER BY [Component Count] DESC

SELECT ccs.ApplicationID, ap.Name AS [Application Name], t.Name AS [Template Name], Count(ccs.ApplicationID) AS [Component Count], 
node.Caption AS [Node Name], e.EngineID, e.ServerName AS [Polling Engine]
FROM [dbo].[APM_CurrentComponentStatus] ccs
JOIN [APM_Application] ap ON ccs.ApplicationID = ap.ID
JOIN [Nodesdata] node ON ap.NodeID = node.NodeID
JOIN [Engines] e ON node.EngineID = e.EngineID
JOIN [APM_ApplicationTemplate] t ON t.ID = ap.TemplateID
GROUP BY ap.Name, t.Name, ccs.ApplicationID, node.Caption, e.EngineID, e.ServerName
ORDER BY [Component Count] DESC