Submit a ticketCall us

Get a crash course on Network Monitoring delivered right to your inbox
This free 7-day email course provides a primer to the philosophy, theory, and fundamental concepts involved in IT monitoring. Lessons will explain not only how to perform various monitoring tasks, but why and when you should use them. Sign up now.

Home > Success Center > Storage Manager (STM) > Collector service stopped working due to huge amount of VMs assigned for collection

Collector service stopped working due to huge amount of VMs assigned for collection

Updated March 10, 2017

Overview

The data collection stopped for all devices assigned to the STM server, therefore Java could not maintain reachability to all Metadata java objects. 

Environment

  • STM 6.x
  • STM 5.x
  • Profiler 4.11.x

Cause 

Collector is being overloaded. The STM Collector is intended to be a 'collector' of collector, ie, a collector of all Java agents.

Resolution

The Collector service is under huge load, causing "data collection issues". First, we need to take care of the Collector. If there is more than 300 VMs assigned for collection, the Collector service will become saturated and when a full Java Garbage collection takes place, it will not be able to maintain reachability to all metadata java objects. Standalone agents with 4GB of RAM assigned can poll approximately 300 VM's. Collector service has 2GB of RAM assigned by default. Collector shares memory between modules, that means collection is not possible from any other device.

  1. Go to Settings, Manage Agent Assignments, click tab ESX.
  2. Select all ESX servers.
  3. Click "Assign to Agent".
  4. Select "Unassign".


Now using services.msc stop "Solarwinds Storage Manager Collector" service. We need to edit config to add bit more memory:

  1. Open Notepad as an Administrator.
  2. Find file SolarWinds.Storage.Collector.ini (by default in c:\Program Files\SolarWinds\Storage Manager Server\webapps\ROOT\bin\).
  3. Find line: param03=-Xmx2048m.
  4. Assuming you have 8GB of free RAM on the server, replace above line with: param03=-Xmx8192m.
  5. Save changes, restart Solarwinds Storage Manager Collector.


See if devices restart collection (may take around 1 - 2h for SMI-S based devices). Once you see devices collecting, you can re-assign some of the ESX hosts to other Java STM agents. Make sure not to exceed 300 VM's per any given STM java agent. To check current figures go to Reports > Quick Reports, using the Left Navigation pane and then check the report labeled  "Vmware to Agent assignment".

 

Also you will want to click on "Device to Agent Mapping Report" and check values there and couple them with the values you see from the VmWare to Agent mapping report.

 

Typical Rule of thumb is that any 1 STM agent is scalable (by default) to be able to maintain reachabilty to any 1 scenario but not cummulative: 

 

* Up to 300 VMs per any 1 STM agent OR

* 4-6 Storage arrays per any 1 STM agent OR

* 15-20 Fibre Channel Switches OR

* Up to 10 Netapps per any 1 STM agent

 

For more detail regarding setting up proper Java Heap Space sizes you can look at this article for reference: 

https://support.solarwinds.com/Success_Center/Storage_Manager_(STM)/Properly_size_Java_heap_space_in_STM#section_3

 

 

 

Last modified
13:42, 20 Jul 2017

Tags

Classifications

Public