Syslog service is crashing due to insufficient memory

11/29/2018 10:37 PM

10/21/2022 8:42 AM

The following errors are reported in the SyslogService.log located in the C:\ProgramData\SolarWinds\Logs\Orion folder.
 

[498] ERROR SolarWinds.Orion.Common.SWEventLogging - Service was unable to open new database connection when requested.

[UDPListenerThread] ERROR SolarWinds.Orion.Common.SWEventLogging - Server Exception Error: System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. Line 152: 2015-11-19 01:59:14,548 [UDPListenerThreadIpv6] ERROR SyslogService.SyslogService - A blocking operation was interrupted by a call to WSACancelBlockingCall

[506] ERROR SolarWinds.Orion.Common.ManagedNodeState - ManagedNodeState.NodeRefresher(): Error refreshing managed node status from Nodes table! Line 160: 2015-11-19 01:59:15,203 [397] ERROR SolarWinds.Orion.Common.ManagedNodeState - ManagedNodeState.NodeRefresher(): Error refreshing managed node status from Nodes table!

[98] ERROR SolarWinds.Orion.Common.SWEventLogging - Syslog Service Unhandled Exception Cannot access a disposed object.

The article discusses why the Syslog service is periodically crashing and how it can be addressed 

Network Performance Monitor

The system is running low on memory, causing the Syslog service to crash due to a large amount of incoming data.

Follow the next steps to increase the time outs for the database connection and command:
 

1. Go to <installationpath>:\Program Files (x86)\Solarwinds\Orion.
2. Open the SWNetPerfMon.db file.
3. Scroll down to the bottom and change the following

   Connection timeout in seconds - Timeout=20 to 200
   Database Command timeout in seconds - CommandTimeout=90 to 900

4. Save the file then stop and restart all services.

 

Alternatively, you can also run the following SQL command to find the Syslog top talkers and attempt to limit the Syslog export at the device end to prevent the flood of Syslog from crashing the service:
 

-- Scripts are not supported under any SolarWinds support program or service.
-- Scripts are provided AS IS without warranty of any kind. SolarWinds further
-- disclaims all warranties including, without limitation, any implied warranties
-- of merchantability or of fitness for a particular purpose. The risk arising
-- out of the use or performance of the scripts and documentation stays with you.
-- In no event shall SolarWinds or anyone else involved in the creation,
-- production, or delivery of the scripts be liable for any damages whatsoever
-- (including, without limitation, damages for loss of business profits, business
-- interruption, loss of business information, or other pecuniary loss) arising
-- out of the use of or inability to use the scripts or documentation.

SELECT TOP 100000 Hostname, COUNT(*) FROM Syslog
WHERE DateTime>DATEADD(DAY, -1, GETDATE())
GROUP BY Hostname

If the above options fail, it is recommended to increase the memory size of the host in question.
Another workaround is to configure the unnecessary devices to stop sending Syslog messages to Solarwinds