Submit a ticketCall us

Welcome to the NEW Success Center. Search all resources (documentation, videos, training, knowledge base articles) or browse resources by product. If you are unable to find what you are looking for, please contact us at customersuccess@solarwinds.com

 

 

 

 

Home > Success Center > Netflow Traffic Analyzer (NTA) > Netflow service will not start: Corrupted memory errors in the flow storage logs

Netflow service will not start: Corrupted memory errors in the flow storage logs

Created by Matthew Lamb, last modified by MindTouch on Jun 23, 2016

Views: 1,034 Votes: 1 Revisions: 4

Overview

This article addresses an issue that can occur and bring the NTA flow storage service down, resulting in the NTA service on the collector from starting as well.

The following error may be found in the FastBitServerService.logs on the Flow storage database server under %programdata%\SolarWinds\Logs\NTA:

2016-04-04 12:49:20,737 [21] WARN  ibis.column - logWarning (column.cpp:5059):  Warning -- column[TN#2016-03-20@11_0.DestinationDomain](CATEGORY)::readIndex -- index file "E:\ProgramData\SolarWinds\NTA\FlowStorage\Data\Flows\TN#2016-03-20@11_0\DestinationDomain-0.idx" contains an incorrect header (
2016-04-04 12:49:20,753 [21] ERROR SolarWinds.Netflow.FastBit.Server.Service.Program - NetFlowService will be abnormally terminated - UnhandledException was caught : System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at FastbitCLR.FastbitColumn.PerformCompleteBackup(FastbitColumn* , basic_ostream<char\,std::char_traits<char> >* stream, BackupTransaction transaction)
   at FastbitCLR.FastbitPartition.PerformBackup(Stream stream, BackupTransaction transaction)
   at FastbitCLR.FastbitTable.PerformBackup(Stream stream, DateTime minTimeSlot, DateTime maxTimeSlot, BackupTransaction transaction)
   at FastbitCLR.FastbitDatabase.PerformBackup(Stream stream, DateTime minTimeSlot, DateTime maxTimeSlot, BackupTransaction transaction)
   at SolarWinds.Netflow.FastBit.Server.Backup.BackupManager.PerformBackup(String directory, FastbitDatabase db, Int64 timeSlot, BackupTransaction transaction)
   at SolarWinds.Netflow.FastBit.Server.Backup.BackupManager.BackupThread()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
2016-04-04 12:49:20,753 [21] ERROR SolarWinds.Netflow.FastBit.Server.Service.Program - Unhandled exception: Message: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
 Stack Trace:    at FastbitCLR.FastbitColumn.PerformCompleteBackup(FastbitColumn* , basic_ostream<char\,std::char_traits<char> >* stream, BackupTransaction transaction)
   at FastbitCLR.FastbitPartition.PerformBackup(Stream stream, BackupTransaction transaction)
   at FastbitCLR.FastbitTable.PerformBackup(Stream stream, DateTime minTimeSlot, DateTime maxTimeSlot, BackupTransaction transaction)
   at FastbitCLR.FastbitDatabase.PerformBackup(Stream stream, DateTime minTimeSlot, DateTime maxTimeSlot, BackupTransaction transaction)
   at SolarWinds.Netflow.FastBit.Server.Backup.BackupManager.PerformBackup(String directory, FastbitDatabase db, Int64 timeSlot, BackupTransaction transaction)
   at SolarWinds.Netflow.FastBit.Server.Backup.BackupManager.BackupThread()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart() 
 Type: System.AccessViolationException

Environment

NTA 4.0+

Cause 

This occurs because a table index is corrupted and cannot be read correctly. The reasons for this can be either due to memory or memory allocation issues on the server in question, but the most typical issue is with antivirus scanning conducted on the Flow Storage directory that can corrupt the table information in the database directory.

Resolution

  1. On the flow storage database server, stop the Netflow Service from starting back up.
  2. Go to E:\ProgramData\SolarWinds\NTA\FlowStorage\Data\Flows\
  3. Locate the directory listed in the error.
  4. Move that directory to the desktop of that server.
  5. Restart the service and observe if the service starts back up.

 

Note: The directory will very likely not be recoverable and placing it back to attempt recovery will result in further outage.

 

SolarWinds recommends that if you have an antivirus application on that server, verify that it is either disabled or configured with exclusions for the SolarWinds directories based off this article here.

 

 

Last modified
20:36, 22 Jun 2016

Tags

Classifications

Public