Submit a ticketCall us

Webinar: Web Help Desk for HR, Facilities and Accounting Departments
This webinar will focus on use cases for HR, Facilities and Accounting.

Having a unified ticketing and asset management system for all the departments in your company can provide end-users with a seamless experience and make things easier for your IT team. Yet, with different business tasks and objectives, many departments don’t fully understand the capabilities of Web Help Desk and how the software can be customized for effective use in their departments.
Register Now.

Home > Success Center > Netflow Traffic Analyzer (NTA) > Netflow service will not start: Corrupted memory errors in the flow storage logs

Netflow service will not start: Corrupted memory errors in the flow storage logs

Created by Matthew Lamb, last modified by MindTouch on Jun 23, 2016

Views: 134 Votes: 2 Revisions: 4

Overview

This article addresses an issue that can occur and bring the NTA flow storage service down, resulting in the NTA service on the collector from starting as well.

The following error may be found in the FastBitServerService.logs on the Flow storage database server under %programdata%\SolarWinds\Logs\NTA:

2016-04-04 12:49:20,737 [21] WARN  ibis.column - logWarning (column.cpp:5059):  Warning -- column[TN#2016-03-20@11_0.DestinationDomain](CATEGORY)::readIndex -- index file "E:\ProgramData\SolarWinds\NTA\FlowStorage\Data\Flows\TN#2016-03-20@11_0\DestinationDomain-0.idx" contains an incorrect header (
2016-04-04 12:49:20,753 [21] ERROR SolarWinds.Netflow.FastBit.Server.Service.Program - NetFlowService will be abnormally terminated - UnhandledException was caught : System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
   at FastbitCLR.FastbitColumn.PerformCompleteBackup(FastbitColumn* , basic_ostream<char\,std::char_traits<char> >* stream, BackupTransaction transaction)
   at FastbitCLR.FastbitPartition.PerformBackup(Stream stream, BackupTransaction transaction)
   at FastbitCLR.FastbitTable.PerformBackup(Stream stream, DateTime minTimeSlot, DateTime maxTimeSlot, BackupTransaction transaction)
   at FastbitCLR.FastbitDatabase.PerformBackup(Stream stream, DateTime minTimeSlot, DateTime maxTimeSlot, BackupTransaction transaction)
   at SolarWinds.Netflow.FastBit.Server.Backup.BackupManager.PerformBackup(String directory, FastbitDatabase db, Int64 timeSlot, BackupTransaction transaction)
   at SolarWinds.Netflow.FastBit.Server.Backup.BackupManager.BackupThread()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
2016-04-04 12:49:20,753 [21] ERROR SolarWinds.Netflow.FastBit.Server.Service.Program - Unhandled exception: Message: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
 Stack Trace:    at FastbitCLR.FastbitColumn.PerformCompleteBackup(FastbitColumn* , basic_ostream<char\,std::char_traits<char> >* stream, BackupTransaction transaction)
   at FastbitCLR.FastbitPartition.PerformBackup(Stream stream, BackupTransaction transaction)
   at FastbitCLR.FastbitTable.PerformBackup(Stream stream, DateTime minTimeSlot, DateTime maxTimeSlot, BackupTransaction transaction)
   at FastbitCLR.FastbitDatabase.PerformBackup(Stream stream, DateTime minTimeSlot, DateTime maxTimeSlot, BackupTransaction transaction)
   at SolarWinds.Netflow.FastBit.Server.Backup.BackupManager.PerformBackup(String directory, FastbitDatabase db, Int64 timeSlot, BackupTransaction transaction)
   at SolarWinds.Netflow.FastBit.Server.Backup.BackupManager.BackupThread()
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart() 
 Type: System.AccessViolationException

Environment

NTA 4.0+

Cause 

This occurs because a table index is corrupted and cannot be read correctly. The reasons for this can be either due to memory or memory allocation issues on the server in question, but the most typical issue is with antivirus scanning conducted on the Flow Storage directory that can corrupt the table information in the database directory.

Resolution

  1. On the flow storage database server, stop the Netflow Service from starting back up.
  2. Go to E:\ProgramData\SolarWinds\NTA\FlowStorage\Data\Flows\
  3. Locate the directory listed in the error.
  4. Move that directory to the desktop of that server.
  5. Restart the service and observe if the service starts back up.

 

Note: The directory will very likely not be recoverable and placing it back to attempt recovery will result in further outage.

 

SolarWinds recommends that if you have an antivirus application on that server, verify that it is either disabled or configured with exclusions for the SolarWinds directories based off this article here.

 

 

Last modified
20:36, 22 Jun 2016

Tags

Classifications

Public