Low disk space on an Orion Server

10/15/2018 6:51 PM

12/21/2022 3:03 PM

Issus presented by los space on Disk could cause:

Some Orion Services failed to start 

Deployment Health will provide a warning for Low Drive Space on particular Engine

This article answers how to investigate whether the SolarWinds software might be the cause of your Orion Servers running out of disk space.

Orion Platform products should not cause disk space issues except for the cases when you enable DEBUG logging or when you store NCM files on your C:\ drive. SolarWinds software does not control Windows Error Reporting files.

It can also happen on whichever drive Orion is installed to such as the D:\ drive, if you are sending logs or other data that can fill up the drive.

Network Performance Monitor

Low disk space caused by Orion Platform software could have various reasons, including the following:

• NCM Configurations stored in the SolarWinds ProgramData folder
• NCM Session tracing is on.
• Orion DEBUG logs are not turned off for other Orion Platform products, such as NCM, VIM, or SAM

SolarWinds file locations 

Orion Platform software has 4 main folder locations: 

• C:\Inetpub\SolarWinds
• C:\Windows\System32\msmq\storage\
• C:\Program Files (x86)\SolarWinds\ 

  • If this folder contains a copy of the NCM Config-Archive, move it to another location to save disk space.

• C:\ProgramData\Solarwinds\
  • This folder can take up 6-8 GB of data. It contains the MIB Database (~2GB) and installer files (~4 GB). These files should not be removed.
  • The folder also contains rolling logs. These should grow only when extra logging is enabled.

File location recommendations 

• The Program Files & Inetpub folders should remain static and can be configured during the installation to point to other disk Drives.
• ProgramData & MSMQ locations should remain in their default locations, that is at C:\Program Files (x86)\SolarWinds\.

 

To save disk space, complete the following steps:

• Validate Download folder and review if a big file can be moved to other volume or location
• Reset all logging to default
• Clean up orphaned log files
• Make Diagnostics tables small
• NCM: Move the config fields from their default location
• NCM: Clean up NCM Session Trace files or move them from the default location

Reset all logging to default:

1. Launch LogAdjuster using one of the following methods:
   • Run c:\Program Files (x86)\SolarWinds\Orion\LogAdjuster.exe
   • Click Start > RUN > All Programs > SolarWinds Orion > Documentation and Support > Orion Log Adjuster
2. In the Log Adjuster, select Reset to defaults, and click Apply.

Clean up orphaned Log Files 

You might have folders with thousands of log files that can be deleted without issues. Delete any files with the.LOG or.LOG.1-5 suffix.

To clean up SolarWinds logs, open the command line as an administrator, and run the following commands to clean up roll-over logs:

del /f /q /s C:\programdata\solarwinds\*.log
del /f /q /s C:\programdata\solarwinds\*.log.*

Keep Diagnostics tables small 

You can edit Diagnostics Config files not to pull back large tables.

• SolarwindsDiagnostics.exe checks the following Folder for Filetypes:
  • c:\Program Files (x86)\SolarWinds\Orion\ folder for any FILETYPE = ‘*.*diagnostics’ = ‘Diagnostics File’
  • Depending on what is returned from these files, it then gathers LOGS and Table samples accordingly.
• One can edit any Diagnostics Config file so next time its execute it exports fewer data.

For example:

1. Open the following file in a text editor: C:\Program Files (x86)\SolarWinds\Orion\NCM\NCMDiagnostics.config

2. Edit the following two lines so it pulls back less data:

Original config:

<database Description="Contents of NCM_Audit table" OutputFile="DB\NCM_Audit.csv">

select * from NCM_Audit WITH(NOLOCK) </database>

<database Description="Contents of NCM_CacheDiffResults table" OutputFile="DB\NCM_CacheDiffResults.csv">

select * from NCM_CacheDiffResults WITH(NOLOCK)

</database>

Updated config:

<database Description="Contents of NCM_Audit table" OutputFile="DB\NCM_Audit.csv">

select top 1000 * from NCM_Audit WITH(NOLOCK) </database>

<database Description="Contents of NCM_CacheDiffResults table" OutputFile="DB\NCM_CacheDiffResults.csv">

select top 1000 * from NCM_CacheDiffResults WITH(NOLOCK)

</database>

 

Move NCM Config files from their default location 

If you do not have the NCM module, you may skip this step.
If your environment site has NCM installed, NCM might store large folders of NCM-specific data.

• If NCM configuration files are stored in SolarWinds folder, move them to another drive, and set up a new default location using NCM web settings pages.
• Disable NCM SESSION-TRACING and delete the files.

Clean up Session Trace files or move them from the default location 

When Session Tracing is enabled, the following files are created:SwTelnetDebug.log and Session Traces data.

If you turn on Session Trace, be sure that you turn it off after you have collected the traces you need. To save disk space, delete the trace files. Do not keep session traces unless you are troubleshooting an issue.

How to set up a Session Trace 

1. Open the Orion Web Console.
2. Click Settings > All Settings, and then CLI settings.
3. Check the box to Enable/Disable Session Tracing.

How do you change where session trace files are stored 

By default, the session trace files are stored at C:\ProgramData\SolarWinds\Logs\Orion\CLI\Session-Trace or "%ALLUSERSPROFILE%\Application Data\SolarWinds\Logs\Orion\CLI\Session-Trace"