Submit a ticketCall us

Webinar: Web Help Desk for HR, Facilities and Accounting Departments
This webinar will focus on use cases for HR, Facilities and Accounting.

Having a unified ticketing and asset management system for all the departments in your company can provide end-users with a seamless experience and make things easier for your IT team. Yet, with different business tasks and objectives, many departments don’t fully understand the capabilities of Web Help Desk and how the software can be customized for effective use in their departments.
Register Now.

Home > Success Center > Database Performance Analyzer (DPA) > Storage I/O polling is disabled because the file limit was exceeded

Storage I/O polling is disabled because the file limit was exceeded

Updated March 11th, 2016

Issue

The Storage I/O feature in DPA frequently collects file statistics from a monitored database instance. If the instance includes a large number of files, collecting I/O data can incur excessive overhead for both the monitored instance and for DPA. To avoid performance issues, DPA automatically disables storage I/O polling when the number of files exceeds a specified limit. The default limit is 250.

 

If storage I/O polling is disabled, you will see a message similar to the one below when you attempt to access Storage I/O:

Storage I/O has been automatically disabled for this database instance due to the large number of files in this database instance. This was done because gathering I/O data for a large number of files may negatively affect the performance of both DPA and the database instance.
353 files detected. Current limit is 250.

Additionally, the idc.log includes a log entry similar to the one below:

IOCollectionServiceImpl - Disabling IO stats collection for database [id=3] because the number of files 315 exceeds the IO stats poll file limit.

Considerations

If you choose to re-enable storage I/O polling for a database instance with a large number of files, be aware of the following options and their effect on performance.

Option

Description

IO_STATS_POLL_ENABLED
(Automatic vs. manual enablement mode)

Determines whether storage I/O polling is enabled, and whether this value automatically changes based on the file limit. The following values are available:

  • AUTO_ON: Polling is currently enabled. It will be automatically disabled if the number of files exceed the file limit.
  • AUTO_OFF: Polling is currently disabled because the file limit is exceeded. It will be automatically re-enabled if the number of files drops below the file limit.
  • MANUAL_ON: Polling is manually enabled. DPA ignores the file limit, and storage I/O data is gathered regardless of the number of files. (Selecting this value can negatively affect performance.)
  • MANUAL_OFF: Polling is manually disabled.

If you see the error described in the previous section, DPA has changed this option to AUTO_OFF.

IO_STATS_POLL_FILE_LIMIT
(File limit)

Specifies the maximum number of files that can reside on a database instance before DPA automatically disables storage I/O polling. Applies only if the enablement mode is AUTO_ON or AUTO_OFF.

IO_STATS_POLL_INTERVAL
(Polling frequency)

Specifies how often storage I/O data is collected. Polling too frequently with a large number of files can negatively affect performance.

Enabling the Storage I/O Feature

To re-enable storage I/O polling for a monitored instance with a large number of files, complete the following steps to increase the file limit and (if necessary) decrease the polling frequency.

Note: Alternatively, you can tell DPA to ignore the file limit and always enable storage I/O polling. To do so, set IO_STATS_POLL_ENABLED (on the DB Instance Options tab) to MANUAL_ON. However, be aware that MANUAL_ON is not recommended because it can negatively affect performance.

We recommend that you follow these instructions under the guidance of SolarWinds Support.

  1. Increase the file limit:
    1. Click Options.
    2. Click the Administration tab.
    3. Click Advanced Options.
    4. Select Support Options in the upper right corner.
    5. On the System Options tab, click IO_STATS_POLL_FILE_LIMIT and enter a value that is larger than the number of files in the error message.
    6. Restart the monitor.

      If the number of files is below the file limit, IO_STATS_POLL_ENABLED automatically changes to AUTO_ON. Storage I/O polling is re-enabled.
       
  2. On a regular basis, review the idc.log file for errors associated with I/O data collection. Examples:
    IOCollectionServiceImpl - Error collecting IO Stats for QASQL2K12. IO Stat collection for this entity will be retried in 5 minutes. Retries will continue indefinitely, however, this error will only be logged after every 10th failed attempt. This error can be ignored if sporadic.

    IOCollectionServiceImpl - IO Stats query gap detected for dbId [41] last collection: 03/14/2015 08:55:11, current collection: 03/14/2015 08:56:31, difference [79995] > 30000

    IOCollectionServiceImpl - Added file to blacklist [dbId:35] [dbId=2, fileId=5 [2|5]] [expires: 03/16/2015 12:05:30] Missing stats detected.
     
  3. If errors occur, increase the polling interval:
    1. Click Options.
    2. Click the Administration tab.
    3. Click Advanced Options.
    4. Select the Support Options checkbox in the upper right corner.
    5. Click the DB Instance Options tab.
    6. Select the database instance.
    7. Click IO_STATS_POLL_INTERVAL and increase the value. We recommend 30 or 60 (seconds).
    8. Restart the monitor.
Last modified
14:18, 7 Apr 2017

Tags

Classifications

Public