Applications Systems

Scale DPA for the number of databases being monitored and the number of custom metrics

SolarWinds DPA is a scalable solution that can grow as your number of monitored database instances increases. You can also scale DPA for a large number of custom metrics. This article describes how to configure DPA for scalability.

First published date

11/21/2018 11:14 PM

Last published date

12/18/2025 10:11 PM

Overview

SolarWinds DPA is a scalable solution that can grow as the number of database instances you are monitoring increases. You might also need to scale DPA if you configure a large number of custom metrics. This article describes how to configure DPA for scalability.

Note: Other components of your monitoring environment also need to be able to handle increased load, such as the repository database and server, the SolarWinds DPA Server machine, and the network.

Please consider having DPA in its own stand-alone environment when you have a medium to large instance count (76 and above).

Product section

Database Performance Analyzer

Resolution

Adjust the following settings to configure DPA to monitor up to 250 instances:

Java heap settings
General settings in the system.properties file

For DPA 2023.1 and later, increase metrics settings in the system.properties file when DPA is collecting a large number of metrics. DPA collects a large number of metrics in either of the following situations:

You are monitoring a large number of instances.
You have configured a large number of custom metrics.

Note: Do you plan to monitor more than 250 database instances? Consider using more than one SolarWinds DPA servers and Link together separate DPA servers.

Changing Java heap settings

By default, the maximum Java heap setting is configured to monitor up to 20 database instances. To change the maximum Java heap setting, follow the instructions in this article: Increase DPA heap memory to monitor large numbers of instances.

Change the settings to the appropriate value below:

Monitored instances	Max Java heap size (64-bit)
0 - 20	4096 MB
21 - 50	8192 MB
51 - 100	16384 MB
101 - 250	32768 MB

Changing settings in the system.properties file

By default, other DPA properties are configured to monitor up to 75 database instances. You can increase property settings to monitor up to 250 instances.

Note: Increasing these settings puts additional load on the repository, so increase them only if your repository has available processing bandwidth.

Edit the following file:
installdir\iwc\tomcat\ignite_config\idc\system.properties
Uncomment or add the line(s) for the properties you want to change. You can cut and paste lines from later in this article. The properties and recommended values are listed in the following sections:
- General property settings
- Metrics property settings (DPA 2023.1 or later)
Update each property to its new value. If multiple property entries are listed (below) for a single setting, update all property entries to the same value.
Save the file.
Restart the SolarWinds DPA server.

General property settings

Change these general property settings when you are monitoring more than 75 database instances. Recommended values are in the following sections:

Values for general property settings when the repository is on a different server than DPA
Values for general property settings when DPA and the repository are on the same server

Setting	Property entry in system.properties
Min Repo Connection Pool Size	com.confio.idc.repositoryDataSourceFactory.minPoolSize
Max Repo Connection Pool Size	com.confio.idc.repositoryDataSourceFactory.maxPoolSize
Trending Thread Pool Size	com.confio.idc.services.trendExecutor.maxPoolSize com.confio.idc.services.trendExecutor.corePoolSize
Cleaner Thread Pool Size	com.confio.idc.services.lowPriorityExecutor.maxPoolSize com.confio.idc.services.lowPriorityExecutor.corePoolSize
C3PO Helper Thread Count	com.confio.idc.repositoryDataSourceFactory.numHelperThreads
SQL Stats Connection Permits	com.confio.idc.repositoryDataSourceSemaphore.permits
DB Manager Queue Capacity	com.confio.idc.databaseManagerExecutor.queueCapacity

Values for general property settings when the repository is on a different server than DPA

If your repository has the available processing bandwidth, you can increase the following settings to monitor more than 75 database instances.

	0 - 75 instances	76 - 150 instances	151 - 200 instances	201 - 250 instances
Min Repo Connection Pool Size	10	20	30	40
Max Repo Connection Pool Size	200	300*	400*	500*
Trending Thread Pool Size	10	15	20	25
Cleaner Thread Pool Size	5	10	15	20
C3PO Helper Thread Count	25	30	40	50
SQL Stats Connection Permits	20	25	30	35
DB Manager Queue Capacity	250	300	400	500

*Make sure your repository database is capable of handling up to this many connections.

Values for general property settings when DPA and the repository are on the same server

Increase the following settings to enable DPA to monitor more than 75 database instances. Increasing these settings enables DPA to do more processing in parallel, which in turn puts additional load on the repository. Because DPA and the repository are "co-resident", increased repository load completes with increased DPA processing, and might cause DPA processing to get behind. Indications of this include:

The number of repository connections equals the maximum pool setting.
The DPA UI is unresponsive at times.
The DPA logs contain "Task rejected" errors.

In a co-resident situation, it is therefore necessary to limit the amount of parallel processing done by DPA. The following table shows values that either decrease or have modest increases as the number of monitored instances increases.

Note: The decreasing values shown here are guidelines, and can vary based on the processing power of the server that is running DPA and the repository. Less-powerful machines can require lower values.

	0 - 75 instances	76 - 150 instances	151 - 200 instances	201 - 250 instances
Min Repo Connection Pool Size	10	20	30	40
Max Repo Connection Pool Size	200	300*	400*	500*
Trending Thread Pool Size	10	15	20	20
Cleaner Thread Pool Size	5	10	15	15
C3PO Helper Thread Count	25	30	40	50
SQL Stats Connection Permits	20	20	20	20
DB Manager Queue Capacity	250	300	400	500

*Make sure your repository database is capable of handling up to this many connections.

Here are the lines for each scale settings needed. This can be cut pasted to the file and the needed lines uncommented.

##################################################################

#Scale DPA settings 0-75

##################################################################

#com.confio.idc.services.trendExecutor.maxPoolSize=10

#com.confio.idc.services.trendExecutor.corePoolSize=10

#com.confio.idc.services.lowPriorityExecutor.maxPoolSize=5

#com.confio.idc.services.lowPriorityExecutor.corePoolSize=5

#com.confio.idc.repositoryDataSourceFactory.numHelperThreads=25

#com.confio.idc.repositoryDataSourceSemaphore.permits=20

#com.confio.idc.databaseManagerExecutor.queueCapacity=250

#com.confio.idc.repositoryDataSourceFactory.minPoolSize=10

#com.confio.idc.repositoryDataSourceFactory.maxPoolSize=200

##################################################################

#Scale DPA settings 76-150

##################################################################

#com.confio.idc.services.trendExecutor.maxPoolSize=15

#com.confio.idc.services.trendExecutor.corePoolSize=15

#com.confio.idc.services.lowPriorityExecutor.maxPoolSize=10

#com.confio.idc.services.lowPriorityExecutor.corePoolSize=10

#com.confio.idc.repositoryDataSourceFactory.numHelperThreads=30

#com.confio.idc.repositoryDataSourceSemaphore.permits=20

#com.confio.idc.databaseManagerExecutor.queueCapacity=300

#com.confio.idc.repositoryDataSourceFactory.minPoolSize=20

#com.confio.idc.repositoryDataSourceFactory.maxPoolSize=300

##################################################################

#Scale DPA settings 151-200

##################################################################

#com.confio.idc.services.trendExecutor.maxPoolSize=20

#com.confio.idc.services.trendExecutor.corePoolSize=20

#com.confio.idc.services.lowPriorityExecutor.maxPoolSize=15

#com.confio.idc.services.lowPriorityExecutor.corePoolSize=15

#com.confio.idc.repositoryDataSourceFactory.numHelperThreads=40

#com.confio.idc.repositoryDataSourceSemaphore.permits=20

#com.confio.idc.databaseManagerExecutor.queueCapacity=400

#com.confio.idc.repositoryDataSourceFactory.minPoolSize=30

#com.confio.idc.repositoryDataSourceFactory.maxPoolSize=400

##################################################################

#Scale DPA settings 200-250

##################################################################

#com.confio.idc.services.trendExecutor.maxPoolSize=20

#com.confio.idc.services.trendExecutor.corePoolSize=20

#com.confio.idc.services.lowPriorityExecutor.maxPoolSize=15

#com.confio.idc.services.lowPriorityExecutor.corePoolSize=15

#com.confio.idc.repositoryDataSourceFactory.numHelperThreads=50

#com.confio.idc.repositoryDataSourceSemaphore.permits=20

#com.confio.idc.databaseManagerExecutor.queueCapacity=500

#com.confio.idc.repositoryDataSourceFactory.minPoolSize=40

#com.confio.idc.repositoryDataSourceFactory.maxPoolSize=500

Metrics execution thread pool (DPA 2023.1 or later)

Change the following metrics thread pool settings when DPA 2023.1 or later is collecting a large number of metrics (default metrics or custom metrics). Recommended values are in the following sections.

Note: To see the default metrics that DPA collects for each monitored database type, see Metrics collected by DPA.

Setting	Property entry in system.properties
Core Pool Size	com.confio.idc.metricsTaskExecutor.corePoolSize
Max Pool Size	com.confio.idc.metricsTaskExecutor.maxPoolSize
Queue Capacity	com.confio.idc.metricsTaskExecutor.queueCapacity

Calculating property values

The values for your DPA deployment can be calculated as follows:

Core Pool Size (C) = The number of monitored instances.
Max Pool Size (M) = C * 3.
Queue Capacity = ((T - M) * 2), where T is the total number of metric query requests which is calculated as:
- (C * the number of default metrics) + (C * the number of custom metrics)

Example:

If you are monitoring 75 SQL Server instances that have 52 default metrics and 100 custom metrics:

Core Pool Size (C): 75
Max Pool Size (M): 75 * 3 = 225
Total metric query requests: (75 * 52) + (75 * 100) = 11400
Queue Capacity: (11,400 - 225) * 2 = 22350

Examples of system.properties entries scaled for different deployment sizes

The following examples of system.properties entries show values scaled for different deployment sizes. These values were calculated using 52 as the number of default metrics per instance and 100 as the number of custom metrics. You can cut and paste these lines into the system.properties file and then uncomment the lines. If needed, adjust the values using the formula above.

##################################################################

#Scale DPA settings 0-75

##################################################################

com.confio.idc.metricsTaskExecutor.corePoolSize=75

com.confio.idc.metricsTaskExecutor.maxPoolSize=225

com.confio.idc.metricsTaskExecutor.queueCapacity=22350

##################################################################

#Scale DPA settings 76-150

##################################################################

com.confio.idc.metricsTaskExecutor.corePoolSize=150

com.confio.idc.metricsTaskExecutor.maxPoolSize=450

com.confio.idc.metricsTaskExecutor.queueCapacity=44700

##################################################################

#Scale DPA settings 151-200

##################################################################

com.confio.idc.metricsTaskExecutor.corePoolSize=200

com.confio.idc.metricsTaskExecutor.maxPoolSize=600

com.confio.idc.metricsTaskExecutor.queueCapacity=59600

##################################################################

#Scale DPA settings 200-250

##################################################################

com.confio.idc.metricsTaskExecutor.corePoolSize=250

com.confio.idc.metricsTaskExecutor.maxPoolSize=750

com.confio.idc.metricsTaskExecutor.queueCapacity=74500

Disclaimer: Please note, any content posted herein is provided as a suggestion or recommendation to you for your internal use. This is not part of the SolarWinds software or documentation that you purchased from SolarWinds, and the information set forth herein may come from third parties. Your organization should internally review and assess to what extent, if any, such custom scripts or recommendations will be incorporated into your environment. You elect to use third-party content at your own risk, and you will be solely responsible for the incorporation of the same if any.