Applications Systems
WPM Worker – high volume of CRL/OCSP traffic and transaction latency when firewall blocks outbound connections on Port 80
The solarwinds.seum.agent.worker.exe process on the WPM Server repeatedly attempts outbound connections to Certificate Authority (CA) Certificate Revocation List (CRL) and OCSP endpoints on port 80. This generates a high volume of requests that may flood proxy logs, trigger security alerts, or be blocked by firewall or proxy policies in restricted environments.
First published date
Last published date
Overview
Each time WPM spawns a Worker process to execute a transaction, two components may independently perform certificate revocation checks:
-
The Windows Automatic Root Certificate Update component — Windows checks whether the SSL certificates of monitored websites are trusted, reaching out to the certificate authority's CRL Distribution Point (CDP) or OCSP endpoint to verify revocation status.
-
The WPM Browser (Chromium/Electron engine) — The embedded browser performs its own CRL/OCSP validation for SSL certificates encountered during transaction playback.
Because the CRL/CTL cache is per-process and discarded when each Worker exits, every new transaction cycle re-triggers these outbound lookups. With multiple Workers running in parallel, this accumulates into a high-volume, continuous stream of HTTP requests on port 80 directed at CA endpoints.
Note: This traffic is expected behavior in internet-connected environments. It does not indicate a security compromise or product defect. The connections are standard Windows and Chromium certificate validation mechanisms.
Symptoms
-
High volume of outbound connections on port 80 from the WPM Server to certificate authority CRL/OCSP endpoints
-
Source process identified as
solarwinds.seum.agent.worker.exe -
Traffic is blocked or denied by a proxy or firewall
-
Proxy logs flooded with denied entries from the WPM Server
-
Security/network teams raise alerts due to volume of outbound requests
-
WPM transaction step durations increase from sub-second to 15–60+ seconds per step after a platform upgrade or firewall policy change
-
Step duration chart shows a consistent ~30-second plateau — actual page load time confirmed normal (3–5 seconds) via browser DevTools
-
Not all transactions on the same player are affected — only transactions monitoring sites that load external resources (CSS, JavaScript, fonts, CRL/OCSP endpoints) from URLs blocked by the firewall
-
Issue is progressive — dispatch gaps between transaction playback cycles worsen over days (1–3 minute gaps initially, escalating to 14–15 minute gaps within 7 days)
-
AgentWorker logs show probe execution times of 15+ seconds but results write-back to the Orion server completes in under 1 second — confirming the delay is in the Chromium browser layer, not the database
-
All probes return exit code 0 — the browser automation completes successfully but slowly
-
Cloud-based WPM Players (AWS/Azure) on the same WPM version are unaffected — only on-premises players behind the firewall exhibit the latency
Product section
Cause
The WPM Worker process is a short-lived, per-transaction process. Each Worker spawn creates a new process context with no inherited CRL cache. As a result:
-
Windows re-initiates root certificate and revocation checks on each Worker spawn
-
The WPM Browser (Chromium/Electron) also independently checks OCSP/CRL for the SSL certificates of monitored websites
-
In environments where the monitored URLs use certificates issued by major public CAs (such as DigiCert, GlobalSign, or Comodo), these checks are directed to the corresponding CA endpoints
-
When the CRL/OCSP endpoints are blocked by proxy or firewall policy, the requests do not fail immediately. Instead, each blocked request waits until the connection timeout expires — typically 15–30 seconds per blocked endpoint. The requests are then re-attempted on every subsequent Worker spawn
-
Additionally, the WPM Browser (Chromium/Electron) loads all resources referenced by the monitored page during transaction playback, including external CSS files, JavaScript libraries, fonts, and images hosted on third-party CDN endpoints. If the firewall blocks outbound HTTP/HTTPS connections to these CDN endpoints, the page load stalls until each blocked resource request times out. Because multiple blocked requests can accumulate per transaction step, step durations inflate from sub-second to 30+ seconds per step, causing false Critical and Down alerts.
This behavior is more likely to surface after a WPM version upgrade. The Electron/Chromium engine version changes between WPM releases (for example, Electron 37.9.0 in WPM 2026.1 to Electron 40.8.5 in WPM 2026.2) can introduce different handling of certificate revocation checking and external resource loading, making previously tolerated firewall blocks cause visible latency under the newer engine version.
Resolution
Choose the appropriate suppression method based on your environment. Option 2 (Electron switch) is recommended for most deployments running WPM 2022.x and later.
Important: Option 2 (RDP into SEUM-User accounts) is not applicable on WPM 2022.x and later. As of version 2022.x, SEUM-User account passwords are randomly generated and are not user-editable. See Specify a password for WPM user accounts for more information.
Diagnostic: How to Confirm Firewall-Blocked Requests Are Causing WPM Latency
-
Check the AgentWorker logs on the affected WPM Player machine at
C:\ProgramData\SolarWinds\Logs\SEUM\Player\AgentWorker_v2026_*_[PID].log. Compare the probe execution time (between "Preparing probe..." and "Processing results...") against the results write-back time (between "Processing results..." and "Sending results END"). If probe execution is 15+ seconds but write-back is under 1 second, the delay is in the Chromium browser layer. -
Compare on-premises WPM Players against cloud-based (AWS/Azure) WPM Players running the same WPM version. If cloud players complete transactions normally (3–5 seconds) while on-premises players show 30+ second step durations, the issue is environment-specific (firewall/network), not a product defect.
-
Check which transactions on the same player are affected versus normal. If some transactions complete in 1–3 seconds while others on the same player take 30+ seconds, the affected transactions are monitoring sites that load external resources from blocked endpoints. This rules out player-level issues (CPU, memory, disk, DNS, proxy).
Option 1 – Whitelist Blocked Endpoints in Firewall (Recommended when endpoints can be identified)
If the firewall or proxy logs show specific URLs being blocked from the WPM Player machines, work with your network/security team to whitelist those endpoints. Common endpoints that the WPM Browser (Chromium/Electron) may contact during transaction playback include:
Certificate Authority CRL/OCSP endpoints (port 80):
CDN and external resource endpoints (port 80/443):
-
Endpoints serving CSS, JavaScript, font, and image files referenced by the monitored websites — these vary per monitored site and can be identified from the firewall deny logs
To identify the specific blocked URLs, check the firewall or proxy deny logs for outbound connections from the WPM Player machine IP addresses on port 80 and port 443. The source process is solarwinds.seum.agent.worker.exe.
After whitelisting, monitor WPM transaction step durations. If the blocked endpoints were the cause, step durations should return to the expected 3–5 second range.
If firewall whitelisting is not permitted or the specific endpoints cannot be identified, proceed with Option 1 (electronSwitches) below.
Option 2 – Suppress Chromium-Level CRL/OCSP via Electron Switch in AgentSettings.dat (Recommended)
This method targets the WPM Browser (Chromium/Electron engine) and is the recommended approach for WPM 2022.x and later.
Before you begin: Make a backup of AgentSettings.dat before editing.
-
Stop the SolarWinds WPM Playback Player service via
services.msc. -
Navigate to:
C:\ProgramData\SolarWinds\SEUM\Data\AgentSettings.dat -
Open the file with Notepad (run as Administrator).
-
Locate the following self-closing tag in the
<browserConfiguration>section:<electronSwitches /> -
Replace it with the expanded form below:
<electronSwitches> <switch>--disable-features=CertificateRevocationChecking</switch> </electronSwitches>Important: Do not modify
<electronProcessSwitches />. That is a separate tag controlling process-level Chromium switches and must remain unchanged unless specifically required. -
Save the file.
-
Start the SolarWinds WPM Playback Player service.
Verification: Monitor the proxy or firewall logs after restarting the service. Traffic from solarwinds.seum.agent.worker.exe to CA CRL/OCSP endpoints should cease or be significantly reduced.
Option 3 – Disable Automatic Root Certificate Update via Group Policy (Offline/Restricted Environments Only)
This method suppresses the Windows-level CRL/OCSP calls by disabling the Automatic Root Certificate Update component. It is intended only for environments with no direct internet access.
Warning: SolarWinds strongly recommends keeping Automatic Root Certificate Update enabled. Disabling this feature can cause:
SSL certificate validation failures
Product installation or upgrade failures
Service disruptions caused by missing or outdated root certificates
Only apply this option after reviewing the risk with your security team.
Via Group Policy:
-
Open Group Policy Editor (
gpedit.msc) on the WPM Server. -
Navigate to:
Computer Configuration → Windows Settings → Security Settings → Public Key Policies → Certificate Path Validation Settings -
Enable the policy and check "Turn off Automatic Root Certificates Update".
-
Run
gpupdate /forcefrom an elevated command prompt. -
Restart the WPM Server if prompted.
Via Registry (alternative):
-
Open Registry Editor (
regedit) on the WPM Server. -
Navigate to:
HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\SystemCertificates\AuthRoot -
Create or set the following DWORD value:
Name: DisableRootAutoUpdate Type: DWORD (32-bit) Value: 1 -
Run
gpupdate /forcefrom an elevated command prompt to apply immediately.
Note: If traffic persists after applying the GPO/registry change and running
gpupdate /force, proceed with Option 1 as an additional or alternative measure.
Additional Information
|
Layer |
Source Process |
Suppression Method |
|---|---|---|
|
WPM Browser (Chromium/Electron) |
|
|
|
Windows OS / .NET Runtime |
|
GPO / Registry |