Observability

Job Engine v3 Error: ResourceExhausted when GetJobResult exceeds gRPC message size limit in the SolarWinds Platform

This article provides information about a Job Engine v3 error that occurs when a worker returns a job-result response larger than the configured gRPC message-size limit. The failure can prevent Job Engine v3 from processing the job successfully and may be followed by repeated processing attempts, worker instability, or increased memory usage.

First published date

9/10/2026 8:41 PM

Last published date

9/10/2026 10:42 PM

Overview

Job Engine v3 uses gRPC to retrieve job results from worker processes. When the returned result exceeds the configured incoming message-size limit, the request fails with a ResourceExhausted error.

A similar message may appear in the Job Engine or worker logs:

[YYYY-MM-DD HH:MM:SS,mmm] ERROR SolarWinds.JobEngine.Worker.GrpcClient.WorkerGrpcClientExceptionHandler -
The Worker gRPC client encountered an 'RpcException' exception -
'Status(StatusCode="ResourceExhausted", Detail="Received message exceeds the maximum configured message size.")'
Grpc.Core.RpcException: Status(StatusCode="ResourceExhausted",
Detail="Received message exceeds the maximum configured message size.")
   at ...WorkerClient.GetJobResult(...)
   at ...WorkerGrpcClient.GetJobResult(...)
[YYYY-MM-DD HH:MM:SS,mmm] ERROR SolarWinds.JobEngine.Engine.WorkerMonitor -
Attempt to process jobs on worker <worker-id> failed.

Observable platform behavior

The following behavior may be noticed by an administrator or monitoring operator when the affected job or plugin cannot complete successfully:

  • Monitoring data for the affected feature or plugin may stop updating or become delayed.
  • Polling or collection jobs may fail, remain incomplete, or repeatedly retry.
  • Status information, metrics, or alerts that depend on the affected job may be delayed or unavailable.
  • Platform, Collector, or Job Engine responsiveness may degrade if resource usage continues to increase.

These are possible external symptoms and may not occur in every environment. They are not sufficient by themselves to confirm this issue.

Internal indicators

The following internal symptoms may also occur:

  • Job Engine v3 fails to process one or more worker jobs.
  • The same job or worker is processed repeatedly.
  • Worker processes are repeatedly created or remain active after failed processing.
  • Job Engine or Collector memory usage increases over time.
  • Job Engine v3 logs contain repeated GetJobResult and ResourceExhausted messages.

The exact plugin or job that produces the oversized response may vary by environment. Confirm the issue by matching the log error and GetJobResult call rather than by memory symptoms alone.

Product section

Hybrid Cloud Observability

Cause

The error occurs when the response returned by the worker through the unary GetJobResult gRPC operation exceeds the configured incoming gRPC message-size limit.

The default gRPC incoming message-size limit is commonly 4 MB unless it has been changed by the application. The affected response may be generated by a job or plugin that returns a large result. Repeated failures may cause additional worker processing attempts and can contribute to worker churn or increased memory usage.

Resolution

As of September 10, 2026, a permanent fix implementation date has not been defined. Monitor this article and the SolarWinds release notes for product fixes and enhancements.

Temporary workaround for environments using the SD-WAN poller

Use this workaround only when all of the following conditions apply:

  • The issue is associated with the SD-WAN poller or its Job Engine v3 workload.

  • The user is aware that SD-WAN polling will be moved from Job Engine v3 to Job Engine v2.

  • A service-restart maintenance window is available.

This workaround does not increase the gRPC message-size limit. It removes the SD-WAN poller from the Job Engine v3 collector-plugin list so that SD-WAN polling can fall back to Job Engine v2.

Before changing the setting

  1. Confirm the affected polling engine and the active HA member.

  2. If HA pools are in use, disable all HA pools before changing the setting or restarting services.

  3. Record the current memory usage and collect relevant diagnostics if the issue is actively occurring.

  4. Confirm that the customer accepts the temporary change in SD-WAN polling behavior.

Change the Job Engine v3 plugin list

  1. Log in to the SolarWinds Platform Web Console.

  2. Open the Advanced Configuration page:

    https://<solarwinds_server>/Orion/admin/advancedconfiguration/global.aspx

  3. Search for SolarWinds.JobEngine.Service.v3.Settings.

  4. Locate JobEngineV3PluginList and select the pencil/edit icon. Do not edit JobEngineV3BusinessLayerPluginList; the SD-WAN poller is a Collector job.

  5. Copy the complete current value to a text editor and retain it as the rollback backup.

  6. Locate this exact entry:

     SolarWinds.Orion.SdWan.Pollers
  7. Remove the entry and one adjacent semicolon separator:

    • If the entry is followed by another plugin, remove SolarWinds.Orion.SdWan.Pollers;.

    • If the entry is the final plugin, remove the semicolon immediately before the entry.

  8. Confirm that the remaining plugin names are still separated by a single semicolon and that no neighboring plugin was removed or changed.

  9. Save the setting.

Restart and verify the services

  1. When the Web Console asks whether to restart services, select Yes.

  2. Confirm that these services restart successfully:

    • SolarWinds Collector Service

    • SolarWinds Job Engine v3

    • SolarWinds Job Engine v2

  3. If no restart prompt appears, manually restart the three services during the maintenance window.

  4. Reopen JobEngineV3PluginList and verify that SolarWinds.Orion.SdWan.Pollers is no longer present.

  5. If HA pools were disabled, wait approximately 1–2 minutes after the service restart, then re-enable them.

    Workaround limitations

    • This workaround moves SD-WAN polling from Job Engine v3 to Job Engine v2; it does not fix the oversized gRPC response or increase the message-size limit.

    • SD-WAN monitoring coverage, performance, or behavior may differ while the workaround is active.

    • This is a temporary mitigation, not a permanent product fix.

    • Do not manually change gRPC message-size settings unless SolarWinds Support provides a supported configuration and value.