Applications Systems

Java poller memory constraints and CPU 100% for Hyper-V polling in VMAN

The Java poller may be constrained by the amount of memory normally allocated to the Java poller when polling very large Hyper-V environments in VMAN Orion, which causes 100% CPU utilization during the polling period and a longer than expected polling time.

First published date

4/17/2019 9:02 PM

Last published date

4/17/2019 9:02 PM

Overview

When polling very large Hyper-V environments in VMAN Orion, the Java poller may be constrained by the amount of memory normally allocated to the Java poller, which causes 100% CPU utilization during the polling period and a longer than expected polling time.

Product section

Virtualization Manager

Cause

Performance bottlenecks are caused by resource limitations. The Java poller has a default limit of 1.5 GB, which may not be enough for large environments.

Resolution

  • One way to fix this issue is to set up an additional polling engine to split the polling load. See the VMAN Deployment Sizing Guide for instructions.

    Since clusters must be monitored entirely by the same poller, only apply additional polling engines for independent host deployments or split a cluster into more than one before setting up an additional polling engine.

  • Another resolution is to increase the amount of memory allocated to the Java poller, which has a default limit of 1.5 GB:
  1. Go to the centralized settings page in the Orion Web Console: [hostname]/Orion/Admin/AdvancedConfiguration/Global.aspx 
  2. Locate the setting "CustomWorkerCommandParams"
  3. Change its value to increase the amount of memory allocated to the Java poller. You can increase this to even higher values if needed. If performance still hasn't improved after increasing the memory, continue to allocate more memory until you see a noticeable improvement. Here are some examples (enter without the quotation marks):
    • "-Xmx3072M"            - allocate 3 GB
    • "-Xmx6144M"    - allocate 6 GB and run the Java poller as a 64 bit process.
    • "-Xmx16384M"  - allocate 16 GB and run the Java poller as a 64 bit process.

    The Java poller may not utilize all of the memory allocated with the -xmx flag. However, increasing the allocation may still result in the Java poller utilizing more memory. See https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/jrdocs/refman/optionX.html (© 2016 Oracle, available at https://docs.oracle.com, obtained on March 29, 2019) for more information about how the -xmx parameter works to increase the Java heap size, and https://docs.oracle.com/cd/E13150_01/jrockit_jvm/jrockit/geninfo/diagnos/garbage_collect.html (© 2016 Oracle, available at https://docs.oracle.com, obtained on March 29, 2019) for more information about general memory management behavior in Java.

  4. Stop the Job Engine service in Orion Service Manager
  5. If running, kill the Java process using Task Manager
  6. Start the Job Engine service again in Orion Service Manager.