Submit a ticketCall us

Get a crash course on Network Monitoring delivered right to your inbox
This free 7-day email course provides a primer to the philosophy, theory, and fundamental concepts involved in IT monitoring. Lessons will explain not only how to perform various monitoring tasks, but why and when you should use them. Sign up now.

Home > Success Center > Virtualization Manager (VMAN) > Virtualization Manager data collection and calculation of rollup information from VMware

Virtualization Manager data collection and calculation of rollup information from VMware

Table of contents
No headers

Updated March 11th, 2016

 

Data Collection

Data is collected in intervals based on the collection schedules. There are two primary types of data collections:

  • Configuration
    • Configuration data include properties such as CPU speed, IOPS, and host name. By default, configuration data are collected in 12 hour intervals.
  • Performance
    • Performance data include properties such as total latency, CPU idle, and throughput. By default, performance data are collected in 10 minute intervals. Performance data are collected for new data sources immediately, but are not displayed until configuration data are collected.

 

If you have calculations based on both configuration and performance data, the configuration data will be older than the performance data and may affect your expected calculations. For example, if you just added a cluster, it may take up to 12 hours (or your configuration collection interval) for the information to appear. You can immediately collect configuration information by clicking the Run Now option in the Collection Schedules section of Virtualization Manager. To see how specific properties are collected, see Virtualization Manager Properties for a list of Virtualization Manager properties. You can change the frequency of individual collections by modifying the collection schedule in the Setup tab in the Collection Schedules section.

 

Data Rollup

Raw performance data is rolled up over time, to provide hourly, daily, weekly, monthly and quarterly averages, maxima, and other statistics. The rollup periods are based on the local server time and do not take business hours into account. The raw and hourly performance data consume large amounts of storage capacity and are discarded after a configurable amount of time. Higher level rollups are retained indefinitely for record keeping purposes, cost accounting, and to act as a data warehouse. This information provides administrators with long term trends in resource consumption.

 

The following is an example of how peak values are calculated:

  • Latest Value (peak) - The highest values of each sample collected from VMware. VMware collects raw data in 20 second intervals.
  • Hourly Rollup (peak) - The highest values from the data collected during the last hour.
  • Daily Rollup (peak) - The highest values from the hourly rollups during the last 24 hour period.
  • Weekly Rollup (peak) - The highest values from the daily rollups during the last seven days.
  • Monthly Rollup (peak) - The highest values from the daily rollups during the last calendar month.
  • Quarterly Rollup (peak) - The highest values from the monthly rollups during the last quarter.

 

Although average and peak are the two most common metrics, other metrics collected by the performance job use different rollups. For example, when the Powered On status is rolled up, Virtualization Manager only retains whether the system was mostly on or off during the rollup period.

 

Data Retention

Raw performance data is saved for 14 days and hourly rollups are saved for 90 days by default. You can change how long Virtualization Manager retains this data by going to Setup > Advanced Setup > System Properties. Increasing the length of time that data is retained may slow down the application or the database if you do not provide adequate storage resources. However, any increase to the raw performance data and hourly rollup retention will significantly impact your database size.

 

This can be configured in Setup > Advanced Setup > System Properties.

The following table provides data retention information per rollup period.

Period

Default Retention Period

Latest values (raw data)

14 days (configurable)

Hourly rollup

90 days (configurable)

Daily rollup

Indefinite

Weekly rollup

Indefinite

Monthly rollup

Indefinite

Yearly rollup

Indefinite

 

Aggregation

Aggregation combines the performance data collected during the same time across your virtual environment. To calculate overall performance and load statistics for datastores. For example, Virtualization Manager collects partial data from each individual host and VM which accesses the datastore and then aggregates that data to get a complete picture of the load and performance of that datastore. Raw data is aggregated by the collection job in real time. Once there are aggregate raw data points, they are stored, processed, indexed, and rolled up by Virtualization Manager.

 

Infrastructure Aging

If no data is collected against a piece of your virtual infrastructure after 24 hours, that piece, or CI, is considered "stale," and data relating to it are grayed out. After 48 hours with no data collected, Virtualization Manager considers that CI to be removed from your infrastructure and stops displaying information about it. The data is not deleted from the database, and if the CI reappears in your virtual environment, Virtualization Manager will recognize it and link the new information to the information already gathered in the database. 

Last modified
10:49, 6 Feb 2017

Tags

Classifications

Public