Virtualization

VMware vRealize Operations 6.x Architecture Details

18 Jan 2018 by Uma Panda

Product Overview:
VMware vRealize Operations, sometimes referred to as vROps and formerly called vCenter Operations Manager, is a software product that provides operations management across physical, virtual and cloud environments, no matter if those environments are based on vSphere, Hyper-V or Amazon Web Services.
Run production operations hands-off and hassle-free with VMware vRealize Operations, delivering continuous performance optimization based on intent, efficient capacity management, proactive planning and intelligent remediation. Optimize, plan and scale SDDC and cloud deployments, from apps to infrastructure, with a unified management platform.

Business Requirement:

Capacity planning and reporting for existing VMware environment.
Proactive workload monitoring for existing VMware environment.

VMware vROPS Architecture Design:

Collector:

The collector requests real-time stats that are pulled directly from the vCenter hosts. vRealize Operations Manager communicates to the hosts through vCenter. All communications between vRealize Operations Manager and vCenter Server take place over an SSL connection and are authenticated by certificates.
vRealize Operations Manager gathers data relating to the physical vSphere environment through vCenter Server. It obtains multiple averaged samples for most statistics every five minutes and computes derived metrics from the raw vSphere metrics (for example, demand, workload, and outstanding I/Os).
The collector talks over the message queue and pushes data to the analytics server. The collector can be on the vRealize Operations Manager server or distributed remotely when using the Enterprise edition for geographical network segmentation and balancing workloads, and it can house one or more adapters.

ActiveMQ:

ActiveMQ is a message broker that is responsible for transferring messages across the different components of the solution and uses KahaDB to provide message persistence. Messages are stored in a journal so that they can rapidly be written to disk. The KahaDB enables the broker to restart quickly. Message references are stored in a B-tree index, which can be quickly updated at run time. JMS transactions are fully supported.
KahaDB employs various strategies to enable recovery after a disorderly shutdown of the broker.

Analytics:

The Analytics element has dynamic thresholds that adjust automatically based on a baseline (termed normal) and trended behavior over the past hours, days, weeks, and months. The Analytics element detects metric-level abnormalities for use in alerts and system health assessments.
Thresholds are calculated by looking at data over a historical period of time and identifying upper and lower values. The difference between these two values defines normal behavior.
- Dynamic thresholds are calculated overnight.
- Metrics are stored in the FSDB.
- Metric relationships are stored and retrieved from the relational database (PostgreSQL) on the vROPS appliance.

FSDB:

FSDB is an optimized time series data storage mechanism composed of flat files on the file system. It stores only time series data (statistics). Relational databases are too slow and have too much overhead to respond to the incoming statistics.

File layout:

One file per object per statistic.
Directories contain a monthís worth of statistics files per object.
Retention can be configured on a monthly basis. Purging old data consists of deleting directories.

File format:

Efficient storage size: 16 bytes/stat instance.
On-disk format identical to in memory format.
Optimized for read performance. No data transformation needed.
Stores statistics at the same resolution at which they were collected (5 minute resolution for most statistics).

VMware vCenter CapacityIQ:

The VMware vCenter CapacityIQô server engine is now an integrated part of vRealize Operations Manager.
The vCenter CapacityIQ Server resides in the vROPS appliance and it has its own PostgreSQL database instance to store capacity related data.
An adapter (CIQ Adapter) collects capacity related data and converts it into a format understood by the vRealize Operations Manager components. This adapter resides inside the collector and pushes data through the ActiveMQ message broker.

Physical Infrastructure Architecture:

vRealize Operations Manager is supplied as a black box appliance from VMware and deployed as a VM.

vRealize Operations Manager UIs are web-based tools that can be connected through the following supported web browsers:

Google Chrome 24 and 25 or higher
Internet Explorer for Windows 8.0 and 9.0 or higher
Mozilla Firefox 18 and higher

Note: There are some limitations with the Dashboards feature in the custom interface when Internet Explorer 8 is used.

Compute Sizing Reference:

Environment: Development:

HA Required: Yes
Appliance Size: Small
Memory Requirement: 16 GB
vCPU Requirement: 4
Storage Requirement: Default

Environment: Pre-Production:

HA Required: Yes
Appliance Size: Medium
Memory Requirement: 32 GB
vCPU Requirement: 8
Storage Requirement: Default

Environment: Production:

HA Required: Yes
Appliance Size: Large
Memory Requirement: 62 GB
vCPU Requirement: 16
Storage Requirement: Default

One year data retention, 3800 objects - 1 TB per node
One year data retention,1700 objects - 500 GB per node
One year data retention, 600 objects - 260 GB per node

vRealize Operations Manager System Dependencies:

Component: DRS-enabled cluster

Impact of Loss on vRealize Operations Manager: vRealize Operations Manager VM can be deployed to a non-DRS cluster if the vRealize Operations Manager VM is deployed to a standalone VMware ESXiô host (not a cluster) with static IP addresses. Move standalone hosts to a cluster if possible. The VM will not work, but vRealize Operations Manager will function unless the IP addresses are changed.

Component: Host systems where the VM is installed

Impact of Loss on vRealize Operations Manager: When a host running vRealize Operations Manager is lost, vRealize Operations Manager stops functioning. HA is supported for VM. This means that when this VM fails on Host1, it will be restarted on Host2. When HA failover happens (for example, if the VM is moved to another host), the VM communication might break. If this occurs, power off and restart the VM to address the issue.

Component: Virtual Networking

Impact of Loss on vRealize Operations Manager: If virtual networking is not working, it might be that vRealize Operations Manager cannot be contacted. Try the following to address the issue:
- Unregister the ESXi host where the VM is deployed and re-register the host to the same or a different vCenter instance.
- Deploy VM to a cluster and move the ESXi host where VM resides out of the cluster.
- Deploy VM to a standalone ESXi host and then move the host into a cluster that does not have DRS enabled.
To repair networking on the VM (if the root cause is not virtual networking):
- DHCP - Power cycle the VM.
- Static - Power off the VM and edit VM settings.
When VM is not working, the ability to detect IP address changes automatically is lost, and user intervention is needed to re-establish communication between the VM. Log in to the console of the VM as admin and run the command:
- vcops-admin repair --ipaddress
Any further change to the IP address will require that you run the repair command.

Component: Virtual Shared Storage

Impact of Loss on vRealize Operations Manager: If shared storage is irretrievably lost, vRealize Operations Manager might require a reinstall. Historical data might be lost.
- vRealize Operations Manager might simply have run out of disk space. An administrative alert is generated when less than 10% of the disk is free (this also appears in the vRealize Operations Manager vSphere UI Alert Window).
- If this occurs, power off the VM, add a new .vmdk file, and restart the VM.

Component: VCenter Server

Impact of Loss on vRealize Operations Manager: If vCenter cannot be reached through the API, vRealize Operations Manager stops collection, and the health of the vCenter objects within vRealize Operations Manager is listed as unknown. All historical data remains in the system. When data collection is able to resume, the existing objects automatically display the new data.

Component: Active Directory Server/Cluster

Impact of Loss on vRealize Operations Manager: If the LDAP connection stops working or is blocked, or the LDAP server or cluster is down, users with LDAP logins are not able to access the system. In this scenario, a local user is created with the relevant permissions for each group.

Component: Exchange Server/Cluster

Impact of Loss on vRealize Operations Manager: If the SMTP connection stops working or is blocked, or the Exchange server or cluster is down, vRealize Operations Manager cannot send alert emails. Alerts can be viewed on the Alert page.

vRealize Operations Manager Storage Requirements:

Detail the server storage by server type, including internal storage array, externally connected storage, and SAN connectivity.

Component Name: User Interface
Type: Resilient shared storage is highly recommended to support the combined IOPS of the vROPS VM
Size: 400 GB

Note: Deploy VM storage as Thick Provisioned Eager Zeroed.

Component Name: Analytics
Type: Resilient shared storage is highly recommended to support the combined IOPS of the vROPS VM.
Size: 3.2 TB (max)

Note: Deploy VM storage as Thick Provisioned Eager Zeroed.

vRealize Operations Manager VM Backup:

The vRealize Operations Manager VM can be backed up as any virtual machine. If the VM does not function when restored, the repair function is required.

Data Purging and Archiving:

By default, historical data is retained for a six monthsí period. This period can be altered by adjusting the configuration setting in the vRealize Operations Manager vSphere user interface.
Data is purged on a rolling basis after the data retention period. VMware best practice specifies data purging after six months of data collection.

Security and Authentication:

The Linux root and application admin local accounts are set during installation and are managed using the Admin user interface. Both passwords can be reset using VM Console rights to the vROPS VM. All passwords are maintained in an encrypted form in the PostgreSQL Metadata DB.
User accounts are managed through the custom UI, where groups are populated with locally created or LDAP users. Permissions are set at the group level and cascaded to members of the group.
LDAP users can be imported from one or more groups into vRealize Operations Manager and can be assigned to vRealize Operations Manager groups in one operation. The process requires connection information for the LDAP server, including host, port, user name, and password to connect to the database.
When LDAP users are imported into vRealize Operations Manager, only the user name is imported ñ the password is not. When an LDAP user logs in to vRealize Operations Manager, the LDAP database is queried to validate the password. An LDAP user cannot change a password in vRealize Operations Manager. All password-related fields in the Edit User window are disabled.
With manual import or auto-synchronization, you can use SSL to communicate securely with the LDAP server. To do this, a security certificate must be imported on the vRealize Operations Manager Server.

Encryption:

SSL is used when accessing the application on all Web user interfaces. By default, vRealize Operations Manager installs a self-signed certificate. You can install your own certificate that satisfies the following prerequisites:
The certificate files contain both a valid private key and a valid certificate chain.
The private key is generated by the RSA or the DSA algorithm.
The private key is not encrypted by a passphrase.
If the certificate is signed by a chain of other certificates, all other certificates must be included in the certificate file that you plan to import.
All the certificates and the private key that are included in the certificate file must be PEM-encoded. vRealize Operations Manager does not support DER-encoded certificates and private keys.
All the certificates and the private key that are included in the certificate file must be in the PEM format. vRealize Operations Manager does not support certificates in PFX, PKCS12, PKCS7, or other formats.

VMware best practice stipulates using a signed certificate for the encryption process on all Web UIs. Signed certificates using customer's internal certificate authority can also be used for vRealize Operations Manager

Active Directory/LDAP:

Customer might decide to import users from LDAP (Active Directory) groups. If this is the case, security groups containing the users to import must be created in Active Directory. VMware recommends that the AD groups align to these general roles:
Administrator - Ability to alter admin and security settings and access the Admin UI (for patching, external adapters, vCenter registration, and so on).
Operator - Same access as the Administrator level users, except that this role does not have the rights to administer security or access the Admin UI.
User - Same access as the Operator, except that this role does not have edit or delete rights.

VMware highly recommends that most users be granted only User access and that user promotion to more powerful groups is done only with change control and careful consideration.

Note: Stage 1 will provide no access to the vRealize Operations Manager UI for end users. Access will be provided through the Active Directory domain through the use of Active Directory groups.

Component Connectivity Matrix:

Source Server: End User Workstation
Destination Server: vRealize Operations Manager Servers
Port Number: 22
Protocol: TCP
Service Description: Enables SSH access to the vRealize Operations Manager VM.

Source Server: End User Web Browser
Destination Server: vRealize Operations Manager Servers
Port Number: 80
Protocol: TCP
Service Description: Redirects to port 443

Source Server: End User Web Browser
Destination Server: vRealize Operations Manager Servers
Port Number: 443
Protocol: TCP
Service Description: Used to access the vRealize Operations Manager Admin portal and the vRealize Operations Manager application.

Source Server: UI VM
Destination Server: Analytics VM
Port Number: 1194, 22
Protocol: TCP
Service Description: Sets the tunnel between UI VM and Analytics VM.

Source Server: UI VM
Destination Server: vCenter Server
Port Number: 443
Protocol: TCP
Service Description: Used for vCenter registration.

Source Server: Analytics VM
Destination Server: vCenter VI ñ SDK
Port Number: 443
Protocol: TCP
Service Description: Used by the Analytics VM to pull counters from the vCenter Server.

Source Server: Analytics VM
Destination Server: vCloud Director REST API
Port Number: 443
Protocol: TCP
Service Description: Used by the Analytics VM to pull counters from the vCloud Director Cell using the REST API.

Source Server: vCenter Server
Destination Server: UI VM
Port Number: 443, 22
Protocol: TCP
Service Description: Used for Web UI and console access.