cloud-panda-logo-img
Cloud Computing

Updating vRealize Automation 6.2 Certificates

blog-default-background-theme

Overview:

When you replace a certificate for a vRealize Automation component, components that have a dependency on this certificate are affected. You must register the new certificate with these components to ensure certificate trust. You must update all components of the same type in a distributed system. For example, if you update a certificate for one vRealize Appliance in a distributed environment, you must update all instances of vRealize Appliance for that environment. vRealize Automation supports both SHA1 and SHA2 certificates. The self-signed certificates generated by the system use SHA-256 With RSA Encryption. You may need to update vRealize Automation components to use SHA2 certificates due to browser requirements.


Backing up Existing Certificates:

  • Connect to any one of the vRA appliance using WinSCP and copy /etc/apache2/server.pem to local machine.
  • Another way, simply login into vRA portal using IE. Post authentication, click on “Lock” icon in the end of address bar -> View Certificates -> Click on “Details” tab -> ”Copy to file” to export the cert. vRA Load Balancer URL - "https://vra.test.lab.com/vcac"
  • Login into any one of the IaaS portal using IE and simply type "https://iaas1.test.lab.com" to view or export the cert using above method.
  • Another way - RDP into any one of the IaaS server -> Open IIS -> Click on Default Web Site in left pane -> click on “Bindings” in Actions pane on right -> select “https” type -> Edit -> “View” the cert from there and export.
  • Open any one of the vRO configurator portal in IE and export the cert using above method. URL- "https://vro1.test.lab.com:8283/vco-config/auth"
  • It’s also important to back up the certificate store. SSH into vRO appliance by using putty and run this command.
    • vro1.test.lab.com#cp –p /etc/vco/app-server/security/jssecacerts  /etc/vco/app-server/security/jssecacerts.old
  • Keep a snapshot for all the appliances and VMs before this activity and also keep a copy of database(s) backup.


Requesting For New Certificates:

  • We should capture the details of existing Subject Alternate Names (SAN) for all components before generating CSRs. We might be using SHA1 certs earlier but now we can also use SHA2 certs. It’s recommended to use the SSL validity for five to ten years, rather than one year.


Generate CSR For vRA and IaaS Servers:

  • Follow the KB2090090 to generate CSR (Certificate Signing Request)
  • Use OpenSSL installed in any of the server. If it’s not installed then it can be downloaded from this given URL https://www.openssl.org/source/ 
  • The KB insists on using specific version of OpenSSL as it was tested using that version only i.e. 1.0.1i
  • Create a configuration file “vra.cfg” as listed below - for vRA appliance and similarly for IaaS server with its own SANs.
  • We should not include the IPs in SANs as it’s recommended to use FQDN only.
  • Open the command prompt with “Run as admin” and run these commands.
  • The below listed commands may need to replace the correct path as per OpenSSL installed location.
  • We can use other relevant names like "vra.csr" instead of "rui.csr" and that should be fine.
    • C:'OpenSSL-Win64'bin'openssl req -new -nodes -out C:'Certs'vra ui.csr -keyout C:'Certs'vra ui-orig.key -config C:'Certs'vra'vra.cfg
    • C:'OpenSSL-Win64'bin'openssl req -new -nodes -out C:'Certs'iaas ui.csr -keyout C:'Certs'iaas ui-orig.key -config C:'Certs'iaas'iaas.cfg
    • C:'OpenSSL-Win64'bin'openssl rsa -in C:'Certs'vra ui-orig.key -out C:'Certs'vra ui.key
    • C:'OpenSSL-Win64'bin'openssl rsa -in C:'Certs'iaas ui-orig.key -out C:'Certs'iaas ui.key
  • Submit a cert request on internal Active Directory server "https://ad.test.lab.com/certsrv" using the CSRs that we have generated in above mentioned steps.
  • Download the root CA cert chain and extract root CA cert as listed below (refer KB2090090). We can copy it to "C:'CertsRoot64.cer" or any other location. 
  • Or buy a certificate from external authorities based on the CSR generated (if needed)

    //=============vra.cfg=============//
    [ req ]
    default_bits = 2048
    default_keyfile = rui.key
    distinguished_name = req_distinguished_name
    encrypt_key = no
    prompt = no
    string_mask = nombstr
    req_extensions = v3_req
    [ v3_req ]
    basicConstraints = CA:FALSE
    keyUsage = digitalSignature, keyEncipherment, dataEncipherment,  onrepudiation
    extendedKeyUsage = serverAuth, clientAuth
    subjectAltName = DNS:vra.test.lab.com, DNS:vra1.test.lab.com, DNS:vra2.test.lab.com, DNS:iaas1.test.lab.com, DNS:iaas2.test.lab.com, DNS:dem1.test.lab.com, DNS:dem2.test.lab.com, DNS:agent1.test.lab.com, DNS:agent2.test.lab.com
    [ req_distinguished_name ]
    countryName = IN
    stateOrProvinceName = KA
    localityName = BLR
    0.organizationName = Test Lab
    organizationalUnitName = Home
    commonName = vra.test.lab.com
    //=============vra.cfg=============//

Sign The Certificates:

  • Navigate to your CA web enrollment portal at "https://ad.test.lab.com/certsrv" and log in with an appropriate account such as domain admin.
  • Submit certificate requests for all the crt certificates. Download the base-64 cert named "rui.crt" using the Web Server certificate template and save each to the appropriate directory.
  • Download the root CA Certificate Chain Base64 root certificate to "C:'Certs" and rename it to cachain.p7b
  • Open the p7b file and find the root cert, export it as base64 and save it to "C:'CertsCertsRoot64.cer"
  • Run these below listed commands to generate the pfx and create the PEM files for vRA. It prompts for password but we can leave blank and just hit enter to proceed.
  • For IaaS, we don’t need to perform these steps and simply use .crt file which we already have.
  • If it’s in .cer format, double click on it and export it as base-64 encoded format and change extension from .cer to .crt
  • C:'OpenSSL-Win64'bin'openssl pkcs12 -export -in C:'Certs'vra ui.crt -inkey C:'Certs'vra ui.key -certfile C:'Certs'Root64.cer -name “vra” -out C:'Certs'vra ui.pfx
  • C:'OpenSSL-Win64'bin'openssl pkcs12 -in C:'Certs'vra ui.pfx -inkey C:'Certs'vra ui.key -out C:'Certs'vra'vra.pem -nodes
  • Now we have cert in .crt format for IaaS and in .pem format for vRA. So let’s get one for vRO too.


Generate a CSR for vRO Appliances:

  • SSH into the vRO appliance(s) by using putty and stop orchestrator server service.
  • Generate a cert request using this command.
    • keytool -certreq -alias dunes -keypass "dunesdunes" -keystore "/etc/vco/app server/security/jssecacerts" -file "/tmp/myCertRequest.csr" -storepass "dunesdunes"  
  • Download this “myCertRequest.csr” using WinSCP tool.
  • Generate a cert by using CSR for vRO.
  • Or buy a certificate from external authorities based on the CSR generated (if needed)
  • For more details please follow the KB 2007032


Applying The Certs:

  • Take snapshot of appliances or VMs. It’s recommended to have an image level backup of appliances or VMs and also the databases before applying the certs.


Applying SSL in vRA Appliances: (Repeat this process for all of the vRA appliances)

  • Login to vRA VAMI console "https://vra1.test.lab.com:5480" and go to vRA Settings -> Host Settings -> SSL Configuration -> Import.
  • Open "rui.key" in notepad, which we have generated in above mentioned steps.
  • Copy and paste into “RSA Private Key” box in VAMI. Make sure there are no spaces before or after the key content that you copied.
  • Open vRA cert (.pem file) that we generated earlier, in a notepad.
  • Copy and paste content in “Certificate Chain” box in VAMI. Make sure there are no spaces before or after the key content that you copied.
  • Leave third box “Passphrase” blank as we didn’t supply any password while creating cert file.
  • Click on “Save Settings” and it may take few minute to update.
  • At last it will show a message like “cert update successful”.
  • Go to SSO tab and type administrator@vsphere.local in SSO Admin User section and appropriate password in SSO Admin Password section.
  • It will update SSO registration for vRA but this process may take several minutes.


Updating IaaS Model Manager Server: (Repeat this process for both of the Model Manager)

  • Log in to the IaaS Model Manager server by using vRealize service account and open the web.config file by using a text editor.
    • C:'Program Files'vCAC'Server'Model Manager Web'web.config
  • Search for the string starting with and make a note of the exact SQL server name and database name.  It needs to be used in below command.
  • Open the command prompt with "Run as Admin" and navigate to the Cafe directory on the Model Manager Data server and run these commands.
  • Make sure you have logged in with vRealize service account that has access to IaaS DB, else there is more possibility you will get some error.  Now we can run the below command.
    • C:'Program Files'vCACServer'Model Manager Data'Cafe'Vcac-Config.exe UpdateServerCertificates -d -s -v
  • Restart IIS services by using the  command “iisreset”


Applying SSL in IaaS Web Servers: (Repeat this process for all of the IaaS web servers)

  • RDP into IaaS web server -> Open IIS -> Click on web server host name in left pane -> Double-click Server Certificates from Features View.
  • Click Import in the Actions pane -> Click browse and import (.crt file) which we already have.
  • Click on the imported certificate and select View. Verify that the certificate and its chain are trusted. If the certificate is un-trusted then this CA root certificate is not trusted.
  • Select the Default web site in left pane -> Click o Bindings in the Action pane -> Click Edit on the https (443) in the Site Bindings dialog box -> Change the SSL certificate to newly imported one.
  • Restart IIS services by using the command “iisreset”
  • Open command prompt with “Run as Admin” and navigate to the following directory and run this command, which will register IaaS component in vRA.
    • C:'Program Files'vCAC'Server'Model Manager Data'Cafe'vcac-Config.exe RegisterEndpoint -EndpointAddress -v
  • SSH into vRA appliance and restart vRA service using the command “service vcac-server restart”


Applying SSL in vRO Appliances: (Repeat this process for all of the vRO appliances)

  • Connect to vRO using WinSCP and upload the cert, let’s say we put at "/tmp/MyCert.crt"
  • Use Putty to SSH into vRO server and run the below listed command.
    • # keytool -importcert -alias dunes -keypass "dunesdunes" -file "/tmp/MyCert.crt" -keystore "/etc/vco/app-server/security/jssecacerts" -storepass "dunesdunes"
  • Restart vRO services i.e. /etc/init.d/vco-server and /etc/init.d/vco-configurator restart
  • Open both vRO configuration portal "https://vro1.test.lab.com:8283/vco-config/auth" and "https://vro2.test.lab.com:8283/vco-config/auth" verifies the cert is updated.
  • For more details please follow the KB 2007032


Applying SSL in vRA Guest Agents:

Linux Guest Agent: (Repeat this process for all of the Linux templates)

  • Convert all Linux templates to VM in vCenter Server, one by one.
  • Power on the VM and SSH into the VM by using putty and run these commands. This command will use IaaS active model manager server to connect i.e. "iaas1.test.lab.com"
    • cd /usr/share/gugent
    • mv cert.pem cert.pem.bak 
    • echo | openssl s_client -connect  | sed -ne ‘/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p’ > cert.pem
    • #./installgugent.sh ssl
  • Now, download "cert.pem" using WinSCP from this VM to local machine, we will use it for windows templates.
  • Shutdown the VM and convert it back to template. Don’t change the template name.
  • For more detail, please follow the VMware Pub


Windows Guest Agent: (Repeat this process for all of the Windows templates)

  • Convert the templates to VM -> power it on -> RDP into it.
  • Copy "cert.pem" file that we got in C:'VRMGuestAgent. (Rename existing "ert.pem" to "cert.pem.bak")
  • Go to windows services and stop the ‘VCACGuestAgentService’ service.
  • Open Windows PowerShell with “Run as Admin” and run below command to uninstall this service.
    • C:'VRMGuestAgent'WinService.exe –u
  • To reinstall the service, run this command. This command will use IaaS active model manager server to connect i.e. "iaas1.test.lab.com"
    • C:'VRMGuestAgent'WinService.exe -i –h -p ssl
  • Start ‘VCACGuestAgentService’ service.
  • Shutdown the VM and convert it back to template. Don’t change the template name.
  • Now we are done with new certificate update. So we are good to perform a reboot of all servers and appliances.
  • For more detail, please follow the VMware KB 2068820


Troubleshooting:

  • Follow this VMware KB 2110207 to re-establish the trust between all components within vRealize Automation 6.2.x environment and at last restart IIS service by executing the command “iisreset” on all web servers.

Additional Information:

Tags:

Write Review

  1. Your email address and mobile number will not be published. Required fields are marked *