RTSA - on the road to becoming a sysadmin

About: The purpose of this document is to provide a step-by-step guide on how one can replace an ESXi's self-signed certificates, which are used to encrypt the management traffic sent over port 443, usually to a vCenter Server.

Work instructions:

Note that it might be a good idea to migrate all of the VMs (even the ones that are shut down) off of the ESXi host for which the certificate will be replaced, as complications with its re-addition to the inventory might mean that the VMs which it used to hold will have to be manually re-registered in vCenter; on a large host, it might not be worth the risk of leaving them in place. This would be done prior to following the below instructions.

place the host in Maintenance Mode
remove the host from the vCenter inventory altogether
connect to the ESXi host via SSH
switch to the /etc/vmware/ssl directory
move the rui.* files found there to /vmfs/volumes/volume-of-your-choice/esxiBackup
- the name of the folder to which the files are to be moved to can be set at will
- the files (which are the host's certificate and its private key) are to be moved for backup purposes
- the files are: rui.crt, the actual certificate, and rui.key, the corresponding private key
run the /sbin/generate-certificates script (no need for parameters)
- run the tool while the working directory is the /etc/vmware/ssl
- should generate an SHA256 signed certificate in the same directory
- ignore any warnings about a missing/unopenable openssl.cnf file
run ls -l in order to confirm that the rui.* files have been generated
run chmod +t rui.* (not clear why this would be required, since the files belong to root:root, but the sticky bit was set on the original files)

^{________________________________________________________________________________________________________________________}

access the host's out-of-band management interface (be it IMM, RSA, iLO, iDRAC or something else) and
- launch a Remote Control session
- log in to the DCUI and restart the Management Agents (under Troubleshooting Options)

run the below commands via the SSH session:
- /etc/init.d/hostd restart
- /etc/init.d/vpxa restart
wait for the agents to restart, as this might take a minute

^{________________________________________________________________________________________________________________________}

re-add the host to the vCenter inventory (via Add Host)
take it out of Maintenance Mode
access https://host's_management_IP_address:443 and verify that the certificate has changed

Note that this implicit certificate-regenerating approach has likely been made possible due to the ESXi 5.5 Update 3a (the version for which these steps have been put together) having the /sbin/generate-certificates script already configured to output SHA256 certificates. Based on the version of the ESXi installation and/or arguments supplied to the script (normally, no arguments are necessary), your mileage may vary.

References: