Upgrade vom vSphere 5.0 to 5.5 – 512 bit certificate issue 10


This week I upgraded the vSphere 5.0 environment to 5.5 Update 1, which is usually not a big deal. I really can’t complain about the upgrade process itself, it’s more the result which I didn’t expect.

Once all components were up to date, I launched the vSphere Web Client which was working fine but at the top I saw the following error message:

Failed to verify the SSL certificate for one or more vCenter Server Systems: https://vCenter_FQDN:443/sdk

I was able to login via the C# client which showed the vCenter as usual, so it seemed to be a problem between the vCenter server and the Web Client.

After I spoke to the VMware Support it turned out that the vCenter Server doesn’t support the old 512 bit certificates. This problem is mentioned in the release notes:

After you upgrade vCenter Server 4.x to 5.5 Update 1, vCenter Server is inaccessible through vSphere Web Client*
When you upgrade from vCenter Server 4.x to 5.5 Update 1, the vCenter Server is not visible through vSphere Web Client. The issue occurs as vCenter Server 4.x supports SSL certificates with 512 bits but vCenter Server 5.5 supports only SSL certificates with greater than or equal to 1024 bits.

Workaround: To resolve this issue, replace the vCenter Server certificates with greater than or equal to 1024 bits

I wasn’t aware of this issue and even if so I would’t have recognize it, since I upgraded a 5.0 environment. The actual problem was that the environment has been upgraded from 4.1 to 5.0 before which is the cause why there still was the 512 bit certificate in use. To see if you are affected by this problem simply open the the rui.crt file (C:\ProgramData\VMware\VMware VirtualCenter\SSL) before upgrading vSphere:

512BitCert

The funny thing is that none of the installation wizards recognizes that the certificate is unsupported so the upgrade went through without any errors.

However there is a way to fix it which I outline below:
1. Backup your vCenter Server / database / old SSL certificates

! This process will cause some downtime to certain vCenter services !

2. Generate new Certificates (KB2037432)

a) Create a temporary directory like c:\certs

b) Create a file called vcenter.cfg with the following content:

[ req ]
default_bits = 2048
default_keyfile = rui.key
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req

 

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = DNS: HOSTNAME, IP: xxx.xxx.xxx.xxx, DNS: FQDN

 

[ req_distinguished_name ]
countryName = DE
stateOrProvinceName = YOURPROVINCE
localityName = YOURCITY
0.organizationName = YOURORGANIZATION
organizationalUnitName = vCenterServer
commonName = FQDN

 

3. Start to create new certificates

The openssl utility can be found in C:\Program Files\VMware\Infrastructure\Inventory Service\bin\openssl.exe

openssl req -new -nodes -out c:\certs\rui.csr -keyout c:\certs\rui-orig.key -config c:\certs\vCenter.cfg

openssl rsa -in c:\certs\rui-orig.key -out c:\certs\rui.key

openssl req -text -noout -in c:\certs\rui.csr

openssl x509 -req -days 3650 -in c:\certs\rui.csr -signkey c:\certs\rui.key -out c:\certs\rui.crt -extensions v3_req -extfile c:\certs\vCenter.cfg

//Update: I just saw that it might be better to add the -sha256 option to use a more secure algorithm instead of the default SHA1

openssl.exe pkcs12 -export -in c:\certs\rui.crt -inkey c:\certs\rui.key -name rui -passout pass:testpassword -out c:\certs\rui.pfx

openssl pkcs12 -in c:\certs\rui.pfx -info

openssl x509 -text -noout -in rui.crt

4. Create a file called chain.pem in C:\certs and then open the rui.crt file with an editor and copy the content into the chain.pem file and finally save it.

5. Use the SSL Certificate Automation Tool 5.5 (for vSphere 5.5 only!) to plan the actions and the order in which they should be performed:PlanSSLUpdateSetps

Take a screenshot of the list or write it down, you will need it in a second.

6. Now replace the certificates just for the vCenter server. The tool will ask you for the certificate chain which is located in C:\certs\chain.pem and the private key c:\certs\rui.key and some credentials:UpdatevCenterCerts

7. Once this is done, you will need to re-establish the trusts between vCenter server and it’s components like the Inventory Service, SSO and so on. When performing these steps, follow the order depicted on the list you have written down. The following screenshot shows the process generic, because it’s pretty similar with all other components:TrustUpdateManager

That’s it. After that I went to the vSphere Web Client and logged in as usual. No errors left, the vCenter server connected to the Web Client correctly and I was able to manage it. So overall this certificate replacement was easier than expected and it fixed the issue as required. I hope this helps!

Print Friendly, PDF & Email

Related Post

It's only fair to share...Tweet about this on TwitterShare on LinkedInEmail this to someone

Leave a comment

Your email address will not be published. Required fields are marked *

10 thoughts on “Upgrade vom vSphere 5.0 to 5.5 – 512 bit certificate issue

  • Jake

    Hi there!

    Nice blog but I am having issues with the last command. Formatting seems to be wrong but I cant figure out where.

    openssl pkcs12 -in c:\certs\rui.pfx -info openssl x509 -text -noout -in rui.crt

    Thanks!

    • Patrick Post author

      Hi

      thanks! I found the problem, this line are actually two commands:

      openssl pkcs12 -in c:\certs\rui.pfx -info

      openssl x509 -text -noout -in rui.crt

  • JD

    Did this and it worked, however it now gives me errors about authenticating with the VMware Inventory service – I’ve renewed the trusts between the webclient and inventory service but still have the same issue. Call has been logged with VMware…

  • Chris

    First of all, THANK YOU for this. Worked as advertised.

    @JD: I also had the inventory service authentication issue. I was able to authenticate using “administrator@vsphere.local” rather than just using “administrator” as the username.

  • francois

    when I enter the last command

    openssl x509 -text -noout -in rui.crt

    I get error

    Error opening Certificate rui.crt
    4112:error:02001002:system library:fopen:No such file or directory:.\crypto\bio\
    bss_file.c:367:fopen(‘rui.crt’,’rb’)
    4112:error:20074002:BIO routines:FILE_CTRL:system lib:.\crypto\bio\bss_file.c:36
    9:
    unable to load certificate

    tried to redo from scratch got the same error can you help me?

    thank you