vRA 8.x Infoblox Plugin throws a certificate chain error during validation

 

vRA 8.x Infoblox Plugin throws a certificate error during validation

When initially setting up infoblox integration or after replacing infoblox certificate for existing integration in vRA 8.x you may see the following error:


Unable to validate the provided access credentials: Failed to validate credentials. AdapterReference: http://provisioningservice.prelude.svc.cluster.local:8282/provisioning/adapter/ipam/endpointconfig. Error: Execution of action Infoblox_ValidateEndpoint failed on provider side: Infoblox HTTP request failed with: HTTPSConnectionPool(host=’pb0infblx01.flexlab.local’, port=443): Max retries exceeded with url: /wapi/v2.7/networkview?_return_fields=name (Caused by SSLError(SSLError(“bad handshake: Error([(‘SSL routines’, ‘tls_process_server_certificate’, ‘certificate verify failed’)],)”,),)) Cloud account: null Task: /provisioning/endpoint-tasks/820902de-bf34-4c91-8217- e3eedd8ea609 (less)

Cause

• The cause is that vRA will not simply allow you to just trust the server certificate of Infoblox certificate.
  
  
• vRA looks to validate the entire certificate chain of the infobox server certificate, if this fails the integration fails.
  
  
• So the actual root cause is a failure of Infoblox server to present full certificate chain & in correct order to vRA 8.x for validation.

Confirming  issue

To confirm you are hitting this particular issue and not something similar you can ssh/putty to vRA 8.x appliance and run the below Openssl query against Infoblox server.


openssl s_client -showcerts -connect <InfoBloxHostname>:443


vRA UI will perform same validation as this query against the certificate chain presented by Infoblox server. In other words the Infoblox integration in the vRA UI will not succeed until the above Openssl command executes without error. Example of an incomplete or incorrect cert chain presented by Infoblox will return a response similar to:

verify error:num=21:unable to verify the first certificate

Workaround

To workaround, you can set the Infoblox.IPAM.DisableCertificateCheck parameter to True and Save the endpoint. This will disable the SSL certificate checks so you won't get any more errors. However, from a security perspective this is not the safest option since this opens the door for MITM attacks.

Resolving  issue

The correct solution is to have Infoblox server present the full certificate chain & in the correct order.

1. Open a browser connection to the Infoblox server click on the lock/security Icon at top of browser and open the certificate object. If you navigate to CertificationPath.




Above example shows a 3 cert chain with root cert , an intermediate cert then the infobox server certificate object.

* In case the browser does not display the intermediate certificate and the CA - contact the Infoblox server administrator and ask him to provide the complete chain of signer certificates that were used for signing the Infoblox server csr.

2. Click on every certificate from the Certification Path tab except the server certificate and export it in PEM format:

• Intermediate CA -> View Certificate -> Details -> Copy to File -> Base 64 encoded X.509 (.CER) -> Save
• Root CA -> View Certificate -> Details -> Copy to File -> Base 64 encoded X.509 (.CER) -> Save

3. Combine the intermediate certificate and the CA into a single .pem file. The order in which the certificates are stored within this .pem file is very important.

The CA must be the last in the file, with each signer from the chain on top.
In our example above the Intermediate CA must be first, followed by Root CA. It should look similar to:


-----BEGIN CERTIFICATE-----
MIIF3TCCA8WgAwIBAgICEAAwDQYJKoZIhvcNAQELBQAweTELMAkGA1UEBhMCR0Ix
EDAOBgNVBAgMB0VuZ2xhbmQxEjAQBgNVBAoMCUFsaWNlIEx0ZDEoMCYGA1UECwwf
.......Content shortened/truncated
45AtvSIDekHL3VJ7J0aipUKBmqqc8ZBLeeUAwo7YRZrAIcFuytWW0YccO4wKTcdT
w6fiPXnlQ8bguriRd939pDOgXfmHtAd6jXpPR+X5U0kMiYovUhXYoMMoDGFjpdN9
w+szwROA2xNyJSqP0pXv2CI=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIF4zCCA8ugAwIBAgIUGfziowvZwdob4sf6zEZVMO4kyEEwDQYJKoZIhvcNAQEL
BQAweTELMAkGA1UEBhMCR0IxEDAOBgNVBAgMB0VuZ2xhbmQxEjAQBgNVBAoMCUFs
.......Content shortened/truncated
WXxBH/Ql11ON88am3zC7pvZn8tvml96PxEm24Ra9WuO4FUInZdHUzRxycDw303nm
/GV2f8dg4yrg3uOd46hk7U/yqm9+gjIFh/Oq/Ha4ixEGszj7f25cAUSfwi4DDJ6+
aRAibW7f3xUTm9VL+wBnQxMu8NHodNM=
-----END CERTIFICATE-----

4. In Infoblox navigate to:

Infoblox -> Grid -> Members -> Certificates -> Manage CA Certificates:

Upload the newly created .pem file from step 3.

5. Wait for 2-3 minutes until Infoblox picks up the changes and verify that the full certificate chain is now returned: Execute again the OpenSSL command:

openssl s_client -showcerts -connect <InfoBloxHostname>:443

The output should confrim the Infoblox appliance now returns the full certificate chain.

Navigate to vRA and change Infoblox.IPAM.DisableCertificateCheck to False(If changed).

Click Validate in vRA UI. You should no longer see the error