Fixing “unable to get local issuer certificate” and “certificate verify failed” in syslog-ng

My Problem

On a Linux host, attempting to set up syslog-ng shipping to an offsite collector over TLS results in errors such as:

Feb 18 16:42:33 my.amazingserver.tld syslog-ng[987]: SSL error while writing stream; tls_error='SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed'
Feb 18 16:42:33 my.amazingserver.tld syslog-ng[987]: I/O error occurred while writing; fd='21', error='Broken pipe (32)'
Feb 18 16:42:33 my.amazingserver.tld syslog-ng[987]: I/O error occurred while writing; fd='21', error='Broken pipe (32)'
Feb 18 16:42:33 my.amazingserver.tld syslog-ng[987]: Syslog connection broken; fd='21', server='AF_INET(1.1.1.1:443)', time_reopen='60'
Feb 18 16:43:33 my.amazingserver.tld syslog-ng[987]: Syslog connection established; fd='21', server='AF_INET(1.1.1.1:443)', local='AF_INET(0.0.0.0:0)'
Feb 18 16:43:33 my.amazingserver.tld syslog-ng[987]: Certificate validation failed; subject='CN=GeoTrust DV SSL CA - G4, OU=Domain Validated SSL, O=GeoTrust Inc., C=US', issuer='CN=GeoTrust Global CA, O=GeoTrust 
Inc., C=US', error='unable to get local issuer certificate', depth='1'
Feb 18 16:34:31 my.amazinghost.com syslog-ng[987]: Certificate validation failed; subject='CN=GeoTrust DV SSL CA - G4, OU=Domain Validated SSL, O=GeoTrust Inc., C=US', issuer='CN=GeoTrust Global CA, O=GeoTrust 
Inc., C=US', error='unable to get local issuer certificate', depth='1'

As well as

SSL error while writing stream; tls_error='SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed'

Running openssl s_client -connect my.syslog.endpoint.tld:443 gets, after the certificate chain, the following error: Verify return code: 20 (unable to get local issuer certificate)

My Solution

Find the CA certificate that is missing as indicated by the error message in the logs (in my case it was the GeoTrust Global CA certificate). Then follow the procedure for configuring the syslog-ng client for TLS.

The Long Story

The first clue to the problem is in the error message:

Feb 18 16:34:31 my.amazinghost.com syslog-ng[987]: Certificate validation failed; subject='CN=GeoTrust DV SSL CA - G4, OU=Domain Validated SSL, O=GeoTrust Inc., C=US', issuer='CN=GeoTrust Global CA, O=GeoTrust
Inc., C=US', error='unable to get local issuer certificate', depth='1'

The connection was being thwarted by not trusting “GeoTrust DV SSL CA – G4” specifically. If you go to GeoTrust’s website and look for their various root certificates, you’ll notice that there are a lot to look through.

I copied the root certificate into a temporary file on the server named geotrust.test.ca.cer. I then ran the same s_client commant as above, but pointed it at the testing CA file I just made:

openssl s_client -connect my.syslog.endpoint.tld:443 -CAfile geotrust.test.ca.cer

Magically, at the end of the connection:

Verify return code: 0 (ok)

Yay! So that’s the cert I need. What I did next was found the root GeoTrust global CA certificate from their site:

 

-----BEGIN CERTIFICATE-----
MIIDVDCCAjygAwIBAgIDAjRWMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT
MRYwFAYDVQQKEw1HZW9UcnVzdCBJbmMuMRswGQYDVQQDExJHZW9UcnVzdCBHbG9i
YWwgQ0EwHhcNMDIwNTIxMDQwMDAwWhcNMjIwNTIxMDQwMDAwWjBCMQswCQYDVQQG
EwJVUzEWMBQGA1UEChMNR2VvVHJ1c3QgSW5jLjEbMBkGA1UEAxMSR2VvVHJ1c3Qg
R2xvYmFsIENBMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA2swYYzD9
9BcjGlZ+W988bDjkcbd4kdS8odhM+KhDtgPpTSEHCIjaWC9mOSm9BXiLnTjoBbdq
fnGk5sRgprDvgOSJKA+eJdbtg/OtppHHmMlCGDUUna2YRpIuT8rxh0PBFpVXLVDv
iS2Aelet8u5fa9IAjbkU+BQVNdnARqN7csiRv8lVK83Qlz6cJmTM386DGXHKTubU
1XupGc1V3sjs0l44U+VcT4wt/lAjNvxm5suOpDkZALeVAjmRCw7+OC7RHQWa9k0+
bw8HHa8sHo9gOeL6NlMTOdReJivbPagUvTLrGAMoUgRx5aszPeE4uwc2hGKceeoW
MPRfwCvocWvk+QIDAQABo1MwUTAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBTA
ephojYn7qwVkDBF9qn1luMrMTjAfBgNVHSMEGDAWgBTAephojYn7qwVkDBF9qn1l
uMrMTjANBgkqhkiG9w0BAQUFAAOCAQEANeMpauUvXVSOKVCUn5kaFOSPeCpilKIn
Z57QzxpeR+nBsqTP3UEaBU6bS+5Kb1VSsyShNwrrZHYqLizz/Tt1kL/6cdjHPTfS
tQWVYrmm3ok9Nns4d0iXrKYgjy6myQzCsplFAMfOEVEiIuCl6rYVSAlk6l5PdPcF
PseKUgzbFbS9bZvlxrFUaKnjaZC2mqUPuLk/IH2uSrW4nOQdtqvmlKXBx4Ot2/Un
hw4EbNX/3aBd7YdStysVAq45pmp06drE57xNNB6pXE0zX5IJL4hmXXeXxx12E6nV
5fEWCRE11azbJHFwLJhWC9kXtNHjUStedejV0NxPNO3CBWaAocvmMw==
-----END CERTIFICATE-----

The next step is to put that in a file and name it something sane, like geotrust-root-ca-g4.crt. From there, I put it in my syslog-ng certificate directory at /etc/syslog-ng/cert.d/.  After that you have to do a funny little two step by making a hash out of the distinguished name in the certificate:

$openssl x509 -noout -hash -in geotrust-root-ca-g4.crt
4c343fd2

After that you symlink geotrust-root-ca-g4.crt with that hash value appended with a .0 within the syslog-ng/cert.d directory:

ln -s geotrust-root-ca-g4.crt 4c343fd2.0

Finally, make sure that, within the syslog-ng configuration file, your destination uses the ca-dir option within the tls option:

destination log_collector { tcp( "my.syslog.endpoint.tld" port(443) template(remoteTemplate) tls(ca_dir("/etc/syslog-ng/cert.d"))); };

Restart services as necessary, and you should be up and sending syslog messages to your syslog collector using TLS.

Leave a Reply

Your email address will not be published. Required fields are marked *

Follow TheNubbyAdmin!

follow us in feedly

Raw RSS Feed:

Contact Me!

Want to hire me as a consultant? Have a job you think I might be interested in? Drop me a line:

Contact Me!

Subscribe via Email

Your email address is handled by Google FeedBurner and never spammed!

The Nubby Archives

Circle Me on Google+!

Photos from Flickr

Me on StackExchange

The IT Crowd Strava Group