Description

This KB article helps troubleshoot problems with Syslog over TLS implementation in a GNF within a Node Slicing setup due to certificate validation errors.

Symptoms

It is observed that after the TCP session is established between the GNF and the syslog server, subsequent communication halts. Eventd traces reveal a certificate error, stating "Certificate is not valid." But no issue is seen with the certs generated.
 
Test Topology:

GNF gnf2-re1 (Eventd client) 10.220.170.1 <=========> rsyslog-gnutls Ubuntu Syslog Server 10.220.170.3 (TLS connection)

Eventd Traces (Snippet):

Nov 22 18:06:25 SSL connection not NULL
Nov 22 18:06:25 SSL Handshake success
Nov 22 18:06:25 Certificate received, and Peer chain is not NULL
Nov 22 18:06:25 LIBPKI pki_request_chain_validation_(opaque[0x1ab78c0], id_type[DER_SUBJECT_DN-0], id_length[103], id[0x1ac1580], chain[0x2c761e0], response_cb[0x57c560], response_arg[0x2bef840]) {
 .
Nov 22 18:06:25 Certificate is not valid
 .

Setup Details:

GNF: Guest Network Function (VM) - MX device partition (TLS client)
BSYS: Base System - MX device; Line cards of BSYS allocated to GNF
Syslog Server: Ubuntu Server with rsyslogd and gnutls driver (TLS server)
CA Server: The same syslog server used as Certificate Authority Server
Auth Mode: Server Authentication enabled on the syslog server

Solution

The TLS session fails due to an invalid certificate message, as the PKI-service in BSYS handles certificate verification requests from eventd in the GNF. If CA certificates aren't loaded on BSYS, the verification request from GNF's eventd to PKI-service on BSYS is invalidated. To resolve this, configure the CA certificate on BSYS as a workaround. 

Below is the detailed config that needs to be done for Syslog over TLS on a GNF

# Generate CA certificates

openssl genrsa -out ca-2k-sha2_512.key 2048
openssl req -new -x509 -days 3650 -key ca-2k-sha2_512.key -out ca-2k-sha2_512.crt -days 365 -subj "/C=US/ST=CA/L=Sunnyvale/O=Juniper/CN=root/OU=QA"
openssl x509 -in ca-2k-sha2_512.crt -out ca-2k-sha2_512.pem -outform PEM
openssl x509 -in ca-2k-sha2_512.pem -text 

# Generate sub CA certificates

openssl genrsa -out subca-2k-sha2_512.key 2048
openssl req -new -key subca-2k-sha2_512.key -out subca-2k-sha2_512.csr -subj "/C=US/ST=CA/L=Sunnyvale/O=Juniper/CN=SUB/OU=QA"
openssl x509 -req -days 3650 -in subca-2k-sha2_512.csr -CA ca-2k-sha2_512.crt -CAkey ca-2k-sha2_512.key -CAcreateserial -out subca-2k-sha2_512.crt -extfile /etc/ssl/openssl.cnf -extensions v3_ca
openssl x509 -in subca-2k-sha2_512.crt -out subca-2k-sha2_512.pem -outform PEM
openssl x509 -in subca-2k-sha2_512.pem -text 

# Generate Client and Server Certificates:

openssl genrsa -out client.key 2048
openssl req -new -key client.key -out client.csr -subj "/C=US/ST=CA/L=Sunnyvale/O=Juniper/CN=localhost/OU=QA"
openssl x509 -req -days 3650 -sha512 -in client.csr -CA subca-2k-sha2_512.crt -CAkey subca-2k sha2_512.key -CAcreateserial -out client.crt -extfile /etc/ssl/openssl.cnf -extensions v3_req
openssl x509 -in client.crt -out client.pem -outform PEM
openssl x509 -in client.pem -text
openssl genrsa -out server.key 2048
openssl req -new -key server.key -out server.csr -subj "/C=US/ST=CA/L=Sunnyvale/O=Juniper/CN=10.220.170.1 /OU=QA"
openssl x509 -req -days 3650 -sha512 -in server.csr -CA subca-2k-sha2_512.crt -CAkey subca-2k-sha2_512.key -CAcreateserial -out server.crt -extfile /etc/ssl/openssl.cnf -extensions v3_req
openssl x509 -in server.crt -out server.pem -outform PEM
openssl x509 -in server.pem -text
cat ca-2k-sha2_512.pem subca-2k-sha2_512.pem > ca-chain.pem 

Copy the files to the BSYS:

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null /etc/ssl/ca-2k-sha2_512.pem [email protected]:/root/
scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null /etc/ssl/subca-2k-sha2_512.pem [email protected]:/root/

The provided configuration steps are for setting up a syslog client on an MX router with additional configurations for secure syslog transport using TLS.
Configure PKI CA Profiles. This needs to be done on both BSYS and GNF

set security pki ca-profile ms-ca ca-identity test
set security pki ca-profile ms-ca revocation-check disable
set security pki ca-profile ms-subca ca-identity test
set security pki ca-profile ms-subca revocation-check disable
set security pki traceoptions flag all
commit 

Load CA Certificates:

This needs to be done on BSYS

request security pki ca-certificate load ca-profile ms-ca filename /root/ca-2k-sha2_512.pem 
request security pki ca-certificate load ca-profile ms-subca filename /root/subca-2k-sha2_512.pem

Configure Syslog Host and TLS Details (On GNF) :

set system syslog host 10.220.170.3 any any
set system syslog host 10.220.170.3 allow-duplicates
set system syslog host 10.220.170.3 port 30013
set system syslog host 10.220.170.3 transport tls
set system syslog host 10.220.170.3 tlsdetails trusted-ca-group abc ca-profiles ms-ca
set system syslog host 10.220.170.3 tlsdetails trusted-ca-group abc ca-profiles ms-subca
set system syslog file messages any any
commit

Modification History

2023-12-17 : Article Created