This article details the procedure to collect error logs in Contrail Service Orchestration (CSO) that can be used for troubleshooting purposes.

Note: Starting from CSO 5.1.0 release, CSO can be installed as On-Prem or in Cloud.

If you are using the Cloud version of CSO, report the JTAC case with a clear description of the issue. JTAC will confirm the issue and report it to CSO Cloud Dev-Ops.

Collect the following information:

1. Error logs, which can be collected from central and regional VMs.

   1. Navigate to /root of the server from which the logs needs to be collected, and run  ./getlogs.sh .
   2. The script logs collected are listed in the script output and stored in /opt/csp/logs/ .

2. Version information:

   1. Log in to the Installer VM.
   2. Navigate to path /root/Contrail_Service_Orchestration_4.0.2 .
   3. Execute the command -  /cso-status.sh version_info regional .

   Example :
   root@installervm:~/Contrail_Service_Orchestration_4.0.2# ./cso-status.sh version_info regional

   {'build_date': datetime.datetime(2018, 10, 18, 3, 1, 5, 569896), 'version': '4.0.2', 'display_name': 'Contrail Service Orchestration', 'release_type': 'XXX', 'package_name': 'Contrail_Service_Orchestration'}

3. Individual service health checkup information:

   1. Log in to the Installer VM.
   2. Navigate to path /root/Contrail_Service_Orchestration_4.0.2 .
   3. Execute the command - ./python.sh monitoring/system_healthcheck/check_infracomponent.py central <service_name> .
   4. Python module to check the cluster health of the following services : Cassandra, ElasticSearch, Etcd, MariaDB, RabbitMQ , ZooKeeper, Redis, ArangoDb, Sim_Cluster, Elk_Logstash, Elk_Kibana, Contrail_Analytics, Keystone, Swift and Kubernetes

   Example :

   root@installervm:~/Contrail_Service_Orchestration_4.0.2# ./python.sh monitoring/system_healthcheck/check_infracomponent.py central arangodb

************************************************************************

HEALTH CHECK FOR INFRASTRUCTURE COMPONENTS STARTED IN CENTRAL ENVIRONMENT

************************************************************************

INFO     Health Check for Component arangodb Started
INFO     The Components arangodb is Healthy in central Environment    

4. All service health checkup:

   1. Log in to the Installer VM.
   2. Navigate to path /root/Contrail_Service_Orchestration_4.0.2 .
   3. Execute ./components_health.sh .
   4. Output is displayed on the screen and logs are stored at /root/Contrail_Service_Orchestration_ <version> /logs/health_check_console.log , /root/Contrail_Service_Orchestration_ <version> /logs/health_check_error.log , /root/Contrail_Service_Orchestration_ <version> /logs/health_check.log .

   Example :

   root@installervm:~/Contrail_Service_Orchestration_4.0.2# ./components_health.sh
Syncing salt data, this will take a while..

************************************************************************

HEALTH CHECK FOR INFRASTRUCTURE COMPONENTS STARTED IN CENTRAL ENVIRONMENT

************************************************************************

INFO     Health Check for Infrastructure Component Cassandra Started
INFO     The Infrastructure Component Cassandra is Healthy

INFO     Health Check for Infrastructure Component ElasticSearch Started
INFO     The Infrastructure Component ElasticSearch is Healthy

INFO     Health Check for Infrastructure Component Etcd Started
INFO     The Infrastructure Component Etcd is Healthy

INFO     Health Check for Infrastructure Component MariaDb Started
INFO     The Infrastructure Component MariaDb is Healthy

INFO     Health Check for Infrastructure Component RabbitMQ Started
INFO     The Infrastructure Component RabbitMQ is Healthy

INFO     Health Check for Infrastructure Component ZooKeeper Started
INFO     The Infrastructure Component ZooKeeper is Healthy

INFO     Health Check for Infrastructure Component Redis Started
INFO     The Infrastructure Component Redis is Healthy

INFO     Health Check for Infrastructure Component ArangoDb Started
INFO     The Infrastructure Component ArangoDb is Healthy

INFO     Health Check for Infrastructure Component Sim_Cluster Started
INFO     The Infrastructure Component Sim_Cluster is Healthy

INFO     Health Check for Infrastructure Component Elk_Logstash Started
INFO     The Infrastructure Component Elk_Logstash is Healthy

INFO     Health Check for Infrastructure Component Elk_Kibana Started
INFO     The Infrastructure Component Elk_Kibana is Healthy

INFO     Health Check for Infrastructure Component Keystone Started
INFO     The Infrastructure Component Keystone is Healthy

INFO     Health Check for Infrastructure Component Swift Started
INFO     The Infrastructure Component Swift is Healthy

INFO     Health Check for Infrastructure Component Kubernetes Started
INFO     The Infrastructure Component Kubernetes is Healthy

INFO     Health Check for Infrastructure Component Contrail_Analytics Started
INFO     The Infrastructure Component Contrail_Analytics is Healthy


Overall result:

     The following Infrastructure Components are Healthy:

         ['Cassandra', 'ElasticSearch', 'Etcd', 'MariaDb', 'RabbitMQ', 'ZooKeeper', 'Redis', 'ArangoDb', 'Sim_Cluster', 'Elk_Logstash', 'Elk_Kibana', 'Keystone', 'Swift', 'Kubernetes', 'Contrail_Analytics']

HEALTH CHECK NOT REQUIRED FOR COLLOCATED REGION

Please check the logs:
Console Logs Path: /root/Contrail_Service_Orchestration_4.0.2/logs/health_check_console.log
Error Logs Path: /root/Contrail_Service_Orchestration_4.0.2/logs/health_check_error.log
Info Logs Path: /root/Contrail_Service_Orchestration_4.0.2/logs/health_check.log

INFO     =============== Health Check start time: 2019-06-12 00:58:58.521790 ==============
INFO     =============== Health Check end time: 2019-06-12 01:00:07.490042   ==============
INFO     =============== Health Check duration: 0:01:08.968252   ==============

    

2019-06-15: Updated

2020-01-09: Added note about Cloud and On-Prem CSO installation

• KB32131 - [Contrail] Data Collection Checklist [juniper.net]