This article assists with troubleshooting Contrail vDNS. This information will also be useful when submitting a JTAC ticket. 
This contains introspect commands and also some logs and other commands that need to be used.

Logs that need to be gathered

• contrail-control: Accepts XMPP connections from vrouter nodes. If the deployment has more than one controller, each vrouter node connects to two Contrail controller nodes running contrail-control process. The controller selection process is handled by the discovery process.  Controller also connects to ifmap server on port 8443 to receive config updates.
• contrail-control-nodemgr: Collects Contrail controller process info, status and reports to Contrail collector.

• contrail-dns: Provides DNS service to the VMs spawned on different compute nodes. It uses Bind.  If the deployment has more than one controller, each vrouter node connects to two Contrail controller nodes running contrail-dns process. The controller selection process is handled by discovery process.     

• contrail-named: Complements Contrail-dns process.

Logs:

• Contrail-control logs:/var/log/contrail/contrail-control.log, /var/log/contrail/contrail-control-stdout.log

• Contrail-dns logs: /var/log/contrail/contrail-dns.log, /var/log/contrail/contrail-dns-stdout.log

• Contrail-named logs: /var/log/contrail/contrail-named.log, /var/log/contrail/contrail-named-stdout.lo g

Note: If Contrail-named logs are empty then please run   “docker logs contrail-named-container-id”

• Access  http://hypervisor-ip:8085/Snh_SandeshTraceRequest?x=DnsBind  to get how vrouter-agent handled request for dns resolution
• Find hypervisor on which VM whose name is not resolving
• Access  http://hypervisor-ip:8085/Snh_SandeshTraceRequest?x=XmppMessageTrace  to see if vrouter-agent sent the VM details to the contrail-dns service
• Access  http://controller-ip:8092/Snh_SandeshTraceRequest?x=XmppMessageTrace  to see if contrail-dns received message from vrouter-agent about VM presence
• Access  http://controller-ip:8092/Snh_SandeshTraceRequest?x=DnsBind  to see whether contrail-dns programmed bind entries correctly on every controller
• Access  http://controller-ip:8092/Snh_ShowDnsConfig  to see which records the controller is aware of for this virtual dns. The last column, installed, must be true which indicates the record was successfuly sent to bind/named

1. Dig command from the VM or Physical server with the DNS IP address being the @ IP address

   EG: dig [email protected] @10.244.8.72

2. On the controller run contrail-rndc -c /etc/contrail/dns/contrail-rndc.conf dumpdb -zones

   The above command creates a file "named_dump.db" that has all records from all zones, which you can grep to look for records

3. grep in /etc/contrail/dns/* in every controller to check presence of entry. zone files are lazily created and thus do not indicate a problem if grep does not find anything. The opposite is true for jnl files, a positive match on the jnl file also does not indicate a that named is aware of such records.

• KB33932 - Understanding vDNS workflow in Contrail Networking [juniper.net]