Contrail Configuration API uses REST to configure contrail SDN cluster. Contrail-api performs all the operations in the back end with cassandra and other contrail-services to create, read, update and delete different contrail objects/resources.
Requests to contrail-api may be originated from end users, contrail-GUI, and all the different applications that utilize vnc-api python library that interacts with contrail-api directly. All the requests to contrail-api are logged under '/var/log/contrail/contrail-api-0-stdout.log' . However, these events only log the requests URL. Details about request headers or even payload of the request are not logged.
Here's an example of api-request log from '/var/log/contrail/contrail-api-0-stdout.log' during a virtual-machine creation:
127.0.0.1 - - [2018-10-23 12:45:44] "POST /virtual-machines HTTP/1.1" 200 365 0.016590 127.0.0.1 - - [2018-10-23 12:45:45] "POST /virtual-machine-interfaces HTTP/1.1" 200 589 0.172543 127.0.0.1 - - [2018-10-23 12:45:45] "POST /instance-ips HTTP/1.1" 200 357 0.114780 127.0.0.1 - - [2018-10-23 12:45:45] "POST /instance-ips HTTP/1.1" 200 357 0.156959
As seen above, log events only display the endpoint URL where the requests are coming in. It is not displaying the payload in these requests. If an end user is trying to troubleshoot any issue where it is critical to know the payload that went in these requests to api-server, then these API logs are not very helpful.
In order to get additional information, Contrail offers an introspect named, 'RestApiTraceBuff' for all the REST calls that are made to the api-server. Access this introspect either by using contrail GUI or by doing a 'curl' on the introspect port (8084) of contrail-api process.
'RestApiTraceBuff'
Navigate to Settings -> Introspect -> Config Nodes
->
Select the IP address of the config node from the drop down menu.
Select Sandesh_trace under module section and SandeshTraceBufferListRequest under Request section.
The above request should display a list of all the different Introspect commands available under sandesh_trace.
In this article we are interested in REST API payload, so we select the corresponding introspect command named "RestApiTraceBuf".
We should now be able to see a list of all the REST calls made to contrail-api endpoint along with request payload.
Here's how an output from "RestApiTraceBuff" for the same API calls looks like as compared to that seen under '/var/log/contrail/contrail-api-0-stdout.log '.
2018-10-23 12:45:44.836 RestApiTrace: req-c7d7fe6d-dad7-4dee-a07a-0148c71aa7f0 http://127.0.0.1:9100/virtual-machines POST {"virtual-machine": { "perms2": { "owner": "8daacafe-6838-4254-abcf-12702d9fb884", "owner_access": 7, "global_access": 0, "share": [] }, "fq_name": ["3e912080-81d6-4722-b6a7-c1f2beecafd7"], "uuid": "3e912080-81d6-4722-b6a7-c1f2beecafd7" } } 200 OK {"virtual-machine": { "href": "http://127.0.0.1:9100/virtual-machine/3e912080-81d6-4722-b6a7-c1f2beecafd7", "fq_name": ["3e912080-81d6-4722-b6a7-c1f2beecafd7"], "name": "3e912080-81d6-4722-b6a7-c1f2beecafd7", "uuid": "3e912080-81d6-4722-b6a7-c1f2beecafd7" } } 2018-10-23 12:45:44.949 RestApiTrace: req-18f30270-5f86-4024-a654-14bdf333c877 http://127.0.0.1:9100/virtual-machine-interfaces POST { "virtual-machine-interface": { "fq_name": [ "default-domain", "admin-bc8d5f44-ef96-4588-9579-99bc2c09b4a1", "7b62a1a6-95db-4c90-8109-3bd7dc93a013" ], "id_perms": { "created": null, "creator": null, "description": null, "enable": true, "last_modified": null, "permissions": null, "user_visible": true, "uuid": null }, "parent_type": "project", "perms2": { "global_access": 0, "owner": "8daacafe68384254abcf12702d9fb884", "owner_access": 7, "share": [] }, "port_security_enabled": true, "security_group_refs": [ { "to": [ "default-domain", "admin-bc8d5f44-ef96-4588-9579-99bc2c09b4a1", "default" ], "uuid": "92814530-37dc-4984-a41a-2de3357bdff2" } ], "uuid": "7b62a1a6-95db-4c90-8109-3bd7dc93a013", "virtual_machine_interface_bindings": { "key_value_pair": [ { "key": "host_id", "value": "Aio328" }, { "key": "vnic_type", "value": "normal" } ] }, "virtual_machine_interface_device_owner": "compute:nova", "virtual_machine_interface_dhcp_option_list": null, "virtual_machine_refs": [ { "to": [ "3e912080-81d6-4722-b6a7-c1f2beecafd7" ], "uuid": "3e912080-81d6-4722-b6a7-c1f2beecafd7" } ], "virtual_network_refs": [ { "to": [ "default-domain", "MNS-25180-P-host", "MNS-25180-P-host_cor_direct_net_1" ], "uuid": "f9a46a2c-dc3a-44b8-8ec9-21be9dcfac39" } ] } } 2018-10-23 12:45:45.235 RestApiTrace: req-d3055e00-186d-4095-8d5d-2278d041b964 http://127.0.0.1:9100/instance-ips POST { "instance-ip": { "fq_name": [ "ba02119f-d634-4c56-91cb-409dcf5dc86c" ], "instance_ip_family": "v4", "perms2": { "global_access": 0, "owner": "8daacafe68384254abcf12702d9fb884", "owner_access": 7, "share": [] }, "uuid": "ba02119f-d634-4c56-91cb-409dcf5dc86c", "virtual_machine_interface_refs": [ { "to": [ "default-domain", "admin-bc8d5f44-ef96-4588-9579-99bc2c09b4a1", "7b62a1a6-95db-4c90-8109-3bd7dc93a013" ], "uuid": "7b62a1a6-95db-4c90-8109-3bd7dc93a013" } ], "virtual_network_refs": [ { "to": [ "default-domain", "MNS-25180-P-host", "MNS-25180-P-host_cor_direct_net_1" ], "uuid": "f9a46a2c-dc3a-44b8-8ec9-21be9dcfac39" } ] } }
The same "RestApiTraceBuf" output can also be retrieved using a curl request to config node introspect port if using Contrail GUI is not desirable or available. By default , contrail-config node has introspect port of 8084.
chg1a replaced with 'host'