This article covers a specific scenario in an environment with an increasing number of Access Control Lists (ACL) in version 3.2.9. It provides the root cause and solution for such scenarios.

The queues in RabbitMQ are high, and the contrail-schema status reports 'timeout':

The schema-zk.log indicates the timeout as shown below:

# grep -5 "Zookeeper connection lost" schema-zk.log
04/18/2019 07:17:25 PM [schema]: Connection dropped: socket connection broken
04/18/2019 07:17:25 PM [schema]: Transition to CONNECTING
04/18/2019 07:17:25 PM [schema]: Zookeeper connection lost

Due to an increasing number of ACLs in the environment, Zookeeper may exhibit a timeout behavior because the schema process can take longer time to complete the transactions.

This issue is fixed in version 3.2.14.

Work-around:

The following work-around can be added in version 3.2.9. This is not applicable to Contrail version 4.x and above.

A. Zookeeper timeout value needs to be added in schema-transformer:
Conf file: /etc/contrail/contrail-schema.conf
Section: DEFAULTS
Variable: zk_timeout=4000

B. The max session value needs to be increased to 4000000 in the Zookeeper configuration file:
Conf file: /etc/zookeeper/conf/zoo.cfg
Variable: maxSessionTimeout=4000000

C. The services need to be restarted to reflect the above changes:

service contrail-schema restart
service contrail-schema status
service zookeeper restart
service zookeeper status

2019-09-07: Minor, non-technical edit.