This article explains 'yield in evaluate', its related terms and application scenario. This is applicable for Contrail versions 3.2.16, release 1908 or higher versions.
 

How can I ​take advantage of 'yield in evaluate' in contrail-schema.conf in scaled setups (with total policy rules > 8000)?

The following excerpt is from opencontrail.org :

When the network policy has more rules, the virtual network evaluate does CPU bound operation continuously beyond the negotiated zookeeper session timeout, which causes schema-transformer to lose primary election, causing backup schema-transfer to re-init and end up in same state. Thus schema-transformer bounces back and forth. Fix is to optionally yield in virtual network evaluate function

yield_in_evaluate = True is an optional parameter, which can be used in scaled setups (Total network policy rules > 8000). When enabled, the tight loop performing network evaluate will yield every zk_timeout/6 seconds for the zookeeper heartbeat greenlet. If the environment has more than 8000 policy rules, yield_in_evaluate should be set to True.

zk_timeout - Configured at client side in seconds. This will be converted to milliseconds and kazoo client will send to server during session timeout negotiation.

maxSessionTimeout - Configured at server side in milli seconds. (Default is 20 times the tickTime, which is 20 * 2000 = 40000ms)

During initial session establishment, zookeeper server and client (schema transformer) negotiates session timeout. The smallest configured among client/server will be used .

Example:

• If zk_timeout is set to 360 sec and maxSessionTimeout is set to 360000ms, the negotiated session timeout will be 360sec/360000ms, because both client/server is configured with the same value.
• If the client side zk_timeout setting is removed from contrail-schema.conf , as the default in schema-transformer code is 120sec, the negotiated session timeout will be 120000ms because the client side setting is smaller than the server side setting (360000ms).
• If the server side setting maxSessionTimeout is removed from zoo.cfg, as the server side default is (20 * 2000 = 40000ms), the negotiated timeout will be 40000ms because the server side setting is smaller than the client side setting (360 sec).
• The minimum yield time is 60 sec.

It is recommended to add zk_timeout(>=360 sec) and maxSessionTimeout(>=360000 ms) in contrail-schema.conf and zoo.cfg respectively because without these settings, the default session timeout will be (tickTime * 20) = (2000ms *20)= 40000ms/40sec. However, the yield default minimum time is 60 sec. Ideally, zk_timeout / maxSessionTimeout should be more than the yield time (60 sec).

Location of config files in respective containers -

(Note: name of container may vary as per deployment method )

Container - config_schema_1 : /etc/contrail/contrail-schema.conf

Container - config_database_zookeeper_1 : /conf/zoo.cfg

Without yield_in_evaluate set to True, a user can only tune zk_timeout / maxSessionTimeout to solve the schema restart issue. However, they need to increase zk_timeout / maxSessionTimeout whenever they scale up the number of network policy rules.

(Recommended) With yield_in_evaluate set to True and zk_timeout / maxSessionTimeout set to 360 sec / 360000 ms respectively, users do not need to change zk_timeout / maxSessionTimeout whenever they scale up the number of network policy rules.

Following is one of the ways to configure these two parameters via instances.yml of contrail-ansible-deployer :

instances:
  myserver:
    ip: X.X.X.X
    provider: bms
    roles:
      config:
          SCHEMA__DEFAULTS__yield_in_evaluate=True
          SCHEMA__DEFAULTS__zk_timeout=360
      config_database:
      control:
      webui:
      analytics: