Description

Starting from Contrail 4.0 and 5.0, users may notice in-balance of generators distribution among analytics nodes in an HA cluster, i.e., one analytics node serves generators from all computes/controllers, and most analytics processes. Therefore, it attracts most of the analytics traffic. Analytics services on all three nodes appear to be healthy and node status is good.

Symptoms

Customer has an HA cluster running 4.1.2.0 release. After two analytics nodes reboot, the below in-balance of generators distribution is seen. Notice the highlighted node serves almost all generator:

alt

Solution

Starting from contrail 4.0, discovery service is removed , hence such load distribution in-balance may be observed.

This behavior is working as designed. ​As seen in the above screenshot, the node that has the maximum generators, has been up for the longest period of time (16 days), as compared to 2 days in the case of the other nodes.Therefore, that collector is assigned to the maximum number of the generators. The collectors are randomly assigned to the generators. So, in cases where one node has been up for a significantly longer time than another, it can obviously happen that that node has more generators.