This KB explains specific issue seen in Contrail 1909 release with respect to config_schema_1 container crash
Post upgrade to 1909, config_schema_1 container is crashing, In case of multiple nodes - causing it to schema failover and schema logs indicate below : Below is the snippet of contrail-schema.log from container config_schema_1 :
[contrail-schema] [INFO]: SANDESH: Logging: LEVEL: [SYS_INFO] -> [SYS_NOTICE] <type 'exceptions.NameError'> Python 2.7.5: /usr/bin/python A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred. /usr/bin/contrail-schema in <module>() 6 7 if __name__ == '__main__': 8 sys.exit( 9 load_entry_point('schema-transformer==0.1dev', 'console_scripts', 'contrail-schema')() 10 ) load_entry_point = <function load_entry_point> /usr/lib/python2.7/site-packages/schema_transformer/to_bgp.py in server_main() 797 def server_main(): 798 vnc_cgitb.enable(format='text') 799 main() 800 # end server_main 801 global main = <function main> /usr/lib/python2.7/site-packages/schema_transformer/to_bgp.py in main(args_str='--conf_file /etc/contrail/contrail-schema.conf --conf_file /etc/contrail/contrail-keystone-auth.conf') 790 st_logger, args) 791 finally: 792 if trasformer: 793 transformer.destroy_instance() 794 # end main trasformer undefined <type 'exceptions.NameError'>: global name 'trasformer' is not defined __class__ = <type 'exceptions.NameError'> __delattr__ = <method-wrapper '__delattr__' of exceptions.NameError object> __dict__ = {} __doc__ = 'Name not found globally.' __format__ = <built-in method __format__ of exceptions.NameError object> __getattribute__ = <method-wrapper '__getattribute__' of exceptions.NameError object> __getitem__ = <method-wrapper '__getitem__' of exceptions.NameError object> __getslice__ = <method-wrapper '__getslice__' of exceptions.NameError object> __hash__ = <method-wrapper '__hash__' of exceptions.NameError object> __init__ = <method-wrapper '__init__' of exceptions.NameError object> __new__ = <built-in method __new__ of type object> __reduce__ = <built-in method __reduce__ of exceptions.NameError object> __reduce_ex__ = <built-in method __reduce_ex__ of exceptions.NameError object> __repr__ = <method-wrapper '__repr__' of exceptions.NameError object> __setattr__ = <method-wrapper '__setattr__' of exceptions.NameError object> __setstate__ = <built-in method __setstate__ of exceptions.NameError object> __sizeof__ = <built-in method __sizeof__ of exceptions.NameError object> __str__ = <method-wrapper '__str__' of exceptions.NameError object> __subclasshook__ = <built-in method __subclasshook__ of type object> __unicode__ = <built-in method __unicode__ of exceptions.NameError object> args = ("global name 'trasformer' is not defined",) message = "global name 'trasformer' is not defined" The above is a description of an error in a Python program. Here is the original traceback: Traceback (most recent call last): File "/usr/bin/contrail-schema", line 9, in <module> load_entry_point('schema-transformer==0.1dev', 'console_scripts', 'contrail-schema')() File "/usr/lib/python2.7/site-packages/schema_transformer/to_bgp.py", line 799, in server_main main() File "/usr/lib/python2.7/site-packages/schema_transformer/to_bgp.py", line 792, in main if trasformer: NameError: global name 'trasformer' is not defined
This is due to error in the code during destroy instances https://github.com/Juniper/contrail-controller/blob/a17676178baff112aeddbd28e3267cad939d2459/src/config/schema-transformer/schema_transformer/to_bgp.py#L792
Note : This solution is specific to 1909 release Below are the instructions for the fix : Login to container config_schema_1 - docker exec -it config_schema_1 bash Edit the python script - vi /usr/lib/python2.7/site-packages/schema_transformer/to_bgp.py Correct the typo ( Line no 792, change from trasformer to transformer) Save and exit the python script - to_bgp.py restart the container - docker restart config_schema_1