A false alarm with the message "A core file has been generated on the node" is always displayed on the Contrail WebUI if any file or directory exists under /var/crashes/ and the core pattern of the file or directory, by default, equals /var/crashes/core.%e.%p.%h.%t .

This article provides the reason for the false alarm to be generated and suggests a workaround.

On the Contrail WebUI, the following warning (as highlighted in the screenshot) may be seen:

If any file or directory exists under /var/crashes/ and the core pattern of the file or directory, by default, equals /var/crashes/core.%e.%p.%h.%t , the above false alarm is generated.

This problem does not seem to exist in Contrail release 3.0.x but can be consistently reproduced in Contrail 3.2.8.0 and later, including Contrail release 5.0.1.0.

Results of the lab test performed by using Contrail release 5.0.1.0 are as follows:

The Contrail version is determined as follows:

(vrouter-agent)[root@aio50 /var/crashes]$ contrail-version
Package Version Build-ID | Repo | RPM Name
-------------------------------------- ------------------------------ ----------------------------------
contrail-lib 5.0-129.el7 @contrail
contrail-vrouter-agent 5.0-129.el7 @contrail
contrail-vrouter-utils 5.0-129.el7 @contrail
python-contrail 5.0-129.el7 @contrail
python-contrail-vrouter-api 5.0-129.el7 @contrail
contrail-utils 5.0-129.el7 @contrail
python-opencontrail-vrouter-netns 5.0-129.el7 @contrail
contrail-setup 5.0-129.el7 @contrail

​core_pattern , by default, is set to the following value:

(vrouter-agent)[root@aio50 /]$ cat /proc/sys/kernel/core_pattern
/var/crashes/core.%e.%p.%h.%t

Now if a lost+found directory is created under /var/crashes , the false alarm is triggered.

Note: On a customer's setup, the lost+found directory may be hidden, so the " -a " switch must be included when the unix list command is run.

(vrouter-agent)[root@aio50 /]$ cd /var/crashes/
(vrouter-agent)[root@aio50 /var/crashes]$ mkdir lost+found
(vrouter-agent)[root@aio50 /var/crashes]$ ls -lart
total 4
drwxr-xr-x 2 root root 4096 Jul 27 21:01 lost+found

​On the Contrail WebUI, the following warning may then be seen:

However, a contrail-status check does not show any valid core files existing in the output.

root@aio50:~# contrail-status -d
Pod        Service         Original Name                          State    Status    
analytics  alarm-gen       contrail-analytics-alarm-gen           running  Up 7 days 
analytics  api             contrail-analytics-api                 running  Up 7 days 
analytics  collector       contrail-analytics-collector           running  Up 7 days 
analytics  nodemgr         contrail-nodemgr                       running  Up 7 days 
analytics  query-engine    contrail-analytics-query-engine        running  Up 7 days 
config     api             contrail-controller-config-api         running  Up 7 days 
config     cassandra       contrail-external-cassandra            running  Up 7 days 
config     device-manager  contrail-controller-config-devicemgr   running  Up 7 days 
config     nodemgr         contrail-nodemgr                       running  Up 7 days 
config     schema          contrail-controller-config-schema      running  Up 7 days 
config     svc-monitor     contrail-controller-config-svcmonitor  running  Up 7 days 
config     zookeeper       contrail-external-zookeeper            running  Up 7 days 
control    control         contrail-controller-control-control    running  Up 7 days 
control    dns             contrail-controller-control-dns        running  Up 7 days 
control    named           contrail-controller-control-named      running  Up 7 days 
control    nodemgr         contrail-nodemgr                       running  Up 7 days 
database   cassandra       contrail-external-cassandra            running  Up 7 days 
database   kafka           contrail-external-kafka                running  Up 7 days 
database   nodemgr         contrail-nodemgr                       running  Up 7 days 
database   zookeeper       contrail-external-zookeeper            running  Up 7 days 
vrouter    agent           contrail-vrouter-agent                 running  Up 7 days 
vrouter    nodemgr         contrail-nodemgr                       running  Up 7 days 
webui      job             contrail-controller-webui-job          running  Up 7 days 
webui      web             contrail-controller-webui-web          running  Up 7 days 

vrouter kernel module is PRESENT
== Contrail control ==
control: active
nodemgr: active
named: active
dns: active

== Contrail database ==
kafka: active
nodemgr: active
zookeeper: active
cassandra: active

== Contrail analytics ==
snmp-collector: inactive
query-engine: active
api: active
alarm-gen: active
nodemgr: active
collector: active
topology: inactive

== Contrail webui ==
web: active
job: active

== Contrail vrouter ==
nodemgr: active
agent: active

== Contrail config ==
api: active
zookeeper: active
svc-monitor: active
nodemgr: active
device-manager: active
cassandra: active
rabbitmq: inactive
schema: active

Now if the details of the alarm are checked in JSON format, the matched file that Contrail believes to be a core dump is displayed, which, in this case, is nothing but lost+found .

The existing file or directory under /var/crashes/ that is generating the false alarm should be deleted. The alarm must subsequently be acknowledged for it to be removed from the Contrail WebUI.