In the process of device onboarding, if we try to add device from the topology under configuration>devices. The device is seen in the device list but unable to see the device listed in the Device Group.
While adding the device to the device group, the newly added device will be not listed under the list.
Newly added device can be seen under Configuration >> Devices but when you try to add this device under a Device Group, the device is not listed in the drop down.
We have to verify the logs for understanding what is happening when trying to onboard the device:
We also need to check the entry for the device under the database.
Ø Connect to a DB
configuration_version | created_by | created_timestamp | description | device_family | device_family_name | device_properties | device_series | name | parent_type | parent_uuid | updated_by | updated_timestamp | uuid | vendor | key_value_pair | app_properties | subscriptions | ipv4_address | ipv6_address | iso_address | router_id | secondary_ip | secondary_ipv6_address | system_id | city | country | timezone | direction | login_method | password | port | privilege_level | privilege_login | privilege_password | retry | secret_key <Snip>
0 | admin | 1727418117903839 | | | | [] | | 172.16.18.108 | | | admin | 1727418261500517 | 17f3ef32-b550-4f45-9c0b-305f1aa84712 | CISCO | [] | null | [] | 172.16.18.108 | | | 11.0.0.108 | | | | | | | | SSH | $9$Xvp7-bji.f5FbsmfT3tpvWLX-wgoG | 0 | 0 | | | 3 | $9$kqmTCtuBRhn6KM8XbwQF36p0KvW7dbvMxdbYoafTQz9ARhSv8Xlebs2gUD/CAuIEeK8-VwW8Di.PQzEcSe8XN-w2oJdVQF36AtuO1IEy | 300 | northstar | t | t | f | f | 22 | 0 | f | 0 | | | | | | | 0 | 275f8a2d-e65d-466c-a7c9-d0c5d3e9e211 | [] | f | f | | | 0 | 161 | 0 | 0 | | STANDALONE | | ios-xr8 | | | | | \x | \x | [] | [] | [] | [] | [] | [] | [] | f | protobuf | | | | | | | | | \x | | \x | |
(1 row)
Ø Under the devicemodel we could see the entry for this device.
Ø Now, you need to check the other DB related to healthbot.
id | authentication | device-id | name | uuid | host | timezone | iAgent | open-config | outbound-ssh | snmp | syslog | flow | sflow | system-id | variable | vendor | tagging-profile | description | owner | use-ingest-receive-time | marked-for-delete
----+----------------+-----------+------+------+------+----------+--------+-------------+--------------+------+--------+------+-------+-----------+----------+--------+-----------------+-------------+-------+-------------------------+-------------------
(0 rows)
Ø From the above outputs we could see that there is no entry for the device under the healthbot DB.
Ø To verify further, let's validate the communication between devicemodel and config-server. To do so, check the connection status.
o Verify the IP’s associated for the device-model & config-server
config-server-799b58c56c-xpnpg 1/1 Running 0 123d 10.248.42.165 172.22.11.13 <none> <none>
[root@davinci-master~]# kubectl get pods -n ems -owide | grep -i devicemana
devicemanager-59599b86d4-k74jc 1/1 Running 1 22d 10.248.47.58 172.22.11.12 <none> <none>
[root@davinci-master~]# kubectl get pods -n ems -owide | grep -i devicemod
devicemodel-76546c896b-9hd9f 1/1 Running 0 5d22h 10.248.43.35 172.22.11.11 <none> <none>
devicemodel-connector-5476f8cfb6-k46v4 1/1 Running 1 23d 10.248.47.1 172.22.11.12 <none> <none>
[root@davinci-master~]
Make a note of the IP addresses for the device models and the config-server
Ø Check the connection between those two services.
[root@davinci-master~]# conntrack -L | grep 9091
tcp 6 118 TIME_WAIT src=10.248.42.160 dst=10.248.51.157 sport=47636 dport=9091 src=10.248.47.50 dst=10.248.42.160 sport=9091 dport=47636 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=2
tcp 6 86398 ESTABLISHED src=10.248.42.155 dst=10.248.47.40 sport=34930 dport=9091 src=10.248.47.40 dst=10.248.42.155 sport=9091 dport=34930 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
tcp 6 86389 ESTABLISHED src=10.248.42.155 dst=10.248.47.40 sport=33410 dport=9091 src=10.248.47.40 dst=10.248.42.155 sport=9091 dport=33410 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
tcp 6 86393 ESTABLISHED src=10.248.42.143 dst=10.248.53.147 sport=19091 dport=8086 src=10.248.42.148 dst=10.248.42.143 sport=8086 dport=19091 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
tcp 6 119 TIME_WAIT src=10.248.42.165 dst=10.248.51.157 sport=34252 dport=9091 src=10.248.47.50 dst=10.248.42.165 sport=9091 dport=34252 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
tcp 6 86393 ESTABLISHED src=10.248.42.164 dst=10.248.43.35 sport=40254 dport=9091 src=10.248.43.35 dst=10.248.42.164 sport=9091 dport=40254 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
conntrack v1.4.4 (conntrack-tools): 64018 flow entries have been shown.
Ø From the above output, we don’t see any communication established between the config-server & Devicemodel. We can verify the config-server and devicemodel logs for any info. If the connection had been there, there would be an entry for the config-server IP as the src and the device models IP addresses as the destination on port number 9091.
Ø To recover the issue we can restart config-server and check.
o kubectl delete pod -n healthbot <config-server-podname>
Ø Post restarting config-server pod, we could see connection established and everything working fine.
[root@davinci-mastertmp]#healthbot -owide | grep config-serv
config-server-864dcb557f-vgh75 1/1 Running 0 18m 10.248.42.175 172.22.11.13 <none> <none>
[root@davinci-mastertmp]# kubectl get po -n healthbot -owide | grep devicemod
devicemodel-76546c896b-9hd9f 1/1 Running 0 6d5h 10.248.43.35 172.22.11.11 <none> <none>
[root@davinci-mastertmp]#
o Connection is established between both the services.
[root@davinci-mastertmp]# conntrack -L | grep 9091
tcp 6 9 TIME_WAIT src=10.248.42.160 dst=10.248.51.157 sport=40832 dport=9091 src=10.248.47.50 dst=10.248.42.160 sport=9091 dport=40832 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
tcp 6 86389 ESTABLISHED src=10.248.42.175 dst=10.248.43.35 sport=50112 dport=9091 src=10.248.43.35 dst=10.248.42.175 sport=9091 dport=50112 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
tcp 6 86398 ESTABLISHED src=10.248.42.143 dst=10.248.53.147 sport=19091 dport=8086 src=10.248.42.148 dst=10.248.42.143 sport=8086 dport=19091 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
tcp 6 86390 ESTABLISHED src=10.248.42.155 dst=10.248.47.40 sport=39346 dport=9091 src=10.248.47.40 dst=10.248.42.155 sport=9091 dport=39346 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
tcp 6 86389 ESTABLISHED src=10.248.42.164 dst=10.248.43.35 sport=40254 dport=9091 src=10.248.43.35 dst=10.248.42.164 sport=9091 dport=40254 [ASSURED] mark=0 secctx=system_u:object_r:unlabeled_t:s0 use=1
conntrack v1.4.4 (conntrack-tools): 65933 flow entries have been shown.
You can also verify the same by checking the entry in the DB with the command:
kubectl exec -it -n common atom-db-0 -- psql -U postgres -d hb-ems-dmon -c "select * from device where name='ios-xr8';"
Now, you should be able to see the device listed when editting the device group.