Description

On MX platforms with Junos node slicing, communication between two JDM(Juniper Device Manager) servers is not possible when Connection Block 0(CB0) link is down, whereas CB1 link is still up. Secure Shell(SSH) connections cannot be established between two JDM servers, however there is no service impact.

Symptoms

First, physically disconnect CB0 interface on Server 0 and Server 1
>>> On Server 0, let p1p1 interface down
        [user@lab01-OS0 ~]$ ifconfig p1p1
        p1p1: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
                        ether b4:83:51:10:6a:9e  txqueuelen 1000  (Ethernet)
                        RX packets 262107940  bytes 139268737986 (129.7 GiB)
                        RX errors 0  dropped 10883109  overruns 0  frame 0
                        TX packets 117555370  bytes 20329364385 (18.9 GiB)
                        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
                                        
        [user@lab01-OS0 ~]$ sudo ifconfig p1p1 down
        [sudo] password for user:
        [user@EPCR2-RCH01-OS0 ~]$                 
        
        [user@lab01-OS0 ~]$ ifconfig p1p1              
        p1p1: flags=4354<BROADCAST,PROMISC,MULTICAST>  mtu 1500
                        ether b4:83:51:10:6a:9e  txqueuelen 1000  (Ethernet)
                        RX packets 262131844  bytes 139283750042 (129.7 GiB)
                        RX errors 0  dropped 10883865  overruns 0  frame 0
                        TX packets 117567950  bytes 20330230386 (18.9 GiB)
                        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        
>>>On Server 1, let p1p1 interface down        
        [user@lab01-OS1 ~]$ ifconfig p1p1
        p1p1: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 1500
                        ether b4:83:51:10:6a:a2  txqueuelen 1000  (Ethernet)
                        RX packets 19953899  bytes 11564595429 (10.7 GiB)
                        RX errors 0  dropped 10852047  overruns 0  frame 0
                        TX packets 83841119  bytes 6007046326 (5.5 GiB)
                        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        
        [user@lab01-OS1 ~]$ sudo ifconfig p1p1 down
        [sudo] password for user:
        [user@lab01-OS1 ~]$
        [user@lab01-OS1 ~]$ ifconfig p1p1         
        p1p1: flags=4354<BROADCAST,PROMISC,MULTICAST>  mtu 1500
                        ether b4:83:51:10:6a:a2  txqueuelen 1000  (Ethernet)
                        RX packets 19954067  bytes 11564608365 (10.7 GiB)
                        RX errors 0  dropped 10852215  overruns 0  frame 0
                        TX packets 83841185  bytes 6007050286 (5.5 GiB)
                        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        
Then verify JDM to JDM link0 is down and JDM to JDM link1 is up.        
        user@lab01-JDM0> show server connections all-servers | no-more   
        Feb 15 13:04:50
        server0:
        --------------------------------------------------------------------------
        Component               Interface                Status  Comments
        Host to JDM port        virbr0                   up    
        Physical CB0 port       p1p1                     down                  >>>  
        Physical CB1 port       p2p1                     up                    >>>    
        Physical JDM mgmt port  em3                      up    
        Physical VNF mgmt port  em4                      up    
        JDM-GNF bridge          bridge_jdm_vm            up    
        CB0                     cb0                      down                  >>>  
        CB1                     cb1                      up                    >>>    
        JDM mgmt port           jmgmt0                   up    
        JDM to HOST port        bme1                     up    
        JDM to GNF port         bme2                     up    
        JDM to JDM link0        cb0.4002                 down    Peer IP not reachable        
        JDM to JDM link1*       cb1.4002                 up      StrictKey peer SSH - OK        
        GNF Mac-Pool Prefix     Primary CB               OK      Prefix: JDM0[0x6e] / JDM1[0x6e]
        
        server1:
        --------------------------------------------------------------------------
        Component               Interface                Status  Comments
        Host to JDM port        virbr0                   up    
        Physical CB0 port       p1p1                     down                  >>>  
        Physical CB1 port       p2p1                     up                    >>>            
        Physical JDM mgmt port  em3                      up    
        Physical VNF mgmt port  em4                      up    
        JDM-GNF bridge          bridge_jdm_vm            up    
        CB0                     cb0                      down                  >>>  
        CB1                     cb1                      up                    >>>    
        JDM mgmt port           jmgmt0                   up    
        JDM to HOST port        bme1                     up    
        JDM to GNF port         bme2                     up    
        JDM to JDM link0        cb0.4002                 down    Peer IP not reachable        
        JDM to JDM link1*       cb1.4002                 up      StrictKey peer SSH - OK        
        GNF Mac-Pool Prefix     Primary CB               OK      Prefix: JDM1[0x6e] / JDM0[0x6e]
        
From JDM 0 cannot ssh to JDM 1 via CB1 and vice versa
        user@lab01-JDM0> start shell
        Feb 15 13:51:51
        % su
        Password:
        root@lab01-JDM0:/var/home/user# ip netns exec jdm_nv_ns ssh root@jdm-server1
        ssh: connect to host jdm-server1 port 22: Network is unreachable
        root@EPCR2-RCH01-JDM0:/var/home/user#
        
        user@lab01-JDM1> start shell
        Feb 15 13:52:48
        % su
        Password:
        root@lab01-JDM1:/var/home/user# ip netns exec jdm_nv_ns ssh root@jdm-server0
        ssh: connect to host jdm-server0 port 22: Network is unreachable
        root@lab01-JDM1:/var/home/user#
 

Solution

Fixed in PR1791981 and resolved in Junos 23.4R2, 24.1R1 and 24.2R1.

Modification History

Steps to disconnect CB ports on servers and SSH between JDMs servers are provided