Description

Transit traffic cannot be sent to another layer-3 device, if it is physically connected on the RSNG or SNG. 

Setup:

NNG--Qfabric--RSNG1--ae44 (xe-0/0/44,1/0/44)======Layer-3-Switch(vlan-10; ip-address:-192.168.10.3)---Host 192.168.20.2 (Gateway on layer-3 switch)

There are a few subnets connected behind the Layer-3 Switch.

IRB configured on the Qfabric:

unit 10 {
     description "to layer-3 switch";
     family inet {
address 192.168.10.2/24;

    • Hosts connected on the layer-3 device cannot reach the qfabric vLAN-10 IP address and vice-versa
    • Hosts can only reach or ping to the layer-3 device.
    • Same status on the qfabric side and Qfabric can only ping to the layer-3 address but not to the hosts.

    Symptoms

    The following symptoms were observed:

    1. Ping connectivity between the end devices fail.
    2. In the test, ping works from qfabric to the layer-3 switch and from the host to the layer-3 switch.
    3. ARP entry shows as "hold" status on the qfabric NNG.
    4. ARP entry may show complete if checked from the RSNG and DG.
    Example

    Topology used for verification:
     
    [Qfabric]---------------------[Qfx5100] irb.10 and irb.20

    irb.10-192.168.10.3
    irb.20-192.168.20.2

    Ping to irb.10 on QFX5100 connected to Qfabric RSNG is successful:

    Ping from Qfabric:

    root@JTAC-QF02# run ping routing-instance JTAC 192.168.10.3
    64 bytes from 192.168.10.3: icmp_seq=0 ttl=64 time=20986 micro seconds
    64 bytes from 192.168.10.3: icmp_seq=1 ttl=64 time=21645 micro seconds

    +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

    Not able to ping irb.20 (192.168.20.2) on QFX5100:
    [edit]
    root@JTAC-QF02# run ping routing-instance JTAC 192.168.20.2
    error: from NW-NG-0: Read timed out

    ++++++++++++++++++++++++++++++++++++++++++++++
    QFX5100 output :
    root# run show interfaces terse irb
    Interface Admin Link Proto Local Remote
    irb up up
    irb.10 up up inet 192.168.10.3/24
    irb.20 up up inet 192.168.20.2/24

    +++++++++++++++++++++++++++++++++++++++++++++
    Qfabric NW-NG-0 Output:

    qfabric-admin@NW-NG-0> show route forwarding-table
    Routing table: JTAC---qfabric.inet
    Internet:
    Destination Type RtRef Next hop Type Index NhRef Netif
    default perm 0 rjct 2056 1
    192.168.10.3/32 dest 0 192.168.10.3       hold 2079 3 vlan.10
    192.168.20.0/24 user 0 192.168.10.3       hold 2079 3 vlan.20 >>>>>>>>||$ hold

    qfabric-admin@NW-NG-0> show route forwarding-table destination 192.168.10.3 extensive

    Routing table: JTAC---qfabric.inet [Index 6]
    Internet:

    Destination: 192.168.10.3/32
    Route type: destination
    Route reference: 0 Route interface-index: 65
    Multicast RPF nh index: 0
    Flags: sent to PFE
    Nexthop:
    Next-hop type: composite Index: 1961 Reference: 3
    Load Balance Label: Push 5, None
    Next-hop type: indirect Index: 131142 Reference: 2
    Next-hop type: Push 23, Push 45(top) Index: 1845 Reference: 6
    Load Balance Label: None
    Next-hop interface: dcfabric.0

    Destination: 192.168.10.3/32
    Route type: destination
    Route reference: 0 Route interface-index: 107
    Multicast RPF nh index: 0
    Flags: none
    Nexthop: 192.168.10.3
    Next-hop type: hold Index: 2000 Reference: 8
    Next-hop interface: vlan.10

    Solution

    This works as per design. In Qfabric, Layer-3 behind RSNG is NOT supported. Only NNG is responsible for L2/L3 network-facing protocols. SNG and RSNG handles host-facing protocols such as LACP,LLDP, ARP, DCBX, etc. 

     As a workaround, move the connection of the Layer-3 switch from RSNG to NNG.