On VC environment there users connected to specific FPC can have connectivity issues.
Topology in CFTS lab as follows;
QFX1 [lo0 – 15.1.1.1] -------[xe-0/2/0] EX4100-VC – irb103 (fpc6)--- IXIA
We have default route configured being exported to EX4100-VC.
{master:0}
root@EX4100-VC> show route
Feb 26 15:48:46
inet.0: 15 destinations, 15 routes (15 active, 0 holddown, 0 hidden)
Restart Complete
Limit/Threshold: 32768/32768 destinations
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[OSPF3/150] 00:01:23, metric 0, tag 0
> to 10.16.4.234 via xe-1/2/2.0
to 10.16.4.206 via xe-0/2/0.0
10.0.0.0/24 *[Static/5] 00:02:33
> to 10.85.156.1 via vme.0
10.16.4.206/31 *[Direct/0] 00:01:38
> via xe-0/2/0.0
10.16.4.207/32 *[Local/0] 00:01:38
Local via xe-0/2/0.0
10.16.4.234/31 *[Direct/0] 00:01:52
> via xe-1/2/2.0
10.16.4.235/32 *[Local/0] 00:01:52
Local via xe-1/2/2.0
10.33.45.1/32 *[Local/0] 00:02:33
Reject
10.33.46.1/32 *[Local/0] 00:02:33
10.33.47.1/32 *[Local/0] 00:02:33
10.33.48.0/24 *[Direct/0] 00:02:05
> via irb.103
10.33.48.1/32 *[Local/0] 00:02:05
Local via irb.103
root@EX4100-VC> show virtual-chassis
Feb 26 15:48:21
Preprovisioned Virtual Chassis
Virtual Chassis ID: 1251.3e22.203c
Virtual Chassis Mode: Enabled
Mstr Mixed Route Neighbor List
Member ID Status Serial No Model prio Role Mode Mode ID Interface
0 (FPC 0) Prsnt FA4822AN0006 ex4100-48p 129 Master* N VC 7 vcp-255/1/1
1 vcp-255/1/0
1 (FPC 1) Prsnt FA4922AN0060 ex4100-48p 129 Backup N VC 2 vcp-255/1/0
0 vcp-255/1/1
2 (FPC 2) Prsnt FA4922AN0070 ex4100-48p 0 Linecard N VC 3 vcp-255/1/0
1 vcp-255/1/1
3 (FPC 3) Prsnt FA4922AN0190 ex4100-48p 0 Linecard N VC 4 vcp-255/1/0
2 vcp-255/1/1
4 (FPC 4) Prsnt FB2722AN0008 ex4100-48t 0 Linecard N VC 5 vcp-255/1/0
3 vcp-255/1/1
5 (FPC 5) Prsnt FB2722AN0007 ex4100-48t 0 Linecard N VC 4 vcp-255/1/1
6 vcp-255/1/0
6 (FPC 6) Prsnt FA3523AN1135 ex4100-48p 0 Linecard N VC 7 vcp-255/1/0
5 vcp-255/1/1
7 (FPC 7) Prsnt FA3523AN0286 ex4100-48p 0 Linecard N VC 6 vcp-255/1/1
0 vcp-255/1/0
root@EX4100-VC>
We have clients connected through fpc6
root@EX4100-VC> show arp no-resolve | match 10.33
Feb 26 15:49:24
00:00:82:5e:b5:f7 10.33.48.5 irb.103 [ge-6/0/12.0] none
We have lo0 configured on QFX devices which we can use it as source.
root@QFX1> show interfaces terse lo0
Interface Admin Link Proto Local Remote
lo0 up up
lo0.0 up up inet 15.1.1.1 --> 0/0
lo0.16385 up up inet
root@QFX1>
When system was in good state, we are able to ping client ip from QFX with sourcing lo0 IP.
root@QFX1> ping 10.33.48.5 source 15.1.1.1
PING 10.33.48.5 (10.33.48.5): 56 data bytes
64 bytes from 10.33.48.5: icmp_seq=0 ttl=63 time=0.836 ms
64 bytes from 10.33.48.5: icmp_seq=1 ttl=63 time=0.760 ms
^C
--- 10.33.48.5 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.760/0.798/0.836/0.038 ms
root@EX4100-VC> start shell
Feb 26 15:51:05
root@EX4100-VC:RE:0% cprod -A fpc0 -c 'set dcb bc "l3 defip show"' | egrep 0.0.0
2689 2 128.0.0.0/2 00:00:00:00:00:00 108194 0 0 0 1 n
3584 1 0.0.0.0/0 00:00:00:00:00:00 200010 0 0 0 0 n (ECMP)
3584 2 0.0.0.0/0 00:00:00:00:00:00 108196 0 0 0 0 n
3585 3 0.0.0.0/0 00:00:00:00:00:00 108196 0 0 0 0 n
3585 4 0.0.0.0/0 00:00:00:00:00:00 108196 0 0 0 0 n
7168 Global 224.0.0.0/4 00:00:00:00:00:00 0 0 0 0 0 n
root@EX4100-VC:RE:0% cprod -A fpc1 -c 'set dcb bc "l3 defip show"' | egrep 0.0.0
root@EX4100-VC:RE:0% cprod -A fpc6 -c 'set dcb bc "l3 defip show"' | egrep 0.0.0
3584 1 0.0.0.0/0 00:00:00:00:00:00 200010 0 0 0 0 y (ECMP)
root@EX4100-VC:RE:0%
If we reboot fpc6 , we see the client is no longer reachable from QFX.
root@EX4100-VC> request system reboot member 6 at now
Feb 26 15:52:06
Reboot the system at now? [yes,no] (no) yes
Feb 26 15:52:07
Rebooting fpc6
root@EX4100-VC> show chassis fpc
Feb 26 15:52:32
Temp CPU Utilization (%) CPU Utilization (%) Memory Utilization (%)
Slot State (C) Total Interrupt 1min 5min 15min DRAM (MB) Heap Buffer
0 Online 50 18 0 19 16 8 4041 35 55
1 Online 50 16 0 14 11 5 4041 35 48
2 Online 49 12 0 12 10 5 4041 35 49
3 Online 49 17 0 13 10 5 4041 35 48
4 Online 49 9 0 10 9 4 4041 35 48
5 Online 47 11 0 11 9 4 4041 35 50
6 Empty
7 Online 44 15 0 12 10 5 4041 35 48
8 Empty
9 Empty
Post reboot of fpc6 I see discrepancy in route and pings does not work.
Feb 26 15:57:07
0 Online 51 17 0 22 18 11 4041 35 55
1 Online 51 16 0 15 13 8 4041 35 48
2 Online 50 14 0 13 11 7 4041 35 49
3 Online 50 10 0 12 11 6 4041 35 48
4 Online 46 11 0 11 10 6 4041 35 48
5 Online 48 10 0 10 10 6 4041 35 50
6 Online 40 50 0 18 11 5 4041 35 49
7 Online 45 13 0 12 11 6 4041 35 48
root@EX4100-VC> show arp no-resolve | match 10.33.
Feb 26 15:57:14
Feb 26 15:57:41
4 packets transmitted, 0 packets received, 100% packet loss
Feb 26 15:58:07
^[[A2689 2 128.0.0.0/2 00:00:00:00:00:00 108194 0 0 0 1 n
1792 2 128.0.0.0/2 00:00:00:00:00:00 108194 0 0 0 1 n
3584 1 0.0.0.0/0 00:00:00:00:00:00 108196 0 0 0 0 y
To recover we can use below;
Fixed in later releases