Description

This article outlines the behavior of Bidirectional Forwarding Detection (BFD) when used in aggregated Ethernet (ae) bundles that have child links in different Flexible PIC Concentrator (FPC) cards.

Symptoms

In case of aggregated Ethernet or IRB interfaces, where the member links are hosted by different FPCs, one FPC will be chosen as the FPC that is responsible for sending and receiving BFD packets. A firewall filter rule will be programmed on the other FPCs to redirect packets to the responsible FPC on the "pfh" interface.

The responsible FPC is not changed when the links from the respective FPC come out of the bundle.

Consider a topology where R1 and R2 are connected back to back.

R1 has an ae bundle towards R2, and the ae bundle in R1 has child links from three FPCs:

[edit]
lab@R1# show | match ae10 | display set
Dec 02 08:30:44
set interfaces xe-0/2/0 gigether-options 802.3ad ae10
set interfaces xe-1/2/0 gigether-options 802.3ad ae10
set interfaces xe-3/2/0 gigether-options 802.3ad ae10
set interfaces ae10 aggregated-ether-options lacp active
set interfaces ae10 unit 0 family inet address 172.19.1.1/24
set protocols ospf area 0.0.0.0 interface ae10.0 bfd-liveness-detection minimum-interval 1000
set protocols ospf area 0.0.0.0 interface ae10.0 bfd-liveness-detection multiplier 5

[edit]
lab@R1# run show interfaces ae10.0 extensive
Dec 02 08:30:50
Logical interface ae10.0 (Index 323) (SNMP ifIndex 6858) (Generation 241)
Flags: SNMP-Traps 0x4004000 Encapsulation: ENET2
Statistics Packets pps Bytes bps
Bundle:
Input : 16556 4 1675388 3720
Output: 5750 1 323076 416
Link:
xe-0/2/0.0
Input : 2239 1 268092 1312
Output: 263 0 23442 0
xe-1/2/0.0
Input : 4821 1 583472 992
Output: 2896 0 152110 0
xe-3/2/0.0
Input : 4353 2 373092 1416
Output: 2282 1 118706 416
LACP info: Role System System Port Port Port
priority identifier priority number key
xe-0/2/0.0 Actor 127 b0:a8:6e:36:9f:c0 127 3 11
xe-0/2/0.0 Partner 127 3c:94:d5:0c:07:c0 127 11 11
xe-1/2/0.0 Actor 127 b0:a8:6e:36:9f:c0 127 2 11
xe-1/2/0.0 Partner 127 3c:94:d5:0c:07:c0 127 10 11
xe-3/2/0.0 Actor 127 b0:a8:6e:36:9f:c0 127 4 11
xe-3/2/0.0 Partner 127 3c:94:d5:0c:07:c0 127 12 11
LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx
xe-0/2/0.0 2042 2029 0 0
xe-1/2/0.0 4640 4676 0 0
xe-3/2/0.0 2038 2032 0 0
Marker Statistics: Marker Rx Resp Tx Unknown Rx Illegal Rx
xe-0/2/0.0 0 0 0 0
xe-1/2/0.0 0 0 0 0
xe-3/2/0.0 0 0 0 0
Protocol inet, MTU: 1500, Generation: 342, Route table: 0
Flags: Sendbcast-pkt-to-re
Addresses, Flags: Is-Preferred Is-Primary
Destination: 172.19.1/24, Local: 172.19.1.1, Broadcast: 172.19.1.255, Generation: 185
Protocol multiservice, MTU: Unlimited, Generation: 343, Route table: 0
Flags: Is-Primary
Policer: Input: __default_arp_policer__

OSPF is running between R1 and R2 with BFD.  The uptime for OSPF and BPD is seen below:

[edit]
lab@R1# run show ospf neighbor detail
Dec 02 08:31:05
Address Interface State ID Pri Dead
172.19.1.2 ae10.0 Full 66.110.0.74 128 32
Area 0.0.0.0, opt 0x52, DR 172.19.1.1, BDR 172.19.1.2
Up 00:36:35, adjacent 00:35:59

[edit]
lab@R1# run show bfd session detail
Dec 02 08:31:15
Detect Transmit
Address State Interface Time Interval Multiplier
172.19.1.2 Up ae10.0 5.000 1.000 5
Client OSPF realm ospf-v2 Area 0.0.0.0, TX interval 1.000, RX interval 1.000
Session up time 00:36:39
Local diagnostic None, remote diagnostic None
Remote state Up, version 1

1 sessions, 1 clients
Cumulative transmit rate 1.0 pps, cumulative receive rate 1.0 pps

The filter that is programmed in the FPC can be seen from the FPC shell:

[edit]
lab@R1# run start shell pfe network fpc0


NPC platform (1067Mhz MPC 8548 processor, 2048MB memory, 512KB flash)

NPC0# show filter
Program Filters:
---------------
Index Dir Cnt Text Bss Name
-------- ------ ------ ------ ------ --------

Term Filters:
------------
Index Semantic Name
-------- ---------- ------
1 Classic __default_bpdu_filter__
17000 Classic __default_arp_policer__
57008 Classic __cfm_filter_shared_lc__
65280 Classic __auto_policer_template__
65281 Classic __auto_policer_template_1__
65282 Classic __auto_policer_template_2__
65283 Classic __auto_policer_template_3__
65284 Classic __auto_policer_template_4__
65285 Classic __auto_policer_template_5__
65286 Classic __auto_policer_template_6__
65287 Classic __auto_policer_template_7__
65288 Classic __auto_policer_template_8__
16777216 Classic fnp-filter-level-all
46137345 Classic HOSTBOUND_IPv4_FILTER
46137346 Classic HOSTBOUND_IPv6_FILTER
54525952 Classic ppmd-bfd-agg-inet4

Resolve Filters:
---------------
Index
--------

Looking at the detailed configuration of the filter named "ppmd-bfd-agg-inet4," the following is observed:

NPC0# show filter index 54525952 program
Filter index = 54525952
Optimization flag: 0xf7
Filter notify host id = 0
Filter properties: None
Filter state = CONSISTENT
term ppmd-172.19.1.2
term priority 0
destination-port
3784
false branch to match action in rule ppmd-default
source-address
172.19.1.2/32
false branch to match action in rule ppmd-default

then
action next-hop, type (nh-id)
968
term ppmd-default
term priority 0

then
accept

The next-hop index points to the responsible FPC on all three FPCs where the ae has child links:

NPC0# show nhdb id 968
ID Type Interface Next Hop Addr Protocol Encap MTU Flags PFE internal Flags
----- -------- ------------- --------------- ---------- ------------ ---- ---------- --------------------
968 Unicast pfh-1/0/0.16383 128.0.0.17 IPv4 Unspecified 65535 0x00000000 0x00000000


lab@R1# run start shell pfe network fpc3


NPC platform (1067Mhz MPC 8548 processor, 2048MB memory, 512KB flash)

NPC3# show filter index 54525952 program
Filter index = 54525952
Optimization flag: 0xf7
Filter notify host id = 0
Filter properties: None
Filter state = CONSISTENT
term ppmd-172.19.1.2
term priority 0
destination-port
3784
false branch to match action in rule ppmd-default
source-address
172.19.1.2/32
false branch to match action in rule ppmd-default

then
action next-hop, type (nh-id)
968
term ppmd-default
term priority 0

then
accept

NPC3# show nhdb id 968
ID Type Interface Next Hop Addr Protocol Encap MTU Flags PFE internal Flags
----- -------- ------------- --------------- ---------- ------------ ---- ---------- --------------------
968 Unicast pfh-1/0/0.16383 128.0.0.17 IPv4 Unspecified 65535 0x00000000 0x00000000

From the pfe shell of FPC1 it is observed the bfd packets are transmitted and received >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

NPC1( vty)# show ppm adjacencies protocol bfd

PPM Adjacency information for BFD

IFL-index Holdtime (msec) PPM handle Discr Absorbed Packets RX Packet Length Dist Do-dist

342 5000 10 20 459 462 0x532f8858 24 no-dist no-do-dist


NPC1( vty)# show ppm adjacencies protocol bfd

PPM Adjacency information for BFD

IFL-index Holdtime (msec) PPM handle Discr Absorbed Packets RX Packet Length Dist Do-dist

342 5000 10 20 475 478 0x532f8858 24 no-dist no-do-dist

Then, link xe-1/2/0 is removed from the bundle:

[edit]
lab@R1# show | compare
Dec 02 08:33:44
[edit interfaces]
- xe-1/2/0 {
- gigether-options {
- 802.3ad ae10;
- }
- }

[edit]
lab@R1# commit
Dec 02 08:33:47
commit complete


After the link comes out of the bundle, the ae link with OSPF and BFD is up and has not reset:

[edit]
lab@R1# run show interfaces terse ae10
Dec 02 08:34:06
Interface Admin Link Proto Local Remote
ae10 up up
ae10.0 up up inet 172.19.1.1/24
multiservice

[edit]
lab@R1# run show ospf neighbor detail
Dec 02 08:34:09
Address Interface State ID Pri Dead
172.19.1.2 ae10.0 Full 66.110.0.74 128 31
Area 0.0.0.0, opt 0x52, DR 172.19.1.1, BDR 172.19.1.2
Up 00:39:40, adjacent 00:39:04

[edit]
lab@R1# run show bfd session detail
Dec 02 08:34:14
Detect Transmit
Address State Interface Time Interval Multiplier
172.19.1.2 Up ae10.0 5.000 1.000 5
Client OSPF realm ospf-v2 Area 0.0.0.0, TX interval 1.000, RX interval 1.000
Session up time 00:39:39
Local diagnostic None, remote diagnostic None
Remote state Up, version 1

1 sessions, 1 clients
Cumulative transmit rate 1.0 pps, cumulative receive rate 1.0 pps

Even when the link from FPC 1 has come out of the bundle, the responsible FPC has not changed, which can be confirmed from the filter programming on the PFE:

NPC0# show filter index 54525952 program
Filter index = 54525952
Optimization flag: 0xf7
Filter notify host id = 0
Filter properties: None
Filter state = CONSISTENT
term ppmd-172.19.1.2
term priority 0
destination-port
3784
false branch to match action in rule ppmd-default
source-address
172.19.1.2/32
false branch to match action in rule ppmd-default

then
action next-hop, type (nh-id)
968
term ppmd-default
term priority 0

then
accept

NPC0# show nhdb id 968
ID Type Interface Next Hop Addr Protocol Encap MTU Flags PFE internal Flags
----- -------- ------------- --------------- ---------- ------------ ---- ---------- --------------------
968 Unicast pfh-1/0/0.16383 128.0.0.17 IPv4 Unspecified 65535 0x00000000 0x00000000

It is observed that the ppm adjacency can be still found on FPC1 and bfd packets are transmitted and received

NPC1( vty)# show ppm adjacencies protocol bfd

PPM Adjacency information for BFD

IFL-index Holdtime (msec) PPM handle Discr Absorbed Packets RX Packet Length Dist Do-dist

342 5000 10 20 5876 5879 0x532f8858 24 no-dist no-do-dist

NPC1( vty)# show ppm adjacencies protocol bfd

PPM Adjacency information for BFD

IFL-index Holdtime (msec) PPM handle Discr Absorbed Packets RX Packet Length Dist Do-dist

342 5000 10 20 5957 5960 0x532f8858 24 no-dist no-do-dist

Now, restarting fpc in slot 1 will cause a BFD flap, which will in turn cause a protocol flap:


lab@R1# run request chassis fpc slot 1 restart
Dec 02 08:35:21
Restart initiated, use "show chassis fpc" to verify

[edit]
lab@R1# run show chassis fpc
Dec 02 08:35:24
Temp CPU Utilization (%) Memory Utilization (%)
Slot State (C) Total Interrupt DRAM (MB) Heap Buffer
0 Online 39 7 0 2048 21 13
1 Offline ---Restarted by cli command---
2 Empty
3 Online 33 22 0 2048 12 13
4 Online 42 6 0 1024 7 41
5 Online 35 5 0 1024 19 29

[edit]
lab@R1# run show bfd session extensive
Dec 02 08:35:28
Detect Transmit
Address State Interface Time Interval Multiplier
172.19.1.2 Up ae10.0 5.000 1.000 5
Client OSPF realm ospf-v2 Area 0.0.0.0, TX interval 1.000, RX interval 1.000
Session up time 00:00:04, previous down time 00:00:01
Local diagnostic None, remote diagnostic None
Remote state Up, version 1
Min async interval 1.000, min slow interval 1.000
Adaptive async TX interval 1.000, RX interval 1.000
Local min TX interval 1.000, minimum RX interval 1.000, multiplier 5
Remote min TX interval 1.000, min RX interval 1.000, multiplier 5
Local discriminator 4, remote discriminator 21
Echo mode disabled/inactive
Remote is control-plane independent

1 sessions, 1 clients
Cumulative transmit rate 1.0 pps, cumulative receive rate 1.0 pps


The log reports:


Dec 2 08:35:23 R1 bfdd[1525]: BFDD_TRAP_SHOP_STATE_DOWN: local discriminator: 4, new state: down, interface: ae10.0, peer addr: 172.19.1.2
Dec 2 08:35:23 R1 rpd[1560]: RPD_OSPF_NBRDOWN: OSPF neighbor 172.19.1.2 (realm ospf-v2 ae10.0 area 0.0.0.0) state changed from Full to Down due to InActiveTimer (event reason: BFD session timed out and
neighbor was declared dead)
Dec 2 08:35:23 R1 rpd[1560]: RPD_OSPF_NBRUP: OSPF neighbor 172.19.1.2 (realm ospf-v2 ae10.0 area 0.0.0.0) state changed from Init to ExStart due to 2WayRcvd (event reason: neighbor detected this router)
Dec 2 08:35:23 R1 rpd[1560]: RPD_OSPF_NBRUP: OSPF neighbor 172.19.1.2 (realm ospf-v2 ae10.0 area 0.0.0.0) state changed from Exchange to Full due to ExchangeDone (event reason: DBD exchange of master
completed)
Dec 2 08:35:24 R1 bfdd[1525]: BFDD_TRAP_SHOP_STATE_UP: local discriminator: 4, new state: up, interface: ae10.0, peer addr: 172.19.1.2

Solution


This is a day one behavior and cannot be changed.  Customers are requested to check the FPC using the above process to ensure that it is not the responsible FPC before any event for a specific FPC is planned.

Related Information