In EVPN-VXLAN setup where the router is configured for a pure type-5 route sharing, and is only sharing a single route, a problem can occur where the peer will fail to create a next hop and decapsulation will not happen and the EVPN-VXLAN traffic will be dropped.
The traffic flowing through the EVPN-VXLAN is being dropped on the peer, and when checking the routes on the peer, the type-5 next-hop for the VRF is missing.
Topology:
irb inf. EVPN-VXLAN irb inf. CE ---------- mineralblue ------------------------ butterrum ------------ CE
Configuration for local router:
user@router1> show configuration routing-instances vrf instance-type vrf; interface irb.1; interface irb.2; interface irb.3; interface irb.4; interface irb.5; ## 'irb.5' is not defined interface irb.6; ## 'irb.6' is not defined route-distinguisher 1:50000; vrf-target target:1:50000; vrf-table-label; protocols { evpn { ip-prefix-routes { advertise direct-nexthop; <-- Pure T5 config, not T5+T2 gateway mode. encapsulation vxlan; vni 5001; } } }
user@router1> show route table vrf.inet vrf.inet.0: 9 destinations, 17 routes (9 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 192.168.5.0/24 *[EVPN/170] 02:07:03 > to 20.0.0.2 via xe-4/0/0.0
This is T5 route in FIB.
root@router1:/var/home/router1> show route forwarding-table ... 192.168.5.0/24 user 0 indr 1048611 2 comp 783 2
Configuration on Peer router:
user2@router2> show configuration routing-instances vrf instance-type vrf; interface irb.1; interface irb.2; interface irb.3; interface irb.4; interface irb.5; interface irb.6; ## 'irb.6' is not defined route-distinguisher 1:50001; vrf-target target:1:50000; vrf-table-label; protocols { evpn { ip-prefix-routes { advertise direct-nexthop; encapsulation vxlan; vni 5000; } } }
[edit] user2@router2# run show route table vrf.inet.0 vrf.inet.0: 11 destinations, 15 routes (11 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 192.168.0.0/16 *[Direct/0] 02:44:39 > via irb.1 [EVPN/170] 02:44:39 > to 10.0.0.1 via xe-1/1/10.0 192.168.0.1/32 *[Local/0] 02:44:39 Local via irb.1 192.168.2.0/24 *[Direct/0] 02:44:39 > via irb.2 [EVPN/170] 02:44:39 > to 10.0.0.1 via xe-1/1/10.0 192.168.2.1/32 *[Local/0] 02:44:39 Local via irb.2 192.168.3.0/24 *[Direct/0] 02:44:39 > via irb.3 [EVPN/170] 02:44:39 > to 10.0.0.1 via xe-1/1/10.0 192.168.3.1/32 *[Local/0] 02:44:39 Local via irb.3 192.168.4.0/24 *[Direct/0] 02:44:39 > via irb.4 [EVPN/170] 02:44:39 > to 10.0.0.1 via xe-1/1/10.0 192.168.4.1/32 *[Local/0] 02:44:39 Local via irb.4 192.168.5.0/24 *[Direct/0] 02:44:39 > via irb.5 <-- this subnet is not forwarding the traffic to local CE. 192.168.5.1/32 *[Local/0] 02:44:39 Local via irb.5 192.168.5.200/32 *[EVPN/7] 00:02:53 > via irb.5 <-- [edit] user2@router2# run show route table vrf.evpn.0 vrf.evpn.0: 25 destinations, 25 routes (25 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 5:1:50000::0::192.168.0.0::16/248 *[EVPN/170] 00:03:52 Indirect 5:1:50000::0::192.168.2.0::24/248 *[EVPN/170] 00:03:52 Indirect 5:1:50000::0::192.168.3.0::24/248 *[EVPN/170] 00:03:52 Indirect 5:1:50000::0::192.168.4.0::24/248 *[EVPN/170] 00:03:52 Indirect 5:1:50000::0::192.168.5.0::24/248 *[EVPN/170] 00:03:52 Indirect 5:1:5000023::0::192.168.0.0::16/248 *[BGP/170] 00:02:08, localpref 100, from 150.1.2.2 AS path: I, validation-state: unverified > to 20.0.0.1 via xe-1/1/8.0 5:1:5000023::0::192.168.2.0::24/248 *[BGP/170] 00:02:08, localpref 100, from 150.1.2.2 AS path: I, validation-state: unverified > to 20.0.0.1 via xe-1/1/8.0 5:1:5000023::0::192.168.3.0::24/248 *[BGP/170] 00:02:08, localpref 100, from 150.1.2.2 AS path: I, validation-state: unverified > to 20.0.0.1 via xe-1/1/8.0 5:1:5000023::0::192.168.4.0::24/248 *[BGP/170] 00:02:08, localpref 100, from 150.1.2.2 AS path: I, validation-state: unverified > to 20.0.0.1 via xe-1/1/8.0 5:1:5000027::0::192.168.0.0::16/248 *[BGP/170] 00:01:57, localpref 100, from 150.1.1.1 AS path: I, validation-state: unverified > to 10.0.0.1 via xe-1/1/10.0 5:1:5000027::0::192.168.2.0::24/248 *[BGP/170] 00:01:57, localpref 100, from 150.1.1.1 AS path: I, validation-state: unverified > to 10.0.0.1 via xe-1/1/10.0 5:1:5000027::0::192.168.3.0::24/248 *[BGP/170] 00:01:57, localpref 100, from 150.1.1.1 AS path: I, validation-state: unverified > to 10.0.0.1 via xe-1/1/10.0 5:1:5000027::0::192.168.4.0::24/248 *[BGP/170] 00:01:57, localpref 100, from 150.1.1.1 AS path: I, validation-state: unverified > to 10.0.0.1 via xe-1/1/10.0 5:1:50000::0::2001:1111:1111::::64/248 *[EVPN/170] 00:03:41 Indirect 5:1:50000::0::2001:2222:2222::::64/248 *[EVPN/170] 00:03:41 Indirect 5:1:50000::0::2001:3333:3333::::64/248 *[EVPN/170] 00:03:41 Indirect 5:1:50000::0::2001:4444:4444::::64/248 *[EVPN/170] 00:03:41 Indirect 5:1:5000023::0::2001:1111:1111::::64/248 *[BGP/170] 00:01:57, localpref 100, from 150.1.2.2 AS path: I, validation-state: unverified > to 20.0.0.1 via xe-1/1/8.0 5:1:5000023::0::2001:2222:2222::::64/248 *[BGP/170] 00:01:57, localpref 100, from 150.1.2.2 AS path: I, validation-state: unverified > to 20.0.0.1 via xe-1/1/8.0 5:1:5000023::0::2001:3333:3333::::64/248 *[BGP/170] 00:01:57, localpref 100, from 150.1.2.2 AS path: I, validation-state: unverified > to 20.0.0.1 via xe-1/1/8.0 5:1:5000023::0::2001:4444:4444::::64/248 *[BGP/170] 00:01:57, localpref 100, from 150.1.2.2 AS path: I, validation-state: unverified > to 20.0.0.1 via xe-1/1/8.0 5:1:5000027::0::2001:1111:1111::::64/248 *[BGP/170] 00:01:46, localpref 100, from 150.1.1.1 AS path: I, validation-state: unverified > to 10.0.0.1 via xe-1/1/10.0 5:1:5000027::0::2001:2222:2222::::64/248 *[BGP/170] 00:01:46, localpref 100, from 150.1.1.1 AS path: I, validation-state: unverified > to 10.0.0.1 via xe-1/1/10.0 5:1:5000027::0::2001:3333:3333::::64/248 *[BGP/170] 00:01:46, localpref 100, from 150.1.1.1 AS path: I, validation-state: unverified > to 10.0.0.1 via xe-1/1/10.0 5:1:5000027::0::2001:4444:4444::::64/248 *[BGP/170] 00:01:46, localpref 100, from 150.1.1.1 AS path: I, validation-state: unverified > to 10.0.0.1 via xe-1/1/10.0 [edit] user2@router2# the arp is there in arp table. user2@router2# [edit] user2@router2# run show arp vpn vrf MAC Address Address Name Interface Flags 00:00:00:00:01:01 192.168.0.2 host1.example.com irb.1 permanent published gateway 00:00:00:00:00:14 192.168.0.22 host2.example.com irb.1 [.local..8] permanent remote 00:00:00:00:01:01 192.168.2.100 192.168.2.100 irb.2 permanent published gateway 00:00:00:00:01:01 192.168.3.100 host3.example.com irb.3 permanent published gateway 00:00:00:00:01:01 192.168.4.100 192.168.4.100 irb.4 permanent published gateway 00:00:00:00:01:01 192.168.5.100 192.168.5.100 irb.5 permanent published gateway 00:00:00:00:00:04 192.168.5.104 192.168.5.104 irb.5 [xe-1/0/0.5] permanent remote <-- CE 00:00:00:00:00:64 192.168.5.200 192.168.5.200 irb.5 [xe-1/0/0.5] permanent remote Total entries: 8 [edit]
PR1386423 shows the issue for MX and QFX when they are configured as type-5 route peers, when only local router advertises routes, the peer might not install the decapsulated next-hop (NH) route. As a result, type-5 encapsulated traffic sent by the peer is dropped until the local router advertises any type-5 route.
As a workaround, configure a static route on the local router and advertise that route as a type-5 route to the peer.
[edit] user1@router1# show routing-instances vrf instance-type vrf; interface irb.1; interface irb.2; interface irb.3; interface irb.4; interface irb.5; ## 'irb.5' is not defined interface irb.6; ## 'irb.6' is not defined route-distinguisher 1:500002; vrf-target target:1:50000; vrf-table-label; routing-options { static { route 180.1.1.0/24 next-hop 192.168.0.50; <-- a dummy route is added to solve this limitation } } protocols { evpn { ip-prefix-routes { advertise direct-nexthop; encapsulation vxlan; vni 5001; } } } [edit] user1@router1#
2020-02-19: minor non-technical updates