This article provides an example configuration for EVPN with BGP confederation topology for customers who may want to scale their iBGP networks. The example covers both EVPN-MPLS and EVPN-VXLAN scenarios.
In the following test setup, the Border Gateway Protocol (BGP) confederation breaks AS 1 into sub-AS 65000, 65003, and 65004. The topology is commonly used for the EVPN-MPLS and EVPN-VXLAN scenarios.
+-------------+ +-------------+ | AS 65000 | ge-0/0/0 | AS 65000 | | R1 +------------+ R2 | | 1.0.0.1 | iBGP | 1.0.0.2 | +------+------+ +------+------+ | | eBGP eBGP ge-0/0/1 ge-0/0/2 | | +------+------+ +------+------+ | AS 65003 | | AS 65004 | | R3 | | R4 | | 1.0.0.3 | | 1.0.0.4 | +------+------+ +------+------+ | | ge-0/0/3 ge-0/0/4 | vlan10 | vlan10 +---+---+ +---+---+ | CE1 | | CE2 | +-------+ +-------+
The following configuration example can be used to set up the BGP confederation:
[R1]
set interfaces ge-0/0/0 unit 0 family inet address 10.0.0.1/24 set interfaces ge-0/0/0 unit 0 family mpls set interfaces ge-0/0/1 unit 0 family inet address 10.0.1.1/24 set interfaces ge-0/0/1 unit 0 family mpls set interfaces lo0 unit 0 family inet address 1.0.0.1/32 set routing-options router-id 1.0.0.1 set routing-options autonomous-system 65000 set routing-options confederation 1 set routing-options confederation members 65000 set routing-options confederation members 65003 set routing-options confederation members 65004 set protocols ldp interface ge-0/0/0.0 set protocols ldp interface ge-0/0/1.0 set protocols bgp group border local-address 1.0.0.1 set protocols bgp group border family evpn signaling set protocols bgp group border peer-as 65003 set protocols bgp group border neighbor 1.0.0.3 multihop ttl 2 set protocols bgp group sub type internal set protocols bgp group sub local-address 1.0.0.1 set protocols bgp group sub family evpn signaling set protocols bgp group sub neighbor 1.0.0.2 set protocols ospf traffic-engineering set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 set protocols ospf area 0.0.0.0 interface lo0.0 passive set protocols mpls interface ge-0/0/0.0 set protocols mpls interface ge-0/0/1.0
[R2]
set interfaces ge-0/0/0 unit 0 family inet address 10.0.0.2/24 set interfaces ge-0/0/0 unit 0 family mpls set interfaces ge-0/0/2 unit 0 family inet address 10.0.2.2/24 set interfaces ge-0/0/2 unit 0 family mpls set interfaces lo0 unit 0 family inet address 1.0.0.2/32 set routing-options router-id 1.0.0.2 set routing-options autonomous-system 65000 set routing-options confederation 1 set routing-options confederation members 65000 set routing-options confederation members 65003 set routing-options confederation members 65004 set protocols ldp interface ge-0/0/0.0 set protocols ldp interface ge-0/0/2.0 set protocols bgp group border local-address 1.0.0.2 set protocols bgp group border family evpn signaling set protocols bgp group border peer-as 65004 set protocols bgp group border neighbor 1.0.0.4 multihop ttl 2 set protocols bgp group sub type internal set protocols bgp group sub local-address 1.0.0.2 set protocols bgp group sub family evpn signaling set protocols bgp group sub neighbor 1.0.0.1 set protocols ospf traffic-engineering set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 set protocols ospf area 0.0.0.0 interface lo0.0 passive set protocols mpls interface ge-0/0/0.0 set protocols mpls interface ge-0/0/2.0
[R3]
set interfaces ge-0/0/1 unit 0 family inet address 10.0.1.3/24 set interfaces ge-0/0/1 unit 0 family mpls set interfaces ge-0/0/3 flexible-vlan-tagging set interfaces ge-0/0/3 encapsulation flexible-ethernet-services set interfaces ge-0/0/3 unit 10 encapsulation vlan-bridge set interfaces ge-0/0/3 unit 10 vlan-id 10 set interfaces ge-0/0/3 unit 10 family bridge set interfaces lo0 unit 0 family inet address 1.0.0.3/32 set routing-options router-id 1.0.0.3 set routing-options autonomous-system 65003 set routing-options confederation 1 set routing-options confederation members 65000 set routing-options confederation members 65003 set routing-options confederation members 65004 set protocols ldp interface ge-0/0/1.0 set protocols bgp group border local-address 1.0.0.3 set protocols bgp group border family evpn signaling set protocols bgp group border peer-as 65000 set protocols bgp group border neighbor 1.0.0.1 multihop ttl 2 set protocols ospf traffic-engineering set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 set protocols ospf area 0.0.0.0 interface lo0.0 passive set protocols mpls interface ge-0/0/1.0 set routing-instances evpn1 instance-type evpn set routing-instances evpn1 vlan-id 10 set routing-instances evpn1 interface ge-0/0/3.10 set routing-instances evpn1 route-distinguisher 1:3 set routing-instances evpn1 vrf-target target:1:1 set routing-instances evpn1 protocols evpn interface ge-0/0/3.10
[R4]
set interfaces ge-0/0/2 unit 0 family inet address 10.0.2.4/24 set interfaces ge-0/0/2 unit 0 family mpls set interfaces ge-0/0/4 flexible-vlan-tagging set interfaces ge-0/0/4 encapsulation flexible-ethernet-services set interfaces ge-0/0/4 unit 10 encapsulation vlan-bridge set interfaces ge-0/0/4 unit 10 vlan-id 10 set interfaces ge-0/0/4 unit 10 family bridge set interfaces lo0 unit 0 family inet address 1.0.0.4/32 set routing-options router-id 1.0.0.4 set routing-options autonomous-system 65004 set routing-options confederation 1 set routing-options confederation members 65000 set routing-options confederation members 65003 set routing-options confederation members 65004 set protocols ldp interface ge-0/0/2.0 set protocols bgp group border local-address 1.0.0.4 set protocols bgp group border family evpn signaling set protocols bgp group border peer-as 65000 set protocols bgp group border neighbor 1.0.0.2 multihop ttl 2 set protocols ospf traffic-engineering set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 set protocols ospf area 0.0.0.0 interface lo0.0 passive set protocols mpls interface ge-0/0/2.0 set routing-instances evpn1 instance-type evpn set routing-instances evpn1 vlan-id 10 set routing-instances evpn1 interface ge-0/0/4.10 set routing-instances evpn1 route-distinguisher 1:4 set routing-instances evpn1 vrf-target target:1:1 set routing-instances evpn1 protocols evpn interface ge-0/0/4.10
[CE1]
set interfaces ge-0/0/3 vlan-tagging set interfaces ge-0/0/3 unit 10 vlan-id 10 set interfaces ge-0/0/3 unit 10 family inet address 192.168.10.1/24
[CE2]
set interfaces ge-0/0/4 vlan-tagging set interfaces ge-0/0/4 unit 10 vlan-id 10 set interfaces ge-0/0/4 unit 10 family inet address 192.168.10.2/24
With the above configuration, the basic EVPN outputs are as follows:
PE(R3,R4)
lab@R3_re# run show evpn instance extensive Sep 17 16:36:08 Instance: __default_evpn__ Route Distinguisher: 1.0.0.3:0 Number of bridge domains: 0 Number of neighbors: 0 Instance: evpn1 Route Distinguisher: 1:3 VLAN ID: 10 Per-instance MAC route label: 300032 Duplicate MAC detection threshold: 5 Duplicate MAC detection window: 180 MAC database status Local Remote MAC advertisements: 1 1 MAC+IP advertisements: 1 1 Default gateway MAC advertisements: 0 0 Number of local interfaces: 2 (2 up) Interface name ESI Mode Status AC-Role .local..8 00:00:00:00:00:00:00:00:00:00 single-homed Up Root ge-0/0/3.10 00:00:00:00:00:00:00:00:00:00 single-homed Up Root Number of IRB interfaces: 0 (0 up) Number of protect interfaces: 0 Number of bridge domains: 1 VLAN Domain ID Intfs / up IRB intf Mode MAC sync IM route label IPv4 SG sync IPv4 IM core nexthop IPv6 SG sync IPv6 IM core nexthop 10 1 1 Extended Enabled 300096 Disabled Disabled Number of neighbors: 1 Address MAC MAC+IP AD IM ES Leaf-label 1.0.0.4 1 1 0 1 0 Number of ethernet segments: 0 SMET Forwarding: Disabled lab@R3_re# run show evpn database Sep 17 15:59:08 Instance: evpn1 VLAN DomainId MAC address Active source Timestamp IP address 10 00:05:86:71:96:03 ge-0/0/3.10 Sep 17 15:58:14 192.168.10.1 10 00:05:86:71:fe:04 1.0.0.4 Sep 17 15:58:14 192.168.10.2
Ping is reachable.
lab@CE2# run ping 192.168.10.1 rapid Sep 17 16:32:21 PING 192.168.10.1 (192.168.10.1): 56 data bytes !!!!! --- 192.168.10.1 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 6.746/9.648/17.030/3.735 ms
set interfaces ge-0/0/0 unit 0 family inet address 10.0.0.1/24 set interfaces ge-0/0/0 unit 0 family mpls set interfaces ge-0/0/1 unit 0 family inet address 10.0.1.1/24 set interfaces ge-0/0/1 unit 0 family mpls set interfaces lo0 unit 0 family inet address 1.0.0.1/32 set routing-options router-id 1.0.0.1 set routing-options autonomous-system 65000 set routing-options confederation 1 set routing-options confederation members 65000 set routing-options confederation members 65003 set routing-options confederation members 65004 set protocols bgp group border local-address 1.0.0.1 set protocols bgp group border family evpn signaling set protocols bgp group border peer-as 65003 set protocols bgp group border neighbor 1.0.0.3 multihop ttl 2 set protocols bgp group sub type internal set protocols bgp group sub local-address 1.0.0.1 set protocols bgp group sub family evpn signaling set protocols bgp group sub neighbor 1.0.0.2 set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 set protocols ospf area 0.0.0.0 interface lo0.0 passive
set interfaces ge-0/0/0 unit 0 family inet address 10.0.0.2/24 set interfaces ge-0/0/0 unit 0 family mpls set interfaces ge-0/0/2 unit 0 family inet address 10.0.2.2/24 set interfaces ge-0/0/2 unit 0 family mpls set interfaces lo0 unit 0 family inet address 1.0.0.2/32 set routing-options router-id 1.0.0.2 set routing-options autonomous-system 65000 set routing-options confederation 1 set routing-options confederation members 65000 set routing-options confederation members 65003 set routing-options confederation members 65004 set protocols bgp group border local-address 1.0.0.2 set protocols bgp group border family evpn signaling set protocols bgp group border peer-as 65004 set protocols bgp group border neighbor 1.0.0.4 multihop ttl 2 set protocols bgp group sub type internal set protocols bgp group sub local-address 1.0.0.2 set protocols bgp group sub family evpn signaling set protocols bgp group sub neighbor 1.0.0.1 set protocols ospf area 0.0.0.0 interface ge-0/0/0.0 set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 set protocols ospf area 0.0.0.0 interface lo0.0 passive
set interfaces ge-0/0/1 unit 0 family inet address 10.0.1.3/24 set interfaces ge-0/0/1 unit 0 family mpls set interfaces ge-0/0/3 flexible-vlan-tagging set interfaces ge-0/0/3 encapsulation flexible-ethernet-services set interfaces ge-0/0/3 unit 10 encapsulation vlan-bridge set interfaces ge-0/0/3 unit 10 vlan-id 10 set interfaces ge-0/0/3 unit 10 family bridge set interfaces lo0 unit 0 family inet address 1.0.0.3/32 set routing-instances evpn2 vtep-source-interface lo0.0 set routing-instances evpn2 instance-type virtual-switch set routing-instances evpn2 route-distinguisher 1:3 set routing-instances evpn2 vrf-target target:1:1 set routing-instances evpn2 protocols evpn encapsulation vxlan set routing-instances evpn2 protocols evpn extended-vni-list 10010 set routing-instances evpn2 protocols evpn multicast-mode ingress-replication set routing-instances evpn2 bridge-domains bd10 vlan-id 10 set routing-instances evpn2 bridge-domains bd10 interface ge-0/0/3.10 set routing-instances evpn2 bridge-domains bd10 vxlan vni 10010 set routing-instances evpn2 bridge-domains bd10 vxlan ingress-node-replication set routing-options router-id 1.0.0.3 set routing-options autonomous-system 65003 set routing-options confederation 1 set routing-options confederation members 65000 set routing-options confederation members 65003 set routing-options confederation members 65004 set protocols bgp group border local-address 1.0.0.3 set protocols bgp group border family evpn signaling set protocols bgp group border peer-as 65000 set protocols bgp group border neighbor 1.0.0.1 multihop ttl 2 set protocols ospf area 0.0.0.0 interface ge-0/0/1.0 set protocols ospf area 0.0.0.0 interface lo0.0 passive
set interfaces ge-0/0/2 unit 0 family inet address 10.0.2.4/24 set interfaces ge-0/0/2 unit 0 family mpls set interfaces ge-0/0/4 flexible-vlan-tagging set interfaces ge-0/0/4 encapsulation flexible-ethernet-services set interfaces ge-0/0/4 unit 10 encapsulation vlan-bridge set interfaces ge-0/0/4 unit 10 vlan-id 10 set interfaces ge-0/0/4 unit 10 family bridge set interfaces lo0 unit 0 family inet address 1.0.0.4/32 set routing-instances evpn2 vtep-source-interface lo0.0 set routing-instances evpn2 instance-type virtual-switch set routing-instances evpn2 route-distinguisher 1:4 set routing-instances evpn2 vrf-target target:1:1 set routing-instances evpn2 protocols evpn encapsulation vxlan set routing-instances evpn2 protocols evpn extended-vni-list 10010 set routing-instances evpn2 protocols evpn multicast-mode ingress-replication set routing-instances evpn2 bridge-domains bd10 vlan-id 10 set routing-instances evpn2 bridge-domains bd10 interface ge-0/0/4.10 set routing-instances evpn2 bridge-domains bd10 vxlan vni 10010 set routing-instances evpn2 bridge-domains bd10 vxlan ingress-node-replication set routing-options router-id 1.0.0.4 set routing-options autonomous-system 65004 set routing-options confederation 1 set routing-options confederation members 65000 set routing-options confederation members 65003 set routing-options confederation members 65004 set protocols bgp group border local-address 1.0.0.4 set protocols bgp group border family evpn signaling set protocols bgp group border peer-as 65000 set protocols bgp group border neighbor 1.0.0.2 multihop ttl 2 set protocols ospf area 0.0.0.0 interface ge-0/0/2.0 set protocols ospf area 0.0.0.0 interface lo0.0 passive
With the above configuration, the following basic EVPN outputs are obtained:
lab@R3_re# run show evpn instance extensive Sep 17 17:27:34 Instance: __default_evpn__ Route Distinguisher: 1.0.0.3:0 Number of bridge domains: 0 Number of neighbors: 0 Instance: evpn2 Route Distinguisher: 1:3 Encapsulation type: VXLAN Duplicate MAC detection threshold: 5 Duplicate MAC detection window: 180 MAC database status Local Remote MAC advertisements: 1 1 MAC+IP advertisements: 0 0 Default gateway MAC advertisements: 0 0 Number of local interfaces: 2 (2 up) Interface name ESI Mode Status AC-Role .local..9 00:00:00:00:00:00:00:00:00:00 single-homed Up Root ge-0/0/3.10 00:00:00:00:00:00:00:00:00:00 single-homed Up Root Number of IRB interfaces: 0 (0 up) Number of protect interfaces: 0 Number of bridge domains: 1 VLAN Domain ID Intfs / up IRB intf Mode MAC sync IM route label IPv4 SG sync IPv4 IM core nexthop IPv6 SG sync IPv6 IM core nexthop 10 10010 1 1 Extended Enabled 10010 Disabled Disabled Number of neighbors: 1 Address MAC MAC+IP AD IM ES Leaf-label 1.0.0.4 1 0 0 1 0 Number of ethernet segments: 0 Router-ID: 1.0.0.3 Source VTEP interface IP: 1.0.0.3 SMET Forwarding: Disabled lab@R3_re# run show evpn database Sep 17 17:27:40 Instance: evpn2 VLAN DomainId MAC address Active source Timestamp IP address 10010 00:05:86:71:96:03 ge-0/0/3.10 Sep 17 17:27:10 10010 00:05:86:71:fe:04 1.0.0.4 Sep 17 17:27:19 lab@R3_re# run show bridge mac-table Sep 17 17:28:40 MAC flags (S -static MAC, D -dynamic MAC, L -locally learned, C -Control MAC O -OVSDB MAC, SE -Statistics enabled, NM -Non configured MAC, R -Remote PE MAC, P -Pinned MAC) Routing instance : evpn2 Bridging domain : bd10, VLAN : 10 MAC MAC Logical Active address flags interface source 00:05:86:71:96:03 D ge-0/0/3.10 00:05:86:71:fe:04 D vtep.32769 1.0.0.4
lab@CE2# run ping 192.168.10.1 rapid Sep 17 17:27:23 PING 192.168.10.1 (192.168.10.1): 56 data bytes !!!!! --- 192.168.10.1 ping statistics --- 5 packets transmitted, 5 packets received, 0% packet loss round-trip min/avg/max/stddev = 6.286/7.668/12.342/2.341 ms