This article covers a basic but complete BGPaaS configuration example from both the controller and virtual network function (VNF) perspective. In this example, the Contrail setup has three controllers that are working in High Availability mode.
To configure BGPaaS from the control node perspective, refer to BGP as a Service .
How BGPaaS works
When BGPaaS is configured in Contrail, the vRouter-agent listens for TCP/179 on the virtual network default-gateway and virtual DNS addresses. BGPaaS is configured as an eBGP session. The vRouter-agent then proxies (NAT) the TCP connection to the control node on behalf of the tenant virtual machine (VM). BGP peers are then established, and can exchange routes with one another. All the routes that are advertised from Contrail to VNF will have the next-hop set to the default-gateway IP address.
This workflow is illustrated in the following diagram:
contrail controllers compute node .................... ................................ . +-------+ . . +-------+ . . | | . . |vrouter| . . | | . . | | . . | | . . | | +-------+ . . | | . . | |.1 .10| VNF | . . |cont101+--------------+ +---------+ | . . | | . . | | | | . . | | . . | | +-------+ . . | |<-------------|<------|<-------- . . | | . . | NAT | . . +-------+ . . +-------+ . .................... ................................
Lab Setup
contrail controllers computes VNF ............................... ................. .............. . +-------+ . . +-------+ . . +-------+ . . | | XMPP BGP . . | |.1 . . | | . . |cont101+-------------------------+bcomp79+----------------+ vsrx | . . | | . . | |.2 . .10| | . . +-------+ BGP . ___.__+-------+ . . +-------+ . .172.18.101.101 __./ . 172.18.79.79 . . . . +-------+/XMPP . . . . . | | . . . . . . |cont103| . . . . . . | | . . . . . . +-------+_ XMPP . . . . .172.18.101.102 \_._ . . . . . +-------+ . \__.__+-------+ . . +-------+ . . | | . . | | . . | | . . |cont102+------------------------+bcomp80+-------- ---+ VM | . . | | XMPP . . | | . . | | . . +-------+ . . +-------+ . . +-------+ . . . . . . . ............................... ................. ..............
In the lab setup, there are:
3 controllers
2 compute nodes (only one will be used for this test)
1 VM running on each (vSRX image will be used for the VM.)
Configuration from the Contrail GUI
The following steps illustrate how to configure a BGPaaS service in Contrail GUI:
Configuration from VNF
The vSRX will be configured with two External Border Gateway Protocol (eBGP) neighbors:
4.4.4.1
4.4.4.2
The two sessions will be proxied by the compute node toward the two controllers that are also XMPP neighbors.
The configuration is as follows:
root> show configuration protocols bgp group baas { neighbor 4.4.4.1 { peer-as 60100; } neighbor 4.4.4.2 { peer-as 60100; } } root> show configuration interfaces ge-0/0/0 { unit 0 { family inet { address 4.4.4.10/24; } } }
Verification: BGP session from vSRX
root> show bgp summary Groups: 1 Peers: 2 Down peers: 0 Table Tot Paths Act Paths Suppressed History Damp State Pending inet.0 18 8 0 0 0 0 Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped... 4.4.4.1 60100 52298 57980 0 3 1d 23:47:33 0/9/9/0 0/0/0/0 4.4.4.2 60100 5749 6359 0 0 1d 23:49:35 8/9/9/0 0/0/0/0
root> show route protocol bgp inet.0: 10 destinations, 20 routes (10 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 4.4.4.3/32 *[BGP/170] 1d 23:54:11, MED 200, localpref 100, from 4.4.4.2 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 [BGP/170] 1d 23:52:09, MED 200, localpref 100 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 4.4.4.4/32 *[BGP/170] 1d 23:54:11, MED 100, localpref 100, from 4.4.4.2 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 [BGP/170] 1d 23:52:09, MED 100, localpref 100 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 4.4.4.5/32 *[BGP/170] 1d 23:54:11, MED 100, localpref 100, from 4.4.4.2 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 [BGP/170] 1d 23:52:09, MED 100, localpref 100 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 4.4.4.6/32 *[BGP/170] 1d 23:54:11, MED 100, localpref 100, from 4.4.4.2 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 [BGP/170] 1d 23:52:09, MED 100, localpref 100 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 4.4.4.7/32 *[BGP/170] 1d 23:54:11, MED 100, localpref 100, from 4.4.4.2 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 [BGP/170] 1d 23:52:09, MED 100, localpref 100 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 4.4.4.8/32 *[BGP/170] 1d 23:54:11, MED 100, localpref 100, from 4.4.4.2 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 [BGP/170] 1d 23:52:09, MED 100, localpref 100 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 4.4.4.9/32 *[BGP/170] 1d 23:54:11, MED 100, localpref 100, from 4.4.4.2 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 [BGP/170] 1d 23:52:09, MED 100, localpref 100 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 4.4.4.10/32 [BGP/170] 1d 23:52:09, MED 100, localpref 100 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 [BGP/170] 1d 23:54:11, MED 100, localpref 100, from 4.4.4.2 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 4.4.4.11/32 *[BGP/170] 1d 23:54:11, MED 100, localpref 100, from 4.4.4.2 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0 [BGP/170] 1d 23:52:09, MED 100, localpref 100 AS path: 60100 ? > to 4.4.4.1 via ge-0/0/0.0
Verification: BGP session from the compute node
curl http://127.0.0.1:8085/Snh_BgpAsAServiceSandeshReq <BgpAsAServiceSandeshResp type="sandesh"> <bgp_as_a_service_list type="list" identifier="1"> <list type="struct" size="1"> <BgpAsAServiceSandeshList> <vm_bgp_peer_ip type="string" identifier="1">4.4.4.10 <vm_nat_source_port type="i32" identifier="2">50003 <vmi_uuid type="string" identifier="3" link="ItfReq">76437055-f294-4640-ba8c-d9744c12d649 <is_shared type="bool" identifier="4">false <more type="bool" identifier="0">false
The information in the above XML output can be shown in table format as below:
+----------------+--------------------+--------------------------------------+-----------+ | vm_bgp_peer_ip | vm_nat_source_port | vmi_uuid | is_shared | +----------------+--------------------+--------------------------------------+-----------+ | 4.4.4.10 | 50003 | 76437055-f294-4640-ba8c-d9744c12d649 | false | +----------------+--------------------+--------------------------------------+-----------+
Verification: BGP session from the control node
curl http://127.0.0.1:8083/Snh_BgpNeighborReq?search_string=4.4.4.10 <BgpNeighborListResp type="sandesh"> <neighbors type="list" identifier="1"> <list type="struct" size="1"> <BgpNeighborResp> <instance_name type="string" identifier="53">default-domain:admin:pings-net:pings-net <peer type="string" identifier="1" link="BgpNeighborReq">76437055-f294-4640-ba8c-d9744c12d649 <deleted type="bool" identifier="36">false <peer_address type="string" identifier="2" link="BgpNeighborReq">4.4.4.10 <peer_id type="string" identifier="25">4.4.4.10 ...snipped... <more type="bool" identifier="0">false
The table-format XML output appears as below:
+--------------------------------------+--------------+----------+----------+-----------+-------------+------------+------------+-----------+ | peer | peer_address | peer_asn | encoding | peer_type | state | send_state | flap_count | flap_time | +--------------------------------------+--------------+----------+----------+-----------+-------------+------------+------------+-----------+ | 76437055-f294-4640-ba8c-d9744c12d649 | 4.4.4.10 | 60101 | BGP | external | Established | in sync | 0 | n/a | +--------------------------------------+--------------+----------+----------+-----------+-------------+------------+------------+-----------+
Flow table
Captures of the flow table entries for the two BGP sessions are as follows. Each flow represents one BGP session, but due to the NAT operation in the compute node, each flow will now have two "sections":
Facing VNF
Facing the controller
However, even though each flow has two entries in the flow table, both entries belong to the same flow and share the same flow index ID. Each of the two entries in a flow represents one section of the TCP connection.
Compute with VNF
root@bcomp79:~# flow --match "4.4.4.10 & 4.4.4.1:179" Index Source:Port/Destination:Port Proto(V) ----------------------------------------------------------------------------------- 247936<=>518924 4.4.4.10:49359 6 (5->0) 4.4.4.1:179 (Gen: 1, K(nh):88, Action:N(SPsD), Flags:, TCP:SSrEEr, QOS:-1, S(nh):0, Stats:13338/1013782, SPort 55945, TTL 255, Sinfo 20.0.0.0) root@bcomp79:~# flow --match "4.4.4.10 & 4.4.4.2:179" ...snipped... Index Source:Port/Destination:Port Proto(V) ----------------------------------------------------------------------------------- 203920<=>522980 4.4.4.10:61795 6 (5->0) 4.4.4.2:179 (Gen: 6, K(nh):88, Action:N(SPsD), Flags:, TCP:SSrEEr, QOS:-1, S(nh):0, Stats:13349/1014603, SPort 57130, TTL 255, Sinfo 20.0.0.0)
Compute with controller
root@bcomp79:~# flow --match "172.18.79.79:50003" ...snipped... Index Source:Port/Destination:Port Proto(V) ----------------------------------------------------------------------------------- 518924<=>247936 172.18.101.102:179 6 (0->5) 172.18.79.79:50003 (Gen: 4, K(nh):5, Action:N(SDPd), Flags:, TCP:SSrEEr, QOS:-1, S(nh):0, Stats:13373/816592, SPort 60113, TTL 2, Sinfo 0.0.0.0) 522980<=>203920 172.18.101.103:179 6 (0->5) 172.18.79.79:50003 (Gen: 5, K(nh):5, Action:N(SDPd), Flags:, TCP:SSrEEr, QOS:-1, S(nh):0, Stats:13382/817136, SPort 58348, TTL 2, Sinfo 0.0.0.0)
Summary
The following diagram summarizes the two sections of each flow:
DestinationNAT ---------------------------- A \ / v |index/reverseidx| VM vrouter | vrouter controller | | ---------------+-----------+-------------+-------------------+------------------| | 247936/518924 | 4.4.4.10:*-->4.4.4.1:179|172.18.79.79:50003-->172.18.101.102:179| | 203920/522980 | 4.4.4.10:*-->4.4.4.2:179|172.18.79.79:50003-->172.18.101.103:179| \ A v / ----------------------- sourceNAT+SourcePortNAT
From the diagram, you can see that three translations are performed in the compute node NAT operation:
Source IP translation (VNF random port > compute node port 50003)
Source port transition (VNF IP 4.4.4.10 > compute node fabric IP 172.18.79.79)
Destination IP translation (vRouter agent GW IP 4.4.4.1 > controller IP 172.18.101.102)
As with other flows, each Contrail flow has two directions:
Forward flow: Flow from the traffic sender to the traffic receiver
Reverse flow: The reverse direction of the forward flow, created automatically when a forward flow is triggered by traffic
The flow --get INDEX command will give details about flows in each direction.
flow --get INDEX
Forward flow: VNF toward controller: index 247936
root@bcomp79:~# flow --get 247936 Flow Index: 247936 Flow Generation ID: 1 Reverse Flow Index: 518924 VRF: 5 Destination VRF: 0 Flow Source: [4.4.4.10]:49359 Flow Destination: [4.4.4.1]:179 Flow Protocol: TCP Flow Action: NAT: SourceNAT, SourcePortNAT, DestinationNAT, NAT(Source, Destination): [172.18.79.79]:50003, [172.18.101.102]:179 Expected Source: Source Information: VRF: 5 Layer 3 Route Information Matching Route: 4.4.4.10/32 NextHop(Index, VRF, Type): 88, 5, ENCAP Ingress Interface(Index, VRF, OS): vif0/20, 5, tap76437055-f2 Interface Statistics(Out, In, Errors): 289105, 290715, 0 Layer 2 Route Information SourceMAC: 2:76:43:70:55:f2 NextHop(Index, VRF, Type): 87, 5, ENCAP Ingress Interface(Index, VRF, OS): vif0/20, 5, tap76437055-f2 Interface Statistics(Out, In, Errors): 289105, 290715, 0 Destination Information: VRF: 0 Layer 3 Route Information Matching Route: 0.0.0.0/0 NextHop(Index, VRF, Type): 10, 0, ENCAP Egress Interface(Index, VRF, OS): vif0/0, 0, bond0.2004 Interface Statistics(Out, In, Errors): 26388539, 25596681, 0 Flow Flags: TCP FLAGS: SYN, SYN(REVERSE), ESTABLISHED, ESTABLISHED(REVERSE), UDP Source Port: 55945 Flow Statistics: 13372/1016368 System Wide Packet Drops: 897848 Reverse Path Failures: 0
Reverse flow: Controller to VNF: index 518924
root@bcomp79:~# flow --get 518924 Flow Index: 518924 Flow Generation ID: 4 Reverse Flow Index: 247936 VRF: 0 Destination VRF: 5 Flow Source: [172.18.101.102]:179 Flow Destination: [172.18.79.79]:50003 Flow Protocol: TCP Flow Action: NAT: SourceNAT, DestinationNAT, DestinationPortNAT, NAT(Source, Destination): [4.4.4.1]:179, [4.4.4.10]:49359 Expected Source: Source Information: VRF: 0 Layer 3 Route Information Matching Route: 0.0.0.0/0 NextHop(Index, VRF, Type): 10, 0, ENCAP Ingress Interface(Index, VRF, OS): vif0/0, 0, bond0.2004 Interface Statistics(Out, In, Errors): 26532518, 25736453, 0 Destination Information: VRF: 5 Layer 3 Route Information Matching Route: 4.4.4.10/32 NextHop(Index, VRF, Type): 88, 5, ENCAP Egress Interface(Index, VRF, OS): vif0/20, 5, tap76437055-f2 Interface Statistics(Out, In, Errors): 291156, 292754, 0 Layer 2 Route Information DestinationMAC: 2:76:43:70:55:f2 NextHop(Index, VRF, Type): 87, 5, ENCAP Egress Interface(Index, VRF, OS): vif0/20, 5, tap76437055-f2 Interface Statistics(Out, In, Errors): 291156, 292754, 0 Flow Flags: TCP FLAGS: SYN, SYN(REVERSE), ESTABLISHED, ESTABLISHED(REVERSE), UDP Source Port: 60113 Flow Statistics: 13989/854191 System Wide Packet Drops: 903169 Reverse Path Failures: 0 Flow Block Drops: 42421