Description

This article describes how to configure bond interfaces on Red Hat Enterprise Linux (RHEL)/CentOS 7.5.

Note: The configuration from subordinate switches EX/QFX are not shown in this article.

 

Symptoms

Link redundancy plays an important role in production networks on Contrail compute, and bond interfaces are used to ensure link redundancy. These bond interfaces could be tagged or untagged.

 

Solution

Perform the following:

  1. Check if the compute node (server) already supports NIC bonding. 

[root@localhost ~]#  lsmod | grep bond
[root@localhost ~]#

  1. Modprobe the bonding. ( Note: The bonding module is not loaded by default. You can load the module by issuing the following command as root .) 

[root@localhost ~]# modprobe --first-time bonding
[root@localhost ~]#

  1. Verify the kernel module for bonding (to display information about the module). 

root@localhost ~]# modinfo bonding
filename:       /lib/modules/3.10.0-1062.el7.x86_64/kernel/drivers/net/bonding/bonding.ko.xz
author:         Thomas Davis, [email protected] and many others
description:    Ethernet Channel Bonding Driver, v3.7.1
version:        3.7.1
license:        GPL

Configuring the Bond Interface

  1. Configure a bond interface.

To create a channel bonding interface, create a file in /etc/sysconfig/network-scripts/ . 

[root@localhost]# cd /etc/sysconfig/network-scripts/
[root@localhost network-scripts]# cat ifcfg-bond0
DEVICE=bond0
NAME=bond0
TYPE=Bond
BONDING_MASTER=yes
ONBOOT=yes
BOOTPROTO=none
BONDING_OPTS="mode=4 miion=100 lacp_rate=1"
NM_CONTROLLED=no
IPADDR=20.10.10.2
PREFIX=24

  1. Configure member interfaces. 

[root@localhost network-scripts]# cat ifcfg-ens1f0
DEVICE=ens1f0
HWADDR=90:e2:ba:cc:27:70
TYPE=ethernet
ONBOOT=yes
NM_CONTROLLED=no
MASTER=bond0
SLAVE=yes

[root@localhost network-scripts]# cat ifcfg-ens1f1
DEVICE=ens1f1
HWADDR=90:e2:ba:cc:27:71
TYPE=ethernet
ONBOOT=yes
NM_CONTROLLED=no
MASTER=bond0
SLAVE=yes

  1. Bring up the member interfaces and the bond interface will be up. 

[root@localhost network-scripts]# ifup ens1f0
[root@localhost network-scripts]# ifup ens1f1

  1. To verify that the bond interface is up, execute the following command: 

[root@localhost ~]#ip addr show
<clipped>​
31: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 90:e2:ba:cc:27:70 brd ff:ff:ff:ff:ff:ff
    inet 20.10.10.2/24 brd 20.10.10.255 scope global bond0
       valid_lft forever preferred_lft forever
    inet6 fe80::92e2:baff:fecc:2770/64 scope link 
       valid_lft forever preferred_lft forever

[root@localhost network-scripts]# ping 20.10.10.1
PING 20.10.10.1 (20.10.10.1) 56(84) bytes of data.
64 bytes from 20.10.10.1: icmp_seq=1 ttl=64 time=13.0 ms
64 bytes from 20.10.10.1: icmp_seq=2 ttl=64 time=0.776 ms
^C
--- 20.10.10.1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.776/6.893/13.011/6.118 ms

Creating a Tagged Bond Interface

  1. Create a vlan-tagged bond0 file. 

[root@localhost ~]# cat /etc/sysconfig/network-scripts/ifcfg-bond0.2004
DEVICE=bond0.2004
NAME=bond0.2004
BOOTPROTO=none
ONPARENT=yes
IPADDR=192.168.10.210
PREFIX=24
VLAN=yes
NM_CONTROLLED=no
[root@localhost ~]#

  1. Bring up the bond tagged interface with logical interfaces. 

[root@localhost ~]# ifup /etc/sysconfig/network-scripts/ifcfg-bond0.2004

  1. Verify that the tagged bond interface is up and communicating: 

[root@localhost ~]#ip addr show
<clipped>
33: bond0.2004@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 90:e2:ba:cc:27:70 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.210/24 brd 192.168.10.255 scope global bond0.2004
       valid_lft forever preferred_lft forever
    inet6 fe80::92e2:baff:fecc:2770/64 scope link 
       valid_lft forever preferred_lft forever

[root@localhost ~]# ping 192.168.10.133 (where this IP is the connected switch l3 IP)
PING 192.168.10.133 (192.168.10.133) 56(84) bytes of data.
64 bytes from 192.168.10.133: icmp_seq=1 ttl=64 time=0.383 ms
64 bytes from 192.168.10.133: icmp_seq=2 ttl=64 time=0.177 ms
^C
--- 192.168.10.133 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.177/0.280/0.383/0.103 ms

Verification Commands

  1. To check the IP address for the bond interfaces: 

[root@localhost ~]#ip address show 
<clipped>
4: ens1f0: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
    link/ether 90:e2:ba:cc:27:70 brd ff:ff:ff:ff:ff:ff
<clipped>
8: ens1f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
    link/ether 90:e2:ba:cc:27:70 brd ff:ff:ff:ff:ff:ff
<clipped>
30: fabric: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 90:e2:ba:cc:27:70 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::92e2:baff:fecc:2770/64 scope link 
       valid_lft forever preferred_lft forever
31: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000

    link/ether 90:e2:ba:cc:27:70 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::92e2:baff:fecc:2770/64 scope link 
       valid_lft forever preferred_lft forever

<clipped>
33: bond0.2004@bond0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master fabric state UP group default qlen 1000
    link/ether 90:e2:ba:cc:27:70 brd ff:ff:ff:ff:ff:ff
    inet 192.168.10.210/24 brd 192.168.10.255 scope global bond0.2004
       valid_lft forever preferred_lft forever
    inet6 fe80::92e2:baff:fecc:2770/64 scope link 
       valid_lft forever preferred_lft forever
[root@localhost ~]#

  1. To verify bonding status from the host (this command can also be used for verification and troubleshooting LACP bonding): 

[root@localhost ~]# cat /proc/net/bonding/bond0 
Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

Bonding Mode: IEEE 802.3ad Dynamic link aggregation
Transmit Hash Policy: layer2 (0)
MII Status: up
MII Polling Interval (ms): 100
Up Delay (ms): 0
Down Delay (ms): 0

802.3ad info
LACP rate: fast
Min links: 0
Aggregator selection policy (ad_select): stable
System priority: 65535
System MAC address: 90:e2:ba:cc:27:70
Active Aggregator Info:
        Aggregator ID: 9
        Number of ports: 2
        Actor Key: 15
        Partner Key: 10
        Partner Mac Address: 4c:16:fc:52:e4:00

Slave Interface: ens1f0
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 90:e2:ba:cc:27:70
Slave queue ID: 0
Aggregator ID: 9
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 90:e2:ba:cc:27:70
    port key: 15
    port priority: 255
    port number: 1
    port state: 63
details partner lacp pdu:
    system priority: 127
    system mac address: 4c:16:fc:52:e4:00
    oper key: 10
    port priority: 127
    port number: 3
    port state: 63

Slave Interface: ens1f1
MII Status: up
Speed: 10000 Mbps
Duplex: full
Link Failure Count: 0
Permanent HW addr: 90:e2:ba:cc:27:71
Slave queue ID: 0
Aggregator ID: 9
Actor Churn State: none
Partner Churn State: none
Actor Churned Count: 0
Partner Churned Count: 0
details actor lacp pdu:
    system priority: 65535
    system mac address: 90:e2:ba:cc:27:70
    port key: 15
    port priority: 255
    port number: 2
    port state: 63
details partner lacp pdu:
    system priority: 127
    system mac address: 4c:16:fc:52:e4:00
    oper key: 10
    port priority: 127
    port number: 2
    port state: 63
[root@localhost ~]#