Configuring a bonded NIC

Network Interface Card (NIC) bonding (also known as teaming) enables you to join two NICs as a single physical device so that they appear as one interface. This is usually performed to provide failover capabilities or load balancing. Bonding can be configured on the BMC Discovery hardware appliance. It is not something that you should consider on a virtual appliance, because failover and load balancing should be provided by the virtualization platform.


Warning

When changing network configuration on your appliance, you should always be able to access the system console in case the new configuration does not work correctly.

Netadmin does not work with bonded NICs

If you configure your appliance to use bonded NICs, you can no longer use the netadmin user to perform any networking configuration.

The process is to create a bonded NIC is to create a bonded (the master) interface and configure a number of slave interfaces for the master. The slave interfaces are those network interfaces that are connected to the physical network infrastructure. Network interface names on BMC Discovery appliances running on CentOS 7 might be something like ens160 or enp2s4, though this depends on the hardware platform. The BMC Discovery appliance running on CentOS 7 no longer relies on eth0 having an IP address. For more information, see BMC-Discovery-on-CentOS-7-compared-to-previous-versions. 


bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 1500
        inet 192.168.10.11  netmask 255.255.255.0  broadcast 192.168.10.255
        inet6 fe80::20c:29ff:fe6c:a2b8  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:6c:a2:b8  txqueuelen 1000  (Ethernet)
        RX packets 955  bytes 189248 (184.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 202  bytes 30720 (30.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens160: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether 00:0c:29:6c:a2:b8  txqueuelen 1000  (Ethernet)
        RX packets 647  bytes 110812 (108.2 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 211  bytes 33238 (32.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens35: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether 00:0c:29:6c:a2:b8  txqueuelen 1000  (Ethernet)
        RX packets 426  bytes 106933 (104.4 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1  bytes 60 (60.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

To create a bond point called bond0

To create a bond point called bond0 you need to:

  1. Configure the network interface scripts in /etc/sysconfig/network-scripts to force the hardware to bond two (or more) interfaces to bond0 using their MAC addresses. Take care with the "SLAVE=yes" and "MASTER=bond0" options in each file.
  2. Create the bond0 interface which is actually assigned the IP configuration options.
  3. Configure the boot process to include the bonding module and to use bond0 as the bond point, with the various options associated with that bond.

To do this, log in as the tideway user and perform the following steps:

  1. Stop the tideway, cluster, omniNames and appliance services. Enter:

    [tideway@localhost] $ tw_service_control --stop
    [tideway@localhost] $ sudo /bin/systemctl stop cluster
    [tideway@localhost] $ sudo /bin/systemctl stop omniNames

  2. Change to the /etc/sysconfig/network-scripts directory and determine the MAC addresses of the network cards that you want to bond. In the example below, there are 6 interfaces; ens160, ens33, ens34, ens35, ens36 and lo. The lo interface is the loopback interface and ignored for the purposes of this exercise. In each block of output pertaining to an interface the ether tag precedes the MAC address for the interface. Enter:

    [tideway@localhost ~]$ cd /etc/sysconfig/network-scripts
    [tideway@localhost network-scripts]$ /sbin/ifconfig -a
    ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            ether 00:0c:29:6c:a2:c2  txqueuelen 1000  (Ethernet)
            RX packets 445  bytes 111840 (109.2 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 15  bytes 1706 (1.6 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    ens34: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            ether 00:0c:29:6c:a2:cc  txqueuelen 1000  (Ethernet)
            RX packets 445  bytes 111840 (109.2 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 17  bytes 1886 (1.8 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    ens35: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            ether 00:0c:29:6c:a2:d6  txqueuelen 1000  (Ethernet)
            RX packets 445  bytes 111840 (109.2 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 16  bytes 1796 (1.7 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    ens36: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            ether 00:0c:29:6c:a2:e0  txqueuelen 1000  (Ethernet)
            RX packets 445  bytes 111840 (109.2 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 15  bytes 1706 (1.6 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            inet 192.168.10.11  netmask 255.255.255.0  broadcast 192.168.10.255
            inet6 fe80::20c:29ff:fe6c:a2b8  prefixlen 64  scopeid 0x20<link>
            ether 00:0c:29:6c:a2:b8  txqueuelen 1000  (Ethernet)
            RX packets 2483  bytes 311589 (304.2 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 1297  bytes 195523 (190.9 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
            inet 127.0.0.1  netmask 255.0.0.0
            inet6 ::1  prefixlen 128  scopeid 0x10<host>
            loop  txqueuelen 1000  (Local Loopback)
            RX packets 2451  bytes 328609 (320.9 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 2451  bytes 328609 (320.9 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    [tideway@localhost network-scripts]$

    In this example the MAC Addresses are as follows:

    • ens33  – 00:0c:29:6c:a2:c2
    • ens34  – 00:0c:29:6c:a2:cc
    • ens35  – 00:0c:29:6c:a2:d6
    • ens36  – 00:0c:29:6c:a2:e0
    • ens160 – 00:0c:29:6c:a2:b8

    The example shows that ens160 is the only interface with an IP address, 192.168.10.11. These steps are going to use ens160 and ens35 to create a bonded interface bond0, using the current IP configuration of ens160.

  3. Change to the root user and using a text editor, edit, or create and edit ifcfg-ens160 to contain the following:

    DEVICE=ens160
    BOOTPROTO=none
    ONBOOT=yes
    HWADDR=00:0c:29:6c:a2:b8 # The MAC address of ens160 as determined in the previous step.
    TYPE=Ethernet
    MASTER=bond0
    SLAVE=yes 

  4. Using a text editor, edit or create and edit ifcfg-ens35 to contain the following:

    DEVICE=ens35
    BOOTPROTO=none
    ONBOOT=yes
    HWADDR=00:0c:29:6c:a2:d6 # The MAC address of ens35 as determined in the previous step.
    TYPE=Ethernet
    MASTER=bond0
    SLAVE=yes 

  5. Using a text editor, edit or create and edit ifcfg-bond0 to contain the following. In the example below, IPADDR, NETMASK, and GATEWAY will be configured using the details from ens160 above. These should be completed as required.

    DEVICE=bond0
    BOOTPROTO=static
    ONBOOT=yes
    IPADDR=192.168.10.11
    NETMASK=255.255.255.0
    GATEWAY=192.168.10.1
    BONDING_MASTER=yes
    BONDING_OPTS="miimon=100 mode=active-backup"

    Options such as BROADCAST and NETWORK must be defined in ifcfg-bond0 if they are required. The bonding options, BONDING_OPTS, might not be required but the ones shown are good defaults. miimon=100 means check the link state every 100 milliseconds and mode=active-backup provides tolerance. There are other mode options available, some of which require changes to other items.

  6. Reboot the system.

To verify the creation of a bond point called bond0

  1. Log in as the tideway user and run the following command. The output should look like the following, note the bond0 interface and that bond0, ens160 and ens35 all report the MAC address of the currently active interface of the bond, ens160. Enter:

    [tideway@localhost ~]$ /sbin/ifconfig -a
    bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 1500
            inet 192.168.10.11  netmask 255.255.255.0  broadcast 192.168.10.255
            inet6 fe80::20c:29ff:fe6c:a2b8  prefixlen 64  scopeid 0x20<link>
            ether 00:0c:29:6c:a2:b8  txqueuelen 1000  (Ethernet)
            RX packets 236  bytes 30522 (29.8 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 168  bytes 24885 (24.3 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            ether 00:0c:29:6c:a2:c2  txqueuelen 1000  (Ethernet)
            RX packets 28  bytes 6384 (6.2 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 0  bytes 0 (0.0 B)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    ens34: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            ether 00:0c:29:6c:a2:cc  txqueuelen 1000  (Ethernet)
            RX packets 28  bytes 6384 (6.2 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 0  bytes 0 (0.0 B)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    ens35: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
            ether 00:0c:29:6c:a2:b8  txqueuelen 1000  (Ethernet)
            RX packets 28  bytes 6384 (6.2 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 0  bytes 0 (0.0 B)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    ens36: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
            ether 00:0c:29:6c:a2:e0  txqueuelen 1000  (Ethernet)
            RX packets 28  bytes 6384 (6.2 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 0  bytes 0 (0.0 B)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    ens160: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
            ether 00:0c:29:6c:a2:b8  txqueuelen 1000  (Ethernet)
            RX packets 212  bytes 24394 (23.8 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 175  bytes 27167 (26.5 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
            inet 127.0.0.1  netmask 255.0.0.0
            inet6 ::1  prefixlen 128  scopeid 0x10<host>
            loop  txqueuelen 1000  (Local Loopback)
            RX packets 2086  bytes 298048 (291.0 KiB)
            RX errors 0  dropped 0  overruns 0  frame 0
            TX packets 2086  bytes 298048 (291.0 KiB)
            TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

    [tideway@localhost ~]$

  2. As the root user, check /proc/net/bonding/bond0 to check for a successful bonding. Note that ens160 is reported as the currently active slave and only ens35 and ens160 show MAC addresses (HW addr) :

    [root@localhost ~]# cat /proc/net/bonding/bond0
    Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)

    Bonding Mode: fault-tolerance (active-backup)
    Primary Slave: None
    Currently Active Slave: ens160
    MII Status: up
    MII Polling Interval (ms): 100
    Up Delay (ms): 0
    Down Delay (ms): 0

    Slave Interface: ens160
    MII Status: up
    Speed: 10000 Mbps
    Duplex: full
    Link Failure Count: 0
    Permanent HW addr: 00:0c:29:6c:a2:b8
    Slave queue ID: 0

    Slave Interface: ens35
    MII Status: up
    Speed: 1000 Mbps
    Duplex: full
    Link Failure Count: 0
    Permanent HW addr: 00:0c:29:6c:a2:d6
    Slave queue ID: 0
    [root@localhost ~]#

    The list of running kernel modules should include the bonding module.

  3. Run the following command:

    [tideway@localhost ~]$ /sbin/lsmod | sort
    ablk_helper            13597  1 aesni_intel
    aesni_intel           189414  0
    ata_generic            12923  0
    ata_piix               35052  0
    *bonding               152656  0*
    cdrom                  42556  1 sr_mod
    coretemp               13444  0
    crc32c_intel           22094  1
    crc32_pclmul           13133  0
    ...

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*