blog/ posts/ LACP configuration using iproute2
← Building an tiny GNS3 FRR linux appliance | posts | TLD for localhost services →

LACP port-channel or bonding configuration using Linux kernel and iproute2 tool.

Following cabling scheme is used in this networking scenario:

       CL1                                           S1
    +--------+                                   +--------+
    |        | eth0                        Gi0/0 |        |
    |        |-----------------------------------|        |
    |        |-----------------------------------|        |
    |        | eth1                        Gi0/1 |        |
    +--------+                                   +--------+

A Linux client called CL1 directly connected with 2 physical interfaces eth0 and eth1 to a IOS switch called S1 with interfaces Gi0/0 and Gi0/1. Interfaces have auto-negotiation enabled. Gigabit ethernet is negotiated 1Gbit/s. In this article only the Linux configuration is explained. The switch configuration is already working at the moment as CL1 is configured. The IOS switch port-channel configuration has been attached at the end of the article and is not explained further.

To make bonding available in Linux load the bonding kernel module with its options to make LACP available. The used module options are also in the Linux bonding kernel module documentation.

  • mandatory miimon option setting for a correct bonding configuration using Linux
  • bonding module mode=4 setting is LACP/IEEE 802.3ad
  • The LACP hello rates can be set either to slow or fast. LACP slow rate means checking for LACP neighbour every 30 seconds, fast rate means checking for LACP neighbour every 1 second. lacp_rate=slow or lacp_rate=fast.

Load the linux module using following options:

root # modprobe bonding mmiimon=100 mode=802.3ad lacp_rate=slow

[ 1122.146183] Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) [ 1122.146187] MII link monitoring set to 100 ms

Mandatory miimon dmesg entry, this dmesg message is generated if the module has been called without the explicit miimon setting.

[  414.850084] Warning: miimon must be specified, otherwise bonding will not detect link failure, speed and duplex which are essential for 802.3ad operation

Create of the bond0 logical bonding interface:

root # ip link add dev bond0 type bond

It is recommended to configure bonding using iproute2 . The preferred Linux way of configuring a logical interface.

Set all links to the state down:

ip link set dev eth0 down
ip link set dev eth1 down
ip link set dev bond0 down

Add interfaces eth0 and eth1 to a port aggregation interface bond0, and set it as master interface:

ip link set dev eth0 master bond0
ip link set dev eth1 master bond0

Put all the links into state up

ip link set dev eth0 up
ip link set dev eth1 up
ip link set dev bond0 up

The ifenslave control utility is considered obsolete. Adding physical links into a bond0 interface. Ensure interface is up

ip link set bond0 up

Create the bond0 interface:

ifenslave bond0 eth0 eth1

After the configuration, verify the linux dmesg output and look out for the messages that has been written there:

root # dmesg | egrep 'bond0|NIC|miimon'

[ 3.974363] e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX [ 414.850025] bonding: unknown parameter 'mmiimon' ignored [ 414.850084] Warning: miimon must be specified, otherwise bonding will not detect link failure, speed and duplex which are essential for 802.3ad operation [ 414.850085] Forcing miimon to 100msec [ 429.694603] bond0: eth0 is up - this may be due to an out of date ifenslave [ 429.700352] bond0: Enslaving eth1 as a backup interface with a down link [ 429.714381] e1000: eth1 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX [ 495.045593] IPv6: ADDRCONF(NETDEV_UP): bond0: link is not ready [ 495.045616] bond0: link status definitely up for interface eth1, 1000 Mbps full duplex [ 495.045618] bond0: now running without any active interface! [ 495.045841] IPv6: ADDRCONF(NETDEV_CHANGE): bond0: link becomes ready [ 498.104480] bond0: Enslaving eth0 as a backup interface with a down link [ 500.134424] e1000: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX [ 500.214021] bond0: link status definitely up for interface eth0, 1000 Mbps full duplex

At this point the linux bonding configuration is finished. And if all commands have worked successfully below verification commands are used on how to assure the configuration is sane. Verify the Linux client bonding interface configuration, look out for the Bonding Mode, Transmit Hash Policy and 'LACP rate' configuration setting.

root # cat /proc/net/bonding/bond0

Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011)   Bonding Mode: IEEE 802.3ad Dynamic link aggregation Transmit Hash Policy: layer2 (0) MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0  
802.3ad info LACP rate: slow Min links: 0 Aggregator selection policy (ad_select): stable System priority: 65535 System MAC address: 0c:43:80:44:d5:00 Active Aggregator Info: Aggregator ID: 2 Number of ports: 2 Actor Key: 9 Partner Key: 1 Partner Mac Address: aa:bb:cc:00:03:00   Slave Interface: eth0 MII Status: up Speed: 1000 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 0c:43:80:44:d5:00 Slave queue ID: 0 Aggregator ID: 2 Actor Churn State: none Partner Churn State: churned Actor Churned Count: 0 Partner Churned Count: 1 details actor lacp pdu: system priority: 65535 system mac address: 0c:43:80:44:d5:00 port key: 9 port priority: 255 port number: 1 port state: 63 details partner lacp pdu: system priority: 32768 system mac address: aa:bb:cc:00:03:00 oper key: 1 port priority: 32768 port number: 1 port state: 61   Slave Interface: eth1 MII Status: up Speed: 1000 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 0c:43:80:44:d5:01 Slave queue ID: 0 Aggregator ID: 2 Actor Churn State: none Partner Churn State: churned Actor Churned Count: 0 Partner Churned Count: 1 details actor lacp pdu: system priority: 65535 system mac address: 0c:43:80:44:d5:00 port key: 9 port priority: 255 port number: 2 port state: 63 details partner lacp pdu: system priority: 32768 system mac address: aa:bb:cc:00:03:00 oper key: 1 port priority: 32768 port number: 2 port state: 61

Verify the LACP neighborship from the switch using show lacp neighbor. Watch for the Flags setting and interface displayed participating in the port-channel.

S1#sh lacp neighbor

Flags:  S - Device is requesting Slow LACPDUs
        F - Device is requesting Fast LACPDUs
        A - Device is in Active mode       P - Device is in Passive mode

Channel group 1 neighbors

Partner's information:

                  LACP port                        Admin  Oper   Port    Port
Port      Flags   Priority  Dev ID          Age    key    Key    Number  State
Gi0/0     SA      255       0c43.8044.d500  11s    0x0    0x9    0x1     0x3F
Gi0/1     SA      255       0c43.8044.d500  12s    0x0    0x9    0x2     0x3F

IOS Switch S1 port-channel configuration:

configure terminal
!
interface Port-channel1
exit
!
interface GigabitEthernet0/0
 channel-group 1 mode active
exit
!
interface GigabitEthernet0/1
 channel-group 1 mode active
end
write

External wiki article about different link aggregation implementations and configurations options available.

Minor side note at the end. The linux configuration explained here will not persist among reboots. To configure permanent bonding refer to the linux bonding module documentation.