Linux: add LAG support and bonding plugin for non-LAG cases (#1624)

docs/caveats.md

@@ -281,6 +281,9 @@ We're not testing Fortinet implementation as part of the regular integration tes
 * Junos cannot have more than one loopback interface per routing instance. Using **loopback** links on Junos devices will result in configuration errors.
 * Junos configuration template configures BFD timers within routing protocol configuration, not on individual interfaces
 
+## Linux caveats
+See [](generic-linux-devices)
+
 (caveats-vptx)=
 ## Juniper vPTX
 

docs/labs/linux.md

@@ -1,3 +1,4 @@
+(generic-linux-devices)=
 # Generic Linux Devices
 
 You can run Linux hosts or routers in virtual machines or containers. The default image used for a Linux virtual machine is Ubuntu 20.04, the default container image is Python 3.9 container running on Alpine Linux.
@@ -144,7 +145,7 @@ _netlab_ initial configuration script will skip Ubuntu package installation if i
 The initial configuration process (**[netlab initial](../netlab/initial.md)**) does not rely on commands executed within Linux containers:
 
 * The `/etc/hosts` file is generated during the **[netlab create](../netlab/create.md)** process from the ```templates/provider/clab/frr/hosts.j2``` template (see [](clab-config-template)).
-* Interface IP addresses and static routes to the default gateway (see [](linux-routes)) are configured with **ip** commands executed on the Linux host but within the container network namespace.
+* Interface IP addresses, static routes to the default gateway (see [](linux-routes)) and any lag bonding interfaces are configured with **ip** commands executed on the Linux host but within the container network namespace.
 * Static default route points to the management interface.
 
 You can, therefore, use any container image as a Linux node.

docs/module/lag.md

@@ -12,8 +12,9 @@ LAG is currently supported on these platforms:
 | Aruba AOS-CX          | ✅ | ✅ | ✅ | ✅ |
 | Cumulus Linux 4.x     | ✅ | ✅ | ❌  | ❌ |
 | Cumulus 5.x (NVUE)    | ✅ | ✅ | ❌  | ❌ |
-| Dell OS10             | ✅ | ✅ | ✅ | ✅ |
+| Dell OS10             | ✅ | ✅ | ✅  | ✅ |
 | FRR                   | ✅ | ✅ | ❌  | ❌ |
+| Generic Linux hosts   | ✅ | ✅ | ❌  | ❌ |
 
 ## Parameters
 
@@ -32,7 +33,7 @@ The following parameters can be set on individual links:
 
 * **lag.members**: Mandatory list of links that form the LAG. It uses the [same format as the topology **links** list](link-formats).
 * **lag.ifindex**: Optional parameter that controls the naming of the LAG (bonding, port-channel) interface.
-* **lag.mlag**: Optional Boolean or dict with peer link parameters; see [below](mlag)
+* **lag.mlag**: Optional dict with peer link parameters; see [below](mlag)
 
 This configuration module creates a virtual link with the link type set to **lag** between the **lag.members** and appends the links described in the **lag.members** list to the topology **links** list.
 
@@ -66,6 +67,10 @@ links:
      ifindex: 50
 ```
 
+### Caveat: Multi-provider Labs
+
+There is a known issue with multi-provider labs, where 'lag' type links get converted into 'lan'; this breaks the lag module
+
 (mlag)=
 ## Multi-chassis Link Aggregation (MLAG)
 

docs/plugins.md

@@ -9,6 +9,7 @@
    plugins/bgp.domain.md
    plugins/bgp.session.md
    plugins/bgp.policy.md
+   plugins/bonding.md
    plugins/ebgp.multihop.md
    plugins/bgp.originate.md
    plugins/check.config.md

docs/plugins/bonding.md

@@ -0,0 +1,71 @@
+(plugin-bonding)=
+# Host-side Link Bonding
+
+Linux networking has long supported *bonding*, the ability to use multiple links simultaneously. Netlab supports bonding with LACP through the *lag* module,
+this plugin adds support for the other bonding modes (that don't require any special configuration on peers)
+
+```eval_rst
+.. contents:: Table of Contents
+   :depth: 2
+   :local:
+   :backlinks: none
+```
+
+## Using the Plugin
+
+* Add `plugin: [ bonding ]` to the lab topology.
+* Include the **bonding.ifindex** attribute in any links that need to be bonded
+
+### Supported attributes
+
+The plugin adds the following attributes defined at global, node or interface level:
+* **bonding.mode** (string, one of active-backup, balance-tlb, or balance-alb) -- the bonding mode to use, default `active-backup`
+
+Additional interface level attributes:
+* **bonding.ifindex** (int,mandatory) -- the interface index for the bonding device; links with matching ifindex are bonded together
+* **bonding.primary** (bool) -- optional flag to mark this interface as primary, default *False*. If none of the interfaces are marked as `primary`, the selection is left to the Linux default behavior
+
+### Caveats
+
+The plugin uses the `ip` command to create bond devices and add member links; in case of Linux VMs that are not Ubuntu, the plugin attempts to install this command when not available.
+This installation uses `apt-get` which may not work on some Linux VMs
+
+## Examples
+
+(active-backup-bonding)=
+### Connect a host to a pair of switches using active-backup bonding
+
+```yaml
+plugin: [ bonding ]
+
+bonding.mode: active-backup  # Default
+
+vlans:
+  v1:
+
+groups:
+  _auto_create: True
+  hosts:
+    members: [ h1 ]
+    device: linux
+  switches:
+    members: [ s1, s2 ]
+    module: [ vlan ]
+
+links:
+- s1:
+  s2:
+  vlan.trunk: [ v1 ]
+
+# Bonded interfaces eth1/eth2
+- s1:
+  h1:
+    bonding.ifindex: 1
+- s2:
+  h1:
+    bonding:
+      ifindex: 1
+      primary: True  # Use this interface as primary
+```
+
+Note how there are no bonding specific modules enabled on the switches

netsim/ansible/templates/initial/frr.j2

@@ -84,7 +84,7 @@ fi
 
 # Disable IPv6 (for IPv4-only interfaces) or SLAAC (if the device is a router)
 #
-{% for l in interfaces if l.type in ['lan','p2p','stub'] %}
+{% for l in interfaces if l.type in ['lan','p2p','stub','lag','bond'] %}
 {%   if l.ipv6 is not defined %}
 sysctl -qw net.ipv6.conf.{{ l.ifname }}.disable_ipv6=1
 {%   elif role|default('router') != 'host' %}
@@ -100,7 +100,7 @@ ip link set {{ l.ifname }} up
 echo "service integrated-vtysh-config" >/etc/frr/vtysh.conf
 #
 # Set Ethernet interface MTU
-{% for l in interfaces if l.mtu is defined and l.get('type',"")!='lag' %}
+{% for l in interfaces if l.mtu is defined and l.type!='lag' %}
 ip link set {{ l.ifname }} mtu {{ l.mtu }}
 {% endfor %}
 

netsim/ansible/templates/initial/linux/create-bond.j2

@@ -0,0 +1,36 @@
+#
+# This macro is used by the initial script to create bond devices
+# (which needs to be done early such that IP addresses can be assigned and/or VLAN interfaces created)
+#
+{% macro create_bond_dev(l,node_provider) %}
+{%   if l.type in ['lag','bond'] %}
+{%     set _m = l.lag.mode|default(l.bonding.mode|default("802.3ad")) %}
+{%     if _m=="802.3ad" %}
+{%       set _lacp = l.lag.lacp|default('fast') %}
+{%       set lacp_act = 'off' if _lacp=='off' else 'on' %}
+{%       set lacp_rate = (' lacp_rate ' + _lacp) if _lacp!='off' else '' %}
+{%       set _m = _m + lacp_rate %}
+{%       if node_provider == 'clab' %}
+{%         set _m = _m + " lacp_active " + lacp_act %}
+{%       endif %}
+{%     elif l.bonding.primary is defined %}
+{%       set _m = _m + " primary " + l.bonding.primary %}
+{%     endif %}
+{%     if _m in ["802.3ad","balance-xor","balance-alb","balance-tlb"] %}
+{%       set _m = _m + " xmit_hash_policy encap3+4" %}
+{%     endif -%}
+
+{%     if node_provider!='clab' %}
+#
+# Make sure 'bonding' module is loaded
+#
+if [ ! -e /sys/module/bonding ]; then
+modprobe bonding miimon=100 mode=802.3ad lacp_rate=fast
+fi
+{%     endif -%}
+
+if [ ! -e /sys/class/net/{{l.ifname}} ]; then
+ip link add dev {{l.ifname}} type bond mode {{ _m }}
+fi
+{%   endif %}
+{% endmacro -%}

netsim/ansible/templates/initial/linux/ubuntu.j2

@@ -64,6 +64,7 @@ if systemctl is-active --quiet lldpd.service; then
 else
   if "$NEED_APT_UPDATE"; then
     apt-get update -qq
+    NEED_APT_UPDATE=
   fi
   apt-get install -qq lldpd
 fi
@@ -114,29 +115,65 @@ network:
 SCRIPT
 {% endif %}
 
-# Interface addressing
-{% for l in interfaces|default([]) if (l.ipv4 is defined or l.ipv6 is defined or l.dhcp is defined)%}
+# Interface addressing and bonds
+{% for l in interfaces|default([]) if (l.ipv4 is defined or l.ipv6 is defined or l.dhcp is defined or l.type in ['lag','bond'])%}
 cat <<SCRIPT > /etc/netplan/03-eth-{{ l.ifname }}.yaml
 network:
   version: 2
   renderer: networkd
   ethernets:
+{%   if l.type in ['lag','bond'] %}
+{%     if l.type=='lag' %}
+{%       for i in interfaces if i.lag._parentindex|default(0)==l.linkindex %}
+    {{ i.ifname }}:
+      dhcp4: no
+{%       endfor %}
+{%     elif l.bonding.members is defined %}
+{%       for m in l.bonding.members %}
+    {{ m }}:
+      dhcp4: no
+{%       endfor %}
+{%     endif %}
+  bonds:
     {{ l.ifname }}:
+      interfaces:
+{%     if l.type=='lag' %}
+{%       for i in interfaces if i.lag._parentindex|default(0)==l.linkindex %}
+        - {{ i.ifname }}
+{%       endfor %}
+{%     elif l.bonding.members is defined %}
+{%       for m in l.bonding.members %}
+        - {{ m }}
+{%       endfor %}
+{%     endif %}
+{%     set _m = l.lag.mode|default(l.bonding.mode|default("802.3ad")) %}
+      parameters:
+        mode: {{ _m }}
+{%     if _m=='802.3ad' %}
+        lacp-rate: {{ l.lag.lacp|default('fast') }}
+{%     elif l.bonding.primary is defined %}
+        primary: {{ l.bonding.primary }}
+{%     endif %}
+        mii-monitor-interval: 100
+        transmit-hash-policy: encap3+4
+{%   else %}
+    {{ l.ifname }}:
+{%     if l.mtu is defined %}
+      mtu: {{ l.mtu }}
+{%     endif %}
+{%   endif %}
 {%   if l.dhcp.client.ipv4|default(False) %}
       dhcp4: true
 {%   endif %}
 {%   if l.dhcp.client.ipv6|default(False) %}
       dhcp6: true
 {%   endif %}
-{%   for af in ('ipv4','ipv6') if af in l %}
+{%   for af in ('ipv4','ipv6') if af in l and l[af] is string %}
 {%     if loop.first %}
       addresses:
 {%     endif %}
         - {{ l[af] }}
 {%   endfor %}
-{%   if l.mtu is defined %}
-      mtu: {{ l.mtu }}
-{%   endif %}
 SCRIPT
 {% endfor %}
 

netsim/ansible/templates/initial/linux/vanilla.j2

@@ -1,3 +1,5 @@
+{% from "initial/linux/create-bond.j2" import create_bond_dev %}
+
 ### One-Shot configuration (non-Ubuntu VM or container)
 #
 # Disable IPv4 and IPv6 forwarding
@@ -24,24 +26,27 @@ ip -6 addr add {{ loopback.ipv6 }} dev lo
 {%   endif %}
 {% endif %}
 #
-# Interface addressing
+# Interface addressing, create any bond devices
 #
 {% for l in interfaces|default([]) %}
+{%   if l.type in ['lag','bond'] %}
+{{     create_bond_dev(l,node_provider) }}
+{%   endif %}
 ip link set dev {{ l.ifname }} up
-{% if l.ipv4 is defined %}
+{% if l.ipv4 is defined and l.ipv4 is string %}
 set +e
 ip addr del {{ l.ipv4 }} dev {{ l.ifname }} 2>/dev/null
 set -e
 ip addr add {{ l.ipv4 }} dev {{ l.ifname }}
 {% endif %}
-{% if l.ipv6 is defined %}
+{% if l.ipv6 is defined and l.ipv6 is string %}
 sysctl -w net.ipv6.conf.{{ l.ifname }}.disable_ipv6=0
 set +e
 ip -6 addr del {{ l.ipv6 }} dev {{ l.ifname }} 2>/dev/null
 set -e
 ip -6 addr add {{ l.ipv6 }} dev {{ l.ifname }}
 {% endif %}
-{% if l.mtu is defined %}
+{% if l.mtu is defined and l.type != 'lag' %}
 ip link set {{ l.ifname }} mtu {{ l.mtu }}
 {% endif %}
 {% endfor %}

netsim/ansible/templates/lag/frr.j2

@@ -1,42 +1 @@
-#!/bin/bash
-#
-set -e # Exit immediately when any command fails
-#
-{% if node_provider != 'clab' %}
-modprobe bonding miimon=100 mode=802.3ad lacp_rate=fast
-{% endif %}
-#
-# Create bonds for LAGs, if any. Requires kernel bonding module loaded
-#
-{% for l in interfaces if 'lag' in l %}
-{%  if l.type=='lag' %}
-{%   set _m = l.lag.mode|default(lag.mode) %}
-{%   if _m=="802.3ad" %}
-{%     set _lacp = l.lag.lacp|default(lag.lacp) %}
-{%     set lacp_act = 'off' if _lacp=='off' else 'on' %}
-{%     set lacp_rate = (' lacp_rate ' + _lacp) if _lacp!='off' else '' %}
-{%     set _m = _m + " xmit_hash_policy encap3+4" + lacp_rate %}
-{%     if node_provider == 'clab' %}
-{%       set _m = _m + " lacp_active " + lacp_act %}
-{%     endif %}
-{%   endif %}
-ip link add dev {{l.ifname}} type bond mode {{_m}}
-{%  endif %}
-ip link set dev {{ l.ifname }} down
-{% endfor %}
-
-{% for l in interfaces if 'lag' in l and l.type != 'lag' %}
-{%   if l.type=='p2p' %}
-{%     if node_provider != 'clab' %}
-ethtool -s {{ l.ifname }} autoneg off speed 1000 duplex full
-{%     endif %}
-ip link set dev {{ l.ifname }} master {% 
-       for i in interfaces if i.type=='lag' and i.linkindex==l.lag._parentindex %}{{ i.ifname }}
-{%     endfor %}
-{%   endif %}
-ip link set dev {{ l.ifname }} up
-{% endfor %}
-{% for l in interfaces if 'lag' in l and l.type == 'lag' %}
-ip link set dev {{ l.ifname }} up
-{% endfor %}
-exit 0
+{% include "linux.j2" %}

netsim/ansible/templates/lag/linux-clab.j2

@@ -0,0 +1,5 @@
+{% include "linux.j2" %}
+
+#
+# Note: The above script is executed within the netns context on the host
+#

netsim/ansible/templates/lag/linux.j2

@@ -0,0 +1,24 @@
+{% if node_provider=='clab' or netlab_linux_distro|default("") != "ubuntu" %}
+#!/bin/bash
+#
+set -e # Exit immediately when any command fails
+#
+# Bond devices are created by 'initial' module - add members
+#
+{% for l in interfaces if 'lag' in l and l.type not in ['lag','bond'] %}
+{%   if l.type=='p2p' %}
+{%     if node_provider!='clab' %}
+ethtool -s {{ l.ifname }} autoneg off speed 1000 duplex full
+{%     endif %}
+ip link set dev {{ l.ifname }} down
+ip link set dev {{ l.ifname }} master {%
+       for i in interfaces if i.type=='lag' and i.linkindex==l.lag._parentindex %}{{ i.ifname }}
+{%     endfor %}
+{%   endif %}
+ip link set dev {{ l.ifname }} up
+{% endfor %}
+{% for l in interfaces if 'lag' in l and l.type in ['lag','bond'] %}
+ip link set dev {{ l.ifname }} up
+{% endfor %}
+exit 0
+{% endif %}

netsim/devices/linux.yml

@@ -1,5 +1,6 @@
 description: Generic Linux host
 interface_name: eth{ifindex}
+lag_interface_name: "bond{lag.ifindex}"
 mgmt_if: eth0
 role: host
 features: