Intel Ethernet 700 Series LACP Configuration

From Thomas-Krenn-Wiki
Jump to navigation Jump to search

Network cards of the Intel Ethernet 700 Series have an integrated Link Layer Discovery Protocol Engine, which processes LLDP frames already in the chip of the network card and therefore does not forward them to the operating system. However, the network stacks of Linux and VMware use LLDP information when the LACP is configured. In this article we therefore show how you can disable the LLDP engine of the Intel Ethernet 700 Series and thus use LACP under Linux and VMware.

Affected hardware

The information provided in this article about disabling the LLDP Engine applies to the following hardware components:

  • Intel Ethernet 700 Series network cards, such as Intel 10 Gigabit X710-T4 RJ45 Quad Port Network Card or Intel 10 Gigabit X710-DA2 SFP+ Dual Port Network Card
  • Network cards from other manufacturers based on the Intel 700 network chipsets, e.g. Supermicro 10 Gigabit AOC-STG-i4S SFP+ Quad Port Network Card (based on the XL710-BM1 network chip from Intel)

Disable LLDP Engine

For correct operation of LACP under Linux or VMware, LLDP frames must be forwarded from the network card to the operating system.

Network cards of the Intel Ethernet 700 series (X710, XL710, XXV710) process these LLDP frames in the standard configuration in the integrated LLDP engine. In the following instructions we will show you how to disable them under Linux or VMware.[1] Please note that at least firmware version NVM 6.01 and a current driver version are required.[2][3][4]

Linux

To disable the LLDP hardware engine of the Intel Ethernet 700 Series network adapter in a Linux environment, run the following command in Linux Shell:

ethtool -set-priv-flags <interface name> disable-fw-lldp on

Alternatively, the LLDP Engine can be deactivated via a start script, for example:

echo lldp stop > /sys/kernel/debug/i40e/0000\:02\:00.0/command

VMware

Disabling the hardware DLDP engine of the Intel Ethernet 700 Series Network Adapter for ESXi 6.5 U2 is accomplished through an i40en module parameter option. To disable the LLDP engine, run the following command on the VMware ESXi console:

esxcfg-module -s LLDP=0,0 i40en

The above command disables the LLDP engine on port 0 and port 1 of the Intel Ethernet Converged Network Adapter X710-DA2. The ESXi host must be rebooted for the command to take effect.

FreeBSD

FreeBSD supports disabling the hardware DLDP engine on the Intel Ethernet 700 Series network adapter, starting with FreeBSD version 11.2.[5][6][7]

Note: According to reports in the Netgate Forum, pfSense (FreeBSD-based) with Intel Ethernet 700 Series network cards with driver version 1.9.9k when using VLANs Queue Hangs and MAC VLAN Error.[8] Driver version 1.11.20 and higher fixes these problems, a failover LAGG works with it (LACP lagg was not tested in this report).[9]

Effects with activated LLDP Engine

With redundant network connections (LACP), connection problems occur in connection with network cards of the Intel Ethernet 700 Series (e.g. X710, XL710, XXV710) if the LLDP Engine is activated. When using LACP with these network cards, we therefore recommend deactivating the LLDP engine as described above.

Further Information

References

  1. LACP Configuration Guide - Using Intel Ethernet 500 and 700 Series Network Adapters and Various Server Operating Systems (Intel Technical Brief, April 2018, Chapter 2.6 Intel Ethernet 700 Series Network Adapters - Special Considerations): Each Intel Ethernet 700 Series Network Adapter has a built-in hardware LLDP engine, which is enabled by default. [...] LACP may not function correctly in certain environments that require LLDP frames containing LCAP information to be forwarded to the network stack. To avoid this situation, the user must disable the Intel Ethernet 700 Series Network Adapter's hardware LLDP engine.
  2. Intel Ethernet Controller X710/XXV710/XL710 (Intel Specification Update, July 2018, Issue 70. LLDP Disable Can Result in Incorrect Configuration of the Receive Packet Buffer): Disabling LLDP when using NVM 4.53 or 5.x can result in the device firmware not configuring the Receive Packet Buffer according to the link mode and flow control settings. [...] Fixed in NVM 6.01 [...] To ensure proper operation with Intel drivers, Intel recommends using NVM 6.01 or later and drivers from Release 22.6 or later.
  3. (net-next 07/15) i40e: Implement an ethtool private flag to stop LLDP in FW (Linux netdev Mailing List, 26.01.2018)
  4. Bug 221530 - if_ixl: add a way to disable/enable lldp handling by the NIC (bugs.freebsd.org)
  5. Query Intel NIC driver version (forum.opnsense.org, 10.08.2018)
  6. FreeBSD 11.2 Release Notes - 6.1. Device Drivers (www.freebsd.org, 21.06.2018) The ixl(4) driver has been updated to version 1.9.9-k.[r333343]
  7. MFC r333149 ixl(4) Update to 1.9.9-k (svnweb.freebsd.org, 07.05.2018) Ability to disable firmware LLDP agent by user
  8. New Version 2.4.4 - Interface Error --> aq_add_macvlan err -53, aq_error 14 - Comment 47 (forum.netgate.com, 07.12.2019) A quick FollowUP running 1.11.20 [...]
  9. New Version 2.4.4 - Interface Error --> aq_add_macvlan err -53, aq_error 14 - Comment 53 (forum.netgate.com, 01.02.2020) It is still super stable. [...] use the latest driver [...] use failover LAGG and not LACP lagg (but we did not test LACP with latest driver so we can't confirm is is not working with it)


Foto Werner Fischer.jpg

Author: Werner Fischer

Werner Fischer, working in the Knowledge Transfer team at Thomas-Krenn, completed his studies of Computer and Media Security at FH Hagenberg in Austria. He is a regular speaker at many conferences like LinuxTag, OSMC, OSDC, LinuxCon, and author for various IT magazines. In his spare time he enjoys playing the piano and training for a good result at the annual Linz marathon relay.


Related articles

BiDi Transceiver
Determine Server Heat Loss for Air Conditioning
Difference between Volt-amperes and Watts