Thomas-Krenn OPNsense Rack-Server Firewall Performance

From Thomas-Krenn-Wiki
Jump to navigation Jump to search

The free firewall solution OPNsense can be used on different devices and servers. To find the suitable hardware for your purpose, the systems must withstand comprehensive performance tests. This article shows these results of the Thomas-Krenn rack servers, that have already been tested, in tabular form. This table is constantly expanded with further systems, when they were tested. An overview of performance data for Thomas-Krenn Low Energy Server (LES) devices and other fanless devices can be also found in Thomas-Krenn-Wiki.

Important hint: These tests were performed under laboratory conditions. Real values in productive environments may deviate. The tested models may have been replaced by new versions in the meantime. The latest hardware selection can be found in the online shop of Thomas-Krenn.

Click here for our OPNsense firewalls in the Thomas-Krenn onlineshop

Test results

The following table shows the results available so far:

Server RI1102A-F (Version 2.0, Supermicro A2SDi-4C-HLN4F with Intel Atom C3558)

(Note: Tests with OPNsense 23.1.1_2-amd64, 25 Gbit/s Broadcom P225P additional card)

RI1102D-F (Version 1, Intel Xeon D-1518)

(Note: Tests withOPNsense 22.7.9_3, 25 Gbit/s Broadcom P225P additional card)

RI1102-SMXDFH (Version 1, Intel Xeon-D D-1718T)

(Note: Tests with OPNsense 25.1.6_4-amd64)

Advantech FWA-3034 (Intel Core i5-13500TE)

(Note: Tests with OPNsense 25.7.2-amd64 und NMC 4x 25GbE SFP28 Intel E810-CAM1 (NMC-2503))

Important hint: Test conducted using the slowest CPU; significantly faster models are available; see HPC-7120S.

Advantech HPC-7120S + ASMB-588 (Intel Core 7 251E)

(Note: Test with OPNsense 26.1.6 and 100G Intel E810 network card)

Important hint: Test with the highest-performing CPU to configure.

RI1101-SMXEFH (Version 1, Intel Xeon E-2334)

(Note: Tests with OPNsense 25.1.9_2-amd64)

Advantech FWA-5072-00A1R (Intel Xeon 6538N)

(Note: Test with OPNsense 25.7. 6, setup with two FWA-5072 and two FWA-5072 as iperf clients, 100G NICs per Intel E810-CAM2 (NMC-6003L) Module)

Review of hardware equipment RAM 16 GB RAM 16 GB RAM 32 GB RAM 16 GB RAM 32 GB RAM 16 GB RAM 64 GB
Settings

(The exact test settings are listed below)

Performance Load (CPU Time in %) Performance Load (CPU Time in %) Performance Load (CPU Time in %) Performance Load (CPU Time in %) Performance Load (CPU Time in %) Performance Load (CPU Time in %) Performance Load (CPU Time in %)
Upload Download Upload Download Upload Download Upload Download Upload Download Upload Download Upload Download Upload Download Upload Download Upload Download Upload Download Upload Download Upload Download Upload Download
Routing
  • NAT deactivated
  • clear routing
9.33 Gbit/s 9.46 Gbit/s 51 66 11.7 Gbit/s 12 Gbit/s 39.5 45.4 13.7 Gbit/s 14.1 Gbit/s 14 14 21.6 Gbit/s 23.5 Gbit/s 31.5 32 40.3 Gbit/s 40.3 Gbit/s 10 10 23.2 Gbit/s 23.5 Gbit/s 24 23.5 70 Gbit/s 71 Gbit/s 12.7 12.8
(Note: 25 Gbit/s Broadcom P225p fully utilized)
Firewall
  • NAT active
  • Spamhaus DROP and EDROP lists (since 2025 only DROP list) on WAN and LAN interface
7.11 Gbit/s 6.31 Gbit/s 83 66 5.9 Gbit/s 6.2 Gbit/s 50 50 7.5 Gbit/s 8.2 Gbit/s 15 14.8 16.4 Gbit/s 17.4 Gbit/s 41 39 35.3 Gbit/s 36.1 Gbit/s 13 12 23.4 Gbit/s 23.5 Gbit/s 29.3 32 50.7 Gbit/s 44.6 Gbit/s 14.1 11.1
(Note: The measured values fluctuated significantly, as did the load on the firewall) (Note: 25 Gbit/s Broadcom P225p fully utilized, The load on the firewall is slightly higher than on the router)
Firewall and IDS
  • Basic settings such as firewall test
  • In addition, intrusion detection (IDS) is enabled on the WAN interface
  • number of rules (missing)
6.51 Gbit/s 6.02 Gbit/s 92 89 5.2 Gbit/s 4.7 Gbit/s 68 61 6.2 Gbit/s 6.7 Gbit/s 20 21 8.1 Gbit/s 8.1 Gbit/s 41 44 19.5 Gbit/s 18.2 Gbit/s 29 29 17.2 Gbit/s 20,8 Gbit/s 44.6 51.7 16.9 Gbit/s 16.4 Gbit/s 19,1 18,2
(Note: The results were questionable, as they fluctuated widely, ranging from 3.2 to 8 Gbps. Consequently, the load on the firewall also fluctuated, ranging from just under 60% to 100%.) (Note: IDS increases the load on the firewall; throughput is very good) (Note: IDS significantly increases the load on the firewall, but throughput remains very high) (Note: IDS throughput is lower compared to the Xeon-E system, but the load is significantly lower)
Firewall and IPS
  • Basic settings such as firewall test
  • In addition, intrusion detection (IDS) is enabled on the WAN interface
  • number of rules (missing)
918 Mbit/s 631 Mbit/s 52 40 5.4 Gbit/s 3.2 Gbit/s 65 41 5.7 Gbit/s 3.8 Gbit/s 26 16 14,4 Gbit/s 10,6 Gbit/s 66 45 32,5 Gbit/s 27,5 Gbit/s 30 28 5,6 Gbit/s 4,7 Gbit/s 23 18,5 - - - -
(Note: Tests performed with 1 Gbit/s onboard NIC) (Note: IPS mode faster than IDS, but also a significantly increased load) (Note: IPS mode is faster than IDS, with a slightly higher load. The CPU behaves similarly to the CPU in the FWA-3034; here, IPS is also better than IDS.) (Note: In IPS mode, the throughput rate drops significantly compared to IDS, but the load on the firewall remains moderate) (Note: Suricata shut down immediately in IPS mode; no measurements were possible)
Firewall and Zenarmor
  • Basic settings such as firewall test
  • In addition, Zenarmor active on LAN interface
33.6 Mbit/s 1.76 Gbit/s 12 44 11 Mbit/s 1.2 Gbit/s 6 27 4.3 Gbit/s 7.6 Gbit/s 15 24.5 2.7 Gbit/s 3.7 Gbit/s 6 15 7 Gbit/s 7.9 Gbit/s 5 7 4.95 Gbit/s 6.5 Gbit/s 16 25 4.3 Gbit/s 5.5 Gbit/s 3 4
(Note: Very poor upload performance; settings: MongoDB database, routed mode with emulated netmap) (Note: Very poor upload performance; settings: MongoDB database, routed mode with emulated netmap) (Note: Local MongoDB database, routed mode with native netmap driver, default policy set to Moderate Control) (Note: Local Elasticsearch 8 database, routed mode with native netmap driver (Intel NICs), default policy set to Moderate Control, low performance but also low load) (Note: Local Elasticsearch 8 database, Routed mode with emulated netmap driver, default policy set to Moderate Control, performance satisfactory, but very low load) (Note: Local Elasticsearch 8 database, routed mode with emulated netmap driver, default policy set to Moderate Control, performance and load comparable to IPS mode) (Note: Local Elasticsearch 8 database, routed mode with native netmap driver (Intel NICs), default policy set to Moderate Control, performance good, load very low)
OpenVPN tunnel
  • Basic settings such as firewall test
  • In addition, OpenVPN tunnel between both firewalls
268 Mbit/s 271 Mbit/s 31 34 372 Mbit/s 373 Mbit/s 17 16 1.2 Gbit/s 1.2 Gbit/s 15.3 13.6 375 Mbit/s 488 Mbit/s 16 7 1,1 Gbit/s 1.1 Gbit/s 4 4 1.5 Gbit/s 1.3 Gbit/s 13.7 13.5 1 Gbit/s 1.1 Gbit/s 2 2
(Note: High performance with extremely light load) (Note: Performance is lower than on a Xeon-E system, but the load on the system is negligible)
IPsec VPN tunnel
  • Basic settings such as firewall test
  • In addition, an IPsec VPN tunnel between both firewall
1.23 Gbit/s 1.13 Gbit/s 61 51 1.6 Gbit/s 814 Mbit/s 42 23 1.3 Gbit/s 1.6 Gbit/s 14 19 2 Gbit/s 947 Mbit/s 16 7 3.1 Gbit/s 5.2 Gbit/s 5 7 2.2 Gbit/s 2.6 Gbit/s 16.4 19.7 533 Mbit/s 587 Mbit/s 1 1
(Anmerkung: Top Performance, sehr niedrige Last) (Note: IPsec tunnel performance is very poor; load is negligible)
WireGuard VPN tunnel
  • Basic settings such as firewall test
  • In addition, a WireGuard VPN tunnel between both firewalls
- - 749 Mbit/s 816 Mbit/s 43 44 3.8 Gbit/s 3.4 Gbit/s 65.9 56.7 1.8 Gbit/s 2.3 Gbit/s 33 30 3.3 Gbit/s 3.5 Gbit/s 10 10 3.9 Gbit/s 3.4 Gbit/s 42.6 38.1 4.5 Gbit/s 4.5 Gbit/s 10 9
(Note: Testing was not possible; the tunnel could not be established reliably due to software issues) (Note: A very surprising and very strong result (the load on the firewall is very high) (Note: WireGuard is CPU-efficient, offering strong performance with low resource usage) (Note: Very strong performance; WireGuard benefits from multithreading; high load on the firewall, but significantly lower than that of the Xeon-D 1718T while delivering similar performance) (Note: Very high performance combined with low power consumption compared to Xeon-E systems)

Setup performance tests

Die nachfolgende Tabelle zeigt die weiteren Komponenten der Firewall Tests, die zu testende Firewall ist immer als Firewall Site 2 gekennzeichnet. Um die maximale Leistung testen zu können, wurde als performante Gegenstelle (Firewall Site 1) ein Server basierend auf einem Supermicro H12SSL-NT Mainboard ausgewählt:

Purpose Hardware BIOS information Software
Firewall Site 1
  • Mainboard: Supermicro H12SSL-NT
  • CPU: AMD EPYC 72F3 (8 kernels)
  • RAM: 32 GB
  • Version: 2.3
  • BIOS settings
    • default
  • Always tested with the latest community version of OPNsense
Firewall Site 2
  • changing the firewall to be tested
Clients up to 2023
Client Site 1
  • Mainboard: Supermicro X11SSH-LN4F
  • CPU: Intel Xeon CPU E3-1230 v6 (3.50GHz, 4 kernels)
  • RAM: 4 GB
  • Version: 2.7
  • Ubuntu Server 22.04.1
Client Site 2
  • Mainboard: Supermicro X10SLH-F
  • CPU: Intel Xeon CPU E3-1220 v3 (3.10GHz, 4 kernels)
  • RAM: 4 GB
  • Version: 3.4
Clients up to 2024 (Two new, identical, and significantly more powerful load clients were procured)
Client Site 1 and 2
  • Mainboard Asus P12R-M/10G-2T
  • CPU: Intel Xeon E-2378 (2,60 GHz, 8 kernels, 16 MB)
  • RAM: 32 GB
  • Version: 1201
  • Ubuntu Server 24.04.2 LTS

Benchmark tools

The performance tests were performed with iperf.

  • The tool iperf was started with the following command on the server site: iperf -p 5000 -f m -s
  • The client site has connected to the iperf server via following command: iperf -p 5000 -f m -c <IP-de-Servers> -t 180 -P 10
  • With the command "vmstat -w 180 -c 2", the load on the firewall was measured

Upload test

To simulate an upload test, iperf was started in client mode on client site 2. On client site 1, iperf was started in server mode with the parameter -s in server mode.

Download test

For a download test, the directions were reversed. Client site 1 was started via parameter -c as client and client site 2 was started with -s in server mode.

Test run

In general, the values in the charts are to be seen as average values up to 10 runs. In some tests (when the network socket was busy) the results were not generated in several runs, as the results were always identical.

Settings

The following settings were made on the OPNsense firewalls. In general, no special optimisation steps were performed. OPNsense was used with default settings. As the settings for the individual VPN technologies are also demanding, the measured values can certainly be regarded as an absolute minimum.

  • Firewall Test
    • Spamhaus DROP and EDROP lists activated on LAN and WAN Interface
    • EDROP list also integrated in DROP and no longer available, for tests of FWA-1112VC-4CA1S and RI1102-SMXDFH only DROP list used
  • OpenVPN Test (legacy Server/Client method)
    • Server Mode: Peer to Peer (SSL/TLS)
    • Protocol: UDP4
    • Device Mode: tun
    • TLS Authentication: Enabled - Authentication & encryption
    • TLS Shared Key: 2048bit OpenVPN static key
    • DH Parameters Length: 4096 bit
    • Encryption Algorithm: AES-256-GCM (256 bit key, 128 bit block, TLS client/server mode only)
    • Auth Digest Algorithm: SHA512 (512-bit)
    • Certificate Depth: Do Not Check
    • Compression: Enabled - LZO algorithm (--compress lzo)
  • OpenVPN Test (Instances method)
    • TLS static key crypt
    • 2048 bit Static Key
    • cipher AES-256-GCM,protocol-flags cc-exit tls-ekm dyn-tls-crypt,tun-mtu 1500
    • Data Channel: cipher 'AES-256-GCM', peer-id: 0
    • Protocol options: protocol-flags cc-exit tls-ekm dyn-tls-crypt
  • IPsec VPN Test (legacy Tunnel Settings method)
    • Phase 1
      • Authentication method: Mutual PSK
      • Pre-Shared Key: yes
      • Encryption algorithm: 256 bit AES-GCM with 128 bit ICV
      • Hash algorithm: SHA256
      • DH key group: 14 (means 2048 bits)
    • Phase 2
      • Protocol: ESP
      • Encryption algorithms: aes256gcm16
      • Hash algorithms: SHA256
      • PFS key group: 14 (2048 bits)
  • WireGuard VPN Test
    • Shared secret (PSK)
  • WireGuard VPN Test (Instances)
    • Public Keys
    • additional Pre-shared key


Author: Thomas Niedermeier

Thomas Niedermeier working in the product management team at Thomas-Krenn, completed his bachelor's degree in business informatics at the Deggendorf University of Applied Sciences. Since 2013 Thomas is employed at Thomas-Krenn and takes care of OPNsense firewalls, the Thomas-Krenn-Wiki and firmware security updates.


Translator: Alina Ranzinger

Alina has been working at Thomas-Krenn.AG since 2024. After her training as multilingual business assistant, she got her job as assistant of the Product Management and is responsible for the translation of texts and for the organisation of the department.


Related articles

Broadcom BCM574xx VLAN problem under FreeBSD 13.2 with bnxt driver
OPNsense LTE connection
Usage of Intel E610 network cards under OPNsense 25.7 with Free BSD 14.3