OpenWrt is the Linux distribution behind a huge number of home routers, access points and small embedded boards. wolfSSL was its default TLS library from the 21.02 release until 23.05, when a href=”https://openwrt.org/releases/23.05/notes-23.05.0″ target=”_blank”>OpenWrt switched to Mbed TLS. One of the reasons given was that Mbed TLS is “considerably smaller”.
A lot has changed in wolfSSL since then, so I wanted to find out how the two compare today. I also added OpenSSL, which OpenWrt packages as well, to make it a three-way comparison. To do this, I wrote a set of custom patches that make an OpenWrt image run entirely on wolfSSL. They are not part of OpenWrt yet, but we are looking to upstream them in the near future.
This is the first of two posts. This one covers a typical low-cost MIPS router, and part two looks at the Raspberry Pi 5.
The short version: on a 650 MHz MIPS router, the wolfSSL image is within 13 KB of the Mbed TLS one and needs about half the heap for an HTTPS download. A TLS 1.3 handshake to the router’s web interface takes 15.7 ms with wolfSSL, against 74.5 ms with Mbed TLS and 44.0 ms with OpenSSL.
The Patches
OpenWrt already packages wolfSSL, but a default image still uses Mbed TLS in several places. The patches add a single build option, USE_WOLFSSL, which switches every default package that uses Mbed TLS to a wolfSSL equivalent:
- HTTPS for uclient-fetch, the uhttpd web server and the LuCI web interface, through ustream-ssl, which gets a new native wolfSSL backend
- Wi-Fi security in hostapd and wpa_supplicant: WPA2, WPA3-SAE, OWE and 802.11r/w
- The apk package manager, with a new wolfCrypt backend
- px5g, which generates the certificate for LuCI’s HTTPS
With the option turned on, there is no Mbed TLS left in the image. With it off, the build is unchanged.
The wolfSSL package also gets a minimal build mode, which only builds the features those packages need. This takes the library from 1.43 MB with the package’s current options down to 643 KB. There is also a choice of ECC code, and the default is wolfSSL’s fast single-precision (SP) code for P-256. P-256 is the curve used for ECDSA certificates and by default in WPA3, so this is where most of the speed in the results below comes from.
Test Setup
I tested on the GL.iNet GL-AR300M16: a Qualcomm QCA9531 with a single MIPS 24Kc core at 650 MHz, 128 MB RAM and 16 MB of flash. This is typical of the lower-cost routers OpenWrt runs on. I also built every image for the TP-Link TL-WR902AC, a travel router with only 8 MB of flash, to see what fits.
All the images were built from the same OpenWrt revision, using wolfSSL 5.9.4, Mbed TLS 3.6.7 and OpenSSL 3.5.9 with each package’s default options (for wolfSSL, the minimal build from the patches). Each image was flashed fresh and measured from a clean boot. A few notes on how things were measured:
- Heap figures come from a small heap-tracking library loaded into each process.
- TLS 1.3 handshakes are the median of 20, from my laptop to the router’s web server, which signs with its P-256 certificate every time.
- WPA3 timings use the image’s own hostapd and wpa_supplicant talking over simulated radios (mac80211_hwsim), so all the crypto runs on the router’s CPU.
- Crypto benchmarks use each library’s own benchmark tool. Each tool uses a different block size by default, so I set or patched them to all process the same sizes (1 KB and 16 KB), with 13 bytes of additional data on the AEAD ciphers, just like a TLS record.
One thing worth knowing: OpenWrt compiles Mbed TLS at -O2, because Mbed TLS’s own build files override OpenWrt’s setting. wolfSSL and OpenSSL are compiled at the size-optimised -Os, so the raw crypto numbers favour Mbed TLS a little.
Flash
On the router, wolfSSL and Mbed TLS are effectively the same size, and OpenSSL is in a different league.
| Bytes | Mbed TLS | wolfSSL | OpenSSL |
|---|---|---|---|
| TLS libraries, xz compressed | 234,820 | 240,120 | 1,245,656 |
| GL-AR300M16 root filesystem | 4,375,752 | 4,385,920 | 5,722,676 |
| GL-AR300M16 firmware image | 7,340,322 | 7,340,322 | 8,716,578 |
| TL-WR902AC root filesystem | 4,619,736 | 4,632,692 | 6,003,708 |
| TL-WR902AC firmware image | 6,684,967 | 6,684,967 | Does not fit |
wolfSSL adds 10 KB to the root filesystem on the GL-AR300M16 and 13 KB on the TL-WR902AC. Firmware images are padded out to the 64 KB flash erase block, so on both devices the wolfSSL image comes out exactly the same size as the Mbed TLS one. That means no less space for settings or extra packages. If flash is really tight, the patches also offer wolfSSL’s smaller generic ECC code, which makes the image 17 to 19 KB smaller than Mbed TLS’s, at the cost of slower P-256.
OpenSSL adds 1.35 MB to the router’s root filesystem. Part of that is the openssl command line tool, which the OpenSSL flavour of LuCI uses to create its certificate. On the 8 MB TL-WR902AC, OpenSSL doesn’t fit at all: the kernel and root filesystem alone come to 8.04 MB, for a 7.54 MB firmware partition, so OpenWrt’s build doesn’t produce one.
Memory
This is where wolfSSL really shines. On the router, it needs the least heap of the three for both the HTTPS client and the web server.
| GL-AR300M16 | Mbed TLS | wolfSSL | OpenSSL |
|---|---|---|---|
| HTTPS download (uclient-fetch), peak heap, bytes | 439,572 | 244,172 | 910,577 |
| HTTPS download, heap allocations | 8,136 | 586 | 45,995 |
| Web server (uhttpd), 20 HTTPS requests, peak heap, bytes | 131,571 | 87,428 | 489,665 |
| Web server, 4 connections at a time, peak heap, bytes | 289,095 | 120,256 | 775,971 |
| Web server, heap held between requests, bytes | 72,163 | 68,932 | 405,684 |
| Web server, heap allocations per request | about 7,000 | about 60 | about 780 |
| Free memory at idle after boot, kB | 64,220 | 64,476 | 61,024 |
wolfSSL needs 44% less heap than Mbed TLS for an HTTPS download, and 58% less for the web server handling four connections at once. OpenSSL needs 3.7x more than wolfSSL for the download and 6.5x more under load. It also keeps 405 KB allocated in the web server between requests, and the OpenSSL image starts with 3.4 MB less free memory than the wolfSSL one. On a router with 128 MB of RAM, that adds up.
The allocation counts stand out too. Mbed TLS makes about 7,000 heap allocations for every HTTPS request to the web server, where wolfSSL makes about 60. Fewer allocations means less time spent in the allocator, and less heap fragmentation in a daemon that runs for months.
I also ran a soak test on each image: 5,000 HTTPS requests to the web server, four at a time, plus 200 downloads and 5 package index updates. None of the three libraries had a failure or leaked any memory.
Performance on the Router
For the things a router actually does with cryptography, wolfSSL is the fastest of the three by a wide margin.
| GL-AR300M16, ms (lower is better) | Mbed TLS | wolfSSL | OpenSSL |
|---|---|---|---|
| TLS 1.3 handshake, X25519 | 74.5 | 15.7 | 44.0 |
| TLS 1.3 handshake, P-256 | 63.0 | 20.8 | 95.4 |
| WPA3-SAE connection | 290 | 162 | 298 |
| WPA3-SAE, access point CPU per connection | 142 | 52 | 139 |
| OWE connection | 173 | 120 | 266 |
A TLS 1.3 handshake to LuCI is 4.7x faster than with Mbed TLS and 2.8x faster than with OpenSSL. WPA3-SAE connections are about 45% quicker than with either of the others, and the access point spends about a third of the CPU time on each one. That matters on a busy router with a lot of clients.
Most of this comes down to the elliptic curve code. wolfSSL’s X25519 is 11x faster than Mbed TLS’s, and its SP code for P-256 signs in 4.5 ms on this CPU, against 13.3 ms for Mbed TLS and 28.4 ms for OpenSSL. OpenSSL has fast P-256 code too, but it needs a 64-bit compiler or assembly that this 32-bit MIPS CPU doesn’t have, so it falls back to its generic code.
| GL-AR300M16, ms per operation (lower is better) | Mbed TLS | wolfSSL | OpenSSL |
|---|---|---|---|
| X25519 key agreement | 26.0 | 2.4 | 2.3 |
| P-256 ECDH | 28.7 | 6.5 | 25.4 |
| P-256 ECDSA sign | 13.3 | 4.5 | 28.4 |
| P-256 ECDSA verify | 43.9 | 7.3 | 23.4 |
| RSA-2048 sign | 862 | 313 | 130 |
| RSA-2048 verify | 9.8 | 5.7 | 3.4 |
| GL-AR300M16, MiB/s with 16 KB blocks (higher is better) | Mbed TLS | wolfSSL | OpenSSL |
|---|---|---|---|
| ChaCha20-Poly1305 | 15.83 | 15.37 | 15.42 |
| AES-128-GCM | 1.39 | 3.48 | 5.04 |
| AES-128-CBC | 7.97 | 5.51 | 10.28 |
| SHA-256 | 13.03 | 12.32 | 15.80 |
Against Mbed TLS, wolfSSL is 3x to 11x faster at the public key operations TLS and WPA3 use, and 2.5x faster at AES-GCM.
OpenSSL does win in places. It has MIPS assembly for AES and RSA, which makes its AES-GCM 1.45x faster than wolfSSL’s and its RSA-2048 signing 2.4x faster. In practice, neither matters much on this router. Without AES instructions in the CPU, TLS 1.3 negotiates ChaCha20-Poly1305, where all three are within 4%, and the router’s certificate is ECDSA. RSA only comes into play when verifying RSA server certificates as a client, which takes 3.4 ms with OpenSSL against wolfSSL’s 5.7 ms.
wolfSSL’s AES-CBC is the slowest of the three here, which is down to the -Os build. At -O2, the level Mbed TLS is built at, it is level with Mbed TLS, and wolfSSL’s ChaCha20-Poly1305 becomes about 30% faster than Mbed TLS’s. TLS 1.3 doesn’t use CBC anyway.
What’s Next
The patched images have been through a lot of testing on the router: valid and invalid certificates in the HTTPS client, LuCI over TLS 1.2 and 1.3 in a browser, WPA2, WPA3 and OWE with a real Wi-Fi client, installing packages from the official feeds, and soak tests totalling 20,000 HTTPS handshakes with no failures or memory growth. wolfCrypt’s own test suite passes on the router too.
The next step is getting the patches upstream into OpenWrt, along with a few improvements to wolfSSL itself that came out of this work. Overall, I’m very happy with the results. On a typical router, wolfSSL costs about the same flash as Mbed TLS, uses much less memory than either Mbed TLS or OpenSSL, and makes TLS and WPA3 several times faster.
In part two, I look at how the three libraries compare on a much faster 64-bit ARM board, the Raspberry Pi 5.
If you are interested in wolfSSL on OpenWrt, or have questions about any of the above, please contact us at facts@wolfssl.com or call us at +1 425 245 8247.
Download wolfSSL Now