Overview of Testing in wolfSSL

    • The security of wolfSSL products is always on the wolfSSL team’s mind and holds high importance. Conducting regular, diligent, and well-planned testing helps maintain wolfSSL’s robustness and security. The wolfSSL team strives to write and maintain clean, readable, and understandable code.

    1. API Unit Testing: Unit tests are in place to test API functions for correct behavior. This helps maintain library consistency across releases and as the code evolves. It ensures delivery of a high quality, well tested API to end users with each software release. API unit tests are run with each “make check” of wolfSSL.
    2. Cipher Suite Testing: wolfSSL supports an extensive list of cipher suites, which are all tested with every “make check” using the wolfSSL example client and example server. Each cipher suite is tested not only in the default configuration, but also in non-blocking mode and with client authentication both turned on and off.
    3. Algorithm Testing: The security of the SSL/TLS implementation depends on the correctness and robustness of the underlying cryptography library, wolfCrypt. The wolfssl team tests all algorithms using NIST test vectors in addition to running a CAVP test harness used for FIPS 140-2 and 140-3 validations. Testing is done on both big and little endian platforms for portability.
    4. Benchmark Testing: The wolfSSL team engages in the ever expanding universe of benchmark testing, taking into consideration sizing, transmission rates, connection speeds, and cryptography performance. A version of the benchmark suite is included in every download for users to enjoy!
    5. Static Analysis: wolfSSL does static analysis on the entire codebase using not only one, but multiple different static analysis tools. Currently Coverity Scan, CodeSonar, clang scan-build, and Facebook infer. These tools help to automatically find bugs including those on low-traffic code paths.
    6. Detecting Memory Errors: Memory errors are mitigated by using valgrind on a regular and automated basis. This helps find memory errors including invalid access, use of undefined values, incorrect freeing of dynamic memory, and memory leaks.
    7. Interop Testing: Interoperability checks are done with other Open Source TLS implementations, including OpenSSL, BoringSSL, and GnuTLS. This helps to catch any protocol implementation errors in either wolfSSL or the implementation being tested against. Testing is also done outside of a closed environment by connecting to servers in the real world running unknown SSL/TLS implementations.
    8. Real World Builds: Builds with a series of ‘real’ applications, like cURL, wget, pppd, OpenSSH, stunnel, lighttpd, etc. For some wolfSSL customers with top level support, the wolfSSL team also tests building the latest library version against those customer applications.
    9. Compiler Testing: Some users compile wolfSSL with a variety of different compilers. As such, wolfSSL tests compilation with many different compilers and toolchains including but not limited to: gcc/g++, clang, icc, Visual Studio, CodeWarrior, KDS, LPCXpresso, MPLAB XC, TI CCS, Keil, IAR, Espressif IDF, Cygwin, MinGW, CrossWorks, Arduino, Wind River Workbench.
    10. Peer Review: More eyes on a codebase reduces bugs that end up in a final product. Internally, the wolfSSL team operates using a “Fork and Pull Request” model. This means that every commit that makes it into the wolfSSL master branch has been reviewed and tested by a minimum of two engineers.
    11. Code size increase monitoring: wolfSSL has tests that give alarms when a particular commit greatly increases the code size for a particular build. Active Monitoring of how much the code has grown since the last release ensures that it is within an acceptable range.
    12. Third Party Testing: wolfSSL code is regularly reviewed by university researchers, customer and user security teams, FIPS and certification labs, and other Open Source users. This helps put more eyes on the code and product architecture.
    13. Fuzz Testing: wolfSSL tests using several different software fuzzers, including an in-memory fuzzer, a network fuzzer, OSS-fuzz, libfuzzer, tlsfuzzer, and AFL. Fuzz testing bombards the program with invalid, unexpected, and random data that then allows for observing if there are potential memory leaks or logic errors. This allows us to catch bugs that could turn into potential vulnerabilities if released in a final release.
    14. Protocol Analysis: TLS-Attacker, a Java-based framework for analyzing TLS libraries, helps us analyze that wolfSSL correctly conforms to the SSL/TLS specification.
    15. Continuous Integration (CI): Leveraging Jenkins as well as GitHub Actions, allows wolfSSL to run tests on each commit submitted to the wolfSSL code repository. Tests run on each commit include testing of the FIPS build, numerous build options (customer/user/common), running valgrind, and doing static analysis with scan-build.
    16. Nightly Test Cycle: Each night extended tests are run that last longer than the typical ones during the work day. These extended tests are more in-depth than CI testing and puts results in wolfSSL engineers’ inboxes each morning. Some tests included in the nightly cycle include extended build option testing on multiple platforms with multiple compilers, and extended fuzz testing.

    If you have specific questions about how the wolfSSL team tests, please contact us at facts@wolfssl.com or call us at +1 425 245 8247. If you would like the wolfSSL team to include your SSL/TLS or crypto implementation in interop testing, please let us know! Likewise, if you would like to include wolfSSL in your own test framework, the wolfSSL team would be happy to discuss.

    Download wolfSSL Now