Topic: wc_InitRsaKey causes stack smashing

I'm trying to create an RSA keypair using the following code:

int main()
    RsaKey rsakey;
    printf("Memory allocated\n");

    RNG rng;
    printf("Ring initialized\n");
    wc_InitRsaKey(&rsakey, 0);
    printf("RSA key initialized\n");

    int ret = wc_MakeRsaKey(&rsakey, 1024, 65537, &rng);

    printf("RSA key made\n");

    if (ret != 0)
        printf("Error creating RSA key\n");
        printf("Successfully created the RSA key\n");

    return 0;

I have even placed the return 0 after the wc_MakeRsaKey line. When I run this program, I see the following output:

$ ./main
Memory allocated
Ring initialized
RSA key initialized
RSA key made
Successfully created the RSA key
*** stack smashing detected ***: <unknown> terminated
Aborted (core dumped)

I have tried including the wolfssl/wolfcrypt/rsa.h as well as the cyassl/ctaocrypt/rsa.h files, both seem to have the same problem.

I have downloaded the wolfssl3.9.6 file, configured it with

./configure --enable-keygen

, I can verify the install with

wolfssl-config --version

The code is compiled with the following Makefile (excerpt)

CFLAGS=-c -Wall -I/usr/local/lib

all: main

main: main.o
    $(CC) main.o -o main -lwolfssl

main.o: main.c
    $(CC) $(CFLAGS) main.c

Is there a step that I am missing? Why does this seem to always lead to stack smashing / seg faults?

Thanks for the help!


Re: wc_InitRsaKey causes stack smashing


Have you compiled your application using the same preprocessor defines that were used when compiling wolfSSL?

When using the Autoconf system (./configure), the easiest way to make sure this happens is to include the <wolfssl/options.h> header as the first wolfSSL include in your application:

#include <wolfssl/options.h>
#include <wolfssl/ssl.h>

Best Regards,

Re: wc_InitRsaKey causes stack smashing

Hi Chris,

Thanks! It seems like I was missing the options.h include that you mentioned. I realize now that I neglected to cite my includes from above.

Adding those lines seems to have fixed the problem.


Re: wc_InitRsaKey causes stack smashing

Hi Colin,

Great! Glad that fixed your problem.

Best Regards,