Topic: RSA verify callback


I would like to use my own RSA verify function instead of the built-in one.
A way to do this a to use set the callback function using wolfSSL_CTX_SetRsaVerifyCb().

I don't understand well what exactly is the return value as well as the parameter char ** out as this not specified in the documentation.

typedef int (*CallbackRsaVerify)(WOLFSSL* ssl,

unsigned char* sig, unsigned int sigSz,

unsigned char** out, const unsigned char* keyDer,

unsigned int keySz, void* ctx);

What I understand is :

- The parameter out  is a pointer on pointer of the decrypted signature (in other words the hash)
- The return value is the size of the decrypted signature (the size of the hash)

Is my understanding correct? otherwise, what do the parameters represent?

Thanks in advance,


Re: RSA verify callback

Hi Okba,

It's kind of strange that that function isn't being documented. From what I can tell, it should be documented on this page here: In regards to the actual explanation of what that function does, let me share the snippet that should be describing the function on that page.

    Allows caller to set the Public Key Callback for RSA Verification.
    The callback should return the number of plaintext bytes for success or
    < 0 for an error.  The ssl and ctx pointers are available for the user’s
    convenience.  sig is the signature to verify and sigSz denotes the length
    of the signature.  out should be set to the beginning of the verification
    buffer after the decryption process and any padding.  keyDer is the RSA
    Public key in ASN1 format and keySz is the length of the key in bytes.
    An example callback can be found wolfssl/test.h myRsaVerify().

The function "wolfSSL_CTX_SetRsaVerifyCb" should not return anything on its own, as it's return type is void.

For future reference, when these functions are not being described in the documentation, many wolfSSL API have descriptions located within the wolfSSL library itself, at the following location: <wolfssl-root>/doc/dox_comments/header_files/.

Please let me know if there was anything I missed or if you'd like any further information.