The target is a STM32F417 with gcc arm-elf cross compiler. No configure script; options are set using settings.h

I'm not certain why the more portable bigint library did not work for me. I am confident that there are no stack or heap issues. I noticed that many of the other embedded targets (ie. PIC32) define USE_FAST_MATH and TFM_TIMING_RESISTANT and when including these implementations RsaSSL_Verify() is working correctly when loading the public key from a DER file.

Still CyaSSL_KeyPemToDer() fails because there is no attempt to match the PEM public key header "-----BEGIN PUBLIC KEY-----" etc. I didn't look into it further than this, but if I revisit this software and implement reading a public key from PEM format I will send a pull request.

To update, I've resolved the problem of RsaSSL_Verify() returning error codes by adding USE_FAST_MATH and TFM_TIMING_RESISTANT to my build preprocessor definitions.

It appears that parsing of public keys in PEM format is not supported.

My goal is to have a user supply a public key in PEM format which is then used for verification of signed firmware update files.

I have also tried exporting the public key to DER format and loading with RsaPublicKeyDecode() but RsaSSL_Verify() is failing with RSA_PAD_E (-201). I've added the preprocessor define RSA_DECODE_EXTRA to enable parsing the public key from the DER. The key and signature is 2048 bits in length.

The signature is being generated with openssl:
openssl dgst -sha1 -out build/firmware.bin.sign -sign private.pem firmware.bin