Topic: [SOLVED] WolfSSL Interoperation with OpenSSL (AES CTR)
I'm currently trying to get a microcontroller running FreeRTOS with WolfSSL working with an x86 server using OpenSSL.
I'm encrypting messages server-side with the OpenSSL EVP Cipher functions, using EVP_aes-256_ctr() as the type:
EVP_CIPHER_CTX_new() EVP_EncryptInit_ex(ctx, EVP_aes_256_ctr(), NULL, TEST_KEY, TEST_IV) // EVP_CIPHER_CTX_set_padding(ctx, 0) /* tried adding this for wolfSSL compat */ EVP_EncryptUpdate() EVP_EncryptFinal_ex()
Now this currently works with OpenSSL on the other side running decryption. However, I'm trying to get decryption working with WolfSSL and I've been having problems. I've currently tried a few approaches (mostly based on the FreeRTOS example code), including:
// attempt with bare AesCtrEncrypt: byte cipher[AES_BLOCK_SIZE * 8]; // this is ~twice as large as the messages Aes aes; wc_AesSetKey(&aes, TEST_KEY, AES_BLOCK_SIZE, TEST_IV, AES_ENCRYPTION); wc_AesCtrEncrypt(&aes, cipher, msg, out_len); // attempt with wolfssl EVP Ciphers: EVP_CIPHER_CTX ctx; byte plain [AES_BLOCK_SIZE * 12]; EVP_CIPHER_CTX_init(&ctx); EVP_CipherInit(&ctx, EVP_aes_256_ctr(), TEST_KEY, TEST_IV, 0/*decrypt*/) EVP_Cipher(&ctx, plain, (byte*)msg, len)
But neither of these two main approaches have worked for decrypting the encrypted bytes the server is sending over. Based on the docs I have no idea what to look for - the page for AesCbcEncrypt specifically mentions not using padding for OpenSSL interoperability, but the page for AesCtrEncrypt makes no mention.
Thanks in advance!