• tjko
  • Member
  • Offline
  • Registered: 2025-06-30
  • Posts: 6

Topic: Enabling Post-Quantum (ML-KEM) support

As new OpenSSH versions have started nagging about not using post-quantum safe key exchange algorithms, I decided to test enabling ML-KEM on wolfSSH.

I added following in user_settings.h:

#define WOLFSSL_HAVE_MLKEM
#define WOLFSSL_WC_MLKEM
#define WOLFSSL_SHAKE128
#define WOLFSSL_SHAKE256

And that seemed to work, as (after re-compiling) wolfSSH server started advertising two new "mlkem" algorithms:

debug2: peer server KEXINIT proposal
debug2: KEX algorithms: mlkem768x25519-sha256,mlkem768nistp256-sha256,curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256
debug2: host key algorithms: ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: aes256-gcm@openssh.com,aes192-gcm@openssh.com,aes128-gcm@openssh.com
debug2: ciphers stoc: aes256-gcm@openssh.com,aes192-gcm@openssh.com,aes128-gcm@openssh.com
debug2: MACs ctos: hmac-sha2-256,hmac-sha2-512
debug2: MACs stoc: hmac-sha2-256,hmac-sha2-512
debug2: compression ctos: none
debug2: compression stoc: none

However, when trying to connect using OpenSSH client (OpenSSH_10.2p1, LibreSSL 3.3.6), connection seems to hang indefinitely in key exchange:

debug1: kex: algorithm: mlkem768x25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: aes128-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes128-gcm@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY

(client hangs here...)


If I try any of the other advertised kex algorithms (curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256) connection works just fine.... (I was testing on latest 'version' pulled from github)

Share

  • Registered: 2021-08-12
  • Posts: 172

Re: Enabling Post-Quantum (ML-KEM) support

Hi!!
My name is Anthony and I am a member of the wolfSSL team.
Interesting!! Can you tell me the exact steps I need to perform to reproduce this issue?
Warm regards Anthony

Share

  • tjko
  • Member
  • Offline
  • Registered: 2025-06-30
  • Posts: 6

Re: Enabling Post-Quantum (ML-KEM) support

I'm seeing this on Raspberry Pi Pico W (RP2040).

Here is link to changes I made to enable ML-KEM support: https://github.com/tjko/fanpico/compare/mlkem

If you have a Pico W (or Pico 2 W should yield same results), you should be able to compile firmware image from "mlkem" branch to reproduce the issue:
https://github.com/tjko/fanpico/tree/mlkem


It would seem like its the server that fails to respond to SSH packet "30" (?), since with other kex algorihms client gets response:

debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: aes128-gcm@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: aes128-gcm@openssh.com MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_ECDH_REPLY received
...

(and SSH connection negotiation succeeds normally)

Share

4 Reply by tjko (edited by tjko 2026-03-03 19:23:37)

  • tjko
  • Member
  • Offline
  • Registered: 2025-06-30
  • Posts: 6

Re: Enabling Post-Quantum (ML-KEM) support

I enabled debugging (DEBUG_WOLFSSH), and from the logs it would seem that it is the server that gets stuck in a "loop", getting same error repeatedly:

accept error: SERVER_KEXINIT_SENT, -1010

Log can be found here: https://gist.github.com/tjko/53122b4165 … 4ec6f0c5f6

Share

  • Registered: 2021-08-12
  • Posts: 172

Re: Enabling Post-Quantum (ML-KEM) support

Hi tjko.

Could it possibly be that you're running out of stack?  After all these are very large artifacts.

Have you tried building with your user_settings.h on a linux machine to see if you get different results?

Warm regards, Anthony

Share

6 Reply by tjko (edited by tjko 2026-03-06 11:31:18)

  • tjko
  • Member
  • Offline
  • Registered: 2025-06-30
  • Posts: 6

Re: Enabling Post-Quantum (ML-KEM) support

Thanks for the tips. Comparing differences in debug output from Linux (where things work fine) and Pico helped me to track down the issue. It wasn't stack overflow, but too small network (rx) buffer that wasn't large enough to fit the ML-KEM key exchange.

I know that running SSH server (and everything else) in 264Kb of RAM is pushing the limits, but looks like its possible...

Share

  • Registered: 2021-08-12
  • Posts: 172

Re: Enabling Post-Quantum (ML-KEM) support

Its totally possible!  You might need to optimize though.  For example, turning off certain features and algorithms and looking for configuration macros that lower memory usage.  Happy hunting!!

Share