Does that mean that I just need to protect against multiple calls to wolfSSL_read() and multiple calls to wolfSSL_write() and so can have a separate mutex for each or do I also need to prevent a call to wolfSSL_write() while another thread is in wolfSSL_read() and vice-versa?

To better clarify wolfSSL, for embedded design reasons, uses a single I/O buffer since TLS packets can be up to 16k we thought it unreasonable for resource constrained devices to have to maintain 32k for thread safe reads and writes, the overhead was too much. Because there is only one buffer used for both sending and receiving send calls can corrupt data that is not yet read. Similarly read calls can corrupt data that is not yet sent. A single mutex should be used to protect against both wolfSSL_read and wolfSSL_write calls on the same SSL object.

For blocking setups we added a feature called "write duplicate". This is where you use one SSL object to connect to a server. Once the connection is established and the handshake is complete you can make a duplicate of the SSL object for writing only. Then the first SSL object becomes a read only object. This gives a solution for the scenario SieRaybould noted but also adds that overhead of 32k for thread safe reading and writing. See wolfSSL_write_dup(ssl); for creating a write duplicate of an SSL object.

Warm Regards,

K

"You'll need to protect calls to wolfSSL_read() and wolfSSL_write() on the same WOLFSSL object with a mutex or similar to avoid more than one thread trying to call it simultaneously."

Does that mean that I just need to protect against multiple calls to wolfSSL_read() and multiple calls to wolfSSL_write() and so can have a separate mutex for each or do I also need to prevent a call to wolfSSL_write() while another thread is in wolfSSL_read() and vice-versa?

I hope not or this would mean it is impossible to use a blocking receive interface at the socket transport layer in a multi-threaded environment, as while the receive is waiting for data, no-one would be able to send anything.

Kind Regards,
Simon Raybould,
Firmware Developer, Landis+Gyr, Switzerland.

Thanks

The problem is - The callback is only being called if the SSL_Read function is called. The threads that are reading are receiving messages asynchronously, meaning I don't know when a message will be received...

One way I can think of is inserting the newly received message into a queue, and then calling to SSL_Read explicitly, overriding the read callback to get the message from the queue.
But that seems kind of crooked (And needs to be well designed thread-safety-wise)

The higher levels (i.e. the Business Logic that uses the TCP server) are built based on asynchronous messaging as well, so I'd like to keep the reading method as it is

Thanks
Nitay

It sounds like you're on the right track with registering your own I/O callbacks.  The callbacks were designed to allow applications to control how they read/write data to/from wolfSSL.  For your Recv callback, you'll just need to pass the encrypted data buffer received over your transport medium to wolfSSL through the provided buffer, "buf", in your callback.  For an example, you can reference the EmbedReceive() function in <wolfssl_root>/src/io.c.

I'm not familiar with IOCP myself, so if I'm misunderstanding what you are trying to do, please correct me.

Best Regards,
Chris

I'm less concerned about thread safety since I can limit thread access per session. I'm just not sure about how can I use wolfSSL if my reads are done outside the library, and not through the SSL_Read function

Thanks
Nitay

Section 9.4 of the wolfSSL Manual (http://www.yassl.com/yaSSL/Docs-cyassl- … esign.html) talks a little about wolfSSL's thread safety.  Have you looked through that?

wolfSSL is generally thread safe, but reading from or writing to the same WOLFSSL object with multiple threads at one time is not supported.  You'll need to protect calls to wolfSSL_read() and wolfSSL_write() on the same WOLFSSL object with a mutex or similar to avoid more than one thread trying to call it simultaneously.

Best Regards,
Chris

I have tcp server which uses multiple threads for reading through an IO Completion Port. I want to add SSL to this server.

What, in your opinion, is the best way to do so?
The writes are less of a problem, but how do I decrypt a SSL packet without calling read?

For now, I've set the write and read callbacks using wolfSSL_SetIORecv / wolfSSL_SetIOSend.
I didn't implement the reads yet.
Also, the accept could be done to a new client before attaching it to the IOCP, so its not really a non-blocking IO...

What do you think?

EDIT: The reads are done through IOCP, so the threads are not blocked on recv() or select(), but on GetQueuedCompletionStatus()

Thanks
Nitay