• Registered: 2022-08-15
  • Posts: 6

Topic: chunk size in datastream

Hi there,

In a server response, the data is transmitted in separate chunks. When processing these data chunks, the size of the next chunk appears interspersed with the data itself. Is this expected behavior? Is there a setting to hide this within the data stream? The application is developed in the Arduino IDE for STM32 using a W5500 Ethernet interface and utilizes wolfSSL 5.8.4.

              <Point>
                <position>45</position>
                  <price.amount>145.69</price.amount>
              </Point>
              <Point>
                <position>46</position>
         <price.amount>146.02</price.a
1b70
mount>
              </Point>
              <Point>
                <position>47</position>
                  <price.amount>142.13</price.amount>
              </Point>

The 0x1b70 is the size of the next chunk.

The data is read into 256-byte buffers. This part works correctly, but the chunk size appearing in the middle of the stream makes interpretation difficult.

Share

  • Registered: 2021-07-06
  • Posts: 197

Re: chunk size in datastream

Hi BerHav,

It looks like you're seeing HTTP chunked transfer-encoding.  This is coming from a layer higher than what wolfSSL operates at.  wolfSSL_read returns the raw decrypted bytes from the connection, it is up to the application to further process these bytes.
You will need to parse the size in your application, or you can look into using an HTTP library, for example curl.

Are you able to share any information about your project?  Are you working on a personal or commercial project?

Thanks,
Kareem

Share

  • Registered: 2022-08-15
  • Posts: 6

Re: chunk size in datastream

Hello Kareem,
Thanks for your quick reply. Unfortunately, I already suspected that I would have to handle this within the application itself; I couldn't find any information about it.

The application is for personal use and manages the control logic for solar panels combined with a home battery. This component retrieves electricity market prices to determine whether the battery should be charged from the grid or discharged. The prices are fetched from web-api.tp.entsoe.eu.

Thanks,
Bert

Share

  • Registered: 2022-08-15
  • Posts: 6

Re: chunk size in datastream

Probably the most simple solution is to send a request  with HTTP/1.0 instead of HTTP/1.1. In 1.0 chunking is not supported and a 1.1 client must support chunking. Otherwise the application has to handle the chunks.

Regards,
Bert

Share