FreeRTOS TCP/IP stack: question about order of packets when sending file

Kernel
1

dibosco wrote on Wednesday, February 25, 2015:

I have a simple web server running using the FreeRTOS TCP/IP stack and can have a browser pick a file from the computer and send the file to the FreeRTOS device. However, the file is much bigger than the amount of RAM I have and I need to write this file to flash without assembling it in RAM first.

At the moment, I can prove that each time I send the file from my computer to the FreeRTOS device, the file segments appear in exactly the right order. I am assuming that the trasmission goes like this:

Compueter -> FreeRTOS: Here is the first packet
FreeRTOS -> Computer: Got it
Computer -> FreeRTOS: Here is the second packet
FreeRTOS -> Computer: Got it
…
…
Computer -> FreeRTOS: Here is the last packet
FreeRTOS -> Computer: Got it

Each time the computer sends that packet out it doesn’t send the next packet until it’s had the acknowledge from FreeRTOS. My end customer is saying that even though this set-up will only ever be used on a LAN, their switches on the LAN might not send out packets in the correct order as, and I don’t know whether this can be true, the switch the traffic is going thorugh could take more than one packet from the computer before getting an acknowledge for a TCP packet and then send it out to FreeRTOS in the wrong order. Unless we set some sort of system variable on FreeRTOS to specifically tell the connection to the computer that only one packet should ever be sent before the acknowledge comes back from FreeRTOS.

There may be a technical term for this and I am hoping that either:

a. This is how it works with the FreeRTOS stack anyway
or
b. There is a FreeRTOS TCP/IP variable to ensure that the client only ever sends out one packet to the FreeRTOS unit when it gets the acknowledge for the previous packet, thus ensuring the file is received in the correct order.

Hope that makes sense? Can anyone advise one way or the other whether what I need is feasible. I am not even convinced that a switch would ever send a TCP acknowledge to the computer, I thought switches just passed Ethernet packets and didn’t look at what was actually inside them. I thought the latter was something a gateway our router would do!

Thanks!

2

heinbali01 wrote on Wednesday, February 25, 2015:

Each time the computer sends that packet out it doesn’t send
the next packet until it’s had the acknowledge from FreeRTOS

Do you mean that this is the normal TCP ACK-ing mechanism, or something on top of it?

their switches on the LAN might not send out packets in the correct order

I think that chances are very small, even if the traffic would run through the Internet.
But nevertheless, the function FreeRTOS_recv() will deliver you the data in exactly the same order as it has been sent by the email client.

If your data don’t contains confidential stuff, you might want to post a PCAP, recorded by Wireshark or so? You can filter the conversation of interest and export the selected packets.

Receiving packets one-by-one can be accomplished by playing with the properties of the (listening) socket: FREERTOS_SO_WIN_PROPERTIES. See the documentation on the website. I would recommend to use at least a window of 2 x MSS, not less.
If your RAM is really small, you may decrease the MTU size to e.g. 1024 bytes.

I thought switches just passed Ethernet packets and
didn’t look at what was actually inside them

That is right, a switch will not send ACK’s, only the end-point (your device) will send an ACK to the web client.
A switch might look at the address fields though and filter-out certain packets.

This is a short and quick answer, please feel free to ask more.

Regards,
Hein

3

dibosco wrote on Thursday, February 26, 2015:

Hi Hein,

Sorry for the delay in replying, been off in DALI land!

To answer your questions:

“Do you mean that this is the normal TCP ACK-ing mechanism, or something on top of it?”

I mean normal TCP ACKing.

It’s actually a web broswer that is sending the file, not email. My set-up has the Atmel SAM4E being a web server and serving a page that enables the browser to send a file to the SAM4E.

What I am looking for is a mechanism to ensure that the broswer absolutely will not send a packet out until it has received an ACK of the previous packet from the server.

I assume that when the broswer sends a file it starts at the beginning of the file and sends each byte in exactly the order it appears in the file it is sending. If that is the case and the browser always waits for the ACK from FreeRTOS before it sends the next packet I can’t see why it would not work perfectly!

I assume that if there is an issue with a TCP packet, the underlying stack sends a please resend message to the broswer and it will send the packet again, with the FreeRTOS_recv() only ever presenting correct data.

Thanks!

Rob

4

heinbali01 wrote on Thursday, February 26, 2015:

Hi Rob,

It’s actually a web broswer that is sending the file

I suppose that the browser uses the POST or PUT method?

ensure that the broswer absolutely will not send a packet out
until it has received an ACK of the previous packet from the server.

Why would you want that?
TCP guarantees that not a single byte will be delivered out-of-order.
The bytes that you collect by calling FreeRTOS_recv() come in exactly the same order as they were sent.
If your end-user suspects that something is wrong with the contents, I think that other things should be checked, the sources being sent, the writing to FLASH.

We have tested +TCP through the Internet. An embedded HTTP server was running in Asia. Connecting to that server (from the UK) went slow, but it never gave any wrong data.

been off in DALI land!

What do you mean, Salvador Dali? Figueras or Cadaques?

Regards,
Hein

5

dibosco wrote on Thursday, February 26, 2015:

Hi Hein,

It is, indeed, a POST command the browser is sending to the FreeRTOS server.

"Why would you want that? "

I seem to remember from university days way back in the Protozoic era that when sending a file across the Internet you could not guarantee that packets arrive in the order they were sent because, in theory, different packets could take different routes.

I also assumed that if you are sending a file over a LAN, then the packets were bound to arrive in the order they sent, but the people at Siemens are claiming this cannot be guaranteed as the switches they use do not guarantee packets going out in the order they were received. They said that this could be solved by setting some sort of “Window Size” (they weren’t sure of the correct terminology) whereby you ensure that the established connection between server and browser would only allow a packet to be sent once the ACK for the previous packet had been received.

Now, if you’re saying it’s nonsense that a switch would not send out packets in the order it was received and with your previous message about FreeRTOS_recv() always passing the data in the order received, there is only one further concern in that this particular aspect of the transfer and that is:

Does the browser itself always send a file starting with the first byte and ending with the last byte and every byte in between being sent out in the exact order of the file. I realise that is totally out of the scope of FreeRTOS, but maybe you happened to know this. I retied asking this question on Stack Overflow, but didn’t even get one single reply!

One important thing to note here is that the browser is sending a new firmware image to write to flash when there is a need for software upgrade in the field. My web server will take the new file in (which will be a slightly modified Intel hex file), erase and program a different area of flash then boot to that new area once the file has been received and verified. This is why I am so concerned about information being sent and received in exactly the right order. If I had lots of RAM I’d just receive the file in any order and assemble it in RAM before writing it to flash. (Not that I am clear on how the TCP/IP stack would inform you that you are receiving packet x of y.)

One potential further issue I have is that writing the a packet of the new image to flash takes a a relatively long time. Am I right in assuming that if the buffer for the socket between FreeRTOS and the browser is full, the stack will just keep the browser waiting, eventually sending an ACK to say, OK, send me the next packet when it has room to receive it.

Phew, hope all that is clear!

DALI land means that I am writing code for a Digital Addressable Light Interface! There are no melted clocks anywhere to be found. ;~)

Cheers,

Rob

6

heinbali01 wrote on Friday, February 27, 2015:

Hi Rob,

This page explains a lot:

Phew, hope all that is clear!

Yes it is all clear.

You and Siemens are right: when packets are sent as “a,b,c”, it may happen that they arrive as “c,a” and that “b” gets lost. On a LAN it would be rare but it is possible. Within each packet we assume data integrity: routers and switches will add Ethernet checksums and the two end-points (browser and embedded device) will also set and check checksums (IPv4 and protocol CRC’s).

If you are sending firmware, it would not hurt to add a proper 32-bit CRC, along with length info.

Siemens is also right that if you use a small-enough TCP window size, all segments will be transmitted one-by-one. It can be done but I don’t think it is necessary and it is inefficient (= slow).

When “a” is expected and “c” arrives first, “c” will be acknowledged and stored in the RX-stream. “c” will be kept hidden until both “a” and “b” have been received. So TCP does the re-ordering for you!
If “h” is received it will be refused because it is out-of-sequence, it falls outside the TCP Window size.

How to set the TCP Window Size and buffer sizes in +TCP:

/* Define in your 'FreeRTOSIPConfig.h' the following
 * Maximum Segment Size (MSS) for TCP.
 * With this define, the TCP payload will carry at most
 * 1024 bytes.
 */

#define ipconfigTCP_MSS     1024

/* After opening the listening socket (port 80), set these
 * default properties: */

static void vSetWindowSize( xSocket_t xSocket )
{
xWinProperties_t xWinProps;

    memset( &xWinProps, '\0', sizeof xWinProps );
    /* Properties for transmission, not important now. */
    xWinProps.lTxBufSize = 2 * ipconfigTCP_MSS;
    xWinProps.lTxWinSize = 2;
    /* The size of the RX stream buffer in bytes: */
    xWinProps.lRxBufSize = 2 * ipconfigTCP_MSS;
    /* Siemens's proposal: make the window size of reception
    small. The number '1' means 1 * MSS */
    xWinProps.lRxWinSize = 1;
}

Setting ‘ipconfigTCP_MSS’ is only a proposal, the real MSS will be negotiated in the SYN phase. The party proposing the smallest MSS will always win.

An MSS of 1024 is probably ideal for writing to flash. If you receive chunks of 1460 bytes you can write 1 KB and keep 436 bytes for the next time.

Slowness when the TCP Window Size (WIN) is small:

Normally (and officially) sending packets one-by-one is extremely slow.
An ACK shall be postponed for the duration of 200 up to 500 ms or more unless:

  • this is a repeated segment
  • the reception buffer is full
  • there already was a previous unacknowledged packet

When +TCP receives data it is a bit more relaxed about these rules in order to get a higher throughput.

PS. I tried to attach some nice PCAP files as examples in an earlier post but the post was not accepted. I try again later.

Regards,
Hein

7

heinbali01 wrote on Friday, February 27, 2015:

Hi Rob,

Once again, now to show some PCAP files.

I uploaded a 50KB PNG file four times to a +TCP embedded device and attached a ZIP file in which you find:

mss_1_1024.pcapng
mss_1_1024_slow.pcapng
mss_2_1024.pcapng
mss_8_1024.pcapng

Where:

192.168.2.3   = laptop
192.168.2.100 = embedded +TCP device

The two examples “mss_1_xxx” have a Window Size of 1024 bytes, just as big as the MSS of 1024.
mss_2_1024 has a Window Size of 2 x 1024, mss_8_1024 uses 8 * 1024 bytes.

“mss_1_1024_slow” shows exactly what will happen if you have slow writes to FLASH.
After every 10th packet, you see a delay of 100 ms:

- ACK + WIN=0
- Writing to flash, taking 100 ms
- ACK + WIN=1024

The browser will understand the “WIN=0” and be patient.
If writing to flash takes too long, the other party may start sending N=1 (N=2, N=4) packets, just to probe / test if the “WIN=0” was a mistake.

Regards,
Hein

8

dibosco wrote on Friday, February 27, 2015:

Hi Hein,

Thanks for that, the stack takes care of more than I realised, so that is excellent. I had not realised that it would resequence packets before handing it to my thread.

I do have a checksum at the end of each line of sixteen bytes of data in my modified Intel hex file. Admittedly, they are not proper CRCs, but just the same type as a normal Intel hex file.

I am happy that what I have will be reliable. Many thanks for your patience and clear explanations.

Cheers,

Rob

9

heinbali01 wrote on Friday, February 27, 2015:

Admittedly, they are not proper CRCs, but just the same type
as a normal Intel hex file.

It is firmware, any wrong bit may make your device unreachable for all further upgrades :slight_smile:

Here is a safe one that I recently wrote:

static portINLINE void vInitialiseCrc32( uint32_t *pulCRC32 )
{
    *pulCRC32 = ~0ul;
}

void vCalculateCrc32( const uint8_t *ucData, int iLength, uint32_t *pulCRC32 )
{
unsigned uBitNr;
const uint32_t ulPolynomial = 0xEDB88320;
const uint8_t *pucCurrent = ucData;
const uint8_t *pucLast = pucCurrent + iLength;
uint32_t ulCRC32 = *pulCRC32;

    /* Calculate  normal CRC32 */
    while( pucCurrent < pucLast )
    {
        ulCRC32 ^= *( pucCurrent++ );
        for( uBitNr = 0; uBitNr < 8u; uBitNr++ )
        {
            if( ( ulCRC32 & 1 ) != 0 )
            {
                ulCRC32 = ( ulCRC32 >> 1 ) ^ ulPolynomial;
            }
            else
            {
                ulCRC32 >>= 1;
            }
        }
    }
    /* Copy it back to the parameter */
    *pulCRC32 = ulCRC32;
}

Regards.