FreeRTOS+TCP: socket stops sending even though there is enough space in the window

rbos36 wrote on Thursday, January 11, 2018:

Hi, we have a pressing question about the FreeRTOS+TCP stack behaviour.

In our situation, we are sending data via TCP/IP from an embedded device to an Atom-based Linux machine.

The embedded device listens on a socket until the Linux machine connects, and then starts to send data chunks of around 1454 bytes every 4ms.

Using FREERTOS_SO_WIN_PROPERTIES, we set TxBufSize and TxWinSize to 26 * MSS (I’ll post a code snippet below).

However, in practice, the embedded device only sends 2 frames and then waits for an ACK to come in from the Linux machine before commencing, even if the advertised window has plenty of space.

What could be the cause?

Wireshark screen-shot; .89 is the embedded device ( see attachment )

Note that in the ~25ms before the ACK came in (highlighted row), the embedded device could have sent 25/4 = around 6 packets more, but it only sends two packets.

Here is the relevant code that creates the socket on the embedded device; do we set the FREERTOS_SO_WIN_PROPERTIES at the right moment?

static Socket_t AcceptDataConnection(void)
    static const TickType_t ACCEPT_TIME_OUT = portMAX_DELAY;
    static const TickType_t RECV_TIME_OUT   = 100 / portTICK_PERIOD_MS;
    static const TickType_t SEND_TIME_OUT   = 100 / portTICK_PERIOD_MS;
    const BaseType_t        TRUE            = pdTRUE;
    const BaseType_t        BACKLOG         = 0;
    const uint16_t          PORT_NUMBER     = DATA_PORT;

    struct freertos_sockaddr bindAddress, clientAddress;
    Socket_t                 listeningSocket, connectedSocket;
    socklen_t                clientAddressSize = sizeof(clientAddress);

    TlAssert(listeningSocket != FREERTOS_INVALID_SOCKET);

    /* Set a time out so accept() will just wait for a connection. */
    FreeRTOS_setsockopt(listeningSocket, 0, FREERTOS_SO_RCVTIMEO, &ACCEPT_TIME_OUT, sizeof(ACCEPT_TIME_OUT));

    /* We only accept one simultaneous connection on the data port */
    FreeRTOS_setsockopt(listeningSocket, 0, FREERTOS_SO_REUSE_LISTEN_SOCKET, &TRUE, sizeof(TRUE));

    /* Bind the socket to the port that the gateway will connect to, then listen for incoming connections. */
    bindAddress.sin_port = PORT_NUMBER;
    bindAddress.sin_port = FreeRTOS_htons(bindAddress.sin_port);
    FreeRTOS_bind(listeningSocket, &bindAddress, sizeof(bindAddress));
    FreeRTOS_listen(listeningSocket, BACKLOG);

    DBG("Listening for data connection on port %u" EOL, PORT_NUMBER);

    /* Wait for a client to connect. */
    connectedSocket = FreeRTOS_accept(listeningSocket, &clientAddress, &clientAddressSize);
    TlAssert(connectedSocket != FREERTOS_INVALID_SOCKET);

    /* Set socket timeouts */
    FreeRTOS_setsockopt(connectedSocket, 0, FREERTOS_SO_RCVTIMEO, &RECV_TIME_OUT, 0);
    FreeRTOS_setsockopt(connectedSocket, 0, FREERTOS_SO_SNDTIMEO, &SEND_TIME_OUT, 0);

    /* Configure sliding window */
    WinProperties_t xWinProps;
    xWinProps.lTxBufSize = 26 * ipconfigTCP_MSS;    /* Unit: bytes */
    xWinProps.lTxWinSize = 26;                      /* Unit: MSS */
    xWinProps.lRxBufSize = 4 * ipconfigTCP_MSS;     /* Unit: bytes */
    xWinProps.lRxWinSize = 2;                       /* Unit: MSS */
    FreeRTOS_setsockopt(connectedSocket, 0, FREERTOS_SO_WIN_PROPERTIES, (void *) &xWinProps, sizeof(xWinProps));

    DBG("New client connected to data port, starting acquisition" EOL);
    return connectedSocket;


heinbali01 wrote on Thursday, January 11, 2018:

FreeRTOS_setsockopt(connectedSocket, 0, FREERTOS_SO_WIN_PROPERTIES,
(void *) &xWinProps, sizeof(xWinProps));

Hi Ronald, you are setting the TCP-Window properties of the “connected socket”. I think that is too late, because at that moment the connection has already been established, and the dimensions of the TCP-window and buffers have already been established.

So your connected socket will take the system default for the TCP-window size, which is probably a lot smaller ( 2 x MSS ? ).

I attached a file in which the order is changed: FREERTOS_SO_WIN_PROPERTIES will be applied to the listening socket, before calling accept(). I think that will solve the problem.

Remember that a child TCP-socket inherits all properties of the parent ( = listening ) socket.

The time-outs (FREERTOS_SO_RCVTIMEO and FREERTOS_SO_SNDTIMEO) can be changed at any moment, also after connecting.

It is nice to see that you are using the FREERTOS_SO_REUSE_LISTEN_SOCKET option. With this option, the listening socket will turn into a connected socket. The two sockets refer to the same object, so make sure that in the end, you either close listeningSocket, or connectedSocket.
The advantage of the re-use option is efficiency: no need to allocate a new socket at the moment a connection comes in. It is only used in combination with passive connections ( i.e. using accept() ).
Without the re-use option, a listening socket will stay in its listening state and keep on accepting connection until a maximum (see backlog parameter to FreeRTOS_listen()) has been reached. As long as the maximum has been reached, the stack will respond to connection attempts with a RST packet.

rbos36 wrote on Thursday, January 11, 2018:

Hi Hein,

We changed the setting of FREERTOS_SO_WIN_PROPERTIES from the client socket to the listening socket, and now indeed the embedded device keeps on sending data until the advertised window size is reached.

Thanks a lot for the quick response! We are very happy to have a fully working system now. We will contact you off-line for a more material thank-you :wink:

Kind regards,