TCP stack performance

Kernel
1

joehinkle wrote on Wednesday, March 29, 2017:

I’m making a TCP connection with a web server and using GET to download an 11.3 meg file.

It takes 670 secs to download - or 16.865KB per second.

That appears awfully slow but I have no reference to properly gauge the performance.

NXP - Kinetis K64 ARM m4 cpu running at 120 MHZ.

Freertos for the RTOS using 5msec tics.

Freertos TCP stack.

Socket usues a 1K buffer to acquire the data.

Stack’s thread is hightest priority with my download thread just below it.

Is 16.9KB an acceptable download speed?

If that performance is very low as I suspect it is, any suggestions as to how to improve it?

Thanks.

Joe

2

joehinkle wrote on Wednesday, March 29, 2017:

One other implementation fact I forgot to mention. My network buffers are 700 bytes in size.

I can’t make them any larger due to ram requirements.

This test is running with the stack having access to 165 network buffers.

The size of the buffers and the number of them is dictated by another requirement.

Thanks.

3

heinbali01 wrote on Thursday, March 30, 2017:

Hi Joe,
Please have a look at my recent post and also the PCAP’s in the attachment.
How much RAM do you have available for +TCP? Could you post the FreeRTOSIPConfig.h that you’re using?
Would you mind to post a zipped PCAP of a current sessions. One second is enough, filtered on e.g. “tcp.port==80” or so. Regards.

4

heinbali01 wrote on Thursday, March 30, 2017:

Hi Joe, if I understand it well, in the PCAP that you emailed me, a host is sending a file to your +TCP device.

Your device advertised a TCP window size of 1320. The MSS is 660 bytes. All perfect.

But the big delays are found between the +TCP ACK and the reception of new data: about 87 ms. After that, two blocks of 660 bytes are received and an ACK is sent immediately.

You’re using these IP addresses: a private 192.168.1.21 and a public IP address 173.x.x.x
This connection goes through the Internet? If that is true, then a 90 ms delay is ‘normal’ and only more buffer space can help you out. Unless you have a low bandwidth, then the case would be hopeless :slight_smile:

But I guess that you have enough bandwitdth, it only has long delays.

Is your +TCP device mostly receiving large files, or also sending?

I assume you have seen the socket option FREERTOS_SO_WIN_PROPERTIES ?

It allows you to set 4 properties of a socket. These must be set before communication starts.

typedef struct xWIN_PROPS {
	/* Properties of the Tx buffer and Tx window */
	int32_t lTxBufSize;	/* Unit: bytes */
	int32_t lTxWinSize;	/* Unit: MSS */

	/* Properties of the Rx buffer and Rx window */
	int32_t lRxBufSize;	/* Unit: bytes */
	int32_t lRxWinSize;	/* Unit: MSS */
} WinProperties_t;

As a test you could set them like this:

xProperties.lTxBufSize = 6 * ipconfigTCP_MSS;
xProperties.lTxWinSize = 4;
xProperties.lRxBufSize = 6 * ipconfigTCP_MSS;
xProperties.lRxWinSize = 4;

or more, depending on how much RAM you have available.

Also augmenting the value of MTU would increase the performance, but that would be costly in terms of RAM usage.

Regards, Hein

5

joehinkle wrote on Friday, March 31, 2017:

Thanks Hein

I like the way your stack performs.

Joe

6

joehinkle wrote on Friday, March 31, 2017:

Hein:

Thanks for suggesting the WinProperties_t.

Here are some test results.

Communication is based on a 10 Mbit stream. I have to use a 10Mbit hub to allow my computer to wireshark and capture the package going to the embedded cpu.

My test file is 11.1 meg in size.

My default setup (don’t know the the default xProperties of the socket) but I used the following in the config file

#define ipconfigTCP_RX_BUFFER_LENGTH (4 * ipconfigTCP_MSS)
#define ipconfigTCP_TX_BUFFER_LENGTH (4 * ipconfigTCP_MSS)
#define ipconfigTCP_WIN_SEG_COUNT 12

I am downloading the file and writing it out to a SD card.

Time to complete is 800 seconds.

I can’t change the MSS size but I did change the following properties for THAT specifc socket:

xProperties.lRxBufSize = 20 * ipconfigTCP_MSS;
xProperties.lRxWinSize = 20;

Download time (and SD writing) was reduced to 110 seconds – so dare close to a times 8 improvment.

Great help on your part.

Thanks.

Joe

7

heinbali01 wrote on Saturday, April 01, 2017:

Hi Joe,

( I am truly sorry to be lengthy again … )

My default setup (don’t know the the default xProperties of the socket)

The defaults of the xProperties of a socket are indeed taken from these two macro’s:

    /* Defines the default size of the TX stream buffer. */
    #define ipconfigTCP_TX_BUFFER_LENGTH (4 * ipconfigTCP_MSS)

    /* Defines the default size of the RX stream buffer. */
    #define ipconfigTCP_RX_BUFFER_LENGTH (4 * ipconfigTCP_MSS)

ipconfigTCP_MSS equals ipconfigNETWORK_MTU, minus the overhead of a TCP package.

These are the default values:

    xProperties.lTxBufSize = ipconfigTCP_TX_BUFFER_LENGTH;
    xProperties.lTxWinSize = ipconfigTCP_TX_BUFFER_LENGTH / ipconfigTCP_MSS;
    xProperties.lRxBufSize = ipconfigTCP_RX_BUFFER_LENGTH;
    xProperties.lRxWinSize = ipconfigTCP_RX_BUFFER_LENGTH / ipconfigTCP_MSS;

The driver will make sure that lTxWinSize and lRxWinSize are at least 1.
Also, lTxBufSize will always be a multiple of MSS, and if necessary, it will be rounded-up.

In case no TCP sliding windows are used ( FREERTOS_SO_WIN_PROPERTIES = 0 ), the actual window size will be one MSS.

The following macro is something different:

#define ipconfigTCP_WIN_SEG_COUNT    12

It says that all TCP sockets together ( at any time ) may have a total of 12 outstanding ( i.e. unconfirmed ) packets. It determines the number of buffers ( of type TCPSegment_t ) that are allocated in FreeRTOS_TCP_WIN.c.

I am downloading the file and writing it out to a SD card.

Time to complete is 800 seconds.

I can’t change the MSS size but I did change the following properties for THAT specific socket:

xProperties.lRxBufSize = 20 * ipconfigTCP_MSS;
xProperties.lRxWinSize = 20;

Download time (and SD writing) was reduced to 110 seconds – so dare close to a times 8 improvement.

That is good news! But maybe you get the same performance with 10. In a PCAP you can count the actual number of outstanding packets. It is indicated as:

    TCP -> [SEQ/ACK analysis] -> [Bytes in flight 16060]`.

In my example, there are at most 16060 bytes (11 times MSS) outstanding without an ACK.

About getting UDP faster using an application-hook :

Have you used FREERTOS_SO_UDP_RECV_HANDLER already?
The UDP messages can be delivered to your application with this function:

BaseType_t xOnUDPReceive(
    Socket_t xSocket,
    void * pvData,
    size_t uxLength,
    const struct freertos_sockaddr * pxFrom,
    const struct freertos_sockaddr * pxDestination );

The address of your function can be bound to a socket by calling FreeRTOS_setsockopt() with ‘FREERTOS_SO_UDP_RECV_HANDLER’.

The function must return non-zero to tell the stack that it may discard the packet.

When using this call-back, time is saved: your application task doesn’t not need to become active.
The call-back should be fast though (do not block or wait), because it is called from the IP-task.
Regards.