FreeRTOS TCP keep-alive under heavy load

Kernel
1

Hello, I have a Zynq board with one of the processor running FreeRTOS+TCP. It communicates over both UDP (for sending raw data) and TCP. The keep-alive feature is also enabled.

I noticed that under heavy loads of UDP data (~400 Mbps), sometimes (I believe) the keep-alive packet gets lost or rather the sequence number of the keep-alive packet is off from the previous TCP packet’s sequence number. For example, see packet #118, 157, 210 of the wireshark capture: https://1drv.ms/u/s!ApMttKXHCCb9gR8kbCYtNZ1COzag?e=gBGsyA (sorry, it doesn’t seem like I am able to upload files to FreeRTOS atm).

This isn’t a big deal to me, but I’m wondering if this is a bug? Or if this has happened to anyone else before? The board is running on FreeRTOS v9.0.0

2

I see the FreeRTOS version number in your post - can you please also let us know the TCP stack version number - it should be at the top of each source file. From the FreeRTOS version number I suspect you are using an old stack version.

3

There was a bug in earlier versions of the Xilinx/Zynq driver, that is repaired in PR #1798

This PR has not been merged yet.

You can see the driver here

The bugs had to do with updating peripheral registers: it was necessary to read-back the register after writing to it, which works like a memory barrier.

In case you are using xil_printf, have a look at this change in FreeRTOSIPConfig.h:

#define configPAUSE_AFTER_XIL_PRINTF_MSEC	20

#define configPRINT_STRING( X ) \
	xil_printf( X ); \
	vTaskDelay( pdMS_TO_TICKS( configPAUSE_AFTER_XIL_PRINTF_MSEC ) )

Without the delay, lots of logging messages get lost. The delay takes place in the logging task, so there is no performance loss.

4

Sorry, I forgot to tell that you can find more information about the PR in its predecessor PR #1628

Especially these:

  • In emacps_send_handler() : read-back the XEMACPS_TXSR registers in ISR context after clearing it
  • In emacps_send_message() : read-back the XEMACPS_NWCTRL register after setting it

I hope that this will solve the problem with keep-alive packets.

5

TCP version is FreeRTOS+TCP Labs Build 160919

I tried applying the patch on PR1798/1628. Specifically, I added the calls to read back the registers in:

  • emacps_send_handler(): ( void ) XEmacPs_ReadReg(xemacpsif->emacps.Config.BaseAddress, XEMACPS_TXSR_OFFSET);
  • emacps_send_message(): ( void ) XEmacPs_ReadReg( ulBaseAddress, XEMACPS_NWCTRL_OFFSET );

However, I still see the problem persist. Perhaps I will try updating to the latest FreeRTOS… there seems to be quite a few patches to the TCP library for Xilinx.

For the time-being, I can disable keep-alive as the application regularly sends it’s own keep-alive packet as well.

6

I added the calls to read back the registers

Please use the entire new driver with all source files from the Zynq directory.

What I observed is that sending packets sometimes failed when there was a continuous stream of incoming packets. The new version doesn’t have this problem. I tested this with IPERF3, running it in both directions.

I will try updating to the latest FreeRTOS

It is always good to use the latest kernel version, but it won’t help for the failing transmissions.

as the application regularly sends it’s own keep-alive packet as well

I always recommend using a high-level way of sending are-you-alive packets. When they are responded, it means that the counter application is also still alive, not only the TCP/IP driver.

Hein

7

Hi Hein,
I upgraded to the latest version of FreeRTOS/FreeRTOS+TCP and applied PR1798 and ran the system over night. I did not observe any problems related to keep-alive packets.

I was wondering what’s the timeline like for having PR1798 merged into main?

Thanks