FreeRTOS+TCP Heap Consumption

wavsse wrote on Wednesday, September 05, 2018:

I’m relatively new to the +TCP stack and while I’m having good success, I would welcome any advice or information regarding how +TCP manages heap allocation and release. I have both ipconfigZERO_COPY_TX_DRIVER and ipconfigZERO_COPY_RX_DRIVER set to 1, and I’m using BufferAllocation_1 and, of course, heap4.

At present, the +TCP stack seems to be working well on the surface as I’m able to send and receive UDP packets, and I’ve also implemented and tested a simple HTTP server … thus TCP is working as well.

For diagnostic purposes, I have a console readout that lets me inspect the heap using xPortGetFreeHeapSize() and xPortGetMinimumEverFreeHeapSize().

At startup, my heap consumption is around 29K ( I have several dynamically intantiated tasks that remain intact ) and all runs well. When I retrieve a simple web page using a browser, however, the heap consumption rises to ~45K (expected) … but then never goes back down again. After the HTTP transactions, I’m properly closing all involved sockets. I can retrieve the web page a 2nd time and it doesn’t climb any higher than the ~45K, so there’s no memory leak.

The last time I had +TCP diagnostic messages enabled, I didn’t notice any unusual consumption (leaks) in the buffer descriptors. The count rises with activity, as expected, but remains stable.

**What I’d like to understand better is exactly how and what the ZERO_COPY mechanism does, and how it impacts the heap and what sort of heap behavior is expected. In general, I need help with a bit deeper dive into the inner workings of the +TCP memory management so I can use more than superficial “appearance” for verifying my code integrity. I don’t want to create code that may be configured wrong or unnecessarily impacts TCP performance. **

I realize I may need to provide more configuration details, so please ask. It wont be possible to provide all my code involved as there’s simply too much, but I can post relevant snippets if required.

Thanks for any advice or help.

heinbali01 wrote on Wednesday, September 05, 2018:

Roughly spoken, there are two phases during which heap memory is allocated and that memory is never released.

  1. During FreeRTOS_IPInit(), several system resources are allocated.
  2. When the first TCP connection is established, some buffers are allocated in FreeRTOS_TCP_WIN.c:
    xTCPSegments = ( TCPSegment_t * ) pvPortMallocLarge( ipconfigTCP_WIN_SEG_COUNT * sizeof( xTCPSegments[ 0 ] ) );

Please check the value of ipconfigTCP_WIN_SEG_COUNT. It defines the maximum number of outstanding packets ( both RX and TX ) at any moment in time. The default of 256 is a bit high, 32 would be OK for a small device with few “concurrent” TCP connections.
One TCPSegment_t is about 64 bytes, so by default about 16 KB is being allocated.

A TCP socket will allocate heap memory when created. Beside that, stream buffers will be created when the first bytes are being sent or received.
An UDP socket doesn’t need a stream buffer. It will store the receive packets in buffers of the type NetworkBufferDescriptor_t.
All memory allocated by a socket will be released when FreeRTOS_closesocket() is called.

When using the copy method, both for RX and TX, a number of buffers are usually declared as static space.

So the linker will reserve for instance:

   For RX : 10 x 1536 bytes
   For TX :  2 x 1536 bytes
            12 x 1536 = 18432 bytes

When using the zero-copy method, the memory costs are about the same:

   For RX : 10 Network buffers
   For TX :  0
            10 Network buffers

When receiving packets, DMA will write directly into Network buffer. When sending packets, DMA will read from a Network buffer.

As you probably know:

BufferAllocation_1.c declares the space for Network buffers in static arrays.
BufferAllocation_2.c is more suitable for small memory: it allocates the space dynamically.

I realize I may need to provide more configuration details, so please ask

I won’t ask much: just the type of CPU that you are using and your FreeRTOSIPConfig.h please

wavsse wrote on Wednesday, September 05, 2018:


Thanks for the detailed reply. My ipconfigTCP_WIN_SEG_COUNT is in fact set to 256. Your explanation makes sense, and probably accounts for the additional ~16K of heap consumption I’m seeing. Not sure just yet how much I’m going to reduce that as my product requirement is calling for potentially multiple TCP connections. Just how many … I haven’t yet won that battle.

The CPU we’re using is an STM32F746 with 320K of RAM, so we’re not really pinched in this respect.

Attached is my FreeRTOSIPConfig.h, and thanks for your help.