Unexplained performance drop when memcpy called

Kernel
1

Hello All,

I am encountering an anomaly with the RTOS kernel that has totally baffled me. I am working on a Cortex M7 processor using LWIP sockets and a buffer library that precludes the possibility of concurrent buffer modification. I am highly confident that all of this code is working flawlessly. memcpy is used throughout the code and everything works exactly as it should until I send an instruction for the processor to store a data block in flash.
This performance drop can be seen best from the Wireshark trace. The red line is the rate at which the processor is polled and the grey line represents the rate at which the processor responds to polls.

image
image328×1132 17 KB

Initially, the response rate is equal (or close to) to the poll rate as it should always be. However, after each command to save its configuration settings in flash the response rate sharply declines and then stabilizes at a lower rate! I am totally baffled by this.

When the memory block is written to flash, a few dozen memcpy operations are performed to generate the stash and write page buffers that are then saved to flash. The block of data being written to flash is ~5 kB. All of the memcpy calls and the writing of the buffer to flash are performed by the same thread. An additional mutex is used on the flash level interface to prevent concurrent access to the flash controller or it’s set of memory.

All tasks involved in memory transfer have a floating-point context
and configUSE_TASK_FPU_SUPPORT is set to 0.

An interesting note? placing a taskENTER_CRITICAL()-taskEXIT_CRITICAL() around the write function often results in a hard fault.

Can anyone explain this type of behavior, what’s causing it, or how it can be avoided?

2

I would start by isolating the smallest piece of code that exhibits the issue. For example, you start talking about lwIP, but I’m not sure that is relevant to the issue, so try removing the lwIP functionality and see if the issue still occurs. Likewise, is FreeRTOS even relevant to the issue? Try removing that too so you are performing the flash writes and performance measurements from main(). If you can determine if there is anything specific that is causing the issue report back. You may also want try the same as you have above, but the the M7’s caches turned off.

3

Thanks For the reply,

I reworked the buffer handling and thankfully this behavior went away. It looks like this was a simple software bug, and not anything to do with RTOS. I got hung up on this bug for a couple of days and it was driving me nuts! In hindsight, the poll-response rate dropping by an integer number of pkts/sec (as seen in trace) should have been a red flag to check the buffer handling.