Memory management setup incompatibilities between MCUXpresso IDE, GNU C/Linker for ARM, and FreeRTOS

Partners & Sponsors NXP
1

Dear developers in charge of
making MCUXpresso IDE, GNU C/Linker for ARM, and FreeRTOS
play nice with each other:

As a physicist / mixed signal analog IC design PhD,
I do not have, nor do I really want to have, detailed knowledge of the inner workings of FreeRTOS, MCUXpresso, and GNU linker.

I need to put an existing code base for the MIMXRT1170-EVK
(and later on, the MIMXRT1176DVMAA with non-standard hardware extensions)
with lots of shortcuts and workarounds
on a solid foundation, so I can modify the legacy code to use FreeRTOS reliably and consistently.

At the moment, I am trying to modify an existing project
to use one of the FreeRTOS memory management schemes (preferrably, heap4) consistently for implementing lots of queues with variable size, sometimes large (u-blox binary) data packets,
and large matrices (even tensors) for doing linear algebra.

I’m running into serious problems how to modify an existing project
or the example projects from NXP, like evkmimxrt1170_freertos_queue_cm7,
to allocate the FreeRTOS heap (heap4, preferrably)
into 16777216 bytes of memory in external SDRAM section SDRAM_HEAP, see

Screenshot 2022-11-23 051046
Screenshot 2022-11-23 0510461188Ă—792 176 KB

Due to subtle incompatibilities or undocumented implicit assumptions by FreeRTOS,
which of several possible memory allocation schemes in MCUXpresso / GNU C compiler and linker are supposed to be used,

I do not get the FreeRTOS heap allocated into SDRAM_HEAP and properly recognized by FreeRTOS, let alone the FreeRTOS diagnostics windows in MCUXpresso

  • Task List
  • Queue List
  • Timer List
  • Heap Usage

Through trial and error, I got the FreeRTOS MCUXpresso diagnostics windows to work under Windows, but only from time to time under ubuntu 20.04LTS Linux,
but I cannot reproduce what works and what doesn’t.

In discussion forums (FreeRTOS? NXP??),
I expect to find a lot of patches from typical “move fast and break things” that happened to not break immediately,
but can I find someone here
who can tell me how to making FreeRTOS in NXP MCUXpresso based on the GNU compiler/linker under both Linux and Windows
is supposed to work,
i.e., not to meddle with GNU linker scripts that get automatically overwritten by either FreeRTOS or MCUXpresso,
and get the “move fast and break things” legacy code configured as intended by the designers?

The “pass the bucket to the linker” statement
How the array is placed at a specific memory location is dependent on the compiler being used - refer to your compiler’s documentation.
uint8_t ucHeap[ configTOTAL_HEAP_SIZE ];
in the FreeRTOS
APPLICATION_ALLOCATED_HEAP
documentation
*(posting the link to *
the configAPPLICATION_ALLOCATED_HEAP manual page
is verboten for a newcomer who just needs one specific question answered)
opens a Pandora’s box of subtle interfacing problems,
and when I follow any of the manuals of any one tool,
I’m playing whack-a-mole:
fixing a problem according to the documentation of one tool creates a new problem in the other tool.

I hope that sorting this out is not a Recreational Impossibility,
and I can get an answer by the actual designers of the FreeRTOS/MCUXpresso interface.

Thanks,
Christoph Maier

2

Hi and welcome to the forum, Christoph!

I am afraid that you are asking for a red herring, at least weth respect to the linker. Attempting to understand, let alone implement, RTOS (any, really) based systems without knowlege of memory layout to me sounds like a lot to ask for.

About the IDE, I agree, but since no tool is bug free nor ultimately intuitive, you will need some degree of digging into it. And FreeRTOS - well, you do not need to study the source code in detail, but you should read the documentation carefully because FreeRTOS provides an infra structure to build your embedded system on, so especially if you have next to hard realtime requirements, you will need understand how OS timing interacrts with hardware anmd software timing requirements.

So what exactly is your question?

Cheers Ruediger

3

As far as I understood @tatzelbrumm want’s to place the raw and pretty large heap_4 buffer into external SDRAM. I’ve no idea about MCUXPRESSO but finally the memory regions including the external SDRAM are defined in the linker script along with a dedicated section in the external SDRAM memory region to place the heap buffer into using e.g.

uint8_t ucHeap[ configTOTAL_HEAP_SIZE ] __attribute__ ((section ("exsdheap")));

using the GNU tool chain and enabling configAPPLICATION_ALLOCATED_HEAP.

4

Hi,
these were unexpectedly quick answers,
and @hs2 's is on the spot.
Yes, I want to place the Heap #4 buffer into SDRAM_HEAP.

If

uint8_t ucHeap[ configTOTAL_HEAP_SIZE ] __attribute__ ((section ("SDRAM_HEAP")));

would actually work, that would be a solution.
However, it doesn’t.

By using

#include <cr_section_macros.h>
__BSS(SDRAM_HEAP) uint8_t ucHeap[configTOTAL_HEAP_SIZE];

as recommended by the NXP documentation,
I succeed to place Heap #4 there,
but FreeRTOS doesn’t seem to find it at run time;
xQueueCreate() returns NULL,
and xTaskCreate() also hangs up.

Screenshot 2022-11-26 085721
Screenshot 2022-11-26 0857211920Ă—780 119 KB

(I’ll have to post a follow-up with the screenshot without heap reallocation,
because the FreeRTOS forum administrators believe that restricting newcomers to one screenshot per post is compatible with respecting participants)

I suspect that FreeRTOS makes some undocumented, implicit assumption which of the various different methods to assign a variable to a specific address or section should be used,

and neither the default assumption of the GNU Linker
(assign a pointer to a specific address and recast: tried it, heap size is sizeof(uint8_t*) )

nor the default assumption of MCUXpresso
(#include <cr_section_macros.h>; tried it, allocates correctly, but FreeRTOS runtime doesn’t recognize it)

work.

My question to @RAc would be:

I want to use both the NXP IDE and FreeRTOS specifically so I don’t have to reverse engineer how these tools are supposed to interact with each other,
according to the undocumented, implicit assumptions of their developers.

I wouldn’t even ask in a forum with its unfavorable signal-to-noise-and-distortion ratio if there wasn’t a gap between the documentation of the tools, where both manuals pass the buck by stating: “for interfacing details, refer to the documentation of the other tool.”

So the question would be:
How can I overcome the Betriebsblindheit of the tool developers, who implicitly assume that their job is done when they get their own tool to work consistently, but consider the smooth interplay with other tools in the chain “not my department”, by implicitly assuming that everyone else uses the particular choice among several possible linker commands that their FreeRTOS code accepts without problems.

Clear enough?

E>I
[The Zen of Python]
Christoph

5

And this is what the FreeRTOS heap monitor should look like
(built without explicitly placing the heap)

Screenshot 2022-11-26 083830
Screenshot 2022-11-26 0838301920Ă—780 130 KB

6

Hmm … FreeRTOS makes no special assumptions. heap_4 just uses the provided array of data as raw heap area. Can you simply try to pvPortMalloc some memory and put a breakpoint at this function ? You should halt in heap_4.c. The code is not overly complex and not very hard to step through to see what’s going wrong.
Also in the generated (?) map file when building the application you should find your global ucHeap symbol resp. the array within the external SDRAM address range with the desired size.
Assuming the section macro magic works as documented, of course :wink:

7

so we would need to see the linker command file to figure out where the linker tries to place SDRAM_HEAP. It looks as if there is no entry for explicit placement of the memory area (for Cortex MCUs, the address range 0x2000xxxx is by convention in the internal MCU RAM).

I do not know the tool set, so I can not point you to the place where you need to make the change to match the logical section name with the physical address range. But if you are familiar with Embedded Software design, this is one of the first things that you will recognize and be able to fix yourself.

You appear to expect a completly sanitized software architecture that is so well designed that everything works from scratch across all manufacturer and technology borders. Wouldn’t we all. Welcome to the real world.

8

I’d probably use a different allocator for sdram and leave freertos to use the (probably) faster internal ram. umm_malloc or similar.

Most freertos projects exist on internal ram, so when you want to branch out it’s not going to be completely straightforward, but that’s not really any one projects fault.

9

In defense of FreeRTOS, it doesn’t really do anything with the linker, it has more to do with the compiler and what the chip manufacturer has for tools.

Note that if you use sdram for the heap you will have to make sure the module is initialized before allowing any access to it.

1 Like
10

Well … I tried to adapt the
evkmimxrt1170_hello_world_cm7
example program
to use

#include <cr_section_macros.h>
__BSS(BOARD_SDRAM) uint8_t ucHeap[configTOTAL_HEAP_SIZE];

and whatever the compiler did with the FreeRTOS code bricked one of my evkmimxrt1170 boards after flashing (probably left it in low-power deep sleep mode from which I don’t know how to recover):

Probe Firmware: DAPLink CMSIS-DAP (ARM)
Serial Number:  02440000092d948400000000000000000000000097969905
VID:PID:  0D28:0204
USB Path: /dev/hidraw3
Using memory from core 0 after searching for a good core
connection failed - Em(04). Cannot provide power to DAP bus... Retrying
Failed on connect: Em(04). Cannot provide power to DAP bus.
Connected&Reset. Was: NotConnected. DpID: 6BA02477. CpuID: 00000000. Info: <None>
Last stub error 0: OK
Last sticky error: 0x0 AIndex: 0
Debug bus selected: MemAp 0
DAP Speed test unexecuted or failed
Debug protocol: SWD. RTCK: Disabled. Vector catch: Disabled.
(100) Target Connection Failed
error closing down debug session - Nn(05). Wire ACK Fault in DAP access
Unable to perform operation!
Command failed with exit code 1

I’m aware that making different layers of tools play together smoothly is a more difficult problem than any one software abstraction layer, because you need to think out of your own box, but if the end result is a bricked board, that defeats the purpose of the entire tool chain.

Any ideas how to get the MIMXRT1170-EVK board respond again and how to avoid such catastrophic failures in the future?

An instruction:
Do not use heap4 for external SDRAM
is ok if documented at a conspicuous place in the documentation.
Unfortunately, it isn’t.

Nearly everything that matters is a side effect. — Meredith L. Patterson

11

Sometimes you have to fully power cycle. I’m not familiar enough with nxp to know, also you could check for backup register clear jumpers and boot pins. I really can’t see a reason this is a fault of Freertos, it’s not typically responsible for managing power states unless a manufacturer or someone adds the code specifically.

12

I have power cycled, rebooted, changed host computer before I even wrote my last post.

You really can’t see that not leaking undocumented side effects into other parts of the software stack are of critical importance, because the side effects destabilize the rest of the code in seemingly unrelated places.

The best shot to solve this would be to discuss with the authors of heap4.c directly how they were thinking that their code is supposed to pull the necessary information from the code and the memory map of the compiler, and, much more importantly, what they weren’t thinking about (like, gracefully handling slower external memory).

This is all about unintended side effects.

13

I do understand that your experiences are frustrating, Christoph. But again the heap implementations like heap_4 are just about managing a piece of memory resp. an array either provided or statically defined internally. There is no magic.
The only requirement is that the provided address and size of the memory are valid in the sense that there is accessible memory/RAM. Otherwise … you know.
As seen in the source heap_4.c the internal heap management data is initialized with the 1st pvPortMalloc. So even when using external SDRAM, which probably needs some early low level HW init like setting up the external bus controller, there is no problem doing so until the heap is used by the application/FreeRTOS calls.

1 Like
14

I suspect the download, startup code, or something else to do with program loading is trying to use [edit - I originally incorrectly wrote “initialize” instead of “use”] the external RAM to clear the heap to zero (because you forced an array into it) before the external RAM is enabled - hence the crash. If that is the case using the NXP tools to clear/delete the Flash memory to stop it trying to do that on each boot will fix the issue. If that is the cause then it is a tools usage on the hardware issue, rather than anything to do with FreeRTOS. Likewise for putting the heap in external RAM - if that is too slow for the C stacks then that will be the case if you are using FreeRTOS or not - it’s not something you are going to find in the FreeRTOS documentation because it’s a limitation or running C on that hardware, not a FreeRTOS limitation. FreeRTOS is just C code.

FreeRTOS provides a way of putting the heap in slow RAM and the stacks in fast RAM but these things are more advanced features of FreeRTOS.

15

@rtel,
thanks for the explanation. (did you actually write the heap4 code?)

At the moment, I need to un-brick my eval board from low-power sleep mode, so I’m trying to sort that out with NXP tech support.
The side effects of using one tool on the other tools are always the most difficult problems — I don’t even know where to start looking.

Looking ahead, how big should queues and array storage under FreeRTOS heap management become, anyhow?
And where would I find the documentation for putting FreeRTOS stacks and heaps in different places, in case I want to use FreeRTOS memory management for everything, including allocating and reshaping O(10^2*10^2) floating point matrices?

16

Hi Richard,
you write:

Now that I successfully unbricked an 1176 EVK board and am thinking of allocating heaps and maybe queues too big for on-chip memory of MIMXRT 1176 and 1062 processors,
where in the FreeRTOS documentation should I start looking for these advanced features?
It’s probably important for me to figure out how to hold off any FreeRTOS access to external SDRAM until it has been properly configured by the lower layers of software (the GNU C memory allocation routines and the dedicated SDRAM configuration code, I presume).

Thanks,
Christoph

17

Until you call any FreeRTOS API it won’t do anything by itself.
C library heap (usually newlib) relies on custom/provided _sbrk() implementation returning memory blocks of requested size from raw heap memory.
The latter could be invoked early (before main()) in case of global/static constructors invoking new operator(s) when using C++.
Then you’d need to init the memory HW earlier ie. before __libc_init_array (in case of newlib) is invoked by startup code (e.g. in Reset_Handler on ARM Cortex-M MCUs).
Hmm… of course this would also apply to FreeRTOS API calls when used in global/static constructors.
I think the SDK provides some startup code setting up the clocks/wait states and maybe other low level HW init before branching to main()…

18

Richard probably refers to the xTaskCreateStatic function:

This page describes the FreeRTOS xTaskCreateStatic() API function which allows an RTOS task to be created using statically allocated RAM.

which allows you to explicitly specify the location of a tasks stack in memory.

19

Thanks, but that’s a workaround, not a solution for what I’m trying to do.
I’m dealing with legacy code that,
in order to circumvent low-level FreeRTOS configuration and initialization problems, uses FreeRTOS for tasks,
but haphazardly statically allocated global variables instead of queues, semaphores, and reconfigurable heap space within FreeRTOS.

I’m specifically trying to overcome the FreeRTOS usability obstacle that
when it comes to compatibility problems across software layers,
everyone tries to pass the buck to everyone else
and find patches and workarounds instead of solutions,
that
(through encouraging the use of global variables because FreeRTOS heap4 allocation in slow memory is in the intersection of the “Somebody Elses’s Problem” field of all the domain experts [FreeRTOS, GNU C, NXP])
makes FreeRTOS less than useless.

I want as much as possible of the external SDRAM space on the MIMXRT1170-EVK board under control of FreeRTOS heap4, without sending the processor to sleep because of configuration incompatibilities between software layers.

No static allocation workarounds.

Where in the documentation of FreeRTOS (or GNU C or NXP processors) do I have to look to find the solution, not patches and workarounds?

20

Independent from using FreeRTOS you/the SDK have to ensure that you can seamlessly access the external SDRAM before doing so.
Once that’s solved it’s just RAM, right ? Then also heap_4 will seamlessly work when providing a piece of the external SDRAM as raw heap memory.
There are simply no dark corners or undocumented things going on wrt. FreeRTOS at least.
Using the C library provided/alternative heap is another story. I would expect that the SDK you’re using already provides a working implementation (as mentioned _sbrk() and usually a corresponding heap section in linker script) including multi-tasking awareness (e.g. properly implemented newlib __malloc_lock/unlock callbacks).
If you provide a raw heap buffer to heap_4 from external SDRAM you could verify that everything is setup as expected by doing a test pvPortMalloc before running your actual application code and check that it’s working with the expected heap memory of the expected size by stepping through the code with a debugger as already proposed.
Or try to access the external SDRAM just with the debugger. Usually they provide memory dump/fill/modify functions.