Memory Leak using snprintf with floats

Using version 10.0.0 of FreeRTOS

I’ve been tracking down the source of a memory leak and have determined that it has something to do with using snprintf when floats are included in the argument list.

I stripped down my code to this test task:

void main_test_task(void *p)
{
	size_t freeNow;
	uint16_t fr;
	float testFloat = 1.09773588e-019;
	vTaskDelay(2000);
	freeNow = xPortGetFreeHeapSize();
		
	fr = snprintf(fileBuffer, sizeof(fileBuffer), "%f", testFloat);
	freeNow = xPortGetFreeHeapSize();
	
	vTaskDelete(NULL);
}

This task is created with:

#define MAIN_TASK_STACK_SIZE	800
#define MAIN_TASK_PRIORITY		configMAX_PRIORITIES-1		//second highest priority task
xTaskCreate(main_test_task, "MTest", MAIN_TASK_STACK_SIZE, NULL, MAIN_TASK_PRIORITY, &xMain_Test_task_handle);

fileBuffer is a static global char[128]. I also tried making it a var local to the task, and it made no difference.

The initial delay is to let the heap settle from other tasks that may have been running when this one is started.

Each time I create this task and it runs, I lose 428 words of heap when the snprintf line executes. A variable amount of heap is returned after the task dies, usually 300-336 words. Repeatedly creating/running/killing the task results in a steady decline of available heap.

Here’s the weirder part:
If I change testFloat to be something simple like 1, only 232 words of heap are lost, and all is reclaimed. I can create/run/kill the task over and over with no net loss of heap.

What is going on here?

Using newlib? Which heap implementation are you using for FreeRTOS? (heap3.c?) Which tool provider?

Take a look here for a relevant discussion. There may be an issue with the way your tool provider built newlib.

Yes newlib, and for heap, heap_useNewlib.c

IDE/Toolchain is Atmel Studio.

I’ll take a look at that link, thanks.

That thread does sound an awful lot like what I’m experiencing. Shame that the root cause wasn’t uncovered though.

The issue does go away if I change #define configUSE_NEWLIB_REENTRANT from 1 to 0.

Yes, that’s the issue then. As a workaround can your design avoid deleting tasks? That way you could keep configUSE_NEWLIB_REENTRANT set to 1.

Alternatively you could get support from Atmel. They probably need to change the newlib config. Or you could venture into making a newlib build for yourself.

I may be able to work around not deleting tasks, but that’s hardly ideal.

I’ve figured out which version of newlib the pre-built library is based on, and am looking into rebuilding it, assuming I can figure out what needs to be changed.

I’m actually using the nano version of newlib. Out of curiousity, I tried to build with the regular version and got these errors (all in tasks.c):
Error undefined reference to __sf_fake_stdout' Error undefined reference to __sf_fake_stdin’
Error undefined reference to `__sf_fake_stderr’

So looks like I’ll need to rebuild the nano version, not the regular.

I’d like to add that there are very few (legacy) non-reentrant resp. stateful libc functions like e.g. strtok where a reentrant alternative version strtok_r exists.
Avoiding using those functions allows to go with configUSE_NEWLIB_REENTRANT being disabled. When in doubt the newlib sources can be checked for the functions to omit.
I think nowadays with multi-threaded apps everywhere internally stateful (library) functions should not be used anymore nor implemented.

1 Like

Don’t know your application, but usually the design is better if it doesn’t include task deletion at all. Maybe that’s true in your case too?

To add to Hartmut’s excellent advice, note that in the public domain there are natively reentrant (stack only) alternatives to the problematic newlib functions like printf %f etc. Maybe even all the ones you need.

It seems like I have 3 possible options to move forward:

  1. Set configUSE_NEWLIB_REENTRANT to 0 and switch to _r version of any problematic functions (if it exists)
  2. Leave configUSE_NEWLIB_REENTRANT as 1 and don’t delete/recreate threads
  3. Figure out the root cause and rebuild newlib with it fixed

#1 is appealing provided reentrant versions exist for everything I need, though I’m not sure what other unintended side effects this may cause. Everything seemed to run ok when I temprarily set configUSE_NEWLIB_REENTRANT to 0 for testing, but I’m sure something will get stepped on sooner or later if I don’t use the _r versions.

No, the _r versions in newlib are the ones causing the issue (they are automatically used internally to newlib, so you are already using them). They are safely reentrant when configUSE_NEWLIB_REENTRANT is 1, but with your newlib build they are apparently leaky when you delete tasks.

To make solution #1 work, you would need to find out which newlib functions you are calling utilize the reent struct and find alternatives (not in newlib) using google or github etc. I would suggest some, but I didn’t go that way – I went with solution #2 in my own projects.

Thanks, I follow you now.

#2 does sound like the best approach, though it would take some redesign since I have a few tasks that get spun up to do a certain action, then get deleted when they are finished.

I’m going to waste the part of my day that wasn’t wasted in meetings looking into the 3rd option first.

Am I understand the situation correctly that some versions (builds?) of newlib do not have the issue I’m experiencing? I found a few resources detailing how my version was built, but what is unclear is what, exactly, needs to be changed to mitigate the issue.

I believe so. This post indicates NXP got their newlib build configured “correctly”. But I don’t know from my own experience.

I saw that, but wasn’t sure if it confirmed the hypothesis related to IO control structures, or was more of a “well, I’m not entirely sure what was wrong with A, but B works, so that’s my solution!” way to wrap things up.

I think Dave may have been on track there. I’m reading his page:
https://nadler.com/embedded/NXP_newlibAndFreeRTOS.html

and it has this blurb:

printf or similar function expected to do IO (as opposed to string operation) allocates an IO control structure of 428 bytes.

I see this 428 byte allocation happen for my snprintf call, and only a portion of it is returned when the task is killed. When I did the test with reentrancy turned off, I don’t see this allocation for each instance of the task.

The newlib implementations of printf, strtof and friends are designed to be accurate for all inputs, but are overkill for many embedded applications. If you can be confident that your app will not need to handle stupidly large or small floating point values with absolute precision, you can use smaller functions that don’t need to allocate heap memory or use the newlib reent structure. This is what I do. You can find the printf and strtof functions I use at RRFLibraries/src/General at master · Duet3D/RRFLibraries · GitHub. In creating the SafeVsnprintf.cpp file, I used code from the FreeRTOS printf functions in the Tcp files, refactored it, and added the floating point format support.

1 Like