[PIC32] strange heap behaviour

Kernel
#1

erupter wrote on Thursday, February 16, 2012:

I have the heap_2.c allocation mode linked in the program.
Right now I only create two tasks.
minimal stack allocation in freertosconfig.h is set to 100.
isr stack size is 200.

Heap is 1000.

At first the heap size is 0x3E8 which is 1000, so it’s ok.
After creating the first task, free heap is 0x1E8 which is 488. And it’s NOT correct.
Second task creation fails as there is not enough space available.

if i set the heap to 2000, first task still gets 512 bytes, second does too.
So after creating two tasks (with minimal stack set to 100) my heap is left with 976 bytes free from 2000.

I’m puzzled.
Right now if I set the heap to 0, I still have a RAM usage of 1224 bytes which I assume is the minimal size of the kernel for my configuration.

But I can’t understand the allocations for the tasks.

But I don’t get the memory assignements.
I have the

#2

erupter wrote on Thursday, February 16, 2012:

Also the watermark for one the tasks (currently just blinking leds) is 62.
What does that mean, that it’s filling 450 bytes???
With what???

#3

rtel wrote on Thursday, February 16, 2012:

After creating the first task, free heap is 0x1E8 which is 488. And it’s NOT correct.

Please read: http://www.freertos.org/a00125.html and you will see it is correct.  You are allocating 100 words, or 400 bytes, plus the TCB, plus any alignment bytes.

Also the watermark for one the tasks (currently just blinking leds) is 62.
What does that mean, that it’s filling 450 bytes???

Where did the 450 come from in this?  If you have a stack of 100 words, and the peak usage leaves 62 words, then at its peak there were 38 words on the stack (most of which will be the task context when it is not running).  Therefore you application is using practically no stack.

Regards.

#4

erupter wrote on Thursday, February 16, 2012:

So the stack is in words, but the heap is in bytes…
Ok there came some of my confusion.

So I have my three simple tasks, each one assigned 100 words.
water1 = 60 -> maximum 40 used
water2 = 50 -> maximum 50 used
water3 = 56 -> maximum 44 used

So giving task1 50, task2 60 and task3 50 should allow for them to function.
But instead I get stack overflows.

I’m able to run only the following

-task1 at 55 stack with 14 water (should mean I can get as low as 41)
below 55 get overflow

-task2 at 60 stack with 10 water (at 55 stack it overflows)

-task3 at 60 stack with 16 water (at 55 stack it overflows)

So my numbers are confusing to me.
If the watermark is the highest, I should be safe with it.
Still I can’t get them to run not even with safe margins…

By the way both
vApplicationStackOverflowHook( xTaskHandle *pxTask, signed char *pcTaskName )
and
pxCurrentTCB
never report the correct task name, I always get garbage strings.

#5

erupter wrote on Thursday, February 16, 2012:

Also the total assigned stack is (55 + 60 + 60) = 175, but the used heap is 1752 (reported free heap is 3248 out of 5000).
Now if I assigned 175*4 = 700 bytes, where do the other 1052 come from?

#6

rtel wrote on Thursday, February 16, 2012:

So the stack is in words, but the heap is in bytes…
Ok there came some of my confusion.

This is documented in the API reference.

So I have my three simple tasks, each one assigned 100 words.
water1 = 60 -> maximum 40 used
water2 = 50 -> maximum 50 used
water3 = 56 -> maximum 44 used

So giving task1 50, task2 60 and task3 50 should allow for them to function.
But instead I get stack overflows.

I’m not following your numbers here.  That may be:

A) because I am tired, having traveled far today.
B) because you have mis-understood what the high water mark means.

http://www.freertos.org/uxTaskGetStackHighWaterMark.html

It is the minimum amount of stack there has been free during the lifetime of the task.  Therefore, the closer to 0 it is, the closer the task has come to overflow its stack.  That means the value has meaning no matter how big the stack is.

Regards.

#7

erupter wrote on Thursday, February 16, 2012:

Thank you Richard for your patience all the more if you’re tired.

I think I got the meaning of the watermark.
In fact what I’m saying is: if I have simple-stupid static tasks (without queues, mallocs, strings…) and after some iterations I read that the amount of stack free is 14 words, I feel authorized to assign that task a smaller stack.

In theory that task may overflow, but given it is static it should not. Or should it?

So what I have here is the kernel telling me some task has some free stack, me trying to retake part of that memory, and the task crashing as a consequence.
Although the memory given to it is still more than (stack - watermark).

#8

rtel wrote on Friday, February 17, 2012:

In theory that task may overflow, but given it is static it should not. Or should it?

That really depends on the port you are using.  If you are using something like a Cortex-M3 or PIC32, then the maximum stack usage should be fairly fixed, and whatever the stack usage is when the path through the code taken is such that the stack usage is maximized.  Other ports have to take into account interrupt usage too, in which case it is not so simple.

Although the memory given to it is still more than (stack - watermark).

The stack overflow warning should kick in 20 bytes (note bytes, not words) before the stack limit is actually reached.

Going back to a previous comment.  The parameters to the stack overflow hook are often corrupted by the fact that the stack has overflowed.  It depends on the memory layout in some cases (if the TCB is next to the stack, which it often is), and the way the stack frame is organised in others.  When this is the case, look at the pxCurrentTCB variable in the debugger to see which task is running.

Regards.

#9

erupter wrote on Saturday, February 18, 2012:

The parameters to the stack overflow hook are often corrupted  When this is the case, look at the pxCurrentTCB variable in the debugger to see which task is running.

[Name=, displayName=pxCurrentTCB->pcTaskName]	char[16]	0xA00008B8	"\u0x94\u0x8\u0x0\u0xa0\u0x0\u0x0\u0x0\u0x0\u0x0\u0x0\u0x0\u0x0\u0xf4\u0x8\u0x0\u0xa0"

This is an example of the garbage I get out of the TCB variable.

Getting back to the original topic, here are my two overflowing tasks in their entirety:

static void prvTestTask1( void *pvParameters ) 
    
    mInitAllLEDs();
    mLED_1_On();
    //Toggle RD0 with small delay
    for( ;; )
    {
        mLED_1_Toggle();
        vTaskDelay( smallDelay );
    }
}
static void prvStatTask (void *pvParameters)
{
    while(1)
    {
        heapfree = xPortGetFreeHeapSize();
        task1water = uxTaskGetStackHighWaterMark(task1);
        task2water = uxTaskGetStackHighWaterMark(task2);
        task3water = uxTaskGetStackHighWaterMark(task3);
        vTaskDelay(500);
    }
}

Here you can see their should maintain their memory footprint constant.
At least according to what I know.
Now since I’m most surely ignorant in some ways, I’d like anyone to explain to me why I see the watermark move for these tasks: what would the allocations be? Since my code does not call for anything during execution, I don’t understand where does the increase come from.

All the more for the Stats task which has its variables declared globally: why does the watermark get higher and higher until a stackoverflow occurs?

#10

erupter wrote on Sunday, February 19, 2012:

Doing more tests.

The blink task overflows even with 90 bytes.
the current tcb content is this

org.netbeans.modules.viewmodel.TreeModelNode@1f7b462[Name=, displayName=pxCurrentTCB->pxTopOfStack]	unsigned long*	0xA0000440	0xA0000530	
org.netbeans.modules.viewmodel.TreeModelNode@161066a[Name=, displayName=*pxCurrentTCB->pxTopOfStack]	unsigned long	0xA0001070	0x00000001	
org.netbeans.modules.viewmodel.TreeModelNode@163a99c[Name=, displayName=pxCurrentTCB->xGenericListItem]	xLIST_ITEM	0xA0000444		
org.netbeans.modules.viewmodel.TreeModelNode@144dc9[Name=, displayName=pxCurrentTCB->xEventListItem]	xLIST_ITEM	0xA0000458		
org.netbeans.modules.viewmodel.TreeModelNode@fef760[Name=, displayName=pxCurrentTCB->uxPriority]	unsigned long	0xA000046C	0x00000001	
org.netbeans.modules.viewmodel.TreeModelNode@122c336[Name=, displayName=pxCurrentTCB->pxStack]	unsigned long*	0xA0000470	0xA00004A0	
org.netbeans.modules.viewmodel.TreeModelNode@13e35ca[Name=, displayName=pxCurrentTCB->pcTaskName]	char[16]	0xA0000474	"Blink Task0\u0x0\u0x0\u0x0\u0x0\u0x0"	
org.netbeans.modules.viewmodel.TreeModelNode@13d73e0[Name=, displayName=pxCurrentTCB->uxCriticalNesting]	unsigned long	0xA0000484	0x00000000	
org.netbeans.modules.viewmodel.TreeModelNode@182830b[Name=, displayName=pxCurrentTCB->uxBasePriority]	unsigned long	0xA0000488	0x00000001

which to me doesn’t say much. Any deeper interpretations I’m missing?
Problem is if I assign a stack of 120 words, it crashes right at the first scheduling.
Can’t understand why.

Also I assigned a total of 90+70+125 = 285 words.
That should be 1140 bytes.
I have a heap of 3000, still the reported free heap is 592.
That’s to say that I’m missing (assigned_heap - free_heap) - accounted_for_heap = disappeared_heap
(3000 - 592) - 1140 = 1268 bytes.
Where are they?

Or is my math wrong?

#11

davedoors wrote on Sunday, February 19, 2012:

I’m not following all these numbers but note the demo for the PIC32 has configMINIMAL_STACK_SIZE set to 190, and that is the recommended minimum for the idle task, and a task that doesn’t really do much (have a deep call nesting depth, etc.).

Problem is if I assign a stack of 120 words, it crashes right at the first scheduling.
Can’t understand why.

If you have a consistently repeatable crash at a known point, then you can step through the debugger and see why it is crashing.

That’s to say that I’m missing (assigned_heap - free_heap) - accounted_for_heap = disappeared_heap
(3000 - 592) - 1140 = 1268 bytes.
Where are they?

Put a break point in pvPortMalloc(), watch where the memory goes, and you will have an answer. Are you taking into account that the idle task is created, and depending on the configuration, maybe also a timer task?

#12

erupter wrote on Sunday, February 19, 2012:

Ok I just discovered something.

Using C32 v2.02.
If I debug the project with MplabX I get higher watermarks (between 10 and 20 bytes) and 150 less free heap than if I debug the exact same code with Mplab8.

Now the toolchain is exactly the same, probably the linking order could be different, or something in the X linker script.
But I don’t have enough knowledge in those fields to try to understand where is the difference/problem.

#13

erupter wrote on Sunday, February 19, 2012:

If you have a consistently repeatable crash at a known point, then you can step through the debugger and see why it is crashing.

Yeah well… I have a fast repeating stupid task. I can’t step through it every time, it takes seconds to crash…
And more importantly the task that crashes most of the times is the blinking led task reported above, it consists of just toggling a led and calling the task delay… what should I step through? The task delay call??

Put a break point in pvPortMalloc(), watch where the memory goes, and you will have an answer. Are you taking into account that the idle task is created, and depending on the configuration, maybe also a timer task?

I assumed that the idle task, as well as any other kernel related thing, were assigned statically.
In fact FreeRTOS takes 1K even with no task and an assigned heap of 0.

#14

davedoors wrote on Sunday, February 19, 2012:

Yeah well… I have a fast repeating stupid task. I can’t step through it every time, it takes seconds to crash…

You said in a previous post that with a stack of a certain size it crashed on the first reschedule.  I was referring to that.