ignisuti wrote on Friday, July 20, 2012:
xQueueReceive() function is in once instance in my code causing a Hard Fault. The fault occurs when it gets to this function: vPortYieldFromISR()
I’m new to using RTOS’s. Please help.
MCU: Freescale Kinetis K10 (ARM M4 Cortex)
rtel wrote on Friday, July 20, 2012:
Have you followed all the guidelines on the following pages?
http://www.freertos.org/RTOS-Cortex-M3-M4.html
http://www.freertos.org/FAQHelp.html
Regards.
ignisuti wrote on Friday, July 20, 2012:
Richard,
I did read these articles last night and had a few clarification questions:
1.) None of my MCU’s interrupts call RTOS functions. Therefore, I can set my MCU interrupt priorities to ANY priority (0 through 15), right?
I have configured 10 interrupts in my system (unrelated to the RTOS). Below is also their priority number. I’m using a Periodic Interrupt Timer (PIT) as my MCU SysTick because I wasn’t sure if it was okay to share the standard SysTick with the RTOS. Do you see any concerns with this setup or have suggestions?
= 8,
= 6,
= 5,
= 7,
= 7,
= 0, /* SysTick */
= 10,
= 10,
= 10,
= 1,
2.) Do I need to set the priority for the FreeRTOS interrupts (PendableSrvReq, SysTick, etc…), or are they setup by the ported code?
3.) Step 5 in the FAQ document says: “Do not use any method for disabling and enabling interrupts other than calls to portENTER_CRITICAL() and portEXIT_CRITICAL().”
This caught me completely off guard. Is this referring to my MCU’s interrupts, or the RTOS specific functions? If it’s mine, then I think we’ve identified one major thing I’m doing wrong.
rtel wrote on Friday, July 20, 2012:
Therefore, I can set my MCU interrupt priorities to ANY priority (0 through 15), right?
Yes.
I’m using a Periodic Interrupt Timer (PIT) as my MCU SysTick because I wasn’t sure if it was okay to share the standard SysTick with the RTOS. Do you see any concerns with this setup or have suggestions?
Do you mean you are using the PIT to generate a tick interrupt for your application, and you have left the SysTick for the RTOS kernel? That will be fine.
2.) Do I need to set the priority for the FreeRTOS interrupts (PendableSrvReq, SysTick, etc…), or are they setup by the ported code?
The FreeRTOS interrupts are set by the FreeRTOSConfig.h parameter configKERNEL_INTERRUPT_PRIORITY, which should always be set to 255.
Step 5 in the FAQ document says: “Do not use any method for disabling and enabling interrupts other than calls to portENTER_CRITICAL() and portEXIT_CRITICAL().”
This caught me completely off guard. Is this referring to my MCU’s interrupts, or the RTOS specific functions? If it’s mine, then I think we’ve identified one major thing I’m doing wrong.
It depends what you are doing. The kernel expects to have control of the BASEPRI value, which is used as an interrupt mask. The kernel never globally disables interrupt (for example, by using an cpsid instruction). If you code is setting and clearing the global interrupt flag, but leaving the BASEPRI alone, that should be ok. If you are changing the BASEPRI then that should also be ok provided you always reset it to its previous value and don’t yield (make a blocking API call) while you have it modified. The value you set it to (before resetting it) must not decrease its value (by decrease its value I mean its logical priority value) either.
Regards.
ignisuti wrote on Friday, July 20, 2012:
Richard,
Thanks for the reply.
Sounds like everything there checks out. I’m at a loss?
Any other ideas of things I can check?
vPortYieldFromISR() is only one line of code:
*(portNVIC_INT_CTRL) = portNVIC_PENDSVSET;
When I get to the place in my code that causes this, I set my breakpoint on “*(portNVIC_INT_CTRL) = portNVIC_PENDSVSET;”
The Hard Fault does not occur the first time, in here, but does consistently trigger the 2nd time through.
rtel wrote on Friday, July 20, 2012:
Which interrupt is calling vPortYieldFromISR()? You said before the interrupts were not using the FreeRTOS code, although that line in itself should be safe as all it does is set a low priority interrupt pending, in fact the lowest, so running that line from an interrupt should not have *any* effect on the execution pattern until all executing and pending interrupts have completed.
Regards.
ignisuti wrote on Friday, July 20, 2012:
Richard,
This isn’t being called from an interrupt. I don’t have any FreeRTOS functions called from my interrupts.
This is being called from xQueueReceive().
rtel wrote on Friday, July 20, 2012:
Is it feasible that you are running out of interrupt stack?
On the Cortex-M the stack used by main() is then re-used as the interrupt stack. The stack overflow checking only checks the task stacks, not the interrupt stack.
How big is the stack used by main()? How it is defined depends on your compiler. Can you fill the stack used by main with a known value at start up, then when the crash occurs, view it in the debugger to see if it is full or not?
Some systems will just define the stack at one end of the free RAM and the heap at the other, not really dimension either, and then just let them run into each other to chaotically let you know you ran out of either stack or heap. Which heap implementation are you using (heap_1.c, heap_2.c, heap_3.c, or one you have done yourself)?
Regards.
rtel wrote on Friday, July 20, 2012:
I think the next thing to try is working out which instruction was executing (or attempting to execute) when the hard fault occurs. When the hard fault occurs the stack frame of the fault handler includes the CPU registers as they were when the fault occurred, and the fist thing of interest in the program counter (PC). That tells you the instruction. The other registers will then give you the operand values, which can help track down things like null pointer use as a source of the fault.
To decode the stack frame, provide the following hard fault handler (the function name needs to be correct for the fault handler in your system, and you may need to write it all in assembler depending on how your system is currently set up).
static void HardFault_Handler(void)
{
__asm volatile
(
" tst lr, #4 \n"
" ite eq \n"
" mrseq r0, msp \n"
" mrsne r0, psp \n"
" ldr r1, [r0, #24] \n"
" ldr r2, handler2_address_const \n"
" bx r2 \n"
" handler2_address_const: .word pop_registers_from_fault_stack \n"
);
}
This code just sets up everything ready, then jumps to a function called pop_registers_from_fault_stack - which is defined below:
void pop_registers_from_fault_stack(unsigned int * hardfault_args)
{
unsigned int stacked_r0;
unsigned int stacked_r1;
unsigned int stacked_r2;
unsigned int stacked_r3;
unsigned int stacked_r12;
unsigned int stacked_lr;
unsigned int stacked_pc;
unsigned int stacked_psr;
stacked_r0 = ((unsigned long) hardfault_args[0]);
stacked_r1 = ((unsigned long) hardfault_args[1]);
stacked_r2 = ((unsigned long) hardfault_args[2]);
stacked_r3 = ((unsigned long) hardfault_args[3]);
stacked_r12 = ((unsigned long) hardfault_args[4]);
stacked_lr = ((unsigned long) hardfault_args[5]);
stacked_pc = ((unsigned long) hardfault_args[6]);
stacked_psr = ((unsigned long) hardfault_args[7]);
/* Inspect stacked_pc to locate the offending instruction. */
for( ;; );
}
Put a break point in the for(;
loop in this function, then when the break point is hit, look at the value of stacked_pc to find the instruction that caused the fault.
Once you know the address of the instruction, open up the assembly view in your debugger, go to that address, and put a break point there. Next reset your system so it runs from the start again, and when the break point is hit, you should be able to see the line of C code that is executing.
Regards.
ignisuti wrote on Friday, July 20, 2012:
Richard,
We’ve been beating our heads against the wall for days trying to track down the root of this nasty bug. This Hard Fault Handler code that you’ve provided is brilliant! It showed us exactly what was causing the issue.
A pointer in our code (not the FreeRTOS) is getting filled/overwritten with garbage. I’ve still yet to hunt that down, but that’ll be easy in comparison.
THANK YOU SOOOO MUCH!!!!
Where I can I send the support check? 
rtel wrote on Saturday, July 21, 2012:
I’m pleased you are making progress, and that I could help.
Donations are always welcome ;o) http://www.freertos.org/donate.html
Regards.
porcupineg wrote on Sunday, September 02, 2012:
i have the same problem, it is very weird. Hard fault appears when i write a value to specific filed in a structure,then i have hard fault on xQueueRecieve, did you find couse of that?
ignisuti wrote on Sunday, September 02, 2012:
Richard shared some code with me that helped me understand which instruction caused the hard fault. I’m sorry I’m pressed for time right now, or I’d explain it better. I’m hoping you’ll be able to make sense of this code. In the end, I found it was an issue in my code (outside the RTOS).
/* ===============================================================================================
== FUNCTION DESCRIPTION:
== Arm Core Hard Fault Interrupt (Address 0x0000000C).
== Review ARM documentation to determine which registers are relevant.
==
== OUTPUTS:
== N/A
== =============================================================================================*/
static void INTERRUPT__handle_hard_fault
(
void
)
{/* ARM Core - Hard Fault */
asm
(
tst lr, #4
ite eq
mrseq r0, msp
mrsne r0, psp
ldr r1, [r0, #24]
ldr r2, =pop_registers_from_fault_stack
bx r2
//handler2_address_const: .word
);
} /* INTERRUPT__handle_hard_fault() */
void pop_registers_from_fault_stack(unsigned int * hardfault_args)
{
unsigned int stacked_r0 = ((unsigned long) hardfault_args[0]);
unsigned int stacked_r1 = ((unsigned long) hardfault_args[1]);
unsigned int stacked_r2 = ((unsigned long) hardfault_args[2]);
unsigned int stacked_r3 = ((unsigned long) hardfault_args[3]);
unsigned int stacked_r12 = ((unsigned long) hardfault_args[4]);
unsigned int stacked_lr = ((unsigned long) hardfault_args[5]);
unsigned int stacked_pc = ((unsigned long) hardfault_args[6]);/* stacked_pc should contain offending address. */
unsigned int stacked_psr = ((unsigned long) hardfault_args[7]);
/* Display stacked_pc to user. */
DEBUG__print_error_message( "Hard Fault caused by instruction at address 0x%X", stacked_pc );
snprintf( DEBUG__buffer, LCD_MULTILINE_LEN, "CORE FAULT:|0x%X", stacked_pc );
MENU__write( DEBUG__buffer, UI__LCD, LCD__LINE__1_WITH_CLEAR, LCD__ALIGN__CENTER, FALSE );
EVENT_LOG__add( DIAG_CODE__WARNING, DIAG_CODE__WARNING__HARD_FAULT );
SHUTDOWN__reset_mcu();
}rtel wrote on Sunday, September 02, 2012:
I since wrote this up at http://www.freertos.org/Debugging-Hard-Faults-On-Cortex-M-Microcontrollers.html
Regards.
jaysfk wrote on Thursday, June 08, 2017:
This is a very nice solution.
However, there is problem. If the endless loop at the end of pop_registers_from_fault_stack is removed. The code will return to the offending point that triggers HardFault Handler, resulting in re-entry into the hard fault handler again.
I solved this by calling system reset inside and at the end of pop_registers_from_fault_stack().
However, is it possible to force pop_registers_from_fault_stack to return to the hardfault handler. Because I wish to log the pc counter inside the hardfault handler.