xTaskResumeAll lockup

Hi…

I am using freeRTOS V10.4.5 release

I am using nRF52820 + SDK V17.0.2 with SD112 V7.2.0 along with tickless idle as this is a battery based product…

When my application is just talking over the IIC bus it works perfectly
When my application is just talking over BLE it work perfectly

When I talk over IIC bus and BLE at the same time, somethings gets stuck and the watchdog resets to product… Not a great user experience !!

Here is my investigation
When the firmware freezes, I then paused the CPU…
I see that the CPU is stuck in the freeRTOS tasks.c API xTaskResumeAll() in an endless loop inside while( listLIST_IS_EMPTY( &xPendingReadyList ) == pdFALSE )

When I single step the CPU, the while loop keeps on going around and around
xPendingReadyList.uxNumberOfItems remains at 1, so pdFALSE condition never happens :frowning:

I have looked here: endless loop in function xTaskResumeAll which talks about interrupt priorities clashing… But am not too sure what changes my code needs if this is the underlying problem. My project has the configASSERT() macro setup to call assert_nrf_callback()

What I have done to fix the problem
I added a counter in the while loop that will exit after counter reaches # of tasks created
This works well and my application now works perfectly when IIC and BLE activity happen at the same time

uint8_t retry = 0;
while( (listLIST_IS_EMPTY( &xPendingReadyList ) == pdFALSE) && (retry < uxCurrentNumberOfTasks) ) {

retry++;
}

Thanks for any Help
Ray

A simplistic answer is you have fixed the symptom, but not the actual problem.

Something has corrupted the list so the count doesn’t match the list, and ultimately, that may cause other even harder to find issues.

Interrupt priorities is.a common source of that problem, but not the only one. The way interrupt priorities cause the problem is interrupt with too high a priority can break the critical sections that protect the structures being used by FreeRTOS.

I am not familiar with that processor or its port, so I don’t know how well it tests for this sort of issue. The fact that it occurs when two operations that likely use a number of interrupts points to something interrupt related as a possible cause.

Thanks for the fast response…

I agree, I have fixed the symptom and not the problem…

I think I get the Interrupt priority problem now…
When ISR Pri=3 is updating a freeRTOS structure and a ISR Pri=0 comes along and makes a change to the same freeRTOS structure, then the changes made by ISR Pri=3 will be lost/corrupted

This could definitely be the case as BLE= ISR Pri=0 and IIC Pri=3…
I cannot change the BLE Priorities due to the way Nordic BLE works…

Is the best way to fix this to use critical sections ??? turn off global interrupts ??? have 2 different lists (one for each priority) ???

Thanks
Ray

Even when using nested interrupts this must not happen. It seems that the interrupt priorities resp. the related FreeRTOS configuration (as Richard mentioned to ensure that critical sections are working properly) is not correct.
Just to be sure: You’re using the FreeRTOS API tagged with FromISR in your ISRs, right ?

Since nRF52820 has a Cortex-M4 CPU did you see this related documentation ?

and this very good forum posts:

Check the usual suspects, ie make sure all your ISRs run below MAX_SYSCALL priority, all isr sys call invocations (and only those) are via xxxFromISR() calls, and your startup stack is large enough to serve nested interrupts from all possible sources.

thanks for the info…
Can you post a table showing the ISR priorities that one of your projects use ?

Attached is the priorities that my project is configured with
RTC1 is the timer used by freeRTOS scheduler

Priorities of 0,1,4 are for the Soft Device that does BLE
Priorities of 2,3,5,6,7 can be used by my application

Based on you comments, should RTC1 be Priority 2 ???

If this are Cortex-M NVIC priority values (the values programmed in the NVIC) you might have been read in the doc and posts mentioned before that a lower numerical value means higher priority ! If this is the case it seems that the priorities are not correct.
FreeRTOS tick interrupt should get the lowest priority.
You should post your FreeRTOSConfig.h part with
configMAX_SYSCALL_INTERRUPT_PRIORITY and configKERNEL_INTERRUPT_PRIORITY and I recommend to have a look at the doc for those config items here.

As requested… hope this helps

#define configLIBRARY_LOWEST_INTERRUPT_PRIORITY _PRIO_THREAD

#define configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY _PRIO_APP_MID

#define configKERNEL_INTERRUPT_PRIORITY configLIBRARY_LOWEST_INTERRUPT_PRIORITY

#define configMAX_SYSCALL_INTERRUPT_PRIORITY configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY

where
#define _PRIO_SD_HIGH 0
#define _PRIO_APP_HIGH 1
#define _PRIO_APP_MID 1
#define _PRIO_SD_LOW 2
#define _PRIO_APP_LOW_MID 3
#define _PRIO_APP_LOW 3
#define _PRIO_APP_LOWEST 3
#define _PRIO_THREAD 4

Hi Ray - there are several problems with the FreeRTOSConfig.h contents you posted. The biggest issue being a possible confusion between task priorities and interrupt priorities. Your FreeRTOS interrupt priorities should not be based on task priorities.

Maybe best at this point to start fresh on definitions for configLIBRARY_LOWEST_INTERRUPT_PRIORITY, configKERNEL_INTERRUPT_PRIORITY, configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY, and configMAX_SYSCALL_INTERRUPT_PRIORITY. Be sure to visit the links that Hartmut referenced here, especially the link called “RTOS for ARM Cortex-M”. Here are a few things to note.

  • configLIBRARY_... constants are unshifted, but the corresponding constants with LIBRARY removed must be shifted. There is often a configPRIO_BITS value set in FreeRTOSConfig.h to help with this shifting. It is the number of priority bits implemented in the Cortex M implementation. CMSIS defines __NVIC_PRIO_BITS if that helps you.
  • A convenient shortcut for configKERNEL_INTERRUPT_PRIORITY is to set it to 0xFF. That value is sure to be the lowest priority regardless of the number of priority bits implemented.
  • You must be sure that ISRs with higher priority than configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY (lower numbers) do not make FreeRTOS API calls. (And you must not set configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY to zero.)
1 Like