vTaskDelayUntil() never returns when a task's priority is set bigger than 0

In a project with a single task whenever its priority is set to 0 the system runs as expected, but when its priority is set a value other than 0 (with configMAX_PRIORITIES set to 3) then the function vTaskDelayUntil() never returns. Instead, the IDLE hook fires repeatedly.

For example, the serial output when its priority is set to 0 is:


KRNL_IN
KRNL_FRV
IDL
KRNL_FRV
IDL
KRNL_FRV
IDL

which is ok. But when its priority is set to 1 or 2, then the output is:


KRNL_IN
IDL
IDL
IDL
IDL
IDL

I’ve run out of ideas. Here is the complete code:

#include <Arduino.h>

#include <FreeRTOS.h>
#include <task.h>
#include <event_groups.h>
#include <timers.h>

#define FOREVER while( 1 )



// we pass on the LEONARDO symbol through the Makefile
#ifdef LEONARDO
const int LED_BUILTIN = 13;
#endif

//----------------------------------------------------------------------
//                     plc_kernel_task()
//----------------------------------------------------------------------
static const uint16_t TASK_PLC_KERNEL_STACK_SIZE = 64 * 2;
static StaticTask_t plc_kernel_task_tcb;
static StackType_t plc_kernel_task_ram[ TASK_PLC_KERNEL_STACK_SIZE ];
TaskHandle_t main_task_handler;
void plc_kernel_task( void* pvParameters )
{
   uint8_t ticks{ 5 };
   uint8_t state{ 0 };

   TickType_t last_wake_time = xTaskGetTickCount();

   Serial.println( "KRNL_IN" );

   FOREVER
   {

      vTaskDelayUntil( &last_wake_time, pdMS_TO_TICKS( 100 ) );

      Serial.println( "KRNL_FRV" );

      --ticks;
      if( ticks == 0 ){

         if( state == 0 ){
            ticks = 5;
            state = 1;
            digitalWrite( 13, LOW );
         } else{
            ticks = 5;
            state = 0;
            digitalWrite( 13, HIGH );
         }

      }
   }
}

int main(void)
{

   cli();

   xTaskCreateStatic(
      plc_kernel_task,
      ( const portCHAR* ) "PLC",
      TASK_PLC_KERNEL_STACK_SIZE,
      NULL,
      2,
      plc_kernel_task_ram,
      &plc_kernel_task_tcb );

   init();
   // init the Arduino's hardware and stuff

   Serial.begin( 115200 );

   pinMode( 13, OUTPUT );
   digitalWrite( 13, LOW );

   vTaskStartScheduler();

   while( 1 )
   {
      // nothing to do in here. Everything is done in the task's code
   }

   return 0;
}

#ifdef __cplusplus
extern "C" {
#endif  

void vApplicationIdleHook()
{
   static uint16_t ticks{ 50000 };

   --ticks;
   if( ticks == 0 ){
      ticks = 50000;
      Serial.println( "IDL" );
   }
}

void vApplicationMallocFailedHook()
{
  while( 1 )
  {
     ;
  }
}

void vApplicationStackOverflowHook( TaskHandle_t xTask, signed char *pcTaskName )
{
   while( 1 )
   {
      Serial.println( "SOF!" );
      delay( 500 );
   }
}

#ifdef __cplusplus
}
#endif  

This is a reduced task from a project that involves 3 tasks. T1 and T2 are closely related because of a semaphore, and T3 is kind of independent. Whenever the priorities of T1 and T2 are set other than 0, the system stucks. T3 runs with the highest priority.

That’s way I reduced the code to its minimal, but I cannot see where and how I’m failing.

Any help will be highly appreciated!

Which microcontroller architecture are you using?

Please check the tick count is incrementing. You can do that by inspecting xTickCount in the debugger, or by setting a break point in xTaskIncrementTick in tasks.c.

Please also attach your FreeRTOSConfig.h file so I can see the preemption and time licensing settings.

Do you have configASSERT() defined?

Which version of FreeRTOS you using?

I spent all day trying to debug Arduino under Avarice and GDB, and I made it!

But now I’m clueless about where to start looking for. The file tasks.c has a million of functions and I can’t figure out which one is in charge of selecting the next task ready to run. Any ideas?

Hi Richard,

I forgot to mention the architecture: ATmega328P.

I was writing a long answer to your questions, but I think I need to verify that the function xTaskIncrementTick() is actually being called. I don’t see why not, but I want to be sure. That function is called inside a timer ISR that is fired every 1 ms (look for the last lines):

#if defined(__AVR_ATtiny24__) || defined(__AVR_ATtiny44__) || defined(__AVR_ATtiny84__)
    ISR(TIM0_OVF_vect)
    #else
    ISR(TIMER0_OVF_vect)
    #endif
    {
    	// copy these to local variables so they can be stored in registers
    	// (volatile variables must be read from memory on every access)
    	unsigned long m = timer0_millis;
    	unsigned char f = timer0_fract;

    	m += MILLIS_INC;
    	f += FRACT_INC;
    	if (f >= FRACT_MAX) {
    		f -= FRACT_MAX;
    		m += 1;
    	}

    	timer0_fract = f;
    	timer0_millis = m;
    	timer0_overflow_count++;

    #if USING_FREERTOS == 1
        #if configUSE_PREEMPTION == 1
    	  vPortYieldFromTick();
    	  asm volatile( "reti" );
        #else
    	  xTaskIncrementTick();
        #endif
    #endif
    }

And the config file is:

#define configSUPPORT_STATIC_ALLOCATION     1
    #define configSUPPORT_DYNAMIC_ALLOCATION    0
    //#define configTOTAL_HEAP_SIZE               (( UBaseType_t )( 1500 ))

    #define configMAX_PRIORITIES                3

    #define configUSE_PREEMPTION                1
    #define configUSE_TICK_HOOK                 0
    #define configCPU_CLOCK_HZ                  ( ( uint32_t ) F_CPU )
    #define configTICK_RATE_HZ                  ( ( portTickType ) 1000 )
    #define configMINIMAL_STACK_SIZE            ( ( UBaseType_t ) 85 )
    #define configMAX_TASK_NAME_LEN             ( 5 )
    #define configUSE_TRACE_FACILITY            0
    #define configUSE_16_BIT_TICKS              1

    #define configUSE_IDLE_HOOK                 1
    #define configIDLE_STACK_SIZE               ( ( UBaseType_t ) 192 )
    #define configIDLE_TASK_NAME                "IDLE"
    #define configIDLE_SHOULD_YIELD             1


    #define pdMS_TO_TICKS( xTimeInMs )          (portTickType)( ((portTickType) ( xTimeInMs )) / ((portTickType) portTICK_RATE_MS ) )


    #define configUSE_MUTEXES                   0
    #define configUSE_RECURSIVE_MUTEXES         0
    #define configUSE_COUNTING_SEMAPHORES       0
    #define configUSE_QUEUE_SETS                0
    #define configQUEUE_REGISTRY_SIZE           0
    #define configUSE_TIME_SLICING              0
    #define configCHECK_FOR_STACK_OVERFLOW      0
    #define configUSE_MALLOC_FAILED_HOOK        0

    /* Timer definitions. */
    #define configUSE_TIMERS                    0
    #define configTIMER_TASK_PRIORITY           ( ( UBaseType_t ) 0 )
    #define configTIMER_QUEUE_LENGTH            ( ( UBaseType_t ) 3 )
    #define configTIMER_TASK_STACK_DEPTH        configMINIMAL_STACK_SIZE * 2

    /* Co-routine definitions. */
    #define configUSE_CO_ROUTINES               0
    #define configMAX_CO_ROUTINE_PRIORITIES     ( (UBaseType_t ) 2 )

    /* Set the stack pointer type to be uint16_t, otherwise it defaults to unsigned long */
    #define portPOINTER_SIZE_TYPE			        uint16_t

    /* Set the following definitions to 1 to include the API function, or zero
    to exclude the API function. */

    #define INCLUDE_vTaskPrioritySet                0
    #define INCLUDE_uxTaskPriorityGet               0
    #define INCLUDE_vTaskDelete                     0
    #define INCLUDE_vTaskCleanUpResources           0
    #define INCLUDE_vTaskSuspend                    1
    #define INCLUDE_vResumeFromISR                  1
    #define INCLUDE_vTaskDelayUntil                 1
    #define INCLUDE_vTaskDelay                      1
    #define INCLUDE_xTaskGetSchedulerState          0
    #define INCLUDE_xTaskGetIdleTaskHandle          0
    #define INCLUDE_xTaskGetCurrentTaskHandle       0
    #define INCLUDE_uxTaskGetStackHighWaterMark     0

In this very moment I’m gonna make the tests and I’ll return.

Hi again, Richard

Yes, the function xTaskIncrementTick() is called even when the task’s priority is set to 1 or 2. So the counter is counting!

(gdb) c
Continuando.

Breakpoint 2, __vector_16 ()
    at /home/fjrg76/ard_linux_molcaj/molcajete-9.0.2-Linux-no_inst/arduino-1.8.5/hardware/arduino/avr/cores/arduino/wiring.c:98
98		xTaskIncrementTick();
(gdb) list
93	#if USING_FREERTOS == 1
94	#if configUSE_PREEMPTION == 1
95		vPortYieldFromTick();
96		asm volatile( "reti" );
97	#else
98		xTaskIncrementTick();
99	#endif  
100	#endif  
101	
102	}
(gdb) s
xTaskIncrementTick ()
    at /home/fjrg76/ard_linux_molcaj/molcajete-9.0.2-Linux-no_inst/arduino-1.8.5/libraries/FreeRTOS/src/tasks.c:2599
2599		if( uxSchedulerSuspended == ( UBaseType_t ) pdFALSE )
(gdb) 

I got 2 questions:

1.- I set configUSE_PREEMPTION to 1, but lines 95 and 96 aren’t processed when they should, right?

2.- Inside the xTaskIncrementTick() function, in line 2622:

if( xConstTickCount >= xNextTaskUnblockTime )

I asked for the value of xNextTaskUnblockTime variable and I got the value 65535, is that correct?

(gdb) n
2622			if( xConstTickCount >= xNextTaskUnblockTime )
(gdb) p xNextTaskUnblockTime 
$2 = 65535
(gdb)

I need to sleep, but any other questions you have, or any hint, please let me know them so I can answer them ASAP. Thank you!

That isn’t a proper tick timer interrupt (if you are using preemption) as it doesn’t call xTaskIncrementTick, but only vPortYieldFromTick().

Right, the tick ISR and FreeRTOSConfig.h you posted shows xTaskIncrementTick() will not be called, so time will not pass, so the timeout in vTaskDelayUntil() will not expire.

Where did that code come from?

@richard-damon Hi Richard, in fact xTaskIncrementTick() is called and it counts! Please see my answers. I know it counts because a LED that should blink every 500 ms, blinks! (when its priority is set to 0), and now that I got the debugger I’ve seen the system gets inside that function, and it counts!

@rtel As is, the xTaskIncrementTick() function shouldn’t be called, but it’s! Now I’m going to investigate why. Meanwhile please see all my answers in this post so you can see the debug session.

BTW, I got that solution from the official ATMEGA328 port and I just adapted it to Arduino without modifying such platform a lot.

The problem isn’t whether the counter counts or not (and it counts), I need to find out how the priority’s levels are affecting the overall system, do you guys know which function is in charge of choosing to run the next ready to run task?

Look at the code you posted, if configUSE_PREEMPTION is 1, then it does a vPortYieldFromTick() and then a reti.
Only if configUSE_PREEMPTION isn’t 1, id xTaskIncrementTick() called.
Maybe the port does something strange in vPortYieldFromTick()? Yes, looking up the ATMega328 port the vPortYield from tick does the call to xTaskIncrementTick(), I don’t understand why the extra complexity of testing configUSE_PREEMPTION since xTaskIncrementTick() checks that flag too to see if it should ask for the context switch.

One thing I do note is I don’t see a definition of configASSERT in your FreeRTOSConfig.h, which can help catch a number of configuration errors. One possibility is that an interrupt has a bad priority and is causing FreeRTOS state corruption.

The main part of the scheduler is vTaskSwitchContext()

If you are using an AVR I’m going to assume configUSE_PORT_OPTIMISED_TASK_SELECTION is not set. In which case the following macro in tasks.c selects the next task:

	#define taskSELECT_HIGHEST_PRIORITY_TASK()															\
	{																									\
	UBaseType_t uxTopPriority = uxTopReadyPriority;														\
																										\
		/* Find the highest priority queue that contains ready tasks. */								\
		while( listLIST_IS_EMPTY( &( pxReadyTasksLists[ uxTopPriority ] ) ) )							\
		{																								\
			configASSERT( uxTopPriority );																\
			--uxTopPriority;																			\
		}																								\
																										\
		/* listGET_OWNER_OF_NEXT_ENTRY indexes through the list, so the tasks of						\
		the	same priority get an equal share of the processor time. */									\
		listGET_OWNER_OF_NEXT_ENTRY( pxCurrentTCB, &( pxReadyTasksLists[ uxTopPriority ] ) );			\
		uxTopReadyPriority = uxTopPriority;																\
	} /* taskSELECT_HIGHEST_PRIORITY_TASK */

There is one list for each task priority. The while loop finds the first list that is not empty - starting from the last known highest priority. listGET_OWNER_OF_NEXT_ENTRY() then locates the TCB from the next item in the selected list.

…further you will see taskSELECT_HIGHEST_PRIORITY_TASK() being called from vTaskSwitchContext() (in tasks.c), which in turn should be called if xTaskIncrementTick() returns pdTRUE (meaning xTaskIncrementTick() unblocked a task).

@rtel @richard-damon Thank you for your patience. The counter is counting whether the task’s priority is set to 0 or 1, that doesn’t matter, the counter counts. Here there are new facts:

WHEN TASK’S PRIORITY IS SET TO 0

The program stops at line 2643 in tasks.c (inside xTaskIncrementTick() function) and when

(gdb) p pxTCB
$12 = (TCB_t *) 0x80011e <plc_kernel_task_tcb>

This happens when my task’s delayed time is reached every 100 ms (the counter counts and we should that fact let it go):

(gdb) p xConstTickCount 
$15 = 12300
(gdb) c
Continuando.

Breakpoint 5, xTaskIncrementTick ()
    at /home/fjrg76/ard_linux_molcaj/molcajete-9.0.2-Linux-no_inst/arduino-1.8.5/libraries/FreeRTOS/src/tasks.c:2643
2643						xItemValue = listGET_LIST_ITEM_VALUE( &( pxTCB->xStateListItem ) );
(gdb) p xConstTickCount 
$16 = 12400
(gdb) p pxTCB
$17 = (TCB_t *) 0x80011e <plc_kernel_task_tcb>
(gdb) q

(You can see the counter is counting.)

Besides that I got this:

    Breakpoint 1, xTaskIncrementTick ()
        at /home/fjrg76/ard_linux_molcaj/molcajete-9.0.2-Linux-no_inst/arduino-1.8.5/libraries/FreeRTOS/src/tasks.c:2643
    2643						xItemValue = listGET_LIST_ITEM_VALUE( &( pxTCB->xStateListItem ) );
    (gdb) p *pxDelayedTaskList 
    $1 = {uxNumberOfItems = 1 '\001', pxIndex = 0x800297 <xDelayedTaskList1+3>, xListEnd = {xItemValue = 65535, 
        pxNext = 0x800120 <plc_kernel_task_tcb+2>, pxPrevious = 0x800120 <plc_kernel_task_tcb+2>}}
    (gdb) c
    Continuando.

pxDelayedTaskList->uxNumberOfItems is set to 1, which I think it refers to my task.

So far so good!

WHEN TASK’S PRIORITY IS SET TO 1

The program doesn’t stop at 2643! Even more, from the debugger I see (worried) that there are not items in the pxDelayedTaskList, which might means my task will never wake up:

(gdb) p *pxDelayedTaskList
$10 = {uxNumberOfItems = 0 ‘\000’, pxIndex = 0x80028e <xDelayedTaskList2+3>, xListEnd = {xItemValue = 65535,
pxNext = 0x80028e <xDelayedTaskList2+3>, pxPrevious = 0x80028e <xDelayedTaskList2+3>}}
(gdb) p pxReadyTasksLists
$11 = {{uxNumberOfItems = 1 ‘\001’, pxIndex = 0x800311 <xIdleTaskTCB.1831+2>, xListEnd = {xItemValue = 65535,
pxNext = 0x800311 <xIdleTaskTCB.1831+2>, pxPrevious = 0x800311 <xIdleTaskTCB.1831+2>}}, {uxNumberOfItems = 1 ‘\001’,
pxIndex = 0x8002a9 <pxReadyTasksLists+12>, xListEnd = {xItemValue = 65535, pxNext = 0x800120 <plc_kernel_task_tcb+2>,
pxPrevious = 0x800120 <plc_kernel_task_tcb+2>}}, {uxNumberOfItems = 0 ‘\000’, pxIndex = 0x8002b2 <pxReadyTasksLists+21>,
xListEnd = {xItemValue = 65535, pxNext = 0x8002b2 <pxReadyTasksLists+21>, pxPrevious = 0x8002b2 <pxReadyTasksLists+21>}}}
(gdb)

I think I need to trace the call vTaskDelayUntil() to figure out why my task is not getting a seat in the delayed tasks list.

I’m attaching a screen capture for the source code line 2643, for quick context:

Captura de pantalla de 2020-08-09 14-48-32

And from this link you can see the LED blinking when the task’s priority is set to 0:

https://photos.app.goo.gl/vH7YMBZqDeRcx9hMA

More hints are really appreciated! And I will set up right now the configASSERT() macro.

Agree think you are going to have to step into the call to vTaskDelayUntil() to see what is happening. Make sure the task is added to the delayed list here https://github.com/FreeRTOS/FreeRTOS-Kernel/blob/master/tasks.c#L1326 - you should see the list pointed to by pxDelayedTaskList has one more entry after that call to prvAddCurrentTaskToDelayedList(). Also make a note of the time the task is due to exist the delayed list (xTimeToWake - xConstTickCount). Then break in xTaskIncrementTick() when that time is reached to make sure the task is removed from the blocked list, and added to the correct delayed list.

@richard-damon, @rtel ¡SOLVED! But I’ve got some doubts…

1 THE SOLUTION

The problem was (is, I’m still investigating wether it’s because of my Arduino porting) that in the tick ISR

(the complete code is at the beginning of this thread)...
#if USING_FREERTOS == 1
        #if configUSE_PREEMPTION == 1
    	  vPortYieldFromTick();
    	  asm volatile( "reti" );
        #else
    	  xTaskIncrementTick();
        #endif
    #endif
    }

the function

xTaskIncrementTick()

was called even when configUSE_PREEMPTION is set to 1, instead of

vPortYieldFromTick();
asm volatile( "reti" );

as if configUSE_PREEMPTION were set to 0. I started a new simple project from scratch and it worked, and as I said, I’m still investigating this missbehaviour. Now those 2 LOCs are called as expected.

EDIT: The problem and solution

Now I know what was going on. The Arduino’s file “wiring.c” in which the system tick timer is defined (and for this matter, all files belonging to Arduino) doesn’t know anything about the file “FreeRTOSConfig.h” file. That’s way the conditional compilation takes the “#else” path, so the program behaves as if it were running under a cooperative scheduler, and the lack of the taskYIELD() calls made the system crash.

I found two solutions:

  1. To add the symbol configUSE_PREEMPTION into the overall system makefile (from my project Molcajete):

ARDUINO_DIR = /home/fjrg76/ard_linux_molcaj/molcajete-9.0.2-Linux-no_inst/arduino-1.8.5
BOARD_TAG = uno
ARDUINO_PORT = /dev/ttyUSB*
MONITOR_BAUDRATE = 115200
CFLAGS_STD = -std=gnu11 -Wall -Wpedantic -ggdb3
CXXFLAGS_STD = -std=gnu++11 -Wall -DUNO -Wpedantic -ggdb3
CFLAGS_STD += -DUSING_FREERTOS=1
CFLAGS_STD += -DUSING_USER_TICK_HOOK=0
CFLAGS_STD += -DconfigUSE_PREEMPTION=1
include /usr/share/arduino/Arduino.mk

Drawback: There’s one makefile on a project basis, so the user might forget to add the flag.

  1. An easy one (for me as the project maintener) is to #include the file “FreeRTOSConfig.h” into the “Arduino.h” file once and forever:

Arduino.h (at the end of the file):

(...)
#include "pins_arduino.h"

#if USING_FREERTOS == 1
#include <FreeRTOSConfig.h>
#endif  

#endif //#define Arduino_h

2 THE ISSUES

However, while trying to figure out what the problem was I got two a very strange behaviours related to cooperative kernel. Say you have two task, T1 and T2, almost identical:

T1 blinks a LED with a period of 1000 ms.
T2 blinks a LED with a period of 2000 ms.

Then you assign to T1 a priority of 1, and T2 a priority of 0, the result is that BOTH LEDs blink with period of 2000 ms!

Even worst, when you assign to T1 a priority of 2, and T2 a priority of 2 (among other combinations), the result is that the system stuck and none of both LEDs blinks.

This happens, I think, because none of the tasks releases the CPU, which makes me ask: whenever you use API functions like vTaskDelay() or similar in a cooperative scheduler, where do you place the API function taskYIELD()?

THANK YOU! :smile:

If you use vTaskDelay() (or related) you don’t need to call taskYIELD() as when the delay enters the block state, it will do the yield.

That’s in the theory. In real world that isn’t happening (according to the results I got while using the cooperative schema).

Why both tasks, T1 and T2 blink at the rate impossed by T2 in one scenario, and why the system get stuck when both tasks have the same priority (other than 0) in another scenario? It seems that blocking API functions don’t yield the CPU.

However, as I’m not planning to use such schema I’ll carry over the answer to this questions for the future.

Thank you all (@richard-damon, @rtel)!

@richard-damon

If you use vTaskDelay() (or related) you don’t need to call taskYIELD() as when the delay enters the block state, it will do the yield.

That’s not exactly true. In a cooperative scheduler each task must release the CPU because the API functions don’t yield it. I knew I’ve heard that in Embedded Systems 101.

The blocking API functions do not release the CPU when the kernel runs in the cooperative mode. This fact is very important for the programmers that want to implement such schema, and that they don’t fall into the same problems I did. Please see the text in bold in the quote below (from FreeRTOS):

The critical plant control functionality is once again implemented by a high priority task but the use of the cooperative scheduler necessitates a change to its implementation. Previously the timing was maintained using the vTaskDelayUntil() API function. When the preemptive scheduler was used, assigning the control task the highest priority ensured it started executing at exactly the specified time. Now the cooperative scheduler is being used – therefore a task switch will only occur when explicitly requested from the application source code so the guaranteed timing is lost.

(From FreeRTOS tutorial)

I think calling vTaskDelay() is volunteering to give up the CPU.

I think what they are talking about in that section is that the switch BACK to the high priority task won’t happen until the low priority task does a taskYIELD. ‘Blocking’ IS an explicitly requested yield. It would be totally antithetical for a delay operation in a RTOS to just spin wait for the time period and not let another task get CPU time.