ATSAM4E16C port.c assert in vTaskStepTick()

lantczak wrote on Wednesday, August 22, 2018:

Hi,

After switch from version 9.0.0 to 10.0.1 from time to time I get assert in vTaskStepTick(). CPU slept longer then expected.
configASSERT( ( xTickCount + xTicksToJump ) <= xNextTaskUnblockTime );

Additional info:
CPU: ATSAM4E16C
Compiler: GCC 4.8.3.277
FreeRTOS: 10.0.1
PORT from: FreeRTOSv10.0.1\FreeRTOS\Source\portable\GCC\ARM_CM4F

The issue is quite hard to reproduce. It seems that it is happening when writing to flash is run by CPU (but i’m not sure). Keep in mind that this CPU has one flash controller and during writing to flash RAM function is called and IRQs are blocked

BR
Lukasz Antczak

rtel wrote on Wednesday, August 22, 2018:

Are you using the default tickless implementation that uses the SysTick
timer, or have you implemented your own that makes use of clocks
specific to the chip?

Is there a chance that you are writing to Flash immediately that the
system unblocks, before the time has been checked and corrected?

lantczak wrote on Wednesday, August 22, 2018:

Hi Richard,

I’m using standard implementation (Systick) from FreeRTOSv10.0.1\FreeRTOS\Source\portable\GCC\ARM_CM4F

I never run flash writing in IRQ. It runs always in task context and as I understand freertos code task code should not run before time correction.

lantczak wrote on Thursday, August 23, 2018:

I used vPortSuppressTicksAndSleep() function from v9.0.0 and problem disappered. So the problem has to be somewhere in this function.

Comparing changes I see issue in line 79 (see below code). The IRQ is enabled so it will run and systick could count higher than expected if sleep time + IRQ execution time is higher than expected sleep time. Am I right?

1  	__attribute__((weak)) void vPortSuppressTicksAndSleep( TickType_t xExpectedIdleTime )
2  	{
3  	uint32_t ulReloadValue, ulCompleteTickPeriods, ulCompletedSysTickDecrements;
4  	TickType_t xModifiableIdleTime;
5  
6  		/* Make sure the SysTick reload value does not overflow the counter. */
7  		if( xExpectedIdleTime > xMaximumPossibleSuppressedTicks )
8  		{
9  			xExpectedIdleTime = xMaximumPossibleSuppressedTicks;
10 		}
11 
12 		/* Stop the SysTick momentarily.  The time the SysTick is stopped for
13 		is accounted for as best it can be, but using the tickless mode will
14 		inevitably result in some tiny drift of the time maintained by the
15 		kernel with respect to calendar time. */
16 		portNVIC_SYSTICK_CTRL_REG &= ~portNVIC_SYSTICK_ENABLE_BIT;
17 
18 		/* Calculate the reload value required to wait xExpectedIdleTime
19 		tick periods.  -1 is used because this code will execute part way
20 		through one of the tick periods. */
21 		ulReloadValue = portNVIC_SYSTICK_CURRENT_VALUE_REG + ( ulTimerCountsForOneTick * ( xExpectedIdleTime - 1UL ) );
22 		if( ulReloadValue > ulStoppedTimerCompensation )
23 		{
24 			ulReloadValue -= ulStoppedTimerCompensation;
25 		}
26 
27 		/* Enter a critical section but don't use the taskENTER_CRITICAL()
28 		method as that will mask interrupts that should exit sleep mode. */
29 		__asm volatile( "cpsid i" ::: "memory" );
30 		__asm volatile( "dsb" );
31 		__asm volatile( "isb" );
32 
33 		/* If a context switch is pending or a task is waiting for the scheduler
34 		to be unsuspended then abandon the low power entry. */
35 		if( eTaskConfirmSleepModeStatus() == eAbortSleep )
36 		{
37 			/* Restart from whatever is left in the count register to complete
38 			this tick period. */
39 			portNVIC_SYSTICK_LOAD_REG = portNVIC_SYSTICK_CURRENT_VALUE_REG;
40 
41 			/* Restart SysTick. */
42 			portNVIC_SYSTICK_CTRL_REG |= portNVIC_SYSTICK_ENABLE_BIT;
43 
44 			/* Reset the reload register to the value required for normal tick
45 			periods. */
46 			portNVIC_SYSTICK_LOAD_REG = ulTimerCountsForOneTick - 1UL;
47 
48 			/* Re-enable interrupts - see comments above the cpsid instruction()
49 			above. */
50 			__asm volatile( "cpsie i" ::: "memory" );
51 		}
52 		else
53 		{
54 			/* Set the new reload value. */
55 			portNVIC_SYSTICK_LOAD_REG = ulReloadValue;
56 
57 			/* Clear the SysTick count flag and set the count value back to
58 			zero. */
59 			portNVIC_SYSTICK_CURRENT_VALUE_REG = 0UL;
60 
61 			/* Restart SysTick. */
62 			portNVIC_SYSTICK_CTRL_REG |= portNVIC_SYSTICK_ENABLE_BIT;
63 
64 			/* Sleep until something happens.  configPRE_SLEEP_PROCESSING() can
65 			set its parameter to 0 to indicate that its implementation contains
66 			its own wait for interrupt or wait for event instruction, and so wfi
67 			should not be executed again.  However, the original expected idle
68 			time variable must remain unmodified, so a copy is taken. */
69 			xModifiableIdleTime = xExpectedIdleTime;
70 			configPRE_SLEEP_PROCESSING( xModifiableIdleTime );
71 			if( xModifiableIdleTime > 0 )
72 			{
73 				__asm volatile( "dsb" ::: "memory" );
74 				__asm volatile( "wfi" );
75 				__asm volatile( "isb" );
76 			}
77 			configPOST_SLEEP_PROCESSING( xExpectedIdleTime );
78 
79 			/* Re-enable interrupts to allow the interrupt that brought the MCU
80 			out of sleep mode to execute immediately.  see comments above
81 			__disable_interrupt() call above. */
82 			__asm volatile( "cpsie i" ::: "memory" );
83 			__asm volatile( "dsb" );
84 			__asm volatile( "isb" );
85 
86 			/* Disable interrupts again because the clock is about to be stopped
87 			and interrupts that execute while the clock is stopped will increase
88 			any slippage between the time maintained by the RTOS and calendar
89 			time. */
90 			__asm volatile( "cpsid i" ::: "memory" );
91 			__asm volatile( "dsb" );
92 			__asm volatile( "isb" );
93 
94 			/* Disable the SysTick clock without reading the
95 			portNVIC_SYSTICK_CTRL_REG register to ensure the
96 			portNVIC_SYSTICK_COUNT_FLAG_BIT is not cleared if it is set.  Again,
97 			the time the SysTick is stopped for is accounted for as best it can
98 			be, but using the tickless mode will inevitably result in some tiny
99 			drift of the time maintained by the kernel with respect to calendar
100			time*/
101			portNVIC_SYSTICK_CTRL_REG = ( portNVIC_SYSTICK_CLK_BIT | portNVIC_SYSTICK_INT_BIT );
102
103			/* Determine if the SysTick clock has already counted to zero and
104			been set back to the current reload value (the reload back being
105			correct for the entire expected idle time) or if the SysTick is yet
106			to count to zero (in which case an interrupt other than the SysTick
107			must have brought the system out of sleep mode). */
108			if( ( portNVIC_SYSTICK_CTRL_REG & portNVIC_SYSTICK_COUNT_FLAG_BIT ) != 0 )
109			{
110				uint32_t ulCalculatedLoadValue;
111
112				/* The tick interrupt is already pending, and the SysTick count
113				reloaded with ulReloadValue.  Reset the
114				portNVIC_SYSTICK_LOAD_REG with whatever remains of this tick
115				period. */
116				ulCalculatedLoadValue = ( ulTimerCountsForOneTick - 1UL ) - ( ulReloadValue - portNVIC_SYSTICK_CURRENT_VALUE_REG );
117
118				/* Don't allow a tiny value, or values that have somehow
119				underflowed because the post sleep hook did something
120				that took too long. */
121				if( ( ulCalculatedLoadValue < ulStoppedTimerCompensation ) || ( ulCalculatedLoadValue > ulTimerCountsForOneTick ) )
122				{
123					ulCalculatedLoadValue = ( ulTimerCountsForOneTick - 1UL );
124				}
125
126				portNVIC_SYSTICK_LOAD_REG = ulCalculatedLoadValue;
127
128				/* As the pending tick will be processed as soon as this
129				function exits, the tick value maintained by the tick is stepped
130				forward by one less than the time spent waiting. */
131				ulCompleteTickPeriods = xExpectedIdleTime - 1UL;
132			}
133			else
134			{
135				/* Something other than the tick interrupt ended the sleep.
136				Work out how long the sleep lasted rounded to complete tick
137				periods (not the ulReload value which accounted for part
138				ticks). */
139				ulCompletedSysTickDecrements = ( xExpectedIdleTime * ulTimerCountsForOneTick ) - portNVIC_SYSTICK_CURRENT_VALUE_REG;
140
141				/* How many complete tick periods passed while the processor
142				was waiting? */
143				ulCompleteTickPeriods = ulCompletedSysTickDecrements / ulTimerCountsForOneTick;
144
145				/* The reload value is set to whatever fraction of a single tick
146				period remains. */
147				portNVIC_SYSTICK_LOAD_REG = ( ( ulCompleteTickPeriods + 1UL ) * ulTimerCountsForOneTick ) - ulCompletedSysTickDecrements;
148			}
149
150			/* Restart SysTick so it runs from portNVIC_SYSTICK_LOAD_REG
151			again, then set portNVIC_SYSTICK_LOAD_REG back to its standard
152			value. */
153			portNVIC_SYSTICK_CURRENT_VALUE_REG = 0UL;
154			portNVIC_SYSTICK_CTRL_REG |= portNVIC_SYSTICK_ENABLE_BIT;
155			vTaskStepTick( ulCompleteTickPeriods );
156			portNVIC_SYSTICK_LOAD_REG = ulTimerCountsForOneTick - 1UL;
157
158			/* Exit with interrpts enabled. */
159			__asm volatile( "cpsie i" ::: "memory" );
160		}
161	}

rtel wrote on Thursday, August 23, 2018:

I see what you mean - I’m on a small screen at the moment without access
to the old version - and this is going to take a little digging - so I
will report back after investigating.

lantczak wrote on Friday, August 24, 2018:

Aditional info: I run tests with version 10.0.1 + one change: put stoping of Systick before re-enabling interrupts intead after like in original. Problem is not existing.
My code:

			/* Disable the SysTick clock without reading the
			portNVIC_SYSTICK_CTRL_REG register to ensure the
			portNVIC_SYSTICK_COUNT_FLAG_BIT is not cleared if it is set.  Again,
			the time the SysTick is stopped for is accounted for as best it can
			be, but using the tickless mode will inevitably result in some tiny
			drift of the time maintained by the kernel with respect to calendar
			time*/
			portNVIC_SYSTICK_CTRL_REG = ( portNVIC_SYSTICK_CLK_BIT | portNVIC_SYSTICK_INT_BIT );

			/* Re-enable interrupts to allow the interrupt that brought the MCU
			out of sleep mode to execute immediately.  see comments above
			__disable_interrupt() call above. */
			__asm volatile( "cpsie i" ::: "memory" );
			__asm volatile( "dsb" );
			__asm volatile( "isb" );

			/* Disable interrupts again because the clock is about to be stopped
			and interrupts that execute while the clock is stopped will increase
			any slippage between the time maintained by the RTOS and calendar
			time. */
			__asm volatile( "cpsid i" ::: "memory" );
			__asm volatile( "dsb" );
			__asm volatile( "isb" );

			/* Determine if the SysTick clock has already counted to zero and
			been set back to the current reload value (the reload back being
			correct for the entire expected idle time) or if the SysTick is yet
			to count to zero (in which case an interrupt other than the SysTick
			must have brought the system out of sleep mode). */
			if( ( portNVIC_SYSTICK_CTRL_REG & portNVIC_SYSTICK_COUNT_FLAG_BIT ) != 0 )

lantczak wrote on Friday, August 31, 2018:

Richard any comment? Do you see any risk of my change?

rtel wrote on Friday, August 31, 2018:

Sorry for the delay - I’ve not looked deep into this yet - but the
change you make basically reverts the code back and it was changed for a
very specific reason (which I can’t recall right now, but it was
discussed in the forums).

lctromnml wrote on Saturday, February 16, 2019:

Is there perhaps an update on this issue? I am experiencing the same problem after switching to FreeRTOS v10.1.1 also with an ATMEL uC.

CPU: ATSAM4SA16B
Compiler: GCC 7.4.0
FreeRTOS: 10.1.1
PORT from: FreeRTOSv10.1.1\FreeRTOS\Source\portable\GCC\ARM_CM3

configTICK_RATE_HZ set to 1000
configEXPECTED_IDLE_TIME_BEFORE_SLEEP set to 2 (default)
configSYSTICK_CLOCK_HZ set to configCPU_CLOCK_HZ (default)
configUSE_TICKLESS_IDLE set to 1 so also the default tick suppression

Unfortunately it happens very sporadic, usually not even once per day, maximum was 3 times in 24 hours, so it is hard to trace. I could store some variables when it happens if this would help.

lctromnml wrote on Monday, March 25, 2019:

I think I have solved the problem (at least for me). In the errata of the microcontroller I found the following:

"Issue: Unpredictable Behavior When Entering Sleep Mode

When entering Sleep mode, if an interrupt occurs during WFI or WFE (PMC_FSMR.LPM = 0) instruction processing, the ARM core may read an incorrect data, thus leading to unpredictable behavior of the software. This issue is not present in Wait mode.

Workaround: The following conditions must be met:

  1. The interrupt vector table must be located in Flash.
  2. The Matrix slave interface for the Flash must be set to ‘No default master’. This is done by setting the field DEFMSTR_TYPE to 0 in the register MATRIX_SCFG. The code example below can be used to program the NO_DEFAULT_MASTER state:
    MATRIX_SCFG[2] = MATRIX_SCFG_SLOT_CYCLE(0xFF) | MATRIX_SCFG_DEFMSTR_TYPE(0x0);
    This must be done once in the software before entering Sleep mode."

I followed the workaround and did not have any configASSERTs since then anymore.

lantczak wrote on Wednesday, August 14, 2019:

The problem start occurring again in stress test scenario when a lot of interrupts is generated. In my case it was CAN IRQ. With frequency 2k of interrupts the issue occurs after ~1h of testing.

I added 3x NOP() just after wake up and before enabling IRQ and for some reason it solve the problem. Not nice solution but after 2 days of testing the problem disappear. I’m waiting for answer from Atmel