application crashes in xQueueReceive, vListInsert respectively

hristozov wrote on Wednesday, April 13, 2016:

Hello,

I’m using STM32F405 with FreeRTOS 8.0.1. My application has several tasks. Task B has priority 5 and task S priority 3.
I try to send an item using queue from B to S.
After I call xQueueSend in Task B the application crashes in S in xQueueReceive respectively vListInsert.

I have set all priority bits to preempt priority at the beginning of the main before any freeRTOS initializations with
NVIC_PriorityGroupConfig(NVIC_PriorityGroup_4);

Here is my FreeRTOSConfig.h

#ifndef FREERTOS_CONFIG_H
#define FREERTOS_CONFIG_H

/*-----------------------------------------------------------
 * Application specific definitions.
 *
 * These definitions should be adjusted for your particular hardware and
 * application requirements.
 *
 * THESE PARAMETERS ARE DESCRIBED WITHIN THE 'CONFIGURATION' SECTION OF THE
 * FreeRTOS API DOCUMENTATION AVAILABLE ON THE FreeRTOS.org WEB SITE.
 *
 * See http://www.freertos.org/a00110.html.
 *----------------------------------------------------------*/

/* Ensure stdint is only used by the compiler, and not the assembler. */

#include <stdint.h>
extern uint32_t SystemCoreClock;

#define configUSE_PREEMPTION			1
#define configUSE_IDLE_HOOK				0
#define configUSE_TICK_HOOK				0
#define configCPU_CLOCK_HZ				( SystemCoreClock )
#define configTICK_RATE_HZ				( ( TickType_t ) 1000 )
#define configMAX_PRIORITIES			( 12 ) 						/*This number must be keep as small as possible, in order to safe RAM*/
#define configMINIMAL_STACK_SIZE		( ( unsigned short ) 130 )
#define configTOTAL_HEAP_SIZE			( ( size_t ) ( 75 * 1024 ) )
#define configMAX_TASK_NAME_LEN			( 10 )
#define configUSE_TRACE_FACILITY		1
#define configUSE_16_BIT_TICKS			0
#define configIDLE_SHOULD_YIELD			1
#define configUSE_MUTEXES				1
#define configQUEUE_REGISTRY_SIZE		8
#define configCHECK_FOR_STACK_OVERFLOW	0
#define configUSE_RECURSIVE_MUTEXES		1
#define configUSE_MALLOC_FAILED_HOOK	0
#define configUSE_APPLICATION_TASK_TAG	0
#define configUSE_COUNTING_SEMAPHORES	1
#define configGENERATE_RUN_TIME_STATS	0

/* Co-routine definitions. */
#define configUSE_CO_ROUTINES 		0
#define configMAX_CO_ROUTINE_PRIORITIES ( 2 )

/* Software timer definitions. */
#define configUSE_TIMERS				0
#define configTIMER_TASK_PRIORITY		( 2 )
#define configTIMER_QUEUE_LENGTH		10
#define configTIMER_TASK_STACK_DEPTH	( configMINIMAL_STACK_SIZE * 2 )

/* Set the following definitions to 1 to include the API function, or zero
 to exclude the API function. */
#define INCLUDE_vTaskPrioritySet		1
#define INCLUDE_uxTaskPriorityGet		1
#define INCLUDE_vTaskDelete				1
#define INCLUDE_vTaskCleanUpResources	1
#define INCLUDE_vTaskSuspend			1
#define INCLUDE_vTaskDelayUntil			1
#define INCLUDE_vTaskDelay				1
#define INCLUDE_uxTaskGetStackHighWaterMark     1

/* Cortex-M specific definitions. */
#ifdef __NVIC_PRIO_BITS
/* __BVIC_PRIO_BITS will be specified when CMSIS is being used. */
#define configPRIO_BITS       		__NVIC_PRIO_BITS
#else
#define configPRIO_BITS       		4        /* 15 priority levels */
#endif

/* The lowest interrupt priority that can be used in a call to a "set priority"
 function. */
#define configLIBRARY_LOWEST_INTERRUPT_PRIORITY			0xf

/* The highest interrupt priority that can be used by any interrupt service
 routine that makes calls to interrupt safe FreeRTOS API functions.  DO NOT CALL
 INTERRUPT SAFE FREERTOS API FUNCTIONS FROM ANY INTERRUPT THAT HAS A HIGHER
 PRIORITY THAN THIS! (higher priorities are lower numeric values. */
#define configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY	5

/* Interrupt priorities used by the kernel port layer itself.  These are generic
 to all Cortex-M ports, and do not rely on any particular library functions. */
#define configKERNEL_INTERRUPT_PRIORITY 		( configLIBRARY_LOWEST_INTERRUPT_PRIORITY << (8 - configPRIO_BITS) )
/* !!!! configMAX_SYSCALL_INTERRUPT_PRIORITY must not be set to zero !!!!
 See http://www.FreeRTOS.org/RTOS-Cortex-M3-M4.html. */
#define configMAX_SYSCALL_INTERRUPT_PRIORITY 	( configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY << (8 - configPRIO_BITS) )

/* Normal assert() semantics without relying on the provision of an assert.h
 header file. */
#define configASSERT( x ) if( ( x ) == 0 ) { taskDISABLE_INTERRUPTS(); for( ;; ); }	

/* Definitions that map the FreeRTOS port interrupt handlers to their CMSIS
 standard names. */
#define vPortSVCHandler SVC_Handler
#define xPortPendSVHandler PendSV_Handler
#define xPortSysTickHandler SysTick_Handler

#endif /* FREERTOS_CONFIG_H */

I have set the priorities of all my interrupts to 0xf. with
NVIC_IRQChannelPreemptionPriority = 0xf;

I have also check for:

  1. Stack overflow - using uxTaskGetStackHighWaterMark - there are no stack overflows
  2. I call xQueueSend from a task and not from a ISR
  3. I call xQueueSend after vTaskStartScheduler()

So I think something is not correct with my interrupt priorities.

rtel wrote on Wednesday, April 13, 2016:

The priority bits would only make a difference if you were using the API
in an ISR. Your post suggests you are NOT using the API in an ISR. Can
you confirm that is the case?

Please post the code that is sending to the queue, and the code that is
receiving from the queue.

hristozov wrote on Wednesday, April 13, 2016:

I am NOT using the API in an ISR.

The crash happens after calling xQueueSend in bt_application_spp_disconnect_callback in /bluetooth_lib_v2_0_1/application/application_spp.c. This function is called from task_bluetooth. See call three in the picture.
I receive the queue in task_systemStateLogic.
Please find attached all tasks and the bluetooth library containing application_spp.c.

rtel wrote on Wednesday, April 13, 2016:

I cannot see anything obvious here. The nesting depth to the call in
question is very deep, but it looks like you are obtaining the stack
high water mark on each side, so presumably have ascertained that is not
a problem.

What is vPrintf() doing? Anything? printf() can often cause issues if
called from more than one task, or if it is buffering.

Other than that I think you are going to have to step into the function
to see where it is crashing.

How is it crashing? Does it generate a fault? If so the information on
the following page might help:
http://www.freertos.org/Debugging-Hard-Faults-On-Cortex-M-Microcontrollers.html

hristozov wrote on Wednesday, April 13, 2016:

vPrintf() is turn off with a macro so I don’t use it.

Basically xQueueSend doesn’t break. I check before and after the call of xQueueSend if I there is a new item on the queue:

	debug_uxNumberOfFreeSpaces = uxQueueSpacesAvailable(QueueEvent);
	debug_queue = xQueueSend(QueueEvent, &event_var, 0);
	debug_uxNumberOfFreeSpaces = uxQueueSpacesAvailable(QueueEvent);

And its works - I have one less free space after this lines. xQueueSend also returns 1.

The program stops in

for( pxIterator = ( ListItem_t * ) &( pxList->xListEnd ); pxIterator->pxNext->xItemValue <= xValueOfInsertion; pxIterator = pxIterator->pxNext )

in list.c, when I tray to receive from the queue. See pic.

rtel wrote on Wednesday, April 13, 2016:

The program stops in

That shows there is a basic data corruption, which is normally related
to an interrupt priority problem - presumably why you first started
looking at your interrupt priority settings.

Are you using FreeRTOS API functions in an interrupt anywhere at all?
Which version of FreeRTOS are you using? (relevant to whether or not the
configASSERT() you have defined would catch this problem or not).

hristozov wrote on Wednesday, April 13, 2016:

8.0.1

rtel wrote on Wednesday, April 13, 2016:

That appears to have vPortValidateInterruptPriority() defined (see the
last function in the link below), although it may not check as many
different use cases as the latest code versions:

https://sourceforge.net/p/freertos/code/HEAD/tree/tags/V8.0.1/FreeRTOS/Source/portable/GCC/ARM_CM3/port.c

Regards.

hristozov wrote on Wednesday, April 13, 2016:

I don’t understand? Can you explain how to use vPortValidateInterruptPriority()?

rtel wrote on Wednesday, April 13, 2016:

If configASSERT() is defined, which in your case it is, it gets called
automatically and will trigger a configASSERT() failure if the interrupt
priority is wrong. You don’t need to explicitly use it.

hristozov wrote on Wednesday, April 13, 2016:

I did some more research on that problem and found out an interesting thing.
My starting point was to use a queue holding items of enum type like:

typedef enum {
	power_up_or_reset, //0
	button_pressed_3x, //1
	button_pressed_1x, //2
	time_out_visible, //3
	app_connected_bonded, //4
	app_connected_not_bonded, //5
	select, //6
	app_disconnect, //7
	usb_plugged_in, //8
	pc_usb, //9
	button_hold, //10
	plug_off, //11
	charging_complete, //12
	MAX_EVENTS
} systemEvents_t;

I created the queue like this:
QueueEvent = xQueueCreate(8,sizeof(systemEvents_t));
Using this queue my
xQueueReceive(QueueEvent, &new_event, portMAX_DELAY);
was always crashing.

After this I have change the type of the queue items to uint32_t :
QueueEvent = xQueueCreate(8,sizeof(uint32_t));
Now my
xQueueReceive(QueueEvent, &new_event, portMAX_DELAY);
receives correctly what I was sending but instead to put the task into blocking state at the next iteration of the infinite task loop (because of the portMAX_DELAY) the execution stops again at
for( pxIterator = ( ListItem_t * ) &( pxList->xListEnd ); pxIterator->pxNext->xItemValue <= xValueOfInsertion; pxIterator = pxIterator->pxNext )

I think I misunderstood something according to the usage of the xQueueReceive/Send. Is it possible to use queues with enums?

rtel wrote on Wednesday, April 13, 2016:

For the reasons explained on this page
http://www.freertos.org/Embedded-RTOS-Queues.html queues pass data by copy.

When you create the queue you specify how many bytes required to hold
each item in the queue. So your

QueueEvent = xQueueCreate(8,sizeof(systemEvents_t));

says create a queue that can hold 8 items, where each item is however
many bytes each statemEvent_t variable holds.

Then, when you receive data, you must provide a buffer that is at least
this many bytes, into which the data is copied. So in your call:

xQueueReceive(QueueEvent, &new_event, portMAX_DELAY);

new_event must be a variable (not a pointer) that is at least as big as
systemEvent_t, as that many bytes will copied out of the queue into the
variable.

There is nothing wrong with that provided the compiler always returns
the same value for sizeof( systemEvents_t ) - which is should do. The
size of an enum can be changed at compile time, or the compiler may
choose the most efficient type, but by rights really it should be an int.

In any case, if you were to substitute uint32_t for statemEvent_t then
all the above holds true, except you know uint32_t is always going to
have a size of 4 (4 bytes for each uint32_t).

So, basically, it shouldn’t make a difference.

richard_damon wrote on Thursday, April 14, 2016:

The key is that the variable passed to the send/receive functions should always be the same size, which normally means allways the same type (they could also be an array of bytes long enough, but that just things complicated).

One issue with enums is that they are not required to be any particular size. They typically are ints (not int32_t, unless int = int32_t), but are allowed to be different. In your case, since the maximum value of an enum value is 13, the enum could be as small as a single byte.

hristozov wrote on Friday, April 15, 2016:

You are right Richard this enum is 8 bit long. I also always use the queue send/receive with the same size variable. Is it possible that freeRTOS 32 bit enums expects?

rtel wrote on Friday, April 15, 2016:

Is it possible that
freeRTOS 32 bit enums expects?

No, FreeRTOS is portable and makes no assumptions like that, especially
assumptions that will be wrong most of the time.

As per my previous post, you set the size of the queue item when you
create the queue, in your case in the same way I would have done by
using sizeof(). Whatever is returned from sizeof() is the number of
bytes that will be copied into and out of the queue - the FreeRTOS
source code (remember it is source code) cannot know in advance that you
are going to have an enum inside your call to sizeof() so cannot make
such an assumption.

hristozov wrote on Monday, April 18, 2016:

Hallo,

I have more information on my problem.
As mentioned previously the task that sends to the queue has higher priority then the task that receives.
In the sender task I call xQueueSend(QueueEvent, &event_var, 0);
In the receiver task I get correctly the sent value with xQueueReceive(QueueEvent, &new_event, portMAX_DELAY);
the execution goes ones through the infinite loop of the receiver task and comes again to xQueueReceive(QueueEvent, &new_event, portMAX_DELAY).
Here the task should block on an new item in the queue, but instead it crashes in for( pxIterator = ( ListItem_t * ) &( pxList->xListEnd ); pxIterator->pxNext->xItemValue <= xValueOfInsertion; pxIterator = pxIterator->pxNext )

I also tried to use a semaphore instead of the queue. This works correctly.

rtel wrote on Monday, April 18, 2016:

The semaphore and queue code is basically the same - the difference with the semaphore is that the item size is 0. So what you have determined is that if the item size is 0 or if the item size is 4, then it is ok, but if the item size if 1 then it is not ok. That makes me wonder if it is an alignment issue. The FreeRTOS code is written so alignment doesn’t matter, but some time libraries try and be clever and get this wrong.

I wonder, are you using GCC with an optimisation level greater than 0? I have seen GCC at -O0 optimisation call the C run time memcpy() library function, which behaves as expected, but at -O1 and higher optimisation replace the call with builtin_memcpy(), which attempts word moves on byte data.

If you are using GCC, does the problem still exist at -O0 optimisation?

hristozov wrote on Monday, April 18, 2016:

Yes I am using GCC with -O0. This is my compiler string: -mcpu=cortex-m4; -mfpu=fpv4-sp-d16; -mfloat-abi=hard; -mthumb; -Wall; -ffunction-sections; -g; -O0; -DSTM32F405VG; -DSTM32F4XX; -D__FPU_USED; -DUSE_STDPERIPH_DRIVER; -D__ASSEMBLY__; -I.;

davedoors wrote on Monday, April 18, 2016:

If it was alignment that was wrong it would more likely crash. This sounds like data corruption, so could the library be coping 4 bytes when it should copy 1?

hristozov wrote on Thursday, April 21, 2016:

After few more days of studying the freeRTOS kernel and debugging a have discovered the following:

1.) Set up:
I have only one task that receives from a queue. When i send an item to the queue from the same task it is received without problems.
I used a break point after the xQueueuReceive and checked the value of the queue in the debugger:

“(Queue_t *) QueueEvent” 0x20000978
pcHead 0x200009d0
pcTail 0x20000a10
pcWriteTo 0x200009d4
u {…}
xTasksWaitingToSend {…}
xTasksWaitingToReceive {…}
uxNumberOfItems 0
pxIndex 0x200009a4
xListEnd {…}
uxMessagesWaiting 0
uxLength 16
uxItemSize 4
xRxLock -1
xTxLock -1
uxQueueNumber 0
ucQueueType 0

As we see uxMessagesWaiting	is 0 and xTasksWaitingToReceive	-> uxNumberOfItems	is also 0. 

When I send to the queue from one different task I also receive the sent item correctly but xTasksWaitingToReceive -> uxNumberOfItems = 1 (uxMessagesWaiting = 0)

“(Queue_t *) QueueEvent” 0x20000978
pcHead 0x200009d0
pcTail 0x20000a10
pcWriteTo 0x200009dc
u {…}
xTasksWaitingToSend {…}
xTasksWaitingToReceive {…}
uxNumberOfItems 1
pxIndex 0x200009a4
xListEnd {…}
uxMessagesWaiting 0
uxLength 16
uxItemSize 4
xRxLock -1
xTxLock -1
uxQueueNumber 0
ucQueueType 0

Why I still have items in the TasksWaitingToReceive List?? I have only one task that blocks on this queue and I already had read from it.