Queue data order change?

gigglergigger wrote on Friday, October 23, 2015:

I have a Cortex app with about 15 tasks each with various i/o queues. I’ve enabled stack checking in config and asserts - no issues.

I have a highest priority task RS485 receiving and sending data to a CAN bus queue. The task that receives this CAN data is lower priority than the sender but the sender is filling the queue slow.

I am seeing data order is changing. All the queues use xQueueSendToBack(…)

sent to CAN queue:
80 ed 11 42 37 54 4c 20 31 38 37 39 36 20 20 20 20 20 20 4a

actually received over CAN bus:
80 ed 42 37 54 4c 20 31 38 37 39 36 20 20 20 20 20 20 4a 11

It seems very consistent. The queue size is larger than 32 CAN frames. There is timing data in the frames that I’ve missed out, so the above is just some characters.

This is the basic CAN sender task (simplified) and queue.

while(1) {
	// wait here until we get some data to send
	if(xQueueReceive(gxCANData.txQueue[1], &msg, portMAX_DELAY)) {

The sender in it’s simplest form is basically doing:

for(ucI=0;ucI<ucNFrames;ucI++) {
xQueueSendToBack(…, frame[ucl], portMAX_DELAY )

If I break point in the for(…) loop above, I can see the correct order data.

My CAN analyser shows frames where the squence is out of order.

Strangely I cannot check the queue in the CAN sender task to see if the order there is different. If I breakpoint xQueueReceive(…) I only get 1 entry with first character. However the tasks all still function and send and receive data if I disable the break point to take data from other tasks that are lower or at the same priority. The other tasks using the bus still work sending data. The queue is larger than data being sent in this case and all sends should block waiting for queue space anyway, i.e. send to back with portMAX_DELAY.

Any suggestions?



heinbali01 wrote on Saturday, October 24, 2015:

data order is changing

The FreeRTOS queues always behave like a FIFO. If it doesn’t, then something is wrong. Maybe some memory corruption caused by another task? Maybe the data have been ‘reordered’ earlier without knowing it? Maybe you missed data from the RS485 bus while doing other things?

Do you still have some RAM left for debugging purposes?

What I would do is add some checking: store all events in a circular buffer along with a time-stamp.

I have a highest priority task RS485 receiving and sending data to a CAN bus queue

Does it ever happen that this priority task task has to wait for space in the queue? If so, could it miss data from the RS485 bus?

I would store every relevant event:

    for(ucI=0;ucI<ucNFrames;ucI++) {
    BaseType_t xResult;
        xResult = xQueueSendToBack(..., frame[ucl], portMAX_DELAY );
        vAddEvent( 'S', msg, xResult );

BaseType_t xResult;
    xResult = xQueueReceive(gxCANData.txQueue[1], &msg, portMAX_DELAY);
    if( xResult != pdFALSE ) {
        vAddEvent( 'R', msg, xResult );
        xISO11898Send(LPC_CAN2, &msg, eCANTXBLOCK_WAITFREE);

And here is a sketch of such a circular buffer:

#define EVENT_COUNT      64

typedef struct {
    char cFunction;
    /* In stead of storing the entire messages,
    you might store a readable string here. */
    char pcMsg[ 8 ];
    BaseType_t xResult;
    uint32_t ulTime;
} Event_t;

Event_t xEvents[ EVENT_COUNT ];
volatile BaseType_t xEventIndex, xEventFrozen;

void vAddEvent( char cFunction, Msg_t *pxMsg, BaseType_t xResult )
BaseType_t xIndex = xEventIndex;
    /* Do not touch the event list while
    it is frozen for inspection. */
    if( xEventFrozen != pdFALSE )
        Event_t *pxEvent = &xEvents[ xIndex ];
        if( ++xIndex == EVENT_COUNT )
            xIndex = 0;
        xEventIndex = xIndex;
        pxEvent->cFunction = cFunction;
        vExtractMsg( pxMsg, pxEvent->pcMsg, sizeof( pxEvent->pcMsg ) );
        pxEvent->xResult = xResult;
        /* Get a time-stamp with uS precision. */
        pxEvent->ulTime = ulGetPrecisionTime();

At some point, when the error occurs, you can inspect the data produced in the circular buffer.


gigglergigger wrote on Monday, October 26, 2015:

Hein Tibosch,

Thanks for your reply. I’m not sure why I cannot breakpoint the queue or read data without loosing it but I’ve confirmed it’s getting the data in the correct order.

It’s my CAN bus driver - the output FIFO’s are submitting frames to the bus in a strange order.