Bug in queue management?

davidefer wrote on Friday, April 05, 2019:

we were working with the Windows simulation and have discovered a possible major bug in the xQueueReceive function.

We noticed that sometimes the list iterator looped infinetly:
void vListInsert( List_t * const pxList, ListItem_t * const pxNewListItem )
got stuck here:
for( pxIterator = ( ListItem_t * ) &( pxList->xListEnd ); pxIterator->pxNext->xItemValue <= xValueOfInsertion; pxIterator = pxIterator->pxNext ) /*lint !e826 !e740 The mini list structure is used as the list end to save RAM. This is checked and valid. /
There is nothing to do here, just iterating to the wanted
insertion position. *

After having checked all the possible solutions mentioned in the code, we have noticed that the value pxQueue->xTasksWaitingToReceive.uxNumberOfItems of one of our queues got greater than 1, although only one task was waiting for messages in the queue.

It seems that the function xQueueReceive inserts more than once the same task in the queue rx waiting list.

After having added this code:
if (pxQueue->xTasksWaitingToReceive.uxNumberOfItems == 0)
vTaskPlaceOnEventList(&(pxQueue->xTasksWaitingToReceive), xTicksToWait);
everything works fine.

It seems that the “for” is executed twice and that the first time the task was already placed in the waiting list, so there is no need to place it again.
We call the function with endless timeout.

Is this a bug or just an hint to some other problems in the Win32 port?

davidefer wrote on Friday, April 05, 2019:

Actually my change doesn’t work when more than one task waits for the same queue, anyway it seems that here we have a problem.

The correct solution would be:
if( IsTaskContainedInEventList(&( pxQueue->xTasksWaitingToReceive )) == false)
vTaskPlaceOnEventList(&(pxQueue->xTasksWaitingToReceive), xTicksToWait);
but this function doesn’t exist yet.

rtel wrote on Friday, April 05, 2019:

On first read it looks like you are trying to fix a symptom, not the cause.

As you are using the Windows simulator port I’m going to guess (? - just
from similar support requests in the past) that you are using Windows
system calls in your FreeRTOS tasks. For example, are you calling
printf() (which you can maybe get away with if it is used sparingly and
from only one task as in the demos), or winsock, or disk access, or any
similar Windows system calls? If so, basically don’t, as they can cause
Windows to block threads that are supposed to be under the control of
the FreeRTOS scheduler, resulting in logic errors that can manifest
themselves as per your description.

panicdave wrote on Friday, April 05, 2019:

Here’s another instance of what appears to be the same problem, for the record:

I added a fix for the immediate problem (an item is added to the list twice, causing a short-circuit in the linked list) but it just pushed the hang/crash to somewhere else in the code. I never figured out what the ultimate cause was in my case. :frowning:

richarddamon wrote on Saturday, April 06, 2019:

One thought that came to mind, why is a task that is already waiting on the queue being added again? If it is waiting on the queue, it shouldn’t be running, and thus can’t try to wait on the queue again.

rtel wrote on Saturday, April 06, 2019:

Yep, that is why I say it is the symptom not the cause. Should never be in that state in the first place.

davidefer wrote on Saturday, April 06, 2019:

Thank you all for your time. I’ll try to understand if the situation is really the one Richard describes. I’ll let you know my findings.

P.S. thank you very much for making FreeRTOS available to all of us.

mark1122 wrote on Saturday, April 06, 2019:

I don’t understand why people use a windows simulator. Why not running it on a piece of real hardware like a Raspberry Pi? it’s under $50. Wouldn’t that be better for embedded applications?

heinbali01 wrote on Saturday, April 06, 2019:

Hi Mark, you are right: FreeRTOS was developed to run on embedded hardware, not on a Pentium something. Development boards are indeed very cheap nowadays.

A few arguments in defence of the Windows Simulator though: it behaves very realistic. I was able to discover and trace bugs in FreeRTOS projects, that were otherwise hard to find.

I also like the PCAP interface used in FreeRTOS+TCP, it allowed me to trace all network packets being sent and received. I can not run Wireshark in an embedded device (although I did caption/store packets on a device which has loads of RAM).

Another thing is that debugging of Windows Simulator projects with Visual Studio is very convenient. The IDE will detect any invalid memory access and give a lot of information when there an “invalid memory access” was detected.

To summarise: I think there are some good reasons to play with the Window Simulator project.

davidefer wrote on Monday, April 08, 2019:

Hi Richard
I’ve read again the post https://sourceforge.net/p/freertos/discussion/382005/thread/33c73d4a/ and I wanted you to confirm the following:

Outgoing messages (like demo_logging.c):

  1. Create a stream buffer to communicate with the output task.
  2. Write the stream buffer in the FreeRTOS task
  3. Use SetEvent to wake up the output task, which reads the stream buffer.

Incoming messages
Create a stream buffer to communicate with the input task.
Method 1: input task writes the stream buffer and FreeRTOS task polls ciclically the stream buffer (TCP demo).
Method 2: use vPortSetInterruptHandler to register the FreeRTOS Interrupt handler and then call vPortGenerateSimulatedInterrupt from the input task after having written the stream buffer. The FreeRTOS interrupt handler reads the stream buffer. Since the handler is run in the simulated interrupts task context, no “FromISR” calls must be done.

Is this correct?

Another question: can one use the (now) standard FreeRTOS stream_buffer.c implementation or must one use the FreeRTOS_Stream_Buffer.c implementation provided in the TCP project? I’ve seen that the latter is quite simpler but doesn’timplement concurrency protection at all. Unfortunately both define" StreamBuffer_t"

rtel wrote on Monday, April 08, 2019:

Kind of, but note the stream buffers in the logging example are not the
FreeRTOS stream buffer, but the stream buffer used in the TCP/IP stack
(confusing naming the same) - effectively a lock-less circular buffer so
data can be sent without needing to share locks or other primitive that
is incompatible between Windows and FreeRTOS.

davidefer wrote on Tuesday, April 09, 2019:

Hi Richard
after inspecting our application we have found out that actually we do Win32 calls in this app.

What we have discovered is that these calls can simply be protected with a taskENTER_CRITICAL(); Win32call; taskEXIT_CRITICAL(); sequence. The interrupts mutex is acquired so that no FreeRTOS tasks and/or simulated interrupts can be disturbed by the Win32 calls, provided that the latter are kind of “atomic” in respect to our application. In this way, the queue strange behavior doesn’t appear anymore. Of course, for “incoming” messages from the Win32 system we have to implement the ring buffer approach.