FreeRTOS-Plus-TCP bug with FreeRTOS_select() with TCP socket in another task

I have found a behaviour in FreeRTOS-Plus-TCP that may be a bug. Freertos+TCP version is 4.3.2.

My application has one task using a TCP socket for handling a server function, another task receives data from many UDP sockets and a third task provides a TFTP server also with a UDP socket. All of this works fine until I add a FreeRTOS_select() to handle more than one UDP sockets in the data receiving task.

When FreeRTOS_select() is added, the TCP socket stops working. UDP sockets continue working fine. TCP ACKs continue to work, but data is not received. FreeRTOS_select() is not related to the broken TCP socket, it is used in another task for other sockets.

For the TCP socket I use the following option.

FreeRTOS_setsockopt( modbusSocket, 0, FREERTOS_SO_WAKEUP_CALLBACK, tcp_socket_callback, sizeof(void));

I have also tried with sequential listen-accept-recv method with no difference. I have also tried separate conn and recv callback, no difference with those either.

The “Bug” manifests as not receiving the proper signals that are received when FreeRTOS_select() is not used. Below is a wireshark capture of the broken situation.

And below my simple log from device (tickcount, callback event), notice the time difference of 13s between opening the connection and receiving the receive event 1.

tick 256, eventbits 10
tick 260, eventbits 10
tick 35990, eventbits 0
tick 35990, eventbits 0
tick 35993, eventbits 4
tick 48015, eventbits 1
tick 48016, eventbits 20
tick 48017, eventbits 10
tick 48020, eventbits 10

Callback is:

void tcp_socket_callback( Socket_t pxSocket )
{
	log_serial("tick %d, eventbits %x",xTaskGetTickCount(), pxSocket->xEventBits);
	if (pxSocket->xEventBits & eSOCKET_RECEIVE)	//we can have receive and send callbacks at the same time, therefore eventbits will be eSOCKET_RECEIVE | eSOCKET_SEND, but no need to process the send callback
	{
		xTaskNotify( configuratorTaskHandle, eSOCKET_RECEIVE, eSetValueWithOverwrite );
	}

	if (pxSocket->xEventBits == eSOCKET_ACCEPT)
	{
		connectedSocket = FreeRTOS_accept( modbusSocket, &modbusClientAddr, (socklen_t*)sizeof( struct freertos_sockaddr ));
		FreeRTOS_setsockopt( connectedSocket, 0, FREERTOS_SO_WAKEUP_CALLBACK, tcp_socket_callback, sizeof(void));
	}

	if (pxSocket->xEventBits == eSOCKET_CLOSED)
	{
		if (FreeRTOS_closesocket( pxSocket ) == 1)
		{
			xTaskNotify( configuratorTaskHandle, eSOCKET_CLOSED, eSetValueWithOverwrite );
		}
	}
}

Receive event appears only when the host is already closing the connection.

When FreeRTOS_Select() is not used, then I get the event’s properly:

tick 256, eventbits 10
tick 260, eventbits 10
tick 4114, eventbits 0
tick 4114, eventbits 0
tick 4116, eventbits 4
tick 4118, eventbits 1
tick 4120, eventbits 2
tick 4124, eventbits 0
tick 4125, eventbits 20
tick 4126, eventbits 10
tick 4129, eventbits 10

And equivalent wireshark:

Anyone have any ideas how this could be fixed, or should I continue making an issue report in Github?

A quick response: pxSocket->xEventBits is a bitmask. Do not test ‘xEventBits == eSOCKET_ACCEPT’, but in stead ‘(xEventBits & eSOCKET_ACCEPT)!= 0U

These values are being used:

enum eSOCKET_EVENT
{
    eSOCKET_RECEIVE = 0x0001,
    eSOCKET_SEND = 0x0002,
    eSOCKET_ACCEPT = 0x0004,
    eSOCKET_CONNECT = 0x0008,
    eSOCKET_BOUND = 0x0010,
    eSOCKET_CLOSED = 0x0020,
    eSOCKET_INTR = 0x0040,
    eSOCKET_ALL = 0x007F
};

So please change your code accordingly and test it again.

Also, I think that it would be easier if you choose one single callback mechanism, and don’t mix them.

You are using FREERTOS_SO_WAKEUP_CALLBACK along with FreeRTOS_select().

Another option is ipconfigSOCKET_HAS_USER_SEMAPHORE.

The only important thing is that these methods call your code when there is “work to do”.

If you still find difficulties, please share the code that works with TCP and UDP.

Thanks, Hein

Thanks for the quick replies. I will try those suggestions as soon as possible.

In the meantime, maybe I document the issue better, the tasks and sockets are divided as below:

The TCP socket that breaks, is not related to the sockets used by Freertos_Select(). Select() and FREERTOS_SO_WAKEUP_CALLBACK are not mixed, they are used by different sockets in different tasks. Or are there limitations in how these mechanism are allowed to be used in an application?

I have tried using the TCP socket (nr 5 in task 3) with recv() / send() and also with FREERTOS_SO_TCP_RECV_HANDLER + FREERTOS_SO_TCP_CONN_HANDLER but the situation is the same. No receive signal and no received data if freertos_select() is used in Task 1. Even when TCP socket has broken, task 1 and task 2 work fine.

I believe I can use FREERTOS_SO_UDP_RECV_HANDLER combined with socket IDs instead of freertos_select() in task 1. At least in preliminary test it seems to work and I can go around the problem.

Unfortunately does not affect the issue.

Thank you, that is a clear sketch!

I believe I can use FREERTOS_SO_UDP_RECV_HANDLER combined with socket IDs instead of freertos_select() in task 1. At least in preliminary test it seems to work and I can go around the problem.

Using socket ID’s is indeed very useful:

    BaseType_t xSocketSetSocketID( const Socket_t xSocket,
                                   void * pvSocketID );

    void * pvSocketGetSocketID( const ConstSocket_t xSocket );

So when there is a callback, it is easy to find the corresponding “object”. I use it in C++ where pvSocketID is actually an instance of a class.

About select() : can you please show the code that uses select()? I am not aware of any problem related to select.

As a test, I wrote an FTP and HTTP server that makes use of select(): protocols.

The following functions possibly don’t like to be called during a callback:

FreeRTOS_accept()
FreeRTOS_closesocket()

All you can do in a IP-callback is to wake up the task ( xTaskNotify() ) print some logging, set LEDs. The thing is that your callback is called by the IP-task, and the IP-task is not allowed to call most of the APIs. And if you do you might see deadlocks: the IP-task waiting for the IP-task.

EDIT: please check both callbacks in your code and move any API calls from the callback to the handling task.

I moved the code away from callbacks, there’s only signaling now. Unfortunately there was no effect on the issue with select().

Here’s my task code where the select() function is used. I had to redact it somewhat, but it has all the relevant parts. And I apologize that it’s so ugly. Still prototyping. My aim is to use multicasting (using the hopefully soon officially released mcast support version GitHub - evpopov/FreeRTOS-Plus-TCP at MCast_PR). I have tested select() with both official and the mcast version, but there is no difference. The version below is for testing with the official version.

uint8_t dataBuffer[MESSAGE_UDP_LEN];

void mytask( void *pvParameters )
{
	Socket_t mySocket[NUM_CHANNELS];
	struct freertos_sockaddr inputAddress[NUM_CHANNELS], sourceAddress, destAddress[NUM_CHANNELS];
	BaseType_t status;
	int32_t len = 0, packets = 0, n=0, interval, prevpkt = xTaskGetTickCount();
	uint8_t *dataTxBuffer = NULL;
	volatile UBaseType_t uxHighWaterMark;
	TickType_t timer = 0;
	uint16_t reg;
	EventBits_t dataEventBits;
	
	SocketSet_t xFD_Set;
	xFD_Set = FreeRTOS_CreateSocketSet();

	while(xIPIsNetworkTaskReady() != pdTRUE)
	{
		vTaskDelay(1000);
	}

	TickType_t socketTimeoutRx = pdMS_TO_TICKS(0);
	TickType_t socketTimeoutTx = pdMS_TO_TICKS(2);

	// Setup the local address and socket
	inputAddress[0].sin_address.ulIP_IPv4 = FreeRTOS_htonl(0);
	inputAddress[0].sin_port =  FreeRTOS_htons(5004);
	inputAddress[0].sin_family = FREERTOS_AF_INET4;
	inputAddress[1].sin_address.ulIP_IPv4 = FreeRTOS_htonl(0);
	inputAddress[1].sin_port =  FreeRTOS_htons(5005);
	inputAddress[1].sin_family = FREERTOS_AF_INET4;
	destAddress[0].sin_family = FREERTOS_AF_INET4;
	destAddress[0].sin_port =  FreeRTOS_htons(5004);
	destAddress[1].sin_family = FREERTOS_AF_INET4;
	destAddress[1].sin_port =  FreeRTOS_htons(5005);
	destAddress[0].sin_address.ulIP_IPv4 = FreeRTOS_inet_addr("224.2.0.0");
	destAddress[1].sin_address.ulIP_IPv4 = FreeRTOS_inet_addr("224.2.0.1");

	mySocket[0] = FreeRTOS_socket(inputAddress[0].sin_family, FREERTOS_SOCK_DGRAM,FREERTOS_IPPROTO_UDP);
	FreeRTOS_setsockopt(mySocket[0], 0, FREERTOS_SO_RCVTIMEO, &socketTimeoutRx,	 0);
	FreeRTOS_setsockopt(mySocket[0], 0, FREERTOS_SO_SNDTIMEO, &socketTimeoutTx, 0);
	FreeRTOS_bind(mySocket[0], &inputAddress[0], sizeof(inputAddress));

	mySocket[1] = FreeRTOS_socket(inputAddress[1].sin_family, FREERTOS_SOCK_DGRAM,FREERTOS_IPPROTO_UDP);
	FreeRTOS_setsockopt(mySocket[1], 0, FREERTOS_SO_RCVTIMEO, &socketTimeoutRx,	 0);
	FreeRTOS_setsockopt(mySocket[1], 0, FREERTOS_SO_SNDTIMEO, &socketTimeoutTx, 0);
	FreeRTOS_bind(mySocket[1], &inputAddress[1], sizeof(inputAddress));

	FreeRTOS_FD_SET(mySocket[0], xFD_Set, eSELECT_READ);
	FreeRTOS_FD_SET(mySocket[1], xFD_Set, eSELECT_READ);

	for( ;; )
	{
		status = FreeRTOS_select( xFD_Set, 0 );
		if (status != 0)
		{
			for(n=0; n<2; n++ )
			{
				if( FreeRTOS_FD_ISSET ( mySocket[n], xFD_Set ) )
				{
					len = FreeRTOS_recvfrom( mySocket[n], dataBuffer, sizeof(dataBuffer), 0, &sourceAddress, NULL );	//Receive the waiting data

					if( len > 0 )
					{
						if (getDataPacket(len, dataBuffer, &recvData) == packetOk)
						{
							xEventGroupSetBits(recvEventGroupHandle, RECV_DONE_BIT);
							EventBits_t allDoneBits = RECV_DONE_BIT | USER0_DONE_BIT;	//todo all channels interested in this data need to be processed
							eventBits = xEventGroupWaitBits(eventGroupHandle, allDoneBits, pdTRUE, pdTRUE, pdMS_TO_TICKS(5));
						}
					}
				}
			}
		}

		dataTxBuffer = FreeRTOS_GetUDPPayloadBuffer_Multi( HEADER_LEN+PAYLOAD_TX_LEN, 0, ipTYPE_IPv4);
		if (dataTxBuffer != NULL)
		{
			status = xQueueReceive(txQueue[0], dataTxBuffer, 0);

			if (status == pdTRUE)
			{
				status = FreeRTOS_sendto(mySocket[0], dataTxBuffer, HEADER_LEN+PAYLOAD_TX_LEN, FREERTOS_ZERO_COPY, &destAddress[0], sizeof(destAddress[0]));
				if( status < 0 )
				{
					FreeRTOS_ReleaseUDPPayloadBuffer( ( void * ) dataTxBuffer );	//Release the buffer in case of sendto fail, otherwise it is released internally by the ip stack
				}
			}
			else
			{
				FreeRTOS_ReleaseUDPPayloadBuffer( ( void * ) dataTxBuffer );	//Release the buffer since we didn't get data from queue
			}
		}

		uxHighWaterMark = uxTaskGetStackHighWaterMark( NULL );

		taskYIELD();
	}
}

Should that not be:

status = FreeRTOS_select( xFD_Set, pdMS_TO_TICKS( 5000 ) );

i think that select() should sleep, at least a little.

Wow, thanks! That was it.

I can confirm that with 0 delay in

status = FreeRTOS_select( xFD_Set, 0  );

The TCP socket elsewhere stops working.

But with:

status = FreeRTOS_select( xFD_Set, pdMS_TO_TICKS( 10 )  );

Everything works!

That’s great. Thank you for reporting back.