prvAcceptWaitClient gets interrupted with xSemaphoreTake while Tasks are suspended

Libraries
1

Hi,
I’m setting up a project with FreeRTOS+TCP v4.3.1 and Kernel v10.6.2 using a SAMV71Q21B and a KSZ8795 5 port switch.

While I do use the default MDIO interface I had to make some changes to the SAM drivers and networkinterface.c since they are designed with an SAME70 in mind and I use RMII instead of MII.

The setup and initialization seems to work great as far as I can tell but the moment I try to connect I run into:

    /* Cannot block if the scheduler is suspended. */
    #if ( ( INCLUDE_xTaskGetSchedulerState == 1 ) || ( configUSE_TIMERS == 1 ) )
    {
        configASSERT( !( ( xTaskGetSchedulerState() == taskSCHEDULER_SUSPENDED ) && ( xTicksToWait != 0 ) ) );
    }
    #endif

from queue.c.
After some debugging I found out that FreeRTOS_Sockets.c

    static FreeRTOS_Socket_t * prvAcceptWaitClient( FreeRTOS_Socket_t * pxParentSocket,
                                                    struct freertos_sockaddr * pxAddress,
                                                    socklen_t * pxAddressLength )
    {
        FreeRTOS_Socket_t * pxClientSocket = NULL;

        /* Is there a new client? */
        vTaskSuspendAll();

is suspending the scheduler and while the scheduler is suspended,

NetworkBufferDescriptor_t * pxGetNetworkBufferWithDescriptor( size_t xRequestedSizeBytes,
                                                              TickType_t xBlockTimeTicks )

from BufferAllocation_2.c is called which in turn runs

    if( ( xIntegerOverflowed == pdFALSE ) && ( xAllocatedBytes <= uxMaxAllowedBytes ) && ( xNetworkBufferSemaphore != NULL ) )
    {
        /* If there is a semaphore available, there is a network buffer available. */
        if( xSemaphoreTake( xNetworkBufferSemaphore, xBlockTimeTicks ) == pdPASS )

I already worked my way through Debugging-Hard-Faults-On-Cortex-M-Microcontrollers and ARM-Cortex/RTOS-Cortex-M3-M4 but I could not find any errors through that. (Would have added the links but I am to new a user for it to be allowed)
I set

	#define configMAX_LIBRARY_INTERRUPT_PRIORITY	( 5 )
	#define configMAC_INTERRUPT_PRIORITY			( configMAX_LIBRARY_INTERRUPT_PRIORITY )

while having

#define configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY 5

and ensured it’s not a hard fault. I tried to change the priorities up and down against each other and changed the priorities of the task calling prvAcceptWaitClient and the IP config task prvIPTask to no avail.

The only way to fix it so far was to either add an FreeRTOS_printf(("\r\n"));
after usGenerateProtocolChecksum( pucBuffer, uxLength, pdTRUE ); in void vGMACGenerateChecksum( uint8_t * pucBuffer, size_t uxLength ) or remove the FreeRTOS_printf( ( "prvAcceptWaitClient: client %p parent %p\n", ( void * ) pxClientSocket, ( void * ) pxParentSocket ) ); in static FreeRTOS_Socket_t * prvAcceptWaitClient( FreeRTOS_Socket_t * pxParentSocket, struct freertos_sockaddr * pxAddress, socklen_t * pxAddressLength ).

The latter is my current way to go since I don’t see the necessity of the print either way and I want to generate my checksums with the much more efficient hardware module of my V71.

In any case I am not too happy with the solution since I expect it to be some kind of configuration error or whatnot but I don’t really know what to check now or how to find my root cause and fix it.

Any help would be appreciated!

2

The basic problem is you can’t do a xSemaphoreTake with a non-zero block time while the Scheduler is suspended as you can’t block in that case. That means you can’t call any function that uses it while the scheduler is suspended.

3

Yes but I am not doing that manually. All I do is call xConnectedSocket = FreeRTOS_accept( xListeningSocket, &xClient, &xSize ); in my main task. I did not touch the basic handling inside the TCP stack.

The mentioned changes are - for example - enabling the reset pin of my switch in the setup phase:

vPhyInitialise( &xPhyObject, xPHY_Read, xPHY_Write );
gpio_set_pin_level(ETH_Reset_N, 1);		// toggle reset gpio of ethernet switch
xPhyDiscover( &xPhyObject );
xPhyConfigure( &xPhyObject, &xPHYProperties );

or something like that where I added my SAMV71 define

        #if ( SAME70 != 0 || SAMV71 != 0 )
            if( ( xProtPacket->xICMPPacket.xIPHeader.ucProtocol != ipPROTOCOL_UDP ) &&
                ( xProtPacket->xICMPPacket.xIPHeader.ucProtocol != ipPROTOCOL_TCP ) )
        #endif
        {
            IPHeader_t * pxIPHeader = &( xProtPacket->xTCPPacket.xIPHeader );

            /* Calculate the IP header checksum. */
            pxIPHeader->usHeaderChecksum = 0x00;
            pxIPHeader->usHeaderChecksum = usGenerateChecksum( 0U, ( uint8_t * ) &( pxIPHeader->ucVersionHeaderLength ), ipSIZE_OF_IPv4_HEADER );
            pxIPHeader->usHeaderChecksum = ~FreeRTOS_htons( pxIPHeader->usHeaderChecksum );

            /* Calculate the TCP checksum for an outgoing packet. */
            usGenerateProtocolChecksum( pucBuffer, uxLength, pdTRUE );
        }

Thats why I am so confused as to why the function call is even coming at this point

4

Ok, so you are not disabling the scheduler, but the library is, and then in that disable doing something that may need to block. I am not familiar with that library, but it seems that either there is a bug there, where it needs to check if it needs to allocate the needed resource before suspending the scheduler, or there was some requirement that the application does something to do that before this point.

5

@TDB

Can you share where is FreeRTOS_accept called in your application code?
Whats the priority at which your application code runs, especially the task that calls FreeRTOS_accept?

The recommended priority levels of the tasks when using FreeRTOS+TCP are:

  • Highest : the Network Interface driver (niEMAC_HANDLER_TASK_PRIORITY)
  • Medium : the IP-task (ipconfigIP_TASK_PRIORITY)
  • Lower : the tasks that make use of FreeRTOS+TCP

I set

	#define configMAX_LIBRARY_INTERRUPT_PRIORITY	( 5 )
	#define configMAC_INTERRUPT_PRIORITY			( >configMAX_LIBRARY_INTERRUPT_PRIORITY )

while having

#define configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY 5

Are configMAX_LIBRARY_INTERRUPT_PRIORITY and configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY a typo?
I believe you should be using configMAX_SYSCALL_INTERRUPT_PRIORITY.

Also, please make sure that there aren’t any ISRs that are run with logical priority above configMAX_SYSCALL_INTERRUPT_PRIORITY if they are using any FreeRTOS APIs.

6

My TCP task calling is running

	/* Attempt to open the socket. */
	xListeningSocket = FreeRTOS_socket( FREERTOS_AF_INET, FREERTOS_SOCK_STREAM, FREERTOS_IPPROTO_TCP );
	configASSERT( xListeningSocket != FREERTOS_INVALID_SOCKET );

	#if( ipconfigUSE_TCP_WIN == 1 )
		if (ConfigureAndBindTCPSocket(xListeningSocket, (void *) &xWinProps, sizeof(xWinProps)) != 0)
		{
			/* TODO: implement error handling here according to returned error code */
			FreeRTOS_debug_printf(("Bootloader: ConfigureAndBindTCPSocket failed!\n"));
		}
	#else
		if (ConfigureAndBindTCPSocket(xListeningSocket))
		{
			/* TODO: implement error handling here according to returned error code */
			FreeRTOS_debug_printf(("Bootloader: ConfigureAndBindTCPSocket failed!\n"));
		}
	#endif
		else
		{
			FreeRTOS_listen( xListeningSocket, xBacklog );
		}

	for(;;)
	{
		// Wait for a client to connect
		gpio_set_pin_level(MCU_LED_Green, 0);
		xConnectedSocket = FreeRTOS_accept( xListeningSocket, &xClient, &xSize );
		configASSERT( xConnectedSocket != FREERTOS_INVALID_SOCKET );

and so on has the priority #define TCP_SERVER_TASK_PRIORITY ( tskIDLE_PRIORITY + 3 ). Which maps to priority = 3. I increased configMAX_PRIORITIES to 7 which would place niEMAC_HANDLER_TASK_PRIORITY and ipconfigIP_TASK_PRIORITY to 6 and 5. It did not solve the problem but I kept it anyway for best practice.

Regarding the typo question: That might be a problem since I migrated my code from v10.0.0 to v10.6.2? In any case they are linked through this define in FreeRTOSConfig.h

#define configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY 5

/* Interrupt priorities used by the kernel port layer itself.  These are generic
to all Cortex-M ports, and do not rely on any particular library functions. */
#define configKERNEL_INTERRUPT_PRIORITY (configLIBRARY_LOWEST_INTERRUPT_PRIORITY << (8 - configPRIO_BITS))
#define configMAX_SYSCALL_INTERRUPT_PRIORITY (configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY << (8 - configPRIO_BITS))

where configPRIO_BITS is 3.

Regarding the ISR: At the very least, I did not implement any myself (yet) but managed all my peripheral interfacing wrapped in a task.

7

When does this task start? Does the application make sure that the FreeRTOS+TCP APIs are called once TCP/IP stack is ready?

In the network interface file you are using, how is the configMAC_INTERRUPT_PRIORITY used? Is it passed as an argument to NVIC_SetPriority or directly written to any registers?
Also, I believe you have verified configPRIO_BITS is 3 for your platform (maybe via __NVIC_PRIO_BITS).

8

From NetworkInterface.c:

    /* Note that the gmac_option holds the MAC address that will be enabled for reception.
     * Leave the MAC as zeros. It will be handled properly further down.*/

    gs_gmac_dev.p_hw = GMAC;
    gmac_dev_init( GMAC, &gs_gmac_dev, &gmac_option );

    NVIC_SetPriority( GMAC_IRQn, configMAC_INTERRUPT_PRIORITY );
    NVIC_EnableIRQ( GMAC_IRQn );

Regarding the prio bits:


/*
 * \brief Configuration of the CORTEX-M7 Processor and Core Peripherals
 */

#define __CM7_REV                 0x0101 /**< CM7 Core Revision                                                         */
#define __NVIC_PRIO_BITS               3 /**< Number of Bits used for Priority Levels                                   */

samv71q21b.h defines it accordingly as 3 even through Cortex-M7 processors support 4 bits in theory.

Here is my (slightly shorted) code for my tcp task. I am pretty sure the stack is already initialized at that point regarding the calls of API functions:

static void prvTCPServerTask(void *Parameters_vp)
{
	TCPIP_Errorcodes_t errorcode = No_TCPIP_Error;
	Configure_TCP_Settings_Parameter_t tcp_settings;
	while(xQueueReceive(TCPIP_Config_Queue, &tcp_settings, QUEUE_WAIT_TIME_DEFAULT) != pdPASS)
	{
		/* wait here until tcp config including ip address is loaded and received */
	}

	if(initializeNetworkStack(tcp_settings) != pdPASS)
	{
		FreeRTOS_debug_printf(("Network Stack initialization from file failed!\n"));
		if (initializeNetworkStackFromDefines() != pdPASS)
		{
			FreeRTOS_debug_printf(("Network Stack initialization from defines failed too!\n"));
			gpio_set_pin_level(MCU_LED_Red, 0);
		}
	}

	Queue_Commands_t tcpip_queue_command;
	while(1)
	{
		if(xQueueReceive(TCPIP_Command_Queue, &tcpip_queue_command, QUEUE_WAIT_TIME_DEFAULT) == pdPASS && (TCP_Command_Queue_Commands_E)tcpip_queue_command.Command == NETWORK_UP_EVENT_CAME)
			break;
	}

	struct freertos_sockaddr xClient;
	Socket_t xListeningSocket, xConnectedSocket;
	socklen_t xSize = sizeof( xClient );
	const BaseType_t xBacklog = 20;

	TCP_Command_RX_Handle_t TCP_RX_Command_Handle;

	uint32_t noOfReceivedBytesInMeasurementMessage;
	uint32_t {...} // some variables

	static const TickType_t xReceiveTimeOut = pdMS_TO_TICKS( 5 );
	static const TickType_t xSendTimeOut = pdMS_TO_TICKS( 5000 );
	TickType_t xTimeOnShutdown;
	uint8_t *pucRxBuffer;
	uint8_t *pucTxBuffer;

	#if( ipconfigUSE_TCP_WIN == 1 )
		WinProperties_t xWinProps;
		ConfigureTCPWindow(xWinProps);
	#endif /* ipconfigUSE_TCP_WIN */

	FreeRTOS_debug_printf(("Entered create tcp task\n"));
	/* Just to prevent compiler warnings. */
	( void ) Parameters_vp;

	/* Attempt to open the socket. */
	xListeningSocket = FreeRTOS_socket( FREERTOS_AF_INET, FREERTOS_SOCK_STREAM, FREERTOS_IPPROTO_TCP );
	configASSERT( xListeningSocket != FREERTOS_INVALID_SOCKET );

	#if( ipconfigUSE_TCP_WIN == 1 )
		if (ConfigureAndBindTCPSocket(xListeningSocket, (void *) &xWinProps, sizeof(xWinProps)) != 0)
		{
			FreeRTOS_debug_printf(("Bootloader: ConfigureAndBindTCPSocket failed!\n"));
		}
	#else
		if (ConfigureAndBindTCPSocket(xListeningSocket))
		{			
			FreeRTOS_debug_printf(("Bootloader: ConfigureAndBindTCPSocket failed!\n"));
		}
	#endif
		else
		{
			FreeRTOS_listen( xListeningSocket, xBacklog );
		}

	for(;;)
	{
		// Wait for a client to connect
		gpio_set_pin_level(MCU_LED_Green, 0);
		xConnectedSocket = FreeRTOS_accept( xListeningSocket, &xClient, &xSize );
		configASSERT( xConnectedSocket != FREERTOS_INVALID_SOCKET );

		// Start of actual tcp handling
		pucRxBuffer = ( uint8_t * ) pvPortMalloc( TCP_RX_BUFFER_SIZE );	// Attempt to create the buffer used to receive data
		pucTxBuffer = ( uint8_t * ) pvPortMalloc( TCP_TX_BUFFER_SIZE );	// Attempt to create the buffer used to transmit data

		if( pucRxBuffer != NULL )
		{ ... // basicly a state machine handling incoming messages and queued uped tx messages

Basicly I start my tcp task and wait until another tasks sends a message in the queue that the configureable ip address is loaded from my peripheral storage.
After that I initialize the stack reading a MAC address from an eeprom:

BaseType_t initializeNetworkStack(const Configure_TCP_Settings_Parameter_t tcpconfig)
{
	static NetworkInterface_t xInterfaces[ 1 ];
	static NetworkEndPoint_t xEndPoints[ 1 ];

	uint8_t macaddr[6] = { 0 };
	uint8_t ipaddr[4] = { 0 };
	uint8_t netmask[4] = { 0 };
	uint8_t gatewayaddr[4] = { 0 };
	uint8_t dnsserveraddr[4] = { 0 };

	Read_MAC_Address(macaddr); // basicly an spi read
	if ( (macaddr[0] | macaddr[1] | macaddr[2] | macaddr[3] | macaddr[4] | macaddr[5] ) == 0 ||
		 (macaddr[0] & macaddr[1] & macaddr[2] & macaddr[3] & macaddr[4] & macaddr[5] ) == 0xFF )
	{
		FreeRTOS_debug_printf(("ERROR: Could not read mac address from eeprom\n"));
		return pdFALSE;	// could not read mac address from eeprom
	}

	memcpy(ipaddr, tcpconfig.IP_Address, sizeof(ipaddr));
	memcpy(netmask, tcpconfig.Net_Mask, sizeof(netmask));
	memcpy(gatewayaddr, tcpconfig.Gateway_Address, sizeof(gatewayaddr));
	memcpy(dnsserveraddr, tcpconfig.DNS_Server_Address, sizeof(dnsserveraddr));

	pxSAM_FillInterfaceDescriptor( 0, &(xInterfaces[0]) ); // from NetworkInterface.c
	FreeRTOS_FillEndPoint( &( xInterfaces[ 0 ] ), &( xEndPoints[ 0 ] ), ipaddr, netmask, gatewayaddr, dnsserveraddr, macaddr );
	#if ( ipconfigUSE_DHCP != 0 )
	{
		xEndPoints[ 0 ].bits.bWantDHCP = pdTRUE;
	}
	#endif /* ipconfigUSE_DHCP */
	return FreeRTOS_IPInit_Multi();
}
9

It depends on how you have implemented the sending of NETWORK_UP_EVENT_CAME command to the TCPIP_Command_Queue. Are you using vApplicationIPNetworkEventHook_Multi to send the NETWORK_UP_EVENT_CAME?
You can also take a look at this example implementation of vApplicationIPNetworkEventHook_Multi here, where tasks (that uses networking) are created once the network is up.

10

Since I am using the ipconfigIPv4_BACKWARD_COMPATIBLE== 1 variant I have implemented my hook like this:

void vApplicationIPNetworkEventHook( eIPCallbackEvent_t eNetworkEvent )
{
	static BaseType_t prvCreateTCPServerTaskAlreadyCreated = pdFALSE;

	/* Check this was a network up event, as opposed to a network down event. */
	if( eNetworkEvent == eNetworkUp && prvCreateTCPServerTaskAlreadyCreated == pdFALSE)
	{		
		prvCreateTCPServerTaskAlreadyCreated = pdTRUE;
		Queue_Commands_t tcpip_queue_command;
		tcpip_queue_command.Command = (uint32_t)NETWORK_UP_EVENT_CAME;
		xQueueSendToBack(TCPIP_Command_Queue, &tcpip_queue_command, portMAX_DELAY);	// signal tcp server task that network event came
	}
}

So basicly my TCP task is waiting in the while(1) loop until this event comes

11

Can you check in a debugger which task\ISR calls this pxGetNetworkBufferWithDescriptor when the scheduler is already suspended by the prvAcceptWaitClient?

12

That would be the prvEMACHandlerTask when calling the function prvEMACRxPoll. I placed a breakpoint at every single occurrence of pxGetNetworkBufferWithDescriptor( GMAC_RX_UNITSIZE, xBlockTime ) but this one, since it will come up constantly. No breakpoint was reached and I ran directly into the ASSERT call which checks if the scheduler is disabled.

static uint32_t prvEMACRxPoll( void )
{
    unsigned char * pucUseBuffer;
    uint32_t ulReceiveCount, ulResult, ulReturnValue = 0;
    static NetworkBufferDescriptor_t * pxNextNetworkBufferDescriptor = NULL;
    const UBaseType_t xMinDescriptorsToLeave = 2UL;
    const TickType_t xBlockTime = pdMS_TO_TICKS( 100UL );
    IPStackEvent_t xRxEvent = { eNetworkRxEvent, NULL };
    uint8_t * pucDMABuffer = NULL;

    for( ; ; )
    {
        BaseType_t xRelease = pdFALSE;

        /* If pxNextNetworkBufferDescriptor was not left pointing at a valid
         * descriptor then allocate one now. */
        if( ( pxNextNetworkBufferDescriptor == NULL ) && ( uxGetNumberOfFreeNetworkBuffers() > xMinDescriptorsToLeave ) )
        {
            pxNextNetworkBufferDescriptor = pxGetNetworkBufferWithDescriptor( GMAC_RX_UNITSIZE, xBlockTime );
        }

        if( pxNextNetworkBufferDescriptor != NULL )
        {
            /* Point pucUseBuffer to the buffer pointed to by the descriptor. */
            pucUseBuffer = ( unsigned char * ) ( pxNextNetworkBufferDescriptor->pucEthernetBuffer - ipconfigPACKET_FILLER_SIZE );
        }
        else
        {
            /* As long as pxNextNetworkBufferDescriptor is NULL, the incoming
             * messages will be flushed and ignored. */
            pucUseBuffer = NULL;
        }

        /* Read the next packet from the hardware into pucUseBuffer. */
        ulResult = gmac_dev_read( &gs_gmac_dev, pucUseBuffer, GMAC_RX_UNITSIZE, &ulReceiveCount, &pucDMABuffer );

        if( ( ulResult != GMAC_OK ) || ( ulReceiveCount == 0 ) )
        {
            /* No data from the hardware. */
            break;
        }

        if( pxNextNetworkBufferDescriptor == NULL )
        {
            /* Data was read from the hardware, but no descriptor was available
             * for it, so it will be dropped. */
            iptraceETHERNET_RX_EVENT_LOST();
            continue;
        }

        iptraceNETWORK_INTERFACE_RECEIVE();
        #if ( ipconfigZERO_COPY_RX_DRIVER != 0 )
        {
            pxNextNetworkBufferDescriptor = pxPacketBuffer_to_NetworkBuffer( pucDMABuffer );

            if( pxNextNetworkBufferDescriptor == NULL )
            {
                /* Strange: can not translate from a DMA buffer to a Network Buffer. */
                break;
            }
        }
        #endif /* ipconfigZERO_COPY_RX_DRIVER */

        pxNextNetworkBufferDescriptor->xDataLength = ( size_t ) ulReceiveCount;
        pxNextNetworkBufferDescriptor->pxInterface = pxMyInterface;
        pxNextNetworkBufferDescriptor->pxEndPoint = FreeRTOS_MatchingEndpoint( pxMyInterface, pxNextNetworkBufferDescriptor->pucEthernetBuffer );

        if( pxNextNetworkBufferDescriptor->pxEndPoint == NULL )
        {
            FreeRTOS_printf( ( "NetworkInterface: can not find a proper endpoint\n" ) );
            xRelease = pdTRUE;
        }
        else
        {
            xRxEvent.pvData = ( void * ) pxNextNetworkBufferDescriptor;

            if( xSendEventStructToIPTask( &xRxEvent, xBlockTime ) != pdTRUE )
            {
                /* xSendEventStructToIPTask() timed out. Release the descriptor. */
                FreeRTOS_printf( ( "prvEMACRxPoll: Can not queue a packet!\n" ) );
                xRelease = pdTRUE;
            }
        }

        /* Release the descriptor in case it can not be delivered. */
        if( xRelease == pdTRUE )
        {
            /* The buffer could not be sent to the stack so must be released
             * again. */
            vReleaseNetworkBufferAndDescriptor( pxNextNetworkBufferDescriptor );
            iptraceETHERNET_RX_EVENT_LOST();
        }

        /* Now the buffer has either been passed to the IP-task,
         * or it has been released in the code above. */
        pxNextNetworkBufferDescriptor = NULL;
        ulReturnValue++;
    }

    return ulReturnValue;
}

static void prvEMACHandlerTask( void * pvParameters )
{
    UBaseType_t uxCount;

    uxLowestSemCount = GMAC_TX_BUFFERS + 1;

    #if ( ipconfigZERO_COPY_TX_DRIVER != 0 )
        NetworkBufferDescriptor_t * pxBuffer;
    #endif
    uint8_t * pucBuffer;
    BaseType_t xResult = 0;
    const TickType_t ulMaxBlockTime = pdMS_TO_TICKS( EMAC_MAX_BLOCK_TIME_MS );
    uint32_t ulISREvents = 0U;

    /* Remove compiler warnings about unused parameters. */
    ( void ) pvParameters;

    configASSERT( xEMACTaskHandle );

    for( ; ; )
    {
        xResult = 0;
        vCheckBuffersAndQueue();

        /* Wait for a new event or a time-out. */
        xTaskNotifyWait( 0U,                /* ulBitsToClearOnEntry */
                         EMAC_IF_ALL_EVENT, /* ulBitsToClearOnExit */
                         &( ulISREvents ),  /* pulNotificationValue */
                         ulMaxBlockTime );

        if( ( ulISREvents & EMAC_IF_RX_EVENT ) != 0 )
        {
            /* Wait for the EMAC interrupt to indicate that another packet has been
             * received. */
            xResult = prvEMACRxPoll();
        }

        if( ( ulISREvents & EMAC_IF_TX_EVENT ) != 0 )
        {
            while( xQueueReceive( xTxBufferQueue, &pucBuffer, 0 ) != pdFALSE )
            {
                #if ( ipconfigZERO_COPY_TX_DRIVER != 0 )
                {
                    pxBuffer = pxPacketBuffer_to_NetworkBuffer( pucBuffer );

                    if( pxBuffer != NULL )
                    {
                        vReleaseNetworkBufferAndDescriptor( pxBuffer );
                        TX_STAT_INCREMENT( tx_release_ok );
                    }
                    else
                    {
                        TX_STAT_INCREMENT( tx_release_bad );
                    }
                }
                #else /* if ( ipconfigZERO_COPY_TX_DRIVER != 0 ) */
                {
                    TX_STAT_INCREMENT( tx_release_ok );
                }
                #endif /* if ( ipconfigZERO_COPY_TX_DRIVER != 0 ) */
                uxCount = uxQueueMessagesWaiting( ( QueueHandle_t ) xTXDescriptorSemaphore );

                if( uxCount < ( GMAC_TX_BUFFERS - 1 ) )
                {
                    /* Tell the counting semaphore that one more TX descriptor is available. */
                    xSemaphoreGive( xTXDescriptorSemaphore );
                }
            }
        }

        if( ( ulISREvents & EMAC_IF_ERR_EVENT ) != 0 )
        {
            /* Future extension: logging about errors that occurred. */
        }

        gmac_enable_management( GMAC, true );

        if( xPhyCheckLinkStatus( &xPhyObject, xResult ) != 0 )
        {
            /* Something has changed to a Link Status, need re-check. */
            prvEthernetUpdateConfig( pdFALSE );
        }

        gmac_enable_management( GMAC, false );
	}
}

BTW: Please let me know if I can help you helping me out by highlighting more of my quoted code through comments or cutting some to reduce bloat

13

Have you verified that the ISR that sends EMAC_IF_RX_EVENT notification to the prvEMACHandlerTask uses <>FromISR API?

14

I did indeed. That would be the xRxCallback:

void xRxCallback( uint32_t ulStatus )
{
    if( ( ( ulStatus & GMAC_RSR_REC ) != 0 ) && ( xEMACTaskHandle != NULL ) )
    {
        /* let the prvEMACHandlerTask know that there was an RX event. */
        xTaskNotifyFromISR( xEMACTaskHandle, EMAC_IF_RX_EVENT, eSetBits, &( xGMACSwitchRequired ) );
    }
}

which in turn should be called by the GMAC_Handler. I did not alter them from the official release build and also confirmed that I don’t override the GMAC_Handler with a different version.

15

This is strange; I wonder if there is some other issue causing this behavior. Did you try increasing the stack sizes of the EMAC tasks or any other application tasks to see if there is any change in behavior to rule out stack overflow?

16

I increased the stack from 4 * configMINIMAL_STACK_SIZE to 8 * configMINIMAL_STACK_SIZE but that didn’t change anything.
But it would have been a surprise since I implemented the vApplicationStackOverflowHook to be notified of such situations and I can’t really give this task much more stack without cutting short my actual production code tasks.

17

Unrelated to the actual problem, I would like to add that

if( pxClientSocket != NULL )
{
    FreeRTOS_printf( ( "prvAcceptWaitClient: client %p parent %p\n",
    ( void * ) pxClientSocket, ( void * ) pxParentSocket ) );
}

is generally a very slow operation and should probably not be used lightly in a semi-critical section like prvAcceptWaitClient creates, through disabling the scheduler. I mean everything else between vTaskSuspendAll(); and ( void ) xTaskResumeAll(); is just a handful of assembly lines.
Deleting the print is clearing up this race condition between a GMAC interrupt and the re-enabling of the scheduler but I am still a bit bewildered as how it can come to pass in the first place.

18

Yes, there isnt a need for that printf to be inside the scheduler suspended section, especially when the FreeRTOS_printf is for the user to implement and other FreeRTOS APIs may be called from its implementation.

Does removing that printf solves this your issue?

19

Yes it does seem to solve my issue but I do not know if it is a statistical solve or a systematic one.
It would be pretty bad if I just “get lucky” now but the interrupt could still run into the problematic task.
If the interrupt was only able to produce the faulty behaviour because the printf gave room for a context switch or something like that it is an actual bug of the TCP Stack code which would be solved through the removal.

I tried debugging that but apparently debugging in this situation screws the timings to much for me to reproduce it accordingly.

20

I also thought again about the ISR save call: I am not versed deeply enough in the freeRTOS framework to validate that but it seems to me an ISR save callback is provoking an non ISR save behavior since ISR save should mean it is irrelevant if the scheduler is running since an interrupt can come at any time.
In my mind the behavior of my gmac interrupt is not wrong, it is supposed to come randomly but the code should not have a problem just because the scheduler is disabled for some reason or another.