FreeRTOS SMP Port to VS

wuxuangle · May 11, 2024, 11:46am

Dear AGGARG:
Another point of doubt is: On two cores with prvProcessSimulatedInterrupts, should we send an interrupt to the other core as soon as one of them handles the interrupt, or should we recursively call to send a scheduling interrupt request to the other cores from within vPortGenerateSimulatedInterrupt after sending a scheduling interrupt to the current core and waiting for the current core’s interrupt to be completed?

wuxuangle · May 12, 2024, 10:38am

Regarding this issue, I have thought about it, and the solution is as follows:

#ifndef portYIELD_CORE

    #if ( configNUMBER_OF_CORES == 1 )
        #define portYIELD_CORE( x )    portYIELD()
    #else
        #define portYIELD_CORE( x )    portYIELD()
    #endif /* configNUMBER_OF_CORES */

#endif /* portYIELD_CORE */

#ifndef portYIELD_CORE

    #if ( configNUMBER_OF_CORES == 1 )
        #define portYIELD()  vPortGenerateSimulatedInterrupt( portINTERRUPT_YIELD )

    #else
        #define portYIELD() vPortGenerateSimulatedInterrupt(
portINTERRUPT_YIELD )
    #endif /* configNUMBER_OF_CORES */

#endif /* portYIELD_CORE */

The distinction of cores is placed in vPortGenerateSimulatedInterrupt, thereby unifying the scheduling for single-core and SMP (Symmetric Multi-Processing).

AGGARG, what do you think?

wuxuangle · May 12, 2024, 10:46am

In order to unify the scheduling for both single-core and SMP environments, I will opt for the latter approach. What are your thoughts on this, Aggarg

wuxuangle · May 13, 2024, 12:35am

Dear Aggarg：
The port file has been basically modified according to the aforementioned ideas, but there is currently one type of compilation error (a total of 4 errors). The reason for this error is that I want to access pxCurrentTCBs[xCoreId]->uxCoreAffinityMask within the port file, but I cannot access the TCB_t data structure in port.c. This issue completely prevents me from further debugging the possibility of SMP in the third step of my plan. Please help me think about how to resolve this issue without modifying the source files of FreeRTOS v11.1.0.
I am looking forward to your response.

wuxuangle · May 13, 2024, 1:27am

Simply redefine the TCB_t with a different name within port.c, and then perform a cast to that type。
Aggarg，What do you think, do you think it would work?

aggarg · May 13, 2024, 7:14am

Apologies for delay in my response.

What compiler errors are you getting?

Timer interrupt is generated on one core and it can interrupt another core when needed. The port need to implement portYIELD_CORE( x ) correctly.

wuxuangle:

#ifndef portYIELD_CORE

    #if ( configNUMBER_OF_CORES == 1 )
        #define portYIELD()  vPortGenerateSimulatedInterrupt( portINTERRUPT_YIELD )

    #else
        #define portYIELD() vPortGenerateSimulatedInterrupt(
portINTERRUPT_YIELD )
    #endif /* configNUMBER_OF_CORES */

What is the point of #if ( configNUMBER_OF_CORES == 1 ) if the definition is same in both the cases?

Why do you want that? Is that only for debugging or do you want this de-referencing for implementation?

Also note that my answers may not be 100% correct as I have not implemented SMP for WinSim before. I am answering your questions based on my knowledge of SMP implementation on hardware platforms. I’d also recommend you to take a look at the existing SMP ports -

wuxuangle · May 13, 2024, 7:24am

I currently have two remaining questions:

I am not sure if what I have implemented meets the expected requirements.Regarding the macro definitions listed below, I am not sure if my understanding is correct.Dear brothers of the great FreeRTOS community, can tell me what the following macros are used for? or Or where can I find the rule definitions for their meanings？
portGET_CORE_ID()
portYIELD_CORE( x )
portSET_INTERRUPT_MASK()
portCLEAR_INTERRUPT_MASK( x )
portRELEASE_TASK_LOCK()
portRELEASE_TASK_LOCK()
portGET_TASK_LOCK()
portRELEASE_ISR_LOCK()
portGET_ISR_LOCK()
portENTER_CRITICAL_FROM_ISR()
portEXIT_CRITICAL_FROM_ISR( x )

2.How can I test SMP to know that it is running under SMP rules?

wuxuangle · May 13, 2024, 7:59am

[/quote]

If the modification is made in the following manner：
#if ( configNUMBER_OF_CORES == 1 )
#define portYIELD() vPortGenerateSimulatedInterrupt( portINTERRUPT_YIELD )
#else
#define portYIELD(x) vPortGenerateSimulatedInterrupt(
x,portINTERRUPT_YIELD )
#endif
Where ‘x’ represents the core ID.
then the following compilation errors will occur:
In the task: portTASK_FUNCTION, the corresponding portYield() for taskYIELD() does not exist.The occurrence is located at the following position:

#if ( configNUMBER_OF_CORES > 1 )
static portTASK_FUNCTION( prvPassiveIdleTask, pvParameters )
{
( void ) pvParameters;

    taskYIELD();

    for( ; configCONTROL_INFINITE_LOOP(); )
    {
        #if ( configUSE_PREEMPTION == 0 )
        {

To resolve the aforementioned issues：
#if ( configNUMBER_OF_CORES == 1 )
#define portYIELD() vPortGenerateSimulatedInterrupt( portINTERRUPT_YIELD )
#else
#define portYIELD() vPortGenerateSimulatedInterrupt(portINTERRUPT_YIELD )
#endif
The distinction of cores is placed in vPortGenerateSimulatedInterrupt, thereby unifying the scheduling for single-core and SMP (Symmetric Multi-Processing).

There are the following three reasons:

In the port file, pxPortInitialiseStack creates a task, and the core to which the task is bound is fixed. At this time, the task’s uxCoreAffinityMask information cannot be passed because it is not possible to bind the core corresponding to the task during initialization.
The intention is to implement portGET_CORE_ID() by using pxCurrentTCBs[xCoreId]->uxCoreAffinityMask.
During task switching, re-bind the task’s corresponding core ID by using pxCurrentTCBs[xCoreId]->uxCoreAffinityMask.

Therefore, in the port, it is possible to access the TCB_t, and then use pxCurrentTCBs[xCoreId] to query the uxCoreAffinityMask property.

[wuxu]: Based on your reminder, I realized that my above view might not be entirely correct. I should use GetThreadAffinityMask to obtain the AffinityMask to implement xCoreId = portGET_CORE_ID() . Then, before resuming the thread found by scheduling, Thread = pxCurrentTCBs[xCoreId] , I should set the thread affinity again using SetThreadAffinityMask(Thread, AffinityMask) . This way, there would be no need to access the TCB_t data structure within the port file. Note: The above notation is a form of pseudocode and does not represent the actual implementation.

Aggarg, you are a nice friend, and your experience with SMP is very useful both in terms of hardware and in winSim simulations.I have reviewed the existing SMP port, and it seems to me that the differences are quite significant. If the direction is correct (and the existing SMP approach represents the direction), I think the specific implementation form might not need too much attention.

wuxuangle · May 15, 2024, 12:35am

Current progress:

I can now start testing, but encountered an assert. The details of the assert will be organized later. Currently, I have two questions:

When configuring configNUMBER_OF_CORES to be greater than 1, it is necessary to set portCRITICAL_NESTING_IN_TCB to 1. If the default configuration is used, the following result will not be found:

#define portGET_CRITICAL_NESTING_COUNT()          ( pxCurrentTCBs[ portGET_CORE_ID() ]->uxCriticalNesting )
#define portSET_CRITICAL_NESTING_COUNT( x )       ( pxCurrentTCBs[ portGET_CORE_ID() ]->uxCriticalNesting = ( x ) )
#define portINCREMENT_CRITICAL_NESTING_COUNT()    ( pxCurrentTCBs[ portGET_CORE_ID() ]->uxCriticalNesting++ )
#define portDECREMENT_CRITICAL_NESTING_COUNT()    ( pxCurrentTCBs[ portGET_CORE_ID() ]->uxCriticalNesting-- )

After setting configUSE_CORE_AFFINITY to 1, I only use xTaskCreate to create tasks, but do not bind the created tasks to specific cores. Is there a problem with this?

aggarg · May 15, 2024, 6:36am

portGET_CORE_ID() - Get the core ID of the current Core.
portYIELD_CORE( x ) - Interrupt core with core ID x so that it runs scheduler and picks a task.
portSET_INTERRUPT_MASK() - As the same suggests, mask interrupts.
portCLEAR_INTERRUPT_MASK( x ) - Unmask interrupts.

The following macros are for lcks used to enable cross-core synchronization. One way to implement them is hardware spin locks -
portRELEASE_TASK_LOCK()
portRELEASE_TASK_LOCK()
portGET_TASK_LOCK()
portRELEASE_ISR_LOCK()
portGET_ISR_LOCK()

Critical section enter/exit from ISR -
portENTER_CRITICAL_FROM_ISR()
portEXIT_CRITICAL_FROM_ISR( x )

aggarg · May 15, 2024, 6:40am

I think you can replace the above with just this:

#define portYIELD() vPortGenerateSimulatedInterrupt(portINTERRUPT_YIELD )

That is not correct. portGET_CORE_ID is supposed to represent the core ID of the currently executing core. Besides, a task is not always tied to a core.

Not sure I understand completely, but you cannot change uxCoreAffinityMask of a task.

aggarg · May 15, 2024, 6:41am

No, it is not necessary.

No, there is no problem with this.

wuxuangle · May 15, 2024, 10:01am

Yes, that’s a good idea. I will make the modification according to your suggestion.

Based on your reminder, I realized that my above view might not be entirely correct. I should use GetThreadAffinityMask to obtain the AffinityMask to implement xCoreId = portGET_CORE_ID() . Then, before resuming the thread found by scheduling, Thread = pxCurrentTCBs[xCoreId] , I should set the thread affinity again using SetThreadAffinityMask(Thread, AffinityMask) . This way, there would be no need to access the TCB_t data structure within the port file. Note: The above notation is a form of pseudocode and does not represent the actual implementation.

In fact, I don’t really want to bind the task to the core dynamically myself. In the port file, pxPortInitialiseStack is responsible for creating the initial task. At this point, the uxCoreAffinityMask information cannot be passed to pxPortInitialiseStack , so the created task can only be fixedly bound to core 0. Therefore, it is not possible to bind the task to the corresponding core during initialization. The task is allocated to a specific core based on the scheduler, and then the task is dynamically bound to the corresponding core according to the scheduling assignment before resume this thread。

However, based on the actual linking results, it is necessary, otherwise the compiler cannot find the symbols defined by the following macros during the linking process.

#define portGET_CRITICAL_NESTING_COUNT() ( pxCurrentTCBs[ portGET_CORE_ID() ]->uxCriticalNesting )
#define portSET_CRITICAL_NESTING_COUNT( x ) ( pxCurrentTCBs[ portGET_CORE_ID() ]->uxCriticalNesting = ( x ) )
#define portINCREMENT_CRITICAL_NESTING_COUNT() ( pxCurrentTCBs[ portGET_CORE_ID() ]->uxCriticalNesting++ )
#define portDECREMENT_CRITICAL_NESTING_COUNT() ( pxCurrentTCBs[ portGET_CORE_ID() ]->uxCriticalNesting-- )

Alright, I think it’s unlikely that there will be any problems.

aggarg · May 15, 2024, 2:26pm

I am not sure I completely understand your solution. Let me try to explain the core ID and affinity mask - lets say you have 2 cores, with core ID 0 and 1. A task can be created with -

No core affinity mask - in this case the task can be scheduled on any of the core.
Core affinity mask 0x1 - in this case, the task can only be scheduled on core 0.
Core affinity mask 0x2 - in this case, the task can only be scheduled on core 1.

Hope that clarifies.

That is because if you do not define portCRITICAL_NESTING_IN_TCB, the port layer need to maintain critical nesting in the task context and need to implement these macros.

wuxuangle · May 16, 2024, 12:37am

I understand and agree with your point，but in fact，in the process of xCreateThread，it is impossible to bind the created task to the core. The following is the call processing of the function and so on:
xTaskCreate {
pxNewTCB = prvCreateTask(…){
prvInitialiseNewTask(…){
pxNewTCB->pxTopOfStack = pxPortInitialseStack(…){
//real create task and bind task to core, but cannot get uxCoreAffinityMask
}
}
if(pxNewTCB !=NULL)
{
pxNewTCB->uxCoreAffinityMask = uxCoreAffinityMask;
}
}
}
pxPortInitialseStack is the place where the task is actually bound to the core,but the uxCoreAffinityMask cannot be obtained at pxPortInitialseStack,which leads to the failure of core binding.

xTaskCreate {
prvAddNewTaskToReadyList(…){
taskYIELD_ANY_CORE_IF_USING_PREEMPTION(pxNewTCB) = prvYieldForTask（pxNewTCB）
{
configASSERT( portGET_CRITICAL_NESTING_COUNT() > 0U){
At the point，the value of uxCriticalNexting is 0；
}
}
}
}
what i don’t understand now is why this assert appears when the thread is initially create，or
is there something configured incorrectly？

The prerequisite for that approach is the following approach becoming successful. the following is：

#ifndef portYIELD_CORE

    #if ( configNUMBER_OF_CORES == 1 )
        #define portYIELD_CORE( x )    portYIELD()
    #else
        #define portYIELD_CORE( x )    portYIELD()
    #endif /* configNUMBER_OF_CORES */

#endif /* portYIELD_CORE */

#ifndef portYIELD_CORE

    #if ( configNUMBER_OF_CORES == 1 )
        #define portYIELD()  vPortGenerateSimulatedInterrupt( portINTERRUPT_YIELD )

    #else
        #define portYIELD() vPortGenerateSimulatedInterrupt(
portINTERRUPT_YIELD )
    #endif /* configNUMBER_OF_CORES */

#endif /* portYIELD_CORE */

but in

 xTaskIncrementTick()
{
       for( xCoreID = 0; xCoreID < ( BaseType_t ) 
       configNUMBER_OF_CORES;xCoreID++)
       {
             #if ( configUSE_TASK_PREEMPTION_DISABLE == 1 )
             if( pxCurrentTCBs[ xCoreID ]->xPreemptionDisable == pdFALSE )
             #endif
             {
                        if( ( xYieldRequiredForCore[ xCoreID ] != pdFALSE ) || ( xYieldPendings[ xCoreID ] != pdFALSE ) )
                        {
                            if( xCoreID == xCurrentCoreID )
                            {
                                xSwitchRequired = pdTRUE;
                            }
                            else
                            {
                                prvYieldCore( xCoreID );
                            }
                        }
                        else
                        {
                            mtCOVERAGE_TEST_MARKER();
                        }
                    }
                }
}

from the above kernel code， it can be seen #define portYIELD_CORE( x ) portYIELD(X)
when #if ( configNUMBER_OF_CORES > 1 ) . therefore, the above definition is not successful. so at this point ,we need to explain why taskYIELD()(#define taskYIELD() portYIELD()) is called in portTASK_FUNCTION under the condition of #if ( configNUMBER_OF_CORES > 1 ). However, under condition of #if ( configNUMBER_OF_CORES > 1 ), the task scheduling used in the task.c file is all

prvYieldCore(xCoreId){
      if(xx)
      {
             ...
      }
      else
      {
            portYIELD_CORE(xCoreID);
            .....
     }
}

Here is the code snippet that needs to be disscussed

#if ( configNUMBER_OF_CORES > 1 )
static portTASK_FUNCTION( prvPassiveIdleTask, pvParameters )
{
( void ) pvParameters;

    taskYIELD();

    for( ; configCONTROL_INFINITE_LOOP(); )
    {
        #if ( configUSE_PREEMPTION == 0 )
        {

#endif

richard-damon · May 16, 2024, 3:44am

Why can’t you just use xTaskCreateAffinitySet or xTaskCoreAffinitySet as described on Symmetric Multiprocessing (SMP) with FreeRTOS - FreeRTOS ?

aggarg · May 16, 2024, 5:07am

When you create a task using xTaskCreate it is created with no core affinity (i.e. it can be scheduled on any core). You can then set the affinity mask using vTaskCoreAffinitySet API.

I do not think that understanding is correct and I would recommend to read the existing SMP ports that I have linked before. Here is an example.

aggarg · May 16, 2024, 5:18am

wuxuangle:

#if ( configNUMBER_OF_CORES == 1 )
        #define portYIELD_CORE( x )    portYIELD()
    #else
        #define portYIELD_CORE( x )    portYIELD()
    #endif /* configNUMBER_OF_CORES */

Lets say that you have a code which does the same thing in both if and else:

if( x > 0 )
    printf("%d", x);
else
    printf("%d", x);

The if condition in the above code is useless because the same code is executed in both if and else clause. The above code can be simplified to:

printf("%d", x);

In the same manner, if your definition of portYIELD_CORE( x ) is same in both #if and #else, there is no need of #if and #else.

This is not clear. If you want to ask how to implement portYIELD_CORE, you can see here - FreeRTOS-Kernel/portable/ThirdParty/GCC/RP2040/include/portmacro.h at main · FreeRTOS/FreeRTOS-Kernel · GitHub.

Please enclose your code snippets in three backticks, otherwise your posts are almost unreadable.

wuxuangle · May 16, 2024, 6:14am

Why can’t you just use xTaskCreateAffinitySet or xTaskCoreAffinitySet as described on Symmetric Multiprocessing (SMP) with FreeRTOS - FreeRTOS ?

I understand what you’re saying, but you’re looking at this issue from a hardware perspective, such as from the viewpoint of an ARM-SMP hard core. If it were purely a hardware issue, core binding might simply involve a line like pxNewTCB->uxCoreAffinityMask = uxCoreAffinityMask; to accomplish the task. However, we are now also using the Windows API (SetThreadAffinityMask) to simulate SMP (Symmetric Multi-Processing), which requires us to further bind to the x86 core using uxCoreAffinityMask. Otherwise, any tasks that are created would, in fact, not be bound to a specific core. Do you understand what I mean now?

I understand what you mean. You are discussing the syntax of the C language itself, whereas I am referring to the condition #if (configNUMBER_OF_CORES > 1). My question is, why do you sometimes use taskYIELD() (defined as #define taskYIELD() portYIELD()) for scheduling, and at other times use prvYieldCore(xCoreId)/portYIELD_CORE(xCoreID);? In other words, why do you sometimes pass xCoreId to the scheduling interrupt and sometimes not? This inconsistency makes it difficult for me to create a unified macro substitution within the port. Or, to phrase it another way, what is the difference between using portYIELD() and portYIELD_CORE(xCoreID) in FreeRTOS-SMP when #if (configNUMBER_OF_CORES > 1) ?

Alright, I will follow your suggestion and enclose the code snippets with three backticks.

wuxuangle · May 16, 2024, 7:26am

Here is the call stack at the point where my system crashed:

xTaskCreate →
prvAddNewTaskToReadyList->
taskYIELD_ANY_CORE_IF_USING_PREEMPTION(pxNewTCB)->
configASSERT( portGET_CRITICAL_NESTING_COUNT() > 0U)

where portGET_CRITICAL_NESTING_COUNT() is 0;

To resolve the issue, vTaskEnterCritical must be called to ensure that portGET_CRITICAL_NESTING_COUNT is greater than 0. However, why is it necessary to call vTaskEnterCritical during the initial task creation?

Please help me review it to see if there is a problem with the configuration, as to why an assertion failure occurred at the initial state of task creation in the window.

Topic		Replies	Views
Richard:Bug in the LPC2000 port of FreeRTOS ? Kernel	17	237	August 27, 2012
Windows port and prvProcessSimulatedInterrupts() Kernel	11	513	April 19, 2018
FreeRTOS porting to ARM926ejs Kernel	29	666	October 10, 2013
Can I implement SMP with Cortex-R5F dual core in zynqultraScale+ mpsoc? Xilinx	26	5282	April 12, 2023
Multicore advice and suggestions Kernel	11	545	April 11, 2019

FreeRTOS SMP Port to VS

Related topics