SMP Performance Benchmarking

Kernel
21

I am not sure I completely follow it. It seems that the word mutex has been used to describe multiple things, so lets disambiguate to ensure that we are talking about the same things-

  1. Mutex - A FreeRTOS object which is created using xSemaphoreCreateMutex API.
  2. Spin Lock - Used by the FreeRTOS implementation for cross core synchronization in short critical sections.

If you want to access a resource from more than one task, you would use a FreeRTOS mutex (which you create using xSemaphoreCreateMutex API) to ensure that the task that cannot get the mutex does not use CPU cycles. The granular locking proposal ensures that 2 tasks trying to grab 2 different FreeRTOS mutex do not compete for the same spin lock.

Are you saying that you will use spin lock directly in the application? If yes, why do you want to do that?

22

Using the same argument, we can take this change too. Would you be willing to raise a PR?

1 Like
23

Sure, I’ve opened a PR for review. I should have an execution time comparison ready next week.

3 Likes
25

Thank you for the PR @fvo! Please share the performance comparison data when you have it ready.

1 Like
26

problem with this one is that it works on single core only.

#define xSemaphoreTake( xSemaphore, xBlockTime )
→ xQueueSemaphoreTake( ( xSemaphore ), ( xBlockTime ) )
→ taskENTER_CRITICAL
→ portENTER_CRITICAL()
→ vPortEnterCritical()
→ void vPortEnterCritical( void )
{
portDISABLE_INTERRUPTS();
uxCriticalNesting[ portGET_CORE_ID() ] ++;
}

what we need is something that works on SMP

27

That isn’t the code path should exist if configNUMBER_OF_CORES is greater than 1.

portENTER_CRITICAL should call vTaskEnterCritical which will disable interrupts and get the needed spin locks.

The SMP version of the port should be different than the single core version.

28

As @richard-damon explained already, you are looking at the single core code path and a single core port. You need to set configNUMBER_OF_CORES to the number of cores available on your platform and use a SMP port.

29

hello,

I could not find any example where portENTER_CRITICAL calls vTaskEnterCritical besides the template and I looked at all the GCC/ARM ports and none have this.

Even so, if vTaskEnterCritical is called this further maps on only 2 spinlocks through portGET_TASK_LOCK/portGET_ISR_LOCK thus same problem of user and kernel sharing 2 spinlocks iven if it makes no sense.

Is there any public FreeRTOS ARM SMP port ?

Regards,
Geo

30

As far as I know, there aren’t many processors fully released with SMP ports. I suspect that will improve over time. Did any of the ports you looked at check for SMP mode? (I suspect not).

Yes, sort of by definition, the built in critical section is the one used by FreeRTOS for its kernel. Most applications sharing can be handled by a Mutex unless it needs protection for an ISR too. IF you don’t want to share the lock with the kernel, don’t us its lock, but make your own.

31

For some reason, most silicon vendors (NXP, AMD, etc.) choose to release their FreeRTOS ports (some of which support SMP) directly to their customers instead of publicly contributing them to the FreeRTOS repositories. Perhaps someone from the AWS team (@aggarg?) can comment further on why this is the case and what their preference is in this situation?

I suspect that the vendors don’t want to deal with upstreaming their changs and supporting customers publicly, but as far as I’m aware they also can’t prevent a 3rd party from contributing the vendor ports since this is permitted under the MIT license.

In the NXP CM7 and R55 ports, portENTER_CRITICAL() does indeed call vTaskEnterCritical().

32

Here is one - FreeRTOS-Kernel/portable/ThirdParty/GCC/RP2040/include/portmacro.h at main · FreeRTOS/FreeRTOS-Kernel · GitHub.

Yes, here it is - FreeRTOS-Kernel/portable/ThirdParty/GCC/RP2040 at main · FreeRTOS/FreeRTOS-Kernel · GitHub.

FreeRTOS is MIT licensed and we do not prevent anyone from distributing it. If you got a FreeRTOS port from your vendor, you can reach out to them for support.

I’d still suggest to check with the vendor before doing that.

33

I think a better idea is for FreeRTOS to offer this API such that each project doesn’t invent it’s own.

34

The problem is that this interface isn’t IN FreeRTOS itself, but in the port layer. A given port needs to supply the two locks, but not some generic method to create other locks, in part because it may not be possible on a given set of hardware to generate arbitrary versions of the locks.

The SMP version is fairly new, and largely supported with third-party ports. Perhaps as the technology matures, minimum expected capabilities will be revealed, and a generic API created. For now, you need to look at how the given port works, and use whatever API it provides.

35

Any SMP OS should provide an api to work with spinlocks, GetSpinLock(), ReleaseSpinLock(), TryGetSpinLock() … something like this.

From HW point of view there is no SMP on those cores if there is not hw support for spinlocks.
In the case of ARM we’re talking about Load/Store Exclusive, on other architectures some specific peripherals to implement the exclusive lock.

Thus, if the port doesn’t provide the spinlock hw feature at all then there is no SMP on that port.
If there is a limited number of spinlocks, ok, the API that “creates” the spinlocks with throw an error when the limit is reached.
On the ARM architecture if the LDRX/STRX are supported the number of spinlocks is limited by the amount of memory available.

36

And my point is that it becomes the ports jb of providing the API, not the kernal. Remember, we are in the first version of FreeRTOS that has had SMP capabilities on the trunk, and thus its SMP API is still evolving.

Yes, ideally, an generic API will develop to allow portable use of spinlocks, but until there is some experience on what the boundries for them would be, it is hard to design. Once and API is created, then it will just exclude processors that can’t support that particular API. Since one of the goals for FreeRTOS is wide processor support, it makes sense to wait a bit to lock down that API.

Perhaps the way to look at it is that FreeRTOS is not a “SMP OS” but a “Real Time OS” that supports some SMP features. It should be noted that adding SMP to an OS tends to increase uncertanty of timing, and thus impacts the real time nature of the OS, and spin-waiting, as spin-locks naturally do, goes against the basic nature of real-time design, you want to be able to bound the time of such spins.

2 Likes
37

Why do you want to use spin locks in your application? You earlier said that you want to use it to share resources among tasks running on different cores and as I replied before, you should use mutex for that. Would you please share why do you want to use spin locks?

38

Not sure what mutex you recommended before, is this one working on SMP ?
My understanding is that a spinlock can be used to implement a mutex that works on SMP, do we have this in FreeRTOS ?

39

If you use a working SMP port (such as the RP2040 port I linked before), the following application code would work:

int main( void )
{
    xMutex = xSemaphoreCreateMutex();
    xTaskCreate( TaskA, ... );
    xTaskCreate( TaskB, ... );
    vTaskStartScheduler();
}

void TaskA( void * pvParameters )
{
    for( ;; )
    {
        xSemaphoreTake( xMutex, portMAX_DELAY );
        {
            /* Access the resource shared among multiple tasks. */
        }
        xSemaphoreGive( xMutex );
    }
}

void TaskB( void * pvParameters )
{
    for( ;; )
    {
        xSemaphoreTake( xMutex, portMAX_DELAY );
        {
            /* Access the resource shared among multiple tasks. */
        }
        xSemaphoreGive( xMutex );
    }
}

In other words, the application DOES NOT need to use spin locks to protect a resource shared among multiple tasks. The application should use FreeRTOS mutex. The FreeRTOS mutex implementation internally using spin locks to ensure cross core synchronization whenever needed.

40

here is the call path for the pi-pico smp port.
Same problem, we end up again on the same 2 spinlocks.

Even if xSemaphore can take specific parameters from taskENTER_CRITICAL and below there is no parameter to differentiate, and they end on the same 2 spinlocks.

#define xSemaphoreTake( xSemaphore, xBlockTime )
    -> xQueueSemaphoreTake
        -> taskENTER_CRITICAL
            ->  portENTER_CRITICAL()
                ->  vTaskEnterCritical()
                    -> portGET_TASK_LOCK / portGET_ISR_LOCK
                        ->  vPortRecursiveLock( 1 / 0, spin_lock_instance( configSMP_SPINLOCK_1/0 ), pdTRUE/False )
                        
                        
    #define portGET_ISR_LOCK()         vPortRecursiveLock( 0, spin_lock_instance( configSMP_SPINLOCK_0 ), pdTRUE )
    #define portRELEASE_ISR_LOCK()     vPortRecursiveLock( 0, spin_lock_instance( configSMP_SPINLOCK_0 ), pdFALSE )
    #define portGET_TASK_LOCK()        vPortRecursiveLock( 1, spin_lock_instance( configSMP_SPINLOCK_1 ), pdTRUE )
    #define portRELEASE_TASK_LOCK()    vPortRecursiveLock( 1, spin_lock_instance( configSMP_SPINLOCK_1 ), pdFALSE )
41

Which problem are you talking about? Are you facing any functional issue i.e. mutex not behaving as expected? In other words, are tasks running on different cores able to enter the code section protected by mutex simultaneously? Or are you trying to point out the lock contention? If later, this PR is addressing it - Feature/smp granular locks v4 by sudeep-mohanty · Pull Request #1154 · FreeRTOS/FreeRTOS-Kernel · GitHub.