RP2040, FreeRTOS SMP and Timer ISR

sdbbs · June 21, 2022, 11:16am

Hi all,

Apologies in advance for the verbosity, but I have been stumbling into this problem for a while, which I hope I can get some assistance with - and I think I have finally made a minimal reproducible example at https://gist.github.com/sdbbs/b1410cd45106e0c0ee599f7fcdbb8f90/48e79e193537a9346682b014ffd50d34c940cc1b.

The gist contains the project files CMakeLists.txt, FreeRTOSConfig.h and main.c (the only other files in my project are copy-pasted pico_sdk_import.cmake and FreeRTOS_Kernel_import.cmake).

The code (built against the smp branch of FreeRTOS) is very simple: I simply try to start a single FreeRTOS task, led_task, which I want to run from CPU core 1 - which, when it starts, enables a hardware timer interrupt service routine, and from that point on, simply toggles pins with a semiperiod of 500 ms. There is otherwise no other FreeRTOS interaction (queues and such); and the only thing the timer ISR does is generate some random data, and toggles a pin at the ISR entry and exit.

Therefore, I do not expect anything else from this code, but to see the led_task pin toggled with a semiperiod of 500 ms - and the timer ISR pin toggled with a period close to 250 µs - together/in parallel, continuously.

However, I have had some trouble getting this behavior from the code; in fact, I get the following behaviors (note that the first, behavior A, is the state of the main.c as is in the gist; the others would require changes in the code - commenting or uncommenting accordingly, - and recompilation; click on thumbnails for full image might be ignored so try right-click/Open Image in New Tab, top trace is led pin, bottom trace isr pin):


(A)	//#define USE_HARDWARE_ALARM_API	vTaskCoreAffinitySet (xLedTaskHandle, 0b10);	led_task inits and stops, timer ISR runs for some ms, then stops
(B)	#define USE_HARDWARE_ALARM_API	vTaskCoreAffinitySet (xLedTaskHandle, 0b10);	led_task inits and stops, timer ISR runs continuously
(C)	#define USE_HARDWARE_ALARM_API	//vTaskCoreAffinitySet (xLedTaskHandle, 0b10);	led_task runs continuously, timer ISR never starts
(D)	//#define USE_HARDWARE_ALARM_API	//vTaskCoreAffinitySet (xLedTaskHandle, 0b10);	both led_task runs continuously, and timer ISR runs continuously

So, essentially, I can get the code to work as I expect it to - but in that case, I cannot specify the led_task to run on core 1.

So in general, my main question is: is it possible to both specify the led_task to run on core 1, and have the code (both led_task and timer ISR) run continuously? And if so, how?

However, there were some other subquestions that arose from this ordeal, so I hope I’ll be able to get some assistance/hints with them as well; and for that purpose, I’ll include some more details below.

For most of my development, I had stumbled into behavior A, and it puzzles me to no end. As the screenshot image shows:

… the led_task manages to start the timer ISR, but then stops toggling; I tried stepping in gdb in this case, and basically, after you step over the first vTaskDelay(500); in led_task, execution goes back to the program, and gdb never breaks in led_task again.

But then, the timer ISR runs for some 70-90 ms (82 ms on the image), then there is a “gap” where there are no ISRs (on the image 1.48 ms, but I have seen 2.x, 3.x, 6.x, 8x ms for that “gap”), then there are a couple of more runs of the timer ISR (2 on the screenshot, but I’ve seen anything from 1 to 4), and then the timer ISR… stops?!
And I’ve noticed, that when this happens, then FreeRTOS usually keeps on running (if I break in gdb at a random time afterwards, I can see scheduling functions change in the backtrace) …

I simply cannot wrap my head around why would the timer ISR simply stop: since after each timer ISR, corresponding pin toggle goes back to zero, to me that means that restart_timer_alarm() command has ran, and the next “run” of the timer ISR should therefore be “scheduled” - what on earth could prevent that?

The “gap” however, does imply that something could have disabled all interrupts, and then enabled them again - could this be some FreeRTOS initialization? But then, why does this “gap” consistently happen 70-90 ms after the ISR had started running (so, quite a bit later from where I’d expect FreeRTOS initialization to happen)?

…

Note that I encountered behavior A, by simply setting the led_task affinity to core 1, and then setting up the timer ISR as in the RP2040 pico-examples code in timer_lowlevel.c.
I thought - well, it’s an official example, why shouldn’t it work?

Then I tried “fixing” this timer ISR “stop” by:

Changed exit from timer ISR from portYIELD_FROM_ISR(xHigherPriorityTaskWoken) to portEND_SWITCHING_ISR(xHigherPriorityTaskWoken) - that did not help
Having realized that RP2040 has only one hardware timer with four “alarm” ISRs (beyond the SysTick timer, which FreeRTOS already uses for its own systick), I tried changing the original Alarm 0 to 1, and then 2 (to avoid possible conflicts with FreeRTOS using these) - that did not help
While I still had more tasks in my code, having realized that FreeRTOS might create “Tmr Svc”, “IDLE0” and “IDLE1” tasks in the background (which all run on both cores), and having realized “Tmr Svc” likely has task priority of configMAX_PRIORITIES-1, I tried to reduce all of my task priorities to less than that - that did not help (and ultimately I removed all tasks I had, but the led_task)
I realized interrupts have different priorities from the FreeRTOS task priorities; and the RP2040/Cortex M0+ has only four: 0b11000000 = 192 = 0xc0; 0b10000000 = 128 = 0x80; 0b01000000 = 64 = 0x40; 0b00000000 = 0 = 0x00; and noting that configMAX_SYSCALL_INTERRUPT_PRIORITY has been commented in all FreeRTOSConfig.h I had seen online so far, I tried to set it (so I could give the timer ISR higher interrupt priority, by giving it a lower priority number) - that did not help either, possibly because:

$ grep -r configMAX_SYSCALL_INTERRUPT_PRIORITY FreeRTOS-Kernel-SMP/portable/GCC/ARM_CM0 FreeRTOS-Kernel-SMP/portable/ThirdParty/GCC/{RP2040,rpi_pico}
$

… configMAX_SYSCALL_INTERRUPT_PRIORITY is not found anywhere in (what I think are) the port specific files of RP2040/Cortex M0+.

Finally, I somehow managed to end up revisiting IntQueueTimer.c from the FreeRTOS-SMP-Demos for RP2040, and realized it also sets up a hardware timer alarm - but using a different “API” if you will:

timer_lowlevel.c uses irq_set_exclusive_handler(ALARM_IRQ, timer_alarm_isr), and to restart timer_hw->alarm[ALARM_NUM] = (uint32_t) timer_hw->timerawl + delay_us
IntQueueTimer.c uses hardware_alarm_claim(ALARM_NUM); hardware_alarm_set_callback(ALARM_NUM, timer_alarm_isr), and to restart hardware_alarm_set_target(ALARM_NUM, make_timeout_time_us( delay_us ) );

So, changing to this hardware_alarm_* API, which is handled by the #define USE_HARDWARE_ALARM_API in the code, finally made the timer ISR run continuosly; however the led_task did not run - this is behavior B.

And after some experimentation and commenting random stuff, I finally realized that by not using the hardware_alarm_* API, and not setting the CPU core affinity of led_task to core 1 - I can get both the led_task and the timer ISR to run continuously in parallel (behavior D), as intended/expected … except, I don’t understand why?

So, to sumarize; my main question is:

Is it possible to both specify the led_task to run on core 1, and have the code (both led_task and timer ISR) run continuously? And if so, how?

… and my subquestions:

For RP2040, should I use portYIELD_FROM_ISR(xHigherPriorityTaskWoken), or portEND_SWITCHING_ISR(xHigherPriorityTaskWoken), if I want to exit an ISR which otherwise communicates with FreeRTOS?
- For RP2040, must I use these macros as the very last command in the ISR function, or is there leeway (e.g. can I toggle the pin down after having called these macros)?
For RP2040/Cortex M0+, does configMAX_SYSCALL_INTERRUPT_PRIORITY have any effect, or not?
What on earth could cause the timer ISR to just stop in behavior A, after some tens of milliseconds? By that I mean, could someone provide a plausible sequence of events as an example (even if it is not the actual sequence of events that causes the problem here), that would cause an already queued hardware timer alarm to not fire?
Why do I have seeminly two methods ("API"s) to start and run a hardware timer alarm on the RP2040, with different behavior in respect to multicore/SMP operation? How do I know which is the “right” one to use?
- Why does the hardware_alarm_* API make the timer ISR run, when led_task has affinity to core 1 but otherwise does not run - and yet, if I want both led_task and timer ISR to run continuously as intended, I must use neither the hardware_alarm_* API, nor affinity setting of led_task to core 1?

Thanks in advance for any answers!

jasonpcarroll · June 22, 2022, 5:36pm

Hi @sdbbs,
Thank you for reaching out. I will notify the team and have them take a look.
Best,
Jason Carroll

sdbbs · June 22, 2022, 11:17pm

Many thanks for the response, @jasonpcarroll !

Just a few notes from a further inspection of behavior B:

Note that, regardless of how I setup the timer ISR, in either case I “start” it via the pico-sdk command irq_set_enabled, whose description reads “Enable or disable a specific interrupt on the executing core.”. And, since this command runs from led_task, which via vTaskCoreAffinitySet is set to run from cpu core 1 - that means, the resulting timer_alarm_isr will also run from cpu core 1.
I tried to step through prvSelectHighestPriorityTask for core 1, after led_task has run once, and started the timer ISR; I get this:

(gdb) b prvSelectHighestPriorityTask
Breakpoint 4 at 0x10001714: file C:/path/to/FreeRTOS-Kernel-SMP/tasks.c, line 821.
(gdb) c
Continuing.
[Switching to Thread 1]

Thread 1 hit Breakpoint 4, prvSelectHighestPriorityTask (xCoreID=xCoreID@entry=0) at C:/path/to/FreeRTOS-Kernel-SMP/tasks.c:821
821         {
(gdb) c
Continuing.
[Switching to Thread 2]

Thread 2 hit Breakpoint 4, prvSelectHighestPriorityTask (xCoreID=xCoreID@entry=1) at C:/path/to/FreeRTOS-Kernel-SMP/tasks.c:821
821         {
(gdb) n
822             UBaseType_t uxCurrentPriority = uxTopReadyPriority;
(gdb)
833             while( xTaskScheduled == pdFALSE )
(gdb) p uxCurrentPriority
$20 = 0
(gdb) p xTaskScheduled
$21 = 0
(gdb) n
847                 if( listLIST_IS_EMPTY( &( pxReadyTasksLists[ uxCurrentPriority ] ) ) == pdFALSE )
(gdb)
849                     List_t * const pxReadyList = &( pxReadyTasksLists[ uxCurrentPriority ] );
(gdb)
850                     ListItem_t * pxLastTaskItem = pxReadyList->pxIndex->pxPrevious;
(gdb)
853                     if( ( void * ) pxLastTaskItem == ( void * ) &( pxReadyList->xListEnd ) )
(gdb)
860                     xDecrementTopPriority = pdFALSE;
(gdb)
866                         pxTaskItem = pxTaskItem->pxNext;
(gdb)
868                         if( ( void * ) pxTaskItem == ( void * ) &( pxReadyList->xListEnd ) )
(gdb)
870                             pxTaskItem = pxTaskItem->pxNext;
(gdb)
873                         pxTCB = pxTaskItem->pvOwner;
(gdb)
892                         if( pxTCB->xTaskRunState == taskTASK_NOT_RUNNING )
(gdb)
910                         else if( pxTCB == pxCurrentTCBs[ xCoreID ] )
(gdb)
912                             configASSERT( ( pxTCB->xTaskRunState == xCoreID ) || ( pxTCB->xTaskRunState == taskTASK_YIELDING ) );
(gdb)
915                                     if( ( pxTCB->uxCoreAffinityMask & ( 1 << xCoreID ) ) != 0 )
(gdb)
920                                 pxTCB->xTaskRunState = ( TaskRunning_t ) xCoreID;
(gdb)
925                         if( xTaskScheduled != pdFALSE )
(gdb)
929                             uxListRemove( pxTaskItem );
(gdb)
930                             vListInsertEnd( pxReadyList, pxTaskItem );
(gdb)
931                             break;
(gdb)
951                 if( ( xSchedulerRunning == pdFALSE ) && ( uxCurrentPriority == tskIDLE_PRIORITY ) && ( xTaskScheduled == pdFALSE ) )
(gdb)
956                 configASSERT( ( uxCurrentPriority > tskIDLE_PRIORITY ) || ( xTaskScheduled == pdTRUE ) );
(gdb) p uxCurrentPriority
$22 = 0
(gdb) n
957                 uxCurrentPriority--;
(gdb) p uxCurrentPriority
$23 = 0
(gdb) n
833             while( xTaskScheduled == pdFALSE )
(gdb) p uxCurrentPriority
$24 = 4294967295
(gdb) n
960             configASSERT( taskTASK_IS_RUNNING( pxCurrentTCBs[ xCoreID ]->xTaskRunState ) );
(gdb) p pxCurrentTCBs[ xCoreID ]
$25 = (TCB_t * volatile) 0x20003900 <ucHeap+7864>
(gdb) p xCoreID
$26 = 1
(gdb) p *pxCurrentTCBs[ xCoreID ]
$27 = {pxTopOfStack = 0x200038b0 <ucHeap+7784>, uxCoreAffinityMask = 4294967295, xStateListItem = {xItemValue = 0,
    pxNext = 0x200015a4 <pxReadyTasksLists+8>, pxPrevious = 0x20003480 <ucHeap+6712>,
    pvOwner = 0x20003900 <ucHeap+7864>, pxContainer = 0x2000159c <pxReadyTasksLists>}, xEventListItem = {
    xItemValue = 32, pxNext = 0x0, pxPrevious = 0x0, pvOwner = 0x20003900 <ucHeap+7864>, pxContainer = 0x0},
  uxPriority = 0, pxStack = 0x20003500 <ucHeap+6840>, xTaskRunState = 1, xIsIdle = 1,
  pcTaskName = "IDLE1\000\000\000\000\000\000\000\000\000\000", uxCriticalNesting = 0, uxTCBNumber = 4,
  uxTaskNumber = 0, uxBasePriority = 0, uxMutexesHeld = 0, pvThreadLocalStoragePointers = {0x0, 0x0, 0x0, 0x0, 0x0},
  ulNotifiedValue = {0, 0, 0}, ucNotifyState = "\000\000", ucDelayAborted = 0 '\000'}
(gdb) n
982                     if( ( pxPreviousTCB != NULL ) && ( listIS_CONTAINED_WITHIN( &( pxReadyTasksLists[ pxPreviousTCB->uxPriority ] ), &( pxPreviousTCB->xStateListItem ) ) != pdFALSE ) )
(gdb)
vTaskSwitchContext (xCoreID=<optimized out>) at C:/path/to/FreeRTOS-Kernel-SMP/tasks.c:3959
3959        portRELEASE_ISR_LOCK();

Thread 2 hit Breakpoint 4, prvSelectHighestPriorityTask (xCoreID=xCoreID@entry=1) at C:/path/to/FreeRTOS-Kernel-SMP/tasks.c:821
821         {
(gdb) p uxTopReadyPriority
$33 = 0

Basically, this function starts with uxCurrentPriority at uxTopReadyPriority, and if it doesn’t find a task that should be scheduled next with that priority, it keeps decrementing uxCurrentPriority and looking for tasks. Here however, uxTopReadyPriority is 0 - and uxCurrentPriority even wraps (and gets value of 4294967295) when decrementing from that value (not sure if this is a bug, or if it has no effect).

But basically, from this point on, it seems that prvSelectHighestPriorityTask never seems to get to led_task with task priority 1 on core 1; so it either schedules IDLE* (task priority 0) or “Tmr Svc” (task priority configMAX_PRIORITIES-1, here 31) tasks on core 1.

So, if uxTopReadyPriority for core 1 became between 1 and 30, my guess is it would likely “notice” to schedule led_task at some point - so maybe something in this setup prevents uxTopReadyPriority from becoming 1 once timer ISR starts running; unfortunately, I cannot tell what. Otherwise, uxTopReadyPriority is indeed 1, in the first and only instance where LED_task gets scheduled:

Thread 1 hit Breakpoint 1, prvSelectHighestPriorityTask (xCoreID=xCoreID@entry=1) at C:/path/to/FreeRTOS-Kernel-SMP/tasks.c:821
821         {
(gdb) p uxTopReadyPriority
$4 = 1
(gdb) finish
Run till exit from #0  prvSelectHighestPriorityTask (xCoreID=xCoreID@entry=1)
    at C:/path/to/FreeRTOS-Kernel-SMP/tasks.c:821
0x10002170 in vTaskSwitchContext (xCoreID=1) at C:/path/to/FreeRTOS-Kernel-SMP/tasks.c:3938
3938                ( void ) prvSelectHighestPriorityTask( xCoreID );
Value returned is $5 = 1
(gdb) p *pxCurrentTCBs[ 1 ]
$6 = {pxTopOfStack = 0x20001e28 <ucHeap+992>, uxCoreAffinityMask = 2, xStateListItem = {xItemValue = 0,
    pxNext = 0x200015b8 <pxReadyTasksLists+28>, pxPrevious = 0x200015b8 <pxReadyTasksLists+28>,
    pvOwner = 0x20001e70 <ucHeap+1064>, pxContainer = 0x200015b0 <pxReadyTasksLists+20>}, xEventListItem = {
    xItemValue = 31, pxNext = 0x0, pxPrevious = 0x0, pvOwner = 0x20001e70 <ucHeap+1064>, pxContainer = 0x0},
  uxPriority = 1, pxStack = 0x20001a70 <ucHeap+40>, xTaskRunState = 1, xIsIdle = 0,
  pcTaskName = "LED_Task\000\000\000\000\000\000\000", uxCriticalNesting = 0, uxTCBNumber = 1, uxTaskNumber = 1,
  uxBasePriority = 1, uxMutexesHeld = 0, pvThreadLocalStoragePointers = {0x0, 0x0, 0x0, 0x0, 0x0}, ulNotifiedValue = {
    0, 0, 0}, ucNotifyState = "\000\000", ucDelayAborted = 0 '\000'}

EDIT: Also, found how to inspect the LED_task in gdb; below again for behavior B - we can look it up in pxReadyTasksLists for priority 1 (i.e. at index 1):

(gdb) p *(TCB_t*)pxReadyTasksLists[1].pxIndex.pxNext.pvOwner
$21 = {pxTopOfStack = 0x20001df8 <ucHeap+944>, uxCoreAffinityMask = 2, xStateListItem = {xItemValue = 500,
    pxNext = 0x200015b8 <pxReadyTasksLists+28>, pxPrevious = 0x200015b8 <pxReadyTasksLists+28>,
    pvOwner = 0x20001e70 <ucHeap+1064>, pxContainer = 0x200015b0 <pxReadyTasksLists+20>}, xEventListItem = {
    xItemValue = 31, pxNext = 0x0, pxPrevious = 0x0, pvOwner = 0x20001e70 <ucHeap+1064>, pxContainer = 0x0},
  uxPriority = 1, pxStack = 0x20001a70 <ucHeap+40>, xTaskRunState = -1, xIsIdle = 0,
  pcTaskName = "LED_Task\000\000\000\000\000\000\000", uxCriticalNesting = 0, uxTCBNumber = 1, uxTaskNumber = 1,
  uxBasePriority = 1, uxMutexesHeld = 0, pvThreadLocalStoragePointers = {0x0, 0x0, 0x0, 0x0, 0x0}, ulNotifiedValue = {
    0, 0, 0}, ucNotifyState = "\000\000", ucDelayAborted = 0 '\000'}

This is once the behavior B code has been running for a while; we can notice xTaskRunState = -1, and tasks.c tells us:

/* Indicates that the task is not actively running on any core. */
#define taskTASK_NOT_RUNNING    ( TaskRunning_t ) ( -1 )

/* Indicates that the task is actively running but scheduled to yield. */
#define taskTASK_YIELDING       ( TaskRunning_t ) ( -2 )

So led_task is in a state taskTASK_NOT_RUNNING apparently; at first, I thought this in itself is a problem (maybe I would have expected eSuspended here instead); however, vTaskSuspend can indeed set this state, so maybe its legitimate, and the question is why we cannot exit this state eventually.

So, I thought, a watchpoint on ((TCB_t*)pxReadyTasksLists[1].pxIndex.pxNext.pvOwner).xTaskRunState might reveal what’s going on, but I’ve had difficulties getting that watch: gdb - How to watch a member of a struct for every struct variable that has that type? - Stack Overflow

sdbbs · June 24, 2022, 2:40pm

Hi again, all & @jasonpcarroll

First: unfortunately the previous post of mine is now locked and I cannot edit it anymore - the SO link at the end is wrong, it should have been c - gdb and getting the absolute address of a struct field for watching? - Stack Overflow

Anyways - sorry about the panic, it turns out it might have been a “false alarm”.

Basically: the way I obtained the screenshots in the OP, was to first, run (in a MINGW64 bash terminal):

/c/path/to/openocd/src/openocd.exe -s /c/path/to/openocd/tcl -f interface/picoprobe.cfg -f target/rp2040.cfg -c "init ; reset halt ; exit"

… so I get everything to halt, and I get a bit of “empty space” at the left side of the screenshots; and then running:

/c/path/to/openocd/src/openocd.exe -s /c/path/to/openocd/tcl -f interface/picoprobe.cfg -f target/rp2040.cfg -c "init ; reset ; exit"

… to restart the code. Now, this openocd init/reset command has mostly been working fine for restarting code for me, … until now.

Somewhat by accident, I discovered that, when I have an openocd running in one terminal, called via:

/c/path/to/openocd/src/openocd.exe -s /c/path/to/openocd/tcl -f interface/picoprobe.cfg -f target/rp2040.cfg

… and then I connect with gdb:

gdb-multiarch rp2040_fros_tmrisr_ex.elf -ex 'target extended-remote localhost:3333'

… and then I issue a run command, which also restarts the code:

(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: C:\path\to\rp2040_fros_tmrisr_ex\build\rp2040_fros_tmrisr_ex.elf

… then, upon starting again, the code actually works fine, in the cases where we saw behaviors A and B previously !! (case for behavior D was already working, and it also keeps working in this case as well)

Strangely, behavior C persists, even when I re-run from gdb - so I guess the question of this thread can be reduced to that: why wouldn’t the timer ISR start, if we use hardware_alarm_* commands, but do not use vTaskCoreAffinitySet to pin led_task to CPU core 1?

Regardless: what matters to me most, is that ultimately there is no problem with using vTaskCoreAffinitySet in FreeRTOS SMP with RP2040, regardless of which way I control the timer ISR, so it’s nice to have confirmed that.

So, now I just have to find an openocd “one-liner” command, that can restart this type of code properly from the shell ( Openocd one-liner for proper multicore MCU reset? - Stack Overflow ) …

sdbbs · June 27, 2022, 7:28pm

Hi again,

Previous post stated that the correct behavior (D) in the OP example can be reproduced in cases A, B (and of course D), by restarting the code in gdb with run. Eventually, it is also found at the last SO link in previous post, that this “proper” restart can also be performed by restarting the MCU code, using this bash one-liner using openocd:

/c/path/to/openocd/src/openocd.exe -s /c/path/to/openocd/tcl -f interface/picoprobe.cfg -f target/rp2040.cfg -c "init ; reset halt ; rp2040.core1 arp_reset assert 0 ; rp2040.core0 arp_reset assert 0 ; exit"

The trick is basically to start core 1 before core 0 when restarting - not exactly sure why this would be needed, though…

Anyways, this means that behaviors A, B and D from OP are now accounted for, which leaves only behavior C. Thankfully, at this point it was somewhat easier to debug - since behavior C persists regardless of how the code is restarted (i.e., whether core 1 is started before core 0 during restart or vice versa). So, it would be logical to inspect the ISR startup in led_task:

(gdb) n
209       if (!(irq_is_enabled(ALARM_IRQ))) {
(gdb) p irq_is_enabled(2)       /// #define TIMER_IRQ_2 2
target halted due to debug-request, current mode: Thread
xPSR: 0x61000000 pc: 0x1000267c psp: 0x20003468
$1 = true
(gdb) n
target halted due to debug-request, current mode: Thread
xPSR: 0x61000000 pc: 0x10002640 psp: 0x20003460
215         gpio_put(PICO_DEFAULT_LED_PIN, 1);

Well, for some reason, here irq_is_enabled(ALARM_IRQ)) returns true, so:

  if (!(irq_is_enabled(ALARM_IRQ))) {
    irq_set_enabled(ALARM_IRQ, true);
    restart_timer_alarm();
  }

… the code never reaches the point where the timer is (re)started (for the first time).

It would be interesting to see what unexpectedly enabled the ISR - we can achieve that by watching the appropriate register address:

#// bool irq_is_enabled(uint num) {
#//     check_irq_param(num);
#//     return 0 != ((1u << num) & *((io_rw_32 *) (PPB_BASE + M0PLUS_NVIC_ISER_OFFSET)));
#// }
#// ...
#// addressmap.h:#define PPB_BASE _u(0xe0000000)
#// m0plus.h:#define M0PLUS_NVIC_ISER_OFFSET _u(0x0000e100)
#// 0xe0000000+0x0000e100 = 0xe000e100

(gdb) x/tb 0xe000e100
0xe000e100:     00000000
(gdb) x/tw 0xe000e100
0xe000e100:     10000000000000001000000000101100

(gdb) watch *(uint32_t*)0xe000e100
Hardware watchpoint 2: *(uint32_t*)0xe000e100

(gdb) r
The program being debugged has been started already.
Start it from the beginning? (y or n) y
...
Thread 1 received signal SIGTRAP, Trace/breakpoint trap.
0x100036b4 in irq_set_enabled (num=num@entry=2, enabled=enabled@entry=true) at C:/path/to/pico-sdk/src/rp2_common/hardware_irq/irq.c:39
39      }
(gdb) x/tw 0xe000e100
0xe000e100:     00000000000000000000000000001100
(gdb) bt
#0  0x100036b4 in irq_set_enabled (num=num@entry=2, enabled=enabled@entry=true)
    at C:/path/to/pico-sdk/src/rp2_common/hardware_irq/irq.c:39
#1  0x100044b8 in hardware_alarm_set_callback (alarm_num=alarm_num@entry=2,
    callback=callback@entry=0x20000115 <timer_alarm_isr>)
    at C:/path/to/pico-sdk/src/rp2_common/hardware_timer/timer.c:157
#2  0x100002fa in setup_timer_isr () at C:/path/to/rp2040_fros_tmrisr_ex/main.c:156
#3  0x10000346 in prvSetupHardware () at C:/path/to/rp2040_fros_tmrisr_ex/main.c:140
#4  0x100003ec in main () at C:/path/to/rp2040_fros_tmrisr_ex/main.c:53

(gdb) c
Continuing.
...

Indeed, it is the hardware_alarm_set_callback(ALARM_NUM, timer_alarm_isr); line, that also enables the interrupt request!

So, a simple solution is just to insert a irq_set_enabled(ALARM_IRQ, false); in setup_timer_isr( void ) after the hardware_alarm_set_callback... call; then, in any case, at start of the program the timer ISR will be disabled, and then it will be explicitly started from led_task. With this change, correct behavior (D) is also reproduced in case (C) (sidenote: for some reason, with this change I also get correct behavior regardless if I restart core 1 or core 0 first) - and thus, correct behavior (D) is reproduced now for all combinations (A, B, C, D) of commands for the program given in OP.

Well, hope this makes sense and is correct, and I’m not just covering over some even deeper bug in SMP behavior … And I wish someone else documented this, before I had to :)