Sharing data between task and fast ISR

Kernel
#1

I have a 10kHz motor control task that needs to have sub-microsecond latency (it is synchronized with a PWM waveform). The forum gods tell me that this is too fast to be a FreeRTOS “task”, and should instead be computation done directly in the ISR.

However, I have another task (lets call it hi_prio_task) that needs to run immediately “in lockstep” with the 10kHz ISR (e.g. every third 10kHz ISR should immediately trigger the task to run on exit from the ISR). The hi_prio_task produces data for the ISR to consume (new motor control setpoints), as well as act upon the ISR outputs (motor control outputs are used as inputs to a slower control loop, made available for query over a network, used for fault checks, etc.). The compute time for the hi_prio_task is not small enough to fit into a 100us window, but should not overflow the 300us window if it runs every 3 motor contol ISRs.

Is it appropriate for the motor control ISR to be at a higher priority than FreeRTOS such that it can never be delayed?

It seems that the answer is “yes, but you can’t use any FreeRTOS APIs”. This makes getting data into and out of the ISR a bit difficult, but I have some thread-safe abstractions available to do so if needed (atomic double buffer)

I have seen it suggested that in scenarios like this, you should do the fast computation in the ISR (motor control), and then unblock a higher priority FreeRTOS task to consume the outputs of said computation.

But how is this possible if I can’t use any FreeRTOS APIs in the motor control ISR? Isn’t the act of unblocking a task an API call? Is there any other way to unblock a task from an ISR at a priority level such that it can’t be delayed by internal operations?

Ideally, I would want to use two stream buffers (to/from ISR versions) to manage getting setpoints and results in/out of the ISR, and then unblock the hi_prio_task every third ISR to drain/populate the stream buffers for the next cycle.

Can FreeRTOS support a use case like this?

Any and all help greatly appreciated.

Edit: This is using the NXP MIMXRT1051xxx port (cortex M7)

#2

A good method (originally proposed by Richard Damon here in the forum time ago) is setting a software interrupt with a FreeRTOS covered priority in the highest-prio (outside FreeRTOS covered interrupt prio range) motor-control ISR.
In this chained ISR you could do the FreeRTOS housekeeping.

#3

FreeRTOS will only switch tasks when the following conditions are true:

  1. Preemptive multitasking is active AND the task timer expires AND a different task is higher priority.
  2. An API call such as a write to a queue or a stream buffer unblocks a higher priority task and the kernel is configured to immediately schedule the higher priority task.

The preemptive task timer interrupt is intended to be the lowest priority interrupt in the system.

With these conditions known, it seems that a 10khz motor interrupt at a high interrupt priority will easily get all the time it needs. The ISR can use a direct task notification to wake the hi_prio_task every third interrupt.

While there are no interrupts and the hi_prio_task is blocked, other tasks will run.
If you call portYIELD_FROM_ISR() at the end of the motor ISR, then the motor ISR will exit directly into the hi_prio_task which seems to be the behavior you need. The link above has an example of this operation.

#4

Yes, when you have a ISR that needs to be very fast and no latency, you want it above the ‘FreeRTOS line’, if is occasionally needs to send a notification, find some other interrupt in the system that it can trigger in software to handle that notification, and put that ‘below the line’.

#5

If the motor control ISR is a higher priority then FreeRTOS, isn’t it illegal for me to call portYIELD_FROM_ISR()? I am interpreting that as being part of the FreeRTOS API…

#6

No that is not illegal. That is the intended function. This will cause the scheduler to immediately reschedule the tasks to the freshly unblocked task can start running immediately. One of the scheduler rules for FreeRTOS is to ALWAYS run the HIGHEST PRIORITY task. So when an event occurs that changes the priority we try to immediately unblock and reschedule.

In fact, you can run the kernel in cooperative mode and only switch tasks on such an event or by intentionally yielding. This eliminates the timer preemption.

#7

I was just reminded. You cannot call an API function from an interrupt that is higher priority than the configuration configMAX_SYSCALL_INTERRUPT_PRIORITY so if you adopt this method, you need to be careful of that setting.

#8

Yeah, that is what I was referencing. And that is what I would need to do if I wanted my motor control ISR to never be delayed by a potential kernel critical section right? So It looks like I cannot call portYIELD_FROM_ISR safely in this case?

#9

I understand your concern. That makes sense. The strategy proposed by Hartmut where you issue a software interrupt to trigger the hi_prio_task makes more sense. Then the question becomes, do you trigger an intermediate interrupt that is below the line and use that to trigger the hi_prio_task or do you make the hi_prio_task into a hi_prio_isr and leave it above the line.

#10

Yeah, at that point it seems like the benefits of using the RTOS seem to dwindle if I’m going to have to lean on the NVIC and its exception priorities as a task scheduler anyway. I guess it might still be useful for scheduling various other background tasks, but I’d need to think on it.

I think “trigger an immediate interrupt that is below the line and use that to trigger the hi_prio_task” would need to be the case here, as if I used the “make the hi_prio_task into a hi_prio_isr and leave it above the line.” strategy, I still wouldn’t be able to use any of the API functions.

But if I “trigger the hi_prio_task” in an ISR below the line, there isn’t necessarily a guarantee that I will immediately fall into it either, correct? Since the exception priority will be less than that of FreeRTOS. Not sure that latter part matters as much, but is something I need to consider, as FreeRTOS could have been in the middle of a critical section/syscall when preempted by the motor control ISR. So even if the motor control ISR is able to trigger some other ISR meant to unblock hi_prio below the line, it wouldn’t directly vector into that handler until the FreeRTOS syscall completes?

Let me know if any of that sounds wrong, still trying to wrap my brain around this

#11

If the kernel is in a critical section when the motor ISR triggered and the motor ISR triggers the hi_prio_trigger… then the hi_prio_trigger will be blocked until the kernel is out of its critical section. At which point the hi_prio_trigger will execute and the kernel will then unblock and schedule the hi_prio_task. This will naturally add some latency to the task start. You will need to characterize that for your system and see if it is too much.

#12

I wonder if you could configure a second timer in 1/3 lockstep with the first timer and just make that timer below the line. That would save the software trigger and may slightly reduce the latency.

#13

Okay, that is what I thought, thank you for clarifying. Seems like I need to keep brainstorming.

For those reading this, the answer I was looking for when I came here is that NO YOU CAN NOT use portYIELD_FROM_ISR() from an ISR who’s execution context is at a higher priority (lower exception prio in Arm-speak) than FreeRTOS syscalls. That function counts as an API call as you would think. No free lunch.

We are on the same wavelength. This chip has an elaborate xbar (mux) network that, in theory, should allow me to upcount one of the timers (QTMR) with the PWM reload signal (or a comparable compare value). That was actually my first attempt, before trying to lean on FreeRTOS, however I haven’t gotten it to work yet. I’ll report back if I can get something worth talking about together using FreeRTOS.

Thanks for the help.

#14

I’d like to add that FreeRTOS tries it’s very best to keep critical sections as short as possible.
So don’t expect a serious jitter when using the forwarding ISR approach also b/c it seems you’ve a pretty beefy MCU (600 MHz clock ? Wow…)

1 Like
#15

Okay, I have an interesting update on this.

I’m doing exactly what @richard-damon and @n9wxu suggested - using the “start data aquisition in an interrupt above the line, then trigger an immediate interrupt that is below the line and use that to trigger/unblock the hi_prio_task” strategy. From an interrupt nesting perspective, it is doing exactly what I want. Data is being acquired with native interrupt latency!

Unfortunately, I’m seeing something very odd in the scheduling of the hi_prio_task. The first time through it work just as expected. But Every other time that it is unblocked by the secondary ISR “below the line” (priority of configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY ) there is an 80us delay between unblocking the task via direct-to-task notification, and the task running. It goes on like this forever… a relatively instantaneous unblocking of the hi_prio_task and then an 80us delay between unblocking it. The 80us delay causes the next PWM ISR to pre-empt the task before it runs to completion.

Is 80us scheduling latency in any way expected on a 528MHz part with only one task in addition to the idle task? I feel like something must be going wrong here. As @hs2 mentioned, I should be expecting low jitter given my clock rate, and I don’t even think this is jitter anyway, given the periodicity.

Below is the pseudocode for the two ISRs and the hi priority motor control tasks, and a corresponding oscilloscope trace showing the issue. The code shows where both yellow and blue scope trace signals are set high/low. You can notice that every other time the blue trace shows an 80us delay between task dispatch and task execution, and this causes it to fire twice in a row.

// exception prio of configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY - 1
PWM_ISR {
  
  // (see oscilloscope trace)
  YELLOW_SCOPE_TRACE_HIGH()

  start_nonblocking_data_aquisition()  

  // based on priorities, we fall directly into "below the line ISR" upon 
  // exiting this ISR (verified)
  trigger_below_the_line_ISR()
}

// exception prio of configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY
BELOW_LINE_ISR() {
   // unblock motor control task, since we are now below the priority threshold by which we can use FreeRTOS APIs
    BaseType_t xHigherPriorityTaskWoken = pdFALSE;
    vTaskNotifyGiveFromISR(MotorCtrlAlgorithmTaskHandle, 
                           &xHigherPriorityTaskWoken);

    // (see oscilloscope trace)
    YELLOW_SCOPE_TRACE_LOW()

    // we should resume execution directly in the motor control task, 
    // as it is the highest priority task in the system
    portYIELD_FROM_ISR(xHigherPriorityTaskWoken);

}

// initialized to a task priority of configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY
MOTOR_CONTROL_TASK() {
    while (1)
    {
        // block on direct to task notification
        ulTaskNotifyTake(pdTRUE, portMAX_DELAY);
        
        // (see oscilloscope trace)
        BLUE_SCOPE_TRACE_HIGH()

        complete_nonblocking_data_acquisition();
        run_motor_control_algo()

        // (see oscilloscope trace)
        BLUE_SCOPE_TRACE_LOW()
    }
}
[Album] imgur.com

Thanks so much for your time.

#16

I’m blind … the latency between the falling yellow trace edge and the rising blue trace edge is sometimes 1 or a few more us, isn’t it ?
Did you try to comment out ?

    complete_nonblocking_data_acquisition();
    run_motor_control_algo();

to measure just the FreeRTOS runtime behavior ?
80 us is by far too much latency. There is something else interfering here.

#17

Apologies, I guess the accessibility of posting a picture isn’t the best. It periodically alternates between 1us and 80us, which is the problem. 1us is fine, 80us is not. It should be ~1us all the time.

Right now, those functions are just loops with NOPs for that exact reason. So we should be seeing pure runtime behavior. Also, the task logic itself isn’t what is causing the delay, since that would be occuring while the blue scope trace is high. The delay I’m seeing is between the falling yellow edge (yellow is time it takes from start of PWM ISR to the end of the “below the line” ISR), and the rising blue edge (just the task itself)

#18

That’s weird… it 1st looked like the task code takes excessive amount of time i.e. 80 us.
But I’was wrong, The time with the blue trace being high is the simulated task processing time.
So just to be sure:

  • task prio is tskIDLE_PRIORITY +1 or higher and it’s the only application task running (besides the idle task) and also FreeRTOS timers are not used ?
    (BTW since you seem to have <= 32 tasks you probably enabled configUSE_PORT_OPTIMISED_TASK_SELECTION)

  • task stack size is large enough ?

  • you’ve selected preemptive and maybe time sliced scheduling ?

  • you double checked the interrupt priorities against configLIBRARY_MAX_SYSCALL_INTERRUPT_PRIORITY ?

Sorry for the stupid questions, but currently I’ve no idea where the lag comes from :thinking:

#19

A few observations:

  1. You might try to run Percepio tracealyzer to determine whether the latency comes from other tasks/ISRs/OS cycles or not.

  2. If that doesn’t yield results, it may be a caching issue. At what clock frequency exactly do you run your MCU, or asking the other way around, how many CPU cycles correspond to 80us?

  3. Does your target/probe/tool chain support tracing? Of course the most reliable way to sort is out is simply look at the instructions in between the delay…

#20

@RAc In fact the clock is

which is blazing fast. Or in other words 80 us is a pretty long time…