Direct task notifications "missed" or "missing": typical causes?

Kernel
1

Hi all. Im having some issues with direct task notifications which appear to go missing, or are missed somehow.

I have an ISR that receives bytes from a UART into a buffer, and on receiving a CR (0x0D) a task is notified that the buffer is ready for processing.

This works the first time out of reset, but the second notification never seems to get through.

I can place a breakpoint in my ISR and see that the CR is recognised and the code to send the notification is executed, but interestingly when I have this breakpoint in place, the notification will always get through to the task and is never missed. As soon as I remove the breakpoint from the ISR, within the next one or two notification attempts things will start being missed again.

What are the typical causes of this behaviour that I should look out for? I havent yet stepped through the notification code to see what is happening, because I cant reproduce the behaviour when I have a breakpoint in my ISR. :frowning:

Im not seeing any crashes or other odd behaviour. The reception is triggered by the same task transmitting a command to a LoRaWAN radio module to check the MAC status, and on my oscilloscope I see that the message has indeed been transmitting and the response has also been sent by the radio module (complete with CR), and the ISR does see this and tries to notify the task, but the notification just doesnt seem to get through.

Im on a PIC32MX this time, so have decent debugging tools available. :slight_smile:

Thanks!

2

Which task notification functions are you using? From your description, it seems like you want to use task notification as counting semaphore. Another possible reason could be that your interrupt is firing too fast and the task is not getting a chance to run. Can you share code snippets showing the problem?

Thanks.

3

I only have a single buffer for receiving, so Im just using eSetBits to notify the receiving task that the buffer is ready to process via a flag. The interrupt rate is very low, at the moment once every ten seconds for testing, and only in response to me sending a command, and the baud rate is 57600, so hopefully that is not too much for it to handle. :slight_smile:

Code is attached.

radio_cmd_task() is created to manage sending commands and handling responses.

lora_mgr_task() sends a command to retrieve the MAC status, which happens through lora_mgr_mac_get_status().

The interrupts are handled in uart1_int_handler(), lines 167-217 for RX. The notification that a response is received happens at line 198 (after receiving a CR).

That notification would then be received in radio_cmd_task() line 329. In my initial case (this is a relatively new project that I am working on), this should then be processed beginning line 402 which would generate another notification (to the task that originally sent the command) allowing lora_mgr_mac_get_status() line 508 to continue.

It all works the first time, but the second time Im not getting a notification at line 329, even though the command has been sent and a response has been generated back to the PIC, and if I set a breakpoint in the ISR (e.g. at line 202) I am seeing the notification being sent every time a response is received from the radio.

Hope that all makes sense. :slight_smile:

task_radio.c (19.4 KB)

4

Not studied in details yet but a few notes.

Not related to your problem, the this code:

            xTaskNotifyFromISR(radio_cmd_handle,
                               RADIO_CMD_NOTIF_TX_COMPL,
                               eSetBits,
                               &yield_required_temp);
            yield_required |= yield_required_temp;

Can be simplified to:

            xTaskNotifyFromISR(radio_cmd_handle,
                               RADIO_CMD_NOTIF_TX_COMPL,
                               eSetBits,
                               &yield_required);

xTaskNotifyFromISR() will only ever set yield_required to pdTRUE - otherwise it won’t touch the variable, so the temp variable is not required.

This code could be problematic:

            result = xTaskNotifyWait((RADIO_CMD_NOTIF_TX_COMPL),
                                     (RADIO_CMD_NOTIF_TX_COMPL),
                                     &actions,
                                     pdMS_TO_TICKS(100));

First you are clearing the bits on entry to the function. If they are already set you will clear them, then wait for them…but they were already set. So there is a race that could result in you missing an interrupt. There is a timeout, so it will resume, but only after missing an event.

Second, you are not handling RADIO_CMD_NOTIF_RX_COMPL events here. That may not be a problem as you don’t clear that event and handle it elsewhere, but thought I would point it out just in case.

As you are using different calls to xTaskNotifyWait() for different bits, you may want to consider switching to using two different notification values with eIncrement action rather than eSetBit action. You can use xTaskNotifyWaitIndexed() on index 0 for Rx complete and then on index 1 for Tx complete - and incrementing the value rather than setting and clearing bits will mean you latch exactly how many interrupts occurred.

5

At a first glance, the only thing that looks problematic is clearing the bits on entry. Can you try not doing that?

task_radio.c (19.3 KB)

Thanks.

6

  1. Neat, thanks!

  2. This is an artifact left over from troubleshooting, I did originally have the clear on entry bits set to 0, and indeed I still seem to have the issue after changing them back.

For RADIO_CMD_NOTIF_RX_COMPL, this gets handled later on in the task. It first transmits a command and waits for a RADIO_CMD_NOTIF_TX_COMPL notification to know that is complete and responses should be following, and then proceeds to wait for responses with the RADIO_CMD_NOTIF_RX_COMPL notification.

  1. Ive tried implementing an indexed notification with eIncrement for the RX side, but still seeing the same behaviour. Placing a watch on the value returned from indexed notify wait I always see 1 event.

I then realised I was working on the development branch of FreeRTOS, rather than a release. Ive since tried 10.4.6, 10.4.5 and 10.4.4 but with the same behaviour. Doh!

Even tried posting the same notification twice in a row, but to no avail. Something isnt happy somewhere. :confused:

7

OOoooook, I think I might be sending you guys on a wild goose chase (again). :frowning:

My UART receive logic was faulty, specifically around the rx_cmd_is_null flag. Ive re-written the receive code and Im now getting all responses passed through correctly.

Trying to be clever, just not clever enough!

Case closed. Sorry! :innocent:

8

Thanks for taking the time to report back.