Callback causing configASSERT in xQueueSend()

I am using a 3rd party library which uses a callback function to send notifications. The callback function is called from an ISR. I’ve been using that callback function to print debug messages. The structure in which these debug messages are printed is via a logging task. I also have the MPU enabled to complicate matters. It sounds complicated, but it comes down to this:

ISR → callback() → logger() → log task FreeRTOS Queue → log task() → printf() → UART

The assert comes in when the logger() puts entries into the log task’s queue. I get the assert intermittently (configASSERT( !( ( xTaskGetSchedulerState() == taskSCHEDULER_SUSPENDED ) && ( xTicksToWait != 0 ) ) ); on line 849 in queue.c). I only have configUSE_TIMERS enabled. When I turn off this configuration, the problem seems to go away. However, if I ever want to use timers, this problem will come back.

I’ve seen a couple of articles on this in the forum and frankly the answers confuse me or are beyond my comprehension as of this moment. I basically want to know the simplest solution to this callback from ISR problem.

Thanks,

Johnas

Do you use the correct FromISR API putting the messages into the logging queue ?
Because the callback is invoked from ISR context.
Did you define configASSERT and enabled stack checking ?
I guess you don’t use FreeRTOS notifications in addition since you already use a queue.
Using FreeRTOS timers shouldn’t matter regarding your problem. They are handled by a separate timer task.

What does your printf() translate to on the back end level? There may be implied assumptions about a task context.

Again, do not use serial output in an isr unless you know in detail what you are doing.

The error would imply using a non-FromISR call inside an ISR (since that is the only place that would have a xTicksToWait.

You don’t say what processor you are on, but it may be one that doesn’t support checking if in an ISR, so the error of calling a non-FromISR function is only caught when an interrupt comes during a suspended scheduler section.

Ultimately, the error is caused by using a non-FromISR routine inside an ISR (which includes all functions called by the ISR).

This could be the case if your logger doesn’t have a “FromISR” version for the callback to call (and depending on the callback, you may need a normal and FromISR for it if it can be called from a task level too.)

I had suspected what all of you are saying. However, I did try to use the from ISR version of the Queue send. I had the logger check if it was in an ISR context or not and tried calling the appropriate Queue send in each case. However, when I did that it immediately crashed when I called the from ISR Queue send coming from the callback function. Perhaps I did something wrong, I’ll try this again. I’m using an ARM Cortex M33 (nRF9160) in the non-secure domain.

In answer to your advice, RAc, I am using a UART. However, the UART is called off of the log task which should not be running in an ISR. Therefore, your scenario should not occur. The data is queued. It is also possibe that the queue’s FIFO has overflowed since the log task cannot run until the ISR has exited. I should get an error if I try to push one entry too many into the log queue. I’ll activate stack checking to further debug. I did double the size of the stack at one point, but I believe the assert still occurred.

OK, good advice from everyone. I’ll give your suggestions a try.

Thanks,

Johnas

By “Crash” could it have been an “Assert”, if so. perhaps your ISR is at an incorrect priority.

No, I mean hard fault. I was able to stop this. It seems that an unprivileged task call to my logger is causing the hard fault. The logger checks to see what context it’s in (normal or interrupt) and then calls the appropriate queue send function. All is fine until an unprivileged task tries to print a message. The fault occurs on the if statement before the send to the queue. The reason behind this may be that I’m accessing a register that can only be accessed by a privileged task. Is there a FreeRTOS system call that can determine the context and is safe to call from an unprivileged task?

Instead of doing that, can you not add another API to your logger which is for ISR only?

I could and I’ve thought of that. I was hoping to make the code cleaner and more elegant though. So, no FreeRTOS system call to determine context?

No, we do not have any.

Just to add here - having a separate API for ISR makes the code more efficient as you are not performing the context check on every call.

Oh I see. Can we see your isr code, or the fragment of it that does the ipc? Maybe you attempt to push a püointer into the ist stack to the queue instead of the queue item itself?

The ISR code is not of much interest, but I can post it. I can also show most of the call chain:

This is the ISR (the call to the callback function is done in the 3rd party library function nrf_modem_application_irq_handler() which I don’t have source code for but I can configure the name/pointer of the callback function):

void EGU1_IRQHandler( void )
{
    nrf_modem_application_irq_handler();

    BaseType_t higher_priority_task_woken = pdFALSE;
    vTaskNotifyGiveFromISR( modem_obj.sleeping_task.handle, &higher_priority_task_woken );
    portYIELD_FROM_ISR( higher_priority_task_woken );
}

This is the callback which is called by the ISR:

void ModemNotificationCb( const char * notification )
{
    uint32_t i;
    int32_t parameters[ 5 ];
    char *result;

    for ( i = 0; i < sizeof( modem_response ) / sizeof( char * ); i++ )
    {
        result = stristr( notification, modem_response[ i ] );
        if ( result != NULL )
        {
            break;
        }
    }

    if ( result != NULL )
    {
        switch ( i )
        {
        case 0:         // CESQ:    RSRP, RSRP Threshold, RSRQ, RSRQ Threshold
            result = stristr( notification, ":" ) + 1;
            sscanf( result, "%d, %d, %d, %d", 
                            &parameters[ 0 ], 
                            &parameters[ 1 ], 
                            &parameters[ 2 ], 
                            &parameters[ 3 ] );
            Log.DebugPrint( "RSRP: %d dBm\tRSRP Threshold: %d\tRSRQ: %d dBm\tRSRQ Threshold: %d", 
                            parameters[ 0 ] - 141, 
                            parameters[ 1 ], 
                            ( parameters[ 2 ] - 40 ) / 2, 
                            parameters[ 3 ] );
            break;
        case 1:         // CSCON:   a,b,c
            result = stristr( notification, ":" ) + 1;
            sscanf( result, "%d", &parameters[ 0 ] );
            Log.DebugPrint( "%s", parameters[ 0 ] == 1 ? "Connected" : "Idle" );
            break;
        case 2:         // CEREG:   a,"b","c",d,,,"e","f"
            result = stristr( notification, ":" ) + 1;
            sscanf( result, "%d", &parameters[ 0 ] );
            Log.DebugPrint( "%s", parameters[ 0 ] == 1 || parameters[ 0 ] == 5 ? "Registered" : "Not registered" );
            if ( ( parameters[ 0 ] == 1 ) || ( parameters[ 0 ] == 5 ) )
            {
                Log.InfoPrint( "Modem is registered on %s network", parameters[ 0 ] == 1 ? "home" : "roaming" );
                modem_obj.is_registered = true;
            }
            else
            {
                Log.InfoPrint( "Modem is unregistered" );
                modem_obj.is_registered = false;
            }
            break;
        case 3:         // XOPNAME: "a","b","c"
            break;
        }
    }
}

This is the last function in a chain for functions of the logger before the queue send to the log task (note the comment that points out the if statement that checks the context):

void log_LevelPrint( log_level_t log_level, const char *fmt, va_list args )
{
    log_qmessage_t log_msg;
    BaseType_t high_priority_task = pdFALSE;

    vsnprintf( log_msg.data, LONG_MSG_MAX - 1, fmt, args );
    log_msg.ticks = xTaskGetTickCount();
    log_msg.log_level = log_level;
    log_msg.handle = xTaskGetCurrentTaskHandle();

    /* Use correct context to send to log queue */
    if ( ( SCB->ICSR & SCB_ICSR_VECTACTIVE_Msk ) != 0 )  // In interrupt context
    {
        xQueueSendFromISR( app.GetLogHandle(), &log_msg, &high_priority_task );
        /* Now the buffer is empty we can switch context if necessary. */
        if ( high_priority_task )
        {
            portYIELD_FROM_ISR( high_priority_task );
        }
    }
    else                                                // In normal context
    {
        if ( xQueueSend( app.GetLogHandle(), &log_msg, pdMS_TO_TICKS( 100 ) ) != pdPASS )
        {
            /* Message failed to send */
        }
    }
}

Finally, the log task itself:

void log_Thread( void *parameter_ptr )
{
    uint32_t i;
    log_qmessage_t msg;
    TickType_t msg_time = pdTICKS_TO_MS( xTaskGetTickCount() );
    char time_buf[128];

    twdt.Configure( TWDT_TIMEOUT );
    #ifdef INIT_LOG_LEVEL
    log_obj.log_level = INIT_LOG_LEVEL;
    #else
    log_obj.log_level = loglevel_info;
    #endif
    log_obj.task_list.type = logtask_all;
    log_GetTimestamp( msg_time, time_buf );
    printf( "[%-12s] <%s>: %s\r\n",
             pcTaskGetName( xTaskGetCurrentTaskHandle() ),          // Task name
             time_buf,                                              // Timestamp
             "Log task started" );                                  // Message
    
    for( ; ; ) 
    {
        twdt.Update();

        /* Wait for the maximum period for data to become available on the queue. 
         * The period will be indefinite if INCLUDE_vTaskSuspend is set to 1 in 
         * FreeRTOSConfig.h. 
         */
        if( xQueueReceive( app.GetLogHandle(), &msg, pdMS_TO_TICKS( TWDT_TIMEOUT / 2 ) ) == pdPASS )
        {
            
            switch ( msg.log_level )
            {
            case loglevel_force:
                printf( "%s", msg.data );
                break;
            
            case loglevel_task:
                log_obj.task_list.type = msg.task_list.type;
                for ( i = 0; i < LOG_MAX_LIST; i++ )
                {
                    log_obj.task_list.handle[ i ] = msg.task_list.handle[ i ];
                }
                break;
                
            default:
                /* xLogMessage now contains the received data. */
                msg_time = pdTICKS_TO_MS( msg.ticks );
                
                if ( log_Show( msg ) )
                {                    
                    log_GetTimestamp( msg_time, time_buf );
                    printf( "[%-12s] <%s>: %s\r\n",
                             pcTaskGetName( msg.handle ),                // Task name
                             time_buf,                                   // Timestamp
                             msg.data );                                 // Message
                }
                break;
            }
        }
    }
}

The callback is no longer causing asserts which fixes the original problem. However, now when an unprivileged task in normal context calls the logger, I get a hard fault. Configuring the unprivileged task as privileged seems to solve this problem. Now, the problem is not as severe, but this means that every task that needs to print will have to be privileged. That is not good.

Are you verifying that the notification strings coming from your modem are always well-formed? I have yet to see a modem that does not send garbage every once in a while, and to me it seems as if all of your parsing assumes well formed response strings.

I search for certain patterns, process, and print my own string. Nevertheless, this has nothing to do with the current problem. The issue is no longer the ISR. The problem is with the normal context printing.

Thanks for sharing the code. The usage constrains for ICSR register are that “Privileged access permitted only. Unprivileged accesses generate a fault.”. Therefore, as you said accessing ICSR register from unprivileged code might be the most probable cause of the fault.

Instead of accessing the ICSR register, can you try the API xPortIsInsideInterrupt and see if that helps.

Absolutely. I was told such a function in FreeRTOS doesn’t exist. Thanks!

xPortIsInsideInterrupt may have the same issue, as it likely assumes that the code is running unprotected, after all, code expecting to be run protected can’t be inside an ISR.

“Multi-mode” code is tricky to write correctly. Simplest is to have versions for each operating mode.

xPortIsInsideInterrupt uses IPSR which is also accessible to privileged code only.

Is there a reason why “snprintf” isn’t being used in place of “printf” ?? I am too new to FreeRTOS to know if there’s a good reason NOT to use it.
Thanks!

Hi madGambol. snprintf, prints to a buffer. I’m redirecting to a UART. printf will call putchar which I’ve replaced with my implementation. I don’t believe snprintf calls putchar. Anyway, my implementation is stable at this point. I have an idea as to how to get it to allow for unprivileged tasks. It doesn’t involve the RTOS though. So, my question is, why doesn’t FreeRTOS have a function that can be called from an unprivileged task to determine the context? There are other system calls that do not have a requirement to be privileged.