ulTaskNotifyTake: taskENTER_CRITICAL with portYIELD_WITHIN_API

The problem is in the implement of ulTaskNotifyTake that it masked the interrupt in the critical section. And our interrupt handler will not be triggered and there is a deadlock.

ulTaskNotifyTake equal to ulTaskGenericNotifyTake
FreeRTOS-Kernel/blob/main/tasks.c

The deadlock is happen in this way.

  1. There is a task A which use ulTaskNotifyTake to get a notification from an interrupt B
  2. In ulTaskNotifyTake, the interrupts are masked. Source code showed bellow.
  3. The interrupt will never be delivered because it is masked. the vTaskNotifyGiveFromISR will not be called in the interrupt handler.
  4. Deadlock happen.
    uint32_t ulTaskGenericNotifyTake( UBaseType_t uxIndexToWait,
                                      BaseType_t xClearCountOnExit,
                                      TickType_t xTicksToWait )
    {
        uint32_t ulReturn;

        configASSERT( uxIndexToWait < configTASK_NOTIFICATION_ARRAY_ENTRIES );

        taskENTER_CRITICAL();
        {
            /* Only block if the notification count is not already non-zero. */
            if( pxCurrentTCB->ulNotifiedValue[ uxIndexToWait ] == 0UL )
            {
                /* Mark this task as waiting for a notification. */
                pxCurrentTCB->ucNotifyState[ uxIndexToWait ] = taskWAITING_NOTIFICATION;

                if( xTicksToWait > ( TickType_t ) 0 )
                {
                    prvAddCurrentTaskToDelayedList( xTicksToWait, pdTRUE );
                    traceTASK_NOTIFY_TAKE_BLOCK( uxIndexToWait );

                    /* All ports are written to allow a yield in a critical
                     * section (some will yield immediately, others wait until the
                     * critical section exits) - but it is not something that
                     * application code should ever do. */
                    portYIELD_WITHIN_API();

The taskEnter_CRITICALA() masked the interrupt.
portYIELD_WITHIN_API(); supposed to be wakeup by the interrupt handler. But it could not as interrupt will not happen after portYIELD_WITHIN_API();