Even when using nested interrupts this must not happen. It seems that the interrupt priorities resp. the related FreeRTOS configuration (as Richard mentioned to ensure that critical sections are working properly) is not correct.
Just to be sure: You’re using the FreeRTOS API tagged with FromISR in your ISRs, right ?
Since nRF52820 has a Cortex-M4 CPU did you see this related documentation ?