After some more investigation I found that out that the configTIMER_TASK_PRIORITY was set to 2. Increasing the priority to be the highest in the system got rid of this issue. It looks like that when setting an event from an ISR, it goes through the Timer Service task that is set to priority configTIMER_TASK_PRIORITY. Having it set so low caused it not to wake up to tell the higher priority thread to run when any task above a priority of 2 was running. Not sure why it defaults so low considering how important this Timer service task is.