portable/GCC/ARM_CM3/port.c:vPortValidateInterruptPriority requires that SCB_AIRCR.PRIGROUP be zero. It assumes that zero means that all the bits are used for group/preemption priority and none for a subpriority.
On STM32F2/4, only the values 3 through 7 are documented for PRIGROUP, with 3 meaning that there are no subpriority bits. I tried setting PRIGROUP to zero in an attempt to please vPortValidateInterruptPriority, but this causes great confusion in interrupt handling (I still get some interrupts, but others fire just once, never to be seen again).
Setting PRIGROUP to 3 and adapting the test in vPortValidateInterruptPriority accordingly makes things work as expected.
I suppose this has something to do with the STM32 only having four priority bits. Perhaps the correct value for PRIGRP should be calculated as (configPRIO_BITS == 8 ? 0 : 7-configPRIO_BITS) ?
The check for 8 is necessary because - according to Table 4.18 on http://infocenter.arm.com/help/topic/com.arm.doc.dui0552a/Cihehdge.html - even 0 still leaves one subpriority bit, so on a core that actually implemented all 8 bits, the calculation would wrap around 0.
I’ve seen this on an STM32F207 (CM3) and it seems that the same should also apply to STM32F4xxx (CM4F), and possibly others.