freeRTOS doesnt task swap on APU #2,3 on Xilinx ultrascale+ MPSoc Cortex A53 MP

sarnovian wrote on Saturday, November 02, 2019:

freeRTOS tasks do not run on APU#2,#3 on mpsoc+
Is anyone succesfully running freeRTOS on a MPSoc+ APU #2 or 3? If so did you have to do anything special in the hardware setup or BSP alterations to make it work?
I’ve found that our application (freeRTOS on A53) works fine on APU #0, #1, but when we use it on APU #2 or 3 the task never execute. E.g. it appears the tickInterrupt isnt occuring so the scheduler never swaps which task is run.
I’ve tried using different TTC counter sources, and I have verified they are correctly setup by the BSP to generate the IRQ to the specific core, and that the counter is interval setup and counting correctly. The GIG SPI IRQ also seems to be set correctly.

If I Change the vTaskDelay(500) statements to vTaskDelay(0), then the seperate two threads do execute and continously swap between them.
This implies the scheduler and its associate interrupt is running ok when the vTaskDelay(0) is set since this tells the RTOS to swap tasks every TICK. I’ve verified that in portASM.s, FreeRTOS_SWI_Handler ASM function is also being called.

However, FreeRTOS_IRQ_Handler ASM function in same file never gets called. I think this is the IRQ used by the schedular when tasks are put to IDLE for >0 ticks…

Anyone have any ideas?

This is repeatable with even a very simple test design:
NOTE: This test design and BSP works fine on CPU 0 and CPU 1 if I change the BSP CPU ID parameter (which controls where the SPI IRQ from the GIG400 Distributor is sent). For this test I’m only running one of the CPU’s at a time to be sure there is no corruption of shared memory, or IRQ conflicts between the CPU applications.

#include <sleep.h>
#include <stdbool.h>
#include <stdlib.h>
/* FreeRTOS includes. */
#include “FreeRTOS.h”
#include “semphr.h”
#include “task.h”
#include “queue.h”
#include “timers.h”
void testThread1 (void *p)
{
while(1) {
vTaskDelay (500);
xil_printf(" TASK Thread1\n");
}
}
void testThread2 (void *p)
{
while(1) {
vTaskDelay (500);
xil_printf(" TASK Thread2\n");
}
}

int main( void )
{
BaseType_t threadCreateStatus[2];

xil_printf(" APU2 Starting APP...\n");

threadCreateStatus[0] =xTaskCreate(testThread1, "testThread1", 1024, NULL, 2, NULL);
threadCreateStatus[1] =xTaskCreate(testThread2, "testThread2", 1024, NULL, 2, NULL);


// monitor main big threads to see if we have memory configured correctly:
if (threadCreateStatus[0] == pdPASS &&
	threadCreateStatus[1] == pdPASS)
{
	vTaskStartScheduler();
} else {
	// memory allocation issues:
	while (1);
}

while (1);
return;

}

sarnovian wrote on Saturday, November 02, 2019:

I’ve confirmed that on CPU 0 or 1, the port_asm_vecotors.S:
.org(FREERTOS_VBAR)
_freertos_vector_table:
b FreeRTOS_SWI_Handler

.org (FREERTOS_VBAR + 0x80)
b FreeRTOS_IRQ_Handler <= THIS IRQ is hit repeatably when run on CPU[0,1]

BUT on CPU[2,3] that IRQ Vector is never hit. So I think the issue is in how the Xilinx PORT sets up the GIG.

rtel wrote on Saturday, November 02, 2019:

I’ve only run on CPU 0 I’m afraid so interest in what you find.

sarnovian wrote on Saturday, November 02, 2019:

Hopefully I figure it out… I’ll keep it updated here.

You dont by chance have the ability to run on CPU[2,3] on a mpsoc board? E.g. just to confirm what I’m seeing is not unique?

rtel wrote on Saturday, November 02, 2019:

Not at the moment unfortunately.

sarnovian wrote on Saturday, November 02, 2019:

Ok, after digging into the ARM IHI 0048B GIC400 v2 specification to understand how the ARM IRQ Distributor works, I found that after the RTOS boots and configures the scheduler, that the GIC IRQ Target registers are not configured correctly.

On CPU#2 (3rd ARM A53 core in mpsoc), if I read
mr 0x0000F9010844 1
=> Return value is 0x03030303
Which I interpret as routing of the IRQ#68 (TTC0 Timer Interval IRQ) to CPU[0] & CPU[1].

If I manually overide it and set the CPU target to CPU[3]:
mw 0x0000F9010844 0x04040404
Then the RTOS switches tasks correctly and IRQ processing works…
I can fix this myself in my application or BSP, but I think this is a bug in the BSP code driver for the xscugic (Xilinx code).

I posted this on the XIlinx Forums to see if anyone From Xilinx BSP support wants to take a look an confirm this so if needed it can be fixed in future. This definately is not a FreeRTOS issue as it runs fine after I added the register fix above.