Task is starving

Kernel
1

Hi,

We are running FreeRTOS and FreeRTOS TCP IP Stack on TI uController TM4C129 series.
We have only one task (we call task1) with priority of tskIDL_PRIORITY other than the FreeRTOS_IPInit task.

Task1 has bunch of items to take care of at 250m sec. Task1 also blinks an LED and gives us a visual indication that the task1 is running.

We are able to communicate with the device remotely using FreeRTOS-TCP Stack using UDP-IP protocol and initially the program runs without any issue. We collect data from the device at approximately 30Hz via Ethernet.

However we have noticed the Task1 will stop running after 24-30 hours while the device still communicate on UDP-IP to the remote server.

Here is how I initiate the Task1 and the FreeRTOS_IPInit().

xTaskCreate(Task1, (const portCHAR *)"Task 1", configMINIMAL_STACK_SIZE, NULL, tskIDLE_PRIORITY, NULL);
FreeRTOS_IPInit( ucIPAddress, ucNetMask, ucGatewayAddress, ucDNSServerAddress, ucMACAddress );
vTaskStartScheduler();

Following is the call back associated with Task 1;

void Task1(void *p) 
{

    for (;;)
    {

        Faults_Status = I2C_read(IOChip_I2C_BASE, IOChip_I2C_CHIP_ADDRESS, IOChip_I2C_INPORT0);

        IntADC_Collect_Data(IntADCRawData);
//Read variables
        Fan1.data.value16bits   =   0x0000FFFF & IntADCRawData[4];
        Fan2.data.value16bits   =   0x0000FFFF & IntADCRawData[3];
        volt3.data.value16bits  =   0x0000FFFF & IntADCRawData[2];
        volt5.data.value16bits  =   0x0000FFFF & IntADCRawData[1];
        volt15.data.value16bits =   0x0000FFFF & IntADCRawData[0];


        if(GPIOPinRead(ComMode_PORT,COM_MODE_PIN) == LOCAL)
            {
                ComMode = LOCAL;
                GPIOPinWrite(ComMode_PORT,COM_REMOTESTATUS_PIN,0);
                GPIOPinWrite(ComMode_PORT,COM_LOCALSTATUS_PIN,COM_LOCALSTATUS_PIN);
               

            }
        else
            {
                ComMode = REMOTE;
                GPIOPinWrite(ComMode_PORT,COM_LOCALSTATUS_PIN,0);
                GPIOPinWrite(ComMode_PORT,COM_REMOTESTATUS_PIN,COM_REMOTESTATUS_PIN);

            }

        if(ComMode == LOCAL)
        {
            if(Reset_PS_Flag == 1)
                {
                    Reset_PS_Flag = 0;
                    GPIOPinWrite(PS_PORT1,PS_RESET_PIN,0);
                    vTaskDelay(1/portTICK_PERIOD_MS);
                    GPIOPinWrite(PS_PORT1,PS_RESET_PIN,PS_RESET_PIN);                    
                    DAC_Init(); 
                    
                }

            if(OnOff_PS_Flag == 1 )
                {
                    OnOff_PS_Flag = 0;
                    if(PS_Status == PS_OFF){Switch_PS_ON();}
                    else {Switch_PS_OFF();}
                }
        }

        if(EncoderServiceReqFlag == 1)
        {
            switch (EncoderServiceReqType)
            {
            case EncoderCW:
                if(Iref_SetType == 0){ DAC_SetVal += DAC_FineSetVal; } 
                else{ DAC_SetVal += DAC_CoarseSetVal; }
                if(DAC_SetVal > DAC_MaxVal){DAC_SetVal = DAC_MaxVal;}
                DAC_Write(DAC_DACREG, DAC_SetVal);
                break;

            case EncoderCCW:
                if(DAC_SetVal != DAC_MinVal) //Make sure the counter doesnt go bellow 0x0 as it will result in negative value.
                   {
                       if(Iref_SetType == 0){ DAC_SetVal -= DAC_FineSetVal; }
                       else{ DAC_SetVal -= DAC_CoarseSetVal; }
                       if((DAC_SetVal <= 8191) | DAC_SetVal >= DAC_MaxVal){DAC_SetVal = DAC_MinVal;}
                   }
                DAC_Write(DAC_DACREG, DAC_SetVal);
                break;
            case EncoderMBtn:
                Iref_SetType = ~Iref_SetType;
                break;

            default:
                break;
            }

            EncoderServiceReqFlag = 0;
        }


        LED1_HIGH;
        vTaskDelay(50/portTICK_PERIOD_MS);
        LED1_LOW;
        //counter += 1;
        vTaskDelay(200/portTICK_PERIOD_MS);

    }
}

Please note the following defines in FreeRTOSConfig.h. Specially the Stack size and Heap size.

#define configTICK_RATE_HZ                  ( ( portTickType ) 1000 )
#define configMINIMAL_STACK_SIZE            ( ( unsigned short ) 500 )
#define configTOTAL_HEAP_SIZE               ( ( size_t ) ( 128*1024 ) )

#define INCLUDE_vTaskPrioritySet            1
#define INCLUDE_uxTaskPriorityGet           1
#define INCLUDE_vTaskDelete                 1
#define INCLUDE_vTaskCleanUpResources       0
#define INCLUDE_vTaskSuspend                1
#define INCLUDE_vTaskDelayUntil             1
#define INCLUDE_vTaskDelay                  1
#define INCLUDE_uxTaskGetStackHighWaterMark 1
#define INCLUDE_eTaskGetState                   1
#define INCLUDE_pcTaskGetTaskName               1
#define INCLUDE_xTaskGetSchedulerState          1

Please note above INCLUDE_vTaskCleanUpResources is set to 0.
I thank you all for your time and help.

2

well, the usual technique here is to keep incrementing counters in several pivot points of your task function and thus determine where in its loop it gets stuck.

I’m sure you checked on your usual suspects - stack overflow, oom etc? Which memory manager are you using?

3

Hi,

I am not very expert on FreeRTOS. Used it few times. Can you please point me out how to look for memory details. I am using Code Composer Studio from TI. it is based on Eclipse.

4

After you have done the usual debugging steps like defining configASSERT, stack overflow and malloc failure detection, you can try breaking the code in the debugger when it stops and examine the state.

5

Hi Aggarwal,

I have defined the configASSERT call back. However can you point me out some direction how can check Stack Overflow and malloc related problems ?
I have connected JTAG and waiting for the time to when the Thread Task1 stops execution.

6

I just break the code and via JTAG interface caught the place where the thread stuck.
So in Task1, I am reading an I2C chip using the function I2C_read(). However there is another similar function I2C0_read() and this function is not used anywhere in the project. But the JTAG reveals the Task1 thread is stuck at I2C0_read();

This is very strange how can the context switch and end up on this function.

Please see the picture bellow;

temp1
temp1884×460 21.1 KB

The equivalent Assembly version is shown in the following picture.
temp2

When I click “Step Over” in my debugging tool (Code Composer Studio IDE from TI). it will not move to the next instruction and will stay at 00000C50 as indicated in above picture.

Keep in mind I am still able to communicate with the device via Ethernet interface.

7

See https://freertos.org/FAQHelp.html

I2C0_read() is calling functions that look like they are in ROM - I assume you are using a TI part (not read the entire thread). You would need to look at the call stack to see how you got to I2C0_read() but assume either it is coming from ROM code which won’t show up in your software application or it is code specific to the I2C zero peripheral being called form the more generic I2C entry point code.

8

Stack overflow detection - FreeRTOS - stacks and stack overflow checking
Malloc failure detection - FreeRTOS - RTOS hook (callback) functions for task stack overflows, tick interrupts, idle task, daemon task startup, and malloc failure (pvPortMalloc() returning NULL)

May be a memory corruption. Try the above 2.

9

You might want to look at
GitHub - akobyl/TM4C129_FreeRTOS_Demo: Demo of FreeRTOS 10.2.1 for the Tiva TM4C1294 Connected Launchpad from Texas Instruments
for a working demo.

10

It is very possible that the compiler detects code similarities between the two read function and generates common code with the first function jumping into the code body of the second. Do not make generous assumptions about correspondences between source and assembly code, in particular not when optimization is in effect.