FreeRTOS Cellular Library - small memory leak

We have taken the Windows demo and ported it to the Qualcomm QCA4020 processor with a BG95M2. We have got both the Cellular portion and the encrypted Socket/MQTT working. However, we are noticing a small memory leak when running the AT commands. Commenting out the MQTT portion and just starting / stopping the cellular modem (threads and AT commands), we see a memory leak of 104 bytes per cycle on a successful registration and ~480 bytes on unsuccessful registration (aerial removed). The memory leak is reported by the QCA heap API.
We have spent countless hours debugging and matching up the malloc and free calls for the AT commands.
Has any else seen this with the library or a QCA 4020?

Hi Graham,

Thank you for your feedback.

Cellular interface also defines Platform_Malloc and Platform_Free in cellular_platform.h. You can hook your tracing information in these two macros for debugging memory problem.

Currently, malloc and free are mainly used to create AT command response and AT command line in the response. They will be freed after the command handler function is called.

Can you share your example which could lead to the memory leakage?

Hi Graham,

We do some test with the BG96 MQTT demo application and didn’t observe the memory leak.
The highwater mark is 2620 bytes. The allocated memory after Cellular_Cleanup is called is 0.

Below is how we setup the test.

Hook the Platform_Malloc and Platform_Free function with the following code snippet.

uint32_t mallocSize = 0;
uint32_t mallocHigh = 0;

/**
 * Allocate extra memory to keep the allocated memory size informatioin.
 *  -------------------------------------
 *  |  uint32_t size | allocated memory |
 *  -------------------------------------
 */
void * Platform_Malloc( size_t size )
{
    void * ptr = NULL;
    uint32_t * ptrU32 = NULL;

    ptrU32 = pvPortMalloc( size + sizeof( uint32_t ) );
    if( ptrU32 != NULL )
    {
        ptrU32[0] = size;
        ptr = &ptrU32[1];
        mallocSize = mallocSize + size;
        if( mallocSize > mallocHigh )
        {
            mallocHigh = mallocSize;
        }
    }
    return ptr;
}

/*-----------------------------------------------------------*/

/**
 * Free the memory also decrease total allocated memory.
 */
void Platform_Free( void * ptr )
{
    uint32_t * ptrU32 = NULL;

    if( ptr != NULL )
    {
        ptrU32 = ptr;
        ptrU32 = ptrU32 - 1;
        mallocSize = mallocSize - ptrU32[ 0 ];

        vPortFree( ptrU32 );
    }
}

/*-----------------------------------------------------------*/

Update the cellularDemoTask with the following code.

#include "cellular_config.h"
#include "cellular_config_defaults.h"
#include "cellular_types.h"
#include "cellular_api.h"

extern mallocSize;
extern mallocHigh;
extern CellularHandle_t CellularHandle;

static void CellularDemoTask( void * pvParameters )
{
    bool retCellular = true;

    ( void ) pvParameters;
    while( true )
    {
        /* Setup cellular. */
        retCellular = setupCellular();

        /* Stop here if we fail to initialize cellular. */
        configASSERT( retCellular == true );

        /* Run the MQTT demo. */
        LogInfo( ( "---------STARTING DEMO---------\r\n" ) );
        /* Update the MQTT demo to run only 3 times then return. */
        vStartSimpleMQTTDemo();

        Cellular_Cleanup( CellularHandle );
        LogInfo( ( "total malloc size %u high malloc %u\r\n", mallocSize, mallocHigh ) );
    }

    vTaskDelete( NULL );
}

I will check out the differences. In the meantime I was going to add a big zip file of our cellular library test code. However, as a new user I cannot upload files. Here is the list of what we tried. The pointers all seem to line up but there is a difference when we “print the heap status”

#ifdef CONFIG_CDB_PLATFORM
#ifdef DEBUG_CHECK_MEMORY_LEAKS

static uint32_t platformFreeCount = 0;
static uint32_t platformMallocCount = 0;

/**
 * Saved list of pointers
 */
#define NUMBER_OF_POINTERS_IN_LIST 2000
struct debugPointerEntry_t {
    char        type;
    uint32_t    tickCount;
    uint32_t    mfCount;
    size_t      sizeRequested;
    uint32_t    *address;
};
#define GET_TIME_MS (*((volatile uint32_t*)0X5074207C))
//uint32_t time_now = (GET_TIME_MS*31)/1000;

static uint32_t pointerListIdx = 0;
struct debugPointerEntry_t pointerTable[ NUMBER_OF_POINTERS_IN_LIST ];

void prvStoreDebugPointer( bool isMalloc, void *address, size_t sizeRequested );
void prvStoreDebugPointer( bool isMalloc, void *address, size_t sizeRequested )
{
    if( pointerListIdx >= (sizeof pointerTable / sizeof pointerTable[0] ) ) return;
    pointerTable[ pointerListIdx ].type         = ( isMalloc ) ? 'm' : 'f' ;
    pointerTable[ pointerListIdx ].mfCount      = ( isMalloc ) ? platformMallocCount: platformFreeCount;
    pointerTable[ pointerListIdx ].sizeRequested = sizeRequested;
    pointerTable[ pointerListIdx ].tickCount = (GET_TIME_MS*31)/1000;
    pointerTable[ pointerListIdx ].address = address;

    pointerListIdx++;
}


void printPointerTable( void );
void printPointerTable( void )
{
    // Printing in sequence
    for( uint32_t loopIdx = 0; loopIdx < pointerListIdx; loopIdx++ )
    {
        if( 'm' == pointerTable[ loopIdx ].type )
        {
            configPRINTF( ("\x1B[35mMALL-%p,size=%lu,tick=%lu\x1B[0m\r\n", pointerTable[ loopIdx ].address, pointerTable[ loopIdx ].sizeRequested, pointerTable[ loopIdx ].tickCount ) );
        }
        else
        {
            configPRINTF( ( "\x1B[45mFREE-%p,size=%lu,tick=%lu\x1B[0m\r\n", pointerTable[ loopIdx ].address, pointerTable[ loopIdx ].sizeRequested, pointerTable[ loopIdx ].tickCount ) );
        }

    }
}
#endif /* DEBUG_CHECK_MEMORY_LEAKS */
#endif /* CONFIG_CDB_PLATFORM */

/*-----------------------------------------------------------*/
/**
 * Platform dependent abstraction helper layer
 * With simple setup no extra abstraction is required, can rely on macro definition in header file
 */
#ifdef CONFIG_CDB_PLATFORM
#ifdef DEBUG_CHECK_MEMORY_LEAKS
    extern QCLI_Group_Handle_t qcli_bg95_demo_handle; /* Handle for BG95 demo Command Group. */
    // Implemented as a debug abstraction layer function
    void * Platform_Malloc( size_t xWantedSize )
    {
        void *retPtr = NULL;
        retPtr = malloc( xWantedSize );
        prvStoreDebugPointer( true, retPtr, xWantedSize );
//        CellularLogDebug( "\x1B[32m%s ptr=%p, size=%lu\x1B[0m", __func__, retPtr, xWantedSize );
//        configPRINTF( ("\x1B[32m%s ptr=%p, size=%lu\x1B[0m\r\n", __func__, retPtr, xWantedSize) );
//        QCLI_Printf( qcli_bg95_demo_handle, "\x1B[32m%s ptr=%p, size=%lu\x1B[0m", __func__, retPtr, xWantedSize );
        platformMallocCount++;
        return retPtr;
    }
    void Platform_Free( void * pv )
    {
//        CellularLogDebug( "\x1B[32m%s ptr=%p\x1B[0m", __func__, pv );
//        configPRINTF( ("\x1B[32m%s ptr=%p\x1B[0m\r\n", __func__, pv) );
//        QCLI_Printf( qcli_bg95_demo_handle, "\x1B[32m%s ptr=%p\x1B[0m", __func__, pv );
        prvStoreDebugPointer( false, pv, 0 );
        free( pv );
        platformFreeCount++;
    }
#else
    // deliberately empty as macro defines this case
#endif /* DEBUG_CHECK_MEMORY_LEAKS */
#else
    // deliberately empty as macro defines this case
#endif /* CONFIG_CDB_PLATFORM */

@ grahamjengineer - I bumped your status on the forum up a level - can you try attaching the file again.

Since you seem to use standard malloc/free are you sure they’re thread-safe ?
When using newlib as C-library it’s required to provide __malloc_lock/unlock hooks (if supported by your newlib version). See e.g. this pretty good post for further details.

I have attached the entire file(s) for reference to see if there was anything I missed.
cellular_platform.c (16.8 KB)
cellular_platform.h (7.1 KB)

Note I tried to emulate your debugging technique and check the first [0] U32 of the allocated pointer that I was about to “free” in order to get the memory size and this did not work. It was sometimes off by 1 or plain garbage numbers.

I will check out your suggestion.
Under the hood it uses a Qualcomm abstraction implementation of the FreeRTOS allocator. Qualcomm gives the compiler the ability to use ThreadX or FreeRTOS pre-compiled libraries and FreeRTOS is the one we use. The Qualcomm uses it’s own printf layer “QCLI_Printf” and this is statically allocated for transmission but who knows what else it is doing on assembly of the strings.

The first [0] U32 should not be changed in any case. It looks like there is memory corruption.

I also use your cellular_platform file for verification. The malloc/free logs matches well.

I would like to know more about the following questions

  1. We use the mqtt_mutual_auth_demo to simulate your environment. Is this the same demo that you are having problem with? If not, please provide more information about the application you found the memory leakage or share the log to us. Then we can better simulate the problem you encountered.
  2. Try to use a dedicated memory pool for cellular malloc/free to see if the memory corruption still happens.
    For example, FreeRTOS heap_2.c can be modified as simple memory allocator from dedicated pool.