LwIP mqtt fail in freeRTOS after a while & large payloads cant be stored after a subscribe

 Hello. I have a Riverdi screen with STM32H7 microcontroller. I am using LwIP in order to have ethernet connection with static IP enabled. In freertos.c file in the default task where theMX_LWIP_Init(); is defined I am trying to connect with a broker via mqtt using LwIP mqtt library. 
  First and foremost, the most common issue that I am facing is the hardfault which is a bus problem. I figured it out that it has to do with the Rxpool address in linkerscript and after that it work properly for at least 20 minutes and then it crashes again. I opened debug mode and I realize that in Memory browser the program was trying to reach some addresses that exceeds the end address of the RAM_D2. I added a larger value to linker script and it allocates 80% of ram_d2 memory while in run mode. My linker script code, which handles the lwip section is this:
 .lwip_sec (NOLOAD) :
  {
   	. = ABSOLUTE(0x30000000);
   	*(.RxDecripSection)
   	
   	. = ABSOLUTE(0x30000200);
   	*(.TxDecripSection)
   	
 	. = ABSOLUTE(0x30035800);
   	*(.RxPool)
   } >RAM_D2
   Furthermore, another problem that I am facing is that when I am subscribing to a topic with a large payload 1251 length of data, then the rtos task deletes itself and crashes.  I should have mention that I have change the parameters in the mqtt_opts.h file where the incoming payload is handled and specifically here:
  /**
  * Number of bytes in receive buffer, must be at least the size of the longest incoming topic + 8
  * If one wants to avoid fragmented incoming publish, set length to max incoming topic length + max payload length + 8
  */
 #ifndef MQTT_VAR_HEADER_BUFFER_LEN
 #define MQTT_VAR_HEADER_BUFFER_LEN 1271 // This is the proper value for me, according to the comments above.
 #endif

Is there any other parameter that I should change in order to avoid this operation of the task?

I have tried to increase the task size it doesn’t seem to solve the problem.

osThreadId_t defaultTaskHandle;
 const osThreadAttr_t defaultTask_attributes = {
   .name = "defaultTask",
   .stack_size = 9000 * 4,
   .priority = (osPriority_t) osPriorityNormal,
 };

This is my task:

 void StartDefaultTask(void *argument)
 {
   /* init code for USB_HOST */
  MX_USB_HOST_Init();
 
   /* init code for LWIP */
   MX_LWIP_Init();
   /* USER CODE BEGIN StartDefaultTask */
 
   err_t err;
   ip_addr_t mqttIP;
   mqtt_client_t *client=mqtt_client_new(); 	// Initialize the new client instance.
   flag0.clear_flags = 0; 					// Clear all flags.
   struct mqtt_connect_client_info_t ci; 	// Create an information parameter about the client.
   memset(&ci, 0, sizeof(ci));			 	// Initialize the parameter for client information and set the all 0 to entire size of ci.
   /* Declare information about the client */
   ci.keep_alive = 60; 						// With this value to zero means that the client will never checks after the connection with the broker if they are still connected after a period of time.
   ci.client_id = "STM32H7-R"; 				// It is important to be unique. It doesn't matter if it is a random string but it must be unique as I said.
   if (client != NULL){
  	 err = dns_gethostbyname(BROKER_IP, &mqttIP, mqtt_resolved_cb, NULL); // Declare the server IP:192.168.10.26
 	 if (err == ERR_OK){
  		 mqtt_client_connect(client, &mqttIP, BROKER_PORT, mqtt_connection_cb, 0, &ci); 		// Connect with the server IP:192.168.10.26
 		 mqtt_set_inpub_callback(client, mqtt_incoming_publish_cb, mqtt_incoming_data_cb, &ci); // Define callback functions.
  	 }
   }
   for(;;){
 
    	 if (mqtt_client_is_connected(client)){
    		 // Client is connected
    		 if (status){ // If something must be send to the UI
    			osMessageQueuePut(QueueHandle, &action, 0, 0); // Send the data, which is representing the struct through the queue.
    			status=0; // Close the port="status"
   		 }
    	 }
       osDelay(1);
   }
 
   /* Infinite loop */
 
   /* USER CODE END StartDefaultTask */
 }

I am writing code with STM32CubeIDE 1.13.2 and the UI is generated automatically from
TOUCHGFX Designer 4.22.1. I would appreciate any comment or something that could be useful in order to solve the problem.

This implies that all of your Rx descriptor buffers only have 512 bytes taken together? Is that intended? That does not seem right…

1 Like

Just because I am new to embedded I have followed some steps about this in a topic that I had found in the ST community. So this was not intended to be like that it just worked so I left it like that. Could you please recommend to me something to do?

you need to determine the required size for your section and adjust the values in your linker command file accordingly. One way to do that is to generate a linker map file (typically pass the -m option to your linker), inspect the map file and determine what needs to go into the section declaration.

Please do NOT ask for further assistance until you have read up on what linker command files and memory section assignments are all about. This is absolutely mandatory and elementary knowledge. Without understanding how this works, you will get nowhere in the RTOS world. Just query the internet for “linker command file,” there are plenty of acceptable to good tutorials out there (in this forum, there are links to some). A lot of things will fall into place once you have understood how these work.

Edit: A quick search yielded the following as one of the first hits:

TI Linker Command File Primer

At first glance, it looks reasonable. You may want to start there. Best of luck!

It is likely an issue with you linker script as @RAc mentioned. Is it possible for you to share the complete code?

I appreciate that you want to help me and I would like to thank you in advance!
This is my freertos.c where I am using stack and queue(In order to communicate with the UI).

osThreadId_t defaultTaskHandle;
const osThreadAttr_t defaultTask_attributes = {
  .name = "defaultTask",
  .stack_size = 9000 * 4,
  .priority = (osPriority_t) osPriorityNormal,
};
void StartDefaultTask(void *argument)
{
  /* init code for USB_HOST */
  MX_USB_HOST_Init();

  /* init code for LWIP */
  MX_LWIP_Init();
  /* USER CODE BEGIN StartDefaultTask */

  err_t err;
  ip_addr_t mqttIP;
  struct mqtt_connect_client_info_t ci; 	// Create an information parameter about the client.
  flag0.clear_flags = 0; 					// Clear all flags.

  memset(&ci, 0, sizeof(ci));			 	// Initialize the parameter for client information and set the all 0 to entire size of ci.
  /* Declare information about the client */
  ci.client_id = "STM32H7-R"; 				// It is important to be unique. It doesn't matter if it is a random string but it must be unique as I said.
  client=mqtt_client_new(); 				// Initialize the new client instance.
  if (client != NULL){
	  ipaddr_aton("192.168.10.26",&mqttIP);
	  mqtt_client_connect(client, &mqttIP, BROKER_PORT, mqtt_connection_cb, 0, &ci); 		// Connect with the server IP:192.168.10.26
  }

  for(;;){

   	 if (mqtt_client_is_connected(client)){
   		 // Client is connected
   		 if (status){ // If something must be send to the UI
   			osMessageQueuePut(QueueHandle, &action, 0, 0); // Send the data, which is representing the struct through the queue.
   			status=0; // Close the port="status"
   		 }
   	 }
      osDelay(1);
  }

  /* Infinite loop */

  /* USER CODE END StartDefaultTask */
}
static void mqtt_incoming_data_cb(void *arg, const u8_t *data, u16_t len, u8_t flags)
{

	char buf[len+1]; 					// Declare a buffer where the messages will be stored
	strncpy(buf, (char *)data, len);     // Initialize buffer buf with the value of incoming data
	buf[len]='\0';						// Set a terminate character into the the end of buffer

	if (flags & MQTT_DATA_FLAG_LAST){
		// When the last byte received
		if (flag0.fl.f2){ // If the left state is open
			flag0.fl.f3=0;// Close the right state while you are open
			strncpy(action.left_state, buf, len);	// Copy the value to the proper path of the struct
			flag0.fl.f2 = 0;						// Close the state
		}
		if (flag0.fl.f3){ // If the right state is open
			flag0.fl.f2=0;// Close the left state while you are open
			strncpy(action.right_state, buf, len);	// Copy the value to the proper path of the struct
			flag0.fl.f3 = 0;						// Close the state
		}
		if (flag0.fl.f4){ // If the right energy is open
			flag0.fl.f5=0;

			//parseRightJSON(buf);						// Call the function in order to extract data
			flag0.fl.f4=0;							// Close the energy
		}
		if (flag0.fl.f5){ // If the left energy is open
			parseLeftJSON(buf);					// Call the function in order to extract data
			flag0.fl.f5=0;							// Close the energy
		}
		if (flag0.fl.f6){ // If right time is open
			flag0.fl.f7=0;
			setRightTime(buf, len, 2);				// Send the seconds to the proper function in order to create time
			flag0.fl.f6=0;							// Close the time
		}
		if (flag0.fl.f7){ // If the left time is open
			flag0.fl.f6=0;
			setLeftTime(buf, len, 1);				// Send the seconds to the proper function in order to create time
			flag0.fl.f7=0;							// Close the time
		}
	}

	status=1; // Inform the engine that you have something to send to UI, by opening the port="status"
}
static void mqtt_incoming_publish_cb(void *arg, const char *topic, u32_t tot_len)
{

	if (strcmp(topic, state_left)==0){
		flag0.fl.f2=1; // Store left state
	}else if (strcmp(topic, state_right)==0){
		flag0.fl.f3=1; // Store right state
	}else if (strcmp(topic, right_energy_topic)==0){
		flag0.fl.f4=1; // Store right energy data
	}else if (strcmp(topic, left_energy_topic)==0){
		flag0.fl.f5=1; // Store left energy data
	}else if (strcmp(topic, left_time)==0){
		flag0.fl.f7=1; // Store left charging time
	}else if (strcmp(topic, right_time)==0){
		flag0.fl.f6=1; // Store right charging time
	}

}

Furthermore, my linkerscript is like this:

/*
******************************************************************************
**
**  File        : LinkerScript.ld
**
**  Author      : STM32CubeIDE
**
**  Abstract    : Linker script for STM32H7 series
**                      1024Kbytes FLASH
**                       800Kbytes RAM
**
**                Set heap size, stack size and stack location according
**                to application requirements.
**
**                Set memory bank area and size if external memory is used.
**
**  Target      : STMicroelectronics STM32
**
**  Distribution: The file is distributed as is without any warranty
**                of any kind.
**
*****************************************************************************
** @attention
**
** Copyright (c) 2023 STMicroelectronics.
** All rights reserved.
**
** This software is licensed under terms that can be found in the LICENSE file
** in the root directory of this software component.
** If no LICENSE file comes with this software, it is provided AS-IS.
**
*****************************************************************************
*/

/* Entry Point */
ENTRY(Reset_Handler)

/* Highest address of the user mode stack */
_estack = ORIGIN(RAM_D1) + LENGTH(RAM_D1); /* end of "RAM_D1" Ram type memory */

_Min_Heap_Size = 0xc000 ; /* required amount of heap  */
_Min_Stack_Size = 0x8000; /* required amount of stack */

/* Memories definition */
MEMORY
{
  RAM_D1 (xrw)   : ORIGIN = 0x24000000, LENGTH =  512K
  FLASH   (rx)   : ORIGIN = 0x08000000, LENGTH = 1024K    /* Memory is divided. Actual start is 0x08000000 and actual length is 2048K */
  DTCMRAM (xrw)  : ORIGIN = 0x20000000, LENGTH = 128K
  RAM_D2 (xrw)   : ORIGIN = 0x30000000, LENGTH = 288K
  RAM_D3 (xrw)   : ORIGIN = 0x38000000, LENGTH = 64K
  ITCMRAM (xrw)  : ORIGIN = 0x00000000, LENGTH = 64K
  QUADSPI (r)	 : ORIGIN = 0x90000000,	LENGTH = 64M
  SDRAM   (xrw)  : ORIGIN = 0xD0000000,  LENGTH = 3600K
  SDRAM2  (xrw)  : ORIGIN = 0xD0384000,  LENGTH = 4592K
}

/* Sections */
SECTIONS
{
  /* The startup code into "FLASH" Rom type memory */
  .isr_vector :
  {
    . = ALIGN(4);
    KEEP(*(.isr_vector)) /* Startup code */
    . = ALIGN(4);
  } >FLASH

  /* The program code and other data into "FLASH" Rom type memory */
  .text :
  {
    . = ALIGN(4);
    *(.text)           /* .text sections (code) */
    *(.text*)          /* .text* sections (code) */
    *(.glue_7)         /* glue arm to thumb code */
    *(.glue_7t)        /* glue thumb to arm code */
    *(.eh_frame)

    KEEP (*(.init))
    KEEP (*(.fini))

    . = ALIGN(4);
    _etext = .;        /* define a global symbols at end of code */
  } >FLASH

  /* Constant data into "FLASH" Rom type memory */
  .rodata :
  {
    . = ALIGN(4);
    *(.rodata)         /* .rodata sections (constants, strings, etc.) */
    *(.rodata*)        /* .rodata* sections (constants, strings, etc.) */
    . = ALIGN(4);
  } >FLASH

  .ARM.extab   : { 
    . = ALIGN(4);
    *(.ARM.extab* .gnu.linkonce.armextab.*)
    . = ALIGN(4);
  } >FLASH
  
  .ARM : {
    . = ALIGN(4);
    __exidx_start = .;
    *(.ARM.exidx*)
    __exidx_end = .;
    . = ALIGN(4);
  } >FLASH

  .preinit_array :
  {
    . = ALIGN(4);
    PROVIDE_HIDDEN (__preinit_array_start = .);
    KEEP (*(.preinit_array*))
    PROVIDE_HIDDEN (__preinit_array_end = .);
    . = ALIGN(4);
  } >FLASH
  
  .init_array :
  {
    . = ALIGN(4);
    PROVIDE_HIDDEN (__init_array_start = .);
    KEEP (*(SORT(.init_array.*)))
    KEEP (*(.init_array*))
    PROVIDE_HIDDEN (__init_array_end = .);
    . = ALIGN(4);
  } >FLASH
  
  .fini_array :
  {
    . = ALIGN(4);
    PROVIDE_HIDDEN (__fini_array_start = .);
    KEEP (*(SORT(.fini_array.*)))
    KEEP (*(.fini_array*))
    PROVIDE_HIDDEN (__fini_array_end = .);
    . = ALIGN(4);
  } >FLASH

  /* Used by the startup to initialize data */
  _sidata = LOADADDR(.data);

  /* Initialized data sections into "RAM" Ram type memory */
  .data : 
  {
    . = ALIGN(4);
    _sdata = .;        /* create a global symbol at data start */
    *(.data)           /* .data sections */
    *(.data*)          /* .data* sections */
    *(.RamFunc)        /* .RamFunc sections */
    *(.RamFunc*)       /* .RamFunc* sections */

    . = ALIGN(4);
    _edata = .;        /* define a global symbol at data end */
  } >RAM_D1 AT> FLASH

  /* Uninitialized data section into "RAM" Ram type memory */
  . = ALIGN(4);
  .bss :
  {
    /* This is used by the startup in order to initialize the .bss section */
    _sbss = .;         /* define a global symbol at bss start */
    __bss_start__ = _sbss;
    *(.bss)
    *(.bss*)
    *(COMMON)

    . = ALIGN(4);
    _ebss = .;         /* define a global symbol at bss end */
    __bss_end__ = _ebss;
  } >RAM_D1

  /* User_heap_stack section, used to check that there is enough "RAM" Ram  type memory left */
  ._user_heap_stack :
  {
    . = ALIGN(8);
    PROVIDE ( end = . );
    PROVIDE ( _end = . );
    . = . + _Min_Heap_Size;
    . = . + _Min_Stack_Size;
    . = ALIGN(8);
  } >RAM_D1


  
  /* Remove information from the compiler libraries */
  /DISCARD/ : 
  {
    libc.a ( * )
    libm.a ( * )
    libgcc.a ( * )
  }

  .lwip_sec (NOLOAD) :
  {
  	. = ABSOLUTE(0x30000000);
  	*(.RxDecripSection)
  	
  	. = ABSOLUTE(0x30000200);
  	*(.TxDecripSection)
  	
	. = ABSOLUTE(0x30031100);
  	*(.RxPool)
  } >RAM_D2

  .ARM.attributes 0 : { *(.ARM.attributes) }
  
TouchGFX_Framebuffer (NOLOAD) :
 	{
		*(TouchGFX_Framebuffer TouchGFX_Framebuffer.*)
		*(.gnu.linkonce.r.*)
        . = ALIGN(0x4);
	} >SDRAM
	
BufferSection (NOLOAD) :
  {
    
    *(Video_RGB_Buffer Video_RGB_Buffer.*)
    *(.gnu.linkonce.r.*)
    . = ALIGN(0x4);
  } >SDRAM2
 


   
ExtFlashSection :
	{
		*(ExtFlashSection ExtFlashSection.*)
		*(.gnu.linkonce.r.*)
        . = ALIGN(0x4);
	} >QUADSPI
	
}

And also my MPU region configurations are like this:

void MPU_Config(void)
{
  MPU_Region_InitTypeDef MPU_InitStruct = {0};

  /* Disables the MPU */
  HAL_MPU_Disable();

  /** Initializes and configures the Region and the memory to be protected
  */
  MPU_InitStruct.Enable = MPU_REGION_ENABLE;
  MPU_InitStruct.Number = MPU_REGION_NUMBER0;
  MPU_InitStruct.BaseAddress = 0x30000000;
  MPU_InitStruct.Size = MPU_REGION_SIZE_1KB;
  MPU_InitStruct.SubRegionDisable = 0x0;
  MPU_InitStruct.TypeExtField = MPU_TEX_LEVEL0;
  MPU_InitStruct.AccessPermission = MPU_REGION_FULL_ACCESS;
  MPU_InitStruct.DisableExec = MPU_INSTRUCTION_ACCESS_ENABLE;
  MPU_InitStruct.IsShareable = MPU_ACCESS_NOT_SHAREABLE;
  MPU_InitStruct.IsCacheable = MPU_ACCESS_NOT_CACHEABLE;
  MPU_InitStruct.IsBufferable = MPU_ACCESS_BUFFERABLE;

  HAL_MPU_ConfigRegion(&MPU_InitStruct);

  /** Initializes and configures the Region and the memory to be protected
  */
  MPU_InitStruct.Number = MPU_REGION_NUMBER1;
  MPU_InitStruct.BaseAddress = 0x30004000;
  MPU_InitStruct.Size = MPU_REGION_SIZE_16KB;
  MPU_InitStruct.TypeExtField = MPU_TEX_LEVEL1;
  MPU_InitStruct.IsShareable = MPU_ACCESS_SHAREABLE;
  MPU_InitStruct.IsBufferable = MPU_ACCESS_NOT_BUFFERABLE;

  HAL_MPU_ConfigRegion(&MPU_InitStruct);
  /* Enables the MPU */
  HAL_MPU_Enable(MPU_PRIVILEGED_DEFAULT);

}

That is not sufficient. We also need to know what goes into the RxDescripSection which is Xilinx code.

Again, you should familiarize yourself with memory layout techniques first, you will need to know that either way, and in the end you will get answers faster if you can provide them yourself.

I managed to solve this problem. First of all I have changed the LwIP addresses in linker script like this:

.lwip_sec (NOLOAD) :
  {
   	. = ABSOLUTE(0x30000000);
   	*(.RxDecripSection)
   	
   	. = ABSOLUTE(0x30000200);
   	*(.TxDecripSection)
   	
 	. = ABSOLUTE(0x30000400);
   	*(.RxPool)
   } >RAM_D2

The DMA Descriptors starts at the address 0x30000000, and together they occupy 608 bytes, i.e. 0x200 + 4×24. The Heap size is set to 16KB and the Heap pointer is set to address 0x30004000. Also, I set the address of the LWIP_RAM_HEAP_POINTER into the MPU configuration with 64KB. I noticed that cubemx generates code with not so good TCP configurations so I changed the TCP_MSS to 1460, the TCP_SND_BUF to 5840 which is equal to 4*TCP_MSS and finally the TCP_SND_QUEUELEN to 16. Subsequently, I increase the size of my default task and it appears to be running successfully for 4 hours and it continues to run.

1 Like