keepAliveIntervalSec is set but never used because MQTT_ProcessLoop is never called in coreMQTT library

Hi,
We recently updated our codebase from a very early version of the AWS IoT library to the LTS version. As such, we are using the MQTT shim V2.3.1 and FreeRTOS Shadow V2.2.3.

It appears we are now having problems with our LWT. It seems to errantly fire. Investigating, it appears “keepAliveSeconds” is set to “keepAliveIntervalSec”, but never really used. This is because there is nothing invoking a call to “handleKeepAlive()” because “MQTT_ProcessLoop” is never called.

Is this true? The LWT is a pretty ‘core’ part of the mqtt spec, does coreMQTT not support it? If so, would you suggest us going back to the prior AWS library which supports this until the new library has production-level support for MQTT?

There are several different ways to use the coreMQTT library. These rang from using it as a ‘raw’ base library, in which case the application writer manages use of the protocol, to wrapping the library in an agent (or daemon) task, in which case the agent task manages the protocol. Although the examples are still a work in progress you can find the MQTT agent examples in this standalone repository: https://github.com/FreeRTOS/Lab-Project-coreMQTT-Agent - not this sub-modules in the MQTT agent and base libraries so you need to checked using

--recurse-submodules

You will find MQTT_ProcessLoop() used in the demo project there. You can also search the FreeRTOS/FreeRTOS-Plus directory of the main FreeRTOS download for MQTT_ProcessLoop() to see other use case examples.

Thanks Richard.

Is there any update on when MQTT agent will be released? We are a little tentative to pull this in to our production codebase if it is likely to change in the future.

Thanks,

Ben

While we don’t expect any significant changes the code is currently going through various testing, quality checklists and apsec review, which may result in minor changes to things like removing compilers warnings etc.

Note the demo in the repo I linked to in my prior post is building with Visual Studio and as such has a few Windows dependencies in things like the logging implementation. There is a GCC build not yet merged in (https://github.com/RichardBarry/Lab-Project-coreMQTT-Agent/tree/add-gcc-qemu-makefile-build) that will require changes in the application build rather than the library code (for example, moving the Windows dependencies in from the /source directory to the /build/MSVC directory, just as the GCC/QEMU dependencies are in the GCC build directory already).

Finally - those repos contain build projects for convenience of evaluation and as a reference. The MQTT agent code is actually in its own repo for easy submoduling into application projects - https://github.com/FreeRTOS/coreMQTT-Agent. The MQTT agent code brings in the coreMQTT library too so provided you do a recursive clone you get both the agent and the coreMQTT library.

Hello

As pointed out by Richard, coreMQTT is designed for lightweight applications. I would like to clarify though that the coreMQTT library does supports LWT. Your application will have to call ProcessLoop, as the library it self is ‘stateless’ and does not have any threads.

If you choose to use coreMQTT instead of the coreMQTT-Agent you will have to call MQTT_ProcessLoop at certain interval. The timeouts document talks about how various timeout should be handled. ProcessLoop is used for receiving any incoming messages including all Publish messages; so you will have to call the function to receive LWT message sent by the broker. coreMQTT library also support MQTT_Ping function for sending keep-alive messages if you don’t want to use processLoop. But not using processLoop means not able receive anything.

If you are transitioning from MQTT shim, I would recommend using the Agent as Richard has pointed out as the coreMQTT-Agent provides thread safety.

Either way, I would strongly recommend using the new library rather than going back to the old library. If you have any questions, please feel free ask on this forum or github.
Thanks!

Hi guys. Thanks for the additional info.

It seems like the shim (amazon-freertos/libraries/c_sdk/standard/mqtt at master · aws/amazon-freertos · GitHub) actually should handle LWT? Do you know where the documentation lives for this compatibility layer?

Hi @schoeler,

Could you clarify what you mean by LWT? The MQTT Last Will and Testament is sent by the client with the connect packet; in coreMQTT, this is passed in the call to MQTT_Connect. If the client does not send any packets for 1.5x the keep alive interval, then the broker will broadcast the client’s LWT on the LWT topic. If you’re inquiring where the MQTT shim sends pings to the broker, you can find that here in _IotMqtt_ProcessKeepAlive, the job that is created when forming a new connection with IotMqtt_Connect. The job both sends ping requests and checks for ping responses, and reschedules itself - for the keep alive interval if expecting to send a ping request next, or a short wait time if expecting to receive a ping response.

In regards to documentation for the compatibility layer, the README in the directory you have linked serves as the documentation for the shim itself. For the APIs the shim covers, the README links to the documentation for the old MQTT library.