coreMQTT: Unexpected packet type from server: PacketType=7b

Hello!,

The coreMQTT library is throwing an error with the message: “Unexpected packet type from server: PacketType=7b”. This indicates that the client has received a packet from the server that it does not recognize or expect in the current state. According to the MQTT specification, “7b” does not correspond to any standard packet type, indicating a potential issue with packet corruption or incorrect handling.

I have ensured that the packet types being sent and received align with expected MQTT communication patterns. Based on AWS Documentation

Here are the logs from my device which is ESP32 on framework ESP-IDF v4.4.8 (Commit a9d0f221) combined with coreMQTT (LTS 202406.01 Commit 4f11b8ca). Documentation says on step GetStream request 1. Repeat steps 3 and 4 to receive all data blocks. You must repeat these steps if the amount of data requested is larger than 128 KB. You must program your device to use multiple GetStream requests to receive all of the data requested. However, these logs indicate that the stream can start from a different position than requested in the payload. For instance, the requested offset was 0 [“o”:0], but the received offset was 1 [“i”:1]. When the request was made again for offset 0 [“o”:0], the response was with offset 2 [“i”:2]. The logic requires the incoming packets to arrive in order, but this issue only appears in this example. The longest error-free communication I managed, with valid packets in order, was 18 out of the 847 packets expected.

@edit also response clientToken [“c”] is always as the first payload send, never changes.


D (12:45:07.311) _MQTT[PUBSUB]: [627][_mqtt_subscribe] Subscription to topic [$aws/things/<serialNumber>/jobs/get/accepted]:[38] with QoS: [0]

D (12:45:07.485) _MQTT[PUBSUB]: [627][_mqtt_subscribe] Subscription to topic [$aws/things/<serialNumber>/jobs/get/rejected]:[38] with QoS: [0]

I (12:45:07.749) _MQTT[PUBSUB]: [684][_mqtt_publish] Publishing to topic :: [$aws/things/<serialNumber>/jobs/get][29]. payload :: {"clientToken":"HNezYuG6EVzNjJ6PAQZOxdGSC3NIw7f"}][49].

I (12:45:07.937) _MQTT[PUBSUB]: [400][_mqtt_handle_deserialized_info] Message from Topic: [$aws/things/<serialNumber>/jobs/get/accepted][38] Payload [246] {"clientToken":"HNezYuG6EVzNjJ6PAQZOxdGSC3NIw7f","timestamp":1721990707,"inProgressJobs":[{"jobId":"<AFR_OTA>","queuedAt":1721303991,"lastUpdatedAt":1721376545,"startedAt":1721304351,"executionNumber":1,"versionNumber":8}],"queuedJobs":[]}

D (12:45:08.091) _MQTT[PUBSUB]: [627][_mqtt_subscribe] Subscription to topic [$aws/things/<serialNumber>/jobs/<AFR_OTA>/get/accepted]:[55] with QoS: [0]

D (12:45:08.345) _MQTT[PUBSUB]: [627][_mqtt_subscribe] Subscription to topic [$aws/things/<serialNumber>/jobs/<AFR_OTA>/get/rejected]:[55] with QoS: [0]

I (12:45:08.608) _MQTT[PUBSUB]: [684][_mqtt_publish] Publishing to topic :: [$aws/things/<serialNumber>/jobs/<AFR_OTA>/get][46]. payload :: [{"jobId":"<AFR_OTA>","thingName":"<serialNumber>","includeJobDocument":true,"clientToken":"B4aXiEgw0ddWcQtdvbZ02Q7JonKxeQm"}][125].

I (12:45:08.803) _MQTT[PUBSUB]: [400][_mqtt_handle_deserialized_info] Message from Topic: [$aws/things/<serialNumber>/jobs/<AFR_OTA>/get/accepted][55] Payload [559] {"clientToken":"B4aXiEgw0ddWcQtdvbZ02Q7JonKxeQm","timestamp":1721990708,"execution":{"jobId":"<AFR_OTA>","status":"IN_PROGRESS","queuedAt":1721303991,"startedAt":1721304351,"lastUpdatedAt":1721376545,"versionNumber":8,"executionNumber":1,"jobDocument":{"afr_ota":{"protocols":["MQTT"],"streamname":"<AFR_OTA_STREAMNAME>","files":[{"filepath":"/","filesize":3466990,"fileid":0,"certfile":"<KEY>","sig-sha256-ecdsa":"<SIG>"}]}}}}

D (12:45:09.149) _MQTT[PUBSUB]: [627][_mqtt_subscribe] Subscription to topic [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/rejected/json]:[87] with QoS: [0]

D (12:45:09.368) _MQTT[PUBSUB]: [627][_mqtt_subscribe] Subscription to topic [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/description/json]:[90] with QoS: [0]

D (12:45:09.593) _MQTT[PUBSUB]: [627][_mqtt_subscribe] Subscription to topic [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/data/json]:[83] with QoS: [0]

I (12:45:09.827) _MQTT[PUBSUB]: [684][_mqtt_publish] Publishing to topic :: [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/describe/json][87]. payload :: [{"c":"zIwS0VcrhdRNf5jkQPGm9p9qnQs97oX"}][39].

I (12:45:10.040) _MQTT[PUBSUB]: [400][_mqtt_handle_deserialized_info] Message from Topic: [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/description/json][90] Payload [107] {"c":"zIwS0VcrhdRNf5jkQPGm9p9qnQs97oX","s":1,"d":"Stream for OTAUpdate","r":[{"f":0,"z":3466990}]}

I (12:45:10.162) _PART[firmware]: Starting OTA Update

I (12:45:10.181) _PART[firmware]: Running partition type [0] subtype [16] (offset 0x[00010000])

I (12:45:10.191) _PART[firmware]: Writing to partition subtype [17] at offset 0x310000

I (12:45:10.809) _MQTT[JOBS]: [150][_mqtt_jobs_stream_start] File ID: [0]

I (12:45:10.815) _MQTT[JOBS]: [151][_mqtt_jobs_stream_start] File size: [3466990]

I (12:45:10.826) _MQTT[JOBS]: [165][_mqtt_jobs_stream_start] Number of packets of [4]KB to download: [847]

I (12:45:10.835) _MQTT[JOBS]: [166][_mqtt_jobs_stream_start] Size of the last packet: [1774]

I (12:45:10.954) _MQTT[PUBSUB]: [684][_mqtt_publish] Publishing to topic :: [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/get/json][82]. payload :: [{"c":"EMDV25ZNqewSnKgur3ATKR5Un6jTfz3","s":1,"f":0,"l":4096,"o":0}][66].

I (12:45:11.226) _MQTT[PUBSUB]: [400][_mqtt_handle_deserialized_info] Message from Topic: [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/data/json][83] Payload [5531]

{"c":"EMDV25ZNqewSnKgur3ATKR5Un6jTfz3","f":0,"i":1,"l":4096,"p":"<PART_OF_THE_FILE>"}

I (12:45:11.928) _MQTT[PUBSUB]: [684][_mqtt_publish] Publishing to topic :: [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/get/json][82]. payload :: [{"c":"BmC3wKLdhkmZmgNnsrKy1DooD03SJHW","s":1,"f":0,"l":4096,"o":0}][66].

I (12:45:12.145) _MQTT[PUBSUB]: [400][_mqtt_handle_deserialized_info] Message from Topic: [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/data/json][83] Payload [5531]

{"c":"EMDV25ZNqewSnKgur3ATKR5Un6jTfz3","f":0,"i":2,"l":4096,"p":"<PART_OF_THE_FILE>"}

E (12:45:12.659) _MQTT_JSON[CALLBACK]: [442][json_aws_stream_data_accepted] clientToken invalid [BmC3wKLdhkmZmgNnsrKy1DooD03SJHW] :: [EMDV25ZNqewSnKgur3ATKR5Un6jTfz3]

I (12:45:12.857) _MQTT[PUBSUB]: [684][_mqtt_publish] Publishing to topic :: [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/get/json][82]. payload :: [{"c":"Djam59mo56QXEKwYF0XGfYUbkBtu25e","s":1,"f":0,"l":4096,"o":0}][66].

E (12:45:12.902) coreMQTT: Unexpected packet type from server: PacketType=7b.

I (12:45:13.014) _MQTT[PUBSUB]: [684][_mqtt_publish] Publishing to topic :: [$aws/things/<serialNumber>/streams/<AFR_OTA_STREAMNAME>/get/json][82]. payload :: [{"c":"eIXW0dJE5alCR8JmjzEPJv8ixPmWreu","s":1,"f":0,"l":4096,"o":0}][66].

E (12:45:13.065) coreMQTT: Unexpected packet type from server: PacketType=7b

<infinite loop>

Hi, thanks for reaching out and sharing the logs.
I’ll try to reproduce this.

1 Like

@Cero Are you using a newly designed OTA application? Or is it based on the ESP32 reference integration?: GitHub - FreeRTOS/iot-reference-esp32c3

I tried to reproduce the issue with the same version of coreMQTT in your post:

.
.
.


I (1195173) ota_over_mqtt_demo: Data block request sent.

I (1195813) coreMQTT: De-serialized incoming PUBLISH packet: DeserializerResult=MQTTSuccess.
I (1195813) coreMQTT: State record updated. New state=MQTTPublishDone.
I (1195823) ota_over_mqtt_demo: Received File Block event Received.

I (1195833) ota_over_mqtt_demo: Downloaded block 260 of 262. 

I (1195843) ota_over_mqtt_demo: Request File Block event Received.

I (1199843) ota_over_mqtt_demo: Sent PUBLISH packet to broker $aws/things/ExampleThing/streams/AFR_OTA-8703d1e4-40e9-45e1-9c23-49722bff7f75/get/json to broker.


I (1199853) ota_over_mqtt_demo: Data block request sent.

I (1200523) coreMQTT: De-serialized incoming PUBLISH packet: DeserializerResult=MQTTSuccess.
I (1200523) coreMQTT: State record updated. New state=MQTTPublishDone.
I (1200523) ota_over_mqtt_demo: Received File Block event Received.

I (1200533) ota_over_mqtt_demo: Downloaded block 261 of 262. 

I (1200543) ota_over_mqtt_demo: Free OTA buffers 2
I (1200553) ota_over_mqtt_demo: Close file event Received 

I (1200833) AWS_OTA: Signature verification succeeded.
I (1200833) ota_over_mqtt_demo: Activate Image event Received 

I (1200843) esp_image: segment 0: paddr=001b0020 vaddr=3c0c0020 size=35f00h (220928) map
I (1200873) esp_image: segment 1: paddr=001e5f28 vaddr=3fc8f800 size=03114h ( 12564) 
I (1200873) esp_image: segment 2: paddr=001e9044 vaddr=40380000 size=06fd4h ( 28628) 
I (1200883) esp_image: segment 3: paddr=001f0020 vaddr=42000020 size=bd0f4h (774388) map
I (1200983) esp_image: segment 4: paddr=002ad11c vaddr=40386fd4 size=0875ch ( 34652) 
I (1200993) esp_image: segment 0: paddr=001b0020 vaddr=3c0c0020 size=35f00h (220928) map
I (1201023) esp_image: segment 1: paddr=001e5f28 vaddr=3fc8f800 size=03114h ( 12564) 
I (1201023) esp_image: segment 2: paddr=001e9044 vaddr=40380000 size=06fd4h ( 28628) 
I (1201033) esp_image: segment 3: paddr=001f0020 vaddr=42000020 size=bd0f4h (774388) map

As you can see, the OTA succeeds and didn’t result in a similar error.

1 Like

Thank you for your response!

My implementation was based solely on the coreMQTT and coreHTTP submodules without using the ESP-specific libraries. The issue arose when using the coreMQTT module to handle streaming solely over the MQTT protocol. However, after switching to the esp-mqtt library included in ESP-IDF 4.4.8, I am unable to reproduce the same error—everything seems to be functioning as expected.

Additionally, when I switched to using the HTTP protocol for file retrieval instead of MQTT, the problem with handling the event order disappeared entirely.