AWS-FreeRTOS - TICC3220SF-LaunchXL FATAL ERROR: Sync loss detected issue

Partners & Sponsors AWS
1

Hi,

  1. I am using aws-freeRTOS demo example with TI-CC3220SF-LAUNCHXL.

  2. Application performs following tasks.
    a. Provisioning task
    b. Main Application task (performs capturing and publishing sensor data, OTA verification)
    c. Watchdog task (low priority task )
    d. Logging task (Low priority task for DEBUG Traces).

  3. While running the application most of the time in multiple places i am facing the following issue “[ERROR] - FATAL ERROR: Sync loss detected n”

  4. From TI-forum suggestion, I am posting the same issue here, Please find the reference link ti_forum_link

Please find the debug reference traces.
Place #1

22 1845 [pthread] [INFO ][DEMO][1845] [Provisioning task] detected device is CC3220SF
23 1847 [iot_thread] [INFO ][DEMO][1847] [DEBUG] semaphores waiting passed! 
24 1847 [iot_thread] [INFO ][DEMO][1847] [POWER MANAGER] SET TO BURST MODE. 
25 1847 [iot_thread] [INFO ][DEMO][1847] Successfully initialized the demo. Network type for the demo: 1
26 1848 [iot_thread] [INFO ][DEMO][1848] runSms(): watchdog registration done with handle=3
27 1932 [iot_thread] [INFO ][DEMO][1932] [ERROR] - FATAL ERROR: Sync loss detected n
28 1932 [iot_thread] [ERROR][DEMO][1932] FS - Couldn't get info on file. error status -2005
29 1949 [iot_thread] [ERROR][DEMO][1932] GetDeviceConfiguration(): ERROR: getting device configuration from file 

30 1949 [iot_thread] [ERROR][DEMO][1949] ERROR--getting device configuration from file
31 1949 [iot_thread] [INFO ][DEMO][1949] [POWER MANAGER] SET TO BURST MODE. 
32 1977 [iot_thread] [INFO ][DEMO][1977] [RSESET] Restart immediately

Place #2

[2020-07-28 08:38:54.794] 333 30899 [iot_thread] [INFO ][MQTT][30899] (MQTT connection 20009168) MQTT PUBLISH operation queued.
[2020-07-28 08:38:54.803] 335 30904 [iot_thread] [INFO ][MQTT][30904] (MQTT connection 20009168) MQTT PUBLISH operation queued.
[2020-07-28 08:38:54.814] 339 31036 [iot_thread] [INFO ][DEMO][31036] [ERROR] - FATAL ERROR: Sync loss detected n
[2020-07-28 08:38:54.822] 
[2020-07-28 08:38:54.822] 341 31037 [NetRecv] [INFO ][DEMO][31037] 
[2020-07-28 08:38:54.826]  [SECURE-SOCKET-ERR] ERROR: -2005 Socket receive failed.
[2020-07-28 08:38:54.830] 
[2020-07-28 08:38:54.831] 342 31037 [NetRecv] ERROR: -2005 Socket receive failed.

Place #3

[2020-07-29 01:26:27.739] 160 20944 [OTA Task] [INFO ][MQTT][20944] (MQTT connection 20009088) UNSUBSCRIBE operation scheduled.
[2020-07-29 01:26:27.746] 161 20944 [OTA Task] [prvUnSubscribeFromJobNotificationTopic] OK: $aws/things/00000474M/jobs/$next/get/accepted
[2020-07-29 01:26:27.758] 162 20945 [OTA Task] [INFO ][MQTT][20944] (MQTT connection 20009088, UNSUBSCRIBE operation 2000b560) Waiting for operation completion.
[2020-07-29 01:26:27.925] 163 21129 [OTA Task] [INFO ][MQTT][21129] (MQTT connection 20009088, UNSUBSCRIBE operation 2000b560) Wait complete with result SUCCESS.
[2020-07-29 01:26:27.936] 164 21130 [OTA Task] [INFO ][MQTT][21130] (MQTT connection 20009088, UNSUBSCRIBE operation 2000c2b8) Waiting for operation completion.
[2020-07-29 01:26:27.948] 165 21137 [OTA Task] [INFO ][MQTT][21137] (MQTT connection 20009088, UNSUBSCRIBE operation 2000c2b8) Wait complete with result SUCCESS.
[2020-07-29 01:26:27.960] 166 21137 [OTA Task] [prvOTA_Close] Context->0x200000a0
[2020-07-29 01:26:27.964] 167 21137 [OTA Task] [INFO ][DEMO][21137] [Watchdog] unregister handle=4
[2020-07-29 01:26:27.971] 168 21138 [iot_thread] CheckforOtaUpdate() EXIT 
[2020-07-29 01:26:27.976] 169 21138 [iot_thread] [INFO ][DEMO][21138] OtaUpdateSchedule(): ENTRY
[2020-07-29 01:26:27.983] 170 21190 [iot_thread] [INFO ][DEMO][21190] [ERROR] - FATAL ERROR: Sync loss detected n
[2020-07-29 01:26:27.992] 
[2020-07-29 01:26:27.992] 171 21191 [NetRecv] [INFO ][DEMO][21191] 
[2020-07-29 01:26:27.995]  [SECURE-SOCKET-ERR] ERROR: -2005 Socket receive failed.
[2020-07-29 01:26:28.001] 
[2020-07-29 01:26:28.003] 172 21191 [NetRecv] ERROR: -2005 Socket receive failed.
2

Hi, Suresh.
I see this error message in ti_code/network_if.c ( https://github.com/aws/amazon-freertos/blob/f33863740032c59307b1849629c736a416ea2687/vendors/ti/boards/cc3220_launchpad/aws_demos/application_code/ti_code/network_if.c#L203 ). It might be useful to set a breakpoint and get a backtrace. Before going to that effort, do you have another TI dev board you can try so as to rule out a hardware issue?

3

Hi gooddan,
Thanks for your reply.

  1. I tried with multiple boards the problem is same. If hardware issue what might be the cause it will generate this kind of issue?

  2. As this is the random issue, i am encountering during long runs. For every iteration device goes to hibernate so long run with the debug mode to capture backtrace is not possible. However i will try removing hibernate and see.

4

Hi Suresh,

Have you been able to produce a backtrace by removing hibernate? That will help us greatly in finding a solution to the problem.