Problem sending two bytes via Shadow document (Shadow demo)

Hi.

I’m designing a product based on the shadow example and the document that is sent from the aws iot object is the one in the example like this:

{
  "desired": {
    "powerOn": 1
  },
  "reported": {
    "powerOn": 1
  }
}

This works correctly as long as “powerOn” has a value from 0 to 9, but I need to send more values, up to the number 17, which implies two bytes, for example if I send the number 15, the first byte is 1 and the second is 5 .

I have modified the code in the aws_iot_demo_shadow.c file for this case. In the shadowDeltaCallback function to get the second byte I do the following:

deltaFound = _getDelta( pCallbackParam->u.callback.pDocument,
                            pCallbackParam->u.callback.documentLength,
                            "powerOn",
                            &pDelta,
                            &deltaLength );

    if( deltaFound == true )
    {
        if (deltaLength  > 1) //If two bytes arrive, the length is 2
        {
            iDelta = *pDelta - '0'; //In the example of 15, here would be 1
            varInt = *(pDelta + 1) - '0'; //here will be 5
            iDelta = 10*iDelta+varInt;//here the number 15 is created, 1 * 10 + 5
        }
        else
        {
            iDelta = *pDelta - '0'; //here comes a number from 0 to 9
        }
...
}

To generate the Shadow document and report its status I do the following:

if (deltaLength > 1)
        {
            updateDocument.u.update.pUpdateDocument = pUpdateDocument2;
            updateDocument.u.update.updateDocumentLength = EXPECTED_REPORTED_JSON_2_SIZE;
            updateDocumentLength = snprintf( pUpdateDocument2,
                                         EXPECTED_REPORTED_JSON_2_SIZE + 1,
                                         SHADOW_REPORTED_JSON_2,
                                         ( int ) currentState,
                                         ( long unsigned ) ( IotClock_GetTimeMs() % 1000000 ) );
            varInt = EXPECTED_REPORTED_JSON_2_SIZE;
        }
        else
        {
            updateDocument.u.update.pUpdateDocument = pUpdateDocument;
            updateDocument.u.update.updateDocumentLength = EXPECTED_REPORTED_JSON_SIZE;
            updateDocumentLength = snprintf( pUpdateDocument,
                                         EXPECTED_REPORTED_JSON_SIZE + 1,
                                         SHADOW_REPORTED_JSON,
                                         ( int ) currentState,
                                         ( long unsigned ) ( IotClock_GetTimeMs() % 1000000 ) );
            varInt = EXPECTED_REPORTED_JSON_SIZE;
        }

        if( ( size_t ) updateDocumentLength != varInt)//EXPECTED_REPORTED_JSON_SIZE )
        {
            IotLogError( "Failed to generate reported state document for Shadow update." );
        }
        else
        {
            /* Send the Shadow update. Its result is not checked, as the Shadow updated
             * callback will report if the Shadow was successfully updated. Because the
             * Shadow is constantly updated in this demo, the "Keep Subscriptions" flag
             * is passed to this function. */
            updateStatus = AwsIotShadow_Update( pCallbackParam->mqttConnection,
                                                &updateDocument,
                                                AWS_IOT_SHADOW_FLAG_KEEP_SUBSCRIPTIONS,
                                                NULL,
                                                NULL );

            if( updateStatus != AWS_IOT_SHADOW_STATUS_PENDING )
            {
                IotLogWarn( "%.*s failed to report new state.",
                            pCallbackParam->thingNameLength,
                            pCallbackParam->pThingName );
            }
            else
            {
                IotLogInfo( "%.*s sent new state report.",
                            pCallbackParam->thingNameLength,
                            pCallbackParam->pThingName );
            }
        }

I have two strings that represent the JSON one for 1 byte (SHADOW_REPORTED_JSON) and another for two bytes (SHADOW_REPORTED_JSON_2)

#define SHADOW_REPORTED_JSON    \
    "{"                         \
    "\"state\":{"               \
    "\"reported\":{"            \
    "\"powerOn\":%01d"          \
    "}"                         \
    "},"                        \
    "\"clientToken\":\"%06lu\"" \
    "}"

#define SHADOW_REPORTED_JSON_2  \
    "{"                         \
    "\"state\":{"               \
    "\"reported\":{"            \
    "\"powerOn\":%02d"          \
    "}"                         \
    "},"                        \
    "\"clientToken\":\"%06lu\"" \
    "}"

/**
 * @brief The expected size of #SHADOW_REPORTED_JSON.
 *
 * Because all the format specifiers in #SHADOW_REPORTED_JSON include a length,
 * its full size is known at compile-time.
 */
#define EXPECTED_REPORTED_JSON_SIZE    ( sizeof( SHADOW_REPORTED_JSON ) - 3 )
#define EXPECTED_REPORTED_JSON_2_SIZE    ( sizeof( SHADOW_REPORTED_JSON_2 ) - 3 )

All of this apparently works very well with the microcontroller, until it stops responding to commands from the aws iot object (or from a lambda function associated with an alexa ability) after a few seconds.

I decided to connect the RS232 console to see if there was a type of error, and the microcontroller only arrived to send the last message which is “29 3382 [IP-task] Socket 29603 → 34004522ip: 8883 State eCONNECT_SYN-> eESTABLISHED”

16 3298 [iot_thread] [INFO ][DEMO][3298] ---------STARTING DEMO---------

17 3298 [iot_thread] [INFO ][INIT][3298] SDK successfully initialized.
18 3298 [iot_thread] [INFO ][DEMO][3298] Successfully initialized the demo. Network type for the demo: 4
19 3298 [iot_thread] [INFO ][MQTT][3298] MQTT library successfully initialized.
20 3298 [iot_thread] [INFO ][Shadow][3298] Shadow library successfully initialized.
21 3298 [iot_thread] [INFO ][DEMO][3298] Shadow Thing Name is XXXXXXXXX (length 12).
22 3336 [iot_thread] prvProcessDNSCache: add: 'axxxxxx-xxx.iot.us-east-1.amazonaws.com' @ 34004522ip
23 3336 [iot_thread] DNS[0xF357]: The answer to 'axxxxxxxxxxx-xxx.iot.us-east-1.amazonaws.com' (34004522ip) will be stored
24 3336 [iot_thread] FreeRTOS_connect: 29603 to 34004522ip:8883
25 3336 [iot_thread] Socket 29603 -> 34004522ip:8883 State eCLOSED->eCONNECT_SYN
26 3336 [IP-task] prvSocketSetMSS: 1400 bytes for 34004522ip:8883
27 3336 [IP-task] Connect[34004522ip:8883]: next timeout 1: 3000 ms
28 3382 [IP-task] MSS change 1400 -> 1460
29 3382 [IP-task] Socket 29603 -> 34004522ip:8883 State eCONNECT_SYN->eESTABLISHED

After that there are no more messages, although the Shadow document is modified and the hardware activates and deactivates terminals according to “powerOn”, these changes are not displayed.

When the network cable is disconnected or connected, the respective messages do appear (“MAC link: OFF!” Or “MAC link: ON!”)

After a while, it no longer responds to changes in the shadow document and the only option is to reboot the hardware.

But if I undo all the changes above, it works very well, the hardware always reports what happens through the Rs232 console.

Any comments or suggestions are welcome

Hello @DominusDRR

Thanks for reaching out.

One quick problem I see is that the length you are calculating is incorrect for
EXPECTED_REPORTED_JSON_2_SIZE
The explanation for the length calculation is documented here:

It should be

#define EXPECTED_REPORTED_JSON_2_SIZE    ( sizeof( SHADOW_REPORTED_JSON_2 ) - 2 )

Note the change from -3 to -2 due to width changing from 1 char to 2 chars.

Also, for JSON document, you may just use one string with %02d. The size in this case is going to be :

#define EXPECTED_REPORTED_JSON_SIZE    ( sizeof( SHADOW_REPORTED_JSON ) - 2 )

You don’t have to define two strings. So if you just change %01d to %02d and change the size stated above it should able to accommodate numbers 0-99.
Please let us know if that works for you.

Side note: Based on your description and file name, it seems like you are using demo from amazon-freertos repository which almost 1.5 year old. Have you looked at our recent demos and updated libraries from the release in July 2021 ?

Hi.

Thank you very much for your suggestions.

I’m going to try to apply your suggestions, I hope to be successful.

Regarding the amazon-freertos version, what happens is that I’m starting from an example of the manufacturer of the microcontroller that I am using and they still have not updated, and to request technical support from them I must stick to what they have.

When the project is finished, I hope to migrate to the new version

Hi.

I have made the changes to the current project and there was no message on the RS232 console regarding aws, only the network status.

I applied the changes in a previous project and this message appears in the console. It seems to be fine

Both in the current project and in the one that if the messages appear in the console, it seems to work for changes with numbers greater than 9.

What I don’t understand is why the console doesn’t work. I have checked the old and new project settings and it is the same.

The only difference is that the current project has more processes, but they aren’t related to aws.

That is why in the original post, I indicated that it only went as far as displaying the message “29 3382 [IP-task] Socket 29603 → 34004522ip: 8883 State eCONNECT_SYN-> eESTABLISHED” and nothing else.

In these days I was adding other processes and now not even that last message appears, the only thing I see is this:

image

I also tried modifying the stack size, priority or buffer size with respect to the RS232 console but the problem is the same:

image

Hello @DominusDRR
Happy to see that the shadow is working for you.

Just making sure I understand it correctly:

  1. After making the changes to the original demo ( changing
    %01d to %02d" and changing EXPECTED_REPORTED_JSON_SIZE to ( sizeof( SHADOW_REPORTED_JSON ) - 2 ) every thing works as expected. (console and shadow update)

  2. You have have changed the project that adds some additional tasks. This new project does not work as expected. The console print does not happen, but shadow works ?

Is that the correct understanding ? Other than additional tasks what other differences are in the two projects.

Thanks.

Hi.

Thanks for answering.

Yes, they both work very well with changing the size of the JSON.

The difference of the last version of the project doesn’t show anything the console.

I have placed the file aws_iot_demo_shadow.c of the project that the console does work, in the new one to determine if any changes are the cause of the problem, but the console still doesn’t respond.

I also regenerated the code again with the additional tasks, but empty, that is, without the new processes, but the problem is the same.

What I see that works is the detection of the state of the network and that has been with another function of the microcontroller manufacturer:

PIC32_MAC_DbgPrint( "xNetworkInterfaceInitialise: %d %d\r\n", ( int ) xMacInitStatus, ( int ) xResult ); 

But, it’s like the IotLog function is disabled somewhere.

I didn’t detect this problem because until the old version of the project that does show messages in the console and it works fine.

From then on, as the hardware did respond to the changes in the Shadows document, I didn’t feel the need to use the console to see what happened.

This is why the JSON and its size that I reconnected the console to try to see if there was an important message and as I indicated in the original post, until now there was a last message, currently there is nothing regarding it related to aws.

Today I am going to try from the old project to create the (empty) tasks and add the code to determine how far the console works correctly.

I have found out what is causing the problem.

Initially I have three tasks for my own processes, independent from AWS as follows:

I have added two more like this:

image

But just adding one more, the console stops working.

Note that these tasks do not yet have code, only space is reserved and they are created empty. I haven’t written my code yet.

I imagine it is a problem with the size of the stack.

I have been manually decreasing the stack size of those two new tasks to determine when the console should work, but when I get to 32 bytes like the following image, not even the network status messages work (PIC32_MAC_DbgPrint)

I also see that the above tasks despite having been defined to a stack size, in the code are of less value. for example the appledLAN task was defined at 1024, but it is at 128:

I don’t know if it is an error in the tool that generates the code or if it is trying to preserve a fixed size with the total sum of the stack size of each task.

Did you define configASSERT, enabled stack overflow checking and maybe also configUSE_MALLOC_FAILED_HOOK (see Hook Functions doc) for development/debugging ?
Maybe you just run out of heap space by adding those tasks.

The manufacturer’s tool has generated the code as follows:

image

And is one of the hook functions invoked resp. the assert defined by configASSERT ?

It looks as if all your tasks run at the same priority. If the task you added is stuck in a CPU bound loop, it may lock out the other tasks. Do you have time slicing enabled?

1 Like

@hs2

I don’t understand your question, I will try to understand what hook functions are to answer your question

@RAc

Hi

I have seen that they are all created with priority 1, but when the code is added in the new two they work fine.

The first manages a buzzer as an alarm signal, and the other manages information in an EERAM memory. It works very well, since the first one is a TCPIP server and from a client I can access said memory and retrieve or save information.

While the RS232 console is still not working.

Most of time, the two new tasks are idle:

void APPEERAM_Tasks ( void )
{
    switch ( appeeramData.state )
    {
    ...
    case APPERRAM_STATUS_IDLE break;
    ...
    }
} 

And currently, the tasks are out of useful code, this is because I’m to verify the problem that I have mentioned:

void APPLEDAWS_Tasks ( void )
{

    /* Check the application's current state. */
    switch ( appledawsData.state )
    {
        /* Application's initial state. */
        case APPLEDAWS_STATE_INIT:
        {
            bool appInitialized = true;


            if (appInitialized)
            {

                appledawsData.state = APPLEDAWS_STATE_SERVICE_TASKS;
            }
            break;
        }

        case APPLEDAWS_STATE_SERVICE_TASKS:
        {

            break;
        }

        /* TODO: implement your application state machine.*/


        /* The default state should never be executed. */
        default:
        {
            /* TODO: Handle error in application's state machine. */
            break;
        }
    }
}

That’s not idle, but hogging CPU time infinitely (provided the case switch is within your infinite task loop). That would explain your problem…

1 Like

Hi:

I have put the two new tasks in priority 0 (which is supposed to be idle) and the console works correctly.

Conclusion I must investigate how to put a task to Idle mode or lower its priority to zero through code.

 xTaskCreate((TaskFunction_t) _APPBUZZER_Tasks,
                "APPBUZZER_Tasks",
                256,
                NULL,
                0,
                &xAPPBUZZER_Tasks);

    /* Create OS Thread for APPEERAM_Tasks. */
    xTaskCreate((TaskFunction_t) _APPEERAM_Tasks,
                "APPEERAM_Tasks",
                256,
                NULL,
                0,
                &xAPPEERAM_Tasks);
xTaskCreate((TaskFunction_t) _APPBUZZER_Tasks,
                "APPBUZZER_Tasks",
                256,
                NULL,
                0, <---- This parameter is setting the priority. 
                &xAPPBUZZER_Tasks);

See This page describes the RTOS xTaskCreate() FreeRTOS API function which is part of the RTOS task control API. FreeRTOS is a professional grade, small footprint, open source RTOS for microcontrollers..

Thanks.

1 Like

Going back to the original post, it seems to me that something still needs to be changed with respect to updating the Shadow from the microcontroller.

Seems like a problem in the _sendShadowUpdates function

You’re mixing up what is task priority 0 with the idle task, which runs at priority 0 (the lowest priority level). In order for another task to run, the currently running task needs to be in a suspended or blocked state. I recommend looking at the docs for task states.

I’m going to assume that APPLEDAWS_Tasks has a loop block in it so the function doesn’t return after the switch statement. Within there, you need to change the task state from running in order to give processor time to another task/thread.

1 Like

That’s time consuming and unreliable. Create your task with a greater task size than you know it needs, use uxTaskGetStackHighWaterMark.html to get the number of words remaining in that task for your platform, subtract the bytes remaining (convert from words) from the task stack depth (bytes), add a few extra bytes to this value to ensure you don’t accidentally end up with a overflow. Set that as your new stack depth for that task.

1 Like