Bitmap in Modular OTA example

Libraries
1

MQTT streams library in the OTA orchestrator example does not use the bitmap functionality when requesting multiple blocks from iot jobs and data blocks are received out of order.

Can you explain why that is done?

( void ) CBOR_Encode_GetStreamRequestMessage( ( uint8_t * ) getStreamRequest,
                                                          GET_STREAM_REQUEST_BUFFER_SIZE,
                                                          &requestLength,
                                                          "rdy",
                                                          fileId,
                                                          blockSize,
                                                          blockOffset,
                                                          /* coverity[misra_c_2012_rule_7_4_violation] */
                                                          ( const uint8_t * ) "MQ==",
                                                          strlen( "MQ==" ),
                                                          numberOfBlocksRequested );
2

The previous OTA library was capable of requesting multiple blocks at a time (see this code, this code, and this code). The ability to request multiple blocks at a time necessitates functionality to handle out of order blocks. This was taken care of by the bitmap. With this in mind, there really isn’t a need for a sequencing bitmap when you use the default configuration of the old OTA Agent as it requests a single block at a time.

As we moved towards modular OTA, the goal was to avoid placing restrictions on end user solutions. This includes both forcing complexity (through the use of the bitmap) OR prohibiting a user to request multiple blocks. The new OTA orchestrator example is designed to mimmic the behavior of our default configured old OTA agent library - so it requests a single block at a time - without the unnecessary complexity of the bitmap.

If you need the ability to request and receive multiple blocks, or any other feature for that matter, don’t hesitate to build off of the OTA orchestrator example and add functionality you need. Customizability and extensibility remains the most powerful attributes of the new modular OTA libraries.

3

If the otaconfigMAX_NUM_BLOCKS_REQUEST is set to 1 in the default config, it doesn’t imply that user should keep it at 1. It is a configurable parameter to request multiple blocks if the user wants to. Only if this parameter is not defined by the user, it is set to 1.

The previous ota wasn’t prohibiting the user to request multiple blocks, rather it was providing the functionality to request multiple blocks and handle them through bitmap if they arrive out of order. All of this is stripped off in the modular ota and offloaded to the user.

I fail to understand how the time for OTA is not taken into consideration in the modular ota. For instance, if download of one data block of 1K takes around 2 sec after the request, one file of around 5MB will take around 5242 blocks*2 sec per block ~ 174 minutes.

I also want to point out that there is a possibility to submit request multiple blocks by updating NUM_OF_BLOCKS_REQUESTED in the example but when one data block is received, another request is sent immediately, whereas the request should be sent after receiving all packets. Also if some of the packets are lost(or fail to arrive), subsequent requests are already been made without handling the missing packet. The bitmap was present to determine which packet was missed and request it again if needed.

The point being, if parameter is provided to request multiple blocks, there should also be handling for multiple packets. Or probably the documentation for the modular ota example can be improved suggesting that handling is provided only for 1 packet at a time

4

I fail to understand how the time for OTA is not taken into consideration in the modular ota.

The simple OTA orchestrator example exists to give a reference starting point. The point of modular OTA is that it is modular, extensible, and customizable to fit into your use case and constraints. If timing is critical, then you can easily modify the code to request multiple blocks to shorten the overall OTA download time.

I also want to point out that there is a possibility to submit request multiple blocks by updating NUM_OF_BLOCKS_REQUESTED in the example but when one data block is received, another request is sent immediately, whereas the request should be sent after receiving all packets. Also if some of the packets are lost(or fail to arrive), subsequent requests are already been made without handling the missing packet. The bitmap was present to determine which packet was missed and request it again if needed.

I agree with you that this was a massive miss in the implementation. I’ll look into implement the sequencing map today for this example. I would prefer to provide this multiple block functionality in the example.

5

This PR is a quick and dirty attempt at multi-block support. If you have optimizations suggestions or optimizations to add - please let me know on the PR :slight_smile:

6

I’ve gone ahead and addressed your comments and merged in the rough bitmap example changes. If you have any needed tweaks to improve performance (see below) we are always receptive of PRs :slight_smile:

1 Like