AWS IoT Device SDK C: Task Pool

Hi,

I recently came across IoT task pool library provided by FreeRTOS. Is this library no longer supported? I don’t find any recent documentation about this.
If yes, is there a way to use this?

To make them easier to distribute and integrate (as git sub-modules, CMSIS packs, through package managers, etc.), our libraries evolved to remove dependencies on anything other than the standard C library (the FreeRTOS+TCP stack being a notable exception), hence the task pool isn’t required any more. You can of course still use it if it benefits your application design - outside of our library integrations.

Currently, I have added the required header and source files in my /include and /src folders respectively. Can it be used this way? Is this the way when you say outside of your library integrations?
The building guide for the C SDK does say that we have to run CMake to build the library. I am using it for a firmware application for a MCU. So I am assuming that I won’t have to run the CMake build.

Yes.

You need to use the same build system that you are using to build the firmware for your MCU. If you are using an IDE, then you do not need CMake.

Perfect! Thank you for the quick response, Gaurav!

Also, my requirement is to design a task (thread) pool which will service the events in my firmware. I can see that the IoT task pool library implementation provides a dispatch queue from which the worker threads can pick up the events and service them. Can I use this same queue throughout my firmware? My use case is such that certain events will be triggered from anywhere in the firmware and currently I am just adding them to an RTOS Queue. Then I serve them on a first-come-first-serve basis. What I really want to achieve is to use the task pool library and create a fixed number of worker threads (for eg: 3). These threads will then keep a track of events in the firmware and then service them when they come. Also, the task pool will also be responsible for serving other tasks in the application. Any feedback on this design logic?

Is this not working for you? What is your goal for adding more than one worker task? You can use a FreeRTOS queue for posting all the event and have a FreeRTOS task process those events. What kind of processing is done in response to these events?

I want to use the task pool worker threads to service any event. For example, lets say I have 3 worker threads. Currently I have an event in the queue. I will first check for a free thread. Let’s say that thread 2 is free. I will assign the event to thread 2. It will service it and then return itself back to the pool once it is done. So this way I can ensure that no matter the number of events in the queue, any task can pick it up, service and return.
I have currently added all the sequential functionalities in the form of a routine. So there are certain routines which can occur from anywhere in the firmware. I will have to make different queues for the events then and this will take up more memory.

To add to the above point, I can definitely implement this without task pool library but this is also a POC for future firmware where the hardware will be updated. That is when I will really need to use task pool library.

Why do you want to explicitly assign events to the task and not let them pick themselves? Take a look at the concept section of this page - Safer Interrupt Handling Demo for NXP LPCXpresso55S69 Development Board - FreeRTOS

Would something similar not work for you?

This would not work for me as I have used co-operative scheduling. All the tasks have same priorities.
Lets say that I were to use the task pool library. Then do I need other RTOS Queues or the dispatch queue provided by task pool would be enough?

Got it. Thanks.

You do not need any other queue.

1 Like

Thank you for the response. This would work for me. Will post here if anything else comes up.

Just one more thing: when I build my project after adding header and source files, it throws “undefined reference to [function name]” error. This is a linking issue. Do I need to compile the task pool library as a static or dynamic library and then add it to the project?

Hello @hrm2519,

Do I need to compile the task pool library as a static or dynamic library and then add it to the project?

I think it depends on your requirements.
Regardless of whichever methods you use, the library should compile well. If you are not able to fix the issue, can you please let us know what the error is and we’ll be happy to help.

Thanks,
Aniruddha

Most likely the source file for the library is not getting compiled. You can verify that by writing some non-compileable code in that file.

build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x82): In function `.LFB57':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x8e): In function `.LFB57':
: undefined reference to `_Iot_CreateDetachedThread'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x9a): In function `.L26':
: undefined reference to `_IotSemaphore_Wait'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xb8): In function `.L16':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xca): In function `.LBE206':
: undefined reference to `_IotSemaphore_Post'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xec): In function `.LBE209':
: undefined reference to `_IotSemaphore_Post'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xfe): In function `.L15':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x12a): In function `.L25':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x136): In function `.L25':
: undefined reference to `_Iot_CreateDetachedThread'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x15c): In function `.LFB48':
: undefined reference to `_IotClock_TimerDestroy'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x162): In function `.LFB48':
: undefined reference to `_IotSemaphore_Destroy'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x168): In function `.LFB48':
: undefined reference to `_IotSemaphore_Destroy'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x16e): In function `.LFB48':
: undefined reference to `_IotMutex_Destroy'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x1c6): In function `.LBE232':
: undefined reference to `_IotSemaphore_Create'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x1f0): In function `.L51':
: undefined reference to `_IotSemaphore_Post'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x204): In function `.LBE239':
: undefined reference to `_IotSemaphore_Wait'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x22c): In function `.L70':
: undefined reference to `_IotMutex_Create'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x238): In function `.L39':
: undefined reference to `_IotSemaphore_Destroy'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x240): In function `.L42':
: undefined reference to `_IotMutex_Destroy'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x24c): In function `.L43':
: undefined reference to `_IotSemaphore_Destroy'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x276): In function `.L66':
: undefined reference to `_IotSemaphore_Create'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x286): In function `.L66':
: undefined reference to `_IotClock_TimerCreate'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x2a6): In function `.L57':
: undefined reference to `_Iot_CreateDetachedThread'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x2d6): In function `.L48':
: undefined reference to `_IotSemaphore_Wait'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x2f4): In function `.L69':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x310): In function `.LFB65':
: undefined reference to `_IotClock_GetTimeMs'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x356): In function `.L73':
: undefined reference to `_IotClock_TimerArm'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x372): In function `.L79':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x3a4): In function `.LFB59':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x3da): In function `.L84':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x4b6): In function `.LBB271':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x4d8): In function `.LFB63':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x4e0): In function `.LFB63':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x4fe): In function `.LBE281':
: undefined reference to `_IotClock_GetTimeMs'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x52a): In function `.LBE281':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x54c): In function `.L125':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x552): In function `.L125':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x566): In function `.L134':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x56c): In function `.L131':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x592): In function `.LFB49':
: undefined reference to `_IotSemaphore_Post'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x5a6): In function `.L152':
: undefined reference to `_IotSemaphore_TimedWait'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x5ac): In function `.L152':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x5e8): In function `.LBE314':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x5fa): In function `.LBE314':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x628): In function `.LBE316':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x634): In function `.LBE316':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x64a): In function `.L161':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x652): In function `.L161':
: undefined reference to `_IotSemaphore_TimedWait'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x658): In function `.L161':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x670): In function `.L158':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x680): In function `.L158':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x684): In function `.L158':
: undefined reference to `_IotSemaphore_Post'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x6a0): In function `.L144':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x6c0): In function `.L162':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x6d4): In function `.L162':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x6e6): In function `.L159':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x6fc): In function `.L159':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x7be): In function `_IotTaskPool_Destroy':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x7ee): In function `.LBB401':
: undefined reference to `_IotClock_GetTimeMs'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x860): In function `.L208':
: undefined reference to `_IotSemaphore_Post'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x872): In function `.LBE375':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x87c): In function `.L209':
: undefined reference to `_IotSemaphore_Wait'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x8c6): In function `.L230':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x8de): In function `.L207':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x918): In function `.L249':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x934): In function `.LBE432':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x944): In function `.L247':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x964): In function `.L248':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x972): In function `.L239':
: undefined reference to `_IotSemaphore_Post'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x97c): In function `.L239':
: undefined reference to `_IotSemaphore_Post'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x9d0): In function `.LBB448':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0x9de): In function `.LBB448':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xa10): In function `.LBE454':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xa3c): In function `.L272':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xa5a): In function `.L273':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xa5e): In function `.L273':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xa6c): In function `.L273':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xa94): In function `_IotTaskPool_DestroyRecyclableJob':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xaa4): In function `_IotTaskPool_DestroyRecyclableJob':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xac0): In function `.L287':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xae0): In function `.L286':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xae8): In function `.L286':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xb04): In function `_IotTaskPool_RecycleJob':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xb14): In function `.L294':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xb5a): In function `.LBE471':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xb6a): In function `.L292':
: undefined reference to `_IotLog_Generic'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xb72): In function `.L292':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xb7c): In function `.L295':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xbaa): In function `.L313':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xbba): In function `.L304':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xbde): In function `.L311':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xc06): In function `_IotTaskPool_ScheduleDeferred':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xc14): In function `_IotTaskPool_ScheduleDeferred':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xc46): In function `.L318':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xc68): In function `.LBB479':
: undefined reference to `_IotClock_GetTimeMs'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xcc4): In function `.LBE479':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xcd8): In function `.L336':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xd10): In function `.L339':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xd52): In function `_IotTaskPool_GetStatus':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xd60): In function `_IotTaskPool_GetStatus':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xd70): In function `.L350':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xda0): In function `.L354':
: undefined reference to `_IotMutex_Lock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xdae): In function `.L354':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool.o(.text+0xdc8): In function `.L359':
: undefined reference to `_IotMutex_Unlock'
build/default/production/_ext/1360937237/iot_taskpool_static_memory.o(.text+0x14): In function `.L7':
: undefined reference to `_IotStaticMemory_FindFree'
build/default/production/_ext/1360937237/iot_taskpool_static_memory.o(.text+0x32): In function `_IotTaskPool_FreeTaskPool':
: undefined reference to `_IotStaticMemory_ReturnInUse'
build/default/production/_ext/1360937237/iot_taskpool_static_memory.o(.text+0x42): In function `.L18':
: undefined reference to `_IotStaticMemory_FindFree'
build/default/production/_ext/1360937237/iot_taskpool_static_memory.o(.text+0x62): In function `_IotTaskPool_FreeJob':
: undefined reference to `_IotStaticMemory_ReturnInUse'
build/default/production/_ext/1360937237/iot_taskpool_static_memory.o(.text+0x72): In function `.L29':
: undefined reference to `_IotStaticMemory_FindFree'
build/default/production/_ext/1360937237/iot_taskpool_static_memory.o(.text+0x92): In function `_IotTaskPool_FreeTimerEvent':
: undefined reference to `_IotStaticMemory_ReturnInUse'

I get the above errors when I add the .h and .c files to my project. I am using MPLAB X IDE v5.20. I was able to resolve these errors by checking the option “Remove unused sections” under linker options. I know this is not a Microchip forum so I am not sure whether I would get any feedback from here. Just pasting it here since you asked.

Is there any example implementation for task pool library?

All these are dependency of task pool libraries which are just wrappers around FreeRTOS primitives. So you will need to bring in all these dependencies as well. I’d again suggest to see if you could just use FreeRTOS primitives.

I do not think we have a stand alone example for this library but you can take a look at its use in v4_beta_deprecated branch in C SDK repo. Here is one such example - aws-iot-device-sdk-embedded-C/iot_mqtt_operation.c at v4_beta_deprecated · aws/aws-iot-device-sdk-embedded-C · GitHub

I have added all the required dependencies which are nothing but the header files. Somehow the IDE still throws these errors which is strange.
I have added the following header files in my project:

  • iot_clock.h
  • iot_config.h
  • iot_error.h
  • iot_linear_containers.h
  • iot_logging.h
  • iot_logging_setup.h
  • iot_platforms_types.h
  • iot_platforms_types_freertos.h
  • iot_static_memory.h
  • iot_taskpool.h
  • iot_taskpool_types.h
  • iot_threads.h

The following source files are added to my project:

  • iot_taskpool.c
  • iot_taskpool_static_memory.c

Are you suggesting that I replace these task pool wrappers with FreeRTOS APIs? If yes, can you give one example based on the errors shown above?