If those changes still worked and you could post them here, I would be REALLY grateful =)
Unfortunately the way I had changed the project does not work with the new directory structure / build system in 201906.00, and I’ve already spent a day trying to get to work.
Even the 201906.00 release notes just say “we changed the build system and you will have to adapt your project” without giving any guidance, except for porting to new hardware.
It is understandable that Amazon can not give guidance on how to adapt old projects: there was no standardised way of integrating an application before, they can’t tell you what to change now.
Still, that such a big (and Amazon-backed) project has no way to let developers cleanly add an external project without modifying the RTOS build system (and therefore losing the changes at each git pull) is puzzling…
Sorry to response this late and causing so much frustration. In last release, most of our effort were spent to get CMake build working on as many vendors and boards as possible. We believe this is the very first step that enables us to do many things including an easier way to adopt our code in customer application.
It is true that we still haven’t established a standard way of integrating an application with amazon freertos outside of its source tree using CMake, as we’re still evaluating various approaches. Generally speaking, in CMake you can do it in 2 ways:
Add the external library (in your case, it’s amazon freertos) as a subfolder in your project, and use CMake add_subdirectory command to consume it. This is what I suggested before. The downside is that you have to compile external code every time.
Compile the external project and “install” its header files and build artifacts to a directory. Then in your CMake project, use the find_package command to load this dependency. This way you don’t have to bring the external source code into your application, and you’re just using header files and linking to static libraries. However, this needs to be supported by the external project as CMake does not magically provide this.
As of today, option #2 is not implemented in amazon freertos yet, which leaves you only option #1. Assuming you have this directory structure,
- amazon-freertos
- main.c
- CMakeLists.txt
Here’s a small CMake file you can try to start with,
cmake_minimum_required(VERSION 3.13)
project(my_app)
# AFR_BOARD tells which board we need to target.
set(AFR_BOARD espressif.esp32_devkitc CACHE INTERNAL "")
add_subdirectory(amazon-freertos)
add_executable(my_app main.c)
target_link_libraries(my_app PRIVATE AFR::mqtt)
To configure your project, run this command from your application’s directory (make sure the ESP32 compiler is in environment variable PATH),
Thank you so much Tiangang,
this is exactly the kind of information that I would expect to find in the AFR documentation!
Your example works perfectly.
I tried to extend it by adapting my old (pre-201906.00_Major) make files to work in 201906.00_Major.
I have a problem in bringing in components (e.g. the ESP-DSP componenent by Espressif) that are meant to be integrated with ESP-IDF.
It seems the include directories defined by the components I import are not brought in.
cmake_minimum_required(VERSION 3.13)
# AFR_BOARD tells which board we need to target.
set(AFR_BOARD espressif.esp32_devkitc CACHE INTERNAL "")
add_subdirectory(amazon-freertos)
file(GLOB SOURCES "app/common/application_code/*.c")
file(GLOB HEADERS "app/common/application_code/*.h")
# define the components
#get_filename_component(ESPRESSINF_CODE_COMPONENT "${CMAKE_CURRENT_LIST_DIR}/amazon-freertos/vendors/espressif/boards/esp32/aws_demos/application_code/espressif_code" ABSOLUTE)
get_filename_component(ESP_DSP_COMPONENT "${CMAKE_CURRENT_LIST_DIR}/app/common/application_code/components/esp-dsp" ABSOLUTE)
get_filename_component(I2C_BUS_COMPONENT "${CMAKE_CURRENT_LIST_DIR}/app/common/application_code/components/i2c_devices/i2c_bus" ABSOLUTE)
get_filename_component(HDC2010_COMPONENT "${CMAKE_CURRENT_LIST_DIR}/app/common/application_code/components/i2c_devices/sensor/hdc2010" ABSOLUTE)
set(EXTRA_COMPONENT_DIRS ${EXTRA_COMPONENT_DIRS}
${ESPRESSIF_CODE_COMPONENT}
${ESP_DSP_COMPONENT}
${I2C_BUS_COMPONENT}
${HDC2010_COMPONENT}
)
## I have to comment this inclusion, otherwise cmake will not configure AFR, complaining about duplicated components
# get_filename_component(ABS_IDF_PATH "${CMAKE_CURRENT_LIST_DIR}/amazon-freertos/vendors/espressif/esp-idf" ABSOLUTE)
# set(ENV{IDF_PATH} ${ABS_IDF_PATH})
#include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(main_app)
message("++++++++++++++++++++++++++++++++++")
message("++++ MAIN APP ++++")
message("++++++++++++++++++++++++++++++++++")
message("COMPONENT_DIRS: ${COMPONENT_DIRS}")
# Bring the headers manually into the project <==== This should not be needed, if the components above were brought in correctly
set(INCLUDE_DIRS ${INCLUDE_DIRS}
"app/common/config_files/"
"app/common/application_code/components/esp-dsp/modules/common/include"
"app/common/application_code/components/esp-dsp/modules/dotprod/include"
"app/common/application_code/components/esp-dsp/modules/fft/include"
"app/common/application_code/components/esp-dsp/modules/iir/include"
"app/common/application_code/components/esp-dsp/modules/fir/include"
"app/common/application_code/components/esp-dsp/modules/math/include"
"app/common/application_code/components/esp-dsp/modules/windows/hann/include"
"app/common/application_code/components/esp-dsp/modules/support/include"
"app/common/application_code/components/esp-dsp/modules/matrix/include"
"app/common/application_code/components/i2c_devices/i2c_bus/include"
"app/common/application_code/components/i2c_devices/sensor/hdc2010/include"
"amazon-freertos/vendors/espressif/esp-idf/components/esp_adc_cal/include"
"amazon-freertos/demos/dev_mode_key_provisioning/include"
"amazon-freertos/libraries/abstractions/pkcs11/include"
"amazon-freertos/libraries/3rdparty/pkcs11"
"amazon-freertos/libraries/freertos_plus/standard/utils/include"
"amazon-freertos/libraries/abstractions/wifi/include"
)
message("INCLUDE_DIRS: ${INCLUDE_DIRS}")
include_directories(${INCLUDE_DIRS})
add_executable(main_app ${SOURCES} ${HEADERS})
target_link_libraries(main_app PRIVATE AFR::mqtt)
Looks like the problem is related to adding extra components in ESP-IDF. I do see they reference the EXTRA_COMPONENT_DIRS in vendors/espressif/esp-idf/tools/cmake/project.cmake, but I’m not sure how it is supposed to be used. I’ll reach out to Espressif to see if they have any instruction of doing this in CMake.
And you don’t have to define IDF_PATH, this is already handled inside Amazon FreeRTOS, you can find it in vendors/espressif/boards/esp32/CMakeLists.txt.
AFAIK, on a “pure ESP-IDF” build, EXTRA_COMPONENT_DIRS is appended to COMPONENT_DIRS, which is where the ESP-IDF system looks for components to include in the build.
In turn, each component’s CMake files should bring in the include paths for the header that that component exposes for consumption.
My understanding is that for that mechanism to work, I need to include vendors/espressif/esp-idf/tools/cmake/project.cmake in my CMakeLists.txt.
However, if I do, the AFR part of the build breaks and I get these errors:[https://gist.github.com/matteoscordino/69124ce3f446cd10ea9defaf38849511]
What I learned from Espressif is that IDF_EXTRA_COMPONENT_DIRS was overwritten in their CMake file and this PR fixed this problem. They also requires you to set IDF_PROJECT_EXECUTABLE to your application (defined by add_executable) so that IDF can be linked to it.
Please let us know if the code snippet from this PR works for you.
However, I haven't found a solution using menuconfig and sdkconfig which lives in project-root/ rather than in build/. Of course I need menuconfig to configure the various ESP components of my project, and it is possible to tell menuconfig to save the sdkconfig at project-root/. It is important that sdkconfig does not live in build/ because build/ is often cleaned out and I don't want to re-configure my project each time. I saw the Getting Started suggestion for relocating sdkconfig.defults, but what about sdkconfig itself? The problem is: how to tell the CMake build system to look for, and use, sdkconfig in project-root/ rather than in build/?
The reason that sdkconfig is in build folder is because it's generated from sdkconfig.default by ESP-IDF, and it's not meant to be stored together with source code. When modifying the config with menuconfig, it will touch the generated one in build folder, this configuration is suppose to be temporary. So if your intention is to change some configuration forever and not temporarily, what prevents you from putting your configuration in sdkconfig.default?
It turned out I had another bug that appeared to make sdkconfig.default not work properly. Thanks to your clear and specific explanation I re-focused away from sdkconfig, found my bug, and now working
I was using example console source from ESP-IDF 4+ which used eg CONFIG_ESP_CONSOLE_UART_BAUDRATE, while CONFIG_CONSOLE_UART_BAUDRATE was defined in sdkconfig – my eyes tricked me by completely overlooking the ESP so I mistakenly thought sdkconfig wasn't working. ESP-IDF 4+ includes the renaming function for sdkconfig variables that makes those two equivalent for backward compatibility, but of course renaming isn't present in AFR-ESP32 (IDF 3).
Is there a timetable for migrating AFR-ESP32 to the latest ESP32 IDF?
I don't know about the date but I can raise the issue to Espressif so they're aware of. In the meantime, is it possible for you to use the 3.x version for now? You can also open an issue on their GitHub repo and ask if there's an workaround.
