How to modify projects to run on other boards - Microchip AWS

mrpackethead wrote on April 09, 2018:

Hello,

I’m wondering what i need to do, to be able to modify the Code examples, so i can get them running on other hardware. ‘real’ projects are not going to be built on the curoisity boards.

I want to to be able to use the PIC32MZ2048EFM064 for example, with a LAN8742 Phy driver. I can easily do this with Harmony based projects and add/remove thigns as i like.

I’m at a loss to know how to modify these projects without breaking them to reflect different hardware.

Can someone suggest how this shoudl be approached.

Alexa-AWS wrote on April 10, 2018:

Hi,

An overview of the modules that need to be ported for different hardware can be found in our porting guide here: https://docs.aws.amazon.com/freertos/latest/userguide/porting-guide.html

Unfortunately, the Amazon FreeRTOS’s MPLAB X project is not in the folder structure expected by the Harmony Configurator, but these are a few files/folders to take note of while doing the port:

• The Microchip Curiosity example project uses the PIC32 LAN8720A PHY daughter board, and relevant port-specific code can be found in the NetworkInterface file (lib/FreeRTOS-Plus-TCP/source/portable/NetworkInterface/NetworkInterface_eth.c).
• If a Harmony project with the driver of interest is created, Harmony will generate system_xxx.h/.c files. The initializations, parameters, and routines in these functions can be compared to the files under the /demos/microchip/curiosity_pic32mzef/common/application_code/microchip_code folder.
• Any missing Microchip supplied driver files will need to be pulled into the project under /libs/third_party/mcu_vendor/microchip/harmony/v2.04.

Best,
Alexa

mrpackethead wrote on April 11, 2018:

Im sorry this really has’tn helped me udnerstand what i need to do. Previous upgrades of FreeRTOS where really simple. I was able to go from 9.x to 10.x inside of harmony simply by chaing the source folder for RTOS.

Having to move away from the harmony structure is a massive nightmare as there is already a huge investment in exisiting code that i’m loath to have to convert…

Why did AWS FreeRTOS move away from the way that FreeRTOS has been done for so long?

This has made the migration of this code to other Microchip based boards very very difficult.

Edited by: mrpackethead on Apr 10, 2018 8:09 PM

mrpackethead wrote on April 12, 2018:

Hi,

What i’d like to be able to do is use the MQQT and OTA librarys particulary, ( the others woudl be helpful eventually but not immediately ).

We have several exisiting products that use PIC32MZ, are Harmony based with FreeRTOS ( vs 9.) These work well. I have tested these under harmony 2_05_01, and the FreeRTOS manually updated to the vs10 kernel. It all worked pretty much out of the box. Because we have a huge investment in the harmony based projects, i’m apphrensive to move to a new code base completely. We have a lot of TCP/IP things happening, so moving away from teh Microchip IP stack is also a challenge.

Ideally, i’d like to be able to take the AWS FreeRTOS kernel and introduce it into the harmony project, like i did with version 10. I’ve already tried, but there are enough differneces that things did’tn work so well. A simple example was that in the FreeRTOS10 source, the include folder is inside the source folder, where as in AWS freeRTOS, its in a different location. I went through and changed the locations of many of the files, but there are some differences ( croutines.c for example ) and could’nt see how to get that to work.

A baby step may be to run AWS FreeRTOS on another demo board, such as the PIC32MZ EF Ethernet Starter Kit… or the IOT ethernet board. Whats most important is knowing how you coudl go about migrating it from one particular board to another. Obviously, real projects dont’ run on development/eval boards.

So, to directly answer your question.

(1) If we could just bring the librarys in a Harmony project that has a FreeRTOS 10x kernel, that might be a very simple way of doign thigns… Some guidence on how to do that would be really good.

(2) Knowing how to port the exisiting project to other Boards would be helpful, but less helpful than (1) as staying in a harmony environment is a big deal

Thanks for you help. I might just be missing the obvious on how to make this work.

Alexa-AWS wrote on April 11, 2018:

To enable us to assist further, can you tell us if you are trying to either:

  1. Add Amazon FreeRTOS libraries into an existing Harmony project that is already using the FreeRTOS kernel, and if so, which Amazon FreeRTOS libraries do you want to bring into your project
  2. Run the PIC32 project from the Amazon FreeRTOS distribution on a board other than the Curiosity board.
  3. Update the FreeRTOS kernel of an existing Harmony project

Thanks

Alexa-AWS wrote on April 13, 2018:

Hi,

Thanks for the additional information.

Your feedback about Amazon FreeRTOS not integrating well with Harmony is noted.

I know that explaining changes doesn’t make them easier to work with :), but for context here are some comments on the differences you’ve pointed out

  • The include folder was moved out of the Source folder because we wanted to reduce the number of include paths that needed to be used by each project. As you’ve noted, includes must be copied from lib/include to FreeRTOS/Source/include
  • The croutines file is deprecated, but if you are still using this file, you can copy it over. Note that the contents of the file are the same between v9 and v10 except for the header comments.

As for porting the other Amazon FreeRTOS libraries, it would be best to get MQTT running solid before moving to OTA. MQTT is built on the secure sockets layer, which wraps the TCP/IP and TLS layers.

Since you have an IP stack that you would like to keep, the next step would be to map it’s functionality into the sockets API so that it matches the header provided by lib/include/aws_secure_sockets.h. The MQTT library calls into these API. The example project using the Microchip Curiosity board uses the FreeRTOS+TCP library to call into a portable NetworkInterface_eth.h. This layer calls into the Microchip harmony drivers (lib/third-party/mcu_vendor/microchip/harmony) which need to be switched out for the hardware that you are using.

I realize this is still quite high level, so feel free to create another thread as you come across specific questions and issues in the migration. I am sure many are going through this process and we would like to help make porting as smooth as possible (while acknowledging that every port/hardware setup/pre-existing project is a little bit different!).

Best,
Alexa

mrpackethead wrote on April 15, 2018:

“Since you have an IP stack that you would like to keep, the next step would be to map it’s functionality into the sockets API so that it matches the header provided by lib/include/aws_secure_sockets.h. The MQTT library calls into these API.”

It appears that all the calls to the secure sockets are made from aws_mqtt_agent.c Are there any calls made from elsewhere.

This is a balancing act, of which way is the best way to proceed.

Edited by: mrpackethead on Apr 15, 2018 5:07 PM

mrpackethead wrote on April 16, 2018:

I have spent the day looking at this in some more depth. Swapping to a non FreeRTOS stack is a going to be a bigger task than first anticipated and it runs into some licenceing issues that i want to avoid with WolfSSL.

So, the question now is, how best to approach chaning the project so it will run with an arbitory set of hardware.

mrpackethead wrote on April 16, 2018:

I posed this question in the FreeRTOS Forum, and Richard Barry provided a great response.

https://sourceforge.net/p/freertos/discussion/382005/thread/f457c889/

+"The MQTT implementation includes an MQTT client library suitable for use+
+in bare metal applications, and an MQTT agent for FreeRTOS. So you+
+could integrate just the library, but it will be simpler to create an+
+application if you could also integrate the agent.+

+The agent (think daemon task) manages the MQTT library on behalf of the+
+developer, providing the developer with a simple connect, publish, and+
+subscribe interface, and unburdening the developer from having to do+
+anything special to manage the MQTT protocol itself - for example there+
+is no need to periodically call the MQTT library, the agent does that+
+transparently in the background for you.+

+As you point out, the agent has a couple of dependencies. Although the+
+provided examples use the FreeRTOS+TCP stack, it does so through an+
+abstraction layer (called the ‘secure sockets’ layer, as it also+
+abstracts away TLS), so you could provide a port of that abstraction+
+layer to the Harmony TCP/IP stack if you wanted to continue to use that+
+stack.+

+So you could update your project to use the FreeRTOS+TCP stack and the+
+provided secure sockets abstraction layer for that stack, or you could+
+update your project by providing your own secure sockets abstraction for+
+the Harmony stack. Alternatively you could use just the MQTT library+
+component. Let me know which way you think is best."+

In my way of thinking the best way forward ( at least for me ), would be to use Harmony and FreeRTOS 10.x ( which works nicely ). This gives me the best of all the worlds. the ‘cost’ to me is having to port the secure sockets layer. This does seem like the best way to move forward.

Now to start looking at that secure soeckets layer in some depth.

mrpackethead wrote on April 18, 2018:

Progress is being made; albeit a bit slowly.

  • I’ve decided to go down this path as I to be sensible, i really need encryption ( TLS ) of the messages. there is a MQTT library ( WolfMQQT ) that is supplied with Harmony, but the moment that you use the Wolf Librarys you are into ‘interesting’ licencing territory that is largely incompatible with what I need/want to do.

Amazon uses mbedTLS, and that does not have the same license issues.

Now work starts on porting the aws_secure_sockets to use the sockets that are used in Harmony.

In order to make this an easy to use reference project, I am making modifications the the demo project that is supplied with Harmony 2.05.01 ( apps/rtos/freertos/tcpip_client_server_). Work to date is at https://github.com/mrpackethead/harmony_aws_freertos_mqtt

Please NOTE this is NOT yet working. there is still work to be done!

My project ends up at C:\microchip\harmony\v2_05_01\dev\rtos\freertos\harmony_aws_freertos_mqtt

In order to make this work a few things need to be installed;

(1) The AWS Librarys. I have installed them at
C:\microchip\harmony\v2_05_01\third_party\amazon-freertos
The easiest way to get this is to clone it from Github, which has the most recent versions;
https://github.com/aws/amazon-freertos

Note that these librarys also include FreeRTOS, but it is the AWS Version which has been modified, and it is easier to use the generic FreeRTOS10.x

(2) FreeRTOS 10.x;
Harmony 2.05.01 ships with FreeRTOS9.x Fortunately upgrading it to Version 10.x is very simple.
Download FreeRTOS from Source Forge ( https://sourceforge.net/projects/freertos/ ). Copy the Folder ‘FreeRTOS’ when its expanded out ( FreeRTOSv10.0.1\FreeRTOS ) and use it to replace the FreeRTOS Folder that was installed with Harmony ( In my case it was C:\microchip\harmony\v2_05_01\third_party\rtos\FreeRTOS )

freeRToes wrote on April 19, 2018:

So porting libraries is fairly straight forward. But what kind of TCP or any other configurations would be required for Harmony?

mrpackethead wrote on April 19, 2018:

I’m substituing FreeRTOS IP stack for the Microchip Stack. Forutnatly both have Berkeley type sockets and the way that Amazon imlemented the code theres only one place that it seems it needs to be changed which is aws_secure_sockets.

Its not an exact 1:1 swap however, i’m spending quite a bit of time with various types.

Edited by: mrpackethead on Apr 19, 2018 9:32 PM