Upstreaming FreeRTOS/QEMU-RISC-V/VirtIO + AWS-OTA Demo?

Libraries
1

Hello,

I have been working for a few months on FreeRTOS RISC-V port [0], part of my research. That effort includes a new demo, VirtIO lib + drivers that work on QEMU and a publicly available FPGA SoC on AWS/F1 [1].

I then ported the coreMQTT-Agent [2] from Windows to QEMU with VirtIO net and block devices + FAT. The demo uses mutual authentication, code signing, and performs a complete firmware update from AWS S3 Bucket (well, at least until the device reset/activate part). That gives me some confidence on the SW stack I modified/added.

If that work would be of interest, I can work on upstreaming it. If so, I’d appreciate it if you point me to the proper contribution guidelines. I can find some guidelines from AWS FreeRTOS [3] but not in FreeRTOS itself, or at least the master branch of it (though, there’s one on the lts-development branch [3]).

[0] https://github.com/CTSRD-CHERI/FreeRTOS/tree/hmka2
[1] https://github.com/acceleratedtech/ssith-aws-fpga

Thanks,
Hesham

2

Adding the remaining links because new users are only allowed 2 links per topic

[2] https://github.com/FreeRTOS/Lab-Project-coreMQTT-Agent
[3] https://github.com/FreeRTOS/FreeRTOS/blob/lts-development/CONTRIBUTING.md

3

Hi - thanks this is definitely interesting for us. I will concur with my colleagues to decide the best place to have this code. The MQTT agent repo itself might be a good place, at least temporarily while the agent is being completed. I’m also very interested in your RISC-V port - I’ve not looked yet to see if or how you had to make any changes to the existing code.

4

Reporting back - our intent is to move the MQTT agent implementation into the coreMQTT repository when it is more mature, but also leave the existing labs repo (which submodules in the MQTT implementation from the coreMQTT library) as a sandbox for testing and experimentation. Hence, for now at least, we suggest contributing the code to the MQTT agent repo. I will take a look at the links you posted to see how straight forward that will be - presumably it will be quite easy - but will get back to you again after that. Thanks.

5

Hello Richard,

Thanks for your interest and instant replies. Please note I have just pushed the MQTT-Agent demo today. I’ll give a bit more context/details on our work.

We have extended FreeRTOS/RISC-V port to work with our CHERI architecture [1]. This gives us complete spatial memory safety with negligible performance overheads. There’s WIP on providing SW compartmentalization based on that.

For this specific topic, I thought I’d contribute back the vanilla RISC-V libraries (i.e., no CHERI, unless there will be an interest for it in the future).

For the vanilla RISC-V port offered by FreeRTOS, there are almost no changes in the kernel itself. There are, however, a few added libraries (that I placed in FreeRTOS-Labs) to include a portable VirtIO library among others. This library provides networking and block devices support. I added drivers to both FreeRTOS-Plus-TCP and FreeRTOS-Plus-FAT to make use of this library. Such libraries/drivers are independent of a specific demo and can be shared.

The other part is a new RISC-V Demo directory. That directory [2] holds BSPs and a few drivers (interrupt controller, HTIF, UART, syscalls, etc) plus the apps it can run. The design of this Demo aims to be generic enough to build for different variants of RISC-V platforms (QEMU, Spike, FPGAs, etc), different apps (blinky, CLI, FTP, OTA, etc), different toolchains (GCC, LLVM), w/o CHERI support, w/o compartmentalization support, among other command-line options. It uses a build system called waf.

That makes the project structure a bit different from the one you use in the Lab-Project-coreMQTT-Agent [3]; I re-use the existing released libraries in the classical FreeRTOS/ distribution (being the top directory) and I don’t create separate repos/submodules under coreMQTT-Agent.

Back to where things should go. The libraries should be generic enough not to be part of a specific demo. For example, the VirtIO library may go under FreeRTOS-Labs (like FAT), or FreeRTOS-Plus/ThirdParty for instance. The NetworkInterface.c for that needs to be merged with FreeRTOS-Plus-TCP and similarly the ff_virtioblk_disk.c in FreeRTOS-Plus-FAT.

Regarding the changes I made to the coreMQTT Agent demo itself, I basically just added a new ota-pal [4] that uses FreeRTOS-FAT instead of Windows. That’s the relevant part that can be upstreamed (as new files).

[1] https://www.cl.cam.ac.uk/research/security/ctsrd/cheri/
[2] https://github.com/CTSRD-CHERI/FreeRTOS-Demos-CHERI-RISC-V
[3] https://github.com/FreeRTOS/Lab-Project-coreMQTT-Agent
[4] https://github.com/CTSRD-CHERI/Lab-Project-coreMQTT-Agent/commit/b2d705d4cdfbbdcbdfba8a29198b14d3f5629320

6

Looks like https://github.com/CTSRD-CHERI/FreeRTOS-Demos-CHERI-RISC- is where most of the goodies are, but I’m struggling to build. I’m not familiar with waf. When I execute was if asks me to call “waf configure” - after which I get this output:

Setting top to                           : /mnt/c/[truncate_path]/FreeRTOS-Demos-CHERI-RISC-V
Setting out to                           : /mnt/c/[truncate_path]/FreeRTOS-Demos-CHERI-RISC-V/build
Checking for 'gcc' (C compiler)          : not found
Checking for 'clang' (C compiler)        : not found
Checking for 'icc' (C compiler)          : not found
Invalid toolchain

Both the default gcc and riscv64-unknown-elf-gcc are in the path, so I assume I need to pass parameters into configure.

Grateful if you provide guidance on the build instructions.

7

Hello Richard,

I have updated the demo README to contain some build and run instructions [1]. You also might need to update the submodules from the top FreeRTOS directory.

Basically, your command is failing because the default toolchain we use is LLVM/Clang.
To build with GCC you just need to pass –toolchain=gcc flag.

As an example, to build the AWS-OTA demo run:

$ ./waf configure --program-path=coreMQTT-Agent --program=aws_ota --toolchain=gcc --riscv-platform=qemu_virt build

And to run it on QEMU:

$ qemu-system-riscv64 -M virt -m 2048 -nographic -bios build/RISC-V-Generic_aws_ota.elf -device virtio-net-device,netdev=net0 -netdev user,id=net0,ipv6=off

Please let me know if you face any issues and/or have any suggestions.

[1] https://github.com/CTSRD-CHERI/FreeRTOS-Demos-CHERI-RISC-V/blob/master/README.md

8

:+1: Output is now:

Setting top to                           : /mnt/c/temp/delete/GithubRiscVContribution/FreeRTOS-Demos-CHERI-RISC-V
Setting out to                           : /mnt/c/temp/delete/GithubRiscVContribution/FreeRTOS-Demos-CHERI-RISC-V/build
Checking for 'gcc' (C compiler)          : riscv64-unknown-elf-gcc
Checking for program 'gas, gcc'          : riscv64-unknown-elf-gcc
Checking for program 'ar'                : /usr/bin/ar
('Configuring coreMQTT-Agent Demo @', '/mnt/c/temp/delete/GithubRiscVContribution/FreeRTOS-Demos-CHERI-RISC-V/coreMQTT-Agent')
'configure' finished successfully (7.812s)
Waf: Entering directory `/mnt/c/temp/delete/GithubRiscVContribution/FreeRTOS-Demos-CHERI-RISC-V/build'
Building coreMQTT-Agent OTA Demo

After that I get some build errors but should be able to work through those.

9

Hello Richard,

The demo is expected to work within the old classical FreeRTOS distribution [1].

If you clone that, checkout the hmka2 branch along with its submodules then cd to FreeRTOS/Demo/RISC-V-Generic then run waf there, that should work.

[1] https://github.com/CTSRD-CHERI/FreeRTOS/tree/hmka2

10

Update:

I am able to build the servers example, but not the OTA example as ./demo/aws_ota.c is missing. I think I have seen enough to have a good idea of how to bring the various components in to the repos, but would like to see the whole thing building too. Is aws_ota.c a file you created? I can’t see anything with that specific name in our repos.

11

Great to know you got the servers demo built! That OTA demo error sounds to me like the –program-path flag is assigned the wrong path for OTA or the wrong repo/submodule; the build system shouldn’t try to infer aws_ota.c and it doesn’t exist for that demo.

I’d make sure that:

  1. –program-path=coreMQTT-Agent is passed.
  2. The coreMQTT-Agent submodule/directory [1] exists under the Demo directory (where you execute waf from).
  3. The coreMQTT-Agent repo is checked out from my hmka2 branch and does have a wscript

I’ll update the README with that note.

PS you also may find building with cheribuild (documented in the README) more convenient/faster since it takes care of cloning everything and passing the correct flags along with running the demos.

[1] https://github.com/CTSRD-CHERI/Lab-Project-coreMQTT-Agent/tree/hmka2