I’m a student that is going to write a bachelor thesis about RTOS.
Part of the thesis concerns FreeRTOS and I would like to do some little experiments with the FreeRTOS codebase, by adding some APIs, modifying existing implementation, all possibly in a backward-compatible manner.
Since I want to ensure backward-compatibility, the obvious thing to do is to make sure to not break any existing tests.
But I’m having a hard time figuring out how the FreeRTOS code base is tested.
Specifically, I only care about tests concerning the “common code”, since I think I’ll be playing only with that.
I’ve read FreeRTOS - Free RTOS Coding Standard and Style Guide, but didn’t find much specific information. In particular, I didn’t really understand
The standard demo/test files attempt to provide ‘branch’ test coverage
To which files does “standard demo/test files” refer to?
I’m aware of the Demo folder (
https://github.com/FreeRTOS/FreeRTOS/tree/master/FreeRTOS/Demo), but AFAIK these are all platform-specific demos and I don’t think that these represent the tests for the common code, which I would expect to be runnable directly on the PC (also for purpose of Continuous Integration).
I’ve found the Test folder (
https://github.com/FreeRTOS/FreeRTOS/tree/master/FreeRTOS/Test) in the FreeRTOS repository, but only the CMock folder contains a few tests, in the traditional sense of test. The other two folders contain formal proofs.
Can you confirm me that these are all the means FreeRTOS is tested with? I.e. a few tests and two kinds of formal proofs.
Another related thing that I would like to ask is relative to CI. Is it possible to run the FreeRTOS CI checks on my fork? Or, does the FreeRTOS contain some sort of CI configuration that can be reused in the forked repository? My modifications are probably going to be just some esoteric educational proof-of-concept toys and I don’t expect to get them merged. But currently, without making a Pull Request, I don’t know how can I get those CI checks.
I would really appreciate if you could give me some guidance. Thank you in advance!
One thing to note, is that the ‘common code’ is an incomplete program, since some functions that FreeRTOS needs to do can not be done universally in Standard C, so you will need to choose one of the options in the portable directory to use. Note, there IS a port that is designed to run under Windows, but that has some limitations as Windows is not really suitable to true Real Time operations.
There are two sets of ‘common’ code. The source files that are common to all kernel ports - contained in FreeRTOS/source (to which you must add a port specific implementation to tailor the kernel to a particular architecture) and the source files that are common to all demo applications - contained in FreeRTOS/Demo/Common/Minimal. This page describes the directory structure, and I think each major directory also contains a readme telling you what the directory contains. Note the ‘demo’ files are really test files too - all but the really old ones really. For example have a look inside a file such as this and you will see it is getting good test coverage.
We have one large GCC project that we use for code coverage measuring and testing, but that is a bit messy as it includes unrelated tests in common files to hit the code paths that the standard tests don’t. If you want to hit most but not all paths then you can use something like the Windows demo (you will find the docs for that on FreeRTOS.org) or a QEMU demo that have enough RAM to run all the tests - here is an IAR version - if you search the FreeRTOS/Demo for MPS2 you will find a GCC version too. These run all the common demo/tests tasks continuously and self monitor to look for errors on timing going out of predetermined thresholds. Generally we leave these running for weeks at a time.
The CMOCK tests are new and just being created now.
Wow! Thank you for this thorough answer!
I would like to ask also whether there is a CI configuration that I can pick and copy, although that’s not so important; I think the information you gave me is enough for me to setup a rough CI pipeline.
I have found FreeRTOS/CI-CD-Github-Actions, but I guess it’s only for coreMQTT.
May I ask how is coverage of the demo tests checked and how can it be reported? Looking into the Makefile(s) of the POSIX example and the CORTEX_M3_MPS2_QEMU_GCC/ example (that you suggested), I can’t find any code coverage related configurations (e.g. use of gcov, etc.).
Thanks for your help!
We use this project: FreeRTOS/FreeRTOS/Demo/WIN32-MingW at master · FreeRTOS/FreeRTOS · GitHub with gcov. There is an additional file not used by other projects that covers areas missed by the common tests. Ports that support interrupt nesting also have additional tests to ensure the nesting gets covered too.