"Monolithic" kernel

Kernel
1

Hi !

I can see in modern MCU we should avoid situation when privileged kernel execute non-privileged code. I can see FreeRTOS actively use memset/memcpy and some other standard functions. And library is the same for kernel and all other user side code.

Does it possible if kernel have own versions of all standard functions what it need ?
Or some how extract own binary copy of required functions from standard libraries.

And in this case user code will use only mpu_wrappers api and not have any common code with kernel.
Or may be kernel should be possible compile as standalone library and use in own environment.
I just wondering what us the best and fast way to implement this kind of separation.

Regards,
Eugene

2

FreeRTOS isn’t in the same class as what you are talking about with a ‘privileged kernel’. In many uses (the default case), ALL of the user program runs with exactly the same privileges as the kernel, and if you look at program structure, FreeRTOS runs more inside the application than the application running inside the kernel (At boot, we go to the user program which then starts the kernel, as opposed to bigger systems which start up into the kernel which then might load a user application).

Yes, there are ports of FreeRTOS where parts of the user application can run as a ‘Restricted’ task, but even those systems still have much of the user provided code running in unrestricted space.

3

Hi Richard !

Let’s take LPC55S69 GCC port as example.
All own tasks running in unprivileged mode and use some peripherals driver directly or via SVC calls. So start routined,Kernel is fully privileged and Idle task and Timer task callbacks as well.

Common code is some library functions . Memset can be called from kernel and own task and kernel execute non-privileged memset.

I think problem coming with section( “freertos_system_calls” ) what start execution with non-privileged rights, continue with privileged and return to non-privileged after that.
May be some minimal set of APIs can be wrapped to SVC calls for clearly separate sections ?
Or exist mode fundamental problems what are not visible to me.

Regards,
Eugene

4

@Eugene IMHO if you require strict separation of kernel and user space you should switch to the appropriate platform e.g. Linux. I think it’s not really applicable trying to mimic this behavior with a monolith on resource constrained MCUs w/o MMU.
It’s just another class of system and there is a price for that.

5

As Hartmut says, that is a different segment of OS. FreeRTOS is at its heart, a small highly efficient micro kernel Real Time Operating System. It is designed to first be efficient for small processors. It has some capability to use extra features like a memory protection unit, but those are bonus features and not fully implemented. Most machines that it will run on will have the code in Flash Memory, so it really isn’t ‘unprotected’, so the code that is running is safe from harm, and it privilege will be based on what is calling it. Even if code is copied into RAM, if you are running Memory Protection, the code will be marked Read Only, so is still safe.

I will also point out that the FreeRTOS kernel has NO device drivers integrated into it, so all those drivers are, to the kernel ‘User’ code.

I will also add that I personally have never felt a need to enable the MMU/MPU in my programs (if the processor even had one), but then since all my code is running out of Flash Memory, and it is all well tested (and from me and my team), so I don’t see a particular need for the feature.

I suppose another thing is that the concept of a Restricted Task isn’t enough to say that unit can continue to run despite malicious interference from a Restricted Task. Yes, you sigificantly shrink the ‘attack surface’ of the Task, but because FreeRTOS doesn’t have a ‘Process’ level of isolation, a defective task, even if Restricted, can still crash the system.

Fundamentally, FreeRTOS trusts the user application, because the user application has the machine and turned over significant control to FreeRTOS. If your threat model can’t handle that, then FreeRTOS isn’t the system for you.

1 Like
6

LPC55S69 is an ARM Cortex-M33 (ARMv8-M) for which FreeRTOS implements all the features described on this page Using FreeRTOS on ARMv8-M Microcontrollers - FreeRTOS . The ARMv8-M MPU (memory protection unit) does not have the same restrictions as those found on the ARMv7-M cores, so it is likely more people will use ARMv8-M with the MPU enabled than on the ARMv7-M (Cortex-M4 being an example of an ARMv7-M).

Note there have been quite a few updates to the ARMv8-M ports, and how they use memory the heap in particular, since the last release.

There are a few things in the original post I’m not sure about though, in particular:

Is that really what you want? The memory protection unit is used to restrict access to the RAM memcpy() (and similar functions) access. So if an unprivileged task calls the function to access memory it has the rights to access, then that is fine. If an unprivileged task calls the function to access memory that is marked as privileged access only, it will generate a memory protection exception. That would be the same if every task had its own copy of the function or not. I don’t think it would be practical for ever task to have its own copy without using virtual memory or otherwise completely changing the FreeRTOS model so service calls are used to call all functions and the service call handler checked which task was calling it.

7

Hi !

All non-privileged user tasks + standard libraries have common code space and it is configured as one MPU section. On secure side MPU_s is also active and control NSC calls from non-privileged tasks to non-privileged code area on secure side.

TZ-M secure configuration in misc register “Enable secure privilege check for AHB matrix” and “Enable non-secure privilege check for AHB matrix” enforce privilege check and non-privileged code can’t be executed as privileged.
I think this is quite important feature what can help with better separation/isolation of code.
( Privileged code still can have access to privileged and non-privileged data).

I would like to utilize this feature and study for minimal changes what need to be done on secure and non-secure side ( RTOS side).

First problem is to have own set of standard functions. It seems to me kernel use only few of those. second problem is to support limited set of RTOS IPC APIs what are need. Basically FreeRTOS have those APIs as ISR variant and need to be wrapped to SVC calls.
But for example waiting message from queue can be problem a bit.

It means I try to utilize all secure features of ARM Cortex-M33 (ARMv8-M) MCU and still keep RTOS as light as possible. In any case similar MCU is more and more need everywhere and lightweight OS what utilize all security features need as well.

Regards,
Eugene

8

Your last comment I think it the key to your statement. First, your issue isn’t about ‘protected’ mode, (which is generally considered MMU/MPU based) but ‘Secure’ mode, dividing the running program broadly into a Secure side and an Unsecured side.

Again, the key point is that FreeRTOS is focused on simpler processors, which don’t have this capability, and adds some basic capabilities to support this additional feature, but only what can be added at minimal cost in architecture for the simpler processors. As such, it will not likely give you access to ALL the features of such a processor.

One BIG impediment, is that the C language itself doesn’t really support such a mode in a single program, and the basic structure of FreeRTOS is that the operating system is included a part of the application program, and isn’t an external program that the application integrates into. Such a division is inherently less efficient than a ‘Monolithic’ program.

I would say, that to fully use this type of security, you really need to build your program as two separate pieces: One, on the secure side, including FreeRTOS, doing the work that needs to be on the secure side that exports an API that a separately built Unsecure application uses. This API by necessity will be bigger than the FreeRTOS API, (Remember FreeRTOS itself is just a microkernel with no I/O capability) so needs to be built by the user, though FreeRTOS does provide support for its part of the API.

If you do build your application this way, then you do get you library separation you are asking for, as your unsecure application will be built independently of the secure RTOS, so will have its own library, and only access the secure side through the defined API.

9

Hi !

I also don’t like linux style rtos on this type of MCU. But it should be some intermediate solution
what can utilize more ( almost all) TZ-M security features and keep single code for secure and single code for non-secure parts and well defined NSC APIs.
And I try to understand what can be added to keep this approach .

Regards,
Eugene

10

Yes, it sounds like what you want to do is to basically build a two part program, each mostly independent. Part 1 is the secure side, that includes an API segment with the transition code. That segment probably should begin with a ‘jump’ table to put the addresses of the routines at nice fixed addresses. This secure side includes most of the FreeRTOS code, and the code for most of your device drivers and maybe any tasks that need to be kept on the secure side.

The second part would be built all as the unsecured side, whose only connection to the secure side would be through that jump table. FreeRTOS might be able to have some stuff to help support this division, but it can’t just provide it, as almost by definition, there will be pieces of the secure side that will need a connection that will need to be user provided.

The also is the limitation that FreeRTOS won’t want to pick up any inefficiencies for machines that don’t have this division, and wants to limit the code complexity to support this mode.

11

TZ-M secure configuration in misc register “Enable secure privilege check for AHB matrix” and “Enable non-secure privilege check for AHB matrix” enforce privilege check and non-privileged code can’t be executed as privileged.
I think this is quite important feature what can help with better separation/isolation of code.

It seems that you are talking about the “47.3.3: Secure AHB bus and secure AHB Controller” section of LPC55S6x user manual (UM11126). This layer of protection ensures correct security domain (secure vs non-secure) and not privilege level. All access to all the AHB slaves are checked for security violations. To quote from the User Manual:

The secure AHB Controller provides access policies for all the bus slaves via checker
functions. All masters on LPC55S6x outputs security side-band signals HPRIV
(privileged) and HNONSEC (Secure access) as indication of security attributes for a given
access. Secure AHB Bus processes this signals and compares them against security
attributes set for slaves in secure AHB Controller. Access is granted if security attribute of requested access is not violating the security attribute of the slave being accessed.
Security violation interrupt is raised if violation occurs on data or instruction access.

To your second point, the code segment is organized as follows:

                  +-----------------+
                  |                 |
                  |                 |
                  |   Privileged    |
                  |                 |
                  |                 |
                  +-----------------+
                  |                 |
				  |    Sys Calls    |
				  |                 |
				  +-----------------+
                  |                 |
                  |                 |
                  |                 |
                  |                 |
                  |  Unprivileged   |
                  |                 |
                  |                 |
                  |                 |
                  |                 |
                  +-----------------+
  • Privileged - This section contains FreeRTOS kernel code.
  • Sys Calls - This section contains all the system calls i.e. the FreeRTOS APIs which are made available to an unprivileged task.
  • Unprivileged - Everything else.

You are pointing out that standard library functions (memset etc.) live in the unprivileged segment and the FreeRTOS kernel calls them which is correct. If the standard library functions are to move to the privileged segment, there are two possible options:

  1. Make them unavailable to the unprivileged tasks.
  2. Provide MPU wrappers (System calls) for all of them.

The first is undesirable and the second does not provide any benefit. As Richard pointed out, any unprivileged task wont be able to corrupt privileged RAM just by calling memset.

Did I understand your point correctly or did I miss anything?

Thanks.

12

Hi !

Better to look table “47.4.78 Secure control register”.
ENABLE_SECURE_CHECKING track secure vs non-secure and provide minimal level of security.

But real restriction starts when ENABLE_S_PRIV_CHECK and ENABLE_NS_PRIV_CHECK are activated. In this case MPU mapping should much with MPC and PPC settings at the moment of usage. MPU sections for code should be aligned to 32KB and SRAM - 4KB.
And with privileged rights is not possible to execute non-privileged code.
Now only MPU control privileges but better to control privileges by AHB controller as well.
MCUExpresso IDE in ConfigTool section where TZ have own tab provide good looking graphical representation of entire security features.

For what reason you recommend to keep FreeRTOS on secure side ? As usually task what communicate with external word better to keep with minimal privileges , without secure context and on non-secure side.

Regards,
Eugene

13

Are you asking why we recommend using FreeRTOS on the secure side? If so, then we don’t recommend that. While running FreeRTOS on the secure side is possible, doing so would be unusual.

14

But what in this case a minimal secure like combination what can be recommended for beginners ?
FreeRTOS at non-secure side , minimize amount of task with secure context and fixed MPU sectioning on secure side.
What is doable vision if FreeRTOS is taken as a core element ?

Regards,
Eugene

15

But real restriction starts when ENABLE_S_PRIV_CHECK and ENABLE_NS_PRIV_CHECK are activated. In this case MPU mapping should much with MPC and PPC settings at the moment of usage. MPU sections for code should be aligned to 32KB and SRAM - 4KB.
And with privileged rights is not possible to execute non-privileged code.

I just finished reading chapter 47 of the UM1126 doc. I see that Memory Protection Checker (MPC), Peripheral Protection Checker (PPC) and Master Security Wrapper (MSW) provide more fine grained control to provide trusted execution environment. Based on my reading, in the most strict configuration, the following 4 tier levels are possible:

  • Secure - privilege
  • Secure - non-privilege
  • Non-secure - privilege
  • Non-secure - non-privilege

And with privileged rights is not possible to execute non-privileged code.

The above was not very clear (as it also mentioned that access from higher to lower level is possible) and I need to setup a project to verify it. I agree that if it is the case, there may be some issues in running FreeRTOS. This is specific to LPC55Sx device and we will look into how we can ensure that the FreeRTOS MPU setup works with MPC and PPC.

MCUExpresso IDE in ConfigTool section where TZ have own tab provide good looking graphical representation of entire security features.

Thank you for the tip. This tool is really helpful and I will use it to setup the project I mentioned above.

But what in this case a minimal secure like combination what can be recommended for beginners ?

We recommend running FreeRTOS on the non-secure side and some small trusted critical software on the secure side. The secure software can export some APIs to the non-secure software.

Thank you for all your feedback.

Regards,
Gaurav

16

If those bits are not set looks like only MPU define privilege level.
All natual MCU PPC/MPS settings are ignored. I have asked NXP few times and get answer like this:
"
if you enable secure/non-secure privilege check, the AHB secure controller performs additional check for privilege/non-privilege access. In this case privilege/non-privilege access (defined by either secure on non-secure MPU) has to be aligned with AHB secure controller settings. The privilege/non-privilege check can be separately enabled for secure and non-secure bus transactions. The security of transaction is defined by SAU+IDAU.

The AHB secure controller distinguishes between instruction fetch access and data access. In case of instruction fetch, the strict checking is applied. It means that code has to be executed from the same security level. It means that if CPU is in secure-privilege mode, the code memory has to be configured as secure-privilege in AHB secure controller as well. (This is probably reason for your HardFault). In case of data accesses, the privilege code has access to both privilege and non-privilege (user) memory, the non-privilege code has access to non-privilege (user) memory only.

Please note that if CPU is in secure mode, the secure MPU takes effect. In non-secure/normal mode the non-secure MPU takes effect.

Please also note, that if privilege check is enabled, you cannot simply switch to user mode by writing into CONTROL register in thread mode. This is because of strict check of AHB secure controller for instruction fetch. Once you write into CONTROL register in thread mode, the CPU switches immediately into user mode and this causes HardFault. This is because your code memory is configured as privilege. You need to switch to user mode via interrupt (for example SVC). The interrupt is executed always in privilege mode. Now you can set user mode in CONTROL register (because change to user mode will happen during interrupt exit) and modify return address to the new location, which is configured to user access in AHB secure controller. At the end of this interrupt, the CPU change from privilege to user mode and jump to the new location. So code execution continues in user mode from memory, which configured to the same level in AHB secure controller. The secure/non-secure value is selected based actual secure mode of CPU. You cannot switch from secure-privilege to non-secure-user.
"

I’m trying to sort code on Secure side first. Just to be sure if NSC calls form non-privileged task call code what is located in nonprivileged section at secure side and MPU_S setting the same as MPC.

I’m aligning code /data sections in linker script directly.

. = ALIGN(4096);
or
. = ALIGN(32768);

If I not guess privilege level I have hard fault like this:
"
Entering HardFault interrupt!
SCB->BFSR:IBUSERR fault: Instruction bus error on an instruction prefetch.

Additional AHB secure controller error information:
Secure error at AHB layer 1.
Address that caused secure violation is 0x400455C.
Secure error caused by bus master number 0.
Security level of master 3.
Secure error happened during read code access.
"

For PPC situation is not clear yet MPU_S configured for keep non-privileged access right to almost entire peripheral area 0x50000000 - 0x500ABFFF.
But in PPC settings, peripherals what can be accessed in privileged mode only, set as S-Priv.

So this is state where I’m right now and I think on secure side is possible to sort code/data some how. But on non-secure, where RTOS are, it might be more complicate task.

API like this xQueueSelectFromSet should wait event as part of non-privileged task infinite time and how they can work via SVC ?

Regards,
Eugene

17

Thank you for sharing the response from NXP. It’s really helpful and as I said before, there may probably be problems with the most strict configuration of AHB. I will get back on this.

API like this xQueueSelectFromSet should wait event as part of non-privileged task infinite time and how they can work via SVC ?

We raise privilege before calling the actual APIs, so that should be okay? Or are you talking about problem when lowering the privileges by writing to CONTROL register as mentioned in NXP’s response above?

Thanks.

18

Yes, NXP’s documentation is not written well at all. A lot of time need to ask every detail.
But step by step it is possible.

I think RTOS API should be done via SVC . In other case when you arise privilege, some non-privileged code executed as privileged and only after that privileged function called.

How in this case infinitive wait can be implemented:

xActivatedMember = xQueueSelectFromSet(xSetQueueHandle, portMAX_DELAY);

Regards,
Eugene