Hello everyone! I have a platform with four cores, where three cores are assigned to linux kernel and one to FreeRTOS. I need to share a buffer between these two operating systems that run on different cores.
How can I write the buffer in a physical address, where processes in both operating system can read/write? Do you have any suggetions or documentation about it?
Thank you so much!
If the question is how do you place a buffer at a known memory address then there are a few ways you can do it. How you actually do it depends on the compiler and linker you are using. Some compilers have special extensions to enable a variable to be placed at a particular address. Some have extensions that enable you to allocate a variable to a particular linker section - then you use the linker to define where in memory that section goes. You can also just not tell the linker about a valid memory range so it doesn’t put anything in that memory, leaving it free for you to use yourself.
Not really a FreeRTOS question though.
Thank you for the relply! Yes I am sorry, it is a general question. About FreeRTOS side: as I understand, I can’t use FreeRTOS API like xMessageBufferCreate() to do that. I have just to find a way to place in a known address setting up the linker and compiler. It is nothing but create a task that declare a Buffer in C lenguage, and assign to a physical address in some way?!
You could use the static functions (xTaskCreateStatic for example) of FreeRTOS where you supply a buffer yourself, again as @rtel said, you will need a linker script or compiler directives to place these buffers at known locations, you won’t be able to use the functions that allocate memory internally as you won’t be able to control specific fragment locations.
Thats from the FreeRTOS side.
From the linux side, you have multiple options as well. linker scripts are one of them, memory mapping mmap(2) of /dev/mem, or a module device file that you will have to write yourself.
Needless to say that these buffers needs to be volatile as they could change outside the control of the program.
The challenge I would see is avoiding data corruption and synchronization of this shared buffer. The way I suggest is to use atomics and lock free algorithms that do not count on any os feature from both sides (linux and FreeRTOS) .
Ok, now is more clear. But suppose I have declared a buffer in /dev/mem mapped in linux user space with mmap(2). To access the same block of memory I have to declare the same buffer into FreeRtos Task and assign to it the same address with the linker (or compiler as you said)? it is enough?
Thank you very much.
Linux to FreeRTOS comms normally uses a library such as OpenAMP.
Hi,
If I understand your problem correctly it’s an SMP system with 4 cores and you would like to run on one of those cores FreeRTOS. Right?
There are various approaches to deal with this and it’s typically non trivial, since your hardware ideally needs to support hardware virtualization. One possible solution is a hypervisor like this[1]. Here[2] is a FreeRTOS cell example. Keep in mind, that you will need to “virtualize” whatever hardware is used by FreeRTOS and remove it via virtualization that from Linux.
This[3] paper might give you some initial ideas how you can communicate between Linux and FreeRTOS in this setup and you might want to ask for further details here[4].
Some people suggested /dev/mem. I would strongly recommend against that and go for a proper driver. Probably this driver already exists - depending on your needs and your hardware.
For a “generic” solution from the Linux point of view have a look at [5] and look out for rpmsg[6] and virtio.
[1] GitHub - siemens/jailhouse: Linux-based partitioning hypervisor
[2] GitHub - siemens/freertos-cell: FreeRTOS for Jailhouse Cells
[3] https://events.static.linuxfound.org/sites/events/files/slides/LinuxConJapan-2016-Jailhouse-IPC.pdf
[4] https://groups.google.com/g/jailhouse-dev/c/LUUjyKDBccc/m/pSLtS5EAAwAJ
[5] Remote Processor Framework — The Linux Kernel documentation
[6] Remote Processor Messaging (rpmsg) Framework — The Linux Kernel documentation
As I already said in my previous comment, I would avoid accessing /dev/mem from user space. In a proper Linux configuration you disable access to it as much as possible since it’s a huge security hole. Instead, if you want to expose kernel memory to user space via mmap, you should write a device driver. … but, assuming your system is SMP you need many more things up and running to be able to run FreeRTOS on a single core. Typically Linux will take over all your cores and you need to first think about how to avoid this.
Hello dr. Berger,
first of all thank you for the precious information. You completly got the point!
I will be more precise: I have a Jetson tx2, where three cores are assigned to linux kernel and one core to FreeRTOS using jailhouse hypervisor, as you suggest. At this point, I need to share a buffer between one process in FreeRTOS and another one running in Linux. I think its about AMP (not SMP).
I saw in this paper different solution for IPC between domains, but I didn’t find any docs yet that explain how to do it. I read also the solution suggested by dr. Barry in the comment above about OpenAMP but it seems not suitable in a system with Jailhouse.
I will check this solution, but it is just from the linux side? Do you know if its possible recive/send message from FreeRTOS in this way?
“At this point, I need to share a buffer between one process in FreeRTOS and another one running in Linux. I think it’s about AMP (not SMP).”
If you are talking about the Quad-Core ARM® Cortex®-A57 MPCore where 3 cores are running Linux and one FreeRTOS I would say it’s SMP. Symmetric Multi-Processing - 4 identical cores. To be more precise you need to also to check caches. I assume each core has its own L1 cache, but e.g. the L2 cache is probably shared between 2 CPUs. This means Linux will slightly influence your real-time behavior.
The reason you’ll not find much about AMP and jailhouse is that it’s SMP 
I guess the best place to ask for what you want is the jailhouse group[1].
If I understand correctly what you want to do it’s pretty “standard”. Shared memory between Linux and FreeRTOS. You should be able to read/write from FreeRTOS and also from Linux. Of course, this needs some protection not to write from one end while the other end reads and so on.
Yes, exactly.
I will ask for advise in Jailhouse group you linked
I have a similar setup 3 Cortex A53 cores Linux one RTOS, but I have some cache coherency issues. I have a shared memory to send messages from Linux to RTOS and vice versa. The shared memory is mapped on RTOS Core 3 MMU tables/pages as inner/outer shareable memory. And I would like to see the Linux cores share mem messages and RTOS going via the L2 cache, that would speed up communication. When I use inner/outer cache shared pages, the shared memory is has some of the data not all. If I go from the RTCORE and disable the TLB pages of using the cache, everything works fine. Has anyone tried shared mem with Cache flags on mmu pages?
This does not seem like a FreeRTOS question and more related to your hardware. I’d recommend asking on your hardware vendor’s forum.