FreeRTOS Plus FAT extrem slow (compared to FatFS)

Libraries
1

Hi all

For a project I’ve changed my code to use FreeRTOS and therefore I’ve also switched from the Elm Chans FatFS solution to the FreeRTOS Plus FAT library (using the port library from carlk3).

Now I’m running in some performance issues (with a PI PICO @125MHz system clock), where I don’t know to solve them…

  1. Reading a small file with 450 Bytes in SPI mode leads to the following timings (just one simple Task running with max priority)

With 12.5MHz SPI-Clock

ff_fopen = 2.4ms
ff_stat (to get the filesyste for buffer allocation) = 0.8ms
ff_fread = 1.6ms (→ compared to the FatFS without DMA support this is near 6x slower)
ff_fclose = 0.3ms

With 1.25MHz SPI clock

ff_fopen = 5.4ms (→ why does reopen a file after some magic first stack initialization steps, takes that long?)
ff_stat (to get the filesyste for buffer allocation) = 0.8ms
ff_fread = 4.7ms (→ compared to the FatFS without DMA support this is near 2x slower)
ff_fclose = 0.3ms

So based on this measurements the SPI clock doesn’t have the expected impact - there must be something else…

If I read a file with ~20kByte, the things get even worser → reading this file @12.5MHz is about 10seconds (what is not just linear longer to the filesize).

After reading some forum post to get rid of this issue, I’ve now more questions ;). sometime it is recommended

  • to use preallocated files filled with 0s for improving performance and increase power loss stability → how should this be done so that the correct filesize is returend (e.g. by ff_stat or ff_filelength), for allocating reading buffer?
  • to hold the file open (e.g. for high speed logging) → how should this be realized in a real world szenario, when multiple tasks are using differend files (each task doesn’t know the others)?
  • holing files open lead to the question, how to deal with it in case of power-loss detection (e.g. GPIO interrupt → save remainging data and flush it out, close the open files…)

Best regards

2

Hello @P51D , thank you for contacting the forum.

Before responding into detail could you post you copy of FreeRTOSFATConfig.h here?

Try to insert it as code, insert three back-quotes followed by control-V of the contents of the header file.

Thanks

3 
/* FreeRTOSFATConfig.h
Copyright 2021 Carl John Kugler III

Licensed under the Apache License, Version 2.0 (the License); you may not use
this file except in compliance with the License. You may obtain a copy of the
License at

   http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed
under the License is distributed on an AS IS BASIS, WITHOUT WARRANTIES OR
CONDITIONS OF ANY KIND, either express or implied. See the License for the
specific language governing permissions and limitations under the License.
*/
/*
    FreeRTOS V9.0.0 - Copyright (C) 2016 Real Time Engineers Ltd.
    All rights reserved

    VISIT http://www.FreeRTOS.org TO ENSURE YOU ARE USING THE LATEST VERSION.

    This file is part of the FreeRTOS distribution.

    FreeRTOS is free software; you can redistribute it and/or modify it under
    the terms of the GNU General Public License (version 2) as published by the
    Free Software Foundation >>!AND MODIFIED BY!<< the FreeRTOS exception.

    ***************************************************************************
    >>!   NOTE: The modification to the GPL is included to allow you to     !<<
    >>!   distribute a combined work that includes FreeRTOS without being   !<<
    >>!   obliged to provide the source code for proprietary components     !<<
    >>!   outside of the FreeRTOS kernel.                                   !<<
    ***************************************************************************

    FreeRTOS is distributed in the hope that it will be useful, but WITHOUT ANY
    WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
    FOR A PARTICULAR PURPOSE.  Full license text is available on the following
    link: http://www.freertos.org/a00114.html

    ***************************************************************************
     *                                                                       *
     *    FreeRTOS provides completely free yet professionally developed,    *
     *    robust, strictly quality controlled, supported, and cross          *
     *    platform software that is more than just the market leader, it     *
     *    is the industry's de facto standard.                               *
     *                                                                       *
     *    Help yourself get started quickly while simultaneously helping     *
     *    to support the FreeRTOS project by purchasing a FreeRTOS           *
     *    tutorial book, reference manual, or both:                          *
     *    http://www.FreeRTOS.org/Documentation                              *
     *                                                                       *
    ***************************************************************************

    http://www.FreeRTOS.org/FAQHelp.html - Having a problem?  Start by reading
    the FAQ page "My application does not run, what could be wrong?".  Have you
    defined configASSERT()?

    http://www.FreeRTOS.org/support - In return for receiving this top quality
    embedded software for free we request you assist our global community by
    participating in the support forum.

    http://www.FreeRTOS.org/training - Investing in training allows your team to
    be as productive as possible as early as possible.  Now you can receive
    FreeRTOS training directly from Richard Barry, CEO of Real Time Engineers
    Ltd, and the world's leading authority on the world's leading RTOS.

    http://www.FreeRTOS.org/plus - A selection of FreeRTOS ecosystem products,
    including FreeRTOS+Trace - an indispensable productivity tool, a DOS
    compatible FAT file system, and our tiny thread aware UDP/IP stack.

    http://www.FreeRTOS.org/labs - Where new FreeRTOS products go to incubate.
    Come and try FreeRTOS+TCP, our new open source TCP/IP stack for FreeRTOS.

    http://www.OpenRTOS.com - Real Time Engineers ltd. license FreeRTOS to High
    Integrity Systems ltd. to sell under the OpenRTOS brand.  Low cost OpenRTOS
    licenses offer ticketed support, indemnification and commercial middleware.

    http://www.SafeRTOS.com - High Integrity Systems also provide a safety
    engineered and independently SIL3 certified version for use in safety and
    mission critical applications that require provable dependability.

    1 tab == 4 spaces!
*/

#ifndef _FF_CONFIG_H_
#define _FF_CONFIG_H_

#include <time.h>   
    
/* Must be set to either pdFREERTOS_LITTLE_ENDIAN or pdFREERTOS_BIG_ENDIAN,
depending on the endian of the architecture on which FreeRTOS is running. */
#define ffconfigBYTE_ORDER pdFREERTOS_LITTLE_ENDIAN

/* Set to 1 to maintain a current working directory (CWD) for each task that
accesses the file system, allowing relative paths to be used.

Set to 0 not to use a CWD, in which case full paths must be used for each
file access. */
#define ffconfigHAS_CWD 1

/* Set to an index within FreeRTOS's thread local storage array that is free for
use by FreeRTOS+FAT.  FreeRTOS+FAT will use two consecutive indexes from this
that set by ffconfigCWD_THREAD_LOCAL_INDEX.  The number of thread local storage
pointers provided by FreeRTOS is set by configNUM_THREAD_LOCAL_STORAGE_POINTERS
in FreeRTOSConfig.h */
#define ffconfigCWD_THREAD_LOCAL_INDEX 1

/* Set to 1 to include long file name support.  Set to 0 to exclude long
file name support.

If long file name support is excluded then only 8.3 file names can be used.
Long file names will be recognised but ignored.

Users should familiarise themselves with any patent issues that may
potentially exist around the use of long file names in FAT file systems
before enabling long file name support. */
#define ffconfigLFN_SUPPORT 0

/* Only used when ffconfigLFN_SUPPORT is set to 1.

Set to 1 to include a file's short name when listing a directory, i.e. when
calling findfirst()/findnext().  The short name will be stored in the
'pcShortName' field of FF_DirEnt_t.

Set to 0 to only include a file's long name. */
#define ffconfigINCLUDE_SHORT_NAME 0

/* Set to 1 to recognise and apply the case bits used by Windows XP+ when
using short file names - storing file names such as "readme.TXT" or
"SETUP.exe" in a short-name entry.  This is the recommended setting for
maximum compatibility.

Set to 0 to ignore the case bits. */
#define ffconfigSHORTNAME_CASE 1

/* Only used when ffconfigLFN_SUPPORT is set to 1.

Set to 1 to use UTF-16 (wide-characters) for file and directory names.

Set to 0 to use either 8-bit ASCII or UTF-8 for file and directory names
(see the ffconfigUNICODE_UTF8_SUPPORT). */
#define ffconfigUNICODE_UTF16_SUPPORT 0

/* Only used when ffconfigLFN_SUPPORT is set to 1.

Set to 1 to use UTF-8 encoding for file and directory names.

Set to 0 to use either 8-bit ASCII or UTF-16 for file and directory
names (see the ffconfig_UTF_16_SUPPORT setting). */
#define	ffconfigUNICODE_UTF8_SUPPORT 1

/* Set to 1 to include FAT12 support.

Set to 0 to exclude FAT12 support.

FAT16 and FAT32 are always enabled. */
#define	ffconfigFAT12_SUPPORT 0

/* When writing and reading data, i/o becomes less efficient if sizes other
than 512 bytes are being used.  When set to 1 each file handle will
allocate a 512-byte character buffer to facilitate "unaligned access". */
#define	ffconfigOPTIMISE_UNALIGNED_ACCESS	1

/* Input and output to a disk uses buffers that are only flushed at the
following times:

- When a new buffer is needed and no other buffers are available.
- When opening a buffer in READ mode for a sector that has just been changed.
- After creating, removing or closing a file or a directory.

Normally this is quick enough and it is efficient.  If
ffconfigCACHE_WRITE_THROUGH is set to 1 then buffers will also be flushed each
time a buffer is released - which is less efficient but more secure. */
#define	ffconfigCACHE_WRITE_THROUGH	0

/* In most cases, the FAT table has two identical copies on the disk,
allowing the second copy to be used in the case of a read error.  If

Set to 1 to use both FATs - this is less efficient but more	secure.

Set to 0 to use only one FAT - the second FAT will never be written to. */
#define	ffconfigWRITE_BOTH_FATS	1

/* Set to 1 to have the number of free clusters and the first free cluster
to be written to the FS info sector each time one of those values changes.

Set to 0 not to store these values in the FS info sector, making booting
slower, but making changes faster. */
#define	ffconfigWRITE_FREE_COUNT 1

/* Set to 1 to maintain file and directory time stamps for creation, modify
and last access.

Set to 0 to exclude	time stamps.

If time support is used, the following function must be supplied:

	time_t FreeRTOS_time( time_t *pxTime );

FreeRTOS_time has the same semantics as the standard time() function. */
#define	ffconfigTIME_SUPPORT 1

/* Set to 1 if the media is removable (such as a memory card).

Set to 0 if the media is not removable.

When set to 1 all file handles will be "invalidated" if the media is
extracted.  If set to 0 then file handles will not be invalidated.
In that case the user will have to confirm that the media is still present
before every access. */
#define	ffconfigREMOVABLE_MEDIA	1

/* Set to 1 to determine the disk's free space and the disk's first free
cluster when a disk is mounted.

Set to 0 to find these two values when they	are first needed.  Determining
the values can take some time. */
#define	ffconfigMOUNT_FIND_FREE	1

/* Set to 1 to 'trust' the contents of the 'ulLastFreeCluster' and
ulFreeClusterCount fields.

Set to 0 not to 'trust' these fields.*/
#define	ffconfigFSINFO_TRUSTED 1

/* Set to 1 to store recent paths in a cache, enabling much faster access
when the path is deep within a directory structure at the expense of
additional RAM usage.

Set to 0 to not use a path cache. */
#define	ffconfigPATH_CACHE 0

/* Only used if ffconfigPATH_CACHE is 1.

Sets the maximum number of paths that can exist in the patch cache at any
one time. */
#define	ffconfigPATH_CACHE_DEPTH 8

/* Set to 1 to calculate a HASH value for each existing short file name.
Use of HASH values can improve performance when working with large
directories, or with files that have a similar name.

Set to 0 not to calculate a HASH value. */
#define	ffconfigHASH_CACHE	0

/* Only used if ffconfigHASH_CACHE is set to 1

Set to CRC8 or CRC16 to use 8-bit or 16-bit HASH values respectively. */
#define	ffconfigHASH_FUNCTION CRC16

/*_RB_ Not in FreeRTOSFFConfigDefaults.h. */
#define ffconfigHASH_CACHE_DEPTH 64

/* Set to 1 to add a parameter to ff_mkdir() that allows an entire directory
tree to be created in one go, rather than having to create one directory in
the tree at a time.  For example mkdir( "/etc/settings/network", pdTRUE );.

Set to 0 to use the normal mkdir() semantics (without the additional
parameter). */
#define	ffconfigMKDIR_RECURSIVE	 0

/* Set to a function that will be used for all dynamic memory allocations.
Setting to pvPortMalloc() will use the same memory allocator as FreeRTOS. */
#define ffconfigMALLOC( size )	pvPortMalloc( size )

/* Set to a function that matches the above allocator defined with
ffconfigMALLOC.  Setting to vPortFree() will use the same memory free
function as	FreeRTOS. */
#define ffconfigFREE( ptr )  vPortFree( ptr )

/* Set to 1 to calculate the free size and volume size as a 64-bit number.

Set to 0 to calculate these values as a 32-bit number. */
#define	ffconfig64_NUM_SUPPORT	1

/* Defines the maximum number of partitions (and also logical partitions)
that can be recognised. */
#define	ffconfigMAX_PARTITIONS 1

/* Defines how many drives can be combined in total.  Should be set to at
least 2. */
#define	ffconfigMAX_FILE_SYS 5

/* In case the low-level driver returns an error 'FF_ERR_DRIVER_BUSY',
the library will pause for a number of ms, defined in
ffconfigDRIVER_BUSY_SLEEP_MS before re-trying. */
#define	ffconfigDRIVER_BUSY_SLEEP_MS 10

/* Set to 1 to include the ff_fprintf() function.

Set to 0 to exclude the ff_fprintf() function.

ff_fprintf() is quite a heavy function because it allocates RAM and
brings in a lot of string and variable argument handling code.  If
ff_fprintf() is not being used then the code size can be reduced by setting
ffconfigFPRINTF_SUPPORT to 0. */
#define ffconfigFPRINTF_SUPPORT	0

/* ff_fprintf() will allocate a buffer of this size in which it will create
its formatted string.  The buffer will be freed before the function
exits. */
#define ffconfigFPRINTF_BUFFER_LENGTH 128

/* Set to 1 to inline some internal memory access functions.

Set to 0 to not inline the memory access functions. */
#define	ffconfigINLINE_MEMORY_ACCESS		1

/* Officially the only criteria to determine the FAT type (12, 16, or 32
bits) is the total number of clusters:
if( ulNumberOfClusters  <  4085 ) : Volume is FAT12
if( ulNumberOfClusters  < 65525 ) : Volume is FAT16
if( ulNumberOfClusters >= 65525 ) : Volume is FAT32
Not every formatted device follows the above rule.

Set to 1 to perform additional checks over and above inspecting the
number of clusters on a disk to determine the FAT type.

Set to 0 to only look at the number of clusters on a disk to determine the
FAT type. */
#define	ffconfigFAT_CHECK 1

/* Sets the maximum length for file names, including the path.
Note that the value of this define is directly related to the maximum stack
use of the +FAT library. In some API's, a character buffer of size
'ffconfigMAX_FILENAME' will be declared on stack. */
#define	ffconfigMAX_FILENAME 64

/* Defined in main.c as Visual Studio does not provide its own implementation. */
struct tm *gmtime_r( const time_t *pxTime, struct tm *tmStruct );

/* Prototype for the function used to print out.  In this case it prints to the
console before the network is connected then a UDP port after the network has
connected. */
// extern void vLoggingPrintf( const char *pcFormatString, ... ) __attribute__ ((format (printf, 1, 2)));
// #define FF_PRINTF vLoggingPrintf
// #define FF_PRINTF(fmt, args...)    vLoggingPrintf(fmt, ## args)
// #define FF_PRINTF   task_printf
// #define FF_PRINTF   printf
// #define FF_PRINTF error_message_printf_plain


/* Visual studio does not have an implementation of strcasecmp().
_RB_ Cannot use FF_NOSTRCASECMP setting as the internal implementation of
strcasecmp() is in ff_dir, whereas it is used in the http server.   Also not
sure of why FF_NOSTRCASECMP is being tested against 0 to define the internal
implementation, so I have to set it to 1 here, so it is not defined. */
#define FF_NOSTRCASECMP 1

/* Include the recursive function ff_deltree().  The use of recursion does not
conform with the coding standard, so use this function with care! */
#define ffconfigUSE_DELTREE					1

// #define ffconfigDEBUG                       1
#endif /* _FF_CONFIG_H_ */

I’ve tried to simplify / optimize to my needs:

  • no folders needed
  • filenames are <8 chars
  • filetypes are just one ini (config-file with <500 bytes, theoretically used by several threads) and <= 20 log-files as txt (size should not exceed 10MB each, otherwise a new file is created)

so in general there should not be more than 21 files on the sd-card

some other interesting problem: the test-file with 440 bytes contains 35 lines. reading them with line by line (by using ff_gets in a while-loop) increases the reading time from 1.6ms to 11ms. I’m absolutly awear of some overhead, but not in this dimension…

4

I wrote up all I know about SD card performance tuning here: Appendix D: Performance Tuning Tips. However, this looks like there is a more fundamental problem.

Some things to consider:

  • Are you sure that the task is at a sufficient priority?
  • Try porting and running one or more of the examples on your hardware. If you run the command_line example you could compare the performance with my results in Performance.
  • Try stripping down a copy of your project to a single FreeRTOS Plus FAT oriented task and see how that performs. Perhaps make it do the kinds of things that bench.c or big_file_test.c do.
  • Can you put a ‘scope on the (SPI?) bus to see if it is actually clocking at the speed it should? Or, failing that, maybe a $12 USB logic analyzer?

Are you running multi-core? Let’s have a look at your FreeRTOSConfig.h.

5

I do several things:

  • Signal all data producing tasks to stop and close their files (I use Event Groups to know when they’re finished)
  • FF_FS_Remove(mount_point)
  • FF_Invalidate(pxIOManager)
  • FF_FlushCache(pxIOManager)
  • FF_Unmount(pxDisk)

I get only 500 ms or so of warning, so I’d like this complete as quickly as possible. I have long wondered if some of these steps are redundant and if some are unnecessary if we’re going down anyway.

6

Thank you @P51D for the config file.

Thank you @carlk3 for the extensive description of performance.

In my experience, it is worth buying high-quality SD-cards like Ultra.x. They can be faster, and also they retain the data for a longer period of time.

When you can choose between SPI and SDIO, take the latter. Much better.

I have once written an SPI driver for SD-cards, but I was never satisfied. I couldn’t increase the speed. At that point I decided not to support SPI access to SD-cards any more.

On the contrary, ‘serial’ flash memory like e.g. Spansion is a very good choice for industrial applications. SD-cards can get problems of corrosion and small insects.
Serial flash memory is soldered on the print. It has a very simple but efficient 1-wire interface. SD-cards have a complex interface, both for SPI and SDIO.

to use preallocated files filled with 0s for improving performance and increase power loss stability → how should this be done so that the correct filesize is returned (e.g. by ff_stat or ff_filelength), for allocating reading buffer?

This solution works with a file that always has the same size. So that the FAT is accessed read-only. You will need another method of keeping track of where the last data were written.

to hold the file open (e.g. for high speed logging) → how should this be realized in a real world scenario, when multiple tasks are using different files (each task doesn’t know the others)?

Not sure if I understand your question…
Multiple tasks can use the same SD-card and the driver makes sure that all access is protected with a mutex.
It is also possible that two or more tasks get R/W access to the same directories or even the same files.
But still I recommend to make rules like giving each task its own directory. Make it impossible for the tasks to disturb each other.
If two tasks write to the same log file, I would create a logging task which has unique access to the log file, and which runs at a low priority.
I have a logging task using UDP here.

holding files open lead to the question, how to deal with it in case of power-loss detection (e.g. GPIO interrupt → save remaining data and flush it out, close the open files…)

@carlk3 already responded to this, describing a power system that has 500 ms spare time, just enough to close all files and flush the cache.

Reading and writing with SDIO: the times of R/W operations are not linear if you compare them with the number of bytes transferred. The more bytes in a single operation, the better the average access time. Each read or write operation has an overhead, the starting and the stopping. This overhead is the same whether you read a single sector or many sectors.

In SPI mode, there is a kind of multi-sector R/W, but it is not as effective because it uses polling.

7

@carlk3

  • I’m currently using only one Task for this tests with maxPriority-1 → there shouldn’t be any limitts
  • reading back the SPI configuration results at a Clock-Speed of 12.5MHz, but I will check this with a logic analyzer. With 12.5MHz the raw data block should be transmitted in ~0.3ms. So In my oppinion open/reading/closing this file should never be more than 1ms (with adding a unknown processing time).

@htibosch

  • I will check an other SD-Card with a different quality grade, to see if this is the root cause
  • what confuses me in addition is the unpredictable timing behaviour (e.g. reopening the same file takes once the double time, onse only ~1/3). And I’m not sure where this comes from…
  • I’m awear that SDIO would be the better choise (I think about 4x faster), but my hardware is limitted to free availability of an SPI.
8

It doesn’t answer your question; but FYI it’s entirely possible to run FatFs under FreeRTOS. We’ve been doing it for years. Recently we changed the FatFs sector buffers to a LRU scheme to improve performance.

We use DMA for all sector transfers. When writing to an SD card, if you want to get a good speed when writing large files, it’s essential to write blocks as large as possible. For this reason, when writing data files we generally buffer up 8K or 16K of data before calling FatFs to write it. FatFs will write this to SD card in a single operation if the write operation is aligned on a sector boundary. Another trick to getting good speeds (at the expense of wasting SD card space if you use lots of small files) is to make the cluster size as large as possible. We recommend 64kb clusters to our customers.

When appending to log files, we flush the data using FatFs f_flush calls every 15 seconds, if any data has been appended since the last flush. This ensures that the log file is at most 15 seconds out of data if power is lost.

1 Like
9

I’m awear of this - for example reding the file as block and process it (e.g. line by line) is mutch faster than reading just line by line.

My current approach is

  • for the smal config file → cach it (this 450 Bytes aren’t currently the problem)
  • reading the log-file → reading and processing chunks instead of lines (is ~10x faster)
  • writing the log-file → by chunks about 100-200 at once.

On the other side I will invest some time to dive into the performance optimizations.

10

See the SD-cards as small computers. They have an internal program that is managing the flash. This program will say the card is busy as long as it is working.

I will check an other SD-Card with a different quality grade, to see if this is the root cause

Well, not the root cause, but it is recommended. I used SD-cards for many years and we saw the customer complains: it is cheaper to use more expensive cards.

my hardware is limited to free availability of an SPI

That’s a pity

11

I am surprised there is so much difference in the speed when reading in chunks of less than a sector size (512b) instead of line-by-line. I would expect the FreeRTOS+Fat implementation to keep the last read sector in a buffer, so reading lines one by one should only require a physical sector read once per sector - provided that an intervening call to the file system (perhaps from another task) doesn’t re-allocate the sector buffer to use with a different file. Even in that case, an LRU buffering scheme should be able to retain data for a few files, depending on the number of sector buffers allocated.

With no other concurrent SD card activity, we achieve write speeds of around 2.5 Mbytes/sec (10Mb file written in 500b chunks to an 8kb buffer before passing to FatFs) and read speeds around 0.9Mbytes/sec (10Mb file read in 500b chunks directly from FatFs).

12

FatFs indeed works on FreeRTOS. I have used it for a while. They are quite comparable, except that I found that FatFS code is hard to read. When something went wrong, I had a hard time finding out the cause.

What I like about FreeRTOS+FAT is the possibility of “mounting” a file system on a directory. For instance, you can create a RAM disk and mount it as /ram. Or you can mount several partitions of an SD-card as e.g. /part_1, /part_2, and /part_3.

When using FatFs, I had to assign a driver number to each filesystem, which is a bit incompatible with the Linux method:

.
..
/part_1
/part_2
/part_3
/ram
/other

In this example, only /other is a directory in the root.

13

I’m curious about whether anyone has written a FreeRTOS+FAT media driver for raw flash. Presumably, it would have to do things like wear leveling, bad block remapping, address translation, caching… Interesting idea, though. Like you said, an SD card is doing all that onboard, so this would just be moving that processing to the media driver, running on the MCU.

EMBEDDED MULTI-MEDIA CARD (e•MMC) looks like a good compromise. It, too, can be soldered on a printed circuit board. It uses an interface similar to SDIO. In fact, many eMMC modules can be placed in a simple adapter to fit in an SD card slot and act like an SD card. It has bus modes with widths from one to eight. If GPIOs are tight, one can run 1-bit SDIO. If GPIOs are no limitation, one can run twice the bandwidth of SD card by using an 8-bit bus. Unlike SD, eMMC can do Double Data Rate (DDR) in a 3.3 V bus mode.

I have been playing around with one of these: EMMC Module 64GB 32GB 16GB 8GB with Micro SD-compatible Turn eMMC Adapter. Unfortunately, the bus timing is just different enough that I haven’t yet been able to get it to work with my driver.

14

I use FatFs for some things, but it does have its limitations. FatFs has some support for “re-entrancy” (i.e., thread or “task” safety), but it does not appear to have sufficient FAT and directory locking to make operations like f_mkdir, f_chdir and f_getcwd thread safe, whereas FreeRTOS+FAT does. Take a look at ff_locking.c and vMultiTaskStdioWithCWDTest in ff_stdio_tests_with_cwd.c. I ported vMultiTaskStdioWithCWDTest to FatFS and it fails miserably unless fsTASKS_TO_CREATE == 1. For some applications that could be important.

15

Well, FatFs can fake it with what ChaN calls “Unix style drive prefix”:

When FF_STR_VOLUME_ID == 2, Unix style drive prefix can be used. e.g. “/flash/file1.txt”, “/ram/temp.dat” or “/sd”. If a heading separator is exist, it is treated as an absolute path with a heading volume ID. Any form as “root directory in current drive” and “current directory in specified drive” cannot be used. “..” cannot traverse the volumes such as “/flash/../ram/foo.dat”.

See unix_like for an example.

16

Carl, thanks for your comments on FatFs and multithreading.

We don’t use f_chdir or f_getcwd because we don’t need a “current directory” concept except in some of the higher levels of our embedded software, also having a current directory that is common to all tasks makes little sense to me.

I took a look at f_mkdir and it appears to me to be threadsafe if FF_FS_REENTRANT is set, because f_mkdir calls mount_volume which calls lock_volume.

I’ve never taken a deep dive into FreeRTOS+FAT because our project predates it by several years. However, if as this thread suggests it really does perform less well than FatFs (other things being equal) then that would be a reason for us to stick with FatFs. FatFs does of course support ExFAT optionally as well, but given Microsoft’s current stance on licensing ExFAT, that is of little use to most people.

17

I think that both FatFs and +FAT have been highly optimised. Much depends on the hardware and the driver. When you use a fast SD-card, SDIO and DMA, you won’t see a big difference in speed.

With 12.5MHz SPI-Clock

ff_fread = 1.6ms

Is 1.6ms the time it takes to read a sector of 512 bytes?

18

It has been years since I looked at this. Maybe the problem is not reentrency, but that FatFs doesn’t use thread-local storage, so with FatFs you end up with a current directory that is common to all tasks, but with FreeRTOS+FAT each task gets its own.

The performance could also be affected by the driver implementation. I have a media driver for FatFS and one for FreeRTOS Plus FAT. They share a lot of code, but there are differences. A key one that might significantly impact performance is how the driver waits for an SPI transfer to complete. The spi_transfer_wait_complete function waits until the SPI master completes the transfer or a timeout has occurred. See

In the FatFs driver this is a busy wait in a tight loop, but the FreeRTOS Plus FAT driver uses ulTaskNotifyTakeIndexed to wait for a notification from an Interrupt Service Routine (ISR) that is triggered by the SPI DMA. This gives FreeRTOS the opportunity to do a context switch if another task is ready to run. The busy wait is faster as far as I/O to the SD card, but could be bad for overall system performance if the processor could be running another task instead of spinning. The busy wait could also cause more jitter for something like a high priority near-real-time task that needs to run on a schedule. It’s really a trade-off that depends on the application. My assumption was that for most FreeRTOS applications, I/O to the SD card is secondary to the primary mission. I guess there could be some kind of #define or configuration parameter to choose.

The ExFAT support is a big argument in favor of FatFs. More and more SD cards come formatted with ExFAT, and it’s a pain to have to reformat brand new cards. How long until ExFAT becomes unencumbered?

19

And configuration (e.g., FreeRTOSConfig.h and FreeRTOSFATConfig.h). For example, is SMP enabled so that both cores of the Raspberry Pi Pico are used?

And build options. There are lots and lots of asserts and things that go away in an optimized build.

20

That is not correct, in FreeRTOS+FAT, the “current working directory” has an instance per task. Also there is an “errno”, which has an instance per task. It is implemented as a function stdioGET_ERRNO()Both features use “Thread Local Storage Pointers”.

When using ff_stdio.h, +FAT becomes quite compatible with the normal stdio.h, except for the ff_ prefix, as in ff_fopen().