freeRTOS-Plus-FAT file library - corrupted files

I am using the freeRTOS-Plus- FAT and occasionally files are corrupted. I have written my own low-level SPI driver and it all works for a number of days with no problems.

I was just wondering if anybody else has experienced corrupted files?

I’ve tried increasing the thread stack space and checked as mush as I can.

Thanks in advance.

Hi Stephen, thanks for reporting this.

I have tested the FreeRTOS+FAT library many times by FTP-ing like 50,000 files with random sizes and complex long file names.
I then had the SD-card tested on a Linux machine using dosfsck. There were no errors, nor warnings.
Also I read back the entire disk and compared those files with the originals. They were all equal.

Make sure that you’re using the latest release of FreeRTOS+FAT.

I have worked with several SPI SD-card drivers and that gave me many head-aches. It took a long time before I got them stable (both under Linux as well as FreeRTOS).
The 4-bit interfaces are not only faster, but also much more stable.

What platform (MCU) are you using? What clock speed do you use for SPI?
How fast is the MCU?

Can you device be switched off while storing data?

Dear Hein,

Thanks for your response. The answers to your questions are:

MCU = STM32L071
SPI speed = 2MHz
MCU speed = 6 MHz

I know the speeds are low, but it is an embedded project using batteries.

Also,

freeRTOS v10.0.1

FAT-Plus build is 160919

Regards,
Stephen.

I know the speeds are low

No problem with that, I was just wondering if there aren’t any under- or overflows in your SPI peripheral. Does your library log all read/write errors that occur?

Would it be possible to lower the SPI speed to 1 MHz and see if there are still problems?

FAT-Plus build is 160919

Are you using the old 2016 release? It might be important to upgrade to today’s release. Can you upgrade your project to mentioned version?

but it is an embedded project using batteries

That’s a real challenge, I always like doing that.

Latest is here in the git repo: https://github.com/FreeRTOS/FreeRTOS-Labs/tree/master/FreeRTOS-Labs/Source/FreeRTOS-Plus-FAT

Thanks Hein for your answers.

I will update to the latest version when I have the chance.

Slowing down the SPI clock is easy to try.

A question : is it better to close the file after every write? I have to write to the file every 10 seconds so the file is open for a number of write cycles (500) and then closed. A new file is then opened.

Thanks,
Stephen.

Would it be possible to do the following:

  • Collect your data in RAM, for e.g. 5 minutes
  • Then Open/Write/Close the file(s), and clear the buffer(s)

The above solution has the disadvantage that at most N minutes of data may get lost in case of a power fail.
The advantage is that a file is only open for a minimum amount of time.

Within FreeRTOS+FAT, changes to a file are only flushed when the file is closed.

A new file is then opened.

Make sure that a directory never contains too many files: that could slow down the system enormously. FAT is not efficient when using large directories ( e.g. more than 1K files ).

Dear Hein,

I did not mention about error handling. I check for errors after calling the FF_xxx() functions. I also have asserts in my low-level SD card and SPI driver. To date, no errors have been reported.

As for RAM, there is only 20k and 16k has been used.

Regards,
Stephen.

But are you able to implement this approach? I means less frequent access and leaving the file handle open for a very short time.

Slowing down the SPI clock is easy to try.

Worth trying, for sure. I just spent a couple days debugging an error in my block device driver that only showed up when I was running multiple tasks to multiple SD cards. Eventually, I tried halving the SCK frequency from 12.5 MHz to 6ish, and, whaddyaknow? Problems went away. I guess the spec. puts the top limit at 10 MHz, so I was living dangerously.

Also, I highly recommend testing at low level with ChaN’s “Low level disk I/O module function checker” (http://elm-chan.org/fsw/ff/res/app4.c). I’ve attached my messy, hacked-up version.app4 - IO module function checker.c (11.9 KB)

Also, I am working on a FreeRTOS+FAT filesystem test suite where I have collected various tests from Demos and around the web. These include:

  1. ChaN’s “Low level disk I/O module function checker”
  2. A simple test adapted from “FATFileSystem example” at https://os.mbed.com/docs/mbed-os/v5.15/apis/fatfilesystem.html
  3. vCreateAndVerifyExampleFiles(), implemented in /FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_FAT_Demos/CreateAndVerifyExampleFiles.c
  4. vStdioWithCWDTest(), implemented in /FreeRTOS-Plus/Demo/Common/FreeRTOS_Plus_FAT_Demos/test/ff_stdio_tests_with_cwd.c,
  5. vMultiTaskStdioWithCWDTest, also in ff_stdio_tests_with_cwd.c.

I have gone so far as to run multiple instances of vMultiTaskStdioWithCWDTest simultaneously hammering on different cards.

I’ve hooked these up to CLI commands.