FreeRTOS+FAT: ff_ftell() Oddities

The behavior of ff_ftell() surprises me. The FreeRTOS+FAT Standard API Reference claims that it “Returns the current read/write position of an open file in the embedded FAT file system. The position is returned as the number of bytes from the start of the file.” But I don’t think that’s true, unless I’m misunderstanding something.

Here is a little test. I start by simply opening (with truncation) a file, writing 8 bytes, and closing:

19      // FF_FILE *ff_fopen( const char *pcFile, const char *pcMode );
20      /* “w”      Open a file for reading and writing.
21      If the file already exists it will be truncated to zero length.
22      If the file does not already exist it will be created.*/
23      FF_FILE *file = ff_fopen( pcFileName, "w" );
24      configASSERT(file);
25      char buf[8];
26      memset(buf, '1', sizeof(buf) - 1);
27      buf[sizeof(buf) - 1] = 0;
28      // size_t ff_fwrite( const void *pvBuffer, size_t xSize, size_t xItems, FF_FILE * pxStream );
29      size_t ni = ff_fwrite(buf, sizeof buf, 1, file);
30      configASSERT(1 == ni);
31      // long ff_ftell( FF_FILE *pxStream );
32      /* Returns the current read/write position of an open file in the embedded FAT file system.
33      The position is returned as the number of bytes from the start of the file.*/
34      printf("%d: ff_ftell(): %ld\n", __LINE__, ff_ftell(file));
35      // int ff_feof( FF_FILE *pxStream );
36      /*Queries an open file in the embedded FAT file system to see if the file’s read/write pointer is at the end of the file.*/
37      printf("%d: ff_feof(): %d\n", __LINE__, ff_feof(file));
38      // void ff_rewind( FF_FILE *pxStream );
39      ff_rewind(file);
40      printf("%d: ff_ftell(): %ld\n", __LINE__, ff_ftell(file));
41      printf("%d: ff_feof(): %d\n", __LINE__, ff_feof(file));
42      // int ff_fclose( FF_FILE *pxStream );;
43      int ec = ff_fclose(file);
44      configASSERT(!ec);

which produces this output:

34: ff_ftell(): 8
37: ff_feof(): 1
40: ff_ftell(): 0
41: ff_feof(): 0

So far, so good.

Now, I open the same file for reading and writing with append, and it gets surprising (to me):

46      /* “a+”     Open a file for reading and writing.
47      If the file already exists then new data will be appended to the end of the file.
48      If the file does not already exist it will be created. */
49      file = ff_fopen( pcFileName, "a+" );
50      configASSERT(file);
51      // int ff_feof( FF_FILE *pxStream );
52      /*Queries an open file in the embedded FAT file system to see if the file’s read/write pointer is at the end of the file.*/
53      printf("%d: ff_feof(): %d\n", __LINE__, ff_feof(file));
54      printf("%d: ff_ftell(): %ld\n", __LINE__, ff_ftell(file));
55
56      memset(buf, '2', sizeof(buf));
57      ni = ff_fwrite(buf, sizeof buf, 1, file);
58      configASSERT(1 == ni);
59      printf("%d: ff_feof(): %d\n", __LINE__, ff_feof(file));
60      printf("%d: ff_ftell(): %ld\n", __LINE__, ff_ftell(file));
61      ec = ff_fseek( file, 0, FF_SEEK_END );
62      configASSERT(!ec);
63      printf("%d: ff_feof(): %d\n", __LINE__, ff_feof(file));
64      printf("%d: ff_ftell(): %ld\n", __LINE__, ff_ftell(file));
65      ff_rewind(file);
66      printf("%d: ff_feof(): %d\n", __LINE__, ff_feof(file));
67      printf("%d: ff_ftell(): %ld\n", __LINE__, ff_ftell(file));
68      while (!ff_feof(file)) {
69          ni = ff_fread(buf, sizeof buf, 1 , file);
70          if (1 == ni) {
71              printf("%.*s", sizeof(buf), buf);
72          } else {
73                  int error = stdioGET_ERRNO();
74                  printf("%d: read: %s (%d)\n", __LINE__, strerror(error), error);
75          }
76      };
77      putchar('\n');
78      ec = ff_fclose(file);
79      configASSERT(!ec);

which produces:

53: ff_feof(): 0
54: ff_ftell(): 0

Hmm, “If the file already exists then new data will be appended to the end of the file.”, but apparently the “current read/write position of an open file” is 0 in this case.

59: ff_feof(): 1
60: ff_ftell(): 16

So, now I’ve written 8 bytes (from a read/write position 0) and now I’m at 16? I mean, I should be, if the file is really opened for append, but how was the position 0 when I started?

63: ff_feof(): 1
64: ff_ftell(): 16

FF_SEEK_END made no difference, as expected.

66: ff_feof(): 0
67: ff_ftell(): 0
111111122222222

If I ff_rewind(), I’m back to a new position 0, apparently, and I can read all 16 bytes.

This caused a bug for me when I relied on ff_ftell() to determine whether a file opened in mode “a+” was already existing (and new data will be appended to the end of the file) or it was newly created and empty. (ff_feof() is no more helpful in this case).

I’ve gotten in the habit of putting an ff_fseek( file, 0, FF_SEEK_END ) before every ff_ftell().

If I read this correctly, the anomaly seems to be opening a file with a+ and then calling ff_tell() is reporting the position as 0, whereas it should be [in your posted case] 8. Is that correct, or are there other anomalies?

I just tried this on Windows using Visual studio and the Windows file system and got the same results as yourself - so the +FAT file system seems to be consistent with the Windows file system - so what you are seeing, if I understand it correctly, is probably the expected behaviour. The code I ran was:

FILE *f;
char buffer[ 8 ] = { 0 };

	f = fopen( "c:/temp/delete/myfile.txt", "w" );
	configASSERT( f );

	printf( "ftell for new file = %d\r\n", ftell( f ) );
	fwrite( buffer, sizeof( buffer ), 1, f );
	printf( "ftell after writing 8 bytes = %d\r\n", ftell( f ) );
	fclose( f );

	f = fopen( "c:/temp/delete/myfile.txt", "a+" );
	configASSERT( f );
	printf( "ftell for reopened file = %d\r\n", ftell( f ) );
	fwrite( buffer, sizeof( buffer ), 1, f );
	printf( "ftell after writing another 8 bytes = %d\r\n", ftell( f ) );
	fclose( f );

and the output I received was:

ftell for new file = 0
ftell after writing 8 bytes = 8
ftell for reopened file = 0
ftell after writing another 8 bytes = 16

The point to note being when I opened the file again with “a+” and did an ftell(), the returned value was 0, but after writing another 8 bytes, the returned value was 16.

@rtel : thanks for trying it out under Windows.

I think the behaviour is in line with the definition of the “a+” flag that I found here:

“a+” append/update:
Open a file for update (both for input and output) with all output operations
writing data at the end of the file. Repositioning operations (fseek, fsetpos,
rewind) affects the next input operations, but output operations move the
position back to the end of file. The file is created if it does not exist.

It means that a seek takes place implicitly before every output operation, i.e. ff_write(), and not after opening the file.
If you read from the file, the initial position will be zero.

I’ve gotten in the habit of putting an ff_fseek( file, 0, FF_SEEK_END )
before every ff_ftell().

Calling ff_fseek() is only necessary right after opening a file, and also after reading from the file in “append/update” mode.
Once you start appending data, the current file position will be at the end of the file.

Thanks, Richard and Hein!

I learn something new every day. I was expecting ftell() to give me an absolute offset from the beginning of the file at all times, but it looks like that expectation was wrong.

1 Like

I’m still confused. I’m running prvTest_ff_truncate in ff_stdio_tests_with_cwd.c. I picked this up from one of the FreeRTOS_Plus_FAT_Demos.

I’m getting Assertion "cChar == 0xff" failed: file "tests/ff_stdio_tests_with_cwd.c", line 690, function: prvTest_ff_truncate. Here is what it’s doing there:

623         /* Create a 1000 byte file. */
624         pxFile = ff_truncate( pcTestFileName, 1000L );
//...
631         /* The file should have been created full of zeros. */
632         pxFile = ff_fopen( pcTestFileName, "r" );
//...
661         /* Fill the file with 0xff. */
662         ff_fclose( pxFile );
663         pxFile = ff_fopen( pcTestFileName, "r+" );
664         configASSERT( pxFile != NULL );
665 
666         for( x = 0; x < 1000; x++ ) 
667         {
668                 configASSERT( ff_fputc( 0xff, pxFile ) == 0xff );
669         }
670 
671         /* Extend the file to 2000 bytes using ff_truncate(). */
672         ff_fclose( pxFile );
673         pxFile = ff_truncate( pcTestFileName, 2000L );
//...
681         /* Now the first 1000 bytes should be 0xff, and the second 1000 bytes should
682         be 0x00. */
683         pxFile = ff_fopen( pcTestFileName, "r+" );
//...
687         for( x = 0; x < 1000; x++ )
688         {
689                 cChar = ff_fgetc( pxFile );
690                 configASSERT( cChar == 0xff );
691         }
692 
693         for( x = 0; x < 1000; x++ )
694         {
695                 cChar = ff_fgetc( pxFile );
696                 configASSERT( cChar == 0x00 );
697         }

Since the ff_fopen at line 663 is “r+”, wouldn’t the file pointer be at the end of file? So, the ff_truncate at 673 really does nothing? And the first 1000 bytes are 0xff?

EDIT:

I looked it up, and FreeRTOS+FAT Standard API Reference says

“r+” Open the file for reading and writing.

so the file position seems unspecified, but I suppose would be assumed to be the beginning.

I should mention that vStdioWithCWDTest runs fine. I only see this problem when running vMultiTaskStdioWithCWDTest with two or more tasks.

After the assertion, if I pop the SD card into a PC, directory /0 contains:

total 224
-rw-r--r-- 1 carlk carlk   200 Feb 26 05:27 root001.txt
-rw-r--r-- 1 carlk carlk   400 Feb 26 05:27 root002.txt
-rw-r--r-- 1 carlk carlk   600 Feb 26 05:27 root003.txt
-rw-r--r-- 1 carlk carlk   800 Feb 26 05:27 root004.txt
-rw-r--r-- 1 carlk carlk  1000 Feb 26 05:27 root005.txt
drwxr-xr-x 3 carlk carlk 32768 Feb 26 05:27 SUB1
-rw-r--r-- 1 carlk carlk  2000 Feb 26 05:27 truncate.bin

and xxd truncate.bin prints:

00000000: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000010: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000020: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000030: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000040: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000050: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000060: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000070: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000080: ffff ffff ffff ffff 00ff ffff ffff ffff  ................
00000090: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000000a0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000000b0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000000c0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000000d0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000000e0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000000f0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000100: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000110: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000120: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000130: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000140: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000150: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000160: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000170: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000180: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000190: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000001a0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000001b0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000001c0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000001d0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000001e0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000001f0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000200: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000210: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000220: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000230: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000240: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000250: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000260: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000270: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000280: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000290: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000002a0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000002b0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000002c0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000002d0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000002e0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000002f0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000300: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000310: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000320: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000330: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000340: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000350: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000360: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000370: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000380: ffff ffff ffff ffff ffff ffff ffff ffff  ................
00000390: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000003a0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000003b0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000003c0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000003d0: ffff ffff ffff ffff ffff ffff ffff ffff  ................
000003e0: ffff ffff ffff ffff 0000 0000 0000 0000  ................
000003f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000400: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000410: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000420: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000430: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000440: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000450: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000460: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000470: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000480: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000490: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000500: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000510: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000520: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000530: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000540: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000550: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000560: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000570: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000580: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000590: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000600: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000610: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000620: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000630: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000640: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000650: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000660: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000670: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000680: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000690: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000006f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000700: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000710: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000720: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000730: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000740: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000750: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000760: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000770: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000780: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000790: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000007c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................

Oh, I think I found it. At some point I had turned on configUSE_TIME_SLICING. I could see with my logic analyzer that a DMA transfer from SD card over SPI was being interrupted. Apparently, that’s not good. So, I guess I need to add something like vTaskSuspendAll and xTaskResumeAll around my SPI transfer function.

Update: vTaskSuspendAll and xTaskResumeAll don’t seem to be a good solution. It really breaks my scheme of doing a ulTaskNotifyTake to wait for a DMA complete interrupt. Surely, the DMA can run independently and let the processor do other things. The answer, for now, is to just avoid time slicing. But I don’t really understand why time slicing causes a problem. I do have a Mutex locking the whole SPI for the duration of a read or write, so I don’t think another task would be messing with the DMA.

Carl, all access to the SD-card is protected by a mutex. How come that the SPI communication gets interrupted?

Now, I am doubting that it is. Does the vMultiTaskStdioWithCWDTest pass on other platforms when configUSE_TIME_SLICING is 1?

Here is my code: FreeRTOS-FAT-CLI-for-RPi-Pico/spi.c at master · carlk3/FreeRTOS-FAT-CLI-for-RPi-Pico · GitHub

I don’t yet have a debugger in my development environment. I’m working on the Raspberry Pi Pico, and I ordered a second one to use as a “Picoprobe”, but it has been back ordered for ages. I’ve decided to put time slicing on the back burner until that arrives.

I believe so. configUSE_TIME_SLICING defaults to 1 and I rarely run without time slicing.

@rtel wrote:

I believe so. configUSE_TIME_SLICING defaults to 1 and I rarely run without time slicing.

Yes, we enabled time-slicing when testing the vMultiTaskStdioWithCWDTest(). The test will start a couple of FreeRTOS tasks, all running on the IDLE priority, thus sharing the CPU in slices of 1 msec. That was a very good test written by @rtel.

Your SPI driver does not have to worry about locking, you can be sure the driver functions fnWriteBlocks()and fnReadBlocks() will never be called when another instance is still busy. The second task shall wait.

As you must have seen, locking is done in ff_locking.c, for instance in this function:

void FF_PendSemaphore( void *pxSemaphore )
{
	if( xTaskGetSchedulerState() != taskSCHEDULER_RUNNING )
	{
		/* No need to take the semaphore. */
		return;
	}
	configASSERT( pxSemaphore );
	xSemaphoreTakeRecursive( ( SemaphoreHandle_t ) pxSemaphore, portMAX_DELAY );
}
/*-----------------------------------------------------------*/

So you do not need to stop the scheduler or use task-notify to prevent DMA from getting interrupted. As long as you are executing fnWriteBlocks() or fnReadBlocks(), no interruption should take place. Does that make sense?

There are some things that I don’t understand in ff_locking.c :

153                 /* Called when a task want to make changes to a directory.
154                 First it waits for the desired bit to come high. */
155                 xEventGroupWaitBits( pxIOManager->xEventGroup,
156                         FF_DIR_LOCK_EVENT_BITS, /* uxBitsToWaitFor */
157                         ( EventBits_t )0,       /* xClearOnExit */

Why the cast? This parameter should be const BaseType_t xClearOnExit.

158                         pdFALSE,                /* xWaitForAllBits n.a. */
159                         pdMS_TO_TICKS( 10000UL ) );
160 
161                 /* The next operation will only succeed for 1 task at a time,
162                 because it is an atomary test & set operation: */
163                 xBits = xEventGroupClearBits( pxIOManager->xEventGroup, FF_DIR_LOCK_EVENT_BITS );

The comment on lines 161 and 162 seems contradictory to this explanation in Mastering the FreeRTOS Real Time Kernel; A Hands-On Tutorial Guide; Richard Barry; Pre-release 161204 Edition:

Event bits can be cleared using the xEventGroupClearBits() API function, but using that
function to manually clear event bits will lead to race conditions in the application code if:

  • There is more than one task using the same event group.
  • Bits are set in the event group by a different task, or by an interrupt service routine.

The xClearOnExit parameter is provided to avoid these potential race conditions. If
xClearOnExit is set to pdTRUE, then the testing and clearing of event bits appears to the
calling task to be an atomic operation (uninterruptable by other tasks or interrupts).

The same code is repeated a couple of more times. Next,

315 void FF_BufferProceed( FF_IOManager_t *pxIOManager )

322         /* Wake-up all tasks that are waiting for a sector buffer to become available. */
323         xEventGroupSetBits( pxIOManager->xEventGroup, FF_BUF_LOCK_EVENT_BITS );

The comment seems misleading to me. I would expect it to wake up one task. Or, does this have something to do with the way xClearOnExit is set to false, above?

UPDATE: Here is my proposed rewrite: ff_locking.c (8.9 KB)

I’m still struggling with this problem. My latest theory is that it is a problem in the Pico runtime or hardware. See Context switching breaks SPI/DMA.

@carlk3 wrote:

 xBits = xEventGroupWaitBits( pxIOManager->xEventGroup,
         FF_DIR_LOCK_EVENT_BITS, /* uxBitsToWaitFor */
-        0,                      /* xClearOnExit */
+        pdTRUE,                 /* xClearOnExit */
         pdFALSE,                /* xWaitForAllBits n.a. */
         pdMS_TO_TICKS( 10000UL ) );

It has always worked well, but you are right about the parameters: we’d better just call xEventGroupWaitBits() with xClearOnExit set to pdTRUE, so there is no need to clear the bit manually.

Thanks for your suggestion.

I will first wait for PR #5 to get merged, and then create a PR for ff_locking.c
With PR #5 it will be possible to use other sector sizes than 512 bytes.

You also wrote:

I’m still struggling with this problem. My latest theory is that it is a problem in the Pico runtime or hardware. See Context switching breaks SPI/DMA.

Have you checked all status and error registers that belong to the SPI and DMA? In my experience you can not get corrupt data without some error bit going high. I am thinking of an under- or overrun error.

I would also be curious to see if the errors still occur at a lower bus SPI speed?

Hi Hein,

Yeah, I was excited when I saw a potential race condition there, but fixing it made no difference to my test results.

Good point. The one thing I haven’t tried is the built in DMA Sniffer. The RP2040 can watch data from a given channel passing through the data FIFO, and calculate checksums based on this data. (https://datasheets.raspberrypi.org/rp2040/rp2040-datasheet.pdf) I’ll have to play with that and also see if there are any other interesting registers I can probe.

Yes, this is one of those ugly problems that changes behavior when a butterfly in Argentina flaps its wings. Sometimes lowering the SPI speed helps, but in my main test configuration I still get failures even at very low baud rates.

Welp, turns out that the problem has to do with the Pico’s hardware divider. The partial divider state must be correctly saved and restored when switching context. Since I was using a generic ARM_CM0 FreeRTOS port, that wasn’t happening. So, I am looking at enhancing xPortPendSVHandler. For now, if I build with pico_set_divider_implementation(TARGET compiler) in CMakeLists.txt, everything works great with time slicing enabled.

That is interesting. I would be grateful if you could keep us informed so we can see if there is anything we can do to upstream the changes. For example, the RISC-V port has a method that enables users to add additional registers into the set of registers saved and restored as the task context and it may be possible to do the same here.

I have a working solution:

but, apparently, Raspberry is working on their own port that will have a more efficient solution. See Context switching breaks SPI/DMA - Page 2 - Raspberry Pi Forums

Raspberry’s port is on "the FreeRTOS github

https://github.com/FreeRTOS/FreeRTOS-Kernel/tree/main/portable/ThirdParty/GCC/RP2040

or

https://github.com/FreeRTOS/FreeRTOS-Kernel/tree/smp/portable/ThirdParty/GCC/RP2040 for SMP

and standard demos (plus a few test/example sorts of thing)

https://github.com/FreeRTOS/FreeRTOS/tree/main/FreeRTOS/Demo/ThirdParty/Community-Supported/CORTEX_M0%2B_RP2040"

according to kilograham, Raspberry Pi Engineer & Forum Moderator:
https://www.raspberrypi.org/forums/viewtopic.php?f=144&t=316786#p1899886