FreeRTOS + FAT FF_FlushCache Function Implementation

kydong2019 wrote on Monday, May 06, 2019:

Hello

Recently, I am studying the FreeRTOS + FAT source code FreeRTOS-Plus-FAT-160919a-MIT.zip.

I have some confusion for the implementation of the function
FF_FlushCache() >

When after flush the write cache buffer, why it’s needed to search for other buffers that used this sector ?
Because if the only method to get buffer is by the function <FF_GetBuffer()>, then the <FF_GetBuffer()> make sure that there is only one buffer corresponds to one sector, there isn’t a possibility that would make more than one buffer to correspond to one sector.

heinbali01 wrote on Monday, May 06, 2019:

Hi Kevin, you’re right that FF_GetBuffer() is the only function issuing caching buffers.
It is also true that a given sector can either have a READ, or a WRITE buffer.
And finally: a READ buffer can be shared by many, a WRITE sector has temporary exclusive access.

I also think that this if clause never succeeds:

FF_Error_t FF_FlushCache( FF_IOManager_t *pxIOManager )
{
    <snip>
    /* Search for other buffers that used this sector, and mark them as modified
    So that further requests will result in the new sector being fetched. */
    for( xIndex2 = 0; xIndex2 < pxIOManager->usCacheSize; xIndex2++ )
    {
        if( ( xIndex != xIndex2 ) &&
            ( pxIOManager->pxBuffers[ xIndex2 ].ulSector == pxIOManager->pxBuffers[ xIndex ].ulSector ) &&
            ( pxIOManager->pxBuffers[ xIndex2 ].ucMode == FF_MODE_READ ) )
        {
            pxIOManager->pxBuffers[ xIndex2 ].bModified = pdTRUE;
        }
    }

and I think that the code can be removed without getting any side-effects.
Thanks for taking the effort to report this.

kydong2019 wrote on Tuesday, May 07, 2019:

Hi Hein

Thanks for your confirmation.

kydong2019 wrote on Tuesday, May 07, 2019:

Hi Hein

The usage for CacheMemory imposed by FF_GetBuffer() is: a Read buffer can be shared by many, a Write sector has temporary exclusive access.

But the function prvGetFromFATBuffers( ) violate this usage which let Write FATBuffers can be shared. and FATBuffers are initially got by FF_GetBuffer().

So could I know why FATBuffers processed in such way ?

FF_Buffer_t *prvGetFromFATBuffers( FF_IOManager_t *pxIOManager, FF_FATBuffers_t *pxFATBuffers, BaseType_t xBufferIndex,
	uint32_t ulFATSector, FF_Error_t *pxError, uint8_t ucMode )
{
        <snip>
			if(
				( pxBuffer->ulSector == ulFATSector ) &&
				( ( ( ucMode & FF_MODE_WRITE ) == 0 ) ||
				  ( ( pxBuffer->ucMode & FF_MODE_WRITE ) != 0 ) )
			)
			{
				/* Same sector, AND
				write-permission is not required OR the buffer has write permission:
				it can be reused. */
				#if ffconfigFAT_USES_STAT
				{
					fatStat.reuseCount[ ( ucMode & FF_MODE_WRITE ) ? 1 : 0 ]++;
				}
				#endif /* ffconfigFAT_USES_STAT */
			}
			
}

heinbali01 wrote on Tuesday, May 07, 2019:

Kevin, you’re diving deep into it :slight_smile:

A task that calls prvGetFromFATBuffers() is reading from, and/or writing to the FAT area.

It has a buffer of the type FF_FATBuffers_t, which contains either 1 or 2 cache buffers, depending on whether a shadow FAT must be maintained.

In an earlier version, prvGetFromFATBuffers() did not exists yet, and FF_GetBuffer() was called directly.

It turned out useful to reuse buffers whenever possible. You also see that the usefulness was empirically tested by maintaining statistics ( ffconfigFAT_USES_STAT ).

The rule that I mentioned earlier:

  • A WRITE buffer can only have a single user
  • A READ buffer can have multiple users
  • You can not obtain a WRITE buffer as long as a READ buffer exists ( and has owners )
  • You can not obtain a READ buffer as long as a WRITE buffer exists ( and has an owner )

prvGetFromFATBuffers() is obeying the rules except one situation: when it is looking for READ access and it finds that it already owns a WRITE buffer for the same sector. That is considered OK. It saves this action:

	/* Release the buffer with FF_MODE_WRITE access. */
	FF_ReleaseBuffer( pxIOManager, pxBuffer );
	/* Get a buffer with READ access. */
	pxBuffer = FF_GetBuffer( pxIOManager, ulFATSector, FF_MODE_READ );

The above would be quite costly and have no advantage.

kydong2019 wrote on Wednesday, May 08, 2019:

Hi Hein

Although this method saves some costly action, but it also bypass the mechanism that around the pxMatchingBuffer->usNumHandles and pxMatchingBuffer->usPersistance. Does it OK?

Does the situation that when it is looking for READ access and it finds that it already owns a WRITE buffer for the same sector can only be appropriate to FATBuffer, or it can be appropriate to all CacheBuffer ?

heinbali01 wrote on Wednesday, May 08, 2019:

it also bypass the mechanism that around the pxMatchingBuffer->usNumHandles
and pxMatchingBuffer->usPersistance. Does it OK?

It does not bypass these rules, it is not a back-door. usNumHandles and usPersistance are maintained by FF_GetBuffer() and FF_ReleaseBuffer.

The only thing special that it can do is the combination of read/write on a WRITE buffer: which is very efficient when updating FAT sectors.

Something about WRITE buffers: when obtaining a buffer in WRITE mode, you can choose to either read the contents of the sector from disk, or have the buffer cleared.

When you write data to an ever growing file, it is useless to read newly allocated sectors: you will overwrite all data. In this situation, the pseudo attribute FF_MODE_WR_ONLY is used:

#define FF_MODE_READ      0x01  /* Buffer / FILE Mode for Read Access. */
#define	FF_MODE_WRITE     0x02  /* Buffer / FILE Mode for Write Access. */
#define	FF_MODE_WR_ONLY   0x42  /* Buffer for Write-only Access */

The FAT functions will typically not use the write-only mode, and always read the FAT sector before updating it.

But even if FF_MODE_WR_ONLY is being used, you may read back what you have just written to a cache buffer.

No your question:

Does the situation that when it is looking for READ access and it finds that it already
owns a WRITE buffer for the same sector can only be appropriate to FATBuffer, or it can
be appropriate to all CacheBuffer ?

Other functions that access the contents of files and directories, or special sectors can also read from a WRITE buffer.

kydong2019 wrote on Wednesday, May 08, 2019:

Hi Hein

Thanks for your explaination.
So why not change FF_GetBuffer as below to make life easy ?

FF_Buffer_t *FF_GetBuffer( FF_IOManager_t *pxIOManager, uint32_t ulSector, uint8_t ucMode ){
        /* A Match was found process! */
			if( ( ucMode == FF_MODE_READ ) && ( pxMatchingBuffer->ucMode == FF_MODE_READ ) )
			{
				pxMatchingBuffer->usNumHandles += 1;
				pxMatchingBuffer->usPersistance += 1;
				break;
			}
            
          if( ( ( ucMode & FFMODEWRITE ) == 0 ) ||
				        ( ( pxBuffer->ucMode & FFMODEWRITE ) != 0 )
			)
			{
				  pxMatchingBuffer->usNumHandles += 1;
				  pxMatchingBuffer->usPersistance += 1;
				  break;
			}
}

heinbali01 wrote on Wednesday, May 08, 2019:

No, that would violate the rule: “a WRITE buffer has at most one owner”.
It would allow to obtain a WRITE buffer that is already owned by another task.

kydong2019 wrote on Wednesday, May 08, 2019:

So In prvGetFromFATBuffers(), the FF_LockFAT and FF_UnlockFAT make sure the WRITE buffer is owned by one task,

In prvGetFromFATBuffers, it’s a scenario that the same task reuse the Write Buffer it previously used。

There isn’t a possibility that Two Tasks share the Write Buffer through prvGetFromFATBuffers

Do I understand right ?

kydong2019 wrote on Wednesday, May 08, 2019:

So In prvGetFromFATBuffers(), the FF_LockFAT and FF_UnlockFAT make sure the WRITE buffer is owned by one task,

In prvGetFromFATBuffers, it’s a scenario that the same task reuse the Write Buffer it previously used。

There isn’t a possibility that Two Tasks share the Write Buffer through prvGetFromFATBuffers

Do I understand right ?