ftp get truncates file

system · July 14, 2016, 3:42pm

rasty1 wrote on Thursday, July 14, 2016:

Hi,

FTP RETR command truncates files on RAM disk.
I managed to reproduce the same problem with CLI.
It happens if I add “//” before file name as FTP server does.
Beside this FTP hangs on get, dir.

Please see log below:

–>dir
test.txt [writable file] [size=2048] <<< File is OK
. [directory] [size=1024]
–>type test.txt
Hello word
–>type //test.txt
–>dir
test.txt [writable file] [size=0] <<** file is truncated**
. [directory] [size=1024]

target log:

–>type log.txt
test log
TCP socket on port 80
TCP socket on port
21
TPC-server: new FTP client
USER

PORT a041418ip:56940
Client address a041418ip:5694
0
RETR test.txt
FTP: sent: ‘//test.txt’ 0 B
ytes (0 bytes/sec)

Did anyone see anything like that?

Best regards
Rasty

rtel · July 14, 2016, 3:47pm

rtel wrote on Thursday, July 14, 2016:

I can try and replicate the behaviour here. Can you please provide more
information on how to duplicate it. I understand you are sending a file
that has // on the front, but how are you sending it? For example, is
this done from a command line or from an IDE. If it is done from a
command line please show the commands you are using.

system · July 14, 2016, 3:59pm

rasty1 wrote on Thursday, July 14, 2016:

It is very easy to duplicate.
Just create a file in RAM disk (say test.txt) and than in CLI use type command:
type //test.txt

system · July 14, 2016, 4:04pm

rasty1 wrote on Thursday, July 14, 2016:

Maybe small deviation from stock example:
#define mainRAM_DISK_NAME “/”
and it does not allow to mak a RAM disk smaller than 3 MB.

pxRAMDisk = FF_RAMDiskInit( mainRAM_DISK_NAME, ucRAMDisk, mainRAM_DISK_SECTORS, mainIO_MANAGER_CACHE_SIZE );
configASSERT( pxRAMDisk );

/* Print out information on the RAM disk. */
stat = FF_RAMDiskShowPartition( pxRAMDisk );
pxFile = ff_fopen( “test.txt”, “w” );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fclose( pxFile );

rtel · July 14, 2016, 4:07pm

rtel wrote on Thursday, July 14, 2016:

So this is not an FTP issue - from your first post I thought it was.
Simply trying to view a file after adding // to the front of the file
name. I will give that a try, although // is not a valid path to the
file it should not result in the file being truncated.

system · July 14, 2016, 4:14pm

rasty1 wrote on Thursday, July 14, 2016:

Actually I have an issues with FTP as well - it hangs on DIR, RETR command.

after DIR command it enters tight loop and I had to enter some delay in order to get some responce
for( ;; )
{
FreeRTOS_TCPServerWork( pxTCPServer, xInitialBlockTime );
vTaskDelay(10); <<-- added delay
}

Truncated file is something solid that managed to see and reproduce.

rtel · July 14, 2016, 4:37pm

rtel wrote on Thursday, July 14, 2016:

I just tried what I think you tried before, albeit with the head revision code rather than the release code. The output is below:

First, view a small file I FTPed onto the RAM disk, which is mounted in /

[Press ENTER to execute the previous command again]
>
dir

p.txt [writable file] [size=38]
. [directory] [size=1024]


[Press ENTER to execute the previous command again]

Then I view the file without the // on the front.

type p.txt

11111111111111111111111111111111111111

[Press ENTER to execute the previous command again]

Performing a directory listing shows the file has the same length:

dir

p.txt [writable file] [size=38]
. [directory] [size=1024]


[Press ENTER to execute the previous command again]

Then I do the same, but this time add the // to the front of the file name:

type //p.txt

11111111111111111111111111111111111111

[Press ENTER to execute the previous command again]

When I perform a directory listing again the file has not been truncated.

dir

p.txt [writable file] [size=38]
. [directory] [size=1024]


[Press ENTER to execute the previous command again]
>

system · July 15, 2016, 4:48am

rasty1 wrote on Friday, July 15, 2016:

I see a difference in behavior of dir
test.txt [writable file] [size=2048]
On my system it laways reports 2048 no matter what size is.

rtel · July 15, 2016, 8:11am

rtel wrote on Friday, July 15, 2016:

You mentioned you were using a RAM disk, but I didn’t see a mention of the target. Are you using the Windows demo or something else?

system · July 15, 2016, 10:30am

rasty1 wrote on Friday, July 15, 2016:

I’m using port to uBlaze.
Based on Version from 2015. I can check later exact version.
I managed to operate TCP/IP stack.

The difference from standard demo are

RAM disk mounted to “/”
no ssd
Ethernet MTU is set to 200 bytes at host side - I didn’t find how to limit MTU at freertos (or it did not work)
I implemented TCP/IP CLI rather than UDP

Problem that I’ve faced down the road

Cannot make RAM disk smaller than 3mb - format fails.
FTP hangs at dir, send, get - enters end-less tight loop. I mentioned this in prev post.
DIR via CLI reports incorrect size of the file (maybe some sympthom of the problem)

If you didn’t see anything like that before I’ll try to update to the latest version. What version have you mentioned in your post?

rtel · July 15, 2016, 10:38am

rtel wrote on Friday, July 15, 2016:

I’m using the head revision, which is not in a public repository, if you
send me your files (not your own code, just the FreeRTOS, FreeRTOS+FAT
and FreeRTOS+TCP files) then I can update those files to the latest
version and send them back. You can contact me on r dot barry at
freertos [dot] org.

system · July 15, 2016, 11:21am

rasty1 wrote on Friday, July 15, 2016:

Thank you very much for your help! I sent files.

system · July 15, 2016, 11:21am

rasty1 wrote on Friday, July 15, 2016:

Thank you very much for your help! I sent files.

rtel · July 15, 2016, 11:29am

rtel wrote on Friday, July 15, 2016:

I have updated the +TCP and +FAT code to the head revision of the single interface TCP version - pay no attention to the version number in the header files as its not release code just a working copy. It looks like you were actually using quite an old version. I have not updated any of the other files - you may also like to use FreeRTOS V9.0.0, which you can download from SourceForge here.

system · July 15, 2016, 11:34am

rasty1 wrote on Friday, July 15, 2016:

Thank you very much. I’ll check it on Sunday and let you know the results.

htibosch · July 17, 2016, 12:15am

heinbali01 wrote on Sunday, July 17, 2016:

Thanks for reportting this.

You wrote:

It happens if I add “//” before file name as FTP server does
–>type //test.txt
–>dir

Isn’t //test.txt an invalid file name?

+FAT does have code to handle relative paths, it also recognises invalid characters, but it doesn’t always detect invalid paths or file names.
I think it is even possible to create a file with an empty string as a file name.

For some people, this is a short-coming. For others it is a blessing because they want a fast and small library.

We could add some more code checking path validity and make this dependent on some macro ffconfigCHECK_PATH_VALIDITY

system · July 17, 2016, 8:31am

rasty1 wrote on Sunday, July 17, 2016:

I tested a new version, result is the same.
I think that something is wrong with ramdisk/filesystem configuration. DIR reports 2048 bytes, while in your example I see 28 bytes.
Can you send me you setup of RAMDISK?
–>dir
test.txt [writable file] [size=2048]
log.txt [writable file] [size=2048]
. [directory] [size=1024]
–>type test.txt
Hello word
–>type //test.txt

–>dir
test.txt [writable file] [size=0]
log.txt [writable file] [size=2048]
. [directory] [size=1024]

htibosch · July 17, 2016, 11:43am

heinbali01 wrote on Sunday, July 17, 2016:

Hi Rasty,

Do you have ffconfigPATH_CACHE enabled? It remembers the cluster number that belongs to a certain path.
You could try the same test with:

#define ffconfigPATH_CACHE    0

But once again, //test.txt is an invalid file name and I think that any API called with such an entry should return an error.
( at this moment I don’t have any hardware with a RAM-disk. Beginning of August I will replicate the problem and come back to it )

system · July 17, 2016, 1:30pm

rasty1 wrote on Sunday, July 17, 2016:

ffconfigPATH_CACHE is not defined by default.
I know that // is not valid, maybe it is symphom for other problem as well as reporting of wrong size of file.
BTW. Truncation does not happen and size is reported correctly if file is in sub-folder

htibosch · July 17, 2016, 2:32pm

heinbali01 wrote on Sunday, July 17, 2016:

I do have a device for testing with a RAM disk: the Windows demo project!

I just tried what you’ve done with a 4MB RAM-disk mounted on:

#define mainRAM_DISK_NAME    "/"

But I’m afraid I can not repeat your finding. This is what I saw when using the CLI commands:

    >dir
    ff_test [directory] [size=0]
    test.txt [writable file] [size=12]
    . [directory] [size=1024]

    >type test.txt
    Hello world

    >type //test.txt
    Hello world

    >dir
    ff_test [directory] [size=0]
    test.txt [writable file] [size=12]
    . [directory] [size=1024]

I tried both with and without the path caching (ffconfigPATH_CACHE).

Please find attached the file FreeRTOSFATConfig.h that I used for this test.