ftp get truncates file
Hi,
FTP RETR command truncates files on RAM disk.
I managed to reproduce the same problem with CLI.
It happens if I add “//” before file name as FTP server does.
Beside this FTP hangs on get, dir.
Please see log below:
–>dir
test.txt [writable file] [size=2048] <<< File is OK
. [directory] [size=1024]
–>type test.txt
Hello word
–>type //test.txt
–>dir
test.txt [writable file] [size=0] <<** file is truncated**
. [directory] [size=1024]
target log:
–>type log.txt
test log
TCP socket on port 80
TCP socket on port
21
TPC-server: new FTP client
USER
PORT a041418ip:56940
Client address a041418ip:5694
0
RETR test.txt
FTP: sent: ‘//test.txt’ 0 B
ytes (0 bytes/sec)
Did anyone see anything like that?
Best regards
Rasty
ftp get truncates file
I can try and replicate the behaviour here. Can you please provide more
information on how to duplicate it. I understand you are sending a file
that has // on the front, but how are you sending it? For example, is
this done from a command line or from an IDE. If it is done from a
command line please show the commands you are using.
ftp get truncates file
It is very easy to duplicate.
Just create a file in RAM disk (say test.txt) and than in CLI use type command:
type //test.txt
ftp get truncates file
Maybe small deviation from stock example:
define mainRAMDISKNAME “/”
and it does not allow to mak a RAM disk smaller than 3 MB. pxRAMDisk = FFRAMDiskInit( mainRAMDISKNAME, ucRAMDisk, mainRAMDISKSECTORS, mainIOMANAGERCACHESIZE ); configASSERT( pxRAMDisk ); /* Print out information on the RAM disk. */ stat = FF_RAMDiskShowPartition( pxRAMDisk ); pxFile = ff_fopen( “test.txt”, “w” ); ff_fwrite( test_str, sizeof(test_str), 1, pxFile ); ff_fclose( pxFile );ftp get truncates file
So this is not an FTP issue – from your first post I thought it was.
Simply trying to view a file after adding // to the front of the file
name. I will give that a try, although // is not a valid path to the
file it should not result in the file being truncated.
ftp get truncates file
Actually I have an issues with FTP as well – it hangs on DIR, RETR command.
after DIR command it enters tight loop and I had to enter some delay in order to get some responce
for( ;; )
{
FreeRTOS_TCPServerWork( pxTCPServer, xInitialBlockTime );
vTaskDelay(10); <<– added delay
}
Truncated file is something solid that managed to see and reproduce.
ftp get truncates file
I just tried what I think you tried before, albeit with the head revision code rather than the release code. The output is below:
First, view a small file I FTPed onto the RAM disk, which is mounted in /
~~~
[Press ENTER to execute the previous command again]
>
dir
p.txt [writable file] [size=38]
. [directory] [size=1024]
[Press ENTER to execute the previous command again]
~~~
Then I view the file without the // on the front.
~~~
type p.txt
11111111111111111111111111111111111111
[Press ENTER to execute the previous command again]
~~~
Performing a directory listing shows the file has the same length:
~~~
dir
p.txt [writable file] [size=38]
. [directory] [size=1024]
[Press ENTER to execute the previous command again]
~~~
Then I do the same, but this time add the // to the front of the file name:
~~~
type //p.txt
11111111111111111111111111111111111111
[Press ENTER to execute the previous command again]
~~~
When I perform a directory listing again the file has not been truncated.
~~~
dir
p.txt [writable file] [size=38]
. [directory] [size=1024]
[Press ENTER to execute the previous command again]
>
~~~
ftp get truncates file
I see a difference in behavior of dir
test.txt [writable file] [size=2048]
On my system it laways reports 2048 no matter what size is.
ftp get truncates file
You mentioned you were using a RAM disk, but I didn’t see a mention of the target. Are you using the Windows demo or something else?
ftp get truncates file
I’m using port to uBlaze.
Based on Version from 2015. I can check later exact version.
I managed to operate TCP/IP stack.
The difference from standard demo are
1. RAM disk mounted to “/”
2. no ssd
2. Ethernet MTU is set to 200 bytes at host side – I didn’t find how to limit MTU at freertos (or it did not work)
3. I implemented TCP/IP CLI rather than UDP
Problem that I’ve faced down the road
1. Cannot make RAM disk smaller than 3mb – format fails.
2. FTP hangs at dir, send, get – enters end-less tight loop. I mentioned this in prev post.
3. DIR via CLI reports incorrect size of the file (maybe some sympthom of the problem)
If you didn’t see anything like that before I’ll try to update to the latest version. What version have you mentioned in your post?
ftp get truncates file
I’m using the head revision, which is not in a public repository, if you
send me your files (not your own code, just the FreeRTOS, FreeRTOS+FAT
and FreeRTOS+TCP files) then I can update those files to the latest
version and send them back. You can contact me on r dot barry at
freertos [dot] org.
ftp get truncates file
Thank you very much for your help! I sent files.
ftp get truncates file
Thank you very much for your help! I sent files.
ftp get truncates file
I have updated the +TCP and +FAT code to the head revision of the single interface TCP version – pay no attention to the version number in the header files as its not release code just a working copy. It looks like you were actually using quite an old version. I have not updated any of the other files – you may also like to use FreeRTOS V9.0.0, which you can download from SourceForge here.
ftp get truncates file
Thank you very much. I’ll check it on Sunday and let you know the results.
ftp get truncates file
Thanks for reportting this.
You wrote:
It happens if I add “//” before file name as FTP server does –>type //test.txt –>dirIsn’t
//test.txt
an invalid file name?
+FAT does have code to handle relative paths, it also recognises invalid characters, but it doesn’t always detect invalid paths or file names.
I think it is even possible to create a file with an empty string as a file name.
For some people, this is a short-coming. For others it is a blessing because they want a fast and small library.
We could add some more code checking path validity and make this dependent on some macro ffconfigCHECK_PATH_VALIDITY
ftp get truncates file
I tested a new version, result is the same.
I think that something is wrong with ramdisk/filesystem configuration. DIR reports 2048 bytes, while in your example I see 28 bytes.
Can you send me you setup of RAMDISK?
–>dir
test.txt [writable file] [size=2048]
log.txt [writable file] [size=2048]
. [directory] [size=1024]
–>type test.txt
Hello word
–>type //test.txt
–>dir
test.txt [writable file] [size=0]
log.txt [writable file] [size=2048]
. [directory] [size=1024]
ftp get truncates file
Hi Rasty,
Do you have
ffconfigPATH_CACHE
enabled? It remembers the cluster number that belongs to a certain path.
You could try the same test with:
#define ffconfigPATH_CACHE 0
But once again, //test.txt
is an invalid file name and I think that any API called with such an entry should return an error.
( at this moment I don’t have any hardware with a RAM-disk. Beginning of August I will replicate the problem and come back to it )
ftp get truncates file
ffconfigPATH_CACHE is not defined by default.
I know that // is not valid, maybe it is symphom for other problem as well as reporting of wrong size of file.
BTW. Truncation does not happen and size is reported correctly if file is in sub-folder
ftp get truncates file
I do have a device for testing with a RAM disk: the Windows demo project!
I just tried what you’ve done with a 4MB RAM-disk mounted on:
#define mainRAM_DISK_NAME "/"
But I’m afraid I can not repeat your finding. This is what I saw when using the CLI commands:
~~~~
dir ff_test [directory] [size=0] test.txt [writable file] [size=12] . [directory] [size=1024]
>type test.txt
Hello world
>type //test.txt
Hello world
>dir
ff_test [directory] [size=0]
test.txt [writable file] [size=12]
. [directory] [size=1024]
~~~~
I tried both with and without the path caching (ffconfigPATH_CACHE
).
Please find attached the file FreeRTOSFATConfig.h
that I used for this test.
ftp get truncates file
I had to add 2 defines, otherwise it would not compile.
define ffconfigDEV_PATH “/”
define ffconfigDEV_SUPPORT 1
Still the same problemsftp get truncates file
I’m afraid there is some basic problem with file system (please scroll to the end):
char teststr[]=”Hello ——————————- wordnr”;
/* Create the RAM disk. */
pxRAMDisk = FFRAMDiskInit( mainRAMDISKNAME, ucRAMDisk, mainRAMDISKSECTORS, mainIOMANAGERCACHE_SIZE );
configASSERT( pxRAMDisk );
/* Print out information on the RAM disk. */
stat = FF_RAMDiskShowPartition( pxRAMDisk );
pxFile = ff_fopen( "test.txt", "w" );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fwrite( test_str, sizeof(test_str), 1, pxFile );
ff_fclose( pxFile );
File looks like:
–>type test.txt
Hello ——————————- word
——————————- word
————————— word
———————- word
—————– word
———— word
——- word
— word
rd
Hello ——————————- word
——————————- word
————————— word
———————- word
—————– word
———— word
——- word
— word
rd
Hello ——————————- word
ftp get truncates file
Thanks, that was the difference between your and my project. Now it is reproducible for me. I’ll have a detailed look at it.
Regards, Hein
ftp get truncates file
You can either exclude
ff_dev_support.c
from your project and define:
#define ffconfigDEV_SUPPORT 0
… or you can define the following:
#define ffconfigDEV_SUPPORT 1
#define ffconfigDEV_PATH "/dev"
The module was included in the +FAT distribution, but not yet documented. At this moment, the code is only a sketch. The idea is to open a virtual file and talk with a device ( just like e.g. /dev/ttyS0
under Linux ).
ftp get truncates file
char teststr[]=”Hello ——————————- wordnr”; fffwrite( teststr, sizeof(teststr), 1, pxFile );Could you try the same test but without adding the terminating zero of the strings, like this:
ff_fwrite( test_str, sizeof(test_str) + 1, 1, pxFile );
Adding zeros is a bit confusing
ftp get truncates file
Sorry, not +1 but -1 byte of course:
~~~~
-fffwrite( teststr, sizeof(teststr) + 1, 1, pxFile );
+fffwrite( teststr, sizeof(teststr) – 1, 1, pxFile );
~~~~
ftp get truncates file
Sorry, not +1 but -1 byte of course:
~~~~
-fffwrite( teststr, sizeof(teststr) + 1, 1, pxFile );
+fffwrite( teststr, sizeof(teststr) – 1, 1, pxFile );
~~~~
ftp get truncates file
do you means “sizeof(test_str) – 1“?
I’d expect some garbage at the end of the line.
I did this test because my logs that I print to file also look incomplete.
ftp get truncates file
fffwrite( teststr, sizeof(test_str)-1, 1, pxFile );
–>type test.txt
Hello ——————————- word
Hello
——————————- word
Hello ——
————————- word
Hello ————
——————- word
Hello ——————
————- word
Hello ————————
——- word
Hello ——————————
– word
Hello ——————————- word
Hello ——————————- word
Hell
o ——————————- word
Hello —-
————————— word
Hello ———-
——————— word
Hello —————-
————— word
Hello ———————-
——— word
Hello —————————-
— word
Hello ——————————- wo
rd
Hello ——————————- word
He
llo ——————————- word
Hello —
—————————– word
Hello ——–
———————– word
Hello ————–
—————– word
ftp get truncates file
Right, that is what is expected.
I did this test because my logs that I print to file also look incomplete.Have you found out why?
ftp get truncates file
I found the problem with typing file!
unnessesary “strcat( pcWriteBuffer, cliNEW_LINE )” in prvTYPECommand inserts cr-lf every 50 chars.
Now my logs look great! And I approach to the real problem. FTP and HTTP do not work.
ftp get truncates file
Short summary.
1. Patch (mentioned in https://sourceforge.net/p/freertos/discussion/382005/thread/afe1ae9f/#c48a/06af )did not help.
2. File size in sub-folder is reported correctly and files are not truncated.
2. Size of files in root is still reported as 2048 and truncated if opened for read with // before file name.
3. unnessesary “strcat( pcWriteBuffer, cliNEW_LINE )” in prvTYPECommand inserts cr-lf every 50 chars. Not severy but confusing.
Thank you very much dor the help.
ftp get truncates file
What ‘patch’ are you referring to?
Do you mean this:
~~~~
You can either exclude ffdevsupport.c from your project and define:
#define ffconfigDEV_SUPPORT 0
~~~~
That should help for the problem that you reported as:
~~~~
–>type //test.txt
~~~~
And this should also solve it:
~~~~
define ffconfigDEV_SUPPORT 1
define ffconfigDEV_PATH “/dev”
~~~~ The mentioned defines must appear in yourFreeRTOSFATConfig.h
.
In your case, when ffconfigDEV_PATH
is an empty string, the string “//test.txt
” refers to a device and not a file.
A size of 2048 bytes is returned for a device and in your case “//test.txt
” is still interpreted as a divice.
By the way, there is an easy way of sending out logging:
FreeRTOS-Plus/Demo/Common/Utilities/UDPLoggingPrintf.c
It has some buffering of the logging lines and it also precedes each line with a millisecond-time-stamp, such as:
2016-07-18 23:43:12.896 IP Address: 192.168.2.1
It is easy to add TCP logging to this module. I normally connect it to a telnet server.
lUDPLoggingPrintf
is not interrupt-proof 🙁
~~~~
strcat( pcWriteBuffer, cliNEW_LINE )
~~~~
Some telnet clients like this, because only complete lines (with a LF) will be flushed.
For others (also for me), the extra LF can be confusing.
In UDPLoggingPrintf.c
, no LF’s will be added. Also it has an option to turn every LF into a LF/CR:
~~~~
#define configUDPLOGGINGNEEDSCRLF 1
~~~~
“hello worldn
” will be then sent as “hello worldrn
”
Regards.