FAT acces time
In a loop I open, write and close a file on SD-Card. Each time a file is opened, the access time is increased.
How can I solve the problem?
FAT acces time
Have you tried a different SD card? There HUGE variations between
cards. It is worth keeping a few premium brand cards to hand for
comparison.
FAT acces time
Beside the quality of the SD-cards, which indeed varies a lot:
How many files do you create within a single directory?
The FAT system has a known problem with huge directories, I mean directories with thousands of files.
We have tried to optimise the access in huge directories.
There are some options that may increase the efficiency:
~~~
#define ffconfigHASHCACHE 1
/* Set to CRC8 or CRC16 to use 8-bit or 16-bit
* HASH values respectively. */
#define ffconfigHASHFUNCTION CRC16
#define ffconfigHASHCACHEDEPTH 64
~~~
Postpone flushing to disk :
~~~
#define ffconfigCACHEWRITETHROUGH 0
~~~
FAT acces time
I have tested different card.
The controller ist an ATSAM4E @ 120MHz. I use the HSMCI (own driver. not the asf stuff) to write 250 files (96Bytes/file) in a single directory. Hash function brings no improvement and needs too much ram.
Intenso SDHC 4GB class 4
1 16ms
100 36ms
250 74ms
LYNQ SDHC 4GB class 4
1 19ms
100 45ms
250 86ms
Transcend SDHC 4GB class 10
1 14ms
100 36ms
250 69ms
Toshiba SDHC 4GB class 4
1 14ms
100 37ms
250 71ms
I have determined that the time is used in the ff_open function.
FAT acces time
Hi Hendrik, thanks for this overview!
You’re proving that the difference between qualities of the cards tested is not as spectacular as we assumed. Maybe the speed differences will show-up more clearly when writing tons of data.
I assume that the times show ( 16, 36, 74 ms ), are the times you need to create a single file? To me that looks as extremely fast!
How did you measure this? Is that the time it took to call
ff_fopen()
?
We have spend a lot of time on working with directories that contain like 65K files. The hashing of directory entries should really help to determine if a given short file name already exists or not.
I just took out my SAM4E Xplained board and created 250 files. The time it took to open/write/close was much longer: a minimum of 200 ms per file. The open itself took most of the time.
I attached a graph the actual times on my board. I used a cheap Transcend SDHC 4GB SD-card.
I wonder where our enormous difference in time comes from
FAT acces time
Another question that I should have asked you: do your file names confirm to the 8.3 notation?
Or are those Long File Names ?
FAT acces time
Hi, the files already exist on card. In a for loop I open overwrite and close each file. The times I measured, are the times for a single file. I simply measure the time by xTaskGetTickCount().
The file names confirm to the 8.3 notation. For all files I need about 10s.
I tested again (Transcend SDHC 4GB class 4)
file 1: open 4ms, write 0ms, close 7ms
file 100: open 25ms, write 2ms, close 6ms
file 250: open 82ms, write 3ms, close 8ms
FAT acces time
Hi Hendrik, I was measuring the time to create 250 new files, which is a lot more work.
ffconfigHASH_CACHE
only optimises the creation of new files. It calculates a hash table for a directory so it knows quickly if a new short file name already exists or not.
On my SAM4E it also takes 10s to re-write 250 files of 96 bytes each.
These were the times that I measured:
~~~
# Open Write/Close
1 1 9
2 1 7
3 0 8
100 5 8
101 9 8
102 5 7
150 18 9
151 17 9
152 17 9
200 17 9
201 18 9
202 17 10
247 22 9
248 21 10
249 21 10
250 21 17
~~~
And I see some strange peeks while writing/closing the files:
~~~
# Open Write/Close
130 14 9
131 14 154
132 15 305
133 15 316
134 15 151
135 15 9
136 15 9
~~~
I’m not sure what is taking up to 300 ms.
Sometimes an SD-card has some internal clean-up, it will flush cached sectors, which can take time. The card will remain ‘busy’ until the internal work is done.
The files were created from FILE001.TXT
up to FILE250.TXT
. So I think that when you upgrade them it is normal to see access times increasing along with the index of the file.
Regards.
FAT acces time
Hi Hendrick
I have lots of SD card experience, You should NEVER assume the write performance will be deterministic. Hein is absolutely correct with regard to internal housekeeping.
However- stick to the highest class (and most expensive cards).
AND there are MANY counterfeits . Lots of low quality rubbish with “Sandisk Extreme PRO ” printed on them.. fake fake fake fake fake . only buy from reputable and listed retailers. Buying on ebay is trouble…
The cheap cards- well my mother does not care if the photo storage takes an extra second more than usual, but your application might !