The world of storage technology is littered with examples of out-dated technologies that have hung around longer than they should. Take the humble floppy 3.5” disk. Even when 128MB flash memory sticks were widely available, floppies were still being used. There was always one curmudgeon user in every office who insisted on using floppies when everyone else was using USB flash drives or CDs. Tape backup is only now in its final death throes for most commercial uses. And even in the home, VHS tapes took much longer than expected to retire. Up untillast year, Argos the catalogue retailer was still selling VHS tapes.
There is one more piece of storage technology that is still hanging around, past its sell-by-date – the FAT driver. FAT was devised in 1977 along with MS-DOS and even then was a less than efficient file system. For example, a 45KB file which was on a drive that used 32KB clusters would use up 64KB of space. In 1996, an improved version of FAT called FAT32 was brought out to overcome some the file system’s deficiencies. For drives less than 8Gb, the cluster size is 4KB which means a lot less wasted space on a storage device.
But even with the improvement FAT32, it is still a poor and unreliable file system. Its performance starts to lag after 200MB. Its directory has no formal organisation. As files systems go, FAT32 is not really suited to storages devices above 20GB. Other reasons FAT32 does not make a great file system include:
1) On the FAT32 file system, files will more likely “spread out” or fragment over the drive. For efficient storage, data should be contiguous as possible on a storage device.
2) FAT32 has a very low fault tolerance. This can lead to data loss.
3) FAT 32 was really designed to work with small partitions of under 20GB. Of course, it will still work with your 32GB Memory Stick or camera card but not optimally.
Only last week, we were dealing with a distressed PhD student who had all his thesis backed up onto a Kingston 8GB USB drive which was FAT32 formatted. It had a substantial number of Word Documents, AutoCAD, Jpeg and .PDF files stored on it. When he connected it to his computer, the device was registering as 0 bytes. We performed diagnostics on the device. The NAND chip seemed okay, the controller seemed to be functioning but the file allocation table had totally collapsed. Using a binary editor we were able to manually rebuild the FAT32 file system. The data recovery process restored all of his data.
So with all these disadvantages that I have outlined, you would think that FAT32 would be rarely, if ever, be used with files over 20GB. I’m afraid that is not the case. If you buy a 32GB USB flash drive or standard SD camera card to-day, it will probably be formatted with FAT32. The reason being is that FAT32 is freely licensed and has universal operating system support. This helps explain its popularity among flash drive manufacturers. (NTFS is a proprietary format and is heavily patent protected)
We will probably only see the demise of FAT32 coinciding with the demise of Windows XP. Once the prevalence of Windows XP in houses and offices across Ireland and the globe has finally ended, exFAT will most probably become the de facto standard for high-capacity USB flash drives and camera cards.
Why exFAT is better than FAT32
ExFAT is a file system specifically designed for flash memory devices of over 4GB in capacity. It’s file size limit is 16 exabytes. It allows for access control lists to set restrictions on certain files. More importantly, it uses free space bitmaps in order to reduce fragmentation. Using a more robust system such as exFAT should, in theory, reduce the probability of file system corruption and if this file system does go corrupt, thanks to bit mapping the chances of more streamlined data recovery are a lot higher.
The uptake of exFAT by flash memory manufacturers has already begun. All SDXC (SD Extended Capacity) cards now use exFAT.
In the next couple of years, we will probably be saying goodbye to FAT32 and as the de facto file system for USB drives and camera cards. It will be going to storage heaven to meet-up with some old tapes and old 3.5” floppy diskettes.