Microsoft's Resilient File System (ReFS)

Microsoft has a new file system designed to increase data integrity, scalability and availability called the Resilient File System (ReFS).  This file system has leveraged many of the NTFS file system goodies and expanded them to make it more scalable and prevent corruptions.  ReFS was released with Server 2012 and at the moment is designed for use with file shares.  It cannot be used as a boot volume at the present time, but this file system seems poised to replace NTFS down the road.

Torn Writes

One of the new things that ReFS helps with is a “torn write”.  In NTFS the file system overwrites metadata and data structures during a modify operation.  During this modify operation data is read off the disk and then new data is written on top of the old data, becoming the new data.  Torn writes occur when this process can’t complete fully.  Think about what could happen if a power failure occurred during the write operation.  Maybe some of the new data is there, and some of the old data is available, but neither are consistent and there is no way to roll this back.  With ReFS new data will be written to a different location and when the write is complete, the file system then references the new location as the up to date data.  This process is known as “Copy on Write”.  Now during a power failure, if the write doesn’t complete fully the original data is still consistent.

Bit Rot

Storage devices fail on a pretty regular basis.  Solid State drives slowly leak electrical charges due to insulation issues, spinning disks lose their magnetic fields over time, etc.  This expected decay of storage devices is known as “bit rot” and ReFS isn’t going to get rid of it, but now requires a 64-bit checksum on metadata optionally uses a checksum on user data (known as an Integrity Stream) to determine if the data still has integrity.  This checksum will allow the file system to know if something has changed that wasn’t supposed to, eg from a physical component change.

Detecting Bit Rot with a checksum is only a first step.  Obviously the data is not valid if the checksums don’t match, but detecting an error is an important first step.  What ReFS does is use a new feature in Windows called “Storage Spaces”.  I’ve written a previous article about storage spaces here, but to generalize, Storage Spaces are similar to RAID where multiple disks are used in concert to provide fault tolerance or additional disk capacity.

Assuming an ReFS volume is on a “Mirrored” storage space (mirrored meaning copies on two or more disks), the checksum can then be compared across the disks.  If the checksums match the data there is no reason to do anything.  If they checksums don’t match, the disk with the non-matching checksum can have the data replaced by the disk with the matching checksum.  This process will help to fix the “Bit Rot” problem.

Salvage

There are still times even when using RAID or Storage Spaces that data corruption can exist on the disk.  In an NTFS file system this required a CHKDSK to try to fix any logical corruptions on the disk.  If you’ve ever run a CHKDSK on a system, you know that the volume must be offline to run and if it’s a large volume, this may take quite some time.  ReFS uses B+ Trees for managing data and can now isolate the corrupted part of the file system as opposed to the entire volume.

Below we see a typical B+ Tree with no corruption.

The example below shows corruption.  Notice that any data below the corruption in the B+ Tree is then considered corrupted as well.  While ReFS is attempting to fix this corruption, the rest of the file system is still running and available.  Even though ReFS is supposed to be “Always On” it is possible that the root B+ Tree could get corrupted and at that point the entire volume would then be offline.

 Summary

It might not be ready for primetime yet, but ReFS looks poised to be Microsoft’s flagship file system very soon.  Many of these features are a great improvement to NTFS, but probably won’t be found widespread until boot partitions are supported.  Check it out for yourself.

Additional Reading

If you would like to learn more about ReFS, take a look at the following articles which may help to understand this new file system better.

http://blogs.msdn.com/b/b8/archive/2012/01/16/building-the-next-generation-file-system-for-windows-refs.aspx

http://blogs.technet.com/b/askpfeplat/archive/2013/01/02/windows-server-2012-does-refs-replace-ntfs-when-should-i-use-it.aspx