Sparse NTFS files

NTFS has support for sparse files. These are files that take up less disk space than their own size. This technology is not related to the built-in support for NTFS file compression, as the saving of disk space in sparse files is based on a different principle. No data compression is performed. Instead, in the file are released areas occupied only by zeros (0x00). An application that reads a sparse file, when it reaches the area with zeros, reads zeros, but a real disk read will not occur.

In this way, you can create giant-sized files, consisting of zeros, but on the disk they can take only a few kilobytes. Real disk space is allocated when some other data is written instead of 0x00. Sparseness will help save disk space only in such files that have really large empty areas.

I will demonstrate how to work with sparse files using the fsutil command line utility.

Using the utility, create an empty large file:
fsutil file createnew test.nul 10000000000

Give the file the attribute "sparse":
fsutil sparse setflag test.nul

The attribute itself does not yet save disk space. It is also necessary to mark the area to be freed inside the file. We have the entire file empty, so the area can be set to the file size.
fsutil sparse setrange test.nul 0 10000000000

Is done. See the result.

')
How to do these operations in your programs using API functions, you can look at the source code of a small utility that I wrote in 2006. It is also a console, and also as fsutil is able to assign the sparse attribute and set the range of the region to be freed. In addition, my program can itself look for empty areas larger than a certain size in the file and free them. There is no need to calculate the offset itself.

References:

• Details about sparse files
• The program for creating sparse files in NTFS