ZTree.com  | ZEN  | About...  

 Index   Back

It's the Hard Links   [Discuss]

By: John Gruener     Orlando, Florida  
Date: Nov 23,2019 at 20:56
In Response to: [Discuss] 32GB+ Logged files in 24,255,205,376 Used space? (Bill Kingsbury)

> I find that a partition holds 32GB+ of "Logged files" in
> 24,255,205,376 "Used space".
> No compression is involved -- as far as I know...
> How to explain this?

I'm surprised no one has questioned this before. Doesn't everyone see this discrepancy?

I find that logging any entire Windows 10 system drive, with Junction logging disabled and running ZTree as Administrator, the Logged files will always exceed the Used space by at least 8 GB, and as much as 12 GB. And this is without logging the several directories to which Windows denies access. If run as non-Admin, some 60 or so directories will not be logged, reducing the difference by about 3 GB.

So what's causing this? As Ben mentions, NTFS Compressed and Sparse files take up less space than what ZTree logs, so that accounts for some of it. But I think the existence of multiple Hard Links accounts for most of the difference. Windows 10 (and to a lesser extent previous versions of Windows) makes extensive use of additional hard links to the same file. About 80% of these have at least one link in C:\Windows\WinSxS subdirectories.

Using Alt-Info in the WinSxS branch shows many files with two links, and I've found a few with as many as seven. This means that ZTree is logging those files seven times each. There is no practical way for ZTree to discover that it has already logged that same file in a different directory or by a different name.

A useful tool to find all the Hard Links, Symbolic Links and Junction Points on a volume is NirSoft's NTFSLinksView:
As with most NirSoft utilities, it runs stand-alone without installation.
Scanning C:\ with infinite depth can take 10 minutes or more, but will yield a complete list sortable by name, path, link type or created time. The bottom status line shows the count found.

I've run this on several Windows 10 system drives, finding at least 90,000 and as many as 160,000 hard links counted on each. However, this count is a bit misleading since it lists all the hard links to any file that has more than one. So each file with one extra hard link shows two link lines, and those with multiple extra links have many more. For example, a file with four links will have twelve lines since each of the four will have a line entry pointing to the other three. ZTree will have logged such a file four times. For this reason it's difficult to determine just how many actual files these lines represent, and how many times ZTree will have logged them.

Nevertheless, while that listing is greatly inflated, it does point out that there are tens of thousands of files logged more than once by ZTree. For a Windows 10 system drive I'm guessing there are usually at least 30,000 files logged more than once.

Another tool that gives quick info about a Hard Link is SysInternals FindLinks:
Place it in your F9 Menu script to quickly find all the links to a highlighted file. This is designed to list the additional links other than the one selected, so the number shown does not count the selected link. Also be aware it will truncate the results both left and right if the ZTree window is not wide enough for the path.

I think the only way for a program to decipher the actual number of physical files on an NTFS drive and the space they take would be to track the actual cluster location of each file, then not count links that point to an already-located file.

- John


Messages in this Thread

94,800 Postings in 11,952 Threads, 348 registered users, 27 users online (1 registered, 26 guests)
Index | Admin contact |   Forum Time: Mar 28, 2020 - 9:57 am EDT  |  Hits:30,501,288  (3,367 Today )
RSS Feed