ZTree.com  | ZEN  | About...  

 Index   Back

Is getting size on disk quick?   [Wish]

By: Nuno       
Date: Jan 28,2019 at 08:13
In Response to: Is getting size on disk quick? (Peter Shute)

> > Please try out the Alt-Sort by e.g. 'accessed'.
> > The Date/Time column in FW changes according to your choice and
> > displays the selected date/time values, e.g. last-accessed for all
> > files !!
>
> I like the idea of being able to switch between size and size on disk
> the same way we can switch between the date stamp types, but I wonder if
> it's as simple. I suspect that Ztree retrieves and stores all the date
> stamp types during logging, and that it has little or no effect on the
> logging speed.
>
> This might not be true of retrieving the size on disk, so it might not
> be practical to collect it during logging. I suspect some of the extra
> information displayed by Alt-Info, eg owner, might take some time to
> retrieve, because sometimes I see it take a moment to come up.
>
> If this is the case for size on disk, adding it to Alt-Info might be
> the only practical option.
>
> I'm trying to remember whether XtreeNet used to log it all, and
> whether it was optional.


I've been doing some tests.
I wrote a little app to do it, in C++, with direct use of windows APIs.
---
Depending on how ZTree Logs files, calling "GetCompressedFileSize" will (mostly sure) have some overhead, of course.
If ZTree already Logs files/folders using FindFirstFile/FindNextFile, it will have direct access to Attributes, Timestamps and FileSize, without calling any extra API.
In that case, calling the (extra) "GetCompressedFileSize", will have overhead.
Even without going to disk (if read-cache is already filled), there is some overhead (going from user<->kernel, at least).
I've been using an external USB3 (mechanical) HDD for some of my testings.
The results are not "scientific", but there is some consistency between runs, so I think the relative differences are somehow realistic.
(The "absolute" timings are not really relevant, because it depend on everything on my hardware, disk fragmentation, etc).


Files: 1415909, Size: 955430592444, CompressedSize: 954996876263

With empty cache:
Took: 946734 ms (reading compression sizes, using the extra GetCompressedFileSize call)
Took: 879875 ms (NOT reading compression sizes, but calling GetFileAttributes explicitly, forcing an extra API call)
Took: 739594 ms (NOT reading compression sizes, just FindFirstFile/FindNextFile)

Without clearing cache (after a full read/log):
Took: 108156 ms (reading compression sizes, using the extra GetCompressedFileSize call)
Took: 45625 ms (NOT reading compression sizes, but calling GetFileAttributes explicitly, forcing an extra API call)
Took: 21969 ms (NOT reading compression sizes, just FindFirstFile/FindNextFile)


One interesting thing I noticed (in my testings)...
After a Full Disk LOG with (just) FindFirstFile/FindNextFile, there is NO access to disk in order to do another Full LOG with GetCompressedFileSize.
Meaning...
There would be NO physical disk overhead to make that extra GetCompressedFileSize, at least with Local Disks. Network drives will probably be another story...
---
Note: If ZTree checks "Compressed" attribute and only calls GetCompressedFileSize on the actually compressed files, performance will vary only if the disk has a lot of compressed files.
If the disk has only a few compressed files, timing differences will not be noticeable at all.

Nuno

86 views      
 

Messages in this Thread

 
94,242 Postings in 11,887 Threads, 347 registered users, 9 users online (0 registered, 9 guests)
Index | Admin contact |   Forum Time: Apr 26, 2019 - 1:44 am EDT  |  Hits:27,538,624  (182 Today )
RSS Feed