ZTree.com  | ZEN  | About...  

 Index   Back

Do buffers mess with verification?   [Discuss]

By: Ron Metzger       
Date: Jun 17,2018 at 06:53
In Response to: Do buffers mess with verification? (John Leslie)

> > My guess is that Syncback Pro or Qnap are initiating a File Compare
> > independently of the copy process, Correct?
> Yes, and I'd like that option for ZTreeeWin for copying important stuff
> >
> > If that is not correct, consider using ZTree CLP /APIT instead of using
> > ZTreeWin's native copy process. (Or if using /APIT, use ZTreeWin's
> > native copy process.)
> I always use that, partly as I came up with the idea originally :-)
> http://www.ztw3.com/forum/forum_entry.php?id=119982

In order to have ZTreeWin do the file compare, you would have to give up the use of /API or /APIT. This would impact performance significantly as has been shown since ZTreeWin would be have to WAIT for the cache to commit, then read the resulting file stream.

This would not give you the Reason for the failure, just when it happens. Of course, you would need to be At the console when it happens. I usually start a large copy/move process and let it rip while I do other things.

I Still think that Alt-Compare would provide the solution. Perhaps, logging comparison errors, like SyncBack Pro or Qnap does, would help.

This problem reminds me of the Verify process on Tape Backup systems 30 or so years ago. The Verify process use to write to tape, then shoeshine the tape back and re-read what it wrote, and compare against the source. It put so much wear and tear on the tape drive that the manufacturers eventually disallowed that form of verify. (Drives were failing so fast that something had to be done.) Instead they calculate (for x-amount of data) a CRC at the source, write the x-amount of data and store the CRC of that data on tape, later checking the Source CRC to the tape CRC. Even that was not very effective. Still produces a lot of shoe shining of the tape.

Modern Tape drives now use error detection and log Soft and Hard read or write errors on the tape as they happen. This is done at the hardware level and logged in the software (which is fairly easy to do on sequential data streams like tape). When a Error occurs, it skips that part of the tape and tries again on another part of the tape, logging that part of the tape as potentially bad or actually bad.

Today's hard drives and flash drives, use a similar approach for error recovery. NAS' have a lot more moving parts, such as the network (ethernet?), unknown/untested hardware (caches) used for storage, and a NAS OS interaction with the hardware. Each by themselves might be highly reliable, but combined may create failures not easily detected. Diagnosing the failure is a daunting process. Was it a bad packet, a soft error on drive, did the NAS OS not wait long enough for the cache to clear, etc. etc.

Ultimately, I feel that ZTreeWin should not spend too much effort correcting or handling hardware errors, in my humble opinion.

Thanks for reading my diatribe.
Ron Metzger

Thread locked

Messages in this Thread

94,750 Postings in 11,946 Threads, 348 registered users, 21 users online (0 registered, 21 guests)
Index | Admin contact |   Forum Time: Feb 24, 2020 - 12:27 am EST  |  Hits:30,109,209  (305 Today )
RSS Feed