I have been doing some bulk file copies to a USB attached HDD (spinning rust), using ZTree.
The copies were done in a directory tree using Alt-Copy, normally I would use robocopy, but for this job I needed to only copy a subset of the files, which I have in a .zls file.
Windows 10, using Microsoft Defender anti virus.

Some findings from the experience



[Issue] ztwerr.log Corrupted/miss-formed error 121 line
%Temp%\ZTMP000\ztwerr.log
07-05-2023 02:00:12 : 21] [121] (R) : e:\path\file.ext

What is the " 21] "?
I presume a partial duplicate of " [121] "
Can that be fixed?



[ZEP] ztwerr.log 433 error not logged
When the "433 A device which does not exist was specified." error popup happens, there is no entry in ztwerr.log
Please add one.

I have seen some 1117 errors in the ztwerr.log, but not for every 433
07-05-2023 13:48:00 : I/O device error [1117] (R) : e:\path\file.ext


[issue] Too many retries on Error 3, Path not found
%Temp%\ZTMP000\ztwerr.log
07-05-2023 08:11:35 : Path not found [3] (S) : d:\path

I had an instance where there were 1668 entries in the log file over 2 seconds.
Why try so often?
Wouldn't it be better to have at least 100mS sleep between retries?





[Observation] Defender USB scanning
Resource Monitor, Disk, sort by file: all the files on the USB attached disk are being scanned, with many files (1000-3000) in parallel, which will overwhelm a spinning rust drive.
Task manger, Performance tab, the destination disk, Active time=70-95%, before starting the ZTree copy job

If I'm doing the ZTree file copy while the AV scan is running, after a while, a USB reset (all devices I think) occurs (because of the ding-dongs I hear), the USB drive disappears, so ZTree gives the Error popup, [433], "A device which does not exist was specified".
Once the USB drive reappears, Defender sees the drive as newly attached, and starts a fresh scan.

I tried these, which didn't seem to help, maybe they need the defender service to be restarted to take effect, of possibly are no longer supported by the service.
Set-MpPreference -DisableRemovableDriveScanning $false
Set-MpPreference -ExclusionExtension ext1,ext2,...

So I now remove all un-necessary UBS devices (just in case) and wait for the AV scan to finish before staring the ZTree copy job.





[ZEP] Improve copy job error handling
While doing the ZTree copy
Task manger, Performance, the destination disk, Active time=20-70%

Then after a few (5-20) files
Task manger, Performance, the destination disk, Active time=100%
Resource Monitor, Disk, sort by file. The active destination files are flushed. No destination files for a while, but still Active time=100%
ZTree, spinner active, but progress bar times and rate stopped updating. Most of the time the system flush to disk completes, and the ZTree progress bar counters start updating again.

But sometimes
Error popup: The semaphore timeout period has expired [121]
If I wait for the disk active time to reduce to ~0 (often at 10-23 seconds on the ZTree error popup countdown timer), then retry on the ZTree popup the copy job continues. But to do that I have the baby sit the copy job.
If I'm not at the computer, the default action (skip) is taken, so I have to rerun the Alt-Copy command with the Older files option, to process the skipped files.


Could the file copy semaphore timeout be increased/configured?
Could ZTree be configured to retry once after the error popup timeout for some errors, particularly 121? If the first retry fails then skip.
Could the error popup timeout be configured, possibly by error number?

Those configurations would give ZTree the equivalent of robocopy's /R: and /W: command line options.
Given how robocopy handles it, maybe ZTree could have two new configuration options, Error retry count default=0, Error countdown timer default=30S

I'm open to other suggestions of how to improve the on error handling.


Thanks
Ben