ZTree.com  | ZEN  | About...  

 Index   Back

[Help!] Duplicate File Finder Recommendation   [Help!]

By: Laurent Duchastel     Montréal, Québec  
Date: Oct 23,2022 at 16:26
In Response to: [Help!] Duplicate File Finder Recommendation (Andrew Watson)

Hello Andrew,

I use ZTree for such use, with Branch | Alt-F4 (Binary).
If I get two identical files, I delete one of them (choice established by judging best wanted location). If I get three identicals, I delete two. Etc. Sometime I don't delete if duplication is voluntary. This is done manually. Tedious but relatively fast.

It does exclude however any pictures that had been retouched, commented, resized, rotated, etc.

Another solution would be to de-deplicate files but not *organization* of files by hardlinking them.
It means three "copies" but one physical files only. Works on NTFS.
Use however with good understanding of its implications.

Not that some archivers can "de-deduplicate" in similar manner by not archiving identical files, but only their references.
ex: switch -oi with rar.exe
Of course, archived pictures are not accessibles unless uncompressed, but saving the less used ones as archive on hard disk will easily take less space without having to spend hours dedpulicating them.
See below for ZTree script.

Another solution, if possible, would be to change hard disk of new device for a 1Tb one. If however it is something thightlty closed like a Surface however, no luck.

At last, I once used this software (now "old").
I found it most interesting by finding "almost-identical" pictures by comparing image pixel data
Might worth a try?

::A R*A*R [branch] | DEL | 5% | 100Kb | PLAIN
REM #ZTStart "Archiving"

:: a Add to Archive
:: -r Recurse subfolders
:: -MA5 Force archiving format v5.0+
:: -md128m Set dictionary size to 128mb
:: -v100m Create volumes 100mb
:: -m3 Set compression method to 3=Normal
:: -ms Store these files instead of archiving
:: default = 7z, ace, arj, bz2, cab, gz, jpeg, jpg, lha, lzh, mp3, rar, taz, tgz, xz, z, zip
:: -rrp5 Add data recovery record (5%)
:: -oi Save identical files as reference (v5.0+ only)
:: -htb Set hash type to Blake2 (256bits, ensure files are identicale) (v5.0+ only_
:: -tl Set archive time to newest file
:: -hp[pwd] Encrypt both file and headers
:: -t Test files after archiving
:: -k Lock archive (prevents any further modifications)
:: -df Delete files after archiving

#ZTHome\RAR\RAR.exe a -r -MA5 -md128m -v100m -m3 -ms -rr5p -oi -htb -tl -t -k -df %3.rar *.*

Laurent Duchastel


Messages in this Thread

96,456 Postings in 12,197 Threads, 350 registered users, 126 users online (0 registered, 126 guests)
Index | Admin contact |   Forum Time: Sep 30, 2023 - 7:17 am UTC  |  Hits:58,245,937  (10,684 Today )
RSS Feed