ZTree.com  | ZEN  | About...  

 Index   Back

[Q] UnRAR slowdown   [Q]

By: Ben Kent       
Date: Jun 01,2019 at 18:38
In Response to: [Q] UnRAR slowdown (Liviu)

> I am seeing the same behavior described href=http://www.ztw3.com/forum/forum_entry.php?id=120150>here i.e.
> UnRAR'ing gets progressively slower with archives that contain many
> files/directories.
>
> For a random example, I downloaded the href="https://github.com/git/git/releases/tag/v2.21.0">git v2.21 source
> .zip and repacked it as a (*non* solid) RAR file. Extracting (via
> Alt-F5) all files from the ZIP took 30 sec, while extracting the same
> exact files from the RAR took 3.5 times that (= 1:45) and became slower
> and slower towards the end. The results are consistent whether using the
> UnZIP/UnRAR DLLs vs. (temporarily) renaming those so that the command
> lines in the BB2 file are used, instead.
>
> Both the ZIP and RAR files unpack in 2-3 seconds at the cmd prompt, or
> using WinRAR. I realize that there is an overhead in ZTree about updating
> the display and other housekeeping, however I would have expected that
> overhead to be comparable between archive types ZIP vs. RAR. Any insight,
> clues, or tips much appreciated.
>
> Cheers,
> Liviu

Liviu

skip this section, I started posting before checking how ZTree acts on extraction
==
If its getting slower towards the end of solid archives, then I expect that the list file that ZTree passes is causing 7-Zip to process each file in the list file as if they are separate requests.
With solid archives the compression is better, but to extract a file all of the archive up to the file you want to extract needs to be processed, which will lead to exponential slowdown for many files from a large archive.

Does the ZTree list file have the files in archive order?
(probably not)

I would hope that for solid archives, extraction applications should re-order list files into archive order, and then process the archive in one pass, extracting the requested files as they pass by and stopping when the last file is extracted (wildcards might cause longer processing).
Or if they cannot sort into archive order, then process the archive until all requested files are extracted, which at worst case should take a single pass of the archive.
==


But re-reading the ARCHIVER.BB2 file (I should have done that before typing the above), the list file is only used for compression in ZTree, i.e. ZTree does extraction file by file, which will cause heavy performance issues extracting many files from large solid archives. From memory the reason for the ZTree behavior is to cope with destination file overwrite conflicts.

The possible ZEP, is to have an ARCHIVER.BB2 flag, that causes ZTree to extract with a list file to a temporary directory, and then use a Branch, tag all, Alt-Move, delete what is left process.
That assumes that the achiever programs can cope with list files not in archive order.
Doing that could have IOPS impacts, because of the extract to temp and then move to destination actions, so the ARCHIVER.BB2 flag should only have an effect for solid archives, and only when more than one file is being extracted.

Ben

114 views      
 

Messages in this Thread

 
94,417 Postings in 11,916 Threads, 348 registered users, 8 users online (0 registered, 8 guests)
Index | Admin contact |   Forum Time: Aug 22, 2019 - 8:02 pm EDT  |  Hits:28,555,721  (4,520 Today )
RSS Feed