starred/sharpcompress
1
0
Fork 0
mirror of https://github.com/adamhathcock/sharpcompress.git synced 2026-02-03 21:23:38 +00:00
Code Issues 975 Packages Projects Releases 20 Wiki Activity

Really bad performance. #313

New Issue
Open
opened 2026-01-29 22:09:54 +00:00 by claunia · 12 comments
claunia commented 2026-01-29 22:09:54 +00:00
Owner
Copy Link

Originally created by @SuperJMN on GitHub (Jul 8, 2018).

I'm using this code.

It takes like 10 minutes to extract 272 files from a 7-zip archive that is not special by any means. Well, it's been compressed using the Ultra setting… :)

I has some of them that are about 80MB in size.

What could be wrong?

Thanks!!

public class DriverPackageInstaller : IDriverPackageInstaller
    {
        public async Task InstallDriverPackage(string fileName, string destination)
        {
            using (var package = SevenZipArchive.Open(fileName))
            {
                await Extract(package.Entries, "Cityman", @"NewPack\Pre-OOBE");
                await Extract(package.Entries, "PostOOBE", @"NewPack\Post-OOBE");
            }
        }

        public static async Task Extract(ICollection<SevenZipArchiveEntry> entries, string sourceFolder, string destinationFolder)
        {
            var translatedPath = sourceFolder.Replace("\\", "/");

            foreach (var entry in entries.Where(x => x.Key.StartsWith(translatedPath) && !x.IsDirectory))
            {
                await ExtractEntryToFile(entry, sourceFolder, destinationFolder);
            }
        }

        private static async Task ExtractEntryToFile(IArchiveEntry entry, string sourceFolder, string destination)
        {
            using (var input = entry.OpenEntryStream())
            {
                var path = entry.Key.Replace($"{sourceFolder}/", "");
                var fileName = Path.GetFileName(path);
                var directoryName = Path.GetDirectoryName(path);
                var newFolder = Path.Combine(destination, directoryName);
                var outputPath = Path.Combine(newFolder, fileName);

                Directory.CreateDirectory(newFolder);
                using (var output = File.OpenWrite(outputPath))
                {
                    await input.CopyToAsync(output);
                }                
            }
        }
    }
Originally created by @SuperJMN on GitHub (Jul 8, 2018). I'm using this code. It takes like 10 minutes to extract 272 files from a 7-zip archive that is not special by any means. Well, it's been compressed using the Ultra setting… :) I has some of them that are about 80MB in size. What could be wrong? Thanks!! ``` public class DriverPackageInstaller : IDriverPackageInstaller { public async Task InstallDriverPackage(string fileName, string destination) { using (var package = SevenZipArchive.Open(fileName)) { await Extract(package.Entries, "Cityman", @"NewPack\Pre-OOBE"); await Extract(package.Entries, "PostOOBE", @"NewPack\Post-OOBE"); } } public static async Task Extract(ICollection<SevenZipArchiveEntry> entries, string sourceFolder, string destinationFolder) { var translatedPath = sourceFolder.Replace("\\", "/"); foreach (var entry in entries.Where(x => x.Key.StartsWith(translatedPath) && !x.IsDirectory)) { await ExtractEntryToFile(entry, sourceFolder, destinationFolder); } } private static async Task ExtractEntryToFile(IArchiveEntry entry, string sourceFolder, string destination) { using (var input = entry.OpenEntryStream()) { var path = entry.Key.Replace($"{sourceFolder}/", ""); var fileName = Path.GetFileName(path); var directoryName = Path.GetDirectoryName(path); var newFolder = Path.Combine(destination, directoryName); var outputPath = Path.Combine(newFolder, fileName); Directory.CreateDirectory(newFolder); using (var output = File.OpenWrite(outputPath)) { await input.CopyToAsync(output); } } } } ```
claunia commented 2026-01-29 22:09:55 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jul 8, 2018):

The 7zip implementation is far from optimized. I’m sure some quick wins could be from pooling byte arrays but I haven’t looked into it.

@adamhathcock commented on GitHub (Jul 8, 2018): The 7zip implementation is far from optimized. I’m sure some quick wins could be from pooling byte arrays but I haven’t looked into it.
claunia commented 2026-01-29 22:09:55 +00:00
Author
Owner
Copy Link

@SuperJMN commented on GitHub (Jul 8, 2018):

Thank you. I hope it gets better. I'm using it inside this application: https://github.com/SuperJMN/Lumia-WoA-Installer (in the most recent dev branch, still not committed). I will use it to import driver packages. They're .7z files with Ultra compressing level.

@SuperJMN commented on GitHub (Jul 8, 2018): Thank you. I hope it gets better. I'm using it inside this application: https://github.com/SuperJMN/Lumia-WoA-Installer (in the most recent dev branch, still not committed). I will use it to import driver packages. They're .7z files with Ultra compressing level.
claunia commented 2026-01-29 22:09:55 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jul 9, 2018):

There is a perf improvement someone contributed that I haven't released yet that is specific for LZMA but should help everything: https://github.com/adamhathcock/sharpcompress/pull/384

I probably ought to release soon.

@adamhathcock commented on GitHub (Jul 9, 2018): There is a perf improvement someone contributed that I haven't released yet that is specific for LZMA but should help everything: https://github.com/adamhathcock/sharpcompress/pull/384 I probably ought to release soon.
claunia commented 2026-01-29 22:09:56 +00:00
Author
Owner
Copy Link

@SuperJMN commented on GitHub (Jul 9, 2018):

It's already merged. I hope it's published to NuGet.org soon! Thank you.

@SuperJMN commented on GitHub (Jul 9, 2018): It's already merged. I hope it's published to NuGet.org soon! Thank you.
claunia commented 2026-01-29 22:09:56 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jul 9, 2018):

https://www.nuget.org/packages/sharpcompress/0.22.0

@adamhathcock commented on GitHub (Jul 9, 2018): https://www.nuget.org/packages/sharpcompress/0.22.0
claunia commented 2026-01-29 22:09:56 +00:00
Author
Owner
Copy Link

@SuperJMN commented on GitHub (Jul 9, 2018):

Wow, that's FAST. Thanks a lot! I think my tool is now ready for prime time! As soon as I add a "About" section, this wonderful project will be in the credits, for sure.

@SuperJMN commented on GitHub (Jul 9, 2018): Wow, that's FAST. Thanks a lot! I think my tool is now ready for prime time! As soon as I add a "About" section, this wonderful project will be in the credits, for sure.
claunia commented 2026-01-29 22:09:57 +00:00
Author
Owner
Copy Link

@SuperJMN commented on GitHub (Jul 14, 2018):

Hello again!

Unfortunately, the performance is still alarming. It seems the extraction is quite quick at the beginning, but performance degrades as the extraction of the files progresses. I've profiled the extraction process and this is what I've got:

image

Can you, please, take a look? I had to switch to regular .zip files instead because of this. This is a link to the file I'm testing: https://drive.google.com/open?id=1xSWWALufBbe-2H4fgF8GQChn-NKt7mQ3

Thank you!

@SuperJMN commented on GitHub (Jul 14, 2018): Hello again! Unfortunately, the performance is still alarming. It seems the extraction is quite quick at the beginning, but performance degrades as the extraction of the files progresses. I've profiled the extraction process and this is what I've got: ![image](https://user-images.githubusercontent.com/3109851/42725610-eeda7e9c-8786-11e8-88de-7ff62aafb5a9.png) Can you, please, take a look? I had to switch to regular .zip files instead because of this. This is a link to the file I'm testing: https://drive.google.com/open?id=1xSWWALufBbe-2H4fgF8GQChn-NKt7mQ3 Thank you!
claunia commented 2026-01-29 22:09:57 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jul 14, 2018):

Probably places where byte arrays can be pooled.

Not sure if I’ll get a chance soon as I’ve been going through a long and drawn series of personal issues.

@adamhathcock commented on GitHub (Jul 14, 2018): Probably places where byte arrays can be pooled. Not sure if I’ll get a chance soon as I’ve been going through a long and drawn series of personal issues.
claunia commented 2026-01-29 22:09:57 +00:00
Author
Owner
Copy Link

@androschuk commented on GitHub (Feb 22, 2023):

Hello @adamhathcock Any updates?

In my case, there is a 7z archive with 800+ files (Archive size ~9 Mb, Extracted folder size ~54Mb), when I try to make an OpenEntryStream on a 300+ file, it starts to noticeably slow down.

Example:

using (var archive = ArchiveFactory.Open(filePath))
{
    foreach (var entry in archive.Entries.Where(it => !it.IsDirectory))
    {
          using (var stream = entry.OpenEntryStream())
          {
            // Do something.
          }
      }
  }

It looks like related issues:

image

@androschuk commented on GitHub (Feb 22, 2023): Hello @adamhathcock Any updates? In my case, there is a `7z` archive with `800+` files (Archive size `~9 Mb`, Extracted folder size `~54Mb`), when I try to make an `OpenEntryStream` on a `300+` file, it starts to noticeably slow down. Example: ``` using (var archive = ArchiveFactory.Open(filePath)) { foreach (var entry in archive.Entries.Where(it => !it.IsDirectory)) { using (var stream = entry.OpenEntryStream()) { // Do something. } } } ``` It looks like related issues: - https://github.com/adamhathcock/sharpcompress/issues/544 - https://github.com/adamhathcock/sharpcompress/issues/728 - https://github.com/adamhathcock/sharpcompress/issues/710 ![image](https://user-images.githubusercontent.com/5159408/220609362-1f8be3fa-d8b8-4369-8a1e-258079473a27.png)
claunia commented 2026-01-29 22:09:57 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Mar 1, 2023):

PRs are welcome. I've been away for personal reasons.

@adamhathcock commented on GitHub (Mar 1, 2023): PRs are welcome. I've been away for personal reasons.
claunia commented 2026-01-29 22:09:58 +00:00
Author
Owner
Copy Link

@Erior commented on GitHub (Mar 12, 2023):

@androschuk , try use
var reader = archive.ExtractAllEntries();
while (reader.MoveToNextEntry())
{
if (!reader.Entry.IsDirectory)
{
// Do something
}
}

@Erior commented on GitHub (Mar 12, 2023): @androschuk , try use var reader = archive.ExtractAllEntries(); while (reader.MoveToNextEntry()) { if (!reader.Entry.IsDirectory) { // Do something } }
claunia commented 2026-01-29 22:09:58 +00:00
Author
Owner
Copy Link

@androschuk commented on GitHub (Mar 14, 2023):

@Erior Unfortunately it doesn't help. Because the main problem is inside OpenEntryStream.

@androschuk commented on GitHub (Mar 14, 2023): @Erior Unfortunately it doesn't help. Because the main problem is inside `OpenEntryStream`.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
duplicate
enhancement
feedback requested
not bug
pull-request
Mirrored from GitHub Pull Request
 question
up for grabs
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/sharpcompress#313
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?