starred/sharpcompress
1
0
Fork 0
mirror of https://github.com/adamhathcock/sharpcompress.git synced 2026-02-10 21:22:11 +00:00
Code Issues 975 Packages Projects Releases 20 Wiki Activity

AWS S3 HashStream #532

New Issue
Closed
opened 2026-01-29 22:13:23 +00:00 by claunia · 2 comments
claunia commented 2026-01-29 22:13:23 +00:00
Owner
Copy Link

Originally created by @i2um1 on GitHub (Aug 18, 2022).

Hi,

According to the documentation:
The major feature is support for non-seekable streams so large files can be processed on the fly (i.e. download stream).

But the following code does not work:

using Amazon;
using Amazon.S3;

using SharpCompress.Archives.Zip;

const string accessKey = "";
const string secretKey = "";
const string bucketName = "";
const string fileKey = "";

var s3Client = new AmazonS3Client(accessKey, secretKey, RegionEndpoint.EUWest1);
var response = await s3Client.GetObjectAsync(bucketName, fileKey);

await using var stream = response.ResponseStream;
var archive = ZipArchive.Open(stream);
foreach (var entry in archive.Entries)
{
    Console.WriteLine(entry.Key);
}

Exception:

System.NotSupportedException: HashStream does not support seeking
   at Amazon.Runtime.Internal.Util.HashStream.get_Position()
   at SharpCompress.IO.SourceStream.get_Position()
   at SharpCompress.IO.SourceStream.Seek(Int64 offset, SeekOrigin origin)
   at SharpCompress.IO.SourceStream.set_Position(Int64 value)
   at SharpCompress.Archives.Zip.ZipArchive.LoadVolumes(SourceStream srcStream)
   at SharpCompress.Archives.AbstractArchive`2..ctor(ArchiveType type, SourceStream srcStream)
   at SharpCompress.Archives.AbstractWritableArchive`2..ctor(ArchiveType type, SourceStream srcStream)
   at SharpCompress.Archives.Zip.ZipArchive..ctor(SourceStream srcStream)
   at SharpCompress.Archives.Zip.ZipArchive.Open(Stream stream, ReaderOptions readerOptions)
   at Program.<Main>$(String[] args) in V:\Test\Test\Program.cs:line 15
   at Program.<Main>$(String[] args) in V:\Test\Test\Program.cs:line 16
   at Program.<Main>(String[] args)

Please see https://github.com/aws/aws-sdk-net/blob/master/sdk/src/Core/Amazon.Runtime/Internal/Util/HashStream.cs

Is it possible to handle HashStream in memory?

Originally created by @i2um1 on GitHub (Aug 18, 2022). Hi, According to the documentation: *The major feature is support for **non-seekable** streams so large files can be processed on the fly (i.e. download stream).* But the following code does not work: ```cs using Amazon; using Amazon.S3; using SharpCompress.Archives.Zip; const string accessKey = ""; const string secretKey = ""; const string bucketName = ""; const string fileKey = ""; var s3Client = new AmazonS3Client(accessKey, secretKey, RegionEndpoint.EUWest1); var response = await s3Client.GetObjectAsync(bucketName, fileKey); await using var stream = response.ResponseStream; var archive = ZipArchive.Open(stream); foreach (var entry in archive.Entries) { Console.WriteLine(entry.Key); } ``` Exception: ``` System.NotSupportedException: HashStream does not support seeking at Amazon.Runtime.Internal.Util.HashStream.get_Position() at SharpCompress.IO.SourceStream.get_Position() at SharpCompress.IO.SourceStream.Seek(Int64 offset, SeekOrigin origin) at SharpCompress.IO.SourceStream.set_Position(Int64 value) at SharpCompress.Archives.Zip.ZipArchive.LoadVolumes(SourceStream srcStream) at SharpCompress.Archives.AbstractArchive`2..ctor(ArchiveType type, SourceStream srcStream) at SharpCompress.Archives.AbstractWritableArchive`2..ctor(ArchiveType type, SourceStream srcStream) at SharpCompress.Archives.Zip.ZipArchive..ctor(SourceStream srcStream) at SharpCompress.Archives.Zip.ZipArchive.Open(Stream stream, ReaderOptions readerOptions) at Program.<Main>$(String[] args) in V:\Test\Test\Program.cs:line 15 at Program.<Main>$(String[] args) in V:\Test\Test\Program.cs:line 16 at Program.<Main>(String[] args) ``` Please see https://github.com/aws/aws-sdk-net/blob/master/sdk/src/Core/Amazon.Runtime/Internal/Util/HashStream.cs Is it possible to handle HashStream in memory?
claunia commented 2026-01-29 22:13:23 +00:00
Author
Owner
Copy Link

@Erior commented on GitHub (Aug 25, 2022):

ZipArchive uses the CentralDirectory stored at the end of the file, and requires the Seek functionality, you can as you propose copy the stream into a temporary file or MemoryStream and use ZipArchive on that.
Or use the ReaderFactory.

@Erior commented on GitHub (Aug 25, 2022): ZipArchive uses the CentralDirectory stored at the end of the file, and requires the Seek functionality, you can as you propose copy the stream into a temporary file or MemoryStream and use ZipArchive on that. Or use the ReaderFactory.
claunia commented 2026-01-29 22:13:24 +00:00
Author
Owner
Copy Link

@i2um1 commented on GitHub (Aug 26, 2022):

So, it looks like a temporary file is the best solution when an input file is extremely big and there's no enough information about a file. ReaderFactory is a partial solution. Thank you!

@i2um1 commented on GitHub (Aug 26, 2022): So, it looks like a temporary file is the best solution when an input file is extremely big and there's no enough information about a file. ReaderFactory is a partial solution. Thank you!
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
duplicate
enhancement
feedback requested
not bug
pull-request
Mirrored from GitHub Pull Request
 question
up for grabs
wontfix
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/sharpcompress#532
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?