starred/sharpcompress
1
0
Fork 0
mirror of https://github.com/adamhathcock/sharpcompress.git synced 2026-02-03 21:23:38 +00:00
Code Issues 975 Packages Projects Releases 20 Wiki Activity

Failed to read stream from HttpClient.Get with option HttpCompletionOption.ResponseHeadersRead #196

New Issue
Closed
opened 2026-01-29 22:08:07 +00:00 by claunia · 11 comments
claunia commented 2026-01-29 22:08:07 +00:00
Owner
Copy Link

Originally created by @jiabiao on GitHub (Jun 3, 2017).

The following code works fine under full framework, but get exception under .net core 1.0 and 2.0.

var res = new HttpClient().GetAsync("https://github.com/adamhathcock/sharpcompress/raw/master/tests/TestArchives/Archives/Tar.tar",
	HttpCompletionOption.ResponseHeadersRead).Result;
var stream = res.Content.ReadAsStreamAsync().Result;
var reader = ReaderFactory.Open(stream);
reader.MoveToNextEntry();
reader.WriteEntryTo(new MemoryStream());  //System.IO.IOException: 'Unable to read data from the transport connection. The connection was closed before all data could be read. Expected 105472 bytes, read 512 bytes.'

Changing the HttpClient.Get option to HttpCompletionOption.ResponseContentRead also works. But for large files, it can not start decompressing until the entire file is downloaded.

Originally created by @jiabiao on GitHub (Jun 3, 2017). The following code works fine under full framework, but get exception under .net core 1.0 and 2.0. ```csharp var res = new HttpClient().GetAsync("https://github.com/adamhathcock/sharpcompress/raw/master/tests/TestArchives/Archives/Tar.tar", HttpCompletionOption.ResponseHeadersRead).Result; var stream = res.Content.ReadAsStreamAsync().Result; var reader = ReaderFactory.Open(stream); reader.MoveToNextEntry(); reader.WriteEntryTo(new MemoryStream()); //System.IO.IOException: 'Unable to read data from the transport connection. The connection was closed before all data could be read. Expected 105472 bytes, read 512 bytes.' ``` Changing the HttpClient.Get option to ```HttpCompletionOption.ResponseContentRead``` also works. But for large files, it can not start decompressing until the entire file is downloaded.
claunia added the bugquestion labels 2026-01-29 22:08:07 +00:00
claunia commented 2026-01-29 22:08:08 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jun 3, 2017):

Worked for me

    class Program
    {
        static void Main(string[] args)
        {
            Test().Wait();
        }

        public static async Task Test()
        {
            using(var client = new HttpClient())
            using (var response = await client.GetAsync("https://github.com/adamhathcock/sharpcompress/raw/master/tests/TestArchives/Archives/Tar.tar",
                    HttpCompletionOption.ResponseHeadersRead))
            {
                var stream = await response.Content.ReadAsStreamAsync();
                using (var reader = ReaderFactory.Open(stream))
                {
                    reader.MoveToNextEntry();
                    reader.WriteEntryTo(new MemoryStream());  //System
                }
            }
        }
    }

Always use async/await and using statements please.

@adamhathcock commented on GitHub (Jun 3, 2017): Worked for me ```csharp class Program { static void Main(string[] args) { Test().Wait(); } public static async Task Test() { using(var client = new HttpClient()) using (var response = await client.GetAsync("https://github.com/adamhathcock/sharpcompress/raw/master/tests/TestArchives/Archives/Tar.tar", HttpCompletionOption.ResponseHeadersRead)) { var stream = await response.Content.ReadAsStreamAsync(); using (var reader = ReaderFactory.Open(stream)) { reader.MoveToNextEntry(); reader.WriteEntryTo(new MemoryStream()); //System } } } } ``` Always use `async`/`await` and `using` statements please.
claunia commented 2026-01-29 22:08:08 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jun 3, 2017):

This netcoreapp1.1 haven't tried 2.0

@adamhathcock commented on GitHub (Jun 3, 2017): This `netcoreapp1.1` haven't tried 2.0
claunia commented 2026-01-29 22:08:08 +00:00
Author
Owner
Copy Link

@jiabiao commented on GitHub (Jun 4, 2017):

Weird, your code throw the same exception on my end.
sharpcompress

I did use async/await and using statements in my actual code. The code in the original post is simplified for demonstration purposes.

@jiabiao commented on GitHub (Jun 4, 2017): Weird, your code throw the same exception on my end. ![sharpcompress](https://cloud.githubusercontent.com/assets/1818225/26758688/2b6fd446-4918-11e7-8fd9-5462360794e0.PNG) I did use ```async```/```await``` and ```using``` statements in my actual code. The code in the original post is simplified for demonstration purposes.
claunia commented 2026-01-29 22:08:08 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jun 4, 2017):

Weird.

Just in case I put my code here: https://github.com/adamhathcock/sc-http-test

@adamhathcock commented on GitHub (Jun 4, 2017): Weird. Just in case I put my code here: https://github.com/adamhathcock/sc-http-test
claunia commented 2026-01-29 22:08:09 +00:00
Author
Owner
Copy Link

@jiabiao commented on GitHub (Jun 4, 2017):

git clone & dotnet run still get the same exception.
I will debug with source to see what happened.

D:\Code\netcore>git clone https://github.com/adamhathcock/sc-http-test.git
Cloning into 'sc-http-test'...
remote: Counting objects: 6, done.
remote: Compressing objects: 100% (6/6), done.
Unpacking objects: 100% (6/6), done.
remote: Total 6 (delta 0), reused 6 (delta 0), pack-reused 0
Checking connectivity... done.

D:\Code\netcore>cd sc-http-test

D:\Code\netcore\sc-http-test>dotnet restore
  Restoring packages for D:\Code\netcore\sc-http-test\test.csproj...
  Generating MSBuild file D:\Code\netcore\sc-http-test\obj\test.csproj.nuget.g.props.
  Generating MSBuild file D:\Code\netcore\sc-http-test\obj\test.csproj.nuget.g.targets.
  Writing lock file to disk. Path: D:\Code\netcore\sc-http-test\obj\project.assets.json
  Restore completed in 1.46 sec for D:\Code\netcore\sc-http-test\test.csproj.

  NuGet Config files used:
      C:\Users\jiabiao\AppData\Roaming\NuGet\NuGet.Config
      C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config

  Feeds used:
      https://api.nuget.org/v3/index.json
      C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\

D:\Code\netcore\sc-http-test>dotnet run

Unhandled Exception: System.AggregateException: One or more errors occurred. (Unable to read data from the transport connection. The connection was closed before all data could be read. Expected 105472 bytes, read 512 bytes.) ---> System.IO.IOException: Unable to read data from the transport connection. The connection was closed before all data could be read. Expected 105472 bytes, read 512 bytes.
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at System.Net.Http.WinHttpResponseStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.Net.Http.DelegatingStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at SharpCompress.IO.RewindableStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at SharpCompress.IO.RewindableStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at SharpCompress.Common.Tar.TarReadOnlySubStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at SharpCompress.Common.EntryStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at SharpCompress.Common.EntryStream.SkipEntry()
   at SharpCompress.Common.EntryStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at SharpCompress.Readers.AbstractReader`2.Write(Stream writeStream)
   at SharpCompress.Readers.AbstractReader`2.WriteEntryTo(Stream writableStream)
   at test.Program.<Test>d__1.MoveNext() in D:\Code\netcore\sc-http-test\Program.cs:line 26
   --- End of inner exception stack trace ---
   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
   at System.Threading.Tasks.Task.Wait()
   at test.Program.Main(String[] args) in D:\Code\netcore\sc-http-test\Program.cs:line 13

D:\Code\netcore\sc-http-test>
@jiabiao commented on GitHub (Jun 4, 2017): git clone & dotnet run still get the same exception. I will debug with source to see what happened. ``` D:\Code\netcore>git clone https://github.com/adamhathcock/sc-http-test.git Cloning into 'sc-http-test'... remote: Counting objects: 6, done. remote: Compressing objects: 100% (6/6), done. Unpacking objects: 100% (6/6), done. remote: Total 6 (delta 0), reused 6 (delta 0), pack-reused 0 Checking connectivity... done. D:\Code\netcore>cd sc-http-test D:\Code\netcore\sc-http-test>dotnet restore Restoring packages for D:\Code\netcore\sc-http-test\test.csproj... Generating MSBuild file D:\Code\netcore\sc-http-test\obj\test.csproj.nuget.g.props. Generating MSBuild file D:\Code\netcore\sc-http-test\obj\test.csproj.nuget.g.targets. Writing lock file to disk. Path: D:\Code\netcore\sc-http-test\obj\project.assets.json Restore completed in 1.46 sec for D:\Code\netcore\sc-http-test\test.csproj. NuGet Config files used: C:\Users\jiabiao\AppData\Roaming\NuGet\NuGet.Config C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config Feeds used: https://api.nuget.org/v3/index.json C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\ D:\Code\netcore\sc-http-test>dotnet run Unhandled Exception: System.AggregateException: One or more errors occurred. (Unable to read data from the transport connection. The connection was closed before all data could be read. Expected 105472 bytes, read 512 bytes.) ---> System.IO.IOException: Unable to read data from the transport connection. The connection was closed before all data could be read. Expected 105472 bytes, read 512 bytes. at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult() at System.Net.Http.WinHttpResponseStream.Read(Byte[] buffer, Int32 offset, Int32 count) at System.Net.Http.DelegatingStream.Read(Byte[] buffer, Int32 offset, Int32 count) at SharpCompress.IO.RewindableStream.Read(Byte[] buffer, Int32 offset, Int32 count) at SharpCompress.IO.RewindableStream.Read(Byte[] buffer, Int32 offset, Int32 count) at SharpCompress.Common.Tar.TarReadOnlySubStream.Read(Byte[] buffer, Int32 offset, Int32 count) at SharpCompress.Common.EntryStream.Read(Byte[] buffer, Int32 offset, Int32 count) at SharpCompress.Common.EntryStream.SkipEntry() at SharpCompress.Common.EntryStream.Dispose(Boolean disposing) at System.IO.Stream.Close() at SharpCompress.Readers.AbstractReader`2.Write(Stream writeStream) at SharpCompress.Readers.AbstractReader`2.WriteEntryTo(Stream writableStream) at test.Program.<Test>d__1.MoveNext() in D:\Code\netcore\sc-http-test\Program.cs:line 26 --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at System.Threading.Tasks.Task.Wait() at test.Program.Main(String[] args) in D:\Code\netcore\sc-http-test\Program.cs:line 13 D:\Code\netcore\sc-http-test> ```
claunia commented 2026-01-29 22:08:09 +00:00
Author
Owner
Copy Link

@jiabiao commented on GitHub (Jun 4, 2017):

I created an issue https://github.com/dotnet/corefx/issues/20676.
Reading 0 byte from network stream will cause this exception.
So if the count parameter is 0, the RewindableStream.Read method should return 0 directly.

        public override int Read(byte[] buffer, int offset, int count)
        {
            if (count == 0)
            {
                return 0;
            }
@jiabiao commented on GitHub (Jun 4, 2017): I created an issue https://github.com/dotnet/corefx/issues/20676. Reading 0 byte from network stream will cause this exception. So if the count parameter is 0, the RewindableStream.Read method should return 0 directly. ```csharp public override int Read(byte[] buffer, int offset, int count) { if (count == 0) { return 0; } ```
claunia commented 2026-01-29 22:08:09 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jun 4, 2017):

I don't get how I didn't get the exception but you did. Just happened to read zero bytes and I didn't?

@adamhathcock commented on GitHub (Jun 4, 2017): I don't get how I didn't get the exception but you did. Just happened to read zero bytes and I didn't?
claunia commented 2026-01-29 22:08:10 +00:00
Author
Owner
Copy Link

@jiabiao commented on GitHub (Jun 5, 2017):

This issue occurs only on windows, Are you using macOS or Linux?
Otherwise, I don't know why you didn't get the exception. You can add a conditional breakpoints at the beginning of RewindableStream.Read method to investigate.

@jiabiao commented on GitHub (Jun 5, 2017): This issue occurs only on windows, Are you using macOS or Linux? Otherwise, I don't know why you didn't get the exception. You can add a conditional breakpoints at the beginning of RewindableStream.Read method to investigate.
claunia commented 2026-01-29 22:08:10 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jun 5, 2017):

You're right. I get this on Windows but not macOS

@adamhathcock commented on GitHub (Jun 5, 2017): You're right. I get this on Windows but not macOS
claunia commented 2026-01-29 22:08:10 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jun 9, 2017):

https://www.nuget.org/packages/sharpcompress/0.17.1

@adamhathcock commented on GitHub (Jun 9, 2017): https://www.nuget.org/packages/sharpcompress/0.17.1
claunia commented 2026-01-29 22:08:10 +00:00
Author
Owner
Copy Link

@jiabiao commented on GitHub (Jun 9, 2017):

thanks.

@jiabiao commented on GitHub (Jun 9, 2017): thanks.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
duplicate
enhancement
feedback requested
not bug
pull-request
Mirrored from GitHub Pull Request
 question
up for grabs
wontfix
No Label bug question
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/sharpcompress#196
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?