TarWriter using GZip compression yields InvalidFormatException "Not a tar file." #140

Closed
opened 2026-01-29 22:07:17 +00:00 by claunia · 3 comments
Owner

Originally created by @michaelnoonan on GitHub (Oct 26, 2016).

Thanks for keeping up the work on this library! We just updated Octopus Deploy to use 0.14.0 and after fixing all of the compiler changes one of our tests started failing. The test confirms a class we use which converts Zip files to Tar files and it started failing with an InvalidFormatException: Not a tar file. when trying to read the resulting Tar stream. Inspecting the file contents it has the GZip magic bytes, but the file looks incomplete, like stream isn't flushed properly before the file stream is closed.

We worked around the problem by changing the usage of TarWriter to LeaveStreamOpen = false: using (var writer = new TarWriter(outputStream, new WriterOptions(CompressionType.GZip) { LeaveStreamOpen = false }))

It looks like this is a slight behavioural change possibly introduced in this PR: https://github.com/adamhathcock/sharpcompress/pull/138

A LINQPad reproduction: http://share.linqpad.net/mpow8f.linq

We have reconfigured our code to let the TarWriter close the output stream for us, and this works.

Wasn't sure if this is an indication of how you expected the library to be used, or whether a nuanced behavioural bug has slipped through. The interesting thing we noticed is how BZip streams are explicitly detected and "Finished" before being disposed - should GZip streams get the same treatment?

Hope that helps!
Mike

Originally created by @michaelnoonan on GitHub (Oct 26, 2016). Thanks for keeping up the work on this library! We just updated Octopus Deploy to use `0.14.0` and after fixing all of the compiler changes one of our tests started failing. The test confirms a class we use which converts Zip files to Tar files and it started failing with an `InvalidFormatException: Not a tar file.` when trying to read the resulting Tar stream. Inspecting the file contents it has the GZip magic bytes, but the file looks incomplete, like stream isn't flushed properly before the file stream is closed. We worked around the problem by changing the usage of `TarWriter` to `LeaveStreamOpen = false`: `using (var writer = new TarWriter(outputStream, new WriterOptions(CompressionType.GZip) { LeaveStreamOpen = false }))` It looks like this is a slight behavioural change possibly introduced in this PR: https://github.com/adamhathcock/sharpcompress/pull/138 A LINQPad reproduction: http://share.linqpad.net/mpow8f.linq We have reconfigured our code to let the `TarWriter` close the output stream for us, and this works. Wasn't sure if this is an indication of how you expected the library to be used, or whether a nuanced behavioural bug has slipped through. The interesting thing we noticed is how BZip streams are explicitly detected and "Finished" before being disposed - should GZip streams get the same treatment? Hope that helps! Mike
claunia added the question label 2026-01-29 22:07:17 +00:00
Author
Owner

@adamhathcock commented on GitHub (Oct 26, 2016):

Thanks for finding this inconsistency. It's always hard to know what the correct behavior is supposed to be as things are inconsistent everywhere.

I'm going to go for everything to NOT close a stream it didn't create. So writers/readers taking Streams in their constructor won't close a stream.

I tried to document this but maybe I wasn't clear enough: https://github.com/adamhathcock/sharpcompress/blob/master/USAGE.md

As for the difference between BZip2 and GZip, it's likely just a behavioral difference in the algorithms or implementations (which I didn't make :) ) As long as everything is good when invoking Flush or Dispose then I don't consider it an issue.

Any suggestions to help you or anyone else would be welcome.

@adamhathcock commented on GitHub (Oct 26, 2016): Thanks for finding this inconsistency. It's always hard to know what the correct behavior is supposed to be as things are inconsistent everywhere. I'm going to go for everything to NOT close a stream it didn't create. So writers/readers taking Streams in their constructor won't close a stream. I tried to document this but maybe I wasn't clear enough: https://github.com/adamhathcock/sharpcompress/blob/master/USAGE.md As for the difference between BZip2 and GZip, it's likely just a behavioral difference in the algorithms or implementations (which I didn't make :) ) As long as everything is good when invoking `Flush` or `Dispose` then I don't consider it an issue. Any suggestions to help you or anyone else would be welcome.
Author
Owner

@michaelnoonan commented on GitHub (Oct 26, 2016):

Hi @adamhathcock, thanks for getting back to me. I agree with your decision for consistency, and I also prefer the convention of "if someone handed me a stream, don't close it, just do the job I promised to do" and let the owner determine the lifetime.

I did see the documentation, and it's part of what led us to the final solution.

At this point, we're happy with the implementation we're using, mainly just wanted to share our findings.

Thanks!
Mike

@michaelnoonan commented on GitHub (Oct 26, 2016): Hi @adamhathcock, thanks for getting back to me. I agree with your decision for consistency, and I also prefer the convention of "if someone handed me a stream, don't close it, just do the job I promised to do" and let the owner determine the lifetime. I did see the documentation, and it's part of what led us to the final solution. At this point, we're happy with the implementation we're using, mainly just wanted to share our findings. Thanks! Mike
Author
Owner

@adamhathcock commented on GitHub (Oct 28, 2016):

Thanks for sharing!

@adamhathcock commented on GitHub (Oct 28, 2016): Thanks for sharing!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/sharpcompress#140