starred/sharpcompress
1
0
Fork 0
mirror of https://github.com/adamhathcock/sharpcompress.git synced 2026-02-07 21:22:04 +00:00
Code Issues 975 Packages Projects Releases 20 Wiki Activity

Use Native ZLib from .NET Core when possible. #177

New Issue
Open
opened 2026-01-29 22:07:51 +00:00 by claunia · 3 comments
claunia commented 2026-01-29 22:07:51 +00:00
Owner
Copy Link

Originally created by @adamhathcock on GitHub (May 19, 2017).

I believe this isn't possible yet as the ZLib internals aren't exposed.

https://github.com/dotnet/corefx/issues/7570

Originally created by @adamhathcock on GitHub (May 19, 2017). I believe this isn't possible yet as the ZLib internals aren't exposed. https://github.com/dotnet/corefx/issues/7570
claunia added the enhancement label 2026-01-29 22:07:51 +00:00
claunia commented 2026-01-29 22:07:52 +00:00
Author
Owner
Copy Link

@vvdb commented on GitHub (Jun 20, 2017):

Starting from .NET 4.5, DeflateStream already uses zlib (see the remarks in https://msdn.microsoft.com/en-us/library/system.io.compression.deflatestream(v=vs.110).aspx). I wonder if in those cases, you could use the built-in .NET version instead of your own. This would avoid an extra dependency while benefitting from the speedup.
There's a small problem, however: DeflateStream avoids writing the zlib CMF FLG header from RFC1950 , which generates interoperability issues. One way of solving this is to write a wrapper:

`
///


/// Two header bytes (CMF and FLG from RFC 1950) . Possible values are:
/// - 0x58 (CINFO CM), 0x85 (FLEVEL FDICT FCHECK) => 0101 1000 10 0 00101
/// - 0x78 (CINFO CM), 0x01 (FLEVEL FDICT FCHECK) => 0111 1000 00 0 00001
/// For the latter, this gives:
/// CINFO == 0111 == 7 == 2^(7+8) == 2^15 == 32K window size
/// CM == 1000 == 8 == deflate
/// FLEVEL == 00 = compressor using fastest algorithm (not used for decompression)
/// FDICT == 0 == no preset dictionary
/// FCHECK = 00001 == checksum bits for this header
/// See https://tools.ietf.org/html/rfc1950 form more details.
/// Starting from .NET 4.5, the uses zlib but omits these bytes, which causes the actual bytes not to be interoperable with zlib.
/// This is OK, even if we change the compression level since it's not used for decompression
///
private static readonly byte[] m_ZLibCompatibleHeader = { 0x78, 0x01 };

	/// <summary>
	/// Assuming the input stream is correctly positioned and initializer, this returns a <see cref="DeflateStream"/> such that whatever
	/// is read or written is compatible with zlib.
	/// </summary>
	/// <param name="stream"></param>
	/// <param name="mode"></param>
	/// <param name="leaveOpen"></param>
	/// <returns></returns>
	public static Stream GetCompressedStream(Stream stream, CompressionMode mode, bool leaveOpen)
	{
		switch (mode)
		{
			case CompressionMode.Compress:
				// We need to add a zlib-compatible header, because the C++ zlib.dll requires it and we want to remain compatible.
				stream.Write(m_ZLibCompatibleHeader, 0, m_ZLibCompatibleHeader.Length);
				return new DeflateStream(stream, CompressionMode.Compress, leaveOpen);
			case CompressionMode.Decompress:
				// we need to seek past zlib's 2-byte magic header, called CMF FLG because Microsoft's DeflateStream doesn't handle them.
				Skip(stream, m_ZLibCompatibleHeader.Length);
				return new DeflateStream(stream, CompressionMode.Decompress, leaveOpen);
			default:
				return null;
		}
	}
@vvdb commented on GitHub (Jun 20, 2017): Starting from .NET 4.5, DeflateStream already uses zlib (see the remarks in https://msdn.microsoft.com/en-us/library/system.io.compression.deflatestream(v=vs.110).aspx). I wonder if in those cases, you could use the built-in .NET version instead of your own. This would avoid an extra dependency while benefitting from the speedup. There's a small problem, however: DeflateStream avoids writing the zlib CMF FLG header from RFC1950 , which generates interoperability issues. One way of solving this is to write a wrapper: ` /// <summary> /// Two header bytes (CMF and FLG from RFC 1950) . Possible values are: /// - 0x58 (CINFO CM), 0x85 (FLEVEL FDICT FCHECK) => 0101 1000 10 0 00101 /// - 0x78 (CINFO CM), 0x01 (FLEVEL FDICT FCHECK) => 0111 1000 00 0 00001 /// For the latter, this gives: /// CINFO == 0111 == 7 == 2^(7+8) == 2^15 == 32K window size /// CM == 1000 == 8 == deflate /// FLEVEL == 00 = compressor using fastest algorithm (not used for decompression) /// FDICT == 0 == no preset dictionary /// FCHECK = 00001 == checksum bits for this header /// See https://tools.ietf.org/html/rfc1950 form more details. /// Starting from .NET 4.5, the <see cref="DeflateStream"/> uses zlib but omits these bytes, which causes the actual bytes not to be interoperable with zlib. /// This is OK, even if we change the compression level since it's not used for decompression /// </summary> private static readonly byte[] m_ZLibCompatibleHeader = { 0x78, 0x01 }; /// <summary> /// Assuming the input stream is correctly positioned and initializer, this returns a <see cref="DeflateStream"/> such that whatever /// is read or written is compatible with zlib. /// </summary> /// <param name="stream"></param> /// <param name="mode"></param> /// <param name="leaveOpen"></param> /// <returns></returns> public static Stream GetCompressedStream(Stream stream, CompressionMode mode, bool leaveOpen) { switch (mode) { case CompressionMode.Compress: // We need to add a zlib-compatible header, because the C++ zlib.dll requires it and we want to remain compatible. stream.Write(m_ZLibCompatibleHeader, 0, m_ZLibCompatibleHeader.Length); return new DeflateStream(stream, CompressionMode.Compress, leaveOpen); case CompressionMode.Decompress: // we need to seek past zlib's 2-byte magic header, called CMF FLG because Microsoft's DeflateStream doesn't handle them. Skip(stream, m_ZLibCompatibleHeader.Length); return new DeflateStream(stream, CompressionMode.Decompress, leaveOpen); default: return null; } }
claunia commented 2026-01-29 22:07:52 +00:00
Author
Owner
Copy Link

@adamhathcock commented on GitHub (Jun 20, 2017):

If you want to create a PR that implements this for net45, please do.

I'm worried there's a lot of little things required for usage. I first started with GZip and it's not possible because I need to get at headers in zlib that aren't exposed. It could be possible to use DeflateStream just for Zip files though.

@adamhathcock commented on GitHub (Jun 20, 2017): If you want to create a PR that implements this for net45, please do. I'm worried there's a lot of little things required for usage. I first started with GZip and it's not possible because I need to get at headers in zlib that aren't exposed. It could be possible to use DeflateStream just for Zip files though.
claunia commented 2026-01-29 22:07:52 +00:00
Author
Owner
Copy Link

@sstock commented on GitHub (Aug 9, 2019):

The above example that appends a zlib header to a deflate stream is not sufficient. The zlib requirements, RFC 1950, state:

A compliant compressor must produce streams with correct CMF, FLG and ADLER32, but need not support preset dictionaries.

It goes on to state that a compliant decompressor must check the ALDER32 checksum. Thus adding the header alone does not result in a valid zlib stream.

@sstock commented on GitHub (Aug 9, 2019): The above example that appends a zlib header to a deflate stream is not sufficient. The zlib requirements, [RFC 1950](https://tools.ietf.org/html/rfc1950), state: > A compliant compressor must produce streams with correct CMF, FLG and ADLER32, but need not support preset dictionaries. It goes on to state that a compliant decompressor must check the ALDER32 checksum. Thus adding the header alone does not result in a valid zlib stream.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
duplicate
enhancement
feedback requested
not bug
pull-request
Mirrored from GitHub Pull Request
 question
up for grabs
wontfix
No Label enhancement
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/sharpcompress#177
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?