diff --git a/.circleci/config.yml b/.circleci/config.yml new file mode 100644 index 00000000..9150de77 --- /dev/null +++ b/.circleci/config.yml @@ -0,0 +1,15 @@ +version: 2 +jobs: + build: + docker: + - image: microsoft/dotnet:2.0.7-sdk-2.1.200 + steps: + - checkout + - run: + name: Install unzip + command: | + apt-get update + apt-get install -y unzip + - run: + name: Build + command: ./build.sh \ No newline at end of file diff --git a/.gitattributes b/.gitattributes index 2b6346c9..0a96b77c 100644 --- a/.gitattributes +++ b/.gitattributes @@ -2,4 +2,4 @@ * text=auto # need original files to be windows -test/TestArchives/Original/*.txt eol=crlf \ No newline at end of file +*.txt text eol=crlf \ No newline at end of file diff --git a/.gitignore b/.gitignore index 71becff1..cc539531 100644 --- a/.gitignore +++ b/.gitignore @@ -14,3 +14,6 @@ tests/TestArchives/Scratch .vs tools .vscode +.idea/ + +.DS_Store diff --git a/.travis.yml b/.travis.yml deleted file mode 100644 index 9737ad20..00000000 --- a/.travis.yml +++ /dev/null @@ -1,13 +0,0 @@ -dist: trusty -language: csharp -cache: - directories: - - $HOME/.dotnet -solution: SharpCompress.sln -matrix: - include: - - dotnet: 1.0.4 - mono: none - env: DOTNETCORE=1 -script: - - ./build.sh \ No newline at end of file diff --git a/FORMATS.md b/FORMATS.md index 09d5ae63..972c9c88 100644 --- a/FORMATS.md +++ b/FORMATS.md @@ -2,47 +2,48 @@ ## Accessing Archives -- Archive classes allow random access to a seekable stream. -- Reader classes allow forward-only reading on a stream. -- Writer classes allow forward-only Writing on a stream. +* Archive classes allow random access to a seekable stream. +* Reader classes allow forward-only reading on a stream. +* Writer classes allow forward-only Writing on a stream. ## Supported Format Table -| Archive Format | Compression Format(s) | Compress/Decompress | Archive API | Reader API | Writer API | -| --- | --- | --- | --- | --- | --- | -| Rar | Rar | Decompress (1) | RarArchive | RarReader | N/A | -| Zip (2) | None, DEFLATE, BZip2, LZMA/LZMA2, PPMd | Both | ZipArchive | ZipReader | ZipWriter | -| Tar | None | Both | TarArchive | TarReader | TarWriter (3) | -| Tar.GZip | DEFLATE | Both | TarArchive | TarReader | TarWriter (3) | -| Tar.BZip2 | BZip2 | Both | TarArchive | TarReader | TarWriter (3) | -| Tar.LZip | LZMA | Both | TarArchive | TarReader | TarWriter (3) | -| Tar.XZ | LZMA2 | Decompress | TarArchive | TarReader | TarWriter (3) | -| GZip (single file) | DEFLATE | Both | GZipArchive | GZipReader | GZipWriter | -| 7Zip (4) | LZMA, LZMA2, BZip2, PPMd, BCJ, BCJ2, Deflate | Decompress | SevenZipArchive | N/A | N/A | -| LZip (single file) (5) | LZip (LZMA) | Both | LZipArchive | LZipReader | LZipWriter | +| Archive Format | Compression Format(s) | Compress/Decompress | Archive API | Reader API | Writer API | +| ---------------------- | ------------------------------------------------- | ------------------- | --------------- | ---------- | ------------- | +| Rar | Rar | Decompress (1) | RarArchive | RarReader | N/A | +| Zip (2) | None, DEFLATE, Deflate64, BZip2, LZMA/LZMA2, PPMd | Both | ZipArchive | ZipReader | ZipWriter | +| Tar | None | Both | TarArchive | TarReader | TarWriter (3) | +| Tar.GZip | DEFLATE | Both | TarArchive | TarReader | TarWriter (3) | +| Tar.BZip2 | BZip2 | Both | TarArchive | TarReader | TarWriter (3) | +| Tar.LZip | LZMA | Both | TarArchive | TarReader | TarWriter (3) | +| Tar.XZ | LZMA2 | Decompress | TarArchive | TarReader | TarWriter (3) | +| GZip (single file) | DEFLATE | Both | GZipArchive | GZipReader | GZipWriter | +| 7Zip (4) | LZMA, LZMA2, BZip2, PPMd, BCJ, BCJ2, Deflate | Decompress | SevenZipArchive | N/A | N/A | +| LZip (single file) (5) | LZip (LZMA) | Both | LZipArchive | LZipReader | LZipWriter | - 1. SOLID Rars are only supported in the RarReader API. - 2. Zip format supports pkware and WinzipAES encryption. However, encrypted LZMA is not supported. Zip64 reading/writing is supported but only with seekable streams as the Zip spec doesn't support Zip64 data in post data descriptors. - 3. The Tar format requires a file size in the header. If no size is specified to the TarWriter and the stream is not seekable, then an exception will be thrown. - 4. The 7Zip format doesn't allow for reading as a forward-only stream so 7Zip is only supported through the Archive API - 5. LZip has no support for extra data like the file name or timestamp. There is a default filename used when looking at the entry Key on the archive. +1. SOLID Rars are only supported in the RarReader API. +2. Zip format supports pkware and WinzipAES encryption. However, encrypted LZMA is not supported. Zip64 reading/writing is supported but only with seekable streams as the Zip spec doesn't support Zip64 data in post data descriptors. Deflate64 is only supported for reading. +3. The Tar format requires a file size in the header. If no size is specified to the TarWriter and the stream is not seekable, then an exception will be thrown. +4. The 7Zip format doesn't allow for reading as a forward-only stream so 7Zip is only supported through the Archive API +5. LZip has no support for extra data like the file name or timestamp. There is a default filename used when looking at the entry Key on the archive. ## Compression Streams -For those who want to directly compress/decompress bits. The single file formats are represented here as well. However, BZip2, LZip and XZ have no metadata (GZip has a little) so using them without something like a Tar file makes little sense. +For those who want to directly compress/decompress bits. The single file formats are represented here as well. However, BZip2, LZip and XZ have no metadata (GZip has a little) so using them without something like a Tar file makes little sense. -| Compressor | Compress/Decompress | -| --- | --- | -| BZip2Stream | Both | -| GZipStream | Both | -| DeflateStream | Both | -| LZMAStream | Both | -| PPMdStream | Both | -| ADCStream | Decompress | -| LZipStream | Both | -| XZStream | Decompress | +| Compressor | Compress/Decompress | +| --------------- | ------------------- | +| BZip2Stream | Both | +| GZipStream | Both | +| DeflateStream | Both | +| Deflate64Stream | Decompress | +| LZMAStream | Both | +| PPMdStream | Both | +| ADCStream | Decompress | +| LZipStream | Both | +| XZStream | Decompress | -## Archive Formats vs Compression +## Archive Formats vs Compression Sometimes the terminology gets mixed. @@ -52,9 +53,8 @@ DEFLATE, LZMA are pure compression algorithms ### Formats -Formats like Zip, 7Zip, Rar are archive formats only. They use other compression methods (e.g. DEFLATE, LZMA, etc.) or propriatory (e.g RAR) +Formats like Zip, 7Zip, Rar are archive formats only. They use other compression methods (e.g. DEFLATE, LZMA, etc.) or propriatory (e.g RAR) ### Overlap -GZip, BZip2 and LZip are single file archival formats. The overlap in the API happens because Tar uses the single file formats as "compression" methods and the API tries to hide this a bit. - +GZip, BZip2 and LZip are single file archival formats. The overlap in the API happens because Tar uses the single file formats as "compression" methods and the API tries to hide this a bit. diff --git a/README.md b/README.md index 104efdcc..47fce793 100644 --- a/README.md +++ b/README.md @@ -2,34 +2,35 @@ SharpCompress is a compression library in pure C# for .NET 3.5, 4.5, .NET Standard 1.0, 1.3 that can unrar, un7zip, unzip, untar unbzip2 and ungzip with forward-only reading and file random access APIs. Write support for zip/tar/bzip2/gzip are implemented. -The major feature is support for non-seekable streams so large files can be processed on the fly (i.e. download stream). +The major feature is support for non-seekable streams so large files can be processed on the fly (i.e. download stream). -AppVeyor Build - +AppVeyor Build - [![Build status](https://ci.appveyor.com/api/projects/status/voxg971oemmvxh1e/branch/master?svg=true)](https://ci.appveyor.com/project/adamhathcock/sharpcompress/branch/master) -Travis CI Build - -[![Build Status](https://travis-ci.org/adamhathcock/sharpcompress.svg?branch=master)](https://travis-ci.org/adamhathcock/sharpcompress) +Circle CI Build - +[![CircleCI](https://circleci.com/gh/adamhathcock/sharpcompress.svg?style=svg)](https://circleci.com/gh/adamhathcock/sharpcompress) ## Need Help? + Post Issues on Github! Check the [Supported Formats](FORMATS.md) and [Basic Usage.](USAGE.md) ## Recommended Formats -In general, I recommend GZip (Deflate)/BZip2 (BZip)/LZip (LZMA) as the simplicity of the formats lend to better long term archival as well as the streamability. Tar is often used in conjunction for multiple files in a single archive (e.g. `.tar.gz`) +In general, I recommend GZip (Deflate)/BZip2 (BZip)/LZip (LZMA) as the simplicity of the formats lend to better long term archival as well as the streamability. Tar is often used in conjunction for multiple files in a single archive (e.g. `.tar.gz`) -Zip is okay, but it's a very hap-hazard format and the variation in headers and implementations makes it hard to get correct. Uses Deflate by default but supports a lot of compression methods. +Zip is okay, but it's a very hap-hazard format and the variation in headers and implementations makes it hard to get correct. Uses Deflate by default but supports a lot of compression methods. -RAR is not recommended as it's a propriatory format and the compression is closed source. Use Tar/LZip for LZMA +RAR is not recommended as it's a propriatory format and the compression is closed source. Use Tar/LZip for LZMA -7Zip and XZ both are overly complicated. 7Zip does not support streamable formats. XZ has known holes explained here: (http://www.nongnu.org/lzip/xz_inadequate.html) Use Tar/LZip for LZMA compression instead. +7Zip and XZ both are overly complicated. 7Zip does not support streamable formats. XZ has known holes explained here: (http://www.nongnu.org/lzip/xz_inadequate.html) Use Tar/LZip for LZMA compression instead. ## A Simple Request Hi everyone. I hope you're using SharpCompress and finding it useful. Please give me feedback on what you'd like to see changed especially as far as usability goes. New feature suggestions are always welcome as well. I would also like to know what projects SharpCompress is being used in. I like seeing how it is used to give me ideas for future versions. Thanks! -Please do not email me directly to ask for help. If you think there is a real issue, please report it here. +Please do not email me directly to ask for help. If you think there is a real issue, please report it here. ## Want to contribute? @@ -37,7 +38,7 @@ I'm always looking for help or ideas. Please submit code or email with ideas. Un ## TODOs (always lots) -* RAR 5 support +* RAR 5 decryption support * 7Zip writing * Zip64 (Need writing and extend Reading) * Multi-volume Zip support. @@ -54,9 +55,9 @@ I'm always looking for help or ideas. Please submit code or email with ideas. Un ### Version 0.17.0 -* New - Full LZip support! Can read and write LZip files and Tars inside LZip files. [Make LZip a first class citizen. #241](https://github.com/adamhathcock/sharpcompress/issues/241) -* New - XZ read support! Can read XZ files and Tars inside XZ files. [XZ in SharpCompress #91](https://github.com/adamhathcock/sharpcompress/issues/94) -* Fix - [Regression - zip file writing on seekable streams always assumed stream start was 0. Introduced with Zip64 writing.](https://github.com/adamhathcock/sharpcompress/issues/244) +* New - Full LZip support! Can read and write LZip files and Tars inside LZip files. [Make LZip a first class citizen. #241](https://github.com/adamhathcock/sharpcompress/issues/241) +* New - XZ read support! Can read XZ files and Tars inside XZ files. [XZ in SharpCompress #91](https://github.com/adamhathcock/sharpcompress/issues/94) +* Fix - [Regression - zip file writing on seekable streams always assumed stream start was 0. Introduced with Zip64 writing.](https://github.com/adamhathcock/sharpcompress/issues/244) * Fix - [Zip files with post-data descriptors can be properly skipped via decompression](https://github.com/adamhathcock/sharpcompress/issues/162) ### Version 0.16.2 @@ -136,7 +137,7 @@ I'm always looking for help or ideas. Please submit code or email with ideas. Un ### Version 0.11.6 * Bug fix for global header in Tar -* Writers now have a leaveOpen `bool` overload. They won't close streams if not-requested to. +* Writers now have a leaveOpen `bool` overload. They won't close streams if not-requested to. ### Version 0.11.5 @@ -155,7 +156,7 @@ I'm always looking for help or ideas. Please submit code or email with ideas. Un ### Version 0.11 -* Been over a year, contains mainly fixes from contributors! +* Been over a year, contains mainly fixes from contributors! * Possible breaking change: ArchiveEncoding is UTF8 by default now. * TAR supports writing long names using longlink * RAR Protect Header added diff --git a/SharpCompress.sln.DotSettings b/SharpCompress.sln.DotSettings index d6a99be9..5b016930 100644 --- a/SharpCompress.sln.DotSettings +++ b/SharpCompress.sln.DotSettings @@ -45,10 +45,14 @@ True 1 1 + NEVER + NEVER False False + NEVER True False + NEVER True True @@ -114,6 +118,11 @@ <Policy Inspect="True" Prefix="" Suffix="" Style="aaBb" /> <Policy Inspect="True" Prefix="" Suffix="" Style="AaBb" /> <Policy Inspect="True" Prefix="" Suffix="" Style="AaBb" /> + True + True + True + True True + True True True diff --git a/USAGE.md b/USAGE.md index 4d612aa4..a7a7f0d6 100644 --- a/USAGE.md +++ b/USAGE.md @@ -1,7 +1,8 @@ # SharpCompress Usage -## Stream Rules -When dealing with Streams, the rule should be that you don't close a stream you didn't create. This, in effect, should mean you should always put a Stream in a using block to dispose it. +## Stream Rules (changed with 0.21) + +When dealing with Streams, the rule should be that you don't close a stream you didn't create. This, in effect, should mean you should always put a Stream in a using block to dispose it. However, the .NET Framework often has classes that will dispose streams by default to make things "easy" like the following: @@ -12,7 +13,7 @@ using (var reader = new StreamReader(File.Open("foo"))) } ``` -In this example, reader should get disposed. However, stream rules should say the the `FileStream` created by `File.Open` should remain open. However, the .NET Framework closes it for you by default unless you override the constructor. In general, you should be writing Stream code like this: +In this example, reader should get disposed. However, stream rules should say the the `FileStream` created by `File.Open` should remain open. However, the .NET Framework closes it for you by default unless you override the constructor. In general, you should be writing Stream code like this: ```C# using (var fileStream = File.Open("foo")) @@ -22,16 +23,15 @@ using (var reader = new StreamReader(fileStream)) } ``` -To deal with the "correct" rules as well as the expectations of users, I've decided on this: - -* When writing, leave streams open. -* When reading, close streams +To deal with the "correct" rules as well as the expectations of users, I've decided to always close wrapped streams as of 0.21. To be explicit though, consider always using the overloads that use `ReaderOptions` or `WriterOptions` and explicitly set `LeaveStreamOpen` the way you want. +If using Compression Stream classes directly and you don't want the wrapped stream to be closed. Use the `NonDisposingStream` as a wrapped to prevent the stream being disposed. The change in 0.21 simplified a lot even though the usage is a bit more convoluted. + ## Samples -Also, look over the tests for more thorough [examples](https://github.com/adamhathcock/sharpcompress/tree/master/test/SharpCompress.Test) +Also, look over the tests for more thorough [examples](https://github.com/adamhathcock/sharpcompress/tree/master/tests/SharpCompress.Test) ### Create Zip Archive from all files in a directory to a file @@ -43,11 +43,9 @@ using (var archive = ZipArchive.Create()) } ``` - ### Create Zip Archive from all files in a directory and save in memory ```C# - var memoryStream = new MemoryStream(); using (var archive = ZipArchive.Create()) { @@ -129,3 +127,20 @@ using (var writer = WriterFactory.Open(stream, ArchiveType.Tar, new WriterOption writer.WriteAll("D:\\temp", "*", SearchOption.AllDirectories); } ``` + +### Extract zip which has non-utf8 encoded filename(cp932) + +```C# +var opts = new SharpCompress.Readers.ReaderOptions(); +var encoding = Encoding.GetEncoding(932); +opts.ArchiveEncoding = new SharpCompress.Common.ArchiveEncoding(); +opts.ArchiveEncoding.CustomDecoder = (data, x, y) => +{ + return encoding.GetString(data); +}; +var tr = SharpCompress.Archives.Zip.ZipArchive.Open("test.zip", opts); +foreach(var entry in tr.Entries) +{ + Console.WriteLine($"{entry.Key}"); +} +``` \ No newline at end of file diff --git a/build.cake b/build.cake index c1d19980..3fc65799 100644 --- a/build.cake +++ b/build.cake @@ -25,13 +25,17 @@ Task("Build") var settings = new DotNetCoreBuildSettings { Framework = "netstandard1.0", - Configuration = "Release" + Configuration = "Release", + NoRestore = true }; DotNetCoreBuild("./src/SharpCompress/SharpCompress.csproj", settings); - settings.Framework = "netcoreapp1.1"; - DotNetCoreBuild("./tests/SharpCompress.Test/SharpCompress.Test.csproj", settings); + settings.Framework = "netstandard1.3"; + DotNetCoreBuild("./src/SharpCompress/SharpCompress.csproj", settings); + + settings.Framework = "netstandard2.0"; + DotNetCoreBuild("./src/SharpCompress/SharpCompress.csproj", settings); } }); @@ -39,23 +43,15 @@ Task("Test") .IsDependentOn("Build") .Does(() => { - if (!bool.Parse(EnvironmentVariable("APPVEYOR") ?? "false") - && !bool.Parse(EnvironmentVariable("TRAVIS") ?? "false")) + var files = GetFiles("tests/**/*.csproj"); + foreach(var file in files) { - var files = GetFiles("tests/**/*.csproj"); - foreach(var file in files) + var settings = new DotNetCoreTestSettings { - var settings = new DotNetCoreTestSettings - { - Configuration = "Release" - }; - - DotNetCoreTest(file.ToString(), settings); - } - } - else - { - Information("Skipping tests as this is AppVeyor or Travis CI"); + Configuration = "Release", + Framework = "netcoreapp2.0" + }; + DotNetCoreTest(file.ToString(), settings); } }); diff --git a/build.sh b/build.sh index 9ed17711..d478e0a7 100755 --- a/build.sh +++ b/build.sh @@ -8,7 +8,7 @@ # Define directories. SCRIPT_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd ) TOOLS_DIR=$SCRIPT_DIR/tools -CAKE_VERSION=0.19.1 +CAKE_VERSION=0.27.1 CAKE_DLL=$TOOLS_DIR/Cake.CoreCLR.$CAKE_VERSION/Cake.dll # Make sure the tools folder exist. diff --git a/reference/APPNOTE.TXT b/reference/APPNOTE.TXT new file mode 100644 index 00000000..c89d8a8e --- /dev/null +++ b/reference/APPNOTE.TXT @@ -0,0 +1,3497 @@ +File: APPNOTE.TXT - .ZIP File Format Specification +Version: 6.3.4 +Status: Final - replaces version 6.3.3 +Revised: October 1, 2014 +Copyright (c) 1989 - 2014 PKWARE Inc., All Rights Reserved. + +1.0 Introduction +--------------- + +1.1 Purpose +----------- + + 1.1.1 This specification is intended to define a cross-platform, + interoperable file storage and transfer format. Since its + first publication in 1989, PKWARE, Inc. ("PKWARE") has remained + committed to ensuring the interoperability of the .ZIP file + format through periodic publication and maintenance of this + specification. We trust that all .ZIP compatible vendors and + application developers that use and benefit from this format + will share and support this commitment to interoperability. + +1.2 Scope +--------- + + 1.2.1 ZIP is one of the most widely used compressed file formats. It is + universally used to aggregate, compress, and encrypt files into a single + interoperable container. No specific use or application need is + defined by this format and no specific implementation guidance is + provided. This document provides details on the storage format for + creating ZIP files. Information is provided on the records and + fields that describe what a ZIP file is. + +1.3 Trademarks +-------------- + + 1.3.1 PKWARE, PKZIP, SecureZIP, and PKSFX are registered trademarks of + PKWARE, Inc. in the United States and elsewhere. PKPatchMaker, + Deflate64, and ZIP64 are trademarks of PKWARE, Inc. Other marks + referenced within this document appear for identification + purposes only and are the property of their respective owners. + + +1.4 Permitted Use +----------------- + + 1.4.1 This document, "APPNOTE.TXT - .ZIP File Format Specification" is the + exclusive property of PKWARE. Use of the information contained in this + document is permitted solely for the purpose of creating products, + programs and processes that read and write files in the ZIP format + subject to the terms and conditions herein. + + 1.4.2 Use of the content of this document within other publications is + permitted only through reference to this document. Any reproduction + or distribution of this document in whole or in part without prior + written permission from PKWARE is strictly prohibited. + + 1.4.3 Certain technological components provided in this document are the + patented proprietary technology of PKWARE and as such require a + separate, executed license agreement from PKWARE. Applicable + components are marked with the following, or similar, statement: + 'Refer to the section in this document entitled "Incorporating + PKWARE Proprietary Technology into Your Product" for more information'. + +1.5 Contacting PKWARE +--------------------- + + 1.5.1 If you have questions on this format, its use, or licensing, or if you + wish to report defects, request changes or additions, please contact: + + PKWARE, Inc. + 201 E. Pittsburgh Avenue, Suite 400 + Milwaukee, WI 53204 + +1-414-289-9788 + +1-414-289-9789 FAX + zipformat@pkware.com + + 1.5.2 Information about this format and copies of this document are publicly + available at: + + http://www.pkware.com/appnote + +1.6 Disclaimer +-------------- + + 1.6.1 Although PKWARE will attempt to supply current and accurate + information relating to its file formats, algorithms, and the + subject programs, the possibility of error or omission cannot + be eliminated. PKWARE therefore expressly disclaims any warranty + that the information contained in the associated materials relating + to the subject programs and/or the format of the files created or + accessed by the subject programs and/or the algorithms used by + the subject programs, or any other matter, is current, correct or + accurate as delivered. Any risk of damage due to any possible + inaccurate information is assumed by the user of the information. + Furthermore, the information relating to the subject programs + and/or the file formats created or accessed by the subject + programs and/or the algorithms used by the subject programs is + subject to change without notice. + +2.0 Revisions +-------------- + +2.1 Document Status +-------------------- + + 2.1.1 If the STATUS of this file is marked as DRAFT, the content + defines proposed revisions to this specification which may consist + of changes to the ZIP format itself, or that may consist of other + content changes to this document. Versions of this document and + the format in DRAFT form may be subject to modification prior to + publication STATUS of FINAL. DRAFT versions are published periodically + to provide notification to the ZIP community of pending changes and to + provide opportunity for review and comment. + + 2.1.2 Versions of this document having a STATUS of FINAL are + considered to be in the final form for that version of the document + and are not subject to further change until a new, higher version + numbered document is published. Newer versions of this format + specification are intended to remain interoperable with with all prior + versions whenever technically possible. + +2.2 Change Log +-------------- + + Version Change Description Date + ------- ------------------ ---------- + 5.2 -Single Password Symmetric Encryption 07/16/2003 + storage + + 6.1.0 -Smartcard compatibility 01/20/2004 + -Documentation on certificate storage + + 6.2.0 -Introduction of Central Directory 04/26/2004 + Encryption for encrypting metadata + -Added OS X to Version Made By values + + 6.2.1 -Added Extra Field placeholder for 04/01/2005 + POSZIP using ID 0x4690 + + -Clarified size field on + "zip64 end of central directory record" + + 6.2.2 -Documented Final Feature Specification 01/06/2006 + for Strong Encryption + + -Clarifications and typographical + corrections + + 6.3.0 -Added tape positioning storage 09/29/2006 + parameters + + -Expanded list of supported hash algorithms + + -Expanded list of supported compression + algorithms + + -Expanded list of supported encryption + algorithms + + -Added option for Unicode filename + storage + + -Clarifications for consistent use + of Data Descriptor records + + -Added additional "Extra Field" + definitions + + 6.3.1 -Corrected standard hash values for 04/11/2007 + SHA-256/384/512 + + 6.3.2 -Added compression method 97 09/28/2007 + + -Documented InfoZIP "Extra Field" + values for UTF-8 file name and + file comment storage + + 6.3.3 -Formatting changes to support 09/01/2012 + easier referencing of this APPNOTE + from other documents and standards + + 6.3.4 -Address change 10/01/2014 + + +3.0 Notations +------------- + + 3.1 Use of the term MUST or SHALL indicates a required element. + + 3.2 MAY NOT or SHALL NOT indicates an element is prohibited from use. + + 3.3 SHOULD indicates a RECOMMENDED element. + + 3.4 SHOULD NOT indicates an element NOT RECOMMENDED for use. + + 3.5 MAY indicates an OPTIONAL element. + + +4.0 ZIP Files +------------- + +4.1 What is a ZIP file +---------------------- + + 4.1.1 ZIP files MAY be identified by the standard .ZIP file extension + although use of a file extension is not required. Use of the + extension .ZIPX is also recognized and MAY be used for ZIP files. + Other common file extensions using the ZIP format include .JAR, .WAR, + .DOCX, .XLXS, .PPTX, .ODT, .ODS, .ODP and others. Programs reading or + writing ZIP files SHOULD rely on internal record signatures described + in this document to identify files in this format. + + 4.1.2 ZIP files SHOULD contain at least one file and MAY contain + multiple files. + + 4.1.3 Data compression MAY be used to reduce the size of files + placed into a ZIP file, but is not required. This format supports the + use of multiple data compression algorithms. When compression is used, + one of the documented compression algorithms MUST be used. Implementors + are advised to experiment with their data to determine which of the + available algorithms provides the best compression for their needs. + Compression method 8 (Deflate) is the method used by default by most + ZIP compatible application programs. + + + 4.1.4 Data encryption MAY be used to protect files within a ZIP file. + Keying methods supported for encryption within this format include + passwords and public/private keys. Either MAY be used individually + or in combination. Encryption MAY be applied to individual files. + Additional security MAY be used through the encryption of ZIP file + metadata stored within the Central Directory. See the section on the + Strong Encryption Specification for information. Refer to the section + in this document entitled "Incorporating PKWARE Proprietary Technology + into Your Product" for more information. + + 4.1.5 Data integrity MUST be provided for each file using CRC32. + + 4.1.6 Additional data integrity MAY be included through the use of + digital signatures. Individual files MAY be signed with one or more + digital signatures. The Central Directory, if signed, MUST use a + single signature. + + 4.1.7 Files MAY be placed within a ZIP file uncompressed or stored. + The term "stored" as used in the context of this document means the file + is copied into the ZIP file uncompressed. + + 4.1.8 Each data file placed into a ZIP file MAY be compressed, stored, + encrypted or digitally signed independent of how other data files in the + same ZIP file are archived. + + 4.1.9 ZIP files MAY be streamed, split into segments (on fixed or on + removable media) or "self-extracting". Self-extracting ZIP + files MUST include extraction code for a target platform within + the ZIP file. + + 4.1.10 Extensibility is provided for platform or application specific + needs through extra data fields that MAY be defined for custom + purposes. Extra data definitions MUST NOT conflict with existing + documented record definitions. + + 4.1.11 Common uses for ZIP MAY also include the use of manifest files. + Manifest files store application specific information within a file stored + within the ZIP file. This manifest file SHOULD be the first file in the + ZIP file. This specification does not provide any information or guidance on + the use of manifest files within ZIP files. Refer to the application developer + for information on using manifest files and for any additional profile + information on using ZIP within an application. + + 4.1.12 ZIP files MAY be placed within other ZIP files. + +4.2 ZIP Metadata +---------------- + + 4.2.1 ZIP files are identified by metadata consisting of defined record types + containing the storage information necessary for maintaining the files + placed into a ZIP file. Each record type MUST be identified using a header + signature that identifies the record type. Signature values begin with the + two byte constant marker of 0x4b50, representing the characters "PK". + + +4.3 General Format of a .ZIP file +--------------------------------- + + 4.3.1 A ZIP file MUST contain an "end of central directory record". A ZIP + file containing only an "end of central directory record" is considered an + empty ZIP file. Files may be added or replaced within a ZIP file, or deleted. + A ZIP file MUST have only one "end of central directory record". Other + records defined in this specification MAY be used as needed to support + storage requirements for individual ZIP files. + + 4.3.2 Each file placed into a ZIP file MUST be preceeded by a "local + file header" record for that file. Each "local file header" MUST be + accompanied by a corresponding "central directory header" record within + the central directory section of the ZIP file. + + 4.3.3 Files MAY be stored in arbitrary order within a ZIP file. A ZIP + file MAY span multiple volumes or it MAY be split into user-defined + segment sizes. All values MUST be stored in little-endian byte order unless + otherwise specified in this document for a specific data element. + + 4.3.4 Compression MUST NOT be applied to a "local file header", an "encryption + header", or an "end of central directory record". Individual "central + directory records" must not be compressed, but the aggregate of all central + directory records MAY be compressed. + + 4.3.5 File data MAY be followed by a "data descriptor" for the file. Data + descriptors are used to facilitate ZIP file streaming. + + + 4.3.6 Overall .ZIP file format: + + [local file header 1] + [encryption header 1] + [file data 1] + [data descriptor 1] + . + . + . + [local file header n] + [encryption header n] + [file data n] + [data descriptor n] + [archive decryption header] + [archive extra data record] + [central directory header 1] + . + . + . + [central directory header n] + [zip64 end of central directory record] + [zip64 end of central directory locator] + [end of central directory record] + + + 4.3.7 Local file header: + + local file header signature 4 bytes (0x04034b50) + version needed to extract 2 bytes + general purpose bit flag 2 bytes + compression method 2 bytes + last mod file time 2 bytes + last mod file date 2 bytes + crc-32 4 bytes + compressed size 4 bytes + uncompressed size 4 bytes + file name length 2 bytes + extra field length 2 bytes + + file name (variable size) + extra field (variable size) + + 4.3.8 File data + + Immediately following the local header for a file + SHOULD be placed the compressed or stored data for the file. + If the file is encrypted, the encryption header for the file + SHOULD be placed after the local header and before the file + data. The series of [local file header][encryption header] + [file data][data descriptor] repeats for each file in the + .ZIP archive. + + Zero-byte files, directories, and other file types that + contain no content MUST not include file data. + + 4.3.9 Data descriptor: + + crc-32 4 bytes + compressed size 4 bytes + uncompressed size 4 bytes + + 4.3.9.1 This descriptor MUST exist if bit 3 of the general + purpose bit flag is set (see below). It is byte aligned + and immediately follows the last byte of compressed data. + This descriptor SHOULD be used only when it was not possible to + seek in the output .ZIP file, e.g., when the output .ZIP file + was standard output or a non-seekable device. For ZIP64(tm) format + archives, the compressed and uncompressed sizes are 8 bytes each. + + 4.3.9.2 When compressing files, compressed and uncompressed sizes + should be stored in ZIP64 format (as 8 byte values) when a + file's size exceeds 0xFFFFFFFF. However ZIP64 format may be + used regardless of the size of a file. When extracting, if + the zip64 extended information extra field is present for + the file the compressed and uncompressed sizes will be 8 + byte values. + + 4.3.9.3 Although not originally assigned a signature, the value + 0x08074b50 has commonly been adopted as a signature value + for the data descriptor record. Implementers should be + aware that ZIP files may be encountered with or without this + signature marking data descriptors and SHOULD account for + either case when reading ZIP files to ensure compatibility. + + 4.3.9.4 When writing ZIP files, implementors SHOULD include the + signature value marking the data descriptor record. When + the signature is used, the fields currently defined for + the data descriptor record will immediately follow the + signature. + + 4.3.9.5 An extensible data descriptor will be released in a + future version of this APPNOTE. This new record is intended to + resolve conflicts with the use of this record going forward, + and to provide better support for streamed file processing. + + 4.3.9.6 When the Central Directory Encryption method is used, + the data descriptor record is not required, but MAY be used. + If present, and bit 3 of the general purpose bit field is set to + indicate its presence, the values in fields of the data descriptor + record MUST be set to binary zeros. See the section on the Strong + Encryption Specification for information. Refer to the section in + this document entitled "Incorporating PKWARE Proprietary Technology + into Your Product" for more information. + + + 4.3.10 Archive decryption header: + + 4.3.10.1 The Archive Decryption Header is introduced in version 6.2 + of the ZIP format specification. This record exists in support + of the Central Directory Encryption Feature implemented as part of + the Strong Encryption Specification as described in this document. + When the Central Directory Structure is encrypted, this decryption + header MUST precede the encrypted data segment. + + 4.3.10.2 The encrypted data segment SHALL consist of the Archive + extra data record (if present) and the encrypted Central Directory + Structure data. The format of this data record is identical to the + Decryption header record preceding compressed file data. If the + central directory structure is encrypted, the location of the start of + this data record is determined using the Start of Central Directory + field in the Zip64 End of Central Directory record. See the + section on the Strong Encryption Specification for information + on the fields used in the Archive Decryption Header record. + Refer to the section in this document entitled "Incorporating + PKWARE Proprietary Technology into Your Product" for more information. + + + 4.3.11 Archive extra data record: + + archive extra data signature 4 bytes (0x08064b50) + extra field length 4 bytes + extra field data (variable size) + + 4.3.11.1 The Archive Extra Data Record is introduced in version 6.2 + of the ZIP format specification. This record MAY be used in support + of the Central Directory Encryption Feature implemented as part of + the Strong Encryption Specification as described in this document. + When present, this record MUST immediately precede the central + directory data structure. + + 4.3.11.2 The size of this data record SHALL be included in the + Size of the Central Directory field in the End of Central + Directory record. If the central directory structure is compressed, + but not encrypted, the location of the start of this data record is + determined using the Start of Central Directory field in the Zip64 + End of Central Directory record. Refer to the section in this document + entitled "Incorporating PKWARE Proprietary Technology into Your + Product" for more information. + + 4.3.12 Central directory structure: + + [central directory header 1] + . + . + . + [central directory header n] + [digital signature] + + File header: + + central file header signature 4 bytes (0x02014b50) + version made by 2 bytes + version needed to extract 2 bytes + general purpose bit flag 2 bytes + compression method 2 bytes + last mod file time 2 bytes + last mod file date 2 bytes + crc-32 4 bytes + compressed size 4 bytes + uncompressed size 4 bytes + file name length 2 bytes + extra field length 2 bytes + file comment length 2 bytes + disk number start 2 bytes + internal file attributes 2 bytes + external file attributes 4 bytes + relative offset of local header 4 bytes + + file name (variable size) + extra field (variable size) + file comment (variable size) + + 4.3.13 Digital signature: + + header signature 4 bytes (0x05054b50) + size of data 2 bytes + signature data (variable size) + + With the introduction of the Central Directory Encryption + feature in version 6.2 of this specification, the Central + Directory Structure MAY be stored both compressed and encrypted. + Although not required, it is assumed when encrypting the + Central Directory Structure, that it will be compressed + for greater storage efficiency. Information on the + Central Directory Encryption feature can be found in the section + describing the Strong Encryption Specification. The Digital + Signature record will be neither compressed nor encrypted. + + 4.3.14 Zip64 end of central directory record + + zip64 end of central dir + signature 4 bytes (0x06064b50) + size of zip64 end of central + directory record 8 bytes + version made by 2 bytes + version needed to extract 2 bytes + number of this disk 4 bytes + number of the disk with the + start of the central directory 4 bytes + total number of entries in the + central directory on this disk 8 bytes + total number of entries in the + central directory 8 bytes + size of the central directory 8 bytes + offset of start of central + directory with respect to + the starting disk number 8 bytes + zip64 extensible data sector (variable size) + + 4.3.14.1 The value stored into the "size of zip64 end of central + directory record" should be the size of the remaining + record and should not include the leading 12 bytes. + + Size = SizeOfFixedFields + SizeOfVariableData - 12. + + 4.3.14.2 The above record structure defines Version 1 of the + zip64 end of central directory record. Version 1 was + implemented in versions of this specification preceding + 6.2 in support of the ZIP64 large file feature. The + introduction of the Central Directory Encryption feature + implemented in version 6.2 as part of the Strong Encryption + Specification defines Version 2 of this record structure. + Refer to the section describing the Strong Encryption + Specification for details on the version 2 format for + this record. Refer to the section in this document entitled + "Incorporating PKWARE Proprietary Technology into Your Product" + for more information applicable to use of Version 2 of this + record. + + 4.3.14.3 Special purpose data MAY reside in the zip64 extensible + data sector field following either a V1 or V2 version of this + record. To ensure identification of this special purpose data + it must include an identifying header block consisting of the + following: + + Header ID - 2 bytes + Data Size - 4 bytes + + The Header ID field indicates the type of data that is in the + data block that follows. + + Data Size identifies the number of bytes that follow for this + data block type. + + 4.3.14.4 Multiple special purpose data blocks MAY be present. + Each MUST be preceded by a Header ID and Data Size field. Current + mappings of Header ID values supported in this field are as + defined in APPENDIX C. + + 4.3.15 Zip64 end of central directory locator + + zip64 end of central dir locator + signature 4 bytes (0x07064b50) + number of the disk with the + start of the zip64 end of + central directory 4 bytes + relative offset of the zip64 + end of central directory record 8 bytes + total number of disks 4 bytes + + 4.3.16 End of central directory record: + + end of central dir signature 4 bytes (0x06054b50) + number of this disk 2 bytes + number of the disk with the + start of the central directory 2 bytes + total number of entries in the + central directory on this disk 2 bytes + total number of entries in + the central directory 2 bytes + size of the central directory 4 bytes + offset of start of central + directory with respect to + the starting disk number 4 bytes + .ZIP file comment length 2 bytes + .ZIP file comment (variable size) + +4.4 Explanation of fields +-------------------------- + + 4.4.1 General notes on fields + + 4.4.1.1 All fields unless otherwise noted are unsigned and stored + in Intel low-byte:high-byte, low-word:high-word order. + + 4.4.1.2 String fields are not null terminated, since the length + is given explicitly. + + 4.4.1.3 The entries in the central directory may not necessarily + be in the same order that files appear in the .ZIP file. + + 4.4.1.4 If one of the fields in the end of central directory + record is too small to hold required data, the field should be + set to -1 (0xFFFF or 0xFFFFFFFF) and the ZIP64 format record + should be created. + + 4.4.1.5 The end of central directory record and the Zip64 end + of central directory locator record MUST reside on the same + disk when splitting or spanning an archive. + + 4.4.2 version made by (2 bytes) + + 4.4.2.1 The upper byte indicates the compatibility of the file + attribute information. If the external file attributes + are compatible with MS-DOS and can be read by PKZIP for + DOS version 2.04g then this value will be zero. If these + attributes are not compatible, then this value will + identify the host system on which the attributes are + compatible. Software can use this information to determine + the line record format for text files etc. + + 4.4.2.2 The current mappings are: + + 0 - MS-DOS and OS/2 (FAT / VFAT / FAT32 file systems) + 1 - Amiga 2 - OpenVMS + 3 - UNIX 4 - VM/CMS + 5 - Atari ST 6 - OS/2 H.P.F.S. + 7 - Macintosh 8 - Z-System + 9 - CP/M 10 - Windows NTFS + 11 - MVS (OS/390 - Z/OS) 12 - VSE + 13 - Acorn Risc 14 - VFAT + 15 - alternate MVS 16 - BeOS + 17 - Tandem 18 - OS/400 + 19 - OS X (Darwin) 20 thru 255 - unused + + 4.4.2.3 The lower byte indicates the ZIP specification version + (the version of this document) supported by the software + used to encode the file. The value/10 indicates the major + version number, and the value mod 10 is the minor version + number. + + 4.4.3 version needed to extract (2 bytes) + + 4.4.3.1 The minimum supported ZIP specification version needed + to extract the file, mapped as above. This value is based on + the specific format features a ZIP program MUST support to + be able to extract the file. If multiple features are + applied to a file, the minimum version MUST be set to the + feature having the highest value. New features or feature + changes affecting the published format specification will be + implemented using higher version numbers than the last + published value to avoid conflict. + + 4.4.3.2 Current minimum feature versions are as defined below: + + 1.0 - Default value + 1.1 - File is a volume label + 2.0 - File is a folder (directory) + 2.0 - File is compressed using Deflate compression + 2.0 - File is encrypted using traditional PKWARE encryption + 2.1 - File is compressed using Deflate64(tm) + 2.5 - File is compressed using PKWARE DCL Implode + 2.7 - File is a patch data set + 4.5 - File uses ZIP64 format extensions + 4.6 - File is compressed using BZIP2 compression* + 5.0 - File is encrypted using DES + 5.0 - File is encrypted using 3DES + 5.0 - File is encrypted using original RC2 encryption + 5.0 - File is encrypted using RC4 encryption + 5.1 - File is encrypted using AES encryption + 5.1 - File is encrypted using corrected RC2 encryption** + 5.2 - File is encrypted using corrected RC2-64 encryption** + 6.1 - File is encrypted using non-OAEP key wrapping*** + 6.2 - Central directory encryption + 6.3 - File is compressed using LZMA + 6.3 - File is compressed using PPMd+ + 6.3 - File is encrypted using Blowfish + 6.3 - File is encrypted using Twofish + + 4.4.3.3 Notes on version needed to extract + + * Early 7.x (pre-7.2) versions of PKZIP incorrectly set the + version needed to extract for BZIP2 compression to be 50 + when it should have been 46. + + ** Refer to the section on Strong Encryption Specification + for additional information regarding RC2 corrections. + + *** Certificate encryption using non-OAEP key wrapping is the + intended mode of operation for all versions beginning with 6.1. + Support for OAEP key wrapping MUST only be used for + backward compatibility when sending ZIP files to be opened by + versions of PKZIP older than 6.1 (5.0 or 6.0). + + + Files compressed using PPMd MUST set the version + needed to extract field to 6.3, however, not all ZIP + programs enforce this and may be unable to decompress + data files compressed using PPMd if this value is set. + + When using ZIP64 extensions, the corresponding value in the + zip64 end of central directory record MUST also be set. + This field should be set appropriately to indicate whether + Version 1 or Version 2 format is in use. + + + 4.4.4 general purpose bit flag: (2 bytes) + + Bit 0: If set, indicates that the file is encrypted. + + (For Method 6 - Imploding) + Bit 1: If the compression method used was type 6, + Imploding, then this bit, if set, indicates + an 8K sliding dictionary was used. If clear, + then a 4K sliding dictionary was used. + + Bit 2: If the compression method used was type 6, + Imploding, then this bit, if set, indicates + 3 Shannon-Fano trees were used to encode the + sliding dictionary output. If clear, then 2 + Shannon-Fano trees were used. + + (For Methods 8 and 9 - Deflating) + Bit 2 Bit 1 + 0 0 Normal (-en) compression option was used. + 0 1 Maximum (-exx/-ex) compression option was used. + 1 0 Fast (-ef) compression option was used. + 1 1 Super Fast (-es) compression option was used. + + (For Method 14 - LZMA) + Bit 1: If the compression method used was type 14, + LZMA, then this bit, if set, indicates + an end-of-stream (EOS) marker is used to + mark the end of the compressed data stream. + If clear, then an EOS marker is not present + and the compressed data size must be known + to extract. + + Note: Bits 1 and 2 are undefined if the compression + method is any other. + + Bit 3: If this bit is set, the fields crc-32, compressed + size and uncompressed size are set to zero in the + local header. The correct values are put in the + data descriptor immediately following the compressed + data. (Note: PKZIP version 2.04g for DOS only + recognizes this bit for method 8 compression, newer + versions of PKZIP recognize this bit for any + compression method.) + + Bit 4: Reserved for use with method 8, for enhanced + deflating. + + Bit 5: If this bit is set, this indicates that the file is + compressed patched data. (Note: Requires PKZIP + version 2.70 or greater) + + Bit 6: Strong encryption. If this bit is set, you MUST + set the version needed to extract value to at least + 50 and you MUST also set bit 0. If AES encryption + is used, the version needed to extract value MUST + be at least 51. See the section describing the Strong + Encryption Specification for details. Refer to the + section in this document entitled "Incorporating PKWARE + Proprietary Technology into Your Product" for more + information. + + Bit 7: Currently unused. + + Bit 8: Currently unused. + + Bit 9: Currently unused. + + Bit 10: Currently unused. + + Bit 11: Language encoding flag (EFS). If this bit is set, + the filename and comment fields for this file + MUST be encoded using UTF-8. (see APPENDIX D) + + Bit 12: Reserved by PKWARE for enhanced compression. + + Bit 13: Set when encrypting the Central Directory to indicate + selected data values in the Local Header are masked to + hide their actual values. See the section describing + the Strong Encryption Specification for details. Refer + to the section in this document entitled "Incorporating + PKWARE Proprietary Technology into Your Product" for + more information. + + Bit 14: Reserved by PKWARE. + + Bit 15: Reserved by PKWARE. + + 4.4.5 compression method: (2 bytes) + + 0 - The file is stored (no compression) + 1 - The file is Shrunk + 2 - The file is Reduced with compression factor 1 + 3 - The file is Reduced with compression factor 2 + 4 - The file is Reduced with compression factor 3 + 5 - The file is Reduced with compression factor 4 + 6 - The file is Imploded + 7 - Reserved for Tokenizing compression algorithm + 8 - The file is Deflated + 9 - Enhanced Deflating using Deflate64(tm) + 10 - PKWARE Data Compression Library Imploding (old IBM TERSE) + 11 - Reserved by PKWARE + 12 - File is compressed using BZIP2 algorithm + 13 - Reserved by PKWARE + 14 - LZMA (EFS) + 15 - Reserved by PKWARE + 16 - Reserved by PKWARE + 17 - Reserved by PKWARE + 18 - File is compressed using IBM TERSE (new) + 19 - IBM LZ77 z Architecture (PFS) + 97 - WavPack compressed data + 98 - PPMd version I, Rev 1 + + + 4.4.6 date and time fields: (2 bytes each) + + The date and time are encoded in standard MS-DOS format. + If input came from standard input, the date and time are + those at which compression was started for this data. + If encrypting the central directory and general purpose bit + flag 13 is set indicating masking, the value stored in the + Local Header will be zero. + + 4.4.7 CRC-32: (4 bytes) + + The CRC-32 algorithm was generously contributed by + David Schwaderer and can be found in his excellent + book "C Programmers Guide to NetBIOS" published by + Howard W. Sams & Co. Inc. The 'magic number' for + the CRC is 0xdebb20e3. The proper CRC pre and post + conditioning is used, meaning that the CRC register + is pre-conditioned with all ones (a starting value + of 0xffffffff) and the value is post-conditioned by + taking the one's complement of the CRC residual. + If bit 3 of the general purpose flag is set, this + field is set to zero in the local header and the correct + value is put in the data descriptor and in the central + directory. When encrypting the central directory, if the + local header is not in ZIP64 format and general purpose + bit flag 13 is set indicating masking, the value stored + in the Local Header will be zero. + + 4.4.8 compressed size: (4 bytes) + 4.4.9 uncompressed size: (4 bytes) + + The size of the file compressed (4.4.8) and uncompressed, + (4.4.9) respectively. When a decryption header is present it + will be placed in front of the file data and the value of the + compressed file size will include the bytes of the decryption + header. If bit 3 of the general purpose bit flag is set, + these fields are set to zero in the local header and the + correct values are put in the data descriptor and + in the central directory. If an archive is in ZIP64 format + and the value in this field is 0xFFFFFFFF, the size will be + in the corresponding 8 byte ZIP64 extended information + extra field. When encrypting the central directory, if the + local header is not in ZIP64 format and general purpose bit + flag 13 is set indicating masking, the value stored for the + uncompressed size in the Local Header will be zero. + + 4.4.10 file name length: (2 bytes) + 4.4.11 extra field length: (2 bytes) + 4.4.12 file comment length: (2 bytes) + + The length of the file name, extra field, and comment + fields respectively. The combined length of any + directory record and these three fields should not + generally exceed 65,535 bytes. If input came from standard + input, the file name length is set to zero. + + + 4.4.13 disk number start: (2 bytes) + + The number of the disk on which this file begins. If an + archive is in ZIP64 format and the value in this field is + 0xFFFF, the size will be in the corresponding 4 byte zip64 + extended information extra field. + + 4.4.14 internal file attributes: (2 bytes) + + Bits 1 and 2 are reserved for use by PKWARE. + + 4.4.14.1 The lowest bit of this field indicates, if set, + that the file is apparently an ASCII or text file. If not + set, that the file apparently contains binary data. + The remaining bits are unused in version 1.0. + + 4.4.14.2 The 0x0002 bit of this field indicates, if set, that + a 4 byte variable record length control field precedes each + logical record indicating the length of the record. The + record length control field is stored in little-endian byte + order. This flag is independent of text control characters, + and if used in conjunction with text data, includes any + control characters in the total length of the record. This + value is provided for mainframe data transfer support. + + 4.4.15 external file attributes: (4 bytes) + + The mapping of the external attributes is + host-system dependent (see 'version made by'). For + MS-DOS, the low order byte is the MS-DOS directory + attribute byte. If input came from standard input, this + field is set to zero. + + 4.4.16 relative offset of local header: (4 bytes) + + This is the offset from the start of the first disk on + which this file appears, to where the local header should + be found. If an archive is in ZIP64 format and the value + in this field is 0xFFFFFFFF, the size will be in the + corresponding 8 byte zip64 extended information extra field. + + 4.4.17 file name: (Variable) + + 4.4.17.1 The name of the file, with optional relative path. + The path stored MUST not contain a drive or + device letter, or a leading slash. All slashes + MUST be forward slashes '/' as opposed to + backwards slashes '\' for compatibility with Amiga + and UNIX file systems etc. If input came from standard + input, there is no file name field. + + 4.4.17.2 If using the Central Directory Encryption Feature and + general purpose bit flag 13 is set indicating masking, the file + name stored in the Local Header will not be the actual file name. + A masking value consisting of a unique hexadecimal value will + be stored. This value will be sequentially incremented for each + file in the archive. See the section on the Strong Encryption + Specification for details on retrieving the encrypted file name. + Refer to the section in this document entitled "Incorporating PKWARE + Proprietary Technology into Your Product" for more information. + + + 4.4.18 file comment: (Variable) + + The comment for this file. + + 4.4.19 number of this disk: (2 bytes) + + The number of this disk, which contains central + directory end record. If an archive is in ZIP64 format + and the value in this field is 0xFFFF, the size will + be in the corresponding 4 byte zip64 end of central + directory field. + + + 4.4.20 number of the disk with the start of the central + directory: (2 bytes) + + The number of the disk on which the central + directory starts. If an archive is in ZIP64 format + and the value in this field is 0xFFFF, the size will + be in the corresponding 4 byte zip64 end of central + directory field. + + 4.4.21 total number of entries in the central dir on + this disk: (2 bytes) + + The number of central directory entries on this disk. + If an archive is in ZIP64 format and the value in + this field is 0xFFFF, the size will be in the + corresponding 8 byte zip64 end of central + directory field. + + 4.4.22 total number of entries in the central dir: (2 bytes) + + The total number of files in the .ZIP file. If an + archive is in ZIP64 format and the value in this field + is 0xFFFF, the size will be in the corresponding 8 byte + zip64 end of central directory field. + + 4.4.23 size of the central directory: (4 bytes) + + The size (in bytes) of the entire central directory. + If an archive is in ZIP64 format and the value in + this field is 0xFFFFFFFF, the size will be in the + corresponding 8 byte zip64 end of central + directory field. + + 4.4.24 offset of start of central directory with respect to + the starting disk number: (4 bytes) + + Offset of the start of the central directory on the + disk on which the central directory starts. If an + archive is in ZIP64 format and the value in this + field is 0xFFFFFFFF, the size will be in the + corresponding 8 byte zip64 end of central + directory field. + + 4.4.25 .ZIP file comment length: (2 bytes) + + The length of the comment for this .ZIP file. + + 4.4.26 .ZIP file comment: (Variable) + + The comment for this .ZIP file. ZIP file comment data + is stored unsecured. No encryption or data authentication + is applied to this area at this time. Confidential information + should not be stored in this section. + + 4.4.27 zip64 extensible data sector (variable size) + + (currently reserved for use by PKWARE) + + + 4.4.28 extra field: (Variable) + + This SHOULD be used for storage expansion. If additional + information needs to be stored within a ZIP file for special + application or platform needs, it SHOULD be stored here. + Programs supporting earlier versions of this specification can + then safely skip the file, and find the next file or header. + This field will be 0 length in version 1.0. + + Existing extra fields are defined in the section + Extensible data fields that follows. + +4.5 Extensible data fields +-------------------------- + + 4.5.1 In order to allow different programs and different types + of information to be stored in the 'extra' field in .ZIP + files, the following structure MUST be used for all + programs storing data in this field: + + header1+data1 + header2+data2 . . . + + Each header should consist of: + + Header ID - 2 bytes + Data Size - 2 bytes + + Note: all fields stored in Intel low-byte/high-byte order. + + The Header ID field indicates the type of data that is in + the following data block. + + Header IDs of 0 thru 31 are reserved for use by PKWARE. + The remaining IDs can be used by third party vendors for + proprietary usage. + + 4.5.2 The current Header ID mappings defined by PKWARE are: + + 0x0001 Zip64 extended information extra field + 0x0007 AV Info + 0x0008 Reserved for extended language encoding data (PFS) + (see APPENDIX D) + 0x0009 OS/2 + 0x000a NTFS + 0x000c OpenVMS + 0x000d UNIX + 0x000e Reserved for file stream and fork descriptors + 0x000f Patch Descriptor + 0x0014 PKCS#7 Store for X.509 Certificates + 0x0015 X.509 Certificate ID and Signature for + individual file + 0x0016 X.509 Certificate ID for Central Directory + 0x0017 Strong Encryption Header + 0x0018 Record Management Controls + 0x0019 PKCS#7 Encryption Recipient Certificate List + 0x0065 IBM S/390 (Z390), AS/400 (I400) attributes + - uncompressed + 0x0066 Reserved for IBM S/390 (Z390), AS/400 (I400) + attributes - compressed + 0x4690 POSZIP 4690 (reserved) + + + 4.5.3 -Zip64 Extended Information Extra Field (0x0001): + + The following is the layout of the zip64 extended + information "extra" block. If one of the size or + offset fields in the Local or Central directory + record is too small to hold the required data, + a Zip64 extended information record is created. + The order of the fields in the zip64 extended + information record is fixed, but the fields MUST + only appear if the corresponding Local or Central + directory record field is set to 0xFFFF or 0xFFFFFFFF. + + Note: all fields stored in Intel low-byte/high-byte order. + + Value Size Description + ----- ---- ----------- +(ZIP64) 0x0001 2 bytes Tag for this "extra" block type + Size 2 bytes Size of this "extra" block + Original + Size 8 bytes Original uncompressed file size + Compressed + Size 8 bytes Size of compressed data + Relative Header + Offset 8 bytes Offset of local header record + Disk Start + Number 4 bytes Number of the disk on which + this file starts + + This entry in the Local header MUST include BOTH original + and compressed file size fields. If encrypting the + central directory and bit 13 of the general purpose bit + flag is set indicating masking, the value stored in the + Local Header for the original file size will be zero. + + + 4.5.4 -OS/2 Extra Field (0x0009): + + The following is the layout of the OS/2 attributes "extra" + block. (Last Revision 09/05/95) + + Note: all fields stored in Intel low-byte/high-byte order. + + Value Size Description + ----- ---- ----------- +(OS/2) 0x0009 2 bytes Tag for this "extra" block type + TSize 2 bytes Size for the following data block + BSize 4 bytes Uncompressed Block Size + CType 2 bytes Compression type + EACRC 4 bytes CRC value for uncompress block + (var) variable Compressed block + + The OS/2 extended attribute structure (FEA2LIST) is + compressed and then stored in its entirety within this + structure. There will only ever be one "block" of data in + VarFields[]. + + 4.5.5 -NTFS Extra Field (0x000a): + + The following is the layout of the NTFS attributes + "extra" block. (Note: At this time the Mtime, Atime + and Ctime values MAY be used on any WIN32 system.) + + Note: all fields stored in Intel low-byte/high-byte order. + + Value Size Description + ----- ---- ----------- +(NTFS) 0x000a 2 bytes Tag for this "extra" block type + TSize 2 bytes Size of the total "extra" block + Reserved 4 bytes Reserved for future use + Tag1 2 bytes NTFS attribute tag value #1 + Size1 2 bytes Size of attribute #1, in bytes + (var) Size1 Attribute #1 data + . + . + . + TagN 2 bytes NTFS attribute tag value #N + SizeN 2 bytes Size of attribute #N, in bytes + (var) SizeN Attribute #N data + + For NTFS, values for Tag1 through TagN are as follows: + (currently only one set of attributes is defined for NTFS) + + Tag Size Description + ----- ---- ----------- + 0x0001 2 bytes Tag for attribute #1 + Size1 2 bytes Size of attribute #1, in bytes + Mtime 8 bytes File last modification time + Atime 8 bytes File last access time + Ctime 8 bytes File creation time + + 4.5.6 -OpenVMS Extra Field (0x000c): + + The following is the layout of the OpenVMS attributes + "extra" block. + + Note: all fields stored in Intel low-byte/high-byte order. + + Value Size Description + ----- ---- ----------- + (VMS) 0x000c 2 bytes Tag for this "extra" block type + TSize 2 bytes Size of the total "extra" block + CRC 4 bytes 32-bit CRC for remainder of the block + Tag1 2 bytes OpenVMS attribute tag value #1 + Size1 2 bytes Size of attribute #1, in bytes + (var) Size1 Attribute #1 data + . + . + . + TagN 2 bytes OpenVMS attribute tag value #N + SizeN 2 bytes Size of attribute #N, in bytes + (var) SizeN Attribute #N data + + OpenVMS Extra Field Rules: + + 4.5.6.1. There will be one or more attributes present, which + will each be preceded by the above TagX & SizeX values. + These values are identical to the ATR$C_XXXX and ATR$S_XXXX + constants which are defined in ATR.H under OpenVMS C. Neither + of these values will ever be zero. + + 4.5.6.2. No word alignment or padding is performed. + + 4.5.6.3. A well-behaved PKZIP/OpenVMS program should never produce + more than one sub-block with the same TagX value. Also, there will + never be more than one "extra" block of type 0x000c in a particular + directory record. + + 4.5.7 -UNIX Extra Field (0x000d): + + The following is the layout of the UNIX "extra" block. + Note: all fields are stored in Intel low-byte/high-byte + order. + + Value Size Description + ----- ---- ----------- +(UNIX) 0x000d 2 bytes Tag for this "extra" block type + TSize 2 bytes Size for the following data block + Atime 4 bytes File last access time + Mtime 4 bytes File last modification time + Uid 2 bytes File user ID + Gid 2 bytes File group ID + (var) variable Variable length data field + + The variable length data field will contain file type + specific data. Currently the only values allowed are + the original "linked to" file names for hard or symbolic + links, and the major and minor device node numbers for + character and block device nodes. Since device nodes + cannot be either symbolic or hard links, only one set of + variable length data is stored. Link files will have the + name of the original file stored. This name is NOT NULL + terminated. Its size can be determined by checking TSize - + 12. Device entries will have eight bytes stored as two 4 + byte entries (in little endian format). The first entry + will be the major device number, and the second the minor + device number. + + 4.5.8 -PATCH Descriptor Extra Field (0x000f): + + 4.5.8.1 The following is the layout of the Patch Descriptor + "extra" block. + + Note: all fields stored in Intel low-byte/high-byte order. + + Value Size Description + ----- ---- ----------- +(Patch) 0x000f 2 bytes Tag for this "extra" block type + TSize 2 bytes Size of the total "extra" block + Version 2 bytes Version of the descriptor + Flags 4 bytes Actions and reactions (see below) + OldSize 4 bytes Size of the file about to be patched + OldCRC 4 bytes 32-bit CRC of the file to be patched + NewSize 4 bytes Size of the resulting file + NewCRC 4 bytes 32-bit CRC of the resulting file + + 4.5.8.2 Actions and reactions + + Bits Description + ---- ---------------- + 0 Use for auto detection + 1 Treat as a self-patch + 2-3 RESERVED + 4-5 Action (see below) + 6-7 RESERVED + 8-9 Reaction (see below) to absent file + 10-11 Reaction (see below) to newer file + 12-13 Reaction (see below) to unknown file + 14-15 RESERVED + 16-31 RESERVED + + 4.5.8.2.1 Actions + + Action Value + ------ ----- + none 0 + add 1 + delete 2 + patch 3 + + 4.5.8.2.2 Reactions + + Reaction Value + -------- ----- + ask 0 + skip 1 + ignore 2 + fail 3 + + 4.5.8.3 Patch support is provided by PKPatchMaker(tm) technology + and is covered under U.S. Patents and Patents Pending. The use or + implementation in a product of certain technological aspects set + forth in the current APPNOTE, including those with regard to + strong encryption or patching requires a license from PKWARE. + Refer to the section in this document entitled "Incorporating + PKWARE Proprietary Technology into Your Product" for more + information. + + 4.5.9 -PKCS#7 Store for X.509 Certificates (0x0014): + + This field MUST contain information about each of the certificates + files may be signed with. When the Central Directory Encryption + feature is enabled for a ZIP file, this record will appear in + the Archive Extra Data Record, otherwise it will appear in the + first central directory record and will be ignored in any + other record. + + + Note: all fields stored in Intel low-byte/high-byte order. + + Value Size Description + ----- ---- ----------- +(Store) 0x0014 2 bytes Tag for this "extra" block type + TSize 2 bytes Size of the store data + TData TSize Data about the store + + + 4.5.10 -X.509 Certificate ID and Signature for individual file (0x0015): + + This field contains the information about which certificate in + the PKCS#7 store was used to sign a particular file. It also + contains the signature data. This field can appear multiple + times, but can only appear once per certificate. + + Note: all fields stored in Intel low-byte/high-byte order. + + Value Size Description + ----- ---- ----------- +(CID) 0x0015 2 bytes Tag for this "extra" block type + TSize 2 bytes Size of data that follows + TData TSize Signature Data + + 4.5.11 -X.509 Certificate ID and Signature for central directory (0x0016): + + This field contains the information about which certificate in + the PKCS#7 store was used to sign the central directory structure. + When the Central Directory Encryption feature is enabled for a + ZIP file, this record will appear in the Archive Extra Data Record, + otherwise it will appear in the first central directory record. + + Note: all fields stored in Intel low-byte/high-byte order. + + Value Size Description + ----- ---- ----------- +(CDID) 0x0016 2 bytes Tag for this "extra" block type + TSize 2 bytes Size of data that follows + TData TSize Data + + 4.5.12 -Strong Encryption Header (0x0017): + + Value Size Description + ----- ---- ----------- + 0x0017 2 bytes Tag for this "extra" block type + TSize 2 bytes Size of data that follows + Format 2 bytes Format definition for this record + AlgID 2 bytes Encryption algorithm identifier + Bitlen 2 bytes Bit length of encryption key + Flags 2 bytes Processing flags + CertData TSize-8 Certificate decryption extra field data + (refer to the explanation for CertData + in the section describing the + Certificate Processing Method under + the Strong Encryption Specification) + + See the section describing the Strong Encryption Specification + for details. Refer to the section in this document entitled + "Incorporating PKWARE Proprietary Technology into Your Product" + for more information. + + 4.5.13 -Record Management Controls (0x0018): + + Value Size Description + ----- ---- ----------- +(Rec-CTL) 0x0018 2 bytes Tag for this "extra" block type + CSize 2 bytes Size of total extra block data + Tag1 2 bytes Record control attribute 1 + Size1 2 bytes Size of attribute 1, in bytes + Data1 Size1 Attribute 1 data + . + . + . + TagN 2 bytes Record control attribute N + SizeN 2 bytes Size of attribute N, in bytes + DataN SizeN Attribute N data + + + 4.5.14 -PKCS#7 Encryption Recipient Certificate List (0x0019): + + This field MAY contain information about each of the certificates + used in encryption processing and it can be used to identify who is + allowed to decrypt encrypted files. This field should only appear + in the archive extra data record. This field is not required and + serves only to aid archive modifications by preserving public + encryption key data. Individual security requirements may dictate + that this data be omitted to deter information exposure. + + Note: all fields stored in Intel low-byte/high-byte order. + + Value Size Description + ----- ---- ----------- +(CStore) 0x0019 2 bytes Tag for this "extra" block type + TSize 2 bytes Size of the store data + TData TSize Data about the store + + TData: + + Value Size Description + ----- ---- ----------- + Version 2 bytes Format version number - must 0x0001 at this time + CStore (var) PKCS#7 data blob + + See the section describing the Strong Encryption Specification + for details. Refer to the section in this document entitled + "Incorporating PKWARE Proprietary Technology into Your Product" + for more information. + + 4.5.15 -MVS Extra Field (0x0065): + + The following is the layout of the MVS "extra" block. + Note: Some fields are stored in Big Endian format. + All text is in EBCDIC format unless otherwise specified. + + Value Size Description + ----- ---- ----------- +(MVS) 0x0065 2 bytes Tag for this "extra" block type + TSize 2 bytes Size for the following data block + ID 4 bytes EBCDIC "Z390" 0xE9F3F9F0 or + "T4MV" for TargetFour + (var) TSize-4 Attribute data (see APPENDIX B) + + + 4.5.16 -OS/400 Extra Field (0x0065): + + The following is the layout of the OS/400 "extra" block. + Note: Some fields are stored in Big Endian format. + All text is in EBCDIC format unless otherwise specified. + + Value Size Description + ----- ---- ----------- +(OS400) 0x0065 2 bytes Tag for this "extra" block type + TSize 2 bytes Size for the following data block + ID 4 bytes EBCDIC "I400" 0xC9F4F0F0 or + "T4MV" for TargetFour + (var) TSize-4 Attribute data (see APPENDIX A) + +4.6 Third Party Mappings +------------------------ + + 4.6.1 Third party mappings commonly used are: + + 0x07c8 Macintosh + 0x2605 ZipIt Macintosh + 0x2705 ZipIt Macintosh 1.3.5+ + 0x2805 ZipIt Macintosh 1.3.5+ + 0x334d Info-ZIP Macintosh + 0x4341 Acorn/SparkFS + 0x4453 Windows NT security descriptor (binary ACL) + 0x4704 VM/CMS + 0x470f MVS + 0x4b46 FWKCS MD5 (see below) + 0x4c41 OS/2 access control list (text ACL) + 0x4d49 Info-ZIP OpenVMS + 0x4f4c Xceed original location extra field + 0x5356 AOS/VS (ACL) + 0x5455 extended timestamp + 0x554e Xceed unicode extra field + 0x5855 Info-ZIP UNIX (original, also OS/2, NT, etc) + 0x6375 Info-ZIP Unicode Comment Extra Field + 0x6542 BeOS/BeBox + 0x7075 Info-ZIP Unicode Path Extra Field + 0x756e ASi UNIX + 0x7855 Info-ZIP UNIX (new) + 0xa220 Microsoft Open Packaging Growth Hint + 0xfd4a SMS/QDOS + + Detailed descriptions of Extra Fields defined by third + party mappings will be documented as information on + these data structures is made available to PKWARE. + PKWARE does not guarantee the accuracy of any published + third party data. + + 4.6.2 Third-party Extra Fields must include a Header ID using + the format defined in the section of this document + titled Extensible Data Fields (section 4.5). + + The Data Size field indicates the size of the following + data block. Programs can use this value to skip to the + next header block, passing over any data blocks that are + not of interest. + + Note: As stated above, the size of the entire .ZIP file + header, including the file name, comment, and extra + field should not exceed 64K in size. + + 4.6.3 In case two different programs should appropriate the same + Header ID value, it is strongly recommended that each + program SHOULD place a unique signature of at least two bytes in + size (and preferably 4 bytes or bigger) at the start of + each data area. Every program SHOULD verify that its + unique signature is present, in addition to the Header ID + value being correct, before assuming that it is a block of + known type. + + Third-party Mappings: + + 4.6.4 -ZipIt Macintosh Extra Field (long) (0x2605): + + The following is the layout of the ZipIt extra block + for Macintosh. The local-header and central-header versions + are identical. This block must be present if the file is + stored MacBinary-encoded and it should not be used if the file + is not stored MacBinary-encoded. + + Value Size Description + ----- ---- ----------- + (Mac2) 0x2605 Short tag for this extra block type + TSize Short total data size for this block + "ZPIT" beLong extra-field signature + FnLen Byte length of FileName + FileName variable full Macintosh filename + FileType Byte[4] four-byte Mac file type string + Creator Byte[4] four-byte Mac creator string + + + 4.6.5 -ZipIt Macintosh Extra Field (short, for files) (0x2705): + + The following is the layout of a shortened variant of the + ZipIt extra block for Macintosh (without "full name" entry). + This variant is used by ZipIt 1.3.5 and newer for entries of + files (not directories) that do not have a MacBinary encoded + file. The local-header and central-header versions are identical. + + Value Size Description + ----- ---- ----------- + (Mac2b) 0x2705 Short tag for this extra block type + TSize Short total data size for this block (12) + "ZPIT" beLong extra-field signature + FileType Byte[4] four-byte Mac file type string + Creator Byte[4] four-byte Mac creator string + fdFlags beShort attributes from FInfo.frFlags, + may be omitted + 0x0000 beShort reserved, may be omitted + + + 4.6.6 -ZipIt Macintosh Extra Field (short, for directories) (0x2805): + + The following is the layout of a shortened variant of the + ZipIt extra block for Macintosh used only for directory + entries. This variant is used by ZipIt 1.3.5 and newer to + save some optional Mac-specific information about directories. + The local-header and central-header versions are identical. + + Value Size Description + ----- ---- ----------- + (Mac2c) 0x2805 Short tag for this extra block type + TSize Short total data size for this block (12) + "ZPIT" beLong extra-field signature + frFlags beShort attributes from DInfo.frFlags, may + be omitted + View beShort ZipIt view flag, may be omitted + + + The View field specifies ZipIt-internal settings as follows: + + Bits of the Flags: + bit 0 if set, the folder is shown expanded (open) + when the archive contents are viewed in ZipIt. + bits 1-15 reserved, zero; + + + 4.6.7 -FWKCS MD5 Extra Field (0x4b46): + + The FWKCS Contents_Signature System, used in + automatically identifying files independent of file name, + optionally adds and uses an extra field to support the + rapid creation of an enhanced contents_signature: + + Header ID = 0x4b46 + Data Size = 0x0013 + Preface = 'M','D','5' + followed by 16 bytes containing the uncompressed file's + 128_bit MD5 hash(1), low byte first. + + When FWKCS revises a .ZIP file central directory to add + this extra field for a file, it also replaces the + central directory entry for that file's uncompressed + file length with a measured value. + + FWKCS provides an option to strip this extra field, if + present, from a .ZIP file central directory. In adding + this extra field, FWKCS preserves .ZIP file Authenticity + Verification; if stripping this extra field, FWKCS + preserves all versions of AV through PKZIP version 2.04g. + + FWKCS, and FWKCS Contents_Signature System, are + trademarks of Frederick W. Kantor. + + (1) R. Rivest, RFC1321.TXT, MIT Laboratory for Computer + Science and RSA Data Security, Inc., April 1992. + ll.76-77: "The MD5 algorithm is being placed in the + public domain for review and possible adoption as a + standard." + + + 4.6.8 -Info-ZIP Unicode Comment Extra Field (0x6375): + + Stores the UTF-8 version of the file comment as stored in the + central directory header. (Last Revision 20070912) + + Value Size Description + ----- ---- ----------- + (UCom) 0x6375 Short tag for this extra block type ("uc") + TSize Short total data size for this block + Version 1 byte version of this extra field, currently 1 + ComCRC32 4 bytes Comment Field CRC32 Checksum + UnicodeCom Variable UTF-8 version of the entry comment + + Currently Version is set to the number 1. If there is a need + to change this field, the version will be incremented. Changes + may not be backward compatible so this extra field should not be + used if the version is not recognized. + + The ComCRC32 is the standard zip CRC32 checksum of the File Comment + field in the central directory header. This is used to verify that + the comment field has not changed since the Unicode Comment extra field + was created. This can happen if a utility changes the File Comment + field but does not update the UTF-8 Comment extra field. If the CRC + check fails, this Unicode Comment extra field should be ignored and + the File Comment field in the header should be used instead. + + The UnicodeCom field is the UTF-8 version of the File Comment field + in the header. As UnicodeCom is defined to be UTF-8, no UTF-8 byte + order mark (BOM) is used. The length of this field is determined by + subtracting the size of the previous fields from TSize. If both the + File Name and Comment fields are UTF-8, the new General Purpose Bit + Flag, bit 11 (Language encoding flag (EFS)), can be used to indicate + both the header File Name and Comment fields are UTF-8 and, in this + case, the Unicode Path and Unicode Comment extra fields are not + needed and should not be created. Note that, for backward + compatibility, bit 11 should only be used if the native character set + of the paths and comments being zipped up are already in UTF-8. It is + expected that the same file comment storage method, either general + purpose bit 11 or extra fields, be used in both the Local and Central + Directory Header for a file. + + + 4.6.9 -Info-ZIP Unicode Path Extra Field (0x7075): + + Stores the UTF-8 version of the file name field as stored in the + local header and central directory header. (Last Revision 20070912) + + Value Size Description + ----- ---- ----------- + (UPath) 0x7075 Short tag for this extra block type ("up") + TSize Short total data size for this block + Version 1 byte version of this extra field, currently 1 + NameCRC32 4 bytes File Name Field CRC32 Checksum + UnicodeName Variable UTF-8 version of the entry File Name + + Currently Version is set to the number 1. If there is a need + to change this field, the version will be incremented. Changes + may not be backward compatible so this extra field should not be + used if the version is not recognized. + + The NameCRC32 is the standard zip CRC32 checksum of the File Name + field in the header. This is used to verify that the header + File Name field has not changed since the Unicode Path extra field + was created. This can happen if a utility renames the File Name but + does not update the UTF-8 path extra field. If the CRC check fails, + this UTF-8 Path Extra Field should be ignored and the File Name field + in the header should be used instead. + + The UnicodeName is the UTF-8 version of the contents of the File Name + field in the header. As UnicodeName is defined to be UTF-8, no UTF-8 + byte order mark (BOM) is used. The length of this field is determined + by subtracting the size of the previous fields from TSize. If both + the File Name and Comment fields are UTF-8, the new General Purpose + Bit Flag, bit 11 (Language encoding flag (EFS)), can be used to + indicate that both the header File Name and Comment fields are UTF-8 + and, in this case, the Unicode Path and Unicode Comment extra fields + are not needed and should not be created. Note that, for backward + compatibility, bit 11 should only be used if the native character set + of the paths and comments being zipped up are already in UTF-8. It is + expected that the same file name storage method, either general + purpose bit 11 or extra fields, be used in both the Local and Central + Directory Header for a file. + + + 4.6.10 -Microsoft Open Packaging Growth Hint (0xa220): + + Value Size Description + ----- ---- ----------- + 0xa220 Short tag for this extra block type + TSize Short size of Sig + PadVal + Padding + Sig Short verification signature (A028) + PadVal Short Initial padding value + Padding variable filled with NULL characters + +4.7 Manifest Files +------------------ + + 4.7.1 Applications using ZIP files may have a need for additional + information that must be included with the files placed into + a ZIP file. Application specific information that cannot be + stored using the defined ZIP storage records SHOULD be stored + using the extensible Extra Field convention defined in this + document. However, some applications may use a manifest + file as a means for storing additional information. One + example is the META-INF/MANIFEST.MF file used in ZIP formatted + files having the .JAR extension (JAR files). + + 4.7.2 A manifest file is a file created for the application process + that requires this information. A manifest file MAY be of any + file type required by the defining application process. It is + placed within the same ZIP file as files to which this information + applies. By convention, this file is typically the first file placed + into the ZIP file and it may include a defined directory path. + + 4.7.3 Manifest files may be compressed or encrypted as needed for + application processing of the files inside the ZIP files. + + Manifest files are outside of the scope of this specification. + + +5.0 Explanation of compression methods +-------------------------------------- + + +5.1 UnShrinking - Method 1 +-------------------------- + + 5.1.1 Shrinking is a Dynamic Ziv-Lempel-Welch compression algorithm + with partial clearing. The initial code size is 9 bits, and the + maximum code size is 13 bits. Shrinking differs from conventional + Dynamic Ziv-Lempel-Welch implementations in several respects: + + 5.1.2 The code size is controlled by the compressor, and is + not automatically increased when codes larger than the current + code size are created (but not necessarily used). When + the decompressor encounters the code sequence 256 + (decimal) followed by 1, it should increase the code size + read from the input stream to the next bit size. No + blocking of the codes is performed, so the next code at + the increased size should be read from the input stream + immediately after where the previous code at the smaller + bit size was read. Again, the decompressor should not + increase the code size used until the sequence 256,1 is + encountered. + + 5.1.3 When the table becomes full, total clearing is not + performed. Rather, when the compressor emits the code + sequence 256,2 (decimal), the decompressor should clear + all leaf nodes from the Ziv-Lempel tree, and continue to + use the current code size. The nodes that are cleared + from the Ziv-Lempel tree are then re-used, with the lowest + code value re-used first, and the highest code value + re-used last. The compressor can emit the sequence 256,2 + at any time. + +5.2 Expanding - Methods 2-5 +--------------------------- + + 5.2.1 The Reducing algorithm is actually a combination of two + distinct algorithms. The first algorithm compresses repeated + byte sequences, and the second algorithm takes the compressed + stream from the first algorithm and applies a probabilistic + compression method. + + 5.2.2 The probabilistic compression stores an array of 'follower + sets' S(j), for j=0 to 255, corresponding to each possible + ASCII character. Each set contains between 0 and 32 + characters, to be denoted as S(j)[0],...,S(j)[m], where m<32. + The sets are stored at the beginning of the data area for a + Reduced file, in reverse order, with S(255) first, and S(0) + last. + + 5.2.3 The sets are encoded as { N(j), S(j)[0],...,S(j)[N(j)-1] }, + where N(j) is the size of set S(j). N(j) can be 0, in which + case the follower set for S(j) is empty. Each N(j) value is + encoded in 6 bits, followed by N(j) eight bit character values + corresponding to S(j)[0] to S(j)[N(j)-1] respectively. If + N(j) is 0, then no values for S(j) are stored, and the value + for N(j-1) immediately follows. + + 5.2.4 Immediately after the follower sets, is the compressed data + stream. The compressed data stream can be interpreted for the + probabilistic decompression as follows: + + let Last-Character <- 0. + loop until done + if the follower set S(Last-Character) is empty then + read 8 bits from the input stream, and copy this + value to the output stream. + otherwise if the follower set S(Last-Character) is non-empty then + read 1 bit from the input stream. + if this bit is not zero then + read 8 bits from the input stream, and copy this + value to the output stream. + otherwise if this bit is zero then + read B(N(Last-Character)) bits from the input + stream, and assign this value to I. + Copy the value of S(Last-Character)[I] to the + output stream. + + assign the last value placed on the output stream to + Last-Character. + end loop + + B(N(j)) is defined as the minimal number of bits required to + encode the value N(j)-1. + + 5.2.5 The decompressed stream from above can then be expanded to + re-create the original file as follows: + + let State <- 0. + + loop until done + read 8 bits from the input stream into C. + case State of + 0: if C is not equal to DLE (144 decimal) then + copy C to the output stream. + otherwise if C is equal to DLE then + let State <- 1. + + 1: if C is non-zero then + let V <- C. + let Len <- L(V) + let State <- F(Len). + otherwise if C is zero then + copy the value 144 (decimal) to the output stream. + let State <- 0 + + 2: let Len <- Len + C + let State <- 3. + + 3: move backwards D(V,C) bytes in the output stream + (if this position is before the start of the output + stream, then assume that all the data before the + start of the output stream is filled with zeros). + copy Len+3 bytes from this position to the output stream. + let State <- 0. + end case + end loop + + The functions F,L, and D are dependent on the 'compression + factor', 1 through 4, and are defined as follows: + + For compression factor 1: + L(X) equals the lower 7 bits of X. + F(X) equals 2 if X equals 127 otherwise F(X) equals 3. + D(X,Y) equals the (upper 1 bit of X) * 256 + Y + 1. + For compression factor 2: + L(X) equals the lower 6 bits of X. + F(X) equals 2 if X equals 63 otherwise F(X) equals 3. + D(X,Y) equals the (upper 2 bits of X) * 256 + Y + 1. + For compression factor 3: + L(X) equals the lower 5 bits of X. + F(X) equals 2 if X equals 31 otherwise F(X) equals 3. + D(X,Y) equals the (upper 3 bits of X) * 256 + Y + 1. + For compression factor 4: + L(X) equals the lower 4 bits of X. + F(X) equals 2 if X equals 15 otherwise F(X) equals 3. + D(X,Y) equals the (upper 4 bits of X) * 256 + Y + 1. + +5.3 Imploding - Method 6 +------------------------ + + 5.3.1 The Imploding algorithm is actually a combination of two + distinct algorithms. The first algorithm compresses repeated byte + sequences using a sliding dictionary. The second algorithm is + used to compress the encoding of the sliding dictionary output, + using multiple Shannon-Fano trees. + + 5.3.2 The Imploding algorithm can use a 4K or 8K sliding dictionary + size. The dictionary size used can be determined by bit 1 in the + general purpose flag word; a 0 bit indicates a 4K dictionary + while a 1 bit indicates an 8K dictionary. + + 5.3.3 The Shannon-Fano trees are stored at the start of the + compressed file. The number of trees stored is defined by bit 2 in + the general purpose flag word; a 0 bit indicates two trees stored, + a 1 bit indicates three trees are stored. If 3 trees are stored, + the first Shannon-Fano tree represents the encoding of the + Literal characters, the second tree represents the encoding of + the Length information, the third represents the encoding of the + Distance information. When 2 Shannon-Fano trees are stored, the + Length tree is stored first, followed by the Distance tree. + + 5.3.4 The Literal Shannon-Fano tree, if present is used to represent + the entire ASCII character set, and contains 256 values. This + tree is used to compress any data not compressed by the sliding + dictionary algorithm. When this tree is present, the Minimum + Match Length for the sliding dictionary is 3. If this tree is + not present, the Minimum Match Length is 2. + + 5.3.5 The Length Shannon-Fano tree is used to compress the Length + part of the (length,distance) pairs from the sliding dictionary + output. The Length tree contains 64 values, ranging from the + Minimum Match Length, to 63 plus the Minimum Match Length. + + 5.3.6 The Distance Shannon-Fano tree is used to compress the Distance + part of the (length,distance) pairs from the sliding dictionary + output. The Distance tree contains 64 values, ranging from 0 to + 63, representing the upper 6 bits of the distance value. The + distance values themselves will be between 0 and the sliding + dictionary size, either 4K or 8K. + + 5.3.7 The Shannon-Fano trees themselves are stored in a compressed + format. The first byte of the tree data represents the number of + bytes of data representing the (compressed) Shannon-Fano tree + minus 1. The remaining bytes represent the Shannon-Fano tree + data encoded as: + + High 4 bits: Number of values at this bit length + 1. (1 - 16) + Low 4 bits: Bit Length needed to represent value + 1. (1 - 16) + + 5.3.8 The Shannon-Fano codes can be constructed from the bit lengths + using the following algorithm: + + 1) Sort the Bit Lengths in ascending order, while retaining the + order of the original lengths stored in the file. + + 2) Generate the Shannon-Fano trees: + + Code <- 0 + CodeIncrement <- 0 + LastBitLength <- 0 + i <- number of Shannon-Fano codes - 1 (either 255 or 63) + + loop while i >= 0 + Code = Code + CodeIncrement + if BitLength(i) <> LastBitLength then + LastBitLength=BitLength(i) + CodeIncrement = 1 shifted left (16 - LastBitLength) + ShannonCode(i) = Code + i <- i - 1 + end loop + + 3) Reverse the order of all the bits in the above ShannonCode() + vector, so that the most significant bit becomes the least + significant bit. For example, the value 0x1234 (hex) would + become 0x2C48 (hex). + + 4) Restore the order of Shannon-Fano codes as originally stored + within the file. + + Example: + + This example will show the encoding of a Shannon-Fano tree + of size 8. Notice that the actual Shannon-Fano trees used + for Imploding are either 64 or 256 entries in size. + + Example: 0x02, 0x42, 0x01, 0x13 + + The first byte indicates 3 values in this table. Decoding the + bytes: + 0x42 = 5 codes of 3 bits long + 0x01 = 1 code of 2 bits long + 0x13 = 2 codes of 4 bits long + + This would generate the original bit length array of: + (3, 3, 3, 3, 3, 2, 4, 4) + + There are 8 codes in this table for the values 0 thru 7. Using + the algorithm to obtain the Shannon-Fano codes produces: + + Reversed Order Original + Val Sorted Constructed Code Value Restored Length + --- ------ ----------------- -------- -------- ------ + 0: 2 1100000000000000 11 101 3 + 1: 3 1010000000000000 101 001 3 + 2: 3 1000000000000000 001 110 3 + 3: 3 0110000000000000 110 010 3 + 4: 3 0100000000000000 010 100 3 + 5: 3 0010000000000000 100 11 2 + 6: 4 0001000000000000 1000 1000 4 + 7: 4 0000000000000000 0000 0000 4 + + The values in the Val, Order Restored and Original Length columns + now represent the Shannon-Fano encoding tree that can be used for + decoding the Shannon-Fano encoded data. How to parse the + variable length Shannon-Fano values from the data stream is beyond + the scope of this document. (See the references listed at the end of + this document for more information.) However, traditional decoding + schemes used for Huffman variable length decoding, such as the + Greenlaw algorithm, can be successfully applied. + + 5.3.9 The compressed data stream begins immediately after the + compressed Shannon-Fano data. The compressed data stream can be + interpreted as follows: + + loop until done + read 1 bit from input stream. + + if this bit is non-zero then (encoded data is literal data) + if Literal Shannon-Fano tree is present + read and decode character using Literal Shannon-Fano tree. + otherwise + read 8 bits from input stream. + copy character to the output stream. + otherwise (encoded data is sliding dictionary match) + if 8K dictionary size + read 7 bits for offset Distance (lower 7 bits of offset). + otherwise + read 6 bits for offset Distance (lower 6 bits of offset). + + using the Distance Shannon-Fano tree, read and decode the + upper 6 bits of the Distance value. + + using the Length Shannon-Fano tree, read and decode + the Length value. + + Length <- Length + Minimum Match Length + + if Length = 63 + Minimum Match Length + read 8 bits from the input stream, + add this value to Length. + + move backwards Distance+1 bytes in the output stream, and + copy Length characters from this position to the output + stream. (if this position is before the start of the output + stream, then assume that all the data before the start of + the output stream is filled with zeros). + end loop + +5.4 Tokenizing - Method 7 +------------------------- + + 5.4.1 This method is not used by PKZIP. + +5.5 Deflating - Method 8 +------------------------ + + 5.5.1 The Deflate algorithm is similar to the Implode algorithm using + a sliding dictionary of up to 32K with secondary compression + from Huffman/Shannon-Fano codes. + + 5.5.2 The compressed data is stored in blocks with a header describing + the block and the Huffman codes used in the data block. The header + format is as follows: + + Bit 0: Last Block bit This bit is set to 1 if this is the last + compressed block in the data. + Bits 1-2: Block type + 00 (0) - Block is stored - All stored data is byte aligned. + Skip bits until next byte, then next word = block + length, followed by the ones compliment of the block + length word. Remaining data in block is the stored + data. + + 01 (1) - Use fixed Huffman codes for literal and distance codes. + Lit Code Bits Dist Code Bits + --------- ---- --------- ---- + 0 - 143 8 0 - 31 5 + 144 - 255 9 + 256 - 279 7 + 280 - 287 8 + + Literal codes 286-287 and distance codes 30-31 are + never used but participate in the huffman construction. + + 10 (2) - Dynamic Huffman codes. (See expanding Huffman codes) + + 11 (3) - Reserved - Flag a "Error in compressed data" if seen. + + 5.5.3 Expanding Huffman Codes + + If the data block is stored with dynamic Huffman codes, the Huffman + codes are sent in the following compressed format: + + 5 Bits: # of Literal codes sent - 256 (256 - 286) + All other codes are never sent. + 5 Bits: # of Dist codes - 1 (1 - 32) + 4 Bits: # of Bit Length codes - 3 (3 - 19) + + The Huffman codes are sent as bit lengths and the codes are built as + described in the implode algorithm. The bit lengths themselves are + compressed with Huffman codes. There are 19 bit length codes: + + 0 - 15: Represent bit lengths of 0 - 15 + 16: Copy the previous bit length 3 - 6 times. + The next 2 bits indicate repeat length (0 = 3, ... ,3 = 6) + Example: Codes 8, 16 (+2 bits 11), 16 (+2 bits 10) will + expand to 12 bit lengths of 8 (1 + 6 + 5) + 17: Repeat a bit length of 0 for 3 - 10 times. (3 bits of length) + 18: Repeat a bit length of 0 for 11 - 138 times (7 bits of length) + + The lengths of the bit length codes are sent packed 3 bits per value + (0 - 7) in the following order: + + 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15 + + The Huffman codes should be built as described in the Implode algorithm + except codes are assigned starting at the shortest bit length, i.e. the + shortest code should be all 0's rather than all 1's. Also, codes with + a bit length of zero do not participate in the tree construction. The + codes are then used to decode the bit lengths for the literal and + distance tables. + + The bit lengths for the literal tables are sent first with the number + of entries sent described by the 5 bits sent earlier. There are up + to 286 literal characters; the first 256 represent the respective 8 + bit character, code 256 represents the End-Of-Block code, the remaining + 29 codes represent copy lengths of 3 thru 258. There are up to 30 + distance codes representing distances from 1 thru 32k as described + below. + + Length Codes + ------------ + Extra Extra Extra Extra + Code Bits Length Code Bits Lengths Code Bits Lengths Code Bits Length(s) + ---- ---- ------ ---- ---- ------- ---- ---- ------- ---- ---- --------- + 257 0 3 265 1 11,12 273 3 35-42 281 5 131-162 + 258 0 4 266 1 13,14 274 3 43-50 282 5 163-194 + 259 0 5 267 1 15,16 275 3 51-58 283 5 195-226 + 260 0 6 268 1 17,18 276 3 59-66 284 5 227-257 + 261 0 7 269 2 19-22 277 4 67-82 285 0 258 + 262 0 8 270 2 23-26 278 4 83-98 + 263 0 9 271 2 27-30 279 4 99-114 + 264 0 10 272 2 31-34 280 4 115-130 + + Distance Codes + -------------- + Extra Extra Extra Extra + Code Bits Dist Code Bits Dist Code Bits Distance Code Bits Distance + ---- ---- ---- ---- ---- ------ ---- ---- -------- ---- ---- -------- + 0 0 1 8 3 17-24 16 7 257-384 24 11 4097-6144 + 1 0 2 9 3 25-32 17 7 385-512 25 11 6145-8192 + 2 0 3 10 4 33-48 18 8 513-768 26 12 8193-12288 + 3 0 4 11 4 49-64 19 8 769-1024 27 12 12289-16384 + 4 1 5,6 12 5 65-96 20 9 1025-1536 28 13 16385-24576 + 5 1 7,8 13 5 97-128 21 9 1537-2048 29 13 24577-32768 + 6 2 9-12 14 6 129-192 22 10 2049-3072 + 7 2 13-16 15 6 193-256 23 10 3073-4096 + + 5.5.4 The compressed data stream begins immediately after the + compressed header data. The compressed data stream can be + interpreted as follows: + + do + read header from input stream. + + if stored block + skip bits until byte aligned + read count and 1's compliment of count + copy count bytes data block + otherwise + loop until end of block code sent + decode literal character from input stream + if literal < 256 + copy character to the output stream + otherwise + if literal = end of block + break from loop + otherwise + decode distance from input stream + + move backwards distance bytes in the output stream, and + copy length characters from this position to the output + stream. + end loop + while not last block + + if data descriptor exists + skip bits until byte aligned + read crc and sizes + endif + +5.6 Enhanced Deflating - Method 9 +--------------------------------- + + 5.6.1 The Enhanced Deflating algorithm is similar to Deflate but uses + a sliding dictionary of up to 64K. Deflate64(tm) is supported + by the Deflate extractor. + +5.7 BZIP2 - Method 12 +--------------------- + + 5.7.1 BZIP2 is an open-source data compression algorithm developed by + Julian Seward. Information and source code for this algorithm + can be found on the internet. + +5.8 LZMA - Method 14 +--------------------- + + 5.8.1 LZMA is a block-oriented, general purpose data compression + algorithm developed and maintained by Igor Pavlov. It is a derivative + of LZ77 that utilizes Markov chains and a range coder. Information and + source code for this algorithm can be found on the internet. Consult + with the author of this algorithm for information on terms or + restrictions on use. + + Support for LZMA within the ZIP format is defined as follows: + + 5.8.2 The Compression method field within the ZIP Local and Central + Header records will be set to the value 14 to indicate data was + compressed using LZMA. + + 5.8.3 The Version needed to extract field within the ZIP Local and + Central Header records will be set to 6.3 to indicate the minimum + ZIP format version supporting this feature. + + 5.8.4 File data compressed using the LZMA algorithm must be placed + immediately following the Local Header for the file. If a standard + ZIP encryption header is required, it will follow the Local Header + and will precede the LZMA compressed file data segment. The location + of LZMA compressed data segment within the ZIP format will be as shown: + + [local header file 1] + [encryption header file 1] + [LZMA compressed data segment for file 1] + [data descriptor 1] + [local header file 2] + + 5.8.5 The encryption header and data descriptor records may + be conditionally present. The LZMA Compressed Data Segment + will consist of an LZMA Properties Header followed by the + LZMA Compressed Data as shown: + + [LZMA properties header for file 1] + [LZMA compressed data for file 1] + + 5.8.6 The LZMA Compressed Data will be stored as provided by the + LZMA compression library. Compressed size, uncompressed size and + other file characteristics about the file being compressed must be + stored in standard ZIP storage format. + + 5.8.7 The LZMA Properties Header will store specific data required + to decompress the LZMA compressed Data. This data is set by the + LZMA compression engine using the function WriteCoderProperties() + as documented within the LZMA SDK. + + 5.8.8 Storage fields for the property information within the LZMA + Properties Header are as follows: + + LZMA Version Information 2 bytes + LZMA Properties Size 2 bytes + LZMA Properties Data variable, defined by "LZMA Properties Size" + + 5.8.8.1 LZMA Version Information - this field identifies which version + of the LZMA SDK was used to compress a file. The first byte will + store the major version number of the LZMA SDK and the second + byte will store the minor number. + + 5.8.8.2 LZMA Properties Size - this field defines the size of the + remaining property data. Typically this size should be determined by + the version of the SDK. This size field is included as a convenience + and to help avoid any ambiguity should it arise in the future due + to changes in this compression algorithm. + + 5.8.8.3 LZMA Property Data - this variable sized field records the + required values for the decompressor as defined by the LZMA SDK. + The data stored in this field should be obtained using the + WriteCoderProperties() in the version of the SDK defined by + the "LZMA Version Information" field. + + 5.8.8.4 The layout of the "LZMA Properties Data" field is a function of + the LZMA compression algorithm. It is possible that this layout may be + changed by the author over time. The data layout in version 4.3 of the + LZMA SDK defines a 5 byte array that uses 4 bytes to store the dictionary + size in little-endian order. This is preceded by a single packed byte as + the first element of the array that contains the following fields: + + PosStateBits + LiteralPosStateBits + LiteralContextBits + + Refer to the LZMA documentation for a more detailed explanation of + these fields. + + 5.8.9 Data compressed with method 14, LZMA, may include an end-of-stream + (EOS) marker ending the compressed data stream. This marker is not + required, but its use is highly recommended to facilitate processing + and implementers should include the EOS marker whenever possible. + When the EOS marker is used, general purpose bit 1 must be set. If + general purpose bit 1 is not set, the EOS marker is not present. + +5.9 WavPack - Method 97 +----------------------- + + 5.9.1 Information describing the use of compression method 97 is + provided by WinZIP International, LLC. This method relies on the + open source WavPack audio compression utility developed by David Bryant. + Information on WavPack is available at www.wavpack.com. Please consult + with the author of this algorithm for information on terms and + restrictions on use. + + 5.9.2 WavPack data for a file begins immediately after the end of the + local header data. This data is the output from WavPack compression + routines. Within the ZIP file, the use of WavPack compression is + indicated by setting the compression method field to a value of 97 + in both the local header and the central directory header. The Version + needed to extract and version made by fields use the same values as are + used for data compressed using the Deflate algorithm. + + 5.9.3 An implementation note for storing digital sample data when using + WavPack compression within ZIP files is that all of the bytes of + the sample data should be compressed. This includes any unused + bits up to the byte boundary. An example is a 2 byte sample that + uses only 12 bits for the sample data with 4 unused bits. If only + 12 bits are passed as the sample size to the WavPack routines, the 4 + unused bits will be set to 0 on extraction regardless of their original + state. To avoid this, the full 16 bits of the sample data size + should be provided. + +5.10 PPMd - Method 98 +--------------------- + + 5.10.1 PPMd is a data compression algorithm developed by Dmitry Shkarin + which includes a carryless rangecoder developed by Dmitry Subbotin. + This algorithm is based on predictive phrase matching on multiple + order contexts. Information and source code for this algorithm + can be found on the internet. Consult with the author of this + algorithm for information on terms or restrictions on use. + + 5.10.2 Support for PPMd within the ZIP format currently is provided only + for version I, revision 1 of the algorithm. Storage requirements + for using this algorithm are as follows: + + 5.10.3 Parameters needed to control the algorithm are stored in the two + bytes immediately preceding the compressed data. These bytes are + used to store the following fields: + + Model order - sets the maximum model order, default is 8, possible + values are from 2 to 16 inclusive + + Sub-allocator size - sets the size of sub-allocator in MB, default is 50, + possible values are from 1MB to 256MB inclusive + + Model restoration method - sets the method used to restart context + model at memory insufficiency, values are: + + 0 - restarts model from scratch - default + 1 - cut off model - decreases performance by as much as 2x + 2 - freeze context tree - not recommended + + 5.10.4 An example for packing these fields into the 2 byte storage field is + illustrated below. These values are stored in Intel low-byte/high-byte + order. + + wPPMd = (Model order - 1) + + ((Sub-allocator size - 1) << 4) + + (Model restoration method << 12) + + +6.0 Traditional PKWARE Encryption +---------------------------------- + + 6.0.1 The following information discusses the decryption steps + required to support traditional PKWARE encryption. This + form of encryption is considered weak by today's standards + and its use is recommended only for situations with + low security needs or for compatibility with older .ZIP + applications. + +6.1 Traditional PKWARE Decryption +--------------------------------- + + 6.1.1 PKWARE is grateful to Mr. Roger Schlafly for his expert + contribution towards the development of PKWARE's traditional + encryption. + + 6.1.2 PKZIP encrypts the compressed data stream. Encrypted files + must be decrypted before they can be extracted to their original + form. + + 6.1.3 Each encrypted file has an extra 12 bytes stored at the start + of the data area defining the encryption header for that file. The + encryption header is originally set to random values, and then + itself encrypted, using three, 32-bit keys. The key values are + initialized using the supplied encryption password. After each byte + is encrypted, the keys are then updated using pseudo-random number + generation techniques in combination with the same CRC-32 algorithm + used in PKZIP and described elsewhere in this document. + + 6.1.4 The following are the basic steps required to decrypt a file: + + 1) Initialize the three 32-bit keys with the password. + 2) Read and decrypt the 12-byte encryption header, further + initializing the encryption keys. + 3) Read and decrypt the compressed data stream using the + encryption keys. + + 6.1.5 Initializing the encryption keys + + Key(0) <- 305419896 + Key(1) <- 591751049 + Key(2) <- 878082192 + + loop for i <- 0 to length(password)-1 + update_keys(password(i)) + end loop + + Where update_keys() is defined as: + + update_keys(char): + Key(0) <- crc32(key(0),char) + Key(1) <- Key(1) + (Key(0) & 000000ffH) + Key(1) <- Key(1) * 134775813 + 1 + Key(2) <- crc32(key(2),key(1) >> 24) + end update_keys + + Where crc32(old_crc,char) is a routine that given a CRC value and a + character, returns an updated CRC value after applying the CRC-32 + algorithm described elsewhere in this document. + + 6.1.6 Decrypting the encryption header + + The purpose of this step is to further initialize the encryption + keys, based on random data, to render a plaintext attack on the + data ineffective. + + Read the 12-byte encryption header into Buffer, in locations + Buffer(0) thru Buffer(11). + + loop for i <- 0 to 11 + C <- buffer(i) ^ decrypt_byte() + update_keys(C) + buffer(i) <- C + end loop + + Where decrypt_byte() is defined as: + + unsigned char decrypt_byte() + local unsigned short temp + temp <- Key(2) | 2 + decrypt_byte <- (temp * (temp ^ 1)) >> 8 + end decrypt_byte + + After the header is decrypted, the last 1 or 2 bytes in Buffer + should be the high-order word/byte of the CRC for the file being + decrypted, stored in Intel low-byte/high-byte order. Versions of + PKZIP prior to 2.0 used a 2 byte CRC check; a 1 byte CRC check is + used on versions after 2.0. This can be used to test if the password + supplied is correct or not. + + 6.1.7 Decrypting the compressed data stream + + The compressed data stream can be decrypted as follows: + + loop until done + read a character into C + Temp <- C ^ decrypt_byte() + update_keys(temp) + output Temp + end loop + + +7.0 Strong Encryption Specification +----------------------------------- + + 7.0.1 Portions of the Strong Encryption technology defined in this + specification are covered under patents and pending patent applications. + Refer to the section in this document entitled "Incorporating + PKWARE Proprietary Technology into Your Product" for more information. + +7.1 Strong Encryption Overview +------------------------------ + + 7.1.1 Version 5.x of this specification introduced support for strong + encryption algorithms. These algorithms can be used with either + a password or an X.509v3 digital certificate to encrypt each file. + This format specification supports either password or certificate + based encryption to meet the security needs of today, to enable + interoperability between users within both PKI and non-PKI + environments, and to ensure interoperability between different + computing platforms that are running a ZIP program. + + 7.1.2 Password based encryption is the most common form of encryption + people are familiar with. However, inherent weaknesses with + passwords (e.g. susceptibility to dictionary/brute force attack) + as well as password management and support issues make certificate + based encryption a more secure and scalable option. Industry + efforts and support are defining and moving towards more advanced + security solutions built around X.509v3 digital certificates and + Public Key Infrastructures(PKI) because of the greater scalability, + administrative options, and more robust security over traditional + password based encryption. + + 7.1.3 Most standard encryption algorithms are supported with this + specification. Reference implementations for many of these + algorithms are available from either commercial or open source + distributors. Readily available cryptographic toolkits make + implementation of the encryption features straight-forward. + This document is not intended to provide a treatise on data + encryption principles or theory. Its purpose is to document the + data structures required for implementing interoperable data + encryption within the .ZIP format. It is strongly recommended that + you have a good understanding of data encryption before reading + further. + + 7.1.4 The algorithms introduced in Version 5.0 of this specification + include: + + RC2 40 bit, 64 bit, and 128 bit + RC4 40 bit, 64 bit, and 128 bit + DES + 3DES 112 bit and 168 bit + + Version 5.1 adds support for the following: + + AES 128 bit, 192 bit, and 256 bit + + + 7.1.5 Version 6.1 introduces encryption data changes to support + interoperability with Smartcard and USB Token certificate storage + methods which do not support the OAEP strengthening standard. + + 7.1.6 Version 6.2 introduces support for encrypting metadata by compressing + and encrypting the central directory data structure to reduce information + leakage. Information leakage can occur in legacy ZIP applications + through exposure of information about a file even though that file is + stored encrypted. The information exposed consists of file + characteristics stored within the records and fields defined by this + specification. This includes data such as a file's name, its original + size, timestamp and CRC32 value. + + 7.1.7 Version 6.3 introduces support for encrypting data using the Blowfish + and Twofish algorithms. These are symmetric block ciphers developed + by Bruce Schneier. Blowfish supports using a variable length key from + 32 to 448 bits. Block size is 64 bits. Implementations should use 16 + rounds and the only mode supported within ZIP files is CBC. Twofish + supports key sizes 128, 192 and 256 bits. Block size is 128 bits. + Implementations should use 16 rounds and the only mode supported within + ZIP files is CBC. Information and source code for both Blowfish and + Twofish algorithms can be found on the internet. Consult with the author + of these algorithms for information on terms or restrictions on use. + + 7.1.8 Central Directory Encryption provides greater protection against + information leakage by encrypting the Central Directory structure and + by masking key values that are replicated in the unencrypted Local + Header. ZIP compatible programs that cannot interpret an encrypted + Central Directory structure cannot rely on the data in the corresponding + Local Header for decompression information. + + 7.1.9 Extra Field records that may contain information about a file that should + not be exposed should not be stored in the Local Header and should only + be written to the Central Directory where they can be encrypted. This + design currently does not support streaming. Information in the End of + Central Directory record, the Zip64 End of Central Directory Locator, + and the Zip64 End of Central Directory records are not encrypted. Access + to view data on files within a ZIP file with an encrypted Central Directory + requires the appropriate password or private key for decryption prior to + viewing any files, or any information about the files, in the archive. + + 7.1.10 Older ZIP compatible programs not familiar with the Central Directory + Encryption feature will no longer be able to recognize the Central + Directory and may assume the ZIP file is corrupt. Programs that + attempt streaming access using Local Headers will see invalid + information for each file. Central Directory Encryption need not be + used for every ZIP file. Its use is recommended for greater security. + ZIP files not using Central Directory Encryption should operate as + in the past. + + 7.1.11 This strong encryption feature specification is intended to provide for + scalable, cross-platform encryption needs ranging from simple password + encryption to authenticated public/private key encryption. + + 7.1.12 Encryption provides data confidentiality and privacy. It is + recommended that you combine X.509 digital signing with encryption + to add authentication and non-repudiation. + + +7.2 Single Password Symmetric Encryption Method +----------------------------------------------- + + 7.2.1 The Single Password Symmetric Encryption Method using strong + encryption algorithms operates similarly to the traditional + PKWARE encryption defined in this format. Additional data + structures are added to support the processing needs of the + strong algorithms. + + The Strong Encryption data structures are: + + 7.2.2 General Purpose Bits - Bits 0 and 6 of the General Purpose bit + flag in both local and central header records. Both bits set + indicates strong encryption. Bit 13, when set indicates the Central + Directory is encrypted and that selected fields in the Local Header + are masked to hide their actual value. + + + 7.2.3 Extra Field 0x0017 in central header only. + + Fields to consider in this record are: + + 7.2.3.1 Format - the data format identifier for this record. The only + value allowed at this time is the integer value 2. + + 7.2.3.2 AlgId - integer identifier of the encryption algorithm from the + following range + + 0x6601 - DES + 0x6602 - RC2 (version needed to extract < 5.2) + 0x6603 - 3DES 168 + 0x6609 - 3DES 112 + 0x660E - AES 128 + 0x660F - AES 192 + 0x6610 - AES 256 + 0x6702 - RC2 (version needed to extract >= 5.2) + 0x6720 - Blowfish + 0x6721 - Twofish + 0x6801 - RC4 + 0xFFFF - Unknown algorithm + + 7.2.3.3 Bitlen - Explicit bit length of key + + 32 - 448 bits + + 7.2.3.4 Flags - Processing flags needed for decryption + + 0x0001 - Password is required to decrypt + 0x0002 - Certificates only + 0x0003 - Password or certificate required to decrypt + + Values > 0x0003 reserved for certificate processing + + + 7.2.4 Decryption header record preceding compressed file data. + + -Decryption Header: + + Value Size Description + ----- ---- ----------- + IVSize 2 bytes Size of initialization vector (IV) + IVData IVSize Initialization vector for this file + Size 4 bytes Size of remaining decryption header data + Format 2 bytes Format definition for this record + AlgID 2 bytes Encryption algorithm identifier + Bitlen 2 bytes Bit length of encryption key + Flags 2 bytes Processing flags + ErdSize 2 bytes Size of Encrypted Random Data + ErdData ErdSize Encrypted Random Data + Reserved1 4 bytes Reserved certificate processing data + Reserved2 (var) Reserved for certificate processing data + VSize 2 bytes Size of password validation data + VData VSize-4 Password validation data + VCRC32 4 bytes Standard ZIP CRC32 of password validation data + + 7.2.4.1 IVData - The size of the IV should match the algorithm block size. + The IVData can be completely random data. If the size of + the randomly generated data does not match the block size + it should be complemented with zero's or truncated as + necessary. If IVSize is 0,then IV = CRC32 + Uncompressed + File Size (as a 64 bit little-endian, unsigned integer value). + + 7.2.4.2 Format - the data format identifier for this record. The only + value allowed at this time is the integer value 3. + + 7.2.4.3 AlgId - integer identifier of the encryption algorithm from the + following range + + 0x6601 - DES + 0x6602 - RC2 (version needed to extract < 5.2) + 0x6603 - 3DES 168 + 0x6609 - 3DES 112 + 0x660E - AES 128 + 0x660F - AES 192 + 0x6610 - AES 256 + 0x6702 - RC2 (version needed to extract >= 5.2) + 0x6720 - Blowfish + 0x6721 - Twofish + 0x6801 - RC4 + 0xFFFF - Unknown algorithm + + 7.2.4.4 Bitlen - Explicit bit length of key + + 32 - 448 bits + + 7.2.4.5 Flags - Processing flags needed for decryption + + 0x0001 - Password is required to decrypt + 0x0002 - Certificates only + 0x0003 - Password or certificate required to decrypt + + Values > 0x0003 reserved for certificate processing + + 7.2.4.6 ErdData - Encrypted random data is used to store random data that + is used to generate a file session key for encrypting + each file. SHA1 is used to calculate hash data used to + derive keys. File session keys are derived from a master + session key generated from the user-supplied password. + If the Flags field in the decryption header contains + the value 0x4000, then the ErdData field must be + decrypted using 3DES. If the value 0x4000 is not set, + then the ErdData field must be decrypted using AlgId. + + + 7.2.4.7 Reserved1 - Reserved for certificate processing, if value is + zero, then Reserved2 data is absent. See the explanation + under the Certificate Processing Method for details on + this data structure. + + 7.2.4.8 Reserved2 - If present, the size of the Reserved2 data structure + is located by skipping the first 4 bytes of this field + and using the next 2 bytes as the remaining size. See + the explanation under the Certificate Processing Method + for details on this data structure. + + 7.2.4.9 VSize - This size value will always include the 4 bytes of the + VCRC32 data and will be greater than 4 bytes. + + 7.2.4.10 VData - Random data for password validation. This data is VSize + in length and VSize must be a multiple of the encryption + block size. VCRC32 is a checksum value of VData. + VData and VCRC32 are stored encrypted and start the + stream of encrypted data for a file. + + + 7.2.5 Useful Tips + + 7.2.5.1 Strong Encryption is always applied to a file after compression. The + block oriented algorithms all operate in Cypher Block Chaining (CBC) + mode. The block size used for AES encryption is 16. All other block + algorithms use a block size of 8. Two IDs are defined for RC2 to + account for a discrepancy found in the implementation of the RC2 + algorithm in the cryptographic library on Windows XP SP1 and all + earlier versions of Windows. It is recommended that zero length files + not be encrypted, however programs should be prepared to extract them + if they are found within a ZIP file. + + 7.2.5.2 A pseudo-code representation of the encryption process is as follows: + + Password = GetUserPassword() + MasterSessionKey = DeriveKey(SHA1(Password)) + RD = CryptographicStrengthRandomData() + For Each File + IV = CryptographicStrengthRandomData() + VData = CryptographicStrengthRandomData() + VCRC32 = CRC32(VData) + FileSessionKey = DeriveKey(SHA1(IV + RD) + ErdData = Encrypt(RD,MasterSessionKey,IV) + Encrypt(VData + VCRC32 + FileData, FileSessionKey,IV) + Done + + 7.2.5.3 The function names and parameter requirements will depend on + the choice of the cryptographic toolkit selected. Almost any + toolkit supporting the reference implementations for each + algorithm can be used. The RSA BSAFE(r), OpenSSL, and Microsoft + CryptoAPI libraries are all known to work well. + + + 7.3 Single Password - Central Directory Encryption + -------------------------------------------------- + + 7.3.1 Central Directory Encryption is achieved within the .ZIP format by + encrypting the Central Directory structure. This encapsulates the metadata + most often used for processing .ZIP files. Additional metadata is stored for + redundancy in the Local Header for each file. The process of concealing + metadata by encrypting the Central Directory does not protect the data within + the Local Header. To avoid information leakage from the exposed metadata + in the Local Header, the fields containing information about a file are masked. + + 7.3.2 Local Header + + Masking replaces the true content of the fields for a file in the Local + Header with false information. When masked, the Local Header is not + suitable for streaming access and the options for data recovery of damaged + archives is reduced. Extra Data fields that may contain confidential + data should not be stored within the Local Header. The value set into + the Version needed to extract field should be the correct value needed to + extract the file without regard to Central Directory Encryption. The fields + within the Local Header targeted for masking when the Central Directory is + encrypted are: + + Field Name Mask Value + ------------------ --------------------------- + compression method 0 + last mod file time 0 + last mod file date 0 + crc-32 0 + compressed size 0 + uncompressed size 0 + file name (variable size) Base 16 value from the + range 1 - 0xFFFFFFFFFFFFFFFF + represented as a string whose + size will be set into the + file name length field + + The Base 16 value assigned as a masked file name is simply a sequentially + incremented value for each file starting with 1 for the first file. + Modifications to a ZIP file may cause different values to be stored for + each file. For compatibility, the file name field in the Local Header + should never be left blank. As of Version 6.2 of this specification, + the Compression Method and Compressed Size fields are not yet masked. + Fields having a value of 0xFFFF or 0xFFFFFFFF for the ZIP64 format + should not be masked. + + 7.3.3 Encrypting the Central Directory + + Encryption of the Central Directory does not include encryption of the + Central Directory Signature data, the Zip64 End of Central Directory + record, the Zip64 End of Central Directory Locator, or the End + of Central Directory record. The ZIP file comment data is never + encrypted. + + Before encrypting the Central Directory, it may optionally be compressed. + Compression is not required, but for storage efficiency it is assumed + this structure will be compressed before encrypting. Similarly, this + specification supports compressing the Central Directory without + requiring that it also be encrypted. Early implementations of this + feature will assume the encryption method applied to files matches the + encryption applied to the Central Directory. + + Encryption of the Central Directory is done in a manner similar to + that of file encryption. The encrypted data is preceded by a + decryption header. The decryption header is known as the Archive + Decryption Header. The fields of this record are identical to + the decryption header preceding each encrypted file. The location + of the Archive Decryption Header is determined by the value in the + Start of the Central Directory field in the Zip64 End of Central + Directory record. When the Central Directory is encrypted, the + Zip64 End of Central Directory record will always be present. + + The layout of the Zip64 End of Central Directory record for all + versions starting with 6.2 of this specification will follow the + Version 2 format. The Version 2 format is as follows: + + The leading fixed size fields within the Version 1 format for this + record remain unchanged. The record signature for both Version 1 + and Version 2 will be 0x06064b50. Immediately following the last + byte of the field known as the Offset of Start of Central + Directory With Respect to the Starting Disk Number will begin the + new fields defining Version 2 of this record. + + 7.3.4 New fields for Version 2 + + Note: all fields stored in Intel low-byte/high-byte order. + + Value Size Description + ----- ---- ----------- + Compression Method 2 bytes Method used to compress the + Central Directory + Compressed Size 8 bytes Size of the compressed data + Original Size 8 bytes Original uncompressed size + AlgId 2 bytes Encryption algorithm ID + BitLen 2 bytes Encryption key length + Flags 2 bytes Encryption flags + HashID 2 bytes Hash algorithm identifier + Hash Length 2 bytes Length of hash data + Hash Data (variable) Hash data + + The Compression Method accepts the same range of values as the + corresponding field in the Central Header. + + The Compressed Size and Original Size values will not include the + data of the Central Directory Signature which is compressed or + encrypted. + + The AlgId, BitLen, and Flags fields accept the same range of values + the corresponding fields within the 0x0017 record. + + Hash ID identifies the algorithm used to hash the Central Directory + data. This data does not have to be hashed, in which case the + values for both the HashID and Hash Length will be 0. Possible + values for HashID are: + + Value Algorithm + ------ --------- + 0x0000 none + 0x0001 CRC32 + 0x8003 MD5 + 0x8004 SHA1 + 0x8007 RIPEMD160 + 0x800C SHA256 + 0x800D SHA384 + 0x800E SHA512 + + 7.3.5 When the Central Directory data is signed, the same hash algorithm + used to hash the Central Directory for signing should be used. + This is recommended for processing efficiency, however, it is + permissible for any of the above algorithms to be used independent + of the signing process. + + The Hash Data will contain the hash data for the Central Directory. + The length of this data will vary depending on the algorithm used. + + The Version Needed to Extract should be set to 62. + + The value for the Total Number of Entries on the Current Disk will + be 0. These records will no longer support random access when + encrypting the Central Directory. + + 7.3.6 When the Central Directory is compressed and/or encrypted, the + End of Central Directory record will store the value 0xFFFFFFFF + as the value for the Total Number of Entries in the Central + Directory. The value stored in the Total Number of Entries in + the Central Directory on this Disk field will be 0. The actual + values will be stored in the equivalent fields of the Zip64 + End of Central Directory record. + + 7.3.7 Decrypting and decompressing the Central Directory is accomplished + in the same manner as decrypting and decompressing a file. + + 7.4 Certificate Processing Method + --------------------------------- + + The Certificate Processing Method for ZIP file encryption + defines the following additional data fields: + + 7.4.1 Certificate Flag Values + + Additional processing flags that can be present in the Flags field of both + the 0x0017 field of the central directory Extra Field and the Decryption + header record preceding compressed file data are: + + 0x0007 - reserved for future use + 0x000F - reserved for future use + 0x0100 - Indicates non-OAEP key wrapping was used. If this + this field is set, the version needed to extract must + be at least 61. This means OAEP key wrapping is not + used when generating a Master Session Key using + ErdData. + 0x4000 - ErdData must be decrypted using 3DES-168, otherwise use the + same algorithm used for encrypting the file contents. + 0x8000 - reserved for future use + + + 7.4.2 CertData - Extra Field 0x0017 record certificate data structure + + The data structure used to store certificate data within the section + of the Extra Field defined by the CertData field of the 0x0017 + record are as shown: + + Value Size Description + ----- ---- ----------- + RCount 4 bytes Number of recipients. + HashAlg 2 bytes Hash algorithm identifier + HSize 2 bytes Hash size + SRList (var) Simple list of recipients hashed public keys + + + RCount This defines the number intended recipients whose + public keys were used for encryption. This identifies + the number of elements in the SRList. + + HashAlg This defines the hash algorithm used to calculate + the public key hash of each public key used + for encryption. This field currently supports + only the following value for SHA-1 + + 0x8004 - SHA1 + + HSize This defines the size of a hashed public key. + + SRList This is a variable length list of the hashed + public keys for each intended recipient. Each + element in this list is HSize. The total size of + SRList is determined using RCount * HSize. + + + 7.4.3 Reserved1 - Certificate Decryption Header Reserved1 Data + + Value Size Description + ----- ---- ----------- + RCount 4 bytes Number of recipients. + + RCount This defines the number intended recipients whose + public keys were used for encryption. This defines + the number of elements in the REList field defined below. + + + 7.4.4 Reserved2 - Certificate Decryption Header Reserved2 Data Structures + + + Value Size Description + ----- ---- ----------- + HashAlg 2 bytes Hash algorithm identifier + HSize 2 bytes Hash size + REList (var) List of recipient data elements + + + HashAlg This defines the hash algorithm used to calculate + the public key hash of each public key used + for encryption. This field currently supports + only the following value for SHA-1 + + 0x8004 - SHA1 + + HSize This defines the size of a hashed public key + defined in REHData. + + REList This is a variable length of list of recipient data. + Each element in this list consists of a Recipient + Element data structure as follows: + + + Recipient Element (REList) Data Structure: + + Value Size Description + ----- ---- ----------- + RESize 2 bytes Size of REHData + REKData + REHData HSize Hash of recipients public key + REKData (var) Simple key blob + + + RESize This defines the size of an individual REList + element. This value is the combined size of the + REHData field + REKData field. REHData is defined by + HSize. REKData is variable and can be calculated + for each REList element using RESize and HSize. + + REHData Hashed public key for this recipient. + + REKData Simple Key Blob. The format of this data structure + is identical to that defined in the Microsoft + CryptoAPI and generated using the CryptExportKey() + function. The version of the Simple Key Blob + supported at this time is 0x02 as defined by + Microsoft. + +7.5 Certificate Processing - Central Directory Encryption +--------------------------------------------------------- + + 7.5.1 Central Directory Encryption using Digital Certificates will + operate in a manner similar to that of Single Password Central + Directory Encryption. This record will only be present when there + is data to place into it. Currently, data is placed into this + record when digital certificates are used for either encrypting + or signing the files within a ZIP file. When only password + encryption is used with no certificate encryption or digital + signing, this record is not currently needed. When present, this + record will appear before the start of the actual Central Directory + data structure and will be located immediately after the Archive + Decryption Header if the Central Directory is encrypted. + + 7.5.2 The Archive Extra Data record will be used to store the following + information. Additional data may be added in future versions. + + Extra Data Fields: + + 0x0014 - PKCS#7 Store for X.509 Certificates + 0x0016 - X.509 Certificate ID and Signature for central directory + 0x0019 - PKCS#7 Encryption Recipient Certificate List + + The 0x0014 and 0x0016 Extra Data records that otherwise would be + located in the first record of the Central Directory for digital + certificate processing. When encrypting or compressing the Central + Directory, the 0x0014 and 0x0016 records must be located in the + Archive Extra Data record and they should not remain in the first + Central Directory record. The Archive Extra Data record will also + be used to store the 0x0019 data. + + 7.5.3 When present, the size of the Archive Extra Data record will be + included in the size of the Central Directory. The data of the + Archive Extra Data record will also be compressed and encrypted + along with the Central Directory data structure. + +7.6 Certificate Processing Differences +-------------------------------------- + + 7.6.1 The Certificate Processing Method of encryption differs from the + Single Password Symmetric Encryption Method as follows. Instead + of using a user-defined password to generate a master session key, + cryptographically random data is used. The key material is then + wrapped using standard key-wrapping techniques. This key material + is wrapped using the public key of each recipient that will need + to decrypt the file using their corresponding private key. + + 7.6.2 This specification currently assumes digital certificates will follow + the X.509 V3 format for 1024 bit and higher RSA format digital + certificates. Implementation of this Certificate Processing Method + requires supporting logic for key access and management. This logic + is outside the scope of this specification. + +7.7 OAEP Processing with Certificate-based Encryption +----------------------------------------------------- + + 7.7.1 OAEP stands for Optimal Asymmetric Encryption Padding. It is a + strengthening technique used for small encoded items such as decryption + keys. This is commonly applied in cryptographic key-wrapping techniques + and is supported by PKCS #1. Versions 5.0 and 6.0 of this specification + were designed to support OAEP key-wrapping for certificate-based + decryption keys for additional security. + + 7.7.2 Support for private keys stored on Smartcards or Tokens introduced + a conflict with this OAEP logic. Most card and token products do + not support the additional strengthening applied to OAEP key-wrapped + data. In order to resolve this conflict, versions 6.1 and above of this + specification will no longer support OAEP when encrypting using + digital certificates. + + 7.7.3 Versions of PKZIP available during initial development of the + certificate processing method set a value of 61 into the + version needed to extract field for a file. This indicates that + non-OAEP key wrapping is used. This affects certificate encryption + only, and password encryption functions should not be affected by + this value. This means values of 61 may be found on files encrypted + with certificates only, or on files encrypted with both password + encryption and certificate encryption. Files encrypted with both + methods can safely be decrypted using the password methods documented. + +8.0 Splitting and Spanning ZIP files +------------------------------------- + + 8.1 Spanned ZIP files + + 8.1.1 Spanning is the process of segmenting a ZIP file across + multiple removable media. This support has typically only + been provided for DOS formatted floppy diskettes. + + 8.2 Split ZIP files + + 8.2.1 File splitting is a newer derivation of spanning. + Splitting follows the same segmentation process as + spanning, however, it does not require writing each + segment to a unique removable medium and instead supports + placing all pieces onto local or non-removable locations + such as file systems, local drives, folders, etc. + + 8.3 File Naming Differences + + 8.3.1 A key difference between spanned and split ZIP files is + that all pieces of a spanned ZIP file have the same name. + Since each piece is written to a separate volume, no name + collisions occur and each segment can reuse the original + .ZIP file name given to the archive. + + 8.3.2 Sequence ordering for DOS spanned archives uses the DOS + volume label to determine segment numbers. Volume labels + for each segment are written using the form PKBACK#xxx, + where xxx is the segment number written as a decimal + value from 001 - nnn. + + 8.3.3 Split ZIP files are typically written to the same location + and are subject to name collisions if the spanned name + format is used since each segment will reside on the same + drive. To avoid name collisions, split archives are named + as follows. + + Segment 1 = filename.z01 + Segment n-1 = filename.z(n-1) + Segment n = filename.zip + + 8.3.4 The .ZIP extension is used on the last segment to support + quickly reading the central directory. The segment number + n should be a decimal value. + + 8.4 Spanned Self-extracting ZIP Files + + 8.4.1 Spanned ZIP files may be PKSFX Self-extracting ZIP files. + PKSFX files may also be split, however, in this case + the first segment must be named filename.exe. The first + segment of a split PKSFX archive must be large enough to + include the entire executable program. + + 8.5 Capacities and Markers + + 8.5.1 Capacities for split archives are as follows: + + Maximum number of segments = 4,294,967,295 - 1 + Maximum .ZIP segment size = 4,294,967,295 bytes + Minimum segment size = 64K + Maximum PKSFX segment size = 2,147,483,647 bytes + + 8.5.2 Segment sizes may be different however by convention, all + segment sizes should be the same with the exception of the + last, which may be smaller. Local and central directory + header records must never be split across a segment boundary. + When writing a header record, if the number of bytes remaining + within a segment is less than the size of the header record, + end the current segment and write the header at the start + of the next segment. The central directory may span segment + boundaries, but no single record in the central directory + should be split across segments. + + 8.5.3 Spanned/Split archives created using PKZIP for Windows + (V2.50 or greater), PKZIP Command Line (V2.50 or greater), + or PKZIP Explorer will include a special spanning + signature as the first 4 bytes of the first segment of + the archive. This signature (0x08074b50) will be + followed immediately by the local header signature for + the first file in the archive. + + 8.5.4 A special spanning marker may also appear in spanned/split + archives if the spanning or splitting process starts but + only requires one segment. In this case the 0x08074b50 + signature will be replaced with the temporary spanning + marker signature of 0x30304b50. Split archives can + only be uncompressed by other versions of PKZIP that + know how to create a split archive. + + 8.5.5 The signature value 0x08074b50 is also used by some + ZIP implementations as a marker for the Data Descriptor + record. Conflict in this alternate assignment can be + avoided by ensuring the position of the signature + within the ZIP file to determine the use for which it + is intended. + +9.0 Change Process +------------------ + + 9.1 In order for the .ZIP file format to remain a viable technology, this + specification should be considered as open for periodic review and + revision. Although this format was originally designed with a + certain level of extensibility, not all changes in technology + (present or future) were or will be necessarily considered in its + design. + + 9.2 If your application requires new definitions to the + extensible sections in this format, or if you would like to + submit new data structures or new capabilities, please forward + your request to zipformat@pkware.com. All submissions will be + reviewed by the ZIP File Specification Committee for possible + inclusion into future versions of this specification. + + 9.3 Periodic revisions to this specification will be published as + DRAFT or as FINAL status to ensure interoperability. We encourage + comments and feedback that may help improve clarity or content. + + +10.0 Incorporating PKWARE Proprietary Technology into Your Product +------------------------------------------------------------------ + + 10.1 The Use or Implementation in a product of APPNOTE technological + components pertaining to either strong encryption or patching requires + a separate, executed license agreement from PKWARE. Please contact + PKWARE at zipformat@pkware.com or +1-414-289-9788 with regard to + acquiring such a license. + + 10.2 Additional information regarding PKWARE proprietray technology is + available at http://www.pkware.com/appnote. + +11.0 Acknowledgements +--------------------- + + In addition to the above mentioned contributors to PKZIP and PKUNZIP, + PKWARE would like to extend special thanks to Robert Mahoney for + suggesting the extension .ZIP for this software. + +12.0 References +--------------- + + Fiala, Edward R., and Greene, Daniel H., "Data compression with + finite windows", Communications of the ACM, Volume 32, Number 4, + April 1989, pages 490-505. + + Held, Gilbert, "Data Compression, Techniques and Applications, + Hardware and Software Considerations", John Wiley & Sons, 1987. + + Huffman, D.A., "A method for the construction of minimum-redundancy + codes", Proceedings of the IRE, Volume 40, Number 9, September 1952, + pages 1098-1101. + + Nelson, Mark, "LZW Data Compression", Dr. Dobbs Journal, Volume 14, + Number 10, October 1989, pages 29-37. + + Nelson, Mark, "The Data Compression Book", M&T Books, 1991. + + Storer, James A., "Data Compression, Methods and Theory", + Computer Science Press, 1988 + + Welch, Terry, "A Technique for High-Performance Data Compression", + IEEE Computer, Volume 17, Number 6, June 1984, pages 8-19. + + Ziv, J. and Lempel, A., "A universal algorithm for sequential data + compression", Communications of the ACM, Volume 30, Number 6, + June 1987, pages 520-540. + + Ziv, J. and Lempel, A., "Compression of individual sequences via + variable-rate coding", IEEE Transactions on Information Theory, + Volume 24, Number 5, September 1978, pages 530-536. + + +APPENDIX A - AS/400 Extra Field (0x0065) Attribute Definitions +-------------------------------------------------------------- + +A.1 Field Definition Structure: + + a. field length including length 2 bytes + b. field code 2 bytes + c. data x bytes + +A.2 Field Code Description + + 4001 Source type i.e. CLP etc + 4002 The text description of the library + 4003 The text description of the file + 4004 The text description of the member + 4005 x'F0' or 0 is PF-DTA, x'F1' or 1 is PF_SRC + 4007 Database Type Code 1 byte + 4008 Database file and fields definition + 4009 GZIP file type 2 bytes + 400B IFS code page 2 bytes + 400C IFS Creation Time 4 bytes + 400D IFS Access Time 4 bytes + 400E IFS Modification time 4 bytes + 005C Length of the records in the file 2 bytes + 0068 GZIP two words 8 bytes + +APPENDIX B - z/OS Extra Field (0x0065) Attribute Definitions +------------------------------------------------------------ + +B.1 Field Definition Structure: + + a. field length including length 2 bytes + b. field code 2 bytes + c. data x bytes + +B.2 Field Code Description + + 0001 File Type 2 bytes + 0002 NonVSAM Record Format 1 byte + 0003 Reserved + 0004 NonVSAM Block Size 2 bytes Big Endian + 0005 Primary Space Allocation 3 bytes Big Endian + 0006 Secondary Space Allocation 3 bytes Big Endian + 0007 Space Allocation Type1 byte flag + 0008 Modification Date Retired with PKZIP 5.0 + + 0009 Expiration Date Retired with PKZIP 5.0 + + 000A PDS Directory Block Allocation 3 bytes Big Endian binary value + 000B NonVSAM Volume List variable + 000C UNIT Reference Retired with PKZIP 5.0 + + 000D DF/SMS Management Class 8 bytes EBCDIC Text Value + 000E DF/SMS Storage Class 8 bytes EBCDIC Text Value + 000F DF/SMS Data Class 8 bytes EBCDIC Text Value + 0010 PDS/PDSE Member Info. 30 bytes + 0011 VSAM sub-filetype 2 bytes + 0012 VSAM LRECL 13 bytes EBCDIC "(num_avg num_max)" + 0013 VSAM Cluster Name Retired with PKZIP 5.0 + + 0014 VSAM KSDS Key Information 13 bytes EBCDIC "(num_length num_position)" + 0015 VSAM Average LRECL 5 bytes EBCDIC num_value padded with blanks + 0016 VSAM Maximum LRECL 5 bytes EBCDIC num_value padded with blanks + 0017 VSAM KSDS Key Length 5 bytes EBCDIC num_value padded with blanks + 0018 VSAM KSDS Key Position 5 bytes EBCDIC num_value padded with blanks + 0019 VSAM Data Name 1-44 bytes EBCDIC text string + 001A VSAM KSDS Index Name 1-44 bytes EBCDIC text string + 001B VSAM Catalog Name 1-44 bytes EBCDIC text string + 001C VSAM Data Space Type 9 bytes EBCDIC text string + 001D VSAM Data Space Primary 9 bytes EBCDIC num_value left-justified + 001E VSAM Data Space Secondary 9 bytes EBCDIC num_value left-justified + 001F VSAM Data Volume List variable EBCDIC text list of 6-character Volume IDs + 0020 VSAM Data Buffer Space 8 bytes EBCDIC num_value left-justified + 0021 VSAM Data CISIZE 5 bytes EBCDIC num_value left-justified + 0022 VSAM Erase Flag 1 byte flag + 0023 VSAM Free CI % 3 bytes EBCDIC num_value left-justified + 0024 VSAM Free CA % 3 bytes EBCDIC num_value left-justified + 0025 VSAM Index Volume List variable EBCDIC text list of 6-character Volume IDs + 0026 VSAM Ordered Flag 1 byte flag + 0027 VSAM REUSE Flag 1 byte flag + 0028 VSAM SPANNED Flag 1 byte flag + 0029 VSAM Recovery Flag 1 byte flag + 002A VSAM WRITECHK Flag 1 byte flag + 002B VSAM Cluster/Data SHROPTS 3 bytes EBCDIC "n,y" + 002C VSAM Index SHROPTS 3 bytes EBCDIC "n,y" + 002D VSAM Index Space Type 9 bytes EBCDIC text string + 002E VSAM Index Space Primary 9 bytes EBCDIC num_value left-justified + 002F VSAM Index Space Secondary 9 bytes EBCDIC num_value left-justified + 0030 VSAM Index CISIZE 5 bytes EBCDIC num_value left-justified + 0031 VSAM Index IMBED 1 byte flag + 0032 VSAM Index Ordered Flag 1 byte flag + 0033 VSAM REPLICATE Flag 1 byte flag + 0034 VSAM Index REUSE Flag 1 byte flag + 0035 VSAM Index WRITECHK Flag 1 byte flag Retired with PKZIP 5.0 + + 0036 VSAM Owner 8 bytes EBCDIC text string + 0037 VSAM Index Owner 8 bytes EBCDIC text string + 0038 Reserved + 0039 Reserved + 003A Reserved + 003B Reserved + 003C Reserved + 003D Reserved + 003E Reserved + 003F Reserved + 0040 Reserved + 0041 Reserved + 0042 Reserved + 0043 Reserved + 0044 Reserved + 0045 Reserved + 0046 Reserved + 0047 Reserved + 0048 Reserved + 0049 Reserved + 004A Reserved + 004B Reserved + 004C Reserved + 004D Reserved + 004E Reserved + 004F Reserved + 0050 Reserved + 0051 Reserved + 0052 Reserved + 0053 Reserved + 0054 Reserved + 0055 Reserved + 0056 Reserved + 0057 Reserved + 0058 PDS/PDSE Member TTR Info. 6 bytes Big Endian + 0059 PDS 1st LMOD Text TTR 3 bytes Big Endian + 005A PDS LMOD EP Rec # 4 bytes Big Endian + 005B Reserved + 005C Max Length of records 2 bytes Big Endian + 005D PDSE Flag 1 byte flag + 005E Reserved + 005F Reserved + 0060 Reserved + 0061 Reserved + 0062 Reserved + 0063 Reserved + 0064 Reserved + 0065 Last Date Referenced 4 bytes Packed Hex "yyyymmdd" + 0066 Date Created 4 bytes Packed Hex "yyyymmdd" + 0068 GZIP two words 8 bytes + 0071 Extended NOTE Location 12 bytes Big Endian + 0072 Archive device UNIT 6 bytes EBCDIC + 0073 Archive 1st Volume 6 bytes EBCDIC + 0074 Archive 1st VOL File Seq# 2 bytes Binary + +APPENDIX C - Zip64 Extensible Data Sector Mappings +--------------------------------------------------- + + -Z390 Extra Field: + + The following is the general layout of the attributes for the + ZIP 64 "extra" block for extended tape operations. + + Note: some fields stored in Big Endian format. All text is + in EBCDIC format unless otherwise specified. + + Value Size Description + ----- ---- ----------- + (Z390) 0x0065 2 bytes Tag for this "extra" block type + Size 4 bytes Size for the following data block + Tag 4 bytes EBCDIC "Z390" + Length71 2 bytes Big Endian + Subcode71 2 bytes Enote type code + FMEPos 1 byte + Length72 2 bytes Big Endian + Subcode72 2 bytes Unit type code + Unit 1 byte Unit + Length73 2 bytes Big Endian + Subcode73 2 bytes Volume1 type code + FirstVol 1 byte Volume + Length74 2 bytes Big Endian + Subcode74 2 bytes FirstVol file sequence + FileSeq 2 bytes Sequence + +APPENDIX D - Language Encoding (EFS) +------------------------------------ + +D.1 The ZIP format has historically supported only the original IBM PC character +encoding set, commonly referred to as IBM Code Page 437. This limits storing +file name characters to only those within the original MS-DOS range of values +and does not properly support file names in other character encodings, or +languages. To address this limitation, this specification will support the +following change. + +D.2 If general purpose bit 11 is unset, the file name and comment should conform +to the original ZIP character encoding. If general purpose bit 11 is set, the +filename and comment must support The Unicode Standard, Version 4.1.0 or +greater using the character encoding form defined by the UTF-8 storage +specification. The Unicode Standard is published by the The Unicode +Consortium (www.unicode.org). UTF-8 encoded data stored within ZIP files +is expected to not include a byte order mark (BOM). + +D.3 Applications may choose to supplement this file name storage through the use +of the 0x0008 Extra Field. Storage for this optional field is currently +undefined, however it will be used to allow storing extended information +on source or target encoding that may further assist applications with file +name, or file content encoding tasks. Please contact PKWARE with any +requirements on how this field should be used. + +D.4 The 0x0008 Extra Field storage may be used with either setting for general +purpose bit 11. Examples of the intended usage for this field is to store +whether "modified-UTF-8" (JAVA) is used, or UTF-8-MAC. Similarly, other +commonly used character encoding (code page) designations can be indicated +through this field. Formalized values for use of the 0x0008 record remain +undefined at this time. The definition for the layout of the 0x0008 field +will be published when available. Use of the 0x0008 Extra Field provides +for storing data within a ZIP file in an encoding other than IBM Code +Page 437 or UTF-8. + +D.5 General purpose bit 11 will not imply any encoding of file content or +password. Values defining character encoding for file content or +password must be stored within the 0x0008 Extended Language Encoding +Extra Field. + +D.6 Ed Gordon of the Info-ZIP group has defined a pair of "extra field" records +that can be used to store UTF-8 file name and file comment fields. These +records can be used for cases when the general purpose bit 11 method +for storing UTF-8 data in the standard file name and comment fields is +not desirable. A common case for this alternate method is if backward +compatibility with older programs is required. + +D.7 Definitions for the record structure of these fields are included above +in the section on 3rd party mappings for "extra field" records. These +records are identified by Header ID's 0x6375 (Info-ZIP Unicode Comment +Extra Field) and 0x7075 (Info-ZIP Unicode Path Extra Field). + +D.8 The choice of which storage method to use when writing a ZIP file is left +to the implementation. Developers should expect that a ZIP file may +contain either method and should provide support for reading data in +either format. Use of general purpose bit 11 reduces storage requirements +for file name data by not requiring additional "extra field" data for +each file, but can result in older ZIP programs not being able to extract +files. Use of the 0x6375 and 0x7075 records will result in a ZIP file +that should always be readable by older ZIP programs, but requires more +storage per file to write file name and/or file comment fields. diff --git a/reference/RAR 5.0 archive format.htm b/reference/RAR 5.0 archive format.htm new file mode 100644 index 00000000..ecaae5e2 --- /dev/null +++ b/reference/RAR 5.0 archive format.htm @@ -0,0 +1,1197 @@ + + + +RAR 5.0 archive format + + + + +

RAR 5.0 archive format

+ +

Here we describe basic data structures of archive format introduced +in RAR 5.0. If you need information about algorithms or more detailed +information on data structures, please use UnRAR source code.

+ +

Contents

+ + +

Data types

+ +

vint

+ +

Variable length integer. Can include one or more bytes, where lower +7 bits of every byte contain integer data and highest bit in every byte +is the continuation flag. If highest bit is 0, this is the last byte +in sequence. So first byte contains 7 least significant bits of integer +and continuation flag. Second byte, if present, contains next 7 bits +and so on.

+

Currently RAR format uses vint to store up to 64 bit integers, +resulting in 10 bytes maximum. This value may be increased in the future +if necessary for some reason.

+

Sometimes RAR needs to pre-allocate space for vint before knowing +its exact value. In such situation it can allocate more space than really +necessary and then fill several leading bytes with 0x80 hexadecimal, +which means 0 with continuation flag set.

+ +

byte, uint16, uint32, uint64

+ +

Byte, 16-, 32-, 64- bit unsigned integer in little endian format.

+ +

Variable length data

+ +

We use ellipsis ... to denote variable length data areas. + +

Hexadecimal values

+

We use 0x prefix to define hexadecimal values, such as 0xf000

+ + + +

General archive structure

+ +

General archive block format

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldSizeDescription
Header CRC32uint32CRC32 of header data starting from Header size field +and up to and including the optional extra area.
Header sizevintSize of header data starting from Header type field +and up to and including the optional extra area. +This field must not be longer than 3 bytes in current implementation, +resulting in 2 MB maximum header size. +
Header typevintType of archive header. Possible values are: +
  1   Main archive header. +
  2   File header. +
  3   Service header. +
  4   Archive encryption header. +
  5   End of archive header. +
Header flagsvintFlags common for all headers: +
  0x0001   Extra area is present in the end of header. +
  0x0002   Data area is present in the end of header. +
  0x0004   Blocks with unknown type and this flag must be skipped when updating an archive. +
  0x0008   Data area is continuing from previous volume. +
  0x0010   Data area is continuing in next volume. +
  0x0020   Block depends on preceding file block. +
  0x0040   Preserve a child block if host block is modified. +
Extra area sizevintSize of extra area. Optional field, present only if 0x0001 header flag +is set.
Data sizevintSize of data area. Optional field, present only if 0x0002 header flag +is set.
......Fields specific for current block type. See concrete block types +description for details.
Extra area...Optional area containing additional header fields, present only +if 0x0001 header flag is set.
Data areavintOptional data area, present only if 0x0002 header flag is set. +Used to store large data amounts, such as compressed file data. +Not counted in Header CRC and Header size fields.
+ +

General extra area format

+ +

Extra area can include one or more records having the following format:

+ + + + + + + + + + + + + + + + + + + + + +
SizevintSize of record data starting from Type. +
TypevintRecord type. Different archive blocks have different associated extra area +record types. Read the concrete archive block description for details. +New record types can be added in the future, so unknown record types +need to be skipped without interrupting an operation. +
Data...Record dependent data. May be missing if record consists only from +size and type. +
+ + +

General archive layout

+ +
    Self-extracting module (optional)
+    RAR 5.0 signature
+    Archive encryption header (optional)
+    Main archive header
+    Archive comment service header (optional)
+
+    File header 1
+    Service headers (NTFS ACL, streams, etc.) for preceding file (optional).
+    ...
+    File header N
+    Service headers (NTFS ACL, streams, etc.) for preceding file (optional).
+
+    Recovery record (optional).
+    End of archive header.
+
+ +

Archive blocks

+ +

Self-extracting module (SFX)

+ +

Any data preceding the archive signature. Self-extracting module size +and contents is not defined. At the moment of writing this documentation +RAR assumes the maximum SFX module size to not exceed 1 MB, but this value +can be increased in the future.

+ +

RAR 5.0 signature

+ +

+RAR 5.0 signature consists of 8 bytes: +0x52 0x61 0x72 0x21 0x1A 0x07 0x01 0x00. +You need to search for this signature in supposed archive from beginning +and up to maximum SFX module size. +Just for comparison this is RAR 4.x 7 byte length signature: +0x52 0x61 0x72 0x21 0x1A 0x07 0x00. +

+ + +

Archive encryption header

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Header CRC32uint32
Header sizevint
Header typevint4
Header flagsvintFlags common for all headers
Encryption versionvint +Version of encryption algorithm. Now only 0 version (AES-256) +is supported.
Encryption flagsvint + 0x0001   Password check data is present. +
KDF count1 byte +Binary logarithm of iteration number for PBKDF2 function. RAR can refuse +to process KDF count exceeding some threshold. Concrete value of threshold +is a version dependent. +
Salt16 bytes +Salt value used globally for all encrypted archive headers. +
Check value12 bytes +Value used to verify the password validity. Present only if 0x0001 +encryption flag is set. First 8 bytes are calculated using additional +PBKDF2 rounds, 4 last bytes is the additional checksum. Together with +the standard header CRC32 we have 64 bit checksum to reliably verify +this field integrity and distinguish invalid password and damaged data. +Further details can be found in UnRAR source code. +
+ +

This header is present only in archives with encrypted headers. +Every next header after this one is started from 16 byte AES-256 +initialization vector followed by encrypted header data. Size of encrypted +header data block is aligned to 16 byte boundary.

+ +

Main archive header

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Header CRC32uint32
Header sizevint
Header typevint1
Header flagsvintFlags common for all headers
Extra area sizevintSize of extra area. Optional field, present only if 0x0001 header flag +is set.
Archive flagsvint +   0x0001   Volume. Archive is a part of multivolume set.
+   0x0002   Volume number field is present. This flag is present + in all volumes except first.
+   0x0004   Solid archive.
+   0x0008   Recovery record is present.
+   0x0010   Locked archive. +
Volume numbervintOptional field, present only if 0x0002 archive flag is set. +Not present for first volume, 1 for second volume, 2 for third and so on.
Extra area...Optional area containing additional header fields, present only +if 0x0001 header flag is set.
+ +

Extra area of main archive header can contain following record +types

+ + + + + + + +
TypeNameDescription
0x01LocatorContains positions of different service blocks, so they can be accessed +quickly, without scanning the entire archive. This record is optional. +If it is missing, it is still necessary to scan the entire archive to +verify presence of service blocks.
+ +

Locator record

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Sizevint
Typevint1
Flagsvint +   0x0001   Quick open record offset is present.
+   0x0002   Recovery record offset is present.
+
Quick open offsetvintDistance from beginning of quick open service block to beginning +of main archive header. Present only if 0x0001 flag is set. +If equal to 0, should be ignored. It can be set to zero if preallocated +space was not enough to store resulting offset. +
Recovery record offsetvintDistance from beginning of recovery record service block to beginning +of main archive header. Present only if 0x0002 flag is set. +If equal to 0, should be ignored. It can be set to zero if preallocated +space was not enough to store resulting offset. +
+ + +

File header and service header

+ +

These two header types use the similar data structure, so we describe them +both here.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Header CRC32uint32
Header sizevint
Header typevint2 for file header, 3 for service header
Header flagsvintFlags common for all headers
Extra area sizevintSize of extra area. Optional field, present only if 0x0001 header flag +is set.
Data sizevintSize of data area. Optional field, present only if 0x0002 header flag +is set. For file header this field contains the packed file size.
File flagsvintFlags specific for these header types:
+   0x0001   Directory file system object (file header only).
+   0x0002   Time field in Unix format is present.
+   0x0004   CRC32 field is present.
+   0x0008   Unpacked size is unknown.
+ +

If flag 0x0008 is set, unpacked size field is still present, + but must be ignored and extraction must be performed until reaching + the end of compression stream. This flag can be set if actual file size + is larger than reported by OS or if file size is unknown such as + for all volumes except last when archiving from stdin to multivolume + archive.

+
Unpacked sizevintUnpacked file or service data size.
AttributesvintOperating system specific file attributes in case of file header. +Might be either used for data specific needs or just reserved and set to 0 +for service header.
mtimeuint32File modification time in Unix time format. +Optional, present if 0x0002 file flag is set.
Data CRC32uint32CRC32 of unpacked file or service data. For files split between volumes +it contains CRC32 of file packed data contained in current volume +for all file parts except the last. +Optional, present if 0x0004 file flag is set.
Compression informationvint +

Lower 6 bits (0x003f mask) contain the version of compression algorithm, +resulting in possible 0 - 63 values. Current version is 0.

+

7th bit (0x0040) defines the solid flag. If it is set, RAR continues +to use the compression dictionary left after processing preceding files. +It can be set only for file headers and is never set for service headers.

+

Bits 8 - 10 (0x0380 mask) define the compression method. Currently only +values 0 - 5 are used. 0 means no compression.

+

Bits 11 - 14 (0x3c00) define the minimum size of dictionary size required +to extract data. Value 0 means 128 KB, 1 - 256 KB, ..., 14 - 2048 MB, +15 - 4096 MB.

+
Host OSvintType of operating system used to create the archive.
+   0x0000   Windows.
+   0x0001   Unix.
+
Name lengthvintFile or service header name length.
Name? bytes +

Variable length field containing Name length bytes in UTF-8 +format without trailing zero.

+

For file header this is a name of archived file. Forward slash character +is used as the path separator both for Unix and Windows names. Backslashes +are treated as a part of name for Unix names and as invalid character +for Windows file names. Type of name is defined by Host OS field.

+

If Unix file name contains any high ASCII characters which cannot be +correctly converted to Unicode and UTF-8, we map such characters to +to 0xE080 - 0xE0FF private use Unicode area and insert 0xFFFE Unicode +non-character to resulting string to indicate that it contains mapped +characters, which need to be converted back when extracting. Concrete +position of 0xFFFE is not defined, we need to search the entire string +for it. Such mapped names are not portable and can be correctly unpacked +only on the same system where they were created.

+

For service header this field contains a name of service header. +Now the following names are used: + + + + + + +
CMTArchive comment
QOArchive quick open data
ACLNTFS file permissions
STMNTFS alternate data stream
RRRecovery record
+ +

+
Extra area...Optional area containing additional header fields, present only +if 0x0001 header flag is set.
Data areavintOptional data area, present only if 0x0002 header flag is set. +Store file data in case of file header or service data for service header. +Depending on the compression method value in Compression information +can be either uncompressed (compression method 0) or compressed. +
+ + +

File and service headers use the same types of extra area +records:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TypeNameDescription
0x01File encryptionFile encryption information.
0x02File hashFile data hash.
0x03File timeHigh precision file time.
0x04File versionFile version number.
0x05RedirectionFile system redirection.
0x06Unix ownerUnix owner and group information.
0x07Service dataService header data array.
+ + +

File encryption record

+ +

This record is present if file data is encrypted.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Sizevint
Typevint0x01
Versionvint +Version of encryption algorithm. Now only 0 version (AES-256) +is supported.
Flagsvint +   0x0001   Password check data is present.
+   0x0002   Use tweaked checksums instead of plain checksums.
+ +

If flag 0x0002 is present, RAR transforms the checksum preserving + file or service data integrity, so it becomes dependent on encryption key. + It makes guessing file contents based on checksum impossible. + It affects both data CRC32 in file header and checksums + in file hash record in extra area.

+
KDF count1 byte +Binary logarithm of iteration number for PBKDF2 function. RAR can refuse +to process KDF count exceeding some threshold. Concrete value of threshold +is version dependent. +
Salt16 bytes +Salt value to set the decryption key for encrypted file. +
IV16 bytes +AES-256 initialization vector. +
Check value12 bytes +Value used to verify the password validity. Present only if 0x0001 +encryption flag is set. First 8 bytes are calculated using additional +PBKDF2 rounds, 4 last bytes is the additional checksum. Together with +the standard header CRC32 we have 64 bit checksum to reliably verify +this field integrity and distinguish invalid password and damaged data. +Further details can be found in UnRAR source code. +
+ + +

File hash record

+ +

Only the standard CRC32 checksum can be stored directly in file header. +If other hash is used, it is stored in this extra area record:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
Sizevint
Typevint0x02
Hash typevint + 0x00   BLAKE2sp hash function.
+
Hash data? bytes + 32 bytes of BLAKE2sp for 0x00 hash type. +
+ +

For files split between volumes it contains a hash of file packed +data contained in current volume for all file parts except the last. +For files not split between volumes and for last parts of split files +it contains an unpacked data hash. +

+ + +

File time record

+ +

This record is used if it is necessary to store creation and last access +time or if 1 second precision of Unix mtime stored in file header is not +enough:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Sizevint
Typevint0x03
Flagsvint + 0x0001   Time is stored in Unix time format if this flags is set + and in Windows FILETIME format otherwise
+ 0x0002   Modification time is present
+ 0x0004   Creation time is present
+ 0x0008   Last access time is present
+
mtimeuint32 or uint64Modification time. Present if 0x0002 flag is set. Depending on 0x0001 +flag can be in Unix time or Windows FILETIME format.
ctimeuint32 or uint64Creation time. Present if 0x0004 flag is set. Depending on 0x0001 +flag can be in Unix time or Windows FILETIME format.
ctimeuint32 or uint64Last access time. Present if 0x0008 flag is set. Depending on 0x0001 +flag can be in Unix time or Windows FILETIME format.
+ + +

File version record

+ +

This record is used in archives created with -ver switch.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
Sizevint
Typevint0x04
Flagsvint + No file version flags are defined yet, so it is set to 0. +
Version numbervintFile version number.
+ + +

File system redirection record

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Sizevint
Typevint0x05
Redirection typevint + 0x0001   Unix symlink
+ 0x0002   Windows symlink
+ 0x0003   Windows junction
+ 0x0004   Hard link
+ 0x0005   File copy
+
Flagsvint + 0x0001   Link target is directory
+
Name lengthvintLength of link target name
NamevintLink target name in UTF-8 format without trailing zero
+ + +

Unix owner record

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Sizevint
Typevint0x06
Flagsvint + 0x0001   User name string is present
+ 0x0002   Group name string is present
+ 0x0004   Numeric user ID is present
+ 0x0008   Numeric group ID is present
+
User name lengthvintLength of owner user name. Present if 0x0001 flag is set.
User name? bytesOwner user name in native encoding. Not zero terminated. +Present if 0x0001 flag is set.
Group name lengthvintLength of owner group name. Present if 0x0002 flag is set.
Group name? bytesOwner group name in native encoding. Not zero terminated. +Present if 0x0002 flag is set.
User IDvintNumeric owner user ID. Present if 0x0004 flag is set.
Group IDvintNumeric owner group ID. Present if 0x0008 flag is set.
+ + +

Service data record

+ +

This record is used only by service headers to store additional +parameters.

+ + + + + + + + + + + + + + + + + + + + + +
Sizevint
Typevint0x07
Data? bytesConcrete contents of service data depends on service header type.
+ +

End of archive header

+ +

End of archive marker. RAR does not read anything after this header +letting to use third party tools to add extra information such as +a digital signature to archive.

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Header CRC32uint32
Header sizevint
Header typevint5
Header flagsvintFlags common for all headers
End of archive flagsvint +0x0001   Archive is volume and it is not last volume in the set
+
+ +

Service headers

+ +

RAR uses service headers based on the file header +data structure to store different supplementary information.

+ +

Archive comment header

+ +

Optional header storing the main archive comment. Contains CMT identifier +in file name field. Placed before any file headers and after the main +archive header. Comment data is stored in UTF-8 immediately after +the archive comment header. Now RAR does not use compression for archive +comments, so packed and unpacked data sizes in header are equal and they +both define the comment data size. Compression method in header is set +to 0.

+ +

Quick open header

+ +

Optional header storing the quick open record. Contains QO identifier +in file name field. Placed after all file headers, but before the recovery +record and end of archive header. It is possible to locate the quick open +header with locator record in main archive header. + +

Quick open record data is stored immediately after the quick open header. +RAR does not use compression for quick open data, so packed and unpacked +data sizes in header are equal and they both define the quick open data size. +Compression method in header is set to 0.

+ +

Quick open data is the array consisting of data cache structures. +Every data cache structure stores a portion of archived data +and has the following format:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
FieldSizeDescription
Structure CRC32uint32CRC32 of structure data starting from Structure size field.
Structure sizevintSize of structure data starting from Flags field. +This field must not be longer than 3 bytes in current implementation, +resulting in 2 MB maximum size. +
FlagsvintCurrently set to 0.
OffsetvintOffset from beginning of quick open header to beginning of archive data +cached in current structure. We can use this value to calculate +the absolute position of archived data stored the current structure. +It is guaranteed that absolute archive positions referred by data cache +structures are always growing when going from beginning of structure array +to end.
Data sizevintSize of archive data stored in the current structure.
Data? bytesArchive data stored in the current structure.
+ +

Normally RAR uses the quick open data to store copies of file and service +headers. It can store either all headers or only a part of them. If required +header is missing in quick open data or if structure CRC32 is invalid, +data are read from its original archive position.

+ +

Using the quick open data is optional. You can skip it completely +and read only standard archive headers. But it is important to use the same +access pattern when reading file names to display them to user and +to extract files. Otherwise it would be possible to see one file name +and extract another in case the quick open data and real archive data +are intentionally created different. It could introduce a security threat. +So if you use the quick open data when displaying the archive contents, +use it when extracting. If you do not use it when displaying +the archive contents, do not use it when extracting.

+ + + + \ No newline at end of file diff --git a/reference/unrar/UnRAR.vcxproj b/reference/unrar/UnRAR.vcxproj new file mode 100644 index 00000000..b55873e3 --- /dev/null +++ b/reference/unrar/UnRAR.vcxproj @@ -0,0 +1,279 @@ + + + + + Debug + Win32 + + + Debug + x64 + + + Release + Win32 + + + Release + x64 + + + + {95CC809B-03FC-4EDB-BB20-FD07A698C05F} + UnRAR + Win32Proj + 8.1 + + + + Application + v140_xp + MultiByte + true + + + Application + v140_xp + MultiByte + + + Application + v140_xp + MultiByte + false + + + Application + v140_xp + MultiByte + + + + + + + + + + + + + + + + + + + <_ProjectFileVersion>14.0.24720.0 + + + build\unrar32\$(Configuration)\ + build\unrar32\$(Configuration)\obj\ + true + false + + + build\unrar64\$(Configuration)\ + build\unrar64\$(Configuration)\obj\ + true + false + + + build\unrar32\$(Configuration)\ + build\unrar32\$(Configuration)\obj\ + false + false + + + build\unrar64\$(Configuration)\ + build\unrar64\$(Configuration)\obj\ + false + false + + + + /MP %(AdditionalOptions) + Disabled + UNRAR;%(PreprocessorDefinitions) + false + EnableFastChecks + MultiThreadedDebug + false + Use + rar.hpp + Level3 + ProgramDatabase + StdCall + 4007;4996;%(DisableSpecificWarnings) + NoExtensions + + + true + Console + MachineX86 + + + + + X64 + + + /MP %(AdditionalOptions) + Disabled + UNRAR;%(PreprocessorDefinitions) + false + EnableFastChecks + MultiThreadedDebug + false + Use + rar.hpp + Level3 + ProgramDatabase + StdCall + 4007;4996;%(DisableSpecificWarnings) + NotSet + + + true + Console + MachineX64 + + + + + /MP %(AdditionalOptions) + MaxSpeed + true + Neither + true + false + UNRAR;%(PreprocessorDefinitions) + false + MultiThreaded + Default + true + true + NoExtensions + Precise + false + Use + rar.hpp + Level3 + ProgramDatabase + StdCall + 4007;4996;%(DisableSpecificWarnings) + + + true + Console + true + true + + MachineX86 + + + + + X64 + + + /MP %(AdditionalOptions) + MinSpace + true + Neither + true + false + UNRAR;%(PreprocessorDefinitions) + false + false + MultiThreaded + true + true + false + Use + rar.hpp + Level3 + ProgramDatabase + StdCall + 4007;4996;%(DisableSpecificWarnings) + NotSet + + + true + Console + true + true + + MachineX64 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Create + Create + Create + Create + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/reference/unrar/UnRARDll.vcxproj b/reference/unrar/UnRARDll.vcxproj new file mode 100644 index 00000000..ecac0db0 --- /dev/null +++ b/reference/unrar/UnRARDll.vcxproj @@ -0,0 +1,420 @@ + + + + + Debug + Win32 + + + Debug + x64 + + + release_nocrypt + Win32 + + + release_nocrypt + x64 + + + Release + Win32 + + + Release + x64 + + + + UnRAR + {E815C46C-36C4-499F-BBC2-E772C6B17971} + UnRAR + Win32Proj + 8.1 + + + + DynamicLibrary + v140_xp + MultiByte + true + + + DynamicLibrary + v140_xp + MultiByte + true + + + DynamicLibrary + v140_xp + MultiByte + + + DynamicLibrary + v140_xp + MultiByte + false + + + DynamicLibrary + v140_xp + MultiByte + false + + + DynamicLibrary + v140_xp + MultiByte + + + + + + + + + + + + + + + + + + + + + + + + + <_ProjectFileVersion>14.0.24720.0 + + + build\unrardll32\$(Configuration)\ + build\unrardll32\$(Configuration)\obj\ + true + true + + + build\unrardll64\$(Configuration)\ + build\unrardll64\$(Configuration)\obj\ + true + true + + + build\unrardll32\$(Configuration)\ + build\unrardll32\$(Configuration)\obj\ + false + true + + + build\unrardll64\$(Configuration)\ + build\unrardll64\$(Configuration)\obj\ + false + true + + + build\unrardll32\$(Configuration)\ + build\unrardll32\$(Configuration)\obj\ + false + true + + + build\unrardll64\$(Configuration)\ + build\unrardll64\$(Configuration)\obj\ + false + true + + + + /MP %(AdditionalOptions) + Disabled + RARDLL;UNRAR;SILENT;%(PreprocessorDefinitions) + false + Sync + EnableFastChecks + MultiThreadedDebug + 4Bytes + false + Use + rar.hpp + Level3 + ProgramDatabase + Cdecl + 4007;4996;%(DisableSpecificWarnings) + NoExtensions + + + $(OutDir)unrar.dll + dll.def + true + Console + MachineX86 + + + + + X64 + + + /MP %(AdditionalOptions) + Disabled + RARDLL;UNRAR;SILENT;%(PreprocessorDefinitions) + false + Sync + EnableFastChecks + MultiThreadedDebug + 4Bytes + false + Use + rar.hpp + Level3 + ProgramDatabase + Cdecl + 4007;4996;%(DisableSpecificWarnings) + NotSet + + + $(OutDir)unrar.dll + dll.def + true + Console + MachineX64 + + + + + /MP %(AdditionalOptions) + MaxSpeed + true + Neither + true + false + RARDLL;UNRAR;SILENT;%(PreprocessorDefinitions) + false + Sync + MultiThreaded + 4Bytes + true + true + NoExtensions + Precise + false + Use + rar.hpp + Level3 + ProgramDatabase + Cdecl + 4007;4996;%(DisableSpecificWarnings) + + + /SAFESEH %(AdditionalOptions) + $(OutDir)unrar.dll + dll.def + true + Console + true + true + + MachineX86 + + + + + X64 + + + /MP %(AdditionalOptions) + MaxSpeed + true + Neither + true + false + RARDLL;UNRAR;SILENT;%(PreprocessorDefinitions) + false + false + Sync + MultiThreaded + 4Bytes + true + true + false + Use + rar.hpp + Level3 + ProgramDatabase + Cdecl + 4007;4996;%(DisableSpecificWarnings) + NotSet + + + $(OutDir)unrar.dll + dll.def + true + Console + true + true + + MachineX64 + + + + + /MP %(AdditionalOptions) + MaxSpeed + true + Neither + true + false + RARDLL;UNRAR;SILENT;RAR_NOCRYPT;%(PreprocessorDefinitions) + false + Sync + MultiThreaded + 4Bytes + true + true + NoExtensions + Precise + false + Use + rar.hpp + Level3 + ProgramDatabase + Cdecl + 4007;4996;%(DisableSpecificWarnings) + + + /SAFESEH %(AdditionalOptions) + $(OutDir)unrar.dll + dll_nocrypt.def + true + Console + true + true + + MachineX86 + + + + + X64 + + + /MP %(AdditionalOptions) + MaxSpeed + true + Neither + true + false + RARDLL;UNRAR;SILENT;RAR_NOCRYPT;%(PreprocessorDefinitions) + false + false + Sync + MultiThreaded + 4Bytes + true + true + false + Use + rar.hpp + Level3 + ProgramDatabase + StdCall + 4007;4996;%(DisableSpecificWarnings) + NotSet + + + $(OutDir)unrar.dll + dll_nocrypt.def + true + Console + true + true + + MachineX64 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Create + Create + Create + Create + Create + Create + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/reference/unrar/acknow.txt b/reference/unrar/acknow.txt new file mode 100644 index 00000000..b982ecee --- /dev/null +++ b/reference/unrar/acknow.txt @@ -0,0 +1,92 @@ + ACKNOWLEDGMENTS + +* We used "Screaming Fast Galois Field Arithmetic Using Intel + SIMD Instructions" paper by James S. Plank, Kevin M. Greenan + and Ethan L. Miller to improve Reed-Solomon coding performance. + Also we are grateful to Artem Drobanov and Bulat Ziganshin + for samples and ideas allowed to make Reed-Solomon coding + more efficient. + +* RAR text compression algorithm is based on Dmitry Shkarin PPMII + and Dmitry Subbotin carryless rangecoder public domain source code. + You may find it in ftp.elf.stuba.sk/pub/pc/pack. + +* RAR encryption includes parts of code from Szymon Stefanek + and Brian Gladman AES implementations also as Steve Reid SHA-1 source. + + --------------------------------------------------------------------------- + Copyright (c) 2002, Dr Brian Gladman < >, Worcester, UK. + All rights reserved. + + LICENSE TERMS + + The free distribution and use of this software in both source and binary + form is allowed (with or without changes) provided that: + + 1. distributions of this source code include the above copyright + notice, this list of conditions and the following disclaimer; + + 2. distributions in binary form include the above copyright + notice, this list of conditions and the following disclaimer + in the documentation and/or other associated materials; + + 3. the copyright holder's name is not used to endorse products + built using this software without specific written permission. + + ALTERNATIVELY, provided that this notice is retained in full, this product + may be distributed under the terms of the GNU General Public License (GPL), + in which case the provisions of the GPL apply INSTEAD OF those given above. + + DISCLAIMER + + This software is provided 'as is' with no explicit or implied warranties + in respect of its properties, including, but not limited to, correctness + and/or fitness for purpose. + --------------------------------------------------------------------------- + + Source code of this package also as other cryptographic technology + and computing project related links are available on Brian Gladman's + web site: http://www.gladman.me.uk + +* RAR uses CRC32 function based on Intel Slicing-by-8 algorithm. + Original Intel Slicing-by-8 code is available here: + + http://sourceforge.net/projects/slicing-by-8/ + + Original Intel Slicing-by-8 code is licensed under BSD License + available at http://www.opensource.org/licenses/bsd-license.html + + Copyright (c) 2004-2006 Intel Corporation. + All Rights Reserved + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions + are met: + + Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer + in the documentation and/or other materials provided with + the distribution. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS + FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + SUCH DAMAGE. + +* RAR archives may optionally include BLAKE2sp hash ( https://blake2.net ), + designed by Jean-Philippe Aumasson, Samuel Neves, Zooko Wilcox-O'Hearn + and Christian Winnerlein. + +* Useful hints provided by Alexander Khoroshev and Bulat Ziganshin allowed + to significantly improve RAR compression and speed. diff --git a/reference/unrar/arccmt.cpp b/reference/unrar/arccmt.cpp new file mode 100644 index 00000000..79963b27 --- /dev/null +++ b/reference/unrar/arccmt.cpp @@ -0,0 +1,166 @@ +static bool IsAnsiEscComment(const wchar *Data,size_t Size); + +bool Archive::GetComment(Array *CmtData) +{ + if (!MainComment) + return false; + SaveFilePos SavePos(*this); + +#ifndef SFX_MODULE + uint CmtLength; + if (Format==RARFMT14) + { + Seek(SFXSize+SIZEOF_MAINHEAD14,SEEK_SET); + CmtLength=GetByte(); + CmtLength+=(GetByte()<<8); + } + else +#endif + { + if (MainHead.CommentInHeader) + { + // Old style (RAR 2.9) archive comment embedded into the main + // archive header. + Seek(SFXSize+SIZEOF_MARKHEAD3+SIZEOF_MAINHEAD3,SEEK_SET); + ReadHeader(); + } + else + { + // Current (RAR 3.0+) version of archive comment. + Seek(GetStartPos(),SEEK_SET); + return SearchSubBlock(SUBHEAD_TYPE_CMT)!=0 && ReadCommentData(CmtData); + } +#ifndef SFX_MODULE + // Old style (RAR 2.9) comment header embedded into the main + // archive header. + if (BrokenHeader) + { + uiMsg(UIERROR_CMTBROKEN,FileName); + return false; + } + CmtLength=CommHead.HeadSize-SIZEOF_COMMHEAD; +#endif + } +#ifndef SFX_MODULE + if (Format==RARFMT14 && MainHead.PackComment || Format!=RARFMT14 && CommHead.Method!=0x30) + { + if (Format!=RARFMT14 && (CommHead.UnpVer < 15 || CommHead.UnpVer > VER_UNPACK || CommHead.Method > 0x35)) + return false; + ComprDataIO DataIO; + DataIO.SetTestMode(true); + uint UnpCmtLength; + if (Format==RARFMT14) + { +#ifdef RAR_NOCRYPT + return false; +#else + UnpCmtLength=GetByte(); + UnpCmtLength+=(GetByte()<<8); + CmtLength-=2; + DataIO.SetCmt13Encryption(); + CommHead.UnpVer=15; +#endif + } + else + UnpCmtLength=CommHead.UnpSize; + DataIO.SetFiles(this,NULL); + DataIO.EnableShowProgress(false); + DataIO.SetPackedSizeToRead(CmtLength); + DataIO.UnpHash.Init(HASH_CRC32,1); + + Unpack CmtUnpack(&DataIO); + CmtUnpack.Init(0x10000,false); + CmtUnpack.SetDestSize(UnpCmtLength); + CmtUnpack.DoUnpack(CommHead.UnpVer,false); + + if (Format!=RARFMT14 && (DataIO.UnpHash.GetCRC32()&0xffff)!=CommHead.CommCRC) + { + uiMsg(UIERROR_CMTBROKEN,FileName); + return false; + } + else + { + byte *UnpData; + size_t UnpDataSize; + DataIO.GetUnpackedData(&UnpData,&UnpDataSize); +#ifdef _WIN_ALL + // If we ever decide to extend it to Android, we'll need to alloc + // 4x memory for OEM to UTF-8 output here. + OemToCharBuffA((char *)UnpData,(char *)UnpData,(DWORD)UnpDataSize); +#endif + CmtData->Alloc(UnpDataSize+1); + memset(CmtData->Addr(0),0,CmtData->Size()*sizeof(wchar)); + CharToWide((char *)UnpData,CmtData->Addr(0),CmtData->Size()); + CmtData->Alloc(wcslen(CmtData->Addr(0))); + } + } + else + { + if (CmtLength==0) + return false; + Array CmtRaw(CmtLength); + Read(&CmtRaw[0],CmtLength); + + if (Format!=RARFMT14 && CommHead.CommCRC!=(~CRC32(0xffffffff,&CmtRaw[0],CmtLength)&0xffff)) + { + uiMsg(UIERROR_CMTBROKEN,FileName); + return false; + } + CmtData->Alloc(CmtLength+1); + CmtRaw.Push(0); +#ifdef _WIN_ALL + // If we ever decide to extend it to Android, we'll need to alloc + // 4x memory for OEM to UTF-8 output here. + OemToCharA((char *)&CmtRaw[0],(char *)&CmtRaw[0]); +#endif + CharToWide((char *)&CmtRaw[0],CmtData->Addr(0),CmtLength); + CmtData->Alloc(wcslen(CmtData->Addr(0))); + } +#endif + return CmtData->Size() > 0; +} + + +bool Archive::ReadCommentData(Array *CmtData) +{ + Array CmtRaw; + if (!ReadSubData(&CmtRaw,NULL)) + return false; + size_t CmtSize=CmtRaw.Size(); + CmtRaw.Push(0); + CmtData->Alloc(CmtSize+1); + if (Format==RARFMT50) + UtfToWide((char *)&CmtRaw[0],CmtData->Addr(0),CmtData->Size()); + else + if ((SubHead.SubFlags & SUBHEAD_FLAGS_CMT_UNICODE)!=0) + { + RawToWide(&CmtRaw[0],CmtData->Addr(0),CmtSize/2); + (*CmtData)[CmtSize/2]=0; + + } + else + { + CharToWide((char *)&CmtRaw[0],CmtData->Addr(0),CmtData->Size()); + } + CmtData->Alloc(wcslen(CmtData->Addr(0))); // Set buffer size to actual comment length. + return true; +} + + +void Archive::ViewComment() +{ + if (Cmd->DisableComment) + return; + Array CmtBuf; + if (GetComment(&CmtBuf)) // In GUI too, so "Test" command detects broken comments. + { + size_t CmtSize=CmtBuf.Size(); + wchar *ChPtr=wcschr(&CmtBuf[0],0x1A); + if (ChPtr!=NULL) + CmtSize=ChPtr-&CmtBuf[0]; + mprintf(L"\n"); + OutComment(&CmtBuf[0],CmtSize); + } +} + + diff --git a/reference/unrar/archive.cpp b/reference/unrar/archive.cpp new file mode 100644 index 00000000..e66bddb9 --- /dev/null +++ b/reference/unrar/archive.cpp @@ -0,0 +1,332 @@ +#include "rar.hpp" + +#include "arccmt.cpp" + + +Archive::Archive(RAROptions *InitCmd) +{ + Cmd=NULL; // Just in case we'll have an exception in 'new' below. + + DummyCmd=(InitCmd==NULL); + Cmd=DummyCmd ? (new RAROptions):InitCmd; + + OpenShared=Cmd->OpenShared; + Format=RARFMT15; + Solid=false; + Volume=false; + MainComment=false; + Locked=false; + Signed=false; + FirstVolume=false; + NewNumbering=false; + SFXSize=0; + LatestTime.Reset(); + Protected=false; + Encrypted=false; + FailedHeaderDecryption=false; + BrokenHeader=false; + LastReadBlock=0; + + CurBlockPos=0; + NextBlockPos=0; + + RecoverySize=-1; + RecoveryPercent=-1; + + memset(&MainHead,0,sizeof(MainHead)); + memset(&CryptHead,0,sizeof(CryptHead)); + memset(&EndArcHead,0,sizeof(EndArcHead)); + + VolNumber=0; + VolWrite=0; + AddingFilesSize=0; + AddingHeadersSize=0; + *FirstVolumeName=0; + + Splitting=false; + NewArchive=false; + + SilentOpen=false; + +} + + +Archive::~Archive() +{ + if (DummyCmd) + delete Cmd; +} + + +void Archive::CheckArc(bool EnableBroken) +{ + if (!IsArchive(EnableBroken)) + { + // If FailedHeaderDecryption is set, we already reported that archive + // password is incorrect. + if (!FailedHeaderDecryption) + uiMsg(UIERROR_BADARCHIVE,FileName); + ErrHandler.Exit(RARX_FATAL); + } +} + + +#if !defined(SFX_MODULE) +void Archive::CheckOpen(const wchar *Name) +{ + TOpen(Name); + CheckArc(false); +} +#endif + + +bool Archive::WCheckOpen(const wchar *Name) +{ + if (!WOpen(Name)) + return false; + if (!IsArchive(false)) + { + uiMsg(UIERROR_BADARCHIVE,FileName); + Close(); + return false; + } + return true; +} + + +RARFORMAT Archive::IsSignature(const byte *D,size_t Size) +{ + RARFORMAT Type=RARFMT_NONE; + if (Size>=1 && D[0]==0x52) +#ifndef SFX_MODULE + if (Size>=4 && D[1]==0x45 && D[2]==0x7e && D[3]==0x5e) + Type=RARFMT14; + else +#endif + if (Size>=7 && D[1]==0x61 && D[2]==0x72 && D[3]==0x21 && D[4]==0x1a && D[5]==0x07) + { + // We check the last signature byte, so we can return a sensible + // warning in case we'll want to change the archive format + // sometimes in the future. + if (D[6]==0) + Type=RARFMT15; + else + if (D[6]==1) + Type=RARFMT50; + else + if (D[6]==2) + Type=RARFMT_FUTURE; + } + return Type; +} + + +bool Archive::IsArchive(bool EnableBroken) +{ + Encrypted=false; + BrokenHeader=false; // Might be left from previous volume. + +#ifndef SFX_MODULE + if (IsDevice()) + { + uiMsg(UIERROR_INVALIDNAME,FileName,FileName); + return false; + } +#endif + if (Read(MarkHead.Mark,SIZEOF_MARKHEAD3)!=SIZEOF_MARKHEAD3) + return false; + SFXSize=0; + + RARFORMAT Type; + if ((Type=IsSignature(MarkHead.Mark,SIZEOF_MARKHEAD3))!=RARFMT_NONE) + { + Format=Type; + if (Format==RARFMT14) + Seek(Tell()-SIZEOF_MARKHEAD3,SEEK_SET); + } + else + { + Array Buffer(MAXSFXSIZE); + long CurPos=(long)Tell(); + int ReadSize=Read(&Buffer[0],Buffer.Size()-16); + for (int I=0;I0 && CurPos<28 && ReadSize>31) + { + char *D=&Buffer[28-CurPos]; + if (D[0]!=0x52 || D[1]!=0x53 || D[2]!=0x46 || D[3]!=0x58) + continue; + } + SFXSize=CurPos+I; + Seek(SFXSize,SEEK_SET); + if (Format==RARFMT15 || Format==RARFMT50) + Read(MarkHead.Mark,SIZEOF_MARKHEAD3); + break; + } + if (SFXSize==0) + return false; + } + if (Format==RARFMT_FUTURE) + { + uiMsg(UIERROR_NEWRARFORMAT,FileName); + return false; + } + if (Format==RARFMT50) // RAR 5.0 signature is by one byte longer. + { + Read(MarkHead.Mark+SIZEOF_MARKHEAD3,1); + if (MarkHead.Mark[SIZEOF_MARKHEAD3]!=0) + return false; + MarkHead.HeadSize=SIZEOF_MARKHEAD5; + } + else + MarkHead.HeadSize=SIZEOF_MARKHEAD3; + +#ifdef RARDLL + // If callback function is not set, we cannot get the password, + // so we skip the initial header processing for encrypted header archive. + // It leads to skipped archive comment, but the rest of archive data + // is processed correctly. + if (Cmd->Callback==NULL) + SilentOpen=true; +#endif + + // Skip the archive encryption header if any and read the main header. + while (ReadHeader()!=0) + { + HEADER_TYPE Type=GetHeaderType(); + // In RAR 5.0 we need to quit after reading HEAD_CRYPT if we wish to + // avoid the password prompt. + if (Type==HEAD_MAIN || SilentOpen && Type==HEAD_CRYPT) + break; + SeekToNext(); + } + + // This check allows to make RS based recovery even if password is incorrect. + // But we should not do it for EnableBroken or we'll get 'not RAR archive' + // messages when extracting encrypted archives with wrong password. + if (FailedHeaderDecryption && !EnableBroken) + return false; + + SeekToNext(); + if (BrokenHeader) // Main archive header is corrupt. + { + uiMsg(UIERROR_MHEADERBROKEN,FileName); + if (!EnableBroken) + return false; + } + + MainComment=MainHead.CommentInHeader; + + // If we process non-encrypted archive or can request a password, + // we set 'first volume' flag based on file attributes below. + // It is necessary for RAR 2.x archives, which did not have 'first volume' + // flag in main header. Also for all RAR formats we need to scan until + // first file header to set "comment" flag when reading service header. + // Unless we are in silent mode, we need to know about presence of comment + // immediately after IsArchive call. + if (!SilentOpen || !Encrypted) + { + SaveFilePos SavePos(*this); + int64 SaveCurBlockPos=CurBlockPos,SaveNextBlockPos=NextBlockPos; + HEADER_TYPE SaveCurHeaderType=CurHeaderType; + + while (ReadHeader()!=0) + { + HEADER_TYPE HeaderType=GetHeaderType(); + if (HeaderType==HEAD_SERVICE) + { + // If we have a split service headers, it surely indicates non-first + // volume. But not split service header does not guarantee the first + // volume, because we can have split file after non-split archive + // comment. So we do not quit from loop here. + FirstVolume=Volume && !SubHead.SplitBefore; + } + else + if (HeaderType==HEAD_FILE) + { + FirstVolume=Volume && !FileHead.SplitBefore; + break; + } + else + if (HeaderType==HEAD_ENDARC) // Might happen if archive contains only a split service header. + break; + SeekToNext(); + } + CurBlockPos=SaveCurBlockPos; + NextBlockPos=SaveNextBlockPos; + CurHeaderType=SaveCurHeaderType; + } + if (!Volume || FirstVolume) + wcscpy(FirstVolumeName,FileName); + + return true; +} + + + + +void Archive::SeekToNext() +{ + Seek(NextBlockPos,SEEK_SET); +} + + + + + + +// Calculate the block size including encryption fields and padding if any. +uint Archive::FullHeaderSize(size_t Size) +{ + if (Encrypted) + { + Size = ALIGN_VALUE(Size, CRYPT_BLOCK_SIZE); // Align to encryption block size. + if (Format == RARFMT50) + Size += SIZE_INITV; + else + Size += SIZE_SALT30; + } + return uint(Size); +} + + + + +#ifdef USE_QOPEN +bool Archive::Open(const wchar *Name,uint Mode) +{ + // Important if we reuse Archive object and it has virtual QOpen + // file position not matching real. For example, for 'l -v volname'. + QOpen.Unload(); + + return File::Open(Name,Mode); +} + + +int Archive::Read(void *Data,size_t Size) +{ + size_t Result; + if (QOpen.Read(Data,Size,Result)) + return (int)Result; + return File::Read(Data,Size); +} + + +void Archive::Seek(int64 Offset,int Method) +{ + if (!QOpen.Seek(Offset,Method)) + File::Seek(Offset,Method); +} + + +int64 Archive::Tell() +{ + int64 QPos; + if (QOpen.Tell(&QPos)) + return QPos; + return File::Tell(); +} +#endif + diff --git a/reference/unrar/archive.hpp b/reference/unrar/archive.hpp new file mode 100644 index 00000000..6e85b9ff --- /dev/null +++ b/reference/unrar/archive.hpp @@ -0,0 +1,151 @@ +#ifndef _RAR_ARCHIVE_ +#define _RAR_ARCHIVE_ + +class PPack; +class RawRead; +class RawWrite; + +enum NOMODIFY_FLAGS +{ + NMDF_ALLOWLOCK=1,NMDF_ALLOWANYVOLUME=2,NMDF_ALLOWFIRSTVOLUME=4 +}; + +enum RARFORMAT {RARFMT_NONE,RARFMT14,RARFMT15,RARFMT50,RARFMT_FUTURE}; + +enum ADDSUBDATA_FLAGS +{ + ASDF_SPLIT = 1, // Allow to split archive just before header if necessary. + ASDF_COMPRESS = 2, // Allow to compress data following subheader. + ASDF_CRYPT = 4, // Encrypt data after subheader if password is set. + ASDF_CRYPTIFHEADERS = 8 // Encrypt data after subheader only in -hp mode. +}; + +class Archive:public File +{ + private: + void UpdateLatestTime(FileHeader *CurBlock); + void ConvertNameCase(wchar *Name); + void ConvertFileHeader(FileHeader *hd); + void WriteBlock50(HEADER_TYPE HeaderType,BaseBlock *wb,bool OnlySetSize,bool NonFinalWrite); + size_t ReadHeader14(); + size_t ReadHeader15(); + size_t ReadHeader50(); + void ProcessExtra50(RawRead *Raw,size_t ExtraSize,BaseBlock *bb); + void RequestArcPassword(); + void UnexpEndArcMsg(); + void BrokenHeaderMsg(); + void UnkEncVerMsg(const wchar *Name); + void UnkEncVerMsg(); + bool ReadCommentData(Array *CmtData); + +#if !defined(RAR_NOCRYPT) + CryptData HeadersCrypt; +#endif + ComprDataIO SubDataIO; + bool DummyCmd; + RAROptions *Cmd; + + int64 RecoverySize; + int RecoveryPercent; + + RarTime LatestTime; + int LastReadBlock; + HEADER_TYPE CurHeaderType; + + bool SilentOpen; +#ifdef USE_QOPEN + QuickOpen QOpen; +#endif + public: + Archive(RAROptions *InitCmd=NULL); + ~Archive(); + static RARFORMAT IsSignature(const byte *D,size_t Size); + bool IsArchive(bool EnableBroken); + size_t SearchBlock(HEADER_TYPE HeaderType); + size_t SearchSubBlock(const wchar *Type); + size_t SearchRR(); + void WriteBlock(HEADER_TYPE HeaderType,BaseBlock *wb=NULL,bool OnlySetSize=false,bool NonFinalWrite=false); + void SetBlockSize(HEADER_TYPE HeaderType,BaseBlock *wb=NULL) {WriteBlock(HeaderType,wb,true);} + size_t ReadHeader(); + void CheckArc(bool EnableBroken); + void CheckOpen(const wchar *Name); + bool WCheckOpen(const wchar *Name); + bool GetComment(Array *CmtData); + void ViewComment(); + void SetLatestTime(RarTime *NewTime); + void SeekToNext(); + bool CheckAccess(); + bool IsArcDir(); + void ConvertAttributes(); + void VolSubtractHeaderSize(size_t SubSize); + uint FullHeaderSize(size_t Size); + int64 GetStartPos(); + void AddSubData(byte *SrcData,uint64 DataSize,File *SrcFile, + const wchar *Name,uint Flags); + bool ReadSubData(Array *UnpData,File *DestFile); + HEADER_TYPE GetHeaderType() {return CurHeaderType;}; + RAROptions* GetRAROptions() {return Cmd;} + void SetSilentOpen(bool Mode) {SilentOpen=Mode;} +#if 0 + void GetRecoveryInfo(bool Required,int64 *Size,int *Percent); +#endif +#ifdef USE_QOPEN + bool Open(const wchar *Name,uint Mode=FMF_READ); + int Read(void *Data,size_t Size); + void Seek(int64 Offset,int Method); + int64 Tell(); + void QOpenUnload() {QOpen.Unload();} +#endif + + BaseBlock ShortBlock; + MarkHeader MarkHead; + MainHeader MainHead; + CryptHeader CryptHead; + FileHeader FileHead; + EndArcHeader EndArcHead; + SubBlockHeader SubBlockHead; + FileHeader SubHead; + CommentHeader CommHead; + ProtectHeader ProtectHead; + AVHeader AVHead; + SignHeader SignHead; + UnixOwnersHeader UOHead; + MacFInfoHeader MACHead; + EAHeader EAHead; + StreamHeader StreamHead; + + int64 CurBlockPos; + int64 NextBlockPos; + + RARFORMAT Format; + bool Solid; + bool Volume; + bool MainComment; + bool Locked; + bool Signed; + bool FirstVolume; + bool NewNumbering; + bool Protected; + bool Encrypted; + size_t SFXSize; + bool BrokenHeader; + bool FailedHeaderDecryption; + +#if !defined(RAR_NOCRYPT) + byte ArcSalt[SIZE_SALT50]; +#endif + + bool Splitting; + + uint VolNumber; + int64 VolWrite; + uint64 AddingFilesSize; + uint64 AddingHeadersSize; + + bool NewArchive; + + wchar FirstVolumeName[NM]; +}; + + +#endif diff --git a/reference/unrar/arcread.cpp b/reference/unrar/arcread.cpp new file mode 100644 index 00000000..1120f682 --- /dev/null +++ b/reference/unrar/arcread.cpp @@ -0,0 +1,1437 @@ +#include "rar.hpp" + +size_t Archive::ReadHeader() +{ + // Once we failed to decrypt an encrypted block, there is no reason to + // attempt to do it further. We'll never be successful and only generate + // endless errors. + if (FailedHeaderDecryption) + return 0; + + CurBlockPos=Tell(); + + size_t ReadSize; + switch(Format) + { +#ifndef SFX_MODULE + case RARFMT14: + ReadSize=ReadHeader14(); + break; +#endif + case RARFMT15: + ReadSize=ReadHeader15(); + break; + case RARFMT50: + ReadSize=ReadHeader50(); + break; + } + + if (ReadSize>0 && NextBlockPos<=CurBlockPos) + { + BrokenHeaderMsg(); + return 0; + } + return ReadSize; +} + + +size_t Archive::SearchBlock(HEADER_TYPE HeaderType) +{ + size_t Size,Count=0; + while ((Size=ReadHeader())!=0 && + (HeaderType==HEAD_ENDARC || GetHeaderType()!=HEAD_ENDARC)) + { + if ((++Count & 127)==0) + Wait(); + if (GetHeaderType()==HeaderType) + return Size; + SeekToNext(); + } + return 0; +} + + +size_t Archive::SearchSubBlock(const wchar *Type) +{ + size_t Size,Count=0; + while ((Size=ReadHeader())!=0 && GetHeaderType()!=HEAD_ENDARC) + { + if ((++Count & 127)==0) + Wait(); + if (GetHeaderType()==HEAD_SERVICE && SubHead.CmpName(Type)) + return Size; + SeekToNext(); + } + return 0; +} + + +size_t Archive::SearchRR() +{ + // If locator extra field is available for recovery record, let's utilize it. + if (MainHead.Locator && MainHead.RROffset!=0) + { + uint64 CurPos=Tell(); + Seek(MainHead.RROffset,SEEK_SET); + size_t Size=ReadHeader(); + if (Size!=0 && !BrokenHeader && GetHeaderType()==HEAD_SERVICE && SubHead.CmpName(SUBHEAD_TYPE_RR)) + return Size; + Seek(CurPos,SEEK_SET); + } + // Otherwise scan the entire archive to find the recovery record. + return SearchSubBlock(SUBHEAD_TYPE_RR); +} + + +void Archive::UnexpEndArcMsg() +{ + int64 ArcSize=FileLength(); + + // If block positions are equal to file size, this is not an error. + // It can happen when we reached the end of older RAR 1.5 archive, + // which did not have the end of archive block. + if (CurBlockPos!=ArcSize || NextBlockPos!=ArcSize) + { + uiMsg(UIERROR_UNEXPEOF,FileName); + ErrHandler.SetErrorCode(RARX_WARNING); + } +} + + +void Archive::BrokenHeaderMsg() +{ + uiMsg(UIERROR_HEADERBROKEN,FileName); + BrokenHeader=true; + ErrHandler.SetErrorCode(RARX_CRC); +} + + +void Archive::UnkEncVerMsg(const wchar *Name) +{ + uiMsg(UIERROR_UNKNOWNENCMETHOD,FileName,Name); + ErrHandler.SetErrorCode(RARX_WARNING); +} + + +size_t Archive::ReadHeader15() +{ + RawRead Raw(this); + + bool Decrypt=Encrypted && CurBlockPos>(int64)SFXSize+SIZEOF_MARKHEAD3; + + if (Decrypt) + { +#ifdef RAR_NOCRYPT // For rarext.dll and unrar_nocrypt.dll. + return 0; +#else + RequestArcPassword(); + + byte Salt[SIZE_SALT30]; + if (Read(Salt,SIZE_SALT30)!=SIZE_SALT30) + { + UnexpEndArcMsg(); + return 0; + } + HeadersCrypt.SetCryptKeys(false,CRYPT_RAR30,&Cmd->Password,Salt,NULL,0,NULL,NULL); + Raw.SetCrypt(&HeadersCrypt); +#endif + } + + Raw.Read(SIZEOF_SHORTBLOCKHEAD); + if (Raw.Size()==0) + { + UnexpEndArcMsg(); + return 0; + } + + ShortBlock.HeadCRC=Raw.Get2(); + + ShortBlock.Reset(); + + uint HeaderType=Raw.Get1(); + ShortBlock.Flags=Raw.Get2(); + ShortBlock.SkipIfUnknown=(ShortBlock.Flags & SKIP_IF_UNKNOWN)!=0; + ShortBlock.HeadSize=Raw.Get2(); + + ShortBlock.HeaderType=(HEADER_TYPE)HeaderType; + if (ShortBlock.HeadSizeReset(); + + *(BaseBlock *)hd=ShortBlock; + + hd->SplitBefore=(hd->Flags & LHD_SPLIT_BEFORE)!=0; + hd->SplitAfter=(hd->Flags & LHD_SPLIT_AFTER)!=0; + hd->Encrypted=(hd->Flags & LHD_PASSWORD)!=0; + hd->SaltSet=(hd->Flags & LHD_SALT)!=0; + hd->Solid=FileBlock && (hd->Flags & LHD_SOLID)!=0; + hd->SubBlock=!FileBlock && (hd->Flags & LHD_SOLID)!=0; + hd->Dir=(hd->Flags & LHD_WINDOWMASK)==LHD_DIRECTORY; + hd->WinSize=hd->Dir ? 0:0x10000<<((hd->Flags & LHD_WINDOWMASK)>>5); + hd->CommentInHeader=(hd->Flags & LHD_COMMENT)!=0; + hd->Version=(hd->Flags & LHD_VERSION)!=0; + + hd->DataSize=Raw.Get4(); + uint LowUnpSize=Raw.Get4(); + hd->HostOS=Raw.Get1(); + + hd->FileHash.Type=HASH_CRC32; + hd->FileHash.CRC32=Raw.Get4(); + + uint FileTime=Raw.Get4(); + hd->UnpVer=Raw.Get1(); + hd->Method=Raw.Get1()-0x30; + size_t NameSize=Raw.Get2(); + hd->FileAttr=Raw.Get4(); + + hd->CryptMethod=CRYPT_NONE; + if (hd->Encrypted) + switch(hd->UnpVer) + { + case 13: hd->CryptMethod=CRYPT_RAR13; break; + case 15: hd->CryptMethod=CRYPT_RAR15; break; + case 20: + case 26: hd->CryptMethod=CRYPT_RAR20; break; + default: hd->CryptMethod=CRYPT_RAR30; break; + } + + hd->HSType=HSYS_UNKNOWN; + if (hd->HostOS==HOST_UNIX || hd->HostOS==HOST_BEOS) + hd->HSType=HSYS_UNIX; + else + if (hd->HostOSHSType=HSYS_WINDOWS; + + hd->RedirType=FSREDIR_NONE; + + // RAR 4.x Unix symlink. + if (hd->HostOS==HOST_UNIX && (hd->FileAttr & 0xF000)==0xA000) + { + hd->RedirType=FSREDIR_UNIXSYMLINK; + *hd->RedirName=0; + } + + hd->Inherited=!FileBlock && (hd->SubFlags & SUBHEAD_FLAGS_INHERITED)!=0; + + hd->LargeFile=(hd->Flags & LHD_LARGE)!=0; + + uint HighPackSize,HighUnpSize; + if (hd->LargeFile) + { + HighPackSize=Raw.Get4(); + HighUnpSize=Raw.Get4(); + hd->UnknownUnpSize=(LowUnpSize==0xffffffff && HighUnpSize==0xffffffff); + } + else + { + HighPackSize=HighUnpSize=0; + // UnpSize equal to 0xffffffff without LHD_LARGE flag indicates + // that we do not know the unpacked file size and must unpack it + // until we find the end of file marker in compressed data. + hd->UnknownUnpSize=(LowUnpSize==0xffffffff); + } + hd->PackSize=INT32TO64(HighPackSize,hd->DataSize); + hd->UnpSize=INT32TO64(HighUnpSize,LowUnpSize); + if (hd->UnknownUnpSize) + hd->UnpSize=INT64NDF; + + char FileName[NM*4]; + size_t ReadNameSize=Min(NameSize,ASIZE(FileName)-1); + Raw.GetB((byte *)FileName,ReadNameSize); + FileName[ReadNameSize]=0; + + if (FileBlock) + { + *hd->FileName=0; + if ((hd->Flags & LHD_UNICODE)!=0) + { + EncodeFileName NameCoder; + size_t Length=strlen(FileName); + Length++; + if (ReadNameSize>Length) + NameCoder.Decode(FileName,(byte *)FileName+Length, + ReadNameSize-Length,hd->FileName, + ASIZE(hd->FileName)); + } + + if (*hd->FileName==0) + ArcCharToWide(FileName,hd->FileName,ASIZE(hd->FileName),ACTW_OEM); + +#ifndef SFX_MODULE + ConvertNameCase(hd->FileName); +#endif + ConvertFileHeader(hd); + } + else + { + CharToWide(FileName,hd->FileName,ASIZE(hd->FileName)); + + // Calculate the size of optional data. + int DataSize=int(hd->HeadSize-NameSize-SIZEOF_FILEHEAD3); + if ((hd->Flags & LHD_SALT)!=0) + DataSize-=SIZE_SALT30; + + if (DataSize>0) + { + // Here we read optional additional fields for subheaders. + // They are stored after the file name and before salt. + hd->SubData.Alloc(DataSize); + Raw.GetB(&hd->SubData[0],DataSize); + if (hd->CmpName(SUBHEAD_TYPE_RR)) + { + byte *D=&hd->SubData[8]; + RecoverySize=D[0]+((uint)D[1]<<8)+((uint)D[2]<<16)+((uint)D[3]<<24); + RecoverySize*=512; // Sectors to size. + int64 CurPos=Tell(); + RecoveryPercent=ToPercent(RecoverySize,CurPos); + // Round fractional percent exceeding .5 to upper value. + if (ToPercent(RecoverySize+CurPos/200,CurPos)>RecoveryPercent) + RecoveryPercent++; + } + } + + if (hd->CmpName(SUBHEAD_TYPE_CMT)) + MainComment=true; + } + if ((hd->Flags & LHD_SALT)!=0) + Raw.GetB(hd->Salt,SIZE_SALT30); + hd->mtime.SetDos(FileTime); + if ((hd->Flags & LHD_EXTTIME)!=0) + { + ushort Flags=Raw.Get2(); + RarTime *tbl[4]; + tbl[0]=&FileHead.mtime; + tbl[1]=&FileHead.ctime; + tbl[2]=&FileHead.atime; + tbl[3]=NULL; // Archive time is not used now. + for (int I=0;I<4;I++) + { + RarTime *CurTime=tbl[I]; + uint rmode=Flags>>(3-I)*4; + if ((rmode & 8)==0 || CurTime==NULL) + continue; + if (I!=0) + { + uint DosTime=Raw.Get4(); + CurTime->SetDos(DosTime); + } + RarLocalTime rlt; + CurTime->GetLocal(&rlt); + if (rmode & 4) + rlt.Second++; + rlt.Reminder=0; + int count=rmode&3; + for (int J=0;JSetLocal(&rlt); + } + } + NextBlockPos+=hd->PackSize; + bool CRCProcessedOnly=hd->CommentInHeader; + ushort HeaderCRC=Raw.GetCRC15(CRCProcessedOnly); + if (hd->HeadCRC!=HeaderCRC) + { + BrokenHeader=true; + ErrHandler.SetErrorCode(RARX_WARNING); + + // If we have a broken encrypted header, we do not need to display + // the error message here, because it will be displayed for such + // headers later in this function. Also such headers are unlikely + // to have anything sensible in file name field, so it is useless + // to display the file name. + if (!Decrypt) + uiMsg(UIERROR_FHEADERBROKEN,Archive::FileName,hd->FileName); + } + } + break; + case HEAD_ENDARC: + *(BaseBlock *)&EndArcHead=ShortBlock; + EndArcHead.NextVolume=(EndArcHead.Flags & EARC_NEXT_VOLUME)!=0; + EndArcHead.DataCRC=(EndArcHead.Flags & EARC_DATACRC)!=0; + EndArcHead.RevSpace=(EndArcHead.Flags & EARC_REVSPACE)!=0; + EndArcHead.StoreVolNumber=(EndArcHead.Flags & EARC_VOLNUMBER)!=0; + if (EndArcHead.DataCRC) + EndArcHead.ArcDataCRC=Raw.Get4(); + if (EndArcHead.StoreVolNumber) + VolNumber=EndArcHead.VolNumber=Raw.Get2(); + break; +#ifndef SFX_MODULE + case HEAD3_CMT: + *(BaseBlock *)&CommHead=ShortBlock; + CommHead.UnpSize=Raw.Get2(); + CommHead.UnpVer=Raw.Get1(); + CommHead.Method=Raw.Get1(); + CommHead.CommCRC=Raw.Get2(); + break; + case HEAD3_SIGN: + *(BaseBlock *)&SignHead=ShortBlock; + SignHead.CreationTime=Raw.Get4(); + SignHead.ArcNameSize=Raw.Get2(); + SignHead.UserNameSize=Raw.Get2(); + break; + case HEAD3_AV: + *(BaseBlock *)&AVHead=ShortBlock; + AVHead.UnpVer=Raw.Get1(); + AVHead.Method=Raw.Get1(); + AVHead.AVVer=Raw.Get1(); + AVHead.AVInfoCRC=Raw.Get4(); + break; + case HEAD3_PROTECT: + *(BaseBlock *)&ProtectHead=ShortBlock; + ProtectHead.DataSize=Raw.Get4(); + ProtectHead.Version=Raw.Get1(); + ProtectHead.RecSectors=Raw.Get2(); + ProtectHead.TotalBlocks=Raw.Get4(); + Raw.GetB(ProtectHead.Mark,8); + NextBlockPos+=ProtectHead.DataSize; + RecoverySize=ProtectHead.RecSectors*512; + break; + case HEAD3_OLDSERVICE: + *(BaseBlock *)&SubBlockHead=ShortBlock; + SubBlockHead.DataSize=Raw.Get4(); + NextBlockPos+=SubBlockHead.DataSize; + SubBlockHead.SubType=Raw.Get2(); + SubBlockHead.Level=Raw.Get1(); + switch(SubBlockHead.SubType) + { + case UO_HEAD: + *(SubBlockHeader *)&UOHead=SubBlockHead; + UOHead.OwnerNameSize=Raw.Get2(); + UOHead.GroupNameSize=Raw.Get2(); + if (UOHead.OwnerNameSize>=ASIZE(UOHead.OwnerName)) + UOHead.OwnerNameSize=ASIZE(UOHead.OwnerName)-1; + if (UOHead.GroupNameSize>=ASIZE(UOHead.GroupName)) + UOHead.GroupNameSize=ASIZE(UOHead.GroupName)-1; + Raw.GetB(UOHead.OwnerName,UOHead.OwnerNameSize); + Raw.GetB(UOHead.GroupName,UOHead.GroupNameSize); + UOHead.OwnerName[UOHead.OwnerNameSize]=0; + UOHead.GroupName[UOHead.GroupNameSize]=0; + break; + case MAC_HEAD: + *(SubBlockHeader *)&MACHead=SubBlockHead; + MACHead.fileType=Raw.Get4(); + MACHead.fileCreator=Raw.Get4(); + break; + case EA_HEAD: + case BEEA_HEAD: + case NTACL_HEAD: + *(SubBlockHeader *)&EAHead=SubBlockHead; + EAHead.UnpSize=Raw.Get4(); + EAHead.UnpVer=Raw.Get1(); + EAHead.Method=Raw.Get1(); + EAHead.EACRC=Raw.Get4(); + break; + case STREAM_HEAD: + *(SubBlockHeader *)&StreamHead=SubBlockHead; + StreamHead.UnpSize=Raw.Get4(); + StreamHead.UnpVer=Raw.Get1(); + StreamHead.Method=Raw.Get1(); + StreamHead.StreamCRC=Raw.Get4(); + StreamHead.StreamNameSize=Raw.Get2(); + if (StreamHead.StreamNameSize>=ASIZE(StreamHead.StreamName)) + StreamHead.StreamNameSize=ASIZE(StreamHead.StreamName)-1; + Raw.GetB(StreamHead.StreamName,StreamHead.StreamNameSize); + StreamHead.StreamName[StreamHead.StreamNameSize]=0; + break; + } + break; +#endif + default: + if (ShortBlock.Flags & LONG_BLOCK) + NextBlockPos+=Raw.Get4(); + break; + } + + ushort HeaderCRC=Raw.GetCRC15(false); + + // Old AV header does not have header CRC properly set. + if (ShortBlock.HeadCRC!=HeaderCRC && ShortBlock.HeaderType!=HEAD3_SIGN && + ShortBlock.HeaderType!=HEAD3_AV) + { + bool Recovered=false; + if (ShortBlock.HeaderType==HEAD_ENDARC && EndArcHead.RevSpace) + { + // Last 7 bytes of recovered volume can contain zeroes, because + // REV files store its own information (volume number, etc.) here. + SaveFilePos SavePos(*this); + int64 Length=Tell(); + Seek(Length-7,SEEK_SET); + Recovered=true; + for (int J=0;J<7;J++) + if (GetByte()!=0) + Recovered=false; + } + if (!Recovered) + { + BrokenHeader=true; + ErrHandler.SetErrorCode(RARX_CRC); + + if (Decrypt) + { + uiMsg(UIERROR_CHECKSUMENC,FileName,FileName); + FailedHeaderDecryption=true; + return 0; + } + } + } + + if (NextBlockPos<=CurBlockPos) + { + BrokenHeaderMsg(); + return 0; + } + + return Raw.Size(); +} + + +size_t Archive::ReadHeader50() +{ + RawRead Raw(this); + + bool Decrypt=Encrypted && CurBlockPos>(int64)SFXSize+SIZEOF_MARKHEAD5; + + if (Decrypt) + { +#if defined(RAR_NOCRYPT) + return 0; +#else + RequestArcPassword(); + + byte HeadersInitV[SIZE_INITV]; + if (Read(HeadersInitV,SIZE_INITV)!=SIZE_INITV) + { + UnexpEndArcMsg(); + return 0; + } + + byte PswCheck[SIZE_PSWCHECK]; + HeadersCrypt.SetCryptKeys(false,CRYPT_RAR50,&Cmd->Password,CryptHead.Salt,HeadersInitV,CryptHead.Lg2Count,NULL,PswCheck); + // Verify password validity. + if (CryptHead.UsePswCheck && memcmp(PswCheck,CryptHead.PswCheck,SIZE_PSWCHECK)!=0) + { + uiMsg(UIERROR_BADPSW,FileName); + FailedHeaderDecryption=true; + ErrHandler.SetErrorCode(RARX_BADPWD); + return 0; + } + + Raw.SetCrypt(&HeadersCrypt); +#endif + } + + // Header size must not occupy more than 3 variable length integer bytes + // resulting in 2 MB maximum header size, so here we read 4 byte CRC32 + // followed by 3 bytes or less of header size. + const size_t FirstReadSize=7; // Smallest possible block size. + if (Raw.Read(FirstReadSize)=ShortBlock.HeadSize) + { + BrokenHeaderMsg(); + return 0; + } + } + + uint64 DataSize=0; + if ((ShortBlock.Flags & HFL_DATA)!=0) + DataSize=Raw.GetV(); + + NextBlockPos=CurBlockPos+FullHeaderSize(ShortBlock.HeadSize)+DataSize; + + switch(ShortBlock.HeaderType) + { + case HEAD_CRYPT: + { + *(BaseBlock *)&CryptHead=ShortBlock; + uint CryptVersion=(uint)Raw.GetV(); + if (CryptVersion>CRYPT_VERSION) + { + UnkEncVerMsg(FileName); + return 0; + } + uint EncFlags=(uint)Raw.GetV(); + CryptHead.UsePswCheck=(EncFlags & CHFL_CRYPT_PSWCHECK)!=0; + CryptHead.Lg2Count=Raw.Get1(); + if (CryptHead.Lg2Count>CRYPT5_KDF_LG2_COUNT_MAX) + { + UnkEncVerMsg(FileName); + return 0; + } + Raw.GetB(CryptHead.Salt,SIZE_SALT50); + if (CryptHead.UsePswCheck) + { + Raw.GetB(CryptHead.PswCheck,SIZE_PSWCHECK); + + byte csum[SIZE_PSWCHECK_CSUM]; + Raw.GetB(csum,SIZE_PSWCHECK_CSUM); + + sha256_context ctx; + sha256_init(&ctx); + sha256_process(&ctx, CryptHead.PswCheck, SIZE_PSWCHECK); + + byte Digest[SHA256_DIGEST_SIZE]; + sha256_done(&ctx, Digest); + + CryptHead.UsePswCheck=memcmp(csum,Digest,SIZE_PSWCHECK_CSUM)==0; + } + Encrypted=true; + } + break; + case HEAD_MAIN: + { + MainHead.Reset(); + *(BaseBlock *)&MainHead=ShortBlock; + uint ArcFlags=(uint)Raw.GetV(); + + Volume=(ArcFlags & MHFL_VOLUME)!=0; + Solid=(ArcFlags & MHFL_SOLID)!=0; + Locked=(ArcFlags & MHFL_LOCK)!=0; + Protected=(ArcFlags & MHFL_PROTECT)!=0; + Signed=false; + NewNumbering=true; + + if ((ArcFlags & MHFL_VOLNUMBER)!=0) + VolNumber=(uint)Raw.GetV(); + else + VolNumber=0; + FirstVolume=Volume && VolNumber==0; + + if (ExtraSize!=0) + ProcessExtra50(&Raw,(size_t)ExtraSize,&MainHead); + +#ifdef USE_QOPEN + if (MainHead.Locator && MainHead.QOpenOffset>0 && Cmd->QOpenMode!=QOPEN_NONE) + { + // We seek to QO block in the end of archive when processing + // QOpen.Load, so we need to preserve current block positions + // to not break normal archive processing by calling function. + int64 SaveCurBlockPos=CurBlockPos,SaveNextBlockPos=NextBlockPos; + HEADER_TYPE SaveCurHeaderType=CurHeaderType; + + QOpen.Init(this,false); + QOpen.Load(MainHead.QOpenOffset); + + CurBlockPos=SaveCurBlockPos; + NextBlockPos=SaveNextBlockPos; + CurHeaderType=SaveCurHeaderType; + } +#endif + } + break; + case HEAD_FILE: + case HEAD_SERVICE: + { + FileHeader *hd=ShortBlock.HeaderType==HEAD_FILE ? &FileHead:&SubHead; + hd->Reset(); + *(BaseBlock *)hd=ShortBlock; + + bool FileBlock=ShortBlock.HeaderType==HEAD_FILE; + + hd->LargeFile=true; + + hd->PackSize=DataSize; + hd->FileFlags=(uint)Raw.GetV(); + hd->UnpSize=Raw.GetV(); + + hd->UnknownUnpSize=(hd->FileFlags & FHFL_UNPUNKNOWN)!=0; + if (hd->UnknownUnpSize) + hd->UnpSize=INT64NDF; + + hd->MaxSize=Max(hd->PackSize,hd->UnpSize); + hd->FileAttr=(uint)Raw.GetV(); + if ((hd->FileFlags & FHFL_UTIME)!=0) + hd->mtime.SetUnix((time_t)Raw.Get4()); + + hd->FileHash.Type=HASH_NONE; + if ((hd->FileFlags & FHFL_CRC32)!=0) + { + hd->FileHash.Type=HASH_CRC32; + hd->FileHash.CRC32=Raw.Get4(); + } + + hd->RedirType=FSREDIR_NONE; + + uint CompInfo=(uint)Raw.GetV(); + hd->Method=(CompInfo>>7) & 7; + + // "+ 50" to not mix with old RAR format algorithms. For example, + // we may need to use the compression algorithm 15 in the future, + // but it was already used in RAR 1.5 and Unpack needs to distinguish + // them. + hd->UnpVer=(CompInfo & 0x3f) + 50; + + hd->HostOS=(byte)Raw.GetV(); + size_t NameSize=(size_t)Raw.GetV(); + hd->Inherited=(ShortBlock.Flags & HFL_INHERITED)!=0; + + hd->HSType=HSYS_UNKNOWN; + if (hd->HostOS==HOST5_UNIX) + hd->HSType=HSYS_UNIX; + else + if (hd->HostOS==HOST5_WINDOWS) + hd->HSType=HSYS_WINDOWS; + + hd->SplitBefore=(hd->Flags & HFL_SPLITBEFORE)!=0; + hd->SplitAfter=(hd->Flags & HFL_SPLITAFTER)!=0; + hd->SubBlock=(hd->Flags & HFL_CHILD)!=0; + hd->Solid=FileBlock && (CompInfo & FCI_SOLID)!=0; + hd->Dir=(hd->FileFlags & FHFL_DIRECTORY)!=0; + hd->WinSize=hd->Dir ? 0:size_t(0x20000)<<((CompInfo>>10)&0xf); + + hd->CryptMethod=hd->Encrypted ? CRYPT_RAR50:CRYPT_NONE; + + char FileName[NM*4]; + size_t ReadNameSize=Min(NameSize,ASIZE(FileName)-1); + Raw.GetB((byte *)FileName,ReadNameSize); + FileName[ReadNameSize]=0; + + UtfToWide(FileName,hd->FileName,ASIZE(hd->FileName)); + + // Should do it before converting names, because extra fields can + // affect name processing, like in case of NTFS streams. + if (ExtraSize!=0) + ProcessExtra50(&Raw,(size_t)ExtraSize,hd); + + if (FileBlock) + { +#ifndef SFX_MODULE + ConvertNameCase(hd->FileName); +#endif + ConvertFileHeader(hd); + } + + if (!FileBlock && hd->CmpName(SUBHEAD_TYPE_CMT)) + MainComment=true; + +#if 0 + // For RAR5 format we read the user specified recovery percent here. + // It would be useful to do it for shell extension too, so we display + // the correct recovery record size in archive properties. But then + // we would need to include the entire recovery record processing + // code to shell extension, which is not done now. + if (!FileBlock && hd->CmpName(SUBHEAD_TYPE_RR) && hd->SubData.Size()>0) + { + RecoveryPercent=hd->SubData[0]; + RSBlockHeader Header; + GetRRInfo(this,&Header); + RecoverySize=Header.RecSectionSize*Header.RecCount; + } +#endif + + if (BadCRC) // Add the file name to broken header message displayed above. + uiMsg(UIERROR_FHEADERBROKEN,Archive::FileName,hd->FileName); + } + break; + case HEAD_ENDARC: + { + *(BaseBlock *)&EndArcHead=ShortBlock; + uint ArcFlags=(uint)Raw.GetV(); + EndArcHead.NextVolume=(ArcFlags & EHFL_NEXTVOLUME)!=0; + EndArcHead.StoreVolNumber=false; + EndArcHead.DataCRC=false; + EndArcHead.RevSpace=false; + } + break; + } + + if (NextBlockPos<=CurBlockPos) + { + BrokenHeaderMsg(); + return 0; + } + return Raw.Size(); +} + + +#if !defined(RAR_NOCRYPT) +void Archive::RequestArcPassword() +{ + if (!Cmd->Password.IsSet()) + { +#ifdef RARDLL + if (Cmd->Callback!=NULL) + { + wchar PasswordW[MAXPASSWORD]; + *PasswordW=0; + if (Cmd->Callback(UCM_NEEDPASSWORDW,Cmd->UserData,(LPARAM)PasswordW,ASIZE(PasswordW))==-1) + *PasswordW=0; + if (*PasswordW==0) + { + char PasswordA[MAXPASSWORD]; + *PasswordA=0; + if (Cmd->Callback(UCM_NEEDPASSWORD,Cmd->UserData,(LPARAM)PasswordA,ASIZE(PasswordA))==-1) + *PasswordA=0; + GetWideName(PasswordA,NULL,PasswordW,ASIZE(PasswordW)); + cleandata(PasswordA,sizeof(PasswordA)); + } + Cmd->Password.Set(PasswordW); + cleandata(PasswordW,sizeof(PasswordW)); + } + if (!Cmd->Password.IsSet()) + { + Close(); + Cmd->DllError=ERAR_MISSING_PASSWORD; + ErrHandler.Exit(RARX_USERBREAK); + } +#else + if (!uiGetPassword(UIPASSWORD_ARCHIVE,FileName,&Cmd->Password) || + !Cmd->Password.IsSet()) + { + Close(); + uiMsg(UIERROR_INCERRCOUNT); + ErrHandler.Exit(RARX_USERBREAK); + } +#endif + Cmd->ManualPassword=true; + } +} +#endif + + +void Archive::ProcessExtra50(RawRead *Raw,size_t ExtraSize,BaseBlock *bb) +{ + // Read extra data from the end of block skipping any fields before it. + size_t ExtraStart=Raw->Size()-ExtraSize; + if (ExtraStartGetPos()) + return; + Raw->SetPos(ExtraStart); + while (Raw->DataLeft()>=2) + { + int64 FieldSize=Raw->GetV(); + if (FieldSize==0 || Raw->DataLeft()==0 || FieldSize>(int64)Raw->DataLeft()) + break; + size_t NextPos=size_t(Raw->GetPos()+FieldSize); + uint64 FieldType=Raw->GetV(); + + FieldSize=int64(NextPos-Raw->GetPos()); // Field size without size and type fields. + + if (bb->HeaderType==HEAD_MAIN) + { + MainHeader *hd=(MainHeader *)bb; + if (FieldType==MHEXTRA_LOCATOR) + { + hd->Locator=true; + uint Flags=(uint)Raw->GetV(); + if ((Flags & MHEXTRA_LOCATOR_QLIST)!=0) + { + uint64 Offset=Raw->GetV(); + if (Offset!=0) // 0 means that reserved space was not enough to write the offset. + hd->QOpenOffset=Offset+CurBlockPos; + } + if ((Flags & MHEXTRA_LOCATOR_RR)!=0) + { + uint64 Offset=Raw->GetV(); + if (Offset!=0) // 0 means that reserved space was not enough to write the offset. + hd->RROffset=Offset+CurBlockPos; + } + } + } + + if (bb->HeaderType==HEAD_FILE || bb->HeaderType==HEAD_SERVICE) + { + FileHeader *hd=(FileHeader *)bb; + switch(FieldType) + { + case FHEXTRA_CRYPT: + { + FileHeader *hd=(FileHeader *)bb; + uint EncVersion=(uint)Raw->GetV(); + if (EncVersion > CRYPT_VERSION) + UnkEncVerMsg(hd->FileName); + else + { + uint Flags=(uint)Raw->GetV(); + hd->UsePswCheck=(Flags & FHEXTRA_CRYPT_PSWCHECK)!=0; + hd->UseHashKey=(Flags & FHEXTRA_CRYPT_HASHMAC)!=0; + hd->Lg2Count=Raw->Get1(); + if (hd->Lg2Count>CRYPT5_KDF_LG2_COUNT_MAX) + UnkEncVerMsg(hd->FileName); + Raw->GetB(hd->Salt,SIZE_SALT50); + Raw->GetB(hd->InitV,SIZE_INITV); + if (hd->UsePswCheck) + { + Raw->GetB(hd->PswCheck,SIZE_PSWCHECK); + + // It is important to know if password check data is valid. + // If it is damaged and header CRC32 fails to detect it, + // archiver would refuse to decompress a possibly valid file. + // Since we want to be sure distinguishing a wrong password + // or corrupt file data, we use 64-bit password check data + // and to control its validity we use 32 bits of password + // check data SHA-256 additionally to 32-bit header CRC32. + byte csum[SIZE_PSWCHECK_CSUM]; + Raw->GetB(csum,SIZE_PSWCHECK_CSUM); + + sha256_context ctx; + sha256_init(&ctx); + sha256_process(&ctx, hd->PswCheck, SIZE_PSWCHECK); + + byte Digest[SHA256_DIGEST_SIZE]; + sha256_done(&ctx, Digest); + + hd->UsePswCheck=memcmp(csum,Digest,SIZE_PSWCHECK_CSUM)==0; + + // RAR 5.21 and earlier set PswCheck field in service records to 0 + // even if UsePswCheck was present. + if (bb->HeaderType==HEAD_SERVICE && memcmp(hd->PswCheck,"\0\0\0\0\0\0\0\0",SIZE_PSWCHECK)==0) + hd->UsePswCheck=0; + } + hd->SaltSet=true; + hd->CryptMethod=CRYPT_RAR50; + hd->Encrypted=true; + } + } + break; + case FHEXTRA_HASH: + { + FileHeader *hd=(FileHeader *)bb; + uint Type=(uint)Raw->GetV(); + if (Type==FHEXTRA_HASH_BLAKE2) + { + hd->FileHash.Type=HASH_BLAKE2; + Raw->GetB(hd->FileHash.Digest,BLAKE2_DIGEST_SIZE); + } + } + break; + case FHEXTRA_HTIME: + if (FieldSize>=5) + { + byte Flags=(byte)Raw->GetV(); + bool UnixTime=(Flags & FHEXTRA_HTIME_UNIXTIME)!=0; + if ((Flags & FHEXTRA_HTIME_MTIME)!=0) + if (UnixTime) + hd->mtime.SetUnix(Raw->Get4()); + else + hd->mtime.SetWin(Raw->Get8()); + if ((Flags & FHEXTRA_HTIME_CTIME)!=0) + if (UnixTime) + hd->ctime.SetUnix(Raw->Get4()); + else + hd->ctime.SetWin(Raw->Get8()); + if ((Flags & FHEXTRA_HTIME_ATIME)!=0) + if (UnixTime) + hd->atime.SetUnix((time_t)Raw->Get4()); + else + hd->atime.SetWin(Raw->Get8()); + if (UnixTime && (Flags & FHEXTRA_HTIME_UNIX_NS)!=0) // Add nanoseconds. + { + uint ns; + if ((Flags & FHEXTRA_HTIME_MTIME)!=0 && (ns=(Raw->Get4() & 0x3fffffff))<1000000000) + hd->mtime.Adjust(ns); + if ((Flags & FHEXTRA_HTIME_CTIME)!=0 && (ns=(Raw->Get4() & 0x3fffffff))<1000000000) + hd->ctime.Adjust(ns); + if ((Flags & FHEXTRA_HTIME_ATIME)!=0 && (ns=(Raw->Get4() & 0x3fffffff))<1000000000) + hd->atime.Adjust(ns); + } + } + break; + case FHEXTRA_VERSION: + if (FieldSize>=1) + { + Raw->GetV(); // Skip flags field. + uint Version=(uint)Raw->GetV(); + if (Version!=0) + { + hd->Version=true; + + wchar VerText[20]; + swprintf(VerText,ASIZE(VerText),L";%u",Version); + wcsncatz(FileHead.FileName,VerText,ASIZE(FileHead.FileName)); + } + } + break; + case FHEXTRA_REDIR: + { + hd->RedirType=(FILE_SYSTEM_REDIRECT)Raw->GetV(); + uint Flags=(uint)Raw->GetV(); + hd->DirTarget=(Flags & FHEXTRA_REDIR_DIR)!=0; + size_t NameSize=(size_t)Raw->GetV(); + + char UtfName[NM*4]; + *UtfName=0; + if (NameSizeGetB(UtfName,NameSize); + UtfName[NameSize]=0; + } +#ifdef _WIN_ALL + UnixSlashToDos(UtfName,UtfName,ASIZE(UtfName)); +#endif + UtfToWide(UtfName,hd->RedirName,ASIZE(hd->RedirName)); + } + break; + case FHEXTRA_UOWNER: + { + uint Flags=(uint)Raw->GetV(); + hd->UnixOwnerNumeric=(Flags & FHEXTRA_UOWNER_NUMUID)!=0; + hd->UnixGroupNumeric=(Flags & FHEXTRA_UOWNER_NUMGID)!=0; + *hd->UnixOwnerName=*hd->UnixGroupName=0; + if ((Flags & FHEXTRA_UOWNER_UNAME)!=0) + { + size_t Length=(size_t)Raw->GetV(); + Length=Min(Length,ASIZE(hd->UnixOwnerName)-1); + Raw->GetB(hd->UnixOwnerName,Length); + hd->UnixOwnerName[Length]=0; + } + if ((Flags & FHEXTRA_UOWNER_GNAME)!=0) + { + size_t Length=(size_t)Raw->GetV(); + Length=Min(Length,ASIZE(hd->UnixGroupName)-1); + Raw->GetB(hd->UnixGroupName,Length); + hd->UnixGroupName[Length]=0; + } +#ifdef _UNIX + if (hd->UnixOwnerNumeric) + hd->UnixOwnerID=(uid_t)Raw->GetV(); + if (hd->UnixGroupNumeric) + hd->UnixGroupID=(gid_t)Raw->GetV(); +#else + // Need these fields in Windows too for 'list' command, + // but uid_t and gid_t are not defined. + if (hd->UnixOwnerNumeric) + hd->UnixOwnerID=(uint)Raw->GetV(); + if (hd->UnixGroupNumeric) + hd->UnixGroupID=(uint)Raw->GetV(); +#endif + hd->UnixOwnerSet=true; + } + break; + case FHEXTRA_SUBDATA: + { + // RAR 5.21 and earlier set FHEXTRA_SUBDATA size to 1 less than + // required. It did not hurt extraction, because UnRAR 5.21 + // and earlier ignored this field and set FieldSize as data left + // in entire extra area. But now we set the correct field size + // and set FieldSize based on actual extra record size, + // so we need to adjust it for those older archives here. + // FHEXTRA_SUBDATA in those archives always belongs to HEAD_SERVICE + // and always is last in extra area. So since its size is by 1 + // less than needed, we always have 1 byte left in extra area, + // which fact we use here to detect such archives. + if (bb->HeaderType==HEAD_SERVICE && Raw->Size()-NextPos==1) + FieldSize++; + + hd->SubData.Alloc((size_t)FieldSize); + Raw->GetB(hd->SubData.Addr(0),(size_t)FieldSize); + } + break; + } + } + + Raw->SetPos(NextPos); + } +} + + +#ifndef SFX_MODULE +size_t Archive::ReadHeader14() +{ + RawRead Raw(this); + if (CurBlockPos<=(int64)SFXSize) + { + Raw.Read(SIZEOF_MAINHEAD14); + MainHead.Reset(); + byte Mark[4]; + Raw.GetB(Mark,4); + uint HeadSize=Raw.Get2(); + byte Flags=Raw.Get1(); + NextBlockPos=CurBlockPos+HeadSize; + CurHeaderType=HEAD_MAIN; + + Volume=(Flags & MHD_VOLUME)!=0; + Solid=(Flags & MHD_SOLID)!=0; + Locked=(Flags & MHD_LOCK)!=0; + MainHead.CommentInHeader=(Flags & MHD_COMMENT)!=0; + MainHead.PackComment=(Flags & MHD_PACK_COMMENT)!=0; + } + else + { + Raw.Read(SIZEOF_FILEHEAD14); + FileHead.Reset(); + + FileHead.HeaderType=HEAD_FILE; + FileHead.DataSize=Raw.Get4(); + FileHead.UnpSize=Raw.Get4(); + FileHead.FileHash.Type=HASH_RAR14; + FileHead.FileHash.CRC32=Raw.Get2(); + FileHead.HeadSize=Raw.Get2(); + uint FileTime=Raw.Get4(); + FileHead.FileAttr=Raw.Get1(); + FileHead.Flags=Raw.Get1()|LONG_BLOCK; + FileHead.UnpVer=(Raw.Get1()==2) ? 13 : 10; + size_t NameSize=Raw.Get1(); + FileHead.Method=Raw.Get1(); + + FileHead.SplitBefore=(FileHead.Flags & LHD_SPLIT_BEFORE)!=0; + FileHead.SplitAfter=(FileHead.Flags & LHD_SPLIT_AFTER)!=0; + FileHead.Encrypted=(FileHead.Flags & LHD_PASSWORD)!=0; + FileHead.CryptMethod=FileHead.Encrypted ? CRYPT_RAR13:CRYPT_NONE; + + FileHead.PackSize=FileHead.DataSize; + FileHead.WinSize=0x10000; + + FileHead.mtime.SetDos(FileTime); + + Raw.Read(NameSize); + + char FileName[NM]; + Raw.GetB((byte *)FileName,Min(NameSize,ASIZE(FileName))); + FileName[NameSize]=0; + IntToExt(FileName,FileName,ASIZE(FileName)); + CharToWide(FileName,FileHead.FileName,ASIZE(FileHead.FileName)); + ConvertNameCase(FileHead.FileName); + + if (Raw.Size()!=0) + NextBlockPos=CurBlockPos+FileHead.HeadSize+FileHead.PackSize; + CurHeaderType=HEAD_FILE; + } + return NextBlockPos>CurBlockPos ? Raw.Size() : 0; +} +#endif + + +#ifndef SFX_MODULE +void Archive::ConvertNameCase(wchar *Name) +{ + if (Cmd->ConvertNames==NAMES_UPPERCASE) + wcsupper(Name); + if (Cmd->ConvertNames==NAMES_LOWERCASE) + wcslower(Name); +} +#endif + + +bool Archive::IsArcDir() +{ + return FileHead.Dir; +} + + +void Archive::ConvertAttributes() +{ +#if defined(_WIN_ALL) || defined(_EMX) + if (FileHead.HSType!=HSYS_WINDOWS) + FileHead.FileAttr=FileHead.Dir ? 0x10 : 0x20; +#endif +#ifdef _UNIX + // umask defines which permission bits must not be set by default + // when creating a file or directory. The typical default value + // for the process umask is S_IWGRP | S_IWOTH (octal 022), + // resulting in 0644 mode for new files. + // Normally umask is applied automatically when creating a file, + // but we set attributes with chmod later, so we need to calculate + // resulting attributes here. We do it only for non-Unix archives. + // We restore native Unix attributes as is, because it can be backup. + static mode_t mask = (mode_t) -1; + + if (mask == (mode_t) -1) + { + // umask call returns the current umask value. Argument (022) is not + // really important here. + mask = umask(022); + + // Restore the original umask value, which was changed to 022 above. + umask(mask); + } + + switch(FileHead.HSType) + { + case HSYS_WINDOWS: + { + // Mapping MSDOS, OS/2 and Windows file attributes to Unix. + + if (FileHead.FileAttr & 0x10) // FILE_ATTRIBUTE_DIRECTORY + { + // For directories we use 0777 mask. + FileHead.FileAttr=0777 & ~mask; + } + else + if (FileHead.FileAttr & 1) // FILE_ATTRIBUTE_READONLY + { + // For read only files we use 0444 mask with 'w' bits turned off. + FileHead.FileAttr=0444 & ~mask; + } + else + { + // umask does not set +x for regular files, so we use 0666 + // instead of 0777 as for directories. + FileHead.FileAttr=0666 & ~mask; + } + } + break; + case HSYS_UNIX: + break; + default: + if (FileHead.Dir) + FileHead.FileAttr=0x41ff & ~mask; + else + FileHead.FileAttr=0x81b6 & ~mask; + break; + } +#endif +} + + +void Archive::ConvertFileHeader(FileHeader *hd) +{ + if (Format==RARFMT15 && hd->UnpVer<20 && (hd->FileAttr & 0x10)) + hd->Dir=true; + if (hd->HSType==HSYS_UNKNOWN) + if (hd->Dir) + hd->FileAttr=0x10; + else + hd->FileAttr=0x20; + +#ifdef _WIN_ALL + if (hd->HSType==HSYS_UNIX) // Convert Unix, OS X and Android decomposed chracters to Windows precomposed. + ConvertToPrecomposed(hd->FileName,ASIZE(hd->FileName)); +#endif + + for (wchar *s=hd->FileName;*s!=0;s++) + { +#ifdef _UNIX + // Backslash is the invalid character for Windows file headers, + // but it can present in Unix file names extracted in Unix. + if (*s=='\\' && Format==RARFMT50 && hd->HSType==HSYS_WINDOWS) + *s='_'; +#endif + +#if defined(_WIN_ALL) || defined(_EMX) + // RAR 5.0 archives do not use '\' as path separator, so if we see it, + // it means that it is a part of Unix file name, which we cannot + // extract in Windows. + if (*s=='\\' && Format==RARFMT50) + *s='_'; + + // ':' in file names is allowed in Unix, but not in Windows. + // Even worse, file data will be written to NTFS stream on NTFS, + // so automatic name correction on file create error in extraction + // routine does not work. In Windows and DOS versions we better + // replace ':' now. + if (*s==':') + *s='_'; +#endif + + // This code must be performed only after other path separator checks, + // because it produces backslashes illegal for some of checks above. + // Backslash is allowed in file names in Unix, but not in Windows. + // Still, RAR 4.x uses backslashes as path separator even in Unix. + // Forward slash is not allowed in both systems. In RAR 5.0 we use + // the forward slash as universal path separator. + if (*s=='/' || *s=='\\' && Format!=RARFMT50) + *s=CPATHDIVIDER; + } +} + + +int64 Archive::GetStartPos() +{ + int64 StartPos=SFXSize+MarkHead.HeadSize; + if (Format==RARFMT15) + StartPos+=MainHead.HeadSize; + else // RAR 5.0. + StartPos+=CryptHead.HeadSize+FullHeaderSize(MainHead.HeadSize); + return StartPos; +} + + +bool Archive::ReadSubData(Array *UnpData,File *DestFile) +{ + if (BrokenHeader) + { + uiMsg(UIERROR_SUBHEADERBROKEN,FileName); + ErrHandler.SetErrorCode(RARX_CRC); + return false; + } + if (SubHead.Method>5 || SubHead.UnpVer>(Format==RARFMT50 ? VER_UNPACK5:VER_UNPACK)) + { + uiMsg(UIERROR_SUBHEADERUNKNOWN,FileName); + return false; + } + + if (SubHead.PackSize==0 && !SubHead.SplitAfter) + return true; + + SubDataIO.Init(); + Unpack Unpack(&SubDataIO); + Unpack.Init(SubHead.WinSize,false); + + if (DestFile==NULL) + { + if (SubHead.UnpSize>0x1000000) + { + // So huge allocation must never happen in valid archives. + uiMsg(UIERROR_SUBHEADERUNKNOWN,FileName); + return false; + } + if (UnpData==NULL) + SubDataIO.SetTestMode(true); + else + { + UnpData->Alloc((size_t)SubHead.UnpSize); + SubDataIO.SetUnpackToMemory(&(*UnpData)[0],(uint)SubHead.UnpSize); + } + } + if (SubHead.Encrypted) + if (Cmd->Password.IsSet()) + SubDataIO.SetEncryption(false,SubHead.CryptMethod,&Cmd->Password, + SubHead.SaltSet ? SubHead.Salt:NULL,SubHead.InitV, + SubHead.Lg2Count,SubHead.HashKey,SubHead.PswCheck); + else + return false; + SubDataIO.UnpHash.Init(SubHead.FileHash.Type,1); + SubDataIO.SetPackedSizeToRead(SubHead.PackSize); + SubDataIO.EnableShowProgress(false); + SubDataIO.SetFiles(this,DestFile); + SubDataIO.UnpVolume=SubHead.SplitAfter; + SubDataIO.SetSubHeader(&SubHead,NULL); + Unpack.SetDestSize(SubHead.UnpSize); + if (SubHead.Method==0) + CmdExtract::UnstoreFile(SubDataIO,SubHead.UnpSize); + else + Unpack.DoUnpack(SubHead.UnpVer,false); + + if (!SubDataIO.UnpHash.Cmp(&SubHead.FileHash,SubHead.UseHashKey ? SubHead.HashKey:NULL)) + { + uiMsg(UIERROR_SUBHEADERDATABROKEN,FileName,SubHead.FileName); + ErrHandler.SetErrorCode(RARX_CRC); + if (UnpData!=NULL) + UnpData->Reset(); + return false; + } + return true; +} diff --git a/reference/unrar/array.hpp b/reference/unrar/array.hpp new file mode 100644 index 00000000..20d258d5 --- /dev/null +++ b/reference/unrar/array.hpp @@ -0,0 +1,191 @@ +#ifndef _RAR_ARRAY_ +#define _RAR_ARRAY_ + +extern ErrorHandler ErrHandler; + +template class Array +{ + private: + T *Buffer; + size_t BufSize; + size_t AllocSize; + size_t MaxSize; + bool Secure; // Clean memory if true. + public: + Array(); + Array(size_t Size); + Array(const Array &Src); // Copy constructor. + ~Array(); + inline void CleanData(); + inline T& operator [](size_t Item) const; + inline T* operator + (size_t Pos); + inline size_t Size(); // Returns the size in items, not in bytes. + void Add(size_t Items); + void Alloc(size_t Items); + void Reset(); + void SoftReset(); + void operator = (Array &Src); + void Push(T Item); + void Append(T *Item,size_t Count); + T* Addr(size_t Item) {return Buffer+Item;} + void SetMaxSize(size_t Size) {MaxSize=Size;} + T* Begin() {return Buffer;} + T* End() {return Buffer==NULL ? NULL:Buffer+BufSize;} + void SetSecure() {Secure=true;} +}; + + +template void Array::CleanData() +{ + Buffer=NULL; + BufSize=0; + AllocSize=0; + MaxSize=0; + Secure=false; +} + + +template Array::Array() +{ + CleanData(); +} + + +template Array::Array(size_t Size) +{ + CleanData(); + Add(Size); +} + + +// Copy constructor in case we need to pass an object as value. +template Array::Array(const Array &Src) +{ + CleanData(); + Alloc(Src.BufSize); + if (Src.BufSize!=0) + memcpy((void *)Buffer,(void *)Src.Buffer,Src.BufSize*sizeof(T)); +} + + +template Array::~Array() +{ + if (Buffer!=NULL) + { + if (Secure) + cleandata(Buffer,AllocSize*sizeof(T)); + free(Buffer); + } +} + + +template inline T& Array::operator [](size_t Item) const +{ + return Buffer[Item]; +} + + +template inline T* Array::operator +(size_t Pos) +{ + return Buffer+Pos; +} + + +template inline size_t Array::Size() +{ + return BufSize; +} + + +template void Array::Add(size_t Items) +{ + BufSize+=Items; + if (BufSize>AllocSize) + { + if (MaxSize!=0 && BufSize>MaxSize) + { + ErrHandler.GeneralErrMsg(L"Maximum allowed array size (%u) is exceeded",MaxSize); + ErrHandler.MemoryError(); + } + + size_t Suggested=AllocSize+AllocSize/4+32; + size_t NewSize=Max(BufSize,Suggested); + + T *NewBuffer; + if (Secure) + { + NewBuffer=(T *)malloc(NewSize*sizeof(T)); + if (NewBuffer==NULL) + ErrHandler.MemoryError(); + if (Buffer!=NULL) + { + memcpy(NewBuffer,Buffer,AllocSize*sizeof(T)); + cleandata(Buffer,AllocSize*sizeof(T)); + free(Buffer); + } + } + else + { + NewBuffer=(T *)realloc(Buffer,NewSize*sizeof(T)); + if (NewBuffer==NULL) + ErrHandler.MemoryError(); + } + Buffer=NewBuffer; + AllocSize=NewSize; + } +} + + +template void Array::Alloc(size_t Items) +{ + if (Items>AllocSize) + Add(Items-BufSize); + else + BufSize=Items; +} + + +template void Array::Reset() +{ + if (Buffer!=NULL) + { + free(Buffer); + Buffer=NULL; + } + BufSize=0; + AllocSize=0; +} + + +// Reset buffer size, but preserve already allocated memory if any, +// so we can reuse it without wasting time to allocation. +template void Array::SoftReset() +{ + BufSize=0; +} + + +template void Array::operator =(Array &Src) +{ + Reset(); + Alloc(Src.BufSize); + if (Src.BufSize!=0) + memcpy((void *)Buffer,(void *)Src.Buffer,Src.BufSize*sizeof(T)); +} + + +template void Array::Push(T Item) +{ + Add(1); + (*this)[Size()-1]=Item; +} + + +template void Array::Append(T *Items,size_t Count) +{ + size_t CurSize=Size(); + Add(Count); + memcpy(Buffer+CurSize,Items,Count*sizeof(T)); +} + +#endif diff --git a/reference/unrar/blake2s.cpp b/reference/unrar/blake2s.cpp new file mode 100644 index 00000000..317603da --- /dev/null +++ b/reference/unrar/blake2s.cpp @@ -0,0 +1,183 @@ +// Based on public domain code written in 2012 by Samuel Neves + +#include "rar.hpp" + +#ifdef USE_SSE +#include "blake2s_sse.cpp" +#endif + +static void blake2s_init_param( blake2s_state *S, uint32 node_offset, uint32 node_depth); +static void blake2s_update( blake2s_state *S, const byte *in, size_t inlen ); +static void blake2s_final( blake2s_state *S, byte *digest ); + +#include "blake2sp.cpp" + +static const uint32 blake2s_IV[8] = +{ + 0x6A09E667UL, 0xBB67AE85UL, 0x3C6EF372UL, 0xA54FF53AUL, + 0x510E527FUL, 0x9B05688CUL, 0x1F83D9ABUL, 0x5BE0CD19UL +}; + +static const byte blake2s_sigma[10][16] = +{ + { 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 } , + { 14, 10, 4, 8, 9, 15, 13, 6, 1, 12, 0, 2, 11, 7, 5, 3 } , + { 11, 8, 12, 0, 5, 2, 15, 13, 10, 14, 3, 6, 7, 1, 9, 4 } , + { 7, 9, 3, 1, 13, 12, 11, 14, 2, 6, 5, 10, 4, 0, 15, 8 } , + { 9, 0, 5, 7, 2, 4, 10, 15, 14, 1, 11, 12, 6, 8, 3, 13 } , + { 2, 12, 6, 10, 0, 11, 8, 3, 4, 13, 7, 5, 15, 14, 1, 9 } , + { 12, 5, 1, 15, 14, 13, 4, 10, 0, 7, 6, 3, 9, 2, 8, 11 } , + { 13, 11, 7, 14, 12, 1, 3, 9, 5, 0, 15, 4, 8, 6, 2, 10 } , + { 6, 15, 14, 9, 11, 3, 0, 8, 12, 2, 13, 7, 1, 4, 10, 5 } , + { 10, 2, 8, 4, 7, 6, 1, 5, 15, 11, 9, 14, 3, 12, 13 , 0 } , +}; + +static inline void blake2s_set_lastnode( blake2s_state *S ) +{ + S->f[1] = ~0U; +} + + +/* Some helper functions, not necessarily useful */ +static inline void blake2s_set_lastblock( blake2s_state *S ) +{ + if( S->last_node ) blake2s_set_lastnode( S ); + + S->f[0] = ~0U; +} + + +static inline void blake2s_increment_counter( blake2s_state *S, const uint32 inc ) +{ + S->t[0] += inc; + S->t[1] += ( S->t[0] < inc ); +} + + +/* init2 xors IV with input parameter block */ +void blake2s_init_param( blake2s_state *S, uint32 node_offset, uint32 node_depth) +{ +#ifdef USE_SSE + if (_SSE_Version>=SSE_SSE2) + blake2s_init_sse(); +#endif + + S->init(); // Clean data. + for( int i = 0; i < 8; ++i ) + S->h[i] = blake2s_IV[i]; + + S->h[0] ^= 0x02080020; // We use BLAKE2sp parameters block. + S->h[2] ^= node_offset; + S->h[3] ^= (node_depth<<16)|0x20000000; +} + + +#define G(r,i,m,a,b,c,d) \ + a = a + b + m[blake2s_sigma[r][2*i+0]]; \ + d = rotr32(d ^ a, 16); \ + c = c + d; \ + b = rotr32(b ^ c, 12); \ + a = a + b + m[blake2s_sigma[r][2*i+1]]; \ + d = rotr32(d ^ a, 8); \ + c = c + d; \ + b = rotr32(b ^ c, 7); + + +static void blake2s_compress( blake2s_state *S, const byte block[BLAKE2S_BLOCKBYTES] ) +{ + uint32 m[16]; + uint32 v[16]; + + for( size_t i = 0; i < 16; ++i ) + m[i] = RawGet4( block + i * 4 ); + + for( size_t i = 0; i < 8; ++i ) + v[i] = S->h[i]; + + v[ 8] = blake2s_IV[0]; + v[ 9] = blake2s_IV[1]; + v[10] = blake2s_IV[2]; + v[11] = blake2s_IV[3]; + v[12] = S->t[0] ^ blake2s_IV[4]; + v[13] = S->t[1] ^ blake2s_IV[5]; + v[14] = S->f[0] ^ blake2s_IV[6]; + v[15] = S->f[1] ^ blake2s_IV[7]; + + for ( uint r = 0; r <= 9; ++r ) // No gain on i7 if unrolled, but exe size grows. + { + G(r,0,m,v[ 0],v[ 4],v[ 8],v[12]); + G(r,1,m,v[ 1],v[ 5],v[ 9],v[13]); + G(r,2,m,v[ 2],v[ 6],v[10],v[14]); + G(r,3,m,v[ 3],v[ 7],v[11],v[15]); + G(r,4,m,v[ 0],v[ 5],v[10],v[15]); + G(r,5,m,v[ 1],v[ 6],v[11],v[12]); + G(r,6,m,v[ 2],v[ 7],v[ 8],v[13]); + G(r,7,m,v[ 3],v[ 4],v[ 9],v[14]); + } + + for( size_t i = 0; i < 8; ++i ) + S->h[i] = S->h[i] ^ v[i] ^ v[i + 8]; +} + + +void blake2s_update( blake2s_state *S, const byte *in, size_t inlen ) +{ + while( inlen > 0 ) + { + size_t left = S->buflen; + size_t fill = 2 * BLAKE2S_BLOCKBYTES - left; + + if( inlen > fill ) + { + memcpy( S->buf + left, in, fill ); // Fill buffer + S->buflen += fill; + blake2s_increment_counter( S, BLAKE2S_BLOCKBYTES ); + +#ifdef USE_SSE +#ifdef _WIN_32 // We use SSSE3 _mm_shuffle_epi8 only in x64 mode. + if (_SSE_Version>=SSE_SSE2) +#else + if (_SSE_Version>=SSE_SSSE3) +#endif + blake2s_compress_sse( S, S->buf ); + else + blake2s_compress( S, S->buf ); // Compress +#else + blake2s_compress( S, S->buf ); // Compress +#endif + + memcpy( S->buf, S->buf + BLAKE2S_BLOCKBYTES, BLAKE2S_BLOCKBYTES ); // Shift buffer left + S->buflen -= BLAKE2S_BLOCKBYTES; + in += fill; + inlen -= fill; + } + else // inlen <= fill + { + memcpy( S->buf + left, in, (size_t)inlen ); + S->buflen += (size_t)inlen; // Be lazy, do not compress + in += inlen; + inlen = 0; + } + } +} + + +void blake2s_final( blake2s_state *S, byte *digest ) +{ + if( S->buflen > BLAKE2S_BLOCKBYTES ) + { + blake2s_increment_counter( S, BLAKE2S_BLOCKBYTES ); + blake2s_compress( S, S->buf ); + S->buflen -= BLAKE2S_BLOCKBYTES; + memcpy( S->buf, S->buf + BLAKE2S_BLOCKBYTES, S->buflen ); + } + + blake2s_increment_counter( S, ( uint32 )S->buflen ); + blake2s_set_lastblock( S ); + memset( S->buf + S->buflen, 0, 2 * BLAKE2S_BLOCKBYTES - S->buflen ); /* Padding */ + blake2s_compress( S, S->buf ); + + for( int i = 0; i < 8; ++i ) /* Output full hash */ + RawPut4( S->h[i], digest + 4 * i ); +} + diff --git a/reference/unrar/blake2s.hpp b/reference/unrar/blake2s.hpp new file mode 100644 index 00000000..7dd71571 --- /dev/null +++ b/reference/unrar/blake2s.hpp @@ -0,0 +1,101 @@ +// Based on public domain code written in 2012 by Samuel Neves +#ifndef _RAR_BLAKE2_ +#define _RAR_BLAKE2_ + +#define BLAKE2_DIGEST_SIZE 32 + +enum blake2s_constant +{ + BLAKE2S_BLOCKBYTES = 64, + BLAKE2S_OUTBYTES = 32 +}; + + +// Alignment to 64 improves performance of both SSE and non-SSE versions. +// Alignment to n*16 is required for SSE version, so we selected 64. +// We use the custom alignment scheme instead of __declspec(align(x)), +// because it is less compiler dependent. Also the compiler directive +// does not help if structure is a member of class allocated through +// 'new' operator. +struct blake2s_state +{ + enum { BLAKE_ALIGNMENT = 64 }; + + // buffer and uint32 h[8], t[2], f[2]; + enum { BLAKE_DATA_SIZE = 48 + 2 * BLAKE2S_BLOCKBYTES }; + + byte ubuf[BLAKE_DATA_SIZE + BLAKE_ALIGNMENT]; + + byte *buf; // byte buf[2 * BLAKE2S_BLOCKBYTES]. + uint32 *h, *t, *f; // uint32 h[8], t[2], f[2]. + + size_t buflen; + byte last_node; + + blake2s_state() + { + set_pointers(); + } + + // Required when we declare and assign in the same command. + blake2s_state(blake2s_state &st) + { + set_pointers(); + *this=st; + } + + void set_pointers() + { + // Set aligned pointers. Must be done in constructor, not in Init(), + // so assignments like 'blake2sp_state res=blake2ctx' work correctly + // even if blake2sp_init is not called for 'res'. + buf = (byte *) ALIGN_VALUE(ubuf, BLAKE_ALIGNMENT); + h = (uint32 *) (buf + 2 * BLAKE2S_BLOCKBYTES); + t = h + 8; + f = t + 2; + } + + void init() + { + memset( ubuf, 0, sizeof( ubuf ) ); + buflen = 0; + last_node = 0; + } + + // Since we use pointers, the default = would work incorrectly. + blake2s_state& operator = (blake2s_state &st) + { + if (this != &st) + { + memcpy(buf, st.buf, BLAKE_DATA_SIZE); + buflen = st.buflen; + last_node = st.last_node; + } + return *this; + } +}; + + +#ifdef RAR_SMP +class ThreadPool; +#endif + +struct blake2sp_state +{ + blake2s_state S[8]; + blake2s_state R; + byte buf[8 * BLAKE2S_BLOCKBYTES]; + size_t buflen; + +#ifdef RAR_SMP + ThreadPool *ThPool; + uint MaxThreads; +#endif +}; + +void blake2sp_init( blake2sp_state *S ); +void blake2sp_update( blake2sp_state *S, const byte *in, size_t inlen ); +void blake2sp_final( blake2sp_state *S, byte *digest ); + +#endif + diff --git a/reference/unrar/blake2s_sse.cpp b/reference/unrar/blake2s_sse.cpp new file mode 100644 index 00000000..1a02f210 --- /dev/null +++ b/reference/unrar/blake2s_sse.cpp @@ -0,0 +1,129 @@ +// Based on public domain code written in 2012 by Samuel Neves + +extern const byte blake2s_sigma[10][16]; + +// Initialization vector. +static __m128i blake2s_IV_0_3, blake2s_IV_4_7; + +#ifdef _WIN_64 +// Constants for cyclic rotation. Used in 64-bit mode in mm_rotr_epi32 macro. +static __m128i crotr8, crotr16; +#endif + +static void blake2s_init_sse() +{ + // We cannot initialize these 128 bit variables in place when declaring + // them globally, because global scope initialization is performed before + // our SSE check and it would make code incompatible with older non-SSE2 + // CPUs. Also we cannot initialize them as static inside of function + // using these variables, because SSE static initialization is not thread + // safe: first thread starts initialization and sets "init done" flag even + // if it is not done yet, second thread can attempt to access half-init + // SSE data. So we moved init code here. + + blake2s_IV_0_3 = _mm_setr_epi32( 0x6A09E667, 0xBB67AE85, 0x3C6EF372, 0xA54FF53A ); + blake2s_IV_4_7 = _mm_setr_epi32( 0x510E527F, 0x9B05688C, 0x1F83D9AB, 0x5BE0CD19 ); + +#ifdef _WIN_64 + crotr8 = _mm_set_epi8( 12, 15, 14, 13, 8, 11, 10, 9, 4, 7, 6, 5, 0, 3, 2, 1 ); + crotr16 = _mm_set_epi8( 13, 12, 15, 14, 9, 8, 11, 10, 5, 4, 7, 6, 1, 0, 3, 2 ); +#endif +} + + +#define LOAD(p) _mm_load_si128( (__m128i *)(p) ) +#define STORE(p,r) _mm_store_si128((__m128i *)(p), r) + +#ifdef _WIN_32 +// 32-bit mode has less SSE2 registers and in MSVC2008 it is more efficient +// to not use _mm_shuffle_epi8 here. +#define mm_rotr_epi32(r, c) ( \ + _mm_xor_si128(_mm_srli_epi32( (r), c ),_mm_slli_epi32( (r), 32-c )) ) +#else +#define mm_rotr_epi32(r, c) ( \ + c==8 ? _mm_shuffle_epi8(r,crotr8) \ + : c==16 ? _mm_shuffle_epi8(r,crotr16) \ + : _mm_xor_si128(_mm_srli_epi32( (r), c ),_mm_slli_epi32( (r), 32-c )) ) +#endif + + +#define G1(row1,row2,row3,row4,buf) \ + row1 = _mm_add_epi32( _mm_add_epi32( row1, buf), row2 ); \ + row4 = _mm_xor_si128( row4, row1 ); \ + row4 = mm_rotr_epi32(row4, 16); \ + row3 = _mm_add_epi32( row3, row4 ); \ + row2 = _mm_xor_si128( row2, row3 ); \ + row2 = mm_rotr_epi32(row2, 12); + +#define G2(row1,row2,row3,row4,buf) \ + row1 = _mm_add_epi32( _mm_add_epi32( row1, buf), row2 ); \ + row4 = _mm_xor_si128( row4, row1 ); \ + row4 = mm_rotr_epi32(row4, 8); \ + row3 = _mm_add_epi32( row3, row4 ); \ + row2 = _mm_xor_si128( row2, row3 ); \ + row2 = mm_rotr_epi32(row2, 7); + +#define DIAGONALIZE(row1,row2,row3,row4) \ + row4 = _mm_shuffle_epi32( row4, _MM_SHUFFLE(2,1,0,3) ); \ + row3 = _mm_shuffle_epi32( row3, _MM_SHUFFLE(1,0,3,2) ); \ + row2 = _mm_shuffle_epi32( row2, _MM_SHUFFLE(0,3,2,1) ); + +#define UNDIAGONALIZE(row1,row2,row3,row4) \ + row4 = _mm_shuffle_epi32( row4, _MM_SHUFFLE(0,3,2,1) ); \ + row3 = _mm_shuffle_epi32( row3, _MM_SHUFFLE(1,0,3,2) ); \ + row2 = _mm_shuffle_epi32( row2, _MM_SHUFFLE(2,1,0,3) ); + +#ifdef _WIN_64 + // MSVC 2008 in x64 mode expands _mm_set_epi32 to store to stack and load + // from stack operations, which are slower than this code. + #define _mm_set_epi32(i3,i2,i1,i0) \ + _mm_unpacklo_epi32(_mm_unpacklo_epi32(_mm_cvtsi32_si128(i0),_mm_cvtsi32_si128(i2)), \ + _mm_unpacklo_epi32(_mm_cvtsi32_si128(i1),_mm_cvtsi32_si128(i3))) +#endif + +// Original BLAKE2 SSE4.1 message loading code was a little slower in x86 mode +// and about the same in x64 mode in our test. Perhaps depends on compiler. +// We also tried _mm_i32gather_epi32 and _mm256_i32gather_epi32 AVX2 gather +// instructions here, but they did not show any speed gain on i7-6700K. +#define SSE_ROUND(m,row,r) \ +{ \ + __m128i buf; \ + buf=_mm_set_epi32(m[blake2s_sigma[r][6]],m[blake2s_sigma[r][4]],m[blake2s_sigma[r][2]],m[blake2s_sigma[r][0]]); \ + G1(row[0],row[1],row[2],row[3],buf); \ + buf=_mm_set_epi32(m[blake2s_sigma[r][7]],m[blake2s_sigma[r][5]],m[blake2s_sigma[r][3]],m[blake2s_sigma[r][1]]); \ + G2(row[0],row[1],row[2],row[3],buf); \ + DIAGONALIZE(row[0],row[1],row[2],row[3]); \ + buf=_mm_set_epi32(m[blake2s_sigma[r][14]],m[blake2s_sigma[r][12]],m[blake2s_sigma[r][10]],m[blake2s_sigma[r][8]]); \ + G1(row[0],row[1],row[2],row[3],buf); \ + buf=_mm_set_epi32(m[blake2s_sigma[r][15]],m[blake2s_sigma[r][13]],m[blake2s_sigma[r][11]],m[blake2s_sigma[r][9]]); \ + G2(row[0],row[1],row[2],row[3],buf); \ + UNDIAGONALIZE(row[0],row[1],row[2],row[3]); \ +} + + +static int blake2s_compress_sse( blake2s_state *S, const byte block[BLAKE2S_BLOCKBYTES] ) +{ + __m128i row[4]; + __m128i ff0, ff1; + + const uint32 *m = ( uint32 * )block; + + row[0] = ff0 = LOAD( &S->h[0] ); + row[1] = ff1 = LOAD( &S->h[4] ); + + row[2] = blake2s_IV_0_3; + row[3] = _mm_xor_si128( blake2s_IV_4_7, LOAD( &S->t[0] ) ); + SSE_ROUND( m, row, 0 ); + SSE_ROUND( m, row, 1 ); + SSE_ROUND( m, row, 2 ); + SSE_ROUND( m, row, 3 ); + SSE_ROUND( m, row, 4 ); + SSE_ROUND( m, row, 5 ); + SSE_ROUND( m, row, 6 ); + SSE_ROUND( m, row, 7 ); + SSE_ROUND( m, row, 8 ); + SSE_ROUND( m, row, 9 ); + STORE( &S->h[0], _mm_xor_si128( ff0, _mm_xor_si128( row[0], row[2] ) ) ); + STORE( &S->h[4], _mm_xor_si128( ff1, _mm_xor_si128( row[1], row[3] ) ) ); + return 0; +} diff --git a/reference/unrar/blake2sp.cpp b/reference/unrar/blake2sp.cpp new file mode 100644 index 00000000..da645883 --- /dev/null +++ b/reference/unrar/blake2sp.cpp @@ -0,0 +1,153 @@ +/* + BLAKE2 reference source code package - reference C implementations + + Written in 2012 by Samuel Neves + + To the extent possible under law, the author(s) have dedicated all copyright + and related and neighboring rights to this software to the public domain + worldwide. This software is distributed without any warranty. + + You should have received a copy of the CC0 Public Domain Dedication along with + this software. If not, see . +*/ + +#define PARALLELISM_DEGREE 8 + +void blake2sp_init( blake2sp_state *S ) +{ + memset( S->buf, 0, sizeof( S->buf ) ); + S->buflen = 0; + + blake2s_init_param( &S->R, 0, 1 ); // Init root. + + for( uint i = 0; i < PARALLELISM_DEGREE; ++i ) + blake2s_init_param( &S->S[i], i, 0 ); // Init leaf. + + S->R.last_node = 1; + S->S[PARALLELISM_DEGREE - 1].last_node = 1; +} + + +struct Blake2ThreadData +{ + void Update(); + blake2s_state *S; + const byte *in; + size_t inlen; +}; + + +void Blake2ThreadData::Update() +{ + size_t inlen__ = inlen; + const byte *in__ = ( const byte * )in; + + while( inlen__ >= PARALLELISM_DEGREE * BLAKE2S_BLOCKBYTES ) + { +#ifdef USE_SSE + // We gain 5% in i7 SSE mode by prefetching next data block. + if (_SSE_Version>=SSE_SSE && inlen__ >= 2 * PARALLELISM_DEGREE * BLAKE2S_BLOCKBYTES) + _mm_prefetch((char*)(in__ + PARALLELISM_DEGREE * BLAKE2S_BLOCKBYTES), _MM_HINT_T0); +#endif + blake2s_update( S, in__, BLAKE2S_BLOCKBYTES ); + in__ += PARALLELISM_DEGREE * BLAKE2S_BLOCKBYTES; + inlen__ -= PARALLELISM_DEGREE * BLAKE2S_BLOCKBYTES; + } +} + +#ifdef RAR_SMP +THREAD_PROC(Blake2Thread) +{ + Blake2ThreadData *td=(Blake2ThreadData *)Data; + td->Update(); +} +#endif + + +void blake2sp_update( blake2sp_state *S, const byte *in, size_t inlen ) +{ + size_t left = S->buflen; + size_t fill = sizeof( S->buf ) - left; + + if( left && inlen >= fill ) + { + memcpy( S->buf + left, in, fill ); + + for( size_t i = 0; i < PARALLELISM_DEGREE; ++i ) + blake2s_update( &S->S[i], S->buf + i * BLAKE2S_BLOCKBYTES, BLAKE2S_BLOCKBYTES ); + + in += fill; + inlen -= fill; + left = 0; + } + + Blake2ThreadData btd_array[PARALLELISM_DEGREE]; + +#ifdef RAR_SMP + uint ThreadNumber = inlen < 0x1000 ? 1 : S->MaxThreads; + + if (ThreadNumber==6 || ThreadNumber==7) // 6 and 7 threads work slower than 4 here. + ThreadNumber=4; +#else + uint ThreadNumber=1; +#endif + + for (size_t id__=0;id__inlen = inlen; + btd->in = in + id__ * BLAKE2S_BLOCKBYTES; + btd->S = &S->S[id__]; + +#ifdef RAR_SMP + if (ThreadNumber>1) + S->ThPool->AddTask(Blake2Thread,(void*)btd); + else + btd->Update(); +#else + btd->Update(); +#endif + id__++; + } +#ifdef RAR_SMP + if (S->ThPool!=NULL) // Can be NULL in -mt1 mode. + S->ThPool->WaitDone(); +#endif // RAR_SMP + } + + in += inlen - inlen % ( PARALLELISM_DEGREE * BLAKE2S_BLOCKBYTES ); + inlen %= PARALLELISM_DEGREE * BLAKE2S_BLOCKBYTES; + + if( inlen > 0 ) + memcpy( S->buf + left, in, (size_t)inlen ); + + S->buflen = left + (size_t)inlen; +} + + +void blake2sp_final( blake2sp_state *S, byte *digest ) +{ + byte hash[PARALLELISM_DEGREE][BLAKE2S_OUTBYTES]; + + for( size_t i = 0; i < PARALLELISM_DEGREE; ++i ) + { + if( S->buflen > i * BLAKE2S_BLOCKBYTES ) + { + size_t left = S->buflen - i * BLAKE2S_BLOCKBYTES; + + if( left > BLAKE2S_BLOCKBYTES ) left = BLAKE2S_BLOCKBYTES; + + blake2s_update( &S->S[i], S->buf + i * BLAKE2S_BLOCKBYTES, left ); + } + + blake2s_final( &S->S[i], hash[i] ); + } + + for( size_t i = 0; i < PARALLELISM_DEGREE; ++i ) + blake2s_update( &S->R, hash[i], BLAKE2S_OUTBYTES ); + + blake2s_final( &S->R, digest ); +} diff --git a/reference/unrar/cmddata.cpp b/reference/unrar/cmddata.cpp new file mode 100644 index 00000000..758cd171 --- /dev/null +++ b/reference/unrar/cmddata.cpp @@ -0,0 +1,1400 @@ +#include "rar.hpp" + +CommandData::CommandData() +{ + Init(); +} + + +void CommandData::Init() +{ + RAROptions::Init(); + + *Command=0; + *ArcName=0; + FileLists=false; + + ListMode=RCLM_AUTO; + + BareOutput=false; + + + FileArgs.Reset(); + ExclArgs.Reset(); + InclArgs.Reset(); + StoreArgs.Reset(); + ArcNames.Reset(); + NextVolSizes.Reset(); +} + + +// Return the pointer to next position in the string and store dynamically +// allocated command line parameter in Par. +static const wchar *AllocCmdParam(const wchar *CmdLine,wchar **Par) +{ + const wchar *NextCmd=GetCmdParam(CmdLine,NULL,0); + if (NextCmd==NULL) + return NULL; + size_t ParSize=NextCmd-CmdLine+2; // Parameter size including the trailing zero. + *Par=(wchar *)malloc(ParSize*sizeof(wchar)); + if (*Par==NULL) + return NULL; + return GetCmdParam(CmdLine,*Par,ParSize); +} + + +#if !defined(SFX_MODULE) +void CommandData::ParseCommandLine(bool Preprocess,int argc, char *argv[]) +{ + *Command=0; + NoMoreSwitches=false; +#ifdef CUSTOM_CMDLINE_PARSER + // In Windows we may prefer to implement our own command line parser + // to avoid replacing \" by " in standard parser. Such replacing corrupts + // destination paths like "dest path\" in extraction commands. + // Also our own parser is Unicode compatible. + const wchar *CmdLine=GetCommandLine(); + + wchar *Par; + for (bool FirstParam=true;;FirstParam=false) + { + if ((CmdLine=AllocCmdParam(CmdLine,&Par))==NULL) + break; + if (!FirstParam) // First parameter is the executable name. + if (Preprocess) + PreprocessArg(Par); + else + ParseArg(Par); + free(Par); + } +#else + Array Arg; + for (int I=1;I EnvStrW(strlen(EnvStr)+1); + CharToWide(EnvStr,&EnvStrW[0],EnvStrW.Size()); + ProcessSwitchesString(&EnvStrW[0]); + } +} +#endif + + + +#if !defined(SFX_MODULE) +// Preprocess those parameters, which must be processed before the rest of +// command line. Return 'false' to stop further processing. +void CommandData::PreprocessArg(const wchar *Arg) +{ + if (IsSwitch(Arg[0]) && !NoMoreSwitches) + { + Arg++; + if (Arg[0]=='-' && Arg[1]==0) // Switch "--". + NoMoreSwitches=true; + if (wcsicomp(Arg,L"cfg-")==0) + ConfigDisabled=true; + if (wcsnicomp(Arg,L"ilog",4)==0) + { + // Ensure that correct log file name is already set + // if we need to report an error when processing the command line. + ProcessSwitch(Arg); + InitLogOptions(LogName,ErrlogCharset); + } + if (wcsnicomp(Arg,L"sc",2)==0) + { + // Process -sc before reading any file lists. + ProcessSwitch(Arg); + if (*LogName!=0) + InitLogOptions(LogName,ErrlogCharset); + } + } + else + if (*Command==0) + wcsncpy(Command,Arg,ASIZE(Command)); // Need for rar.ini. +} +#endif + + +#if !defined(SFX_MODULE) +void CommandData::ReadConfig() +{ + StringList List; + if (ReadTextFile(DefConfigName,&List,true)) + { + wchar *Str; + while ((Str=List.GetString())!=NULL) + { + while (IsSpace(*Str)) + Str++; + if (wcsnicomp(Str,L"switches=",9)==0) + ProcessSwitchesString(Str+9); + if (*Command!=0) + { + wchar Cmd[16]; + wcsncpyz(Cmd,Command,ASIZE(Cmd)); + wchar C0=toupperw(Cmd[0]); + wchar C1=toupperw(Cmd[1]); + if (C0=='I' || C0=='L' || C0=='M' || C0=='S' || C0=='V') + Cmd[1]=0; + if (C0=='R' && (C1=='R' || C1=='V')) + Cmd[2]=0; + wchar SwName[16+ASIZE(Cmd)]; + swprintf(SwName,ASIZE(SwName),L"switches_%s=",Cmd); + size_t Length=wcslen(SwName); + if (wcsnicomp(Str,SwName,Length)==0) + ProcessSwitchesString(Str+Length); + } + } + } +} +#endif + + +#if !defined(SFX_MODULE) +void CommandData::ProcessSwitchesString(const wchar *Str) +{ + wchar *Par; + while ((Str=AllocCmdParam(Str,&Par))!=NULL) + { + if (IsSwitch(*Par)) + ProcessSwitch(Par+1); + free(Par); + } +} +#endif + + +#if !defined(SFX_MODULE) +void CommandData::ProcessSwitch(const wchar *Switch) +{ + + switch(toupperw(Switch[0])) + { + case '@': + ListMode=Switch[1]=='+' ? RCLM_ACCEPT_LISTS:RCLM_REJECT_LISTS; + break; + case 'A': + switch(toupperw(Switch[1])) + { + case 'C': + ClearArc=true; + break; + case 'D': + AppendArcNameToPath=true; + break; +#ifndef SFX_MODULE + case 'G': + if (Switch[2]=='-' && Switch[3]==0) + GenerateArcName=0; + else + { + GenerateArcName=true; + wcsncpyz(GenerateMask,Switch+2,ASIZE(GenerateMask)); + } + break; +#endif + case 'I': + IgnoreGeneralAttr=true; + break; + case 'N': // Reserved for archive name. + break; + case 'O': + AddArcOnly=true; + break; + case 'P': + wcscpy(ArcPath,Switch+2); + break; + case 'S': + SyncFiles=true; + break; + default: + BadSwitch(Switch); + break; + } + break; + case 'C': + if (Switch[2]==0) + switch(toupperw(Switch[1])) + { + case '-': + DisableComment=true; + break; + case 'U': + ConvertNames=NAMES_UPPERCASE; + break; + case 'L': + ConvertNames=NAMES_LOWERCASE; + break; + } + break; + case 'D': + if (Switch[2]==0) + switch(toupperw(Switch[1])) + { + case 'S': + DisableSortSolid=true; + break; + case 'H': + OpenShared=true; + break; + case 'F': + DeleteFiles=true; + break; + } + break; + case 'E': + switch(toupperw(Switch[1])) + { + case 'P': + switch(Switch[2]) + { + case 0: + ExclPath=EXCL_SKIPWHOLEPATH; + break; + case '1': + ExclPath=EXCL_BASEPATH; + break; + case '2': + ExclPath=EXCL_SAVEFULLPATH; + break; + case '3': + ExclPath=EXCL_ABSPATH; + break; + } + break; + default: + if (Switch[1]=='+') + { + InclFileAttr|=GetExclAttr(Switch+2); + InclAttrSet=true; + } + else + ExclFileAttr|=GetExclAttr(Switch+1); + break; + } + break; + case 'F': + if (Switch[1]==0) + FreshFiles=true; + else + BadSwitch(Switch); + break; + case 'H': + switch (toupperw(Switch[1])) + { + case 'P': + EncryptHeaders=true; + if (Switch[2]!=0) + { + Password.Set(Switch+2); + cleandata((void *)Switch,wcslen(Switch)*sizeof(Switch[0])); + } + else + if (!Password.IsSet()) + { + uiGetPassword(UIPASSWORD_GLOBAL,NULL,&Password); + eprintf(L"\n"); + } + break; + default : + BadSwitch(Switch); + break; + } + break; + case 'I': + if (wcsnicomp(Switch+1,L"LOG",3)==0) + { + wcsncpyz(LogName,Switch[4]!=0 ? Switch+4:DefLogName,ASIZE(LogName)); + break; + } + if (wcsicomp(Switch+1,L"SND")==0) + { + Sound=true; + break; + } + if (wcsicomp(Switch+1,L"ERR")==0) + { + MsgStream=MSG_STDERR; + break; + } + if (wcsnicomp(Switch+1,L"EML",3)==0) + { + wcsncpyz(EmailTo,Switch[4]!=0 ? Switch+4:L"@",ASIZE(EmailTo)); + break; + } + if (wcsicomp(Switch+1,L"NUL")==0) + { + MsgStream=MSG_NULL; + break; + } + if (toupperw(Switch[1])=='D') + { + for (uint I=2;Switch[I]!=0;I++) + switch(toupperw(Switch[I])) + { + case 'Q': + MsgStream=MSG_ERRONLY; + break; + case 'C': + DisableCopyright=true; + break; + case 'D': + DisableDone=true; + break; + case 'P': + DisablePercentage=true; + break; + } + break; + } + if (wcsicomp(Switch+1,L"OFF")==0) + { + Shutdown=true; + break; + } + if (wcsicomp(Switch+1,L"VER")==0) + { + PrintVersion=true; + break; + } + break; + case 'K': + switch(toupperw(Switch[1])) + { + case 'B': + KeepBroken=true; + break; + case 0: + Lock=true; + break; + } + break; + case 'M': + switch(toupperw(Switch[1])) + { + case 'C': + { + const wchar *Str=Switch+2; + if (*Str=='-') + for (uint I=0;IMaxPoolThreads || Threads<1) + BadSwitch(Switch); + else + { + } + break; +#endif + default: + Method=Switch[1]-'0'; + if (Method>5 || Method<0) + BadSwitch(Switch); + break; + } + break; + case 'N': + case 'X': + if (Switch[1]!=0) + { + StringList *Args=toupperw(Switch[0])=='N' ? &InclArgs:&ExclArgs; + if (Switch[1]=='@' && !IsWildcard(Switch)) + { + RAR_CHARSET Charset=FilelistCharset; + +#if defined(_WIN_ALL) + // for compatibility reasons we use OEM encoding + // in Win32 console version by default + +// if (Charset==RCH_DEFAULT) +// Charset=RCH_OEM; +#endif + + ReadTextFile(Switch+2,Args,false,true,Charset,true,true,true); + } + else + Args->AddString(Switch+1); + } + break; + case 'O': + switch(toupperw(Switch[1])) + { + case '+': + Overwrite=OVERWRITE_ALL; + break; + case '-': + Overwrite=OVERWRITE_NONE; + break; + case 0: + Overwrite=OVERWRITE_FORCE_ASK; + break; +#ifdef _WIN_ALL + case 'C': + SetCompressedAttr=true; + break; +#endif + case 'H': + SaveHardLinks=true; + break; + + +#ifdef SAVE_LINKS + case 'L': + SaveSymLinks=true; + if (toupperw(Switch[2])=='A') + AbsoluteLinks=true; + break; +#endif +#ifdef _WIN_ALL + case 'N': + if (toupperw(Switch[2])=='I') + AllowIncompatNames=true; + break; +#endif + case 'R': + Overwrite=OVERWRITE_AUTORENAME; + break; +#ifdef _WIN_ALL + case 'S': + SaveStreams=true; + break; +#endif + case 'W': + ProcessOwners=true; + break; + default : + BadSwitch(Switch); + break; + } + break; + case 'P': + if (Switch[1]==0) + { + uiGetPassword(UIPASSWORD_GLOBAL,NULL,&Password); + eprintf(L"\n"); + } + else + { + Password.Set(Switch+1); + cleandata((void *)Switch,wcslen(Switch)*sizeof(Switch[0])); + } + break; +#ifndef SFX_MODULE + case 'Q': + if (toupperw(Switch[1])=='O') + switch(toupperw(Switch[2])) + { + case 0: + QOpenMode=QOPEN_AUTO; + break; + case '-': + QOpenMode=QOPEN_NONE; + break; + case '+': + QOpenMode=QOPEN_ALWAYS; + break; + default: + BadSwitch(Switch); + break; + } + else + BadSwitch(Switch); + break; +#endif + case 'R': + switch(toupperw(Switch[1])) + { + case 0: + Recurse=RECURSE_ALWAYS; + break; + case '-': + Recurse=RECURSE_DISABLE; + break; + case '0': + Recurse=RECURSE_WILDCARDS; + break; + case 'I': + { + Priority=atoiw(Switch+2); + if (Priority<0 || Priority>15) + BadSwitch(Switch); + const wchar *ChPtr=wcschr(Switch+2,':'); + if (ChPtr!=NULL) + { + SleepTime=atoiw(ChPtr+1); + if (SleepTime>1000) + BadSwitch(Switch); + InitSystemOptions(SleepTime); + } + SetPriority(Priority); + } + break; + } + break; + case 'S': + if (IsDigit(Switch[1])) + { + Solid|=SOLID_COUNT; + SolidCount=atoiw(&Switch[1]); + } + else + switch(toupperw(Switch[1])) + { + case 0: + Solid|=SOLID_NORMAL; + break; + case '-': + Solid=SOLID_NONE; + break; + case 'E': + Solid|=SOLID_FILEEXT; + break; + case 'V': + Solid|=Switch[2]=='-' ? SOLID_VOLUME_DEPENDENT:SOLID_VOLUME_INDEPENDENT; + break; + case 'D': + Solid|=SOLID_VOLUME_DEPENDENT; + break; + case 'L': + if (IsDigit(Switch[2])) + FileSizeLess=atoilw(Switch+2); + break; + case 'M': + if (IsDigit(Switch[2])) + FileSizeMore=atoilw(Switch+2); + break; + case 'C': + { + bool AlreadyBad=false; // Avoid reporting "bad switch" several times. + + RAR_CHARSET rch=RCH_DEFAULT; + switch(toupperw(Switch[2])) + { + case 'A': + rch=RCH_ANSI; + break; + case 'O': + rch=RCH_OEM; + break; + case 'U': + rch=RCH_UNICODE; + break; + case 'F': + rch=RCH_UTF8; + break; + default : + BadSwitch(Switch); + AlreadyBad=true; + break; + }; + if (!AlreadyBad) + if (Switch[3]==0) + CommentCharset=FilelistCharset=ErrlogCharset=RedirectCharset=rch; + else + for (uint I=3;Switch[I]!=0 && !AlreadyBad;I++) + switch(toupperw(Switch[I])) + { + case 'C': + CommentCharset=rch; + break; + case 'L': + FilelistCharset=rch; + break; + case 'R': + RedirectCharset=rch; + break; + default: + BadSwitch(Switch); + AlreadyBad=true; + break; + } + } + break; + + } + break; + case 'T': + switch(toupperw(Switch[1])) + { + case 'K': + ArcTime=ARCTIME_KEEP; + break; + case 'L': + ArcTime=ARCTIME_LATEST; + break; + case 'O': + FileTimeBefore.SetAgeText(Switch+2); + break; + case 'N': + FileTimeAfter.SetAgeText(Switch+2); + break; + case 'B': + FileTimeBefore.SetIsoText(Switch+2); + break; + case 'A': + FileTimeAfter.SetIsoText(Switch+2); + break; + case 'S': + { + EXTTIME_MODE Mode=EXTTIME_HIGH3; + bool CommonMode=Switch[2]>='0' && Switch[2]<='4'; + if (CommonMode) + Mode=(EXTTIME_MODE)(Switch[2]-'0'); + if (Mode==EXTTIME_HIGH1 || Mode==EXTTIME_HIGH2) // '2' and '3' not supported anymore. + Mode=EXTTIME_HIGH3; + if (Switch[2]=='-') + Mode=EXTTIME_NONE; + if (CommonMode || Switch[2]=='-' || Switch[2]=='+' || Switch[2]==0) + xmtime=xctime=xatime=Mode; + else + { + if (Switch[3]>='0' && Switch[3]<='4') + Mode=(EXTTIME_MODE)(Switch[3]-'0'); + if (Mode==EXTTIME_HIGH1 || Mode==EXTTIME_HIGH2) // '2' and '3' not supported anymore. + Mode=EXTTIME_HIGH3; + if (Switch[3]=='-') + Mode=EXTTIME_NONE; + switch(toupperw(Switch[2])) + { + case 'M': + xmtime=Mode; + break; + case 'C': + xctime=Mode; + break; + case 'A': + xatime=Mode; + break; + } + } + } + break; + case '-': + Test=false; + break; + case 0: + Test=true; + break; + default: + BadSwitch(Switch); + break; + } + break; + case 'U': + if (Switch[1]==0) + UpdateFiles=true; + else + BadSwitch(Switch); + break; + case 'V': + switch(toupperw(Switch[1])) + { + case 'P': + VolumePause=true; + break; + case 'E': + if (toupperw(Switch[2])=='R') + VersionControl=atoiw(Switch+3)+1; + break; + case '-': + VolSize=0; + break; + default: + VolSize=VOLSIZE_AUTO; // UnRAR -v switch for list command. + break; + } + break; + case 'W': + wcsncpyz(TempPath,Switch+1,ASIZE(TempPath)); + AddEndSlash(TempPath,ASIZE(TempPath)); + break; + case 'Y': + AllYes=true; + break; + case 'Z': + if (Switch[1]==0) + { + // If comment file is not specified, we read data from stdin. + wcscpy(CommentFile,L"stdin"); + } + else + wcsncpyz(CommentFile,Switch+1,ASIZE(CommentFile)); + break; + case '?' : + OutHelp(RARX_SUCCESS); + break; + default : + BadSwitch(Switch); + break; + } +} +#endif + + +#if !defined(SFX_MODULE) +void CommandData::BadSwitch(const wchar *Switch) +{ + mprintf(St(MUnknownOption),Switch); + ErrHandler.Exit(RARX_USERERROR); +} +#endif + + +void CommandData::OutTitle() +{ + if (BareOutput || DisableCopyright) + return; +#if defined(__GNUC__) && defined(SFX_MODULE) + mprintf(St(MCopyrightS)); +#else +#ifndef SILENT + static bool TitleShown=false; + if (TitleShown) + return; + TitleShown=true; + + wchar Version[80]; + if (RARVER_BETA!=0) + swprintf(Version,ASIZE(Version),L"%d.%02d %ls %d",RARVER_MAJOR,RARVER_MINOR,St(MBeta),RARVER_BETA); + else + swprintf(Version,ASIZE(Version),L"%d.%02d",RARVER_MAJOR,RARVER_MINOR); +#if defined(_WIN_32) || defined(_WIN_64) + wcsncatz(Version,L" ",ASIZE(Version)); +#endif +#ifdef _WIN_32 + wcsncatz(Version,St(Mx86),ASIZE(Version)); +#endif +#ifdef _WIN_64 + wcsncatz(Version,St(Mx64),ASIZE(Version)); +#endif + if (PrintVersion) + { + mprintf(L"%s",Version); + exit(0); + } + mprintf(St(MUCopyright),Version,RARVER_YEAR); +#endif +#endif +} + + +inline bool CmpMSGID(MSGID i1,MSGID i2) +{ +#ifdef MSGID_INT + return i1==i2; +#else + // If MSGID is const char*, we cannot compare pointers only. + // Pointers to different instances of same string can differ, + // so we need to compare complete strings. + return wcscmp(i1,i2)==0; +#endif +} + +void CommandData::OutHelp(RAR_EXIT ExitCode) +{ +#if !defined(SILENT) + OutTitle(); + static MSGID Help[]={ +#ifdef SFX_MODULE + // Console SFX switches definition. + MCHelpCmd,MSHelpCmdE,MSHelpCmdT,MSHelpCmdV +#else + // UnRAR switches definition. + MUNRARTitle1,MRARTitle2,MCHelpCmd,MCHelpCmdE,MCHelpCmdL, + MCHelpCmdP,MCHelpCmdT,MCHelpCmdV,MCHelpCmdX,MCHelpSw,MCHelpSwm, + MCHelpSwAT,MCHelpSwAC,MCHelpSwAD,MCHelpSwAG,MCHelpSwAI,MCHelpSwAP, + MCHelpSwCm,MCHelpSwCFGm,MCHelpSwCL,MCHelpSwCU, + MCHelpSwDH,MCHelpSwEP,MCHelpSwEP3,MCHelpSwF,MCHelpSwIDP,MCHelpSwIERR, + MCHelpSwINUL,MCHelpSwIOFF,MCHelpSwKB,MCHelpSwN,MCHelpSwNa,MCHelpSwNal, + MCHelpSwO,MCHelpSwOC,MCHelpSwOL,MCHelpSwOR,MCHelpSwOW,MCHelpSwP, + MCHelpSwPm,MCHelpSwR,MCHelpSwRI,MCHelpSwSC,MCHelpSwSL,MCHelpSwSM, + MCHelpSwTA,MCHelpSwTB,MCHelpSwTN,MCHelpSwTO,MCHelpSwTS,MCHelpSwU, + MCHelpSwVUnr,MCHelpSwVER,MCHelpSwVP,MCHelpSwX,MCHelpSwXa,MCHelpSwXal, + MCHelpSwY +#endif + }; + + for (uint I=0;IRewind(); + while (Args->GetString(CurMask,ASIZE(CurMask))) + { + wchar *LastMaskChar=PointToLastChar(CurMask); + bool DirMask=IsPathDiv(*LastMaskChar); // Mask for directories only. + + if (Dir) + { + // CheckName is a directory. + if (DirMask) + { + // We process the directory and have the directory exclusion mask. + // So let's convert "mask\" to "mask" and process it normally. + + *LastMaskChar=0; + } + else + { + // REMOVED, we want -npath\* to match empty folders too. + // If mask has wildcards in name part and does not have the trailing + // '\' character, we cannot use it for directories. + + // if (IsWildcard(PointToName(CurMask))) + // continue; + } + } + else + { + // If we process a file inside of directory excluded by "dirmask\". + // we want to exclude such file too. So we convert "dirmask\" to + // "dirmask\*". It is important for operations other than archiving + // with -x. When archiving with -x, directory matched by "dirmask\" + // is excluded from further scanning. + + if (DirMask) + wcsncatz(CurMask,L"*",ASIZE(CurMask)); + } + +#ifndef SFX_MODULE + if (CheckFullPath && IsFullPath(CurMask)) + { + // We do not need to do the special "*\" processing here, because + // unlike the "else" part of this "if", now we convert names to full + // format, so they all include the path, which is matched by "*\" + // correctly. Moreover, removing "*\" from mask would break + // the comparison, because now all names have the path. + + if (*FullName==0) + ConvertNameToFull(CheckName,FullName,ASIZE(FullName)); + if (CmpName(CurMask,FullName,MatchMode)) + return true; + } + else +#endif + { + wchar NewName[NM+2],*CurName=Name; + + // Important to convert before "*\" check below, so masks like + // d:*\something are processed properly. + wchar *CmpMask=ConvertPath(CurMask,NULL); + + if (CmpMask[0]=='*' && IsPathDiv(CmpMask[1])) + { + // We want "*\name" to match 'name' not only in subdirectories, + // but also in the current directory. We convert the name + // from 'name' to '.\name' to be matched by "*\" part even if it is + // in current directory. + NewName[0]='.'; + NewName[1]=CPATHDIVIDER; + wcsncpyz(NewName+2,Name,ASIZE(NewName)-2); + CurName=NewName; + } + + if (CmpName(CmpMask,CurName,MatchMode)) + return true; + } + } + return false; +} + + +#ifndef SFX_MODULE +// Now this function performs only one task and only in Windows version: +// it skips symlinks to directories if -e1024 switch is specified. +// Symlinks are skipped in ScanTree class, so their entire contents +// is skipped too. Without this function we would check the attribute +// only directly before archiving, so we would skip the symlink record, +// but not the contents of symlinked directory. +bool CommandData::ExclDirByAttr(uint FileAttr) +{ +#ifdef _WIN_ALL + if ((FileAttr & FILE_ATTRIBUTE_REPARSE_POINT)!=0 && + (ExclFileAttr & FILE_ATTRIBUTE_REPARSE_POINT)!=0) + return true; +#endif + return false; +} +#endif + + + + +#ifndef SFX_MODULE +// Return 'true' if we need to exclude the file from processing. +bool CommandData::TimeCheck(RarTime &ft) +{ + if (FileTimeBefore.IsSet() && ft>=FileTimeBefore) + return true; + if (FileTimeAfter.IsSet() && ft<=FileTimeAfter) + return true; + return false; +} +#endif + + +#ifndef SFX_MODULE +// Return 'true' if we need to exclude the file from processing. +bool CommandData::SizeCheck(int64 Size) +{ + if (FileSizeLess!=INT64NDF && Size>=FileSizeLess) + return(true); + if (FileSizeMore!=INT64NDF && Size<=FileSizeMore) + return(true); + return(false); +} +#endif + + + + +int CommandData::IsProcessFile(FileHeader &FileHead,bool *ExactMatch,int MatchType, + wchar *MatchedArg,uint MatchedArgSize) +{ + if (MatchedArg!=NULL && MatchedArgSize>0) + *MatchedArg=0; + if (wcslen(FileHead.FileName)>=NM) + return 0; + bool Dir=FileHead.Dir; + if (ExclCheck(FileHead.FileName,Dir,false,true)) + return 0; +#ifndef SFX_MODULE + if (TimeCheck(FileHead.mtime)) + return 0; + if ((FileHead.FileAttr & ExclFileAttr)!=0 || InclAttrSet && (FileHead.FileAttr & InclFileAttr)==0) + return 0; + if (!Dir && SizeCheck(FileHead.UnpSize)) + return 0; +#endif + wchar *ArgName; + FileArgs.Rewind(); + for (int StringCount=1;(ArgName=FileArgs.GetString())!=NULL;StringCount++) + if (CmpName(ArgName,FileHead.FileName,MatchType)) + { + if (ExactMatch!=NULL) + *ExactMatch=wcsicompc(ArgName,FileHead.FileName)==0; + if (MatchedArg!=NULL) + wcsncpyz(MatchedArg,ArgName,MatchedArgSize); + return StringCount; + } + return 0; +} + + +void CommandData::ProcessCommand() +{ +#ifndef SFX_MODULE + + const wchar *SingleCharCommands=L"FUADPXETK"; + if (Command[0]!=0 && Command[1]!=0 && wcschr(SingleCharCommands,Command[0])!=NULL || *ArcName==0) + OutHelp(*Command==0 ? RARX_SUCCESS:RARX_USERERROR); // Return 'success' for 'rar' without parameters. + + const wchar *ArcExt=GetExt(ArcName); +#ifdef _UNIX + if (ArcExt==NULL && (!FileExist(ArcName) || IsDir(GetFileAttr(ArcName)))) + wcsncatz(ArcName,L".rar",ASIZE(ArcName)); +#else + if (ArcExt==NULL) + wcsncatz(ArcName,L".rar",ASIZE(ArcName)); +#endif + // Treat arcname.part1 as arcname.part1.rar. + if (ArcExt!=NULL && wcsnicomp(ArcExt,L".part",5)==0 && IsDigit(ArcExt[5]) && + !FileExist(ArcName)) + { + wchar Name[NM]; + wcsncpyz(Name,ArcName,ASIZE(Name)); + wcsncatz(Name,L".rar",ASIZE(Name)); + if (FileExist(Name)) + wcsncpyz(ArcName,Name,ASIZE(ArcName)); + } + + if (wcschr(L"AFUMD",*Command)==NULL) + { + if (GenerateArcName) + GenerateArchiveName(ArcName,ASIZE(ArcName),GenerateMask,false); + + StringList ArcMasks; + ArcMasks.AddString(ArcName); + ScanTree Scan(&ArcMasks,Recurse,SaveSymLinks,SCAN_SKIPDIRS); + FindData FindData; + while (Scan.GetNext(&FindData)==SCAN_SUCCESS) + AddArcName(FindData.Name); + } + else + AddArcName(ArcName); +#endif + + switch(Command[0]) + { + case 'P': + case 'X': + case 'E': + case 'T': + case 'I': + { + CmdExtract Extract(this); + Extract.DoExtract(); + } + break; +#ifndef SILENT + case 'V': + case 'L': + ListArchive(this); + break; + default: + OutHelp(RARX_USERERROR); +#endif + } + if (!BareOutput) + mprintf(L"\n"); +} + + +void CommandData::AddArcName(const wchar *Name) +{ + ArcNames.AddString(Name); +} + + +bool CommandData::GetArcName(wchar *Name,int MaxSize) +{ + return ArcNames.GetString(Name,MaxSize); +} + + +bool CommandData::IsSwitch(int Ch) +{ +#if defined(_WIN_ALL) || defined(_EMX) + return Ch=='-' || Ch=='/'; +#else + return Ch=='-'; +#endif +} + + +#ifndef SFX_MODULE +uint CommandData::GetExclAttr(const wchar *Str) +{ + if (IsDigit(*Str)) + return wcstol(Str,NULL,0); + + uint Attr=0; + while (*Str!=0) + { + switch(toupperw(*Str)) + { +#ifdef _UNIX + case 'D': + Attr|=S_IFDIR; + break; + case 'V': + Attr|=S_IFCHR; + break; +#elif defined(_WIN_ALL) || defined(_EMX) + case 'R': + Attr|=0x1; + break; + case 'H': + Attr|=0x2; + break; + case 'S': + Attr|=0x4; + break; + case 'D': + Attr|=0x10; + break; + case 'A': + Attr|=0x20; + break; +#endif + } + Str++; + } + return Attr; +} +#endif + + + + +#ifndef SFX_MODULE +bool CommandData::CheckWinSize() +{ + // Define 0x100000000 as macro to avoid troubles with older compilers. + const uint64 MaxDictSize=INT32TO64(1,0); + // Limit the dictionary size to 4 GB. + for (uint64 I=0x10000;I<=MaxDictSize;I*=2) + if (WinSize==I) + return true; + WinSize=0x400000; + return false; +} +#endif + + +#ifndef SFX_MODULE +void CommandData::ReportWrongSwitches(RARFORMAT Format) +{ + if (Format==RARFMT15) + { + if (HashType!=HASH_CRC32) + uiMsg(UIERROR_INCOMPATSWITCH,L"-ht",4); +#ifdef _WIN_ALL + if (SaveSymLinks) + uiMsg(UIERROR_INCOMPATSWITCH,L"-ol",4); +#endif + if (SaveHardLinks) + uiMsg(UIERROR_INCOMPATSWITCH,L"-oh",4); + +#ifdef _WIN_ALL + // Do not report a wrong dictionary size here, because we are not sure + // yet about archive format. We can switch to RAR5 mode later + // if we update RAR5 archive. + + +#endif + if (QOpenMode!=QOPEN_AUTO) + uiMsg(UIERROR_INCOMPATSWITCH,L"-qo",4); + } + if (Format==RARFMT50) + { + } +} +#endif diff --git a/reference/unrar/cmddata.hpp b/reference/unrar/cmddata.hpp new file mode 100644 index 00000000..e12d4fbc --- /dev/null +++ b/reference/unrar/cmddata.hpp @@ -0,0 +1,64 @@ +#ifndef _RAR_CMDDATA_ +#define _RAR_CMDDATA_ + + +#define DefaultStoreList L"7z;ace;arj;bz2;cab;gz;jpeg;jpg;lha;lz;lzh;mp3;rar;taz;tgz;xz;z;zip;zipx" + +enum RAR_CMD_LIST_MODE {RCLM_AUTO,RCLM_REJECT_LISTS,RCLM_ACCEPT_LISTS}; + +class CommandData:public RAROptions +{ + private: + void ProcessSwitchesString(const wchar *Str); + void ProcessSwitch(const wchar *Switch); + void BadSwitch(const wchar *Switch); + uint GetExclAttr(const wchar *Str); + + bool FileLists; + bool NoMoreSwitches; + RAR_CMD_LIST_MODE ListMode; + bool BareOutput; + public: + CommandData(); + void Init(); + + void ParseCommandLine(bool Preprocess,int argc, char *argv[]); + void ParseArg(wchar *ArgW); + void ParseDone(); + void ParseEnvVar(); + void ReadConfig(); + void PreprocessArg(const wchar *Arg); + void OutTitle(); + void OutHelp(RAR_EXIT ExitCode); + bool IsSwitch(int Ch); + bool ExclCheck(const wchar *CheckName,bool Dir,bool CheckFullPath,bool CheckInclList); + static bool CheckArgs(StringList *Args,bool Dir,const wchar *CheckName,bool CheckFullPath,int MatchMode); + bool ExclDirByAttr(uint FileAttr); + bool TimeCheck(RarTime &ft); + bool SizeCheck(int64 Size); + bool AnyFiltersActive(); + int IsProcessFile(FileHeader &FileHead,bool *ExactMatch=NULL,int MatchType=MATCH_WILDSUBPATH, + wchar *MatchedArg=NULL,uint MatchedArgSize=0); + void ProcessCommand(); + void AddArcName(const wchar *Name); + bool GetArcName(wchar *Name,int MaxSize); + bool CheckWinSize(); + + int GetRecoverySize(const wchar *Str,int DefSize); + +#ifndef SFX_MODULE + void ReportWrongSwitches(RARFORMAT Format); +#endif + + wchar Command[NM+16]; + + wchar ArcName[NM]; + + StringList FileArgs; + StringList ExclArgs; + StringList InclArgs; + StringList ArcNames; + StringList StoreArgs; +}; + +#endif diff --git a/reference/unrar/coder.cpp b/reference/unrar/coder.cpp new file mode 100644 index 00000000..9d971a8a --- /dev/null +++ b/reference/unrar/coder.cpp @@ -0,0 +1,48 @@ + + +inline unsigned int RangeCoder::GetChar() +{ + return(UnpackRead->GetChar()); +} + + +void RangeCoder::InitDecoder(Unpack *UnpackRead) +{ + RangeCoder::UnpackRead=UnpackRead; + + low=code=0; + range=uint(-1); + for (int i=0;i < 4;i++) + code=(code << 8) | GetChar(); +} + + +// (int) cast before "low" added only to suppress compiler warnings. +#define ARI_DEC_NORMALIZE(code,low,range,read) \ +{ \ + while ((low^(low+range))GetChar(); \ + range <<= 8; \ + low <<= 8; \ + } \ +} + + +inline int RangeCoder::GetCurrentCount() +{ + return (code-low)/(range /= SubRange.scale); +} + + +inline uint RangeCoder::GetCurrentShiftCount(uint SHIFT) +{ + return (code-low)/(range >>= SHIFT); +} + + +inline void RangeCoder::Decode() +{ + low += range*SubRange.LowCount; + range *= SubRange.HighCount-SubRange.LowCount; +} diff --git a/reference/unrar/coder.hpp b/reference/unrar/coder.hpp new file mode 100644 index 00000000..7b36ff21 --- /dev/null +++ b/reference/unrar/coder.hpp @@ -0,0 +1,23 @@ +/**************************************************************************** + * Contents: 'Carryless rangecoder' by Dmitry Subbotin * + ****************************************************************************/ + + +class RangeCoder +{ + public: + void InitDecoder(Unpack *UnpackRead); + inline int GetCurrentCount(); + inline uint GetCurrentShiftCount(uint SHIFT); + inline void Decode(); + inline void PutChar(unsigned int c); + inline unsigned int GetChar(); + + uint low, code, range; + struct SUBRANGE + { + uint LowCount, HighCount, scale; + } SubRange; + + Unpack *UnpackRead; +}; diff --git a/reference/unrar/compress.hpp b/reference/unrar/compress.hpp new file mode 100644 index 00000000..8e4f1ed3 --- /dev/null +++ b/reference/unrar/compress.hpp @@ -0,0 +1,50 @@ +#ifndef _RAR_COMPRESS_ +#define _RAR_COMPRESS_ + +// Combine pack and unpack constants to class to avoid polluting global +// namespace with numerous short names. +class PackDef +{ + public: + static const uint MAX_LZ_MATCH = 0x1001; + static const uint MAX3_LZ_MATCH = 0x101; // Maximum match length for RAR v3. + static const uint LOW_DIST_REP_COUNT = 16; + + static const uint NC = 306; /* alphabet = {0, 1, 2, ..., NC - 1} */ + static const uint DC = 64; + static const uint LDC = 16; + static const uint RC = 44; + static const uint HUFF_TABLE_SIZE = NC + DC + RC + LDC; + static const uint BC = 20; + + static const uint NC30 = 299; /* alphabet = {0, 1, 2, ..., NC - 1} */ + static const uint DC30 = 60; + static const uint LDC30 = 17; + static const uint RC30 = 28; + static const uint BC30 = 20; + static const uint HUFF_TABLE_SIZE30 = NC30 + DC30 + RC30 + LDC30; + + static const uint NC20 = 298; /* alphabet = {0, 1, 2, ..., NC - 1} */ + static const uint DC20 = 48; + static const uint RC20 = 28; + static const uint BC20 = 19; + static const uint MC20 = 257; + + // Largest alphabet size among all values listed above. + static const uint LARGEST_TABLE_SIZE = 306; + + enum { + CODE_HUFFMAN, CODE_LZ, CODE_REPEATLZ, CODE_CACHELZ, CODE_STARTFILE, + CODE_ENDFILE, CODE_FILTER, CODE_FILTERDATA + }; +}; + + +enum FilterType { + // These values must not be changed, because we use them directly + // in RAR5 compression and decompression code. + FILTER_DELTA=0, FILTER_E8, FILTER_E8E9, FILTER_ARM, + FILTER_AUDIO, FILTER_RGB, FILTER_ITANIUM, FILTER_PPM, FILTER_NONE +}; + +#endif diff --git a/reference/unrar/consio.cpp b/reference/unrar/consio.cpp new file mode 100644 index 00000000..2f71a991 --- /dev/null +++ b/reference/unrar/consio.cpp @@ -0,0 +1,352 @@ +#include "rar.hpp" +#include "log.cpp" + +static MESSAGE_TYPE MsgStream=MSG_STDOUT; +static RAR_CHARSET RedirectCharset=RCH_DEFAULT; + +const int MaxMsgSize=2*NM+2048; + +static bool StdoutRedirected=false,StderrRedirected=false,StdinRedirected=false; + +#ifdef _WIN_ALL +static bool IsRedirected(DWORD nStdHandle) +{ + HANDLE hStd=GetStdHandle(nStdHandle); + DWORD Mode; + return GetFileType(hStd)!=FILE_TYPE_CHAR || GetConsoleMode(hStd,&Mode)==0; +} +#endif + + +void InitConsole() +{ +#ifdef _WIN_ALL + // We want messages like file names or progress percent to be printed + // immediately. Use only in Windows, in Unix they can cause wprintf %ls + // to fail with non-English strings. + setbuf(stdout,NULL); + setbuf(stderr,NULL); + + // Detect if output is redirected and set output mode properly. + // We do not want to send Unicode output to files and especially to pipes + // like '|more', which cannot handle them correctly in Windows. + // In Unix console output is UTF-8 and it is handled correctly + // when redirecting, so no need to perform any adjustments. + StdoutRedirected=IsRedirected(STD_OUTPUT_HANDLE); + StderrRedirected=IsRedirected(STD_ERROR_HANDLE); + StdinRedirected=IsRedirected(STD_INPUT_HANDLE); +#ifdef _MSC_VER + if (!StdoutRedirected) + _setmode(_fileno(stdout), _O_U16TEXT); + if (!StderrRedirected) + _setmode(_fileno(stderr), _O_U16TEXT); +#endif +#elif defined(_UNIX) + StdoutRedirected=!isatty(fileno(stdout)); + StderrRedirected=!isatty(fileno(stderr)); + StdinRedirected=!isatty(fileno(stdin)); +#endif +} + + +void InitConsoleOptions(MESSAGE_TYPE MsgStream,RAR_CHARSET RedirectCharset) +{ + ::MsgStream=MsgStream; + ::RedirectCharset=RedirectCharset; +} + + +#ifndef SILENT +static void cvt_wprintf(FILE *dest,const wchar *fmt,va_list arglist) +{ + // This buffer is for format string only, not for entire output, + // so it can be short enough. + wchar fmtw[1024]; + PrintfPrepareFmt(fmt,fmtw,ASIZE(fmtw)); +#ifdef _WIN_ALL + safebuf wchar Msg[MaxMsgSize]; + if (dest==stdout && StdoutRedirected || dest==stderr && StderrRedirected) + { + HANDLE hOut=GetStdHandle(dest==stdout ? STD_OUTPUT_HANDLE:STD_ERROR_HANDLE); + vswprintf(Msg,ASIZE(Msg),fmtw,arglist); + DWORD Written; + if (RedirectCharset==RCH_UNICODE) + WriteFile(hOut,Msg,(DWORD)wcslen(Msg)*sizeof(*Msg),&Written,NULL); + else + { + // Avoid Unicode for redirect in Windows, it does not work with pipes. + safebuf char MsgA[MaxMsgSize]; + if (RedirectCharset==RCH_UTF8) + WideToUtf(Msg,MsgA,ASIZE(MsgA)); + else + WideToChar(Msg,MsgA,ASIZE(MsgA)); + if (RedirectCharset==RCH_DEFAULT || RedirectCharset==RCH_OEM) + CharToOemA(MsgA,MsgA); // Console tools like 'more' expect OEM encoding. + + // We already converted \n to \r\n above, so we use WriteFile instead + // of C library to avoid unnecessary additional conversion. + WriteFile(hOut,MsgA,(DWORD)strlen(MsgA),&Written,NULL); + } + return; + } + // MSVC2008 vfwprintf writes every character to console separately + // and it is too slow. We use direct WriteConsole call instead. + vswprintf(Msg,ASIZE(Msg),fmtw,arglist); + HANDLE hOut=GetStdHandle(dest==stderr ? STD_ERROR_HANDLE:STD_OUTPUT_HANDLE); + DWORD Written; + WriteConsole(hOut,Msg,(DWORD)wcslen(Msg),&Written,NULL); +#else + vfwprintf(dest,fmtw,arglist); + // We do not use setbuf(NULL) in Unix (see comments in InitConsole). + fflush(dest); +#endif +} + + +void mprintf(const wchar *fmt,...) +{ + if (MsgStream==MSG_NULL || MsgStream==MSG_ERRONLY) + return; + + fflush(stderr); // Ensure proper message order. + + va_list arglist; + va_start(arglist,fmt); + FILE *dest=MsgStream==MSG_STDERR ? stderr:stdout; + cvt_wprintf(dest,fmt,arglist); + va_end(arglist); +} +#endif + + +#ifndef SILENT +void eprintf(const wchar *fmt,...) +{ + if (MsgStream==MSG_NULL) + return; + + fflush(stdout); // Ensure proper message order. + + va_list arglist; + va_start(arglist,fmt); + cvt_wprintf(stderr,fmt,arglist); + va_end(arglist); +} +#endif + + +#ifndef SILENT +static void GetPasswordText(wchar *Str,uint MaxLength) +{ + if (MaxLength==0) + return; + if (StdinRedirected) + getwstr(Str,MaxLength); // Read from pipe or redirected file. + else + { +#ifdef _WIN_ALL + HANDLE hConIn=GetStdHandle(STD_INPUT_HANDLE); + HANDLE hConOut=GetStdHandle(STD_OUTPUT_HANDLE); + DWORD ConInMode,ConOutMode; + DWORD Read=0; + GetConsoleMode(hConIn,&ConInMode); + GetConsoleMode(hConOut,&ConOutMode); + SetConsoleMode(hConIn,ENABLE_LINE_INPUT); + SetConsoleMode(hConOut,ENABLE_PROCESSED_OUTPUT|ENABLE_WRAP_AT_EOL_OUTPUT); + + ReadConsole(hConIn,Str,MaxLength-1,&Read,NULL); + Str[Read]=0; + SetConsoleMode(hConIn,ConInMode); + SetConsoleMode(hConOut,ConOutMode); +#else + char StrA[MAXPASSWORD]; +#if defined(_EMX) || defined (__VMS) + fgets(StrA,ASIZE(StrA)-1,stdin); +#elif defined(__sun) + strncpyz(StrA,getpassphrase(""),ASIZE(StrA)); +#else + strncpyz(StrA,getpass(""),ASIZE(StrA)); +#endif + CharToWide(StrA,Str,MaxLength); + cleandata(StrA,sizeof(StrA)); +#endif + } + Str[MaxLength-1]=0; + RemoveLF(Str); +} +#endif + + +#ifndef SILENT +bool GetConsolePassword(UIPASSWORD_TYPE Type,const wchar *FileName,SecPassword *Password) +{ + if (!StdinRedirected) + uiAlarm(UIALARM_QUESTION); + + while (true) + { + if (!StdinRedirected) + if (Type==UIPASSWORD_GLOBAL) + eprintf(L"\n%s: ",St(MAskPsw)); + else + eprintf(St(MAskPswFor),FileName); + + wchar PlainPsw[MAXPASSWORD]; + GetPasswordText(PlainPsw,ASIZE(PlainPsw)); + if (*PlainPsw==0 && Type==UIPASSWORD_GLOBAL) + return false; + if (!StdinRedirected && Type==UIPASSWORD_GLOBAL) + { + eprintf(St(MReAskPsw)); + wchar CmpStr[MAXPASSWORD]; + GetPasswordText(CmpStr,ASIZE(CmpStr)); + if (*CmpStr==0 || wcscmp(PlainPsw,CmpStr)!=0) + { + eprintf(St(MNotMatchPsw)); + cleandata(PlainPsw,sizeof(PlainPsw)); + cleandata(CmpStr,sizeof(CmpStr)); + continue; + } + cleandata(CmpStr,sizeof(CmpStr)); + } + Password->Set(PlainPsw); + cleandata(PlainPsw,sizeof(PlainPsw)); + break; + } + return true; +} +#endif + + +#ifndef SILENT +bool getwstr(wchar *str,size_t n) +{ + // Print buffered prompt title function before waiting for input. + fflush(stderr); + + *str=0; +#if defined(_WIN_ALL) + // fgetws does not work well with non-English text in Windows, + // so we do not use it. + if (StdinRedirected) // ReadConsole does not work if redirected. + { + // fgets does not work well with pipes in Windows in our test. + // Let's use files. + Array StrA(n*4); // Up to 4 UTF-8 characters per wchar_t. + File SrcFile; + SrcFile.SetHandleType(FILE_HANDLESTD); + int ReadSize=SrcFile.Read(&StrA[0],StrA.Size()-1); + if (ReadSize<=0) + { + // Looks like stdin is a null device. We can enter to infinite loop + // calling Ask(), so let's better exit. + ErrHandler.Exit(RARX_USERBREAK); + } + StrA[ReadSize]=0; + CharToWide(&StrA[0],str,n); + cleandata(&StrA[0],StrA.Size()); // We can use this function to enter passwords. + } + else + { + DWORD ReadSize=0; + if (ReadConsole(GetStdHandle(STD_INPUT_HANDLE),str,DWORD(n-1),&ReadSize,NULL)==0) + return false; + str[ReadSize]=0; + } +#else + if (fgetws(str,n,stdin)==NULL) + ErrHandler.Exit(RARX_USERBREAK); // Avoid infinite Ask() loop. +#endif + RemoveLF(str); + return true; +} +#endif + + +#ifndef SILENT +// We allow this function to return 0 in case of invalid input, +// because it might be convenient to press Enter to some not dangerous +// prompts like "insert disk with next volume". We should call this function +// again in case of 0 in dangerous prompt such as overwriting file. +int Ask(const wchar *AskStr) +{ + uiAlarm(UIALARM_QUESTION); + + const int MaxItems=10; + wchar Item[MaxItems][40]; + int ItemKeyPos[MaxItems],NumItems=0; + + for (const wchar *NextItem=AskStr;NextItem!=NULL;NextItem=wcschr(NextItem+1,'_')) + { + wchar *CurItem=Item[NumItems]; + wcsncpyz(CurItem,NextItem+1,ASIZE(Item[0])); + wchar *EndItem=wcschr(CurItem,'_'); + if (EndItem!=NULL) + *EndItem=0; + int KeyPos=0,CurKey; + while ((CurKey=CurItem[KeyPos])!=0) + { + bool Found=false; + for (int I=0;I4 ? L"\n":L" "):L", "); + int KeyPos=ItemKeyPos[I]; + for (int J=0;J[{key};"{string}"p used to redefine + // a keyboard key on some terminals. + if (Data[J]=='\"') + return true; + if (!IsDigit(Data[J]) && Data[J]!=';') + break; + } + return false; +} + + +void OutComment(const wchar *Comment,size_t Size) +{ + if (IsCommentUnsafe(Comment,Size)) + return; + const size_t MaxOutSize=0x400; + for (size_t I=0;I>1)^0xEDB88320 : (C>>1); + CRCTab[I]=C; + } +} + + +static void InitTables() +{ + InitCRC32(crc_tables[0]); + + for (uint I=0;I<256;I++) // Build additional lookup tables. + { + uint C=crc_tables[0][I]; + for (uint J=1;J<8;J++) + { + C=crc_tables[0][(byte)C]^(C>>8); + crc_tables[J][I]=C; + } + } +} + + +struct CallInitCRC {CallInitCRC() {InitTables();}} static CallInit32; + +uint CRC32(uint StartCRC,const void *Addr,size_t Size) +{ + byte *Data=(byte *)Addr; + + // Align Data to 8 for better performance. + for (;Size>0 && ((size_t)Data & 7);Size--,Data++) + StartCRC=crc_tables[0][(byte)(StartCRC^Data[0])]^(StartCRC>>8); + + for (;Size>=8;Size-=8,Data+=8) + { +#ifdef BIG_ENDIAN + StartCRC ^= Data[0]|(Data[1] << 8)|(Data[2] << 16)|(Data[3] << 24); + uint NextData = Data[4]|(Data[5] << 8)|(Data[6] << 16)|(Data[7] << 24); +#else + StartCRC ^= *(uint32 *) Data; + uint NextData = *(uint32 *) (Data +4); +#endif + StartCRC = crc_tables[7][(byte) StartCRC ] ^ + crc_tables[6][(byte)(StartCRC >> 8) ] ^ + crc_tables[5][(byte)(StartCRC >> 16)] ^ + crc_tables[4][(byte)(StartCRC >> 24)] ^ + crc_tables[3][(byte) NextData ] ^ + crc_tables[2][(byte)(NextData >>8 ) ] ^ + crc_tables[1][(byte)(NextData >> 16)] ^ + crc_tables[0][(byte)(NextData >> 24)]; + } + + for (;Size>0;Size--,Data++) // Process left data. + StartCRC=crc_tables[0][(byte)(StartCRC^Data[0])]^(StartCRC>>8); + + return StartCRC; +} + + +#ifndef SFX_MODULE +// For RAR 1.4 archives in case somebody still has them. +ushort Checksum14(ushort StartCRC,const void *Addr,size_t Size) +{ + byte *Data=(byte *)Addr; + for (size_t I=0;I>15))&0xffff; + } + return StartCRC; +} +#endif + + diff --git a/reference/unrar/crc.hpp b/reference/unrar/crc.hpp new file mode 100644 index 00000000..d8fea281 --- /dev/null +++ b/reference/unrar/crc.hpp @@ -0,0 +1,15 @@ +#ifndef _RAR_CRC_ +#define _RAR_CRC_ + +// This function is only to intialize external CRC tables. We do not need to +// call it before calculating CRC32. +void InitCRC32(uint *CRCTab); + +uint CRC32(uint StartCRC,const void *Addr,size_t Size); + +#ifndef SFX_MODULE +ushort Checksum14(ushort StartCRC,const void *Addr,size_t Size); +#endif + + +#endif diff --git a/reference/unrar/crypt.cpp b/reference/unrar/crypt.cpp new file mode 100644 index 00000000..fc2126d4 --- /dev/null +++ b/reference/unrar/crypt.cpp @@ -0,0 +1,134 @@ +#include "rar.hpp" + +#ifndef SFX_MODULE +#include "crypt1.cpp" +#include "crypt2.cpp" +#endif +#include "crypt3.cpp" +#include "crypt5.cpp" + + +CryptData::CryptData() +{ + Method=CRYPT_NONE; + memset(KDF3Cache,0,sizeof(KDF3Cache)); + memset(KDF5Cache,0,sizeof(KDF5Cache)); + KDF3CachePos=0; + KDF5CachePos=0; + memset(CRCTab,0,sizeof(CRCTab)); +} + + +CryptData::~CryptData() +{ + cleandata(KDF3Cache,sizeof(KDF3Cache)); + cleandata(KDF5Cache,sizeof(KDF5Cache)); +} + + + + +void CryptData::DecryptBlock(byte *Buf,size_t Size) +{ + switch(Method) + { +#ifndef SFX_MODULE + case CRYPT_RAR13: + Decrypt13(Buf,Size); + break; + case CRYPT_RAR15: + Crypt15(Buf,Size); + break; + case CRYPT_RAR20: + for (size_t I=0;IIsSet() || Method==CRYPT_NONE) + return false; + + CryptData::Method=Method; + + wchar PwdW[MAXPASSWORD]; + Password->Get(PwdW,ASIZE(PwdW)); + char PwdA[MAXPASSWORD]; + WideToChar(PwdW,PwdA,ASIZE(PwdA)); + + switch(Method) + { +#ifndef SFX_MODULE + case CRYPT_RAR13: + SetKey13(PwdA); + break; + case CRYPT_RAR15: + SetKey15(PwdA); + break; + case CRYPT_RAR20: + SetKey20(PwdA); + break; +#endif + case CRYPT_RAR30: + SetKey30(Encrypt,Password,PwdW,Salt); + break; + case CRYPT_RAR50: + SetKey50(Encrypt,Password,PwdW,Salt,InitV,Lg2Cnt,HashKey,PswCheck); + break; + } + cleandata(PwdA,sizeof(PwdA)); + cleandata(PwdW,sizeof(PwdW)); + return true; +} + + +// Use the current system time to additionally randomize data. +static void TimeRandomize(byte *RndBuf,size_t BufSize) +{ + static uint Count=0; + RarTime CurTime; + CurTime.SetCurrentTime(); + uint64 Random=CurTime.GetWin()+clock(); + for (size_t I=0;I> ( (I & 7) * 8 )); + RndBuf[I]=byte( (RndByte ^ I) + Count++); + } +} + + + + +// Fill buffer with random data. +void GetRnd(byte *RndBuf,size_t BufSize) +{ + bool Success=false; +#if defined(_WIN_ALL) + HCRYPTPROV hProvider = 0; + if (CryptAcquireContext(&hProvider, 0, 0, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT | CRYPT_SILENT)) + { + Success=CryptGenRandom(hProvider, (DWORD)BufSize, RndBuf) == TRUE; + CryptReleaseContext(hProvider, 0); + } +#elif defined(_UNIX) + FILE *rndf = fopen("/dev/urandom", "r"); + if (rndf!=NULL) + { + Success=fread(RndBuf, BufSize, 1, rndf) == BufSize; + fclose(rndf); + } +#endif + // We use this code only as the last resort if code above failed. + if (!Success) + TimeRandomize(RndBuf,BufSize); +} diff --git a/reference/unrar/crypt.hpp b/reference/unrar/crypt.hpp new file mode 100644 index 00000000..f6382ef5 --- /dev/null +++ b/reference/unrar/crypt.hpp @@ -0,0 +1,101 @@ +#ifndef _RAR_CRYPT_ +#define _RAR_CRYPT_ + + +enum CRYPT_METHOD { + CRYPT_NONE,CRYPT_RAR13,CRYPT_RAR15,CRYPT_RAR20,CRYPT_RAR30,CRYPT_RAR50 +}; + +#define SIZE_SALT50 16 +#define SIZE_SALT30 8 +#define SIZE_INITV 16 +#define SIZE_PSWCHECK 8 +#define SIZE_PSWCHECK_CSUM 4 + +#define CRYPT_BLOCK_SIZE 16 +#define CRYPT_BLOCK_MASK (CRYPT_BLOCK_SIZE-1) // 0xf + +#define CRYPT5_KDF_LG2_COUNT 15 // LOG2 of PDKDF2 iteration count. +#define CRYPT5_KDF_LG2_COUNT_MAX 24 // LOG2 of maximum accepted iteration count. +#define CRYPT_VERSION 0 // Supported encryption version. + + +class CryptData +{ + struct KDF5CacheItem + { + SecPassword Pwd; + byte Salt[SIZE_SALT50]; + byte Key[32]; + uint Lg2Count; // Log2 of PBKDF2 repetition count. + byte PswCheckValue[SHA256_DIGEST_SIZE]; + byte HashKeyValue[SHA256_DIGEST_SIZE]; + }; + + struct KDF3CacheItem + { + SecPassword Pwd; + byte Salt[SIZE_SALT30]; + byte Key[16]; + byte Init[16]; + bool SaltPresent; + }; + + + private: + void SetKey13(const char *Password); + void Decrypt13(byte *Data,size_t Count); + + void SetKey15(const char *Password); + void Crypt15(byte *Data,size_t Count); + + void SetKey20(const char *Password); + void Swap20(byte *Ch1,byte *Ch2); + void UpdKeys20(byte *Buf); + void EncryptBlock20(byte *Buf); + void DecryptBlock20(byte *Buf); + + void SetKey30(bool Encrypt,SecPassword *Password,const wchar *PwdW,const byte *Salt); + void SetKey50(bool Encrypt,SecPassword *Password,const wchar *PwdW,const byte *Salt,const byte *InitV,uint Lg2Cnt,byte *HashKey,byte *PswCheck); + + KDF3CacheItem KDF3Cache[4]; + uint KDF3CachePos; + + KDF5CacheItem KDF5Cache[4]; + uint KDF5CachePos; + + CRYPT_METHOD Method; + + Rijndael rin; + + uint CRCTab[256]; // For RAR 1.5 and RAR 2.0 encryption. + + byte SubstTable20[256]; + uint Key20[4]; + + byte Key13[3]; + ushort Key15[4]; + public: + CryptData(); + ~CryptData(); + bool SetCryptKeys(bool Encrypt,CRYPT_METHOD Method,SecPassword *Password, + const byte *Salt,const byte *InitV,uint Lg2Cnt, + byte *HashKey,byte *PswCheck); + void SetAV15Encryption(); + void SetCmt13Encryption(); + void EncryptBlock(byte *Buf,size_t Size); + void DecryptBlock(byte *Buf,size_t Size); + static void SetSalt(byte *Salt,size_t SaltSize); +}; + +void GetRnd(byte *RndBuf,size_t BufSize); + +void hmac_sha256(const byte *Key,size_t KeyLength,const byte *Data, + size_t DataLength,byte *ResDigest); +void pbkdf2(const byte *pass, size_t pass_len, const byte *salt, + size_t salt_len,byte *key, byte *Value1, byte *Value2, + uint rounds); + +void ConvertHashToMAC(HashValue *Value,byte *Key); + +#endif diff --git a/reference/unrar/crypt1.cpp b/reference/unrar/crypt1.cpp new file mode 100644 index 00000000..14263937 --- /dev/null +++ b/reference/unrar/crypt1.cpp @@ -0,0 +1,79 @@ +extern uint CRCTab[256]; + +void CryptData::SetKey13(const char *Password) +{ + Key13[0]=Key13[1]=Key13[2]=0; + for (size_t I=0;Password[I]!=0;I++) + { + byte P=Password[I]; + Key13[0]+=P; + Key13[1]^=P; + Key13[2]+=P; + Key13[2]=(byte)rotls(Key13[2],1,8); + } +} + + +void CryptData::SetKey15(const char *Password) +{ + InitCRC32(CRCTab); + uint PswCRC=CRC32(0xffffffff,Password,strlen(Password)); + Key15[0]=PswCRC&0xffff; + Key15[1]=(PswCRC>>16)&0xffff; + Key15[2]=Key15[3]=0; + for (size_t I=0;Password[I]!=0;I++) + { + byte P=Password[I]; + Key15[2]^=P^CRCTab[P]; + Key15[3]+=P+(CRCTab[P]>>16); + } +} + + +void CryptData::SetAV15Encryption() +{ + InitCRC32(CRCTab); + Method=CRYPT_RAR15; + Key15[0]=0x4765; + Key15[1]=0x9021; + Key15[2]=0x7382; + Key15[3]=0x5215; +} + + +void CryptData::SetCmt13Encryption() +{ + Method=CRYPT_RAR13; + Key13[0]=0; + Key13[1]=7; + Key13[2]=77; +} + + +void CryptData::Decrypt13(byte *Data,size_t Count) +{ + while (Count--) + { + Key13[1]+=Key13[2]; + Key13[0]+=Key13[1]; + *Data-=Key13[0]; + Data++; + } +} + + +void CryptData::Crypt15(byte *Data,size_t Count) +{ + while (Count--) + { + Key15[0]+=0x1234; + Key15[1]^=CRCTab[(Key15[0] & 0x1fe)>>1]; + Key15[2]-=CRCTab[(Key15[0] & 0x1fe)>>1]>>16; + Key15[0]^=Key15[2]; + Key15[3]=rotrs(Key15[3]&0xffff,1,16)^Key15[1]; + Key15[3]=rotrs(Key15[3]&0xffff,1,16); + Key15[0]^=Key15[3]; + *Data^=(byte)(Key15[0]>>8); + Data++; + } +} diff --git a/reference/unrar/crypt2.cpp b/reference/unrar/crypt2.cpp new file mode 100644 index 00000000..5fa4a97d --- /dev/null +++ b/reference/unrar/crypt2.cpp @@ -0,0 +1,133 @@ +#define NROUNDS 32 + +#define substLong(t) ( (uint)SubstTable20[(uint)t&255] | \ + ((uint)SubstTable20[(int)(t>> 8)&255]<< 8) | \ + ((uint)SubstTable20[(int)(t>>16)&255]<<16) | \ + ((uint)SubstTable20[(int)(t>>24)&255]<<24) ) + + +static byte InitSubstTable20[256]={ + 215, 19,149, 35, 73,197,192,205,249, 28, 16,119, 48,221, 2, 42, + 232, 1,177,233, 14, 88,219, 25,223,195,244, 90, 87,239,153,137, + 255,199,147, 70, 92, 66,246, 13,216, 40, 62, 29,217,230, 86, 6, + 71, 24,171,196,101,113,218,123, 93, 91,163,178,202, 67, 44,235, + 107,250, 75,234, 49,167,125,211, 83,114,157,144, 32,193,143, 36, + 158,124,247,187, 89,214,141, 47,121,228, 61,130,213,194,174,251, + 97,110, 54,229,115, 57,152, 94,105,243,212, 55,209,245, 63, 11, + 164,200, 31,156, 81,176,227, 21, 76, 99,139,188,127, 17,248, 51, + 207,120,189,210, 8,226, 41, 72,183,203,135,165,166, 60, 98, 7, + 122, 38,155,170, 69,172,252,238, 39,134, 59,128,236, 27,240, 80, + 131, 3, 85,206,145, 79,154,142,159,220,201,133, 74, 64, 20,129, + 224,185,138,103,173,182, 43, 34,254, 82,198,151,231,180, 58, 10, + 118, 26,102, 12, 50,132, 22,191,136,111,162,179, 45, 4,148,108, + 161, 56, 78,126,242,222, 15,175,146, 23, 33,241,181,190, 77,225, + 0, 46,169,186, 68, 95,237, 65, 53,208,253,168, 9, 18,100, 52, + 116,184,160, 96,109, 37, 30,106,140,104,150, 5,204,117,112, 84 +}; + + +void CryptData::SetKey20(const char *Password) +{ + InitCRC32(CRCTab); + + char Psw[MAXPASSWORD]; + strncpyz(Psw,Password,ASIZE(Psw)); // We'll need to modify it below. + size_t PswLength=strlen(Psw); + + Key20[0]=0xD3A3B879L; + Key20[1]=0x3F6D12F7L; + Key20[2]=0x7515A235L; + Key20[3]=0xA4E7F123L; + + memcpy(SubstTable20,InitSubstTable20,sizeof(SubstTable20)); + for (uint J=0;J<256;J++) + for (size_t I=0;I=0;I--) + { + T=((C+rotls(D,11,32))^Key20[I&3]); + TA=A^substLong(T); + T=((D^rotls(C,17,32))+Key20[I&3]); + TB=B^substLong(T); + A=C; + B=D; + C=TA; + D=TB; + } + RawPut4(C^Key20[0],Buf+0); + RawPut4(D^Key20[1],Buf+4); + RawPut4(A^Key20[2],Buf+8); + RawPut4(B^Key20[3],Buf+12); + UpdKeys20(InBuf); +} + + +void CryptData::UpdKeys20(byte *Buf) +{ + for (int I=0;I<16;I+=4) + { + Key20[0]^=CRCTab[Buf[I]]; + Key20[1]^=CRCTab[Buf[I+1]]; + Key20[2]^=CRCTab[Buf[I+2]]; + Key20[3]^=CRCTab[Buf[I+3]]; + } +} + + +void CryptData::Swap20(byte *Ch1,byte *Ch2) +{ + byte Ch=*Ch1; + *Ch1=*Ch2; + *Ch2=Ch; +} diff --git a/reference/unrar/crypt3.cpp b/reference/unrar/crypt3.cpp new file mode 100644 index 00000000..48406487 --- /dev/null +++ b/reference/unrar/crypt3.cpp @@ -0,0 +1,68 @@ +void CryptData::SetKey30(bool Encrypt,SecPassword *Password,const wchar *PwdW,const byte *Salt) +{ + byte AESKey[16],AESInit[16]; + + bool Cached=false; + for (uint I=0;I>8); + PswNum[2]=(byte)(I>>16); + sha1_process(&c, PswNum, 3); + if (I%(HashRounds/16)==0) + { + sha1_context tempc=c; + uint32 digest[5]; + sha1_done( &tempc, digest ); + AESInit[I/(HashRounds/16)]=(byte)digest[4]; + } + } + uint32 digest[5]; + sha1_done( &c, digest ); + for (int I=0;I<4;I++) + for (int J=0;J<4;J++) + AESKey[I*4+J]=(byte)(digest[I]>>(J*8)); + + KDF3Cache[KDF3CachePos].Pwd=*Password; + if ((KDF3Cache[KDF3CachePos].SaltPresent=(Salt!=NULL))==true) + memcpy(KDF3Cache[KDF3CachePos].Salt,Salt,SIZE_SALT30); + memcpy(KDF3Cache[KDF3CachePos].Key,AESKey,sizeof(AESKey)); + SecHideData(KDF3Cache[KDF3CachePos].Key,sizeof(KDF3Cache[KDF3CachePos].Key),true,false); + memcpy(KDF3Cache[KDF3CachePos].Init,AESInit,sizeof(AESInit)); + KDF3CachePos=(KDF3CachePos+1)%ASIZE(KDF3Cache); + + cleandata(RawPsw,sizeof(RawPsw)); + } + rin.Init(Encrypt, AESKey, 128, AESInit); + cleandata(AESKey,sizeof(AESKey)); + cleandata(AESInit,sizeof(AESInit)); +} + diff --git a/reference/unrar/crypt5.cpp b/reference/unrar/crypt5.cpp new file mode 100644 index 00000000..7562469f --- /dev/null +++ b/reference/unrar/crypt5.cpp @@ -0,0 +1,233 @@ +static void hmac_sha256(const byte *Key,size_t KeyLength,const byte *Data, + size_t DataLength,byte *ResDigest, + sha256_context *ICtxOpt,bool *SetIOpt, + sha256_context *RCtxOpt,bool *SetROpt) +{ + const size_t Sha256BlockSize=64; // As defined in RFC 4868. + + byte KeyHash[SHA256_DIGEST_SIZE]; + if (KeyLength > Sha256BlockSize) // Convert longer keys to key hash. + { + sha256_context KCtx; + sha256_init(&KCtx); + sha256_process(&KCtx, Key, KeyLength); + sha256_done(&KCtx, KeyHash); + + Key = KeyHash; + KeyLength = SHA256_DIGEST_SIZE; + } + + byte KeyBuf[Sha256BlockSize]; // Store the padded key here. + sha256_context ICtx; + + if (ICtxOpt!=NULL && *SetIOpt) + ICtx=*ICtxOpt; // Use already calculated first block context. + else + { + // This calculation is the same for all iterations with same password. + // So for PBKDF2 we can calculate it only for first block and then reuse + // to improve performance. + + for (size_t I = 0; I < KeyLength; I++) // Use 0x36 padding for inner digest. + KeyBuf[I] = Key[I] ^ 0x36; + for (size_t I = KeyLength; I < Sha256BlockSize; I++) + KeyBuf[I] = 0x36; + + sha256_init(&ICtx); + sha256_process(&ICtx, KeyBuf, Sha256BlockSize); // Hash padded key. + } + + if (ICtxOpt!=NULL && !*SetIOpt) // Store constant context for further reuse. + { + *ICtxOpt=ICtx; + *SetIOpt=true; + } + + sha256_process(&ICtx, Data, DataLength); // Hash data. + + byte IDig[SHA256_DIGEST_SIZE]; // Internal digest for padded key and data. + sha256_done(&ICtx, IDig); + + sha256_context RCtx; + + if (RCtxOpt!=NULL && *SetROpt) + RCtx=*RCtxOpt; // Use already calculated first block context. + else + { + // This calculation is the same for all iterations with same password. + // So for PBKDF2 we can calculate it only for first block and then reuse + // to improve performance. + + for (size_t I = 0; I < KeyLength; I++) // Use 0x5c for outer key padding. + KeyBuf[I] = Key[I] ^ 0x5c; + for (size_t I = KeyLength; I < Sha256BlockSize; I++) + KeyBuf[I] = 0x5c; + + sha256_init(&RCtx); + sha256_process(&RCtx, KeyBuf, Sha256BlockSize); // Hash padded key. + } + + if (RCtxOpt!=NULL && !*SetROpt) // Store constant context for further reuse. + { + *RCtxOpt=RCtx; + *SetROpt=true; + } + + sha256_process(&RCtx, IDig, SHA256_DIGEST_SIZE); // Hash internal digest. + + sha256_done(&RCtx, ResDigest); +} + + +// PBKDF2 for 32 byte key length. We generate the key for specified number +// of iteration count also as two supplementary values (key for checksums +// and password verification) for iterations+16 and iterations+32. +void pbkdf2(const byte *Pwd, size_t PwdLength, + const byte *Salt, size_t SaltLength, + byte *Key, byte *V1, byte *V2, uint Count) +{ + const size_t MaxSalt=64; + byte SaltData[MaxSalt+4]; + memcpy(SaltData, Salt, Min(SaltLength,MaxSalt)); + + SaltData[SaltLength + 0] = 0; // Salt concatenated to 1. + SaltData[SaltLength + 1] = 0; + SaltData[SaltLength + 2] = 0; + SaltData[SaltLength + 3] = 1; + + // First iteration: HMAC of password, salt and block index (1). + byte U1[SHA256_DIGEST_SIZE]; + hmac_sha256(Pwd, PwdLength, SaltData, SaltLength + 4, U1, NULL, NULL, NULL, NULL); + byte Fn[SHA256_DIGEST_SIZE]; // Current function value. + memcpy(Fn, U1, sizeof(Fn)); // Function at first iteration. + + uint CurCount[] = { Count-1, 16, 16 }; + byte *CurValue[] = { Key , V1, V2 }; + + sha256_context ICtxOpt,RCtxOpt; + bool SetIOpt=false,SetROpt=false; + + byte U2[SHA256_DIGEST_SIZE]; + for (uint I = 0; I < 3; I++) // For output key and 2 supplementary values. + { + for (uint J = 0; J < CurCount[I]; J++) + { + // U2 = PRF (P, U1). + hmac_sha256(Pwd, PwdLength, U1, sizeof(U1), U2, &ICtxOpt, &SetIOpt, &RCtxOpt, &SetROpt); + memcpy(U1, U2, sizeof(U1)); + for (uint K = 0; K < sizeof(Fn); K++) // Function ^= U. + Fn[K] ^= U1[K]; + } + memcpy(CurValue[I], Fn, SHA256_DIGEST_SIZE); + } + + cleandata(SaltData, sizeof(SaltData)); + cleandata(Fn, sizeof(Fn)); + cleandata(U1, sizeof(U1)); + cleandata(U2, sizeof(U2)); +} + + +void CryptData::SetKey50(bool Encrypt,SecPassword *Password,const wchar *PwdW, + const byte *Salt,const byte *InitV,uint Lg2Cnt,byte *HashKey, + byte *PswCheck) +{ + if (Lg2Cnt>CRYPT5_KDF_LG2_COUNT_MAX) + return; + + byte Key[32],PswCheckValue[SHA256_DIGEST_SIZE],HashKeyValue[SHA256_DIGEST_SIZE]; + bool Found=false; + for (uint I=0;ILg2Count==Lg2Cnt && Item->Pwd==*Password && + memcmp(Item->Salt,Salt,SIZE_SALT50)==0) + { + memcpy(Key,Item->Key,sizeof(Key)); + SecHideData(Key,sizeof(Key),false,false); + + memcpy(PswCheckValue,Item->PswCheckValue,sizeof(PswCheckValue)); + memcpy(HashKeyValue,Item->HashKeyValue,sizeof(HashKeyValue)); + Found=true; + break; + } + } + + if (!Found) + { + char PwdUtf[MAXPASSWORD*4]; + WideToUtf(PwdW,PwdUtf,ASIZE(PwdUtf)); + + pbkdf2((byte *)PwdUtf,strlen(PwdUtf),Salt,SIZE_SALT50,Key,HashKeyValue,PswCheckValue,(1<Lg2Count=Lg2Cnt; + Item->Pwd=*Password; + memcpy(Item->Salt,Salt,SIZE_SALT50); + memcpy(Item->Key,Key,sizeof(Item->Key)); + memcpy(Item->PswCheckValue,PswCheckValue,sizeof(PswCheckValue)); + memcpy(Item->HashKeyValue,HashKeyValue,sizeof(HashKeyValue)); + SecHideData(Item->Key,sizeof(Item->Key),true,false); + } + if (HashKey!=NULL) + memcpy(HashKey,HashKeyValue,SHA256_DIGEST_SIZE); + if (PswCheck!=NULL) + { + memset(PswCheck,0,SIZE_PSWCHECK); + for (uint I=0;IType==HASH_CRC32) + { + byte RawCRC[4]; + RawPut4(Value->CRC32,RawCRC); + byte Digest[SHA256_DIGEST_SIZE]; + hmac_sha256(Key,SHA256_DIGEST_SIZE,RawCRC,sizeof(RawCRC),Digest,NULL,NULL,NULL,NULL); + Value->CRC32=0; + for (uint I=0;ICRC32^=Digest[I] << ((I & 3) * 8); + } + if (Value->Type==HASH_BLAKE2) + { + byte Digest[BLAKE2_DIGEST_SIZE]; + hmac_sha256(Key,BLAKE2_DIGEST_SIZE,Value->Digest,sizeof(Value->Digest),Digest,NULL,NULL,NULL,NULL); + memcpy(Value->Digest,Digest,sizeof(Value->Digest)); + } +} + + +#if 0 +static void TestPBKDF2(); +struct TestKDF {TestKDF() {TestPBKDF2();exit(0);}} GlobalTestKDF; + +void TestPBKDF2() // Test PBKDF2 HMAC-SHA256 +{ + byte Key[32],V1[32],V2[32]; + + pbkdf2((byte *)"password", 8, (byte *)"salt", 4, Key, V1, V2, 1); + byte Res1[32]={0x12, 0x0f, 0xb6, 0xcf, 0xfc, 0xf8, 0xb3, 0x2c, 0x43, 0xe7, 0x22, 0x52, 0x56, 0xc4, 0xf8, 0x37, 0xa8, 0x65, 0x48, 0xc9, 0x2c, 0xcc, 0x35, 0x48, 0x08, 0x05, 0x98, 0x7c, 0xb7, 0x0b, 0xe1, 0x7b }; + mprintf(L"\nPBKDF2 test1: %s", memcmp(Key,Res1,32)==0 ? L"OK":L"Failed"); + + pbkdf2((byte *)"password", 8, (byte *)"salt", 4, Key, V1, V2, 4096); + byte Res2[32]={0xc5, 0xe4, 0x78, 0xd5, 0x92, 0x88, 0xc8, 0x41, 0xaa, 0x53, 0x0d, 0xb6, 0x84, 0x5c, 0x4c, 0x8d, 0x96, 0x28, 0x93, 0xa0, 0x01, 0xce, 0x4e, 0x11, 0xa4, 0x96, 0x38, 0x73, 0xaa, 0x98, 0x13, 0x4a }; + mprintf(L"\nPBKDF2 test2: %s", memcmp(Key,Res2,32)==0 ? L"OK":L"Failed"); + + pbkdf2((byte *)"just some long string pretending to be a password", 49, (byte *)"salt, salt, salt, a lot of salt", 31, Key, V1, V2, 65536); + byte Res3[32]={0x08, 0x0f, 0xa3, 0x1d, 0x42, 0x2d, 0xb0, 0x47, 0x83, 0x9b, 0xce, 0x3a, 0x3b, 0xce, 0x49, 0x51, 0xe2, 0x62, 0xb9, 0xff, 0x76, 0x2f, 0x57, 0xe9, 0xc4, 0x71, 0x96, 0xce, 0x4b, 0x6b, 0x6e, 0xbf}; + mprintf(L"\nPBKDF2 test3: %s", memcmp(Key,Res3,32)==0 ? L"OK":L"Failed"); +} +#endif diff --git a/reference/unrar/dll.cpp b/reference/unrar/dll.cpp new file mode 100644 index 00000000..d12c0b17 --- /dev/null +++ b/reference/unrar/dll.cpp @@ -0,0 +1,474 @@ +#include "rar.hpp" + +static int RarErrorToDll(RAR_EXIT ErrCode); + +struct DataSet +{ + CommandData Cmd; + Archive Arc; + CmdExtract Extract; + int OpenMode; + int HeaderSize; + + DataSet():Arc(&Cmd),Extract(&Cmd) {}; +}; + + +HANDLE PASCAL RAROpenArchive(struct RAROpenArchiveData *r) +{ + RAROpenArchiveDataEx rx; + memset(&rx,0,sizeof(rx)); + rx.ArcName=r->ArcName; + rx.OpenMode=r->OpenMode; + rx.CmtBuf=r->CmtBuf; + rx.CmtBufSize=r->CmtBufSize; + HANDLE hArc=RAROpenArchiveEx(&rx); + r->OpenResult=rx.OpenResult; + r->CmtSize=rx.CmtSize; + r->CmtState=rx.CmtState; + return hArc; +} + + +HANDLE PASCAL RAROpenArchiveEx(struct RAROpenArchiveDataEx *r) +{ + DataSet *Data=NULL; + try + { + r->OpenResult=0; + Data=new DataSet; + Data->Cmd.DllError=0; + Data->OpenMode=r->OpenMode; + Data->Cmd.FileArgs.AddString(L"*"); + + char AnsiArcName[NM]; + *AnsiArcName=0; + if (r->ArcName!=NULL) + { + strncpyz(AnsiArcName,r->ArcName,ASIZE(AnsiArcName)); +#ifdef _WIN_ALL + if (!AreFileApisANSI()) + { + OemToCharBuffA(r->ArcName,AnsiArcName,ASIZE(AnsiArcName)); + AnsiArcName[ASIZE(AnsiArcName)-1]=0; + } +#endif + } + + wchar ArcName[NM]; + GetWideName(AnsiArcName,r->ArcNameW,ArcName,ASIZE(ArcName)); + + Data->Cmd.AddArcName(ArcName); + Data->Cmd.Overwrite=OVERWRITE_ALL; + Data->Cmd.VersionControl=1; + + Data->Cmd.Callback=r->Callback; + Data->Cmd.UserData=r->UserData; + + // Open shared mode is added by request of dll users, who need to + // browse and unpack archives while downloading. + Data->Cmd.OpenShared = true; + if (!Data->Arc.Open(ArcName,FMF_OPENSHARED)) + { + r->OpenResult=ERAR_EOPEN; + delete Data; + return NULL; + } + if (!Data->Arc.IsArchive(true)) + { + if (Data->Cmd.DllError!=0) + r->OpenResult=Data->Cmd.DllError; + else + { + RAR_EXIT ErrCode=ErrHandler.GetErrorCode(); + if (ErrCode!=RARX_SUCCESS && ErrCode!=RARX_WARNING) + r->OpenResult=RarErrorToDll(ErrCode); + else + r->OpenResult=ERAR_BAD_ARCHIVE; + } + delete Data; + return NULL; + } + r->Flags=0; + + if (Data->Arc.Volume) + r->Flags|=0x01; + if (Data->Arc.Locked) + r->Flags|=0x04; + if (Data->Arc.Solid) + r->Flags|=0x08; + if (Data->Arc.NewNumbering) + r->Flags|=0x10; + if (Data->Arc.Signed) + r->Flags|=0x20; + if (Data->Arc.Protected) + r->Flags|=0x40; + if (Data->Arc.Encrypted) + r->Flags|=0x80; + if (Data->Arc.FirstVolume) + r->Flags|=0x100; + + Array CmtDataW; + if (r->CmtBufSize!=0 && Data->Arc.GetComment(&CmtDataW)) + { + Array CmtData(CmtDataW.Size()*4+1); + memset(&CmtData[0],0,CmtData.Size()); + WideToChar(&CmtDataW[0],&CmtData[0],CmtData.Size()-1); + size_t Size=strlen(&CmtData[0])+1; + + r->Flags|=2; + r->CmtState=Size>r->CmtBufSize ? ERAR_SMALL_BUF:1; + r->CmtSize=(uint)Min(Size,r->CmtBufSize); + memcpy(r->CmtBuf,&CmtData[0],r->CmtSize-1); + if (Size<=r->CmtBufSize) + r->CmtBuf[r->CmtSize-1]=0; + } + else + r->CmtState=r->CmtSize=0; + Data->Extract.ExtractArchiveInit(Data->Arc); + return (HANDLE)Data; + } + catch (RAR_EXIT ErrCode) + { + if (Data!=NULL && Data->Cmd.DllError!=0) + r->OpenResult=Data->Cmd.DllError; + else + r->OpenResult=RarErrorToDll(ErrCode); + if (Data != NULL) + delete Data; + return NULL; + } + catch (std::bad_alloc&) // Catch 'new' exception. + { + r->OpenResult=ERAR_NO_MEMORY; + if (Data != NULL) + delete Data; + } + return NULL; // To make compilers happy. +} + + +int PASCAL RARCloseArchive(HANDLE hArcData) +{ + DataSet *Data=(DataSet *)hArcData; + bool Success=Data==NULL ? false:Data->Arc.Close(); + delete Data; + return Success ? ERAR_SUCCESS : ERAR_ECLOSE; +} + + +int PASCAL RARReadHeader(HANDLE hArcData,struct RARHeaderData *D) +{ + struct RARHeaderDataEx X; + memset(&X,0,sizeof(X)); + + int Code=RARReadHeaderEx(hArcData,&X); + + strncpyz(D->ArcName,X.ArcName,ASIZE(D->ArcName)); + strncpyz(D->FileName,X.FileName,ASIZE(D->FileName)); + D->Flags=X.Flags; + D->PackSize=X.PackSize; + D->UnpSize=X.UnpSize; + D->HostOS=X.HostOS; + D->FileCRC=X.FileCRC; + D->FileTime=X.FileTime; + D->UnpVer=X.UnpVer; + D->Method=X.Method; + D->FileAttr=X.FileAttr; + D->CmtSize=0; + D->CmtState=0; + + return Code; +} + + +int PASCAL RARReadHeaderEx(HANDLE hArcData,struct RARHeaderDataEx *D) +{ + DataSet *Data=(DataSet *)hArcData; + try + { + if ((Data->HeaderSize=(int)Data->Arc.SearchBlock(HEAD_FILE))<=0) + { + if (Data->Arc.Volume && Data->Arc.GetHeaderType()==HEAD_ENDARC && + Data->Arc.EndArcHead.NextVolume) + if (MergeArchive(Data->Arc,NULL,false,'L')) + { + Data->Arc.Seek(Data->Arc.CurBlockPos,SEEK_SET); + return RARReadHeaderEx(hArcData,D); + } + else + return ERAR_EOPEN; + + if (Data->Arc.BrokenHeader) + return ERAR_BAD_DATA; + + // Might be necessary if RARSetPassword is still called instead of + // open callback for RAR5 archives and if password is invalid. + if (Data->Arc.FailedHeaderDecryption) + return ERAR_BAD_PASSWORD; + + return ERAR_END_ARCHIVE; + } + FileHeader *hd=&Data->Arc.FileHead; + if (Data->OpenMode==RAR_OM_LIST && hd->SplitBefore) + { + int Code=RARProcessFile(hArcData,RAR_SKIP,NULL,NULL); + if (Code==0) + return RARReadHeaderEx(hArcData,D); + else + return Code; + } + wcsncpy(D->ArcNameW,Data->Arc.FileName,ASIZE(D->ArcNameW)); + WideToChar(D->ArcNameW,D->ArcName,ASIZE(D->ArcName)); + + wcsncpy(D->FileNameW,hd->FileName,ASIZE(D->FileNameW)); + WideToChar(D->FileNameW,D->FileName,ASIZE(D->FileName)); +#ifdef _WIN_ALL + CharToOemA(D->FileName,D->FileName); +#endif + + D->Flags=0; + if (hd->SplitBefore) + D->Flags|=RHDF_SPLITBEFORE; + if (hd->SplitAfter) + D->Flags|=RHDF_SPLITAFTER; + if (hd->Encrypted) + D->Flags|=RHDF_ENCRYPTED; + if (hd->Solid) + D->Flags|=RHDF_SOLID; + if (hd->Dir) + D->Flags|=RHDF_DIRECTORY; + + D->PackSize=uint(hd->PackSize & 0xffffffff); + D->PackSizeHigh=uint(hd->PackSize>>32); + D->UnpSize=uint(hd->UnpSize & 0xffffffff); + D->UnpSizeHigh=uint(hd->UnpSize>>32); + D->HostOS=hd->HSType==HSYS_WINDOWS ? HOST_WIN32:HOST_UNIX; + if (Data->Arc.Format==RARFMT50) + D->UnpVer=Data->Arc.FileHead.UnpVer==0 ? 50 : 200; // If it is not 0, just set it to something big. + else + D->UnpVer=Data->Arc.FileHead.UnpVer; + D->FileCRC=hd->FileHash.CRC32; + D->FileTime=hd->mtime.GetDos(); + + uint64 MRaw=hd->mtime.GetWin(); + D->MtimeLow=(uint)MRaw; + D->MtimeHigh=(uint)(MRaw>>32); + uint64 CRaw=hd->ctime.GetWin(); + D->CtimeLow=(uint)CRaw; + D->CtimeHigh=(uint)(CRaw>>32); + uint64 ARaw=hd->atime.GetWin(); + D->AtimeLow=(uint)ARaw; + D->AtimeHigh=(uint)(ARaw>>32); + + D->Method=hd->Method+0x30; + D->FileAttr=hd->FileAttr; + D->CmtSize=0; + D->CmtState=0; + + D->DictSize=uint(hd->WinSize/1024); + + switch (hd->FileHash.Type) + { + case HASH_RAR14: + case HASH_CRC32: + D->HashType=RAR_HASH_CRC32; + break; + case HASH_BLAKE2: + D->HashType=RAR_HASH_BLAKE2; + memcpy(D->Hash,hd->FileHash.Digest,BLAKE2_DIGEST_SIZE); + break; + default: + D->HashType=RAR_HASH_NONE; + break; + } + + D->RedirType=hd->RedirType; + // RedirNameSize sanity check is useful in case some developer + // did not initialize Reserved area with 0 as required in docs. + // We have taken 'Redir*' fields from Reserved area. We may remove + // this RedirNameSize check sometimes later. + if (hd->RedirType!=FSREDIR_NONE && D->RedirName!=NULL && + D->RedirNameSize>0 && D->RedirNameSize<100000) + wcsncpyz(D->RedirName,hd->RedirName,D->RedirNameSize); + D->DirTarget=hd->DirTarget; + } + catch (RAR_EXIT ErrCode) + { + return Data->Cmd.DllError!=0 ? Data->Cmd.DllError : RarErrorToDll(ErrCode); + } + return ERAR_SUCCESS; +} + + +int PASCAL ProcessFile(HANDLE hArcData,int Operation,char *DestPath,char *DestName,wchar *DestPathW,wchar *DestNameW) +{ + DataSet *Data=(DataSet *)hArcData; + try + { + Data->Cmd.DllError=0; + if (Data->OpenMode==RAR_OM_LIST || Data->OpenMode==RAR_OM_LIST_INCSPLIT || + Operation==RAR_SKIP && !Data->Arc.Solid) + { + if (Data->Arc.Volume && Data->Arc.GetHeaderType()==HEAD_FILE && + Data->Arc.FileHead.SplitAfter) + if (MergeArchive(Data->Arc,NULL,false,'L')) + { + Data->Arc.Seek(Data->Arc.CurBlockPos,SEEK_SET); + return ERAR_SUCCESS; + } + else + return ERAR_EOPEN; + Data->Arc.SeekToNext(); + } + else + { + Data->Cmd.DllOpMode=Operation; + + *Data->Cmd.ExtrPath=0; + *Data->Cmd.DllDestName=0; + + if (DestPath!=NULL) + { + char ExtrPathA[NM]; + strncpyz(ExtrPathA,DestPath,ASIZE(ExtrPathA)-2); +#ifdef _WIN_ALL + // We must not apply OemToCharBuffA directly to DestPath, + // because we do not know DestPath length and OemToCharBuffA + // does not stop at 0. + OemToCharA(ExtrPathA,ExtrPathA); +#endif + CharToWide(ExtrPathA,Data->Cmd.ExtrPath,ASIZE(Data->Cmd.ExtrPath)); + AddEndSlash(Data->Cmd.ExtrPath,ASIZE(Data->Cmd.ExtrPath)); + } + if (DestName!=NULL) + { + char DestNameA[NM]; + strncpyz(DestNameA,DestName,ASIZE(DestNameA)-2); +#ifdef _WIN_ALL + // We must not apply OemToCharBuffA directly to DestName, + // because we do not know DestName length and OemToCharBuffA + // does not stop at 0. + OemToCharA(DestNameA,DestNameA); +#endif + CharToWide(DestNameA,Data->Cmd.DllDestName,ASIZE(Data->Cmd.DllDestName)); + } + + if (DestPathW!=NULL) + { + wcsncpy(Data->Cmd.ExtrPath,DestPathW,ASIZE(Data->Cmd.ExtrPath)); + AddEndSlash(Data->Cmd.ExtrPath,ASIZE(Data->Cmd.ExtrPath)); + } + + if (DestNameW!=NULL) + wcsncpyz(Data->Cmd.DllDestName,DestNameW,ASIZE(Data->Cmd.DllDestName)); + + wcscpy(Data->Cmd.Command,Operation==RAR_EXTRACT ? L"X":L"T"); + Data->Cmd.Test=Operation!=RAR_EXTRACT; + bool Repeat=false; + Data->Extract.ExtractCurrentFile(Data->Arc,Data->HeaderSize,Repeat); + + // Now we process extra file information if any. + // + // Archive can be closed if we process volumes, next volume is missing + // and current one is already removed or deleted. So we need to check + // if archive is still open to avoid calling file operations on + // the invalid file handle. Some of our file operations like Seek() + // process such invalid handle correctly, some not. + while (Data->Arc.IsOpened() && Data->Arc.ReadHeader()!=0 && + Data->Arc.GetHeaderType()==HEAD_SERVICE) + { + Data->Extract.ExtractCurrentFile(Data->Arc,Data->HeaderSize,Repeat); + Data->Arc.SeekToNext(); + } + Data->Arc.Seek(Data->Arc.CurBlockPos,SEEK_SET); + } + } + catch (std::bad_alloc&) + { + return ERAR_NO_MEMORY; + } + catch (RAR_EXIT ErrCode) + { + return Data->Cmd.DllError!=0 ? Data->Cmd.DllError : RarErrorToDll(ErrCode); + } + return Data->Cmd.DllError; +} + + +int PASCAL RARProcessFile(HANDLE hArcData,int Operation,char *DestPath,char *DestName) +{ + return(ProcessFile(hArcData,Operation,DestPath,DestName,NULL,NULL)); +} + + +int PASCAL RARProcessFileW(HANDLE hArcData,int Operation,wchar *DestPath,wchar *DestName) +{ + return(ProcessFile(hArcData,Operation,NULL,NULL,DestPath,DestName)); +} + + +void PASCAL RARSetChangeVolProc(HANDLE hArcData,CHANGEVOLPROC ChangeVolProc) +{ + DataSet *Data=(DataSet *)hArcData; + Data->Cmd.ChangeVolProc=ChangeVolProc; +} + + +void PASCAL RARSetCallback(HANDLE hArcData,UNRARCALLBACK Callback,LPARAM UserData) +{ + DataSet *Data=(DataSet *)hArcData; + Data->Cmd.Callback=Callback; + Data->Cmd.UserData=UserData; +} + + +void PASCAL RARSetProcessDataProc(HANDLE hArcData,PROCESSDATAPROC ProcessDataProc) +{ + DataSet *Data=(DataSet *)hArcData; + Data->Cmd.ProcessDataProc=ProcessDataProc; +} + + +#ifndef RAR_NOCRYPT +void PASCAL RARSetPassword(HANDLE hArcData,char *Password) +{ + DataSet *Data=(DataSet *)hArcData; + wchar PasswordW[MAXPASSWORD]; + GetWideName(Password,NULL,PasswordW,ASIZE(PasswordW)); + Data->Cmd.Password.Set(PasswordW); + cleandata(PasswordW,sizeof(PasswordW)); +} +#endif + + +int PASCAL RARGetDllVersion() +{ + return RAR_DLL_VERSION; +} + + +static int RarErrorToDll(RAR_EXIT ErrCode) +{ + switch(ErrCode) + { + case RARX_FATAL: + return ERAR_EREAD; + case RARX_CRC: + return ERAR_BAD_DATA; + case RARX_WRITE: + return ERAR_EWRITE; + case RARX_OPEN: + return ERAR_EOPEN; + case RARX_CREATE: + return ERAR_ECREATE; + case RARX_MEMORY: + return ERAR_NO_MEMORY; + case RARX_BADPWD: + return ERAR_BAD_PASSWORD; + case RARX_SUCCESS: + return ERAR_SUCCESS; // 0. + default: + return ERAR_UNKNOWN; + } +} diff --git a/reference/unrar/dll.def b/reference/unrar/dll.def new file mode 100644 index 00000000..54420e2a --- /dev/null +++ b/reference/unrar/dll.def @@ -0,0 +1,12 @@ +EXPORTS + RAROpenArchive + RAROpenArchiveEx + RARCloseArchive + RARReadHeader + RARReadHeaderEx + RARProcessFile + RARSetCallback + RARSetChangeVolProc + RARSetProcessDataProc + RARSetPassword + RARGetDllVersion diff --git a/reference/unrar/dll.hpp b/reference/unrar/dll.hpp new file mode 100644 index 00000000..7f829066 --- /dev/null +++ b/reference/unrar/dll.hpp @@ -0,0 +1,185 @@ +#ifndef _UNRAR_DLL_ +#define _UNRAR_DLL_ + +#pragma pack(1) + +#define ERAR_SUCCESS 0 +#define ERAR_END_ARCHIVE 10 +#define ERAR_NO_MEMORY 11 +#define ERAR_BAD_DATA 12 +#define ERAR_BAD_ARCHIVE 13 +#define ERAR_UNKNOWN_FORMAT 14 +#define ERAR_EOPEN 15 +#define ERAR_ECREATE 16 +#define ERAR_ECLOSE 17 +#define ERAR_EREAD 18 +#define ERAR_EWRITE 19 +#define ERAR_SMALL_BUF 20 +#define ERAR_UNKNOWN 21 +#define ERAR_MISSING_PASSWORD 22 +#define ERAR_EREFERENCE 23 +#define ERAR_BAD_PASSWORD 24 + +#define RAR_OM_LIST 0 +#define RAR_OM_EXTRACT 1 +#define RAR_OM_LIST_INCSPLIT 2 + +#define RAR_SKIP 0 +#define RAR_TEST 1 +#define RAR_EXTRACT 2 + +#define RAR_VOL_ASK 0 +#define RAR_VOL_NOTIFY 1 + +#define RAR_DLL_VERSION 8 + +#define RAR_HASH_NONE 0 +#define RAR_HASH_CRC32 1 +#define RAR_HASH_BLAKE2 2 + + +#ifdef _UNIX +#define CALLBACK +#define PASCAL +#define LONG long +#define HANDLE void * +#define LPARAM long +#define UINT unsigned int +#endif + +#define RHDF_SPLITBEFORE 0x01 +#define RHDF_SPLITAFTER 0x02 +#define RHDF_ENCRYPTED 0x04 +#define RHDF_SOLID 0x10 +#define RHDF_DIRECTORY 0x20 + + +struct RARHeaderData +{ + char ArcName[260]; + char FileName[260]; + unsigned int Flags; + unsigned int PackSize; + unsigned int UnpSize; + unsigned int HostOS; + unsigned int FileCRC; + unsigned int FileTime; + unsigned int UnpVer; + unsigned int Method; + unsigned int FileAttr; + char *CmtBuf; + unsigned int CmtBufSize; + unsigned int CmtSize; + unsigned int CmtState; +}; + + +struct RARHeaderDataEx +{ + char ArcName[1024]; + wchar_t ArcNameW[1024]; + char FileName[1024]; + wchar_t FileNameW[1024]; + unsigned int Flags; + unsigned int PackSize; + unsigned int PackSizeHigh; + unsigned int UnpSize; + unsigned int UnpSizeHigh; + unsigned int HostOS; + unsigned int FileCRC; + unsigned int FileTime; + unsigned int UnpVer; + unsigned int Method; + unsigned int FileAttr; + char *CmtBuf; + unsigned int CmtBufSize; + unsigned int CmtSize; + unsigned int CmtState; + unsigned int DictSize; + unsigned int HashType; + char Hash[32]; + unsigned int RedirType; + wchar_t *RedirName; + unsigned int RedirNameSize; + unsigned int DirTarget; + unsigned int MtimeLow; + unsigned int MtimeHigh; + unsigned int CtimeLow; + unsigned int CtimeHigh; + unsigned int AtimeLow; + unsigned int AtimeHigh; + unsigned int Reserved[988]; +}; + + +struct RAROpenArchiveData +{ + char *ArcName; + unsigned int OpenMode; + unsigned int OpenResult; + char *CmtBuf; + unsigned int CmtBufSize; + unsigned int CmtSize; + unsigned int CmtState; +}; + +typedef int (CALLBACK *UNRARCALLBACK)(UINT msg,LPARAM UserData,LPARAM P1,LPARAM P2); + +#define ROADF_VOLUME 0x0001 +#define ROADF_COMMENT 0x0002 +#define ROADF_LOCK 0x0004 +#define ROADF_SOLID 0x0008 +#define ROADF_NEWNUMBERING 0x0010 +#define ROADF_SIGNED 0x0020 +#define ROADF_RECOVERY 0x0040 +#define ROADF_ENCHEADERS 0x0080 +#define ROADF_FIRSTVOLUME 0x0100 + +struct RAROpenArchiveDataEx +{ + char *ArcName; + wchar_t *ArcNameW; + unsigned int OpenMode; + unsigned int OpenResult; + char *CmtBuf; + unsigned int CmtBufSize; + unsigned int CmtSize; + unsigned int CmtState; + unsigned int Flags; + UNRARCALLBACK Callback; + LPARAM UserData; + unsigned int Reserved[28]; +}; + +enum UNRARCALLBACK_MESSAGES { + UCM_CHANGEVOLUME,UCM_PROCESSDATA,UCM_NEEDPASSWORD,UCM_CHANGEVOLUMEW, + UCM_NEEDPASSWORDW +}; + +typedef int (PASCAL *CHANGEVOLPROC)(char *ArcName,int Mode); +typedef int (PASCAL *PROCESSDATAPROC)(unsigned char *Addr,int Size); + +#ifdef __cplusplus +extern "C" { +#endif + +HANDLE PASCAL RAROpenArchive(struct RAROpenArchiveData *ArchiveData); +HANDLE PASCAL RAROpenArchiveEx(struct RAROpenArchiveDataEx *ArchiveData); +int PASCAL RARCloseArchive(HANDLE hArcData); +int PASCAL RARReadHeader(HANDLE hArcData,struct RARHeaderData *HeaderData); +int PASCAL RARReadHeaderEx(HANDLE hArcData,struct RARHeaderDataEx *HeaderData); +int PASCAL RARProcessFile(HANDLE hArcData,int Operation,char *DestPath,char *DestName); +int PASCAL RARProcessFileW(HANDLE hArcData,int Operation,wchar_t *DestPath,wchar_t *DestName); +void PASCAL RARSetCallback(HANDLE hArcData,UNRARCALLBACK Callback,LPARAM UserData); +void PASCAL RARSetChangeVolProc(HANDLE hArcData,CHANGEVOLPROC ChangeVolProc); +void PASCAL RARSetProcessDataProc(HANDLE hArcData,PROCESSDATAPROC ProcessDataProc); +void PASCAL RARSetPassword(HANDLE hArcData,char *Password); +int PASCAL RARGetDllVersion(); + +#ifdef __cplusplus +} +#endif + +#pragma pack() + +#endif diff --git a/reference/unrar/dll.rc b/reference/unrar/dll.rc new file mode 100644 index 00000000..7aa83145 --- /dev/null +++ b/reference/unrar/dll.rc @@ -0,0 +1,28 @@ +#include +#include + +VS_VERSION_INFO VERSIONINFO +FILEVERSION 5, 50, 100, 2418 +PRODUCTVERSION 5, 50, 100, 2418 +FILEOS VOS__WINDOWS32 +FILETYPE VFT_APP +{ + BLOCK "StringFileInfo" + { + BLOCK "040904E4" + { + VALUE "CompanyName", "Alexander Roshal\0" + VALUE "ProductName", "RAR decompression library\0" + VALUE "FileDescription", "RAR decompression library\0" + VALUE "FileVersion", "5.50.0\0" + VALUE "ProductVersion", "5.50.0\0" + VALUE "LegalCopyright", "Copyright © Alexander Roshal 1993-2017\0" + VALUE "OriginalFilename", "Unrar.dll\0" + } + } + BLOCK "VarFileInfo" + { + VALUE "Translation", 0x0409, 0x04E4 + } +} + diff --git a/reference/unrar/encname.cpp b/reference/unrar/encname.cpp new file mode 100644 index 00000000..5556af3e --- /dev/null +++ b/reference/unrar/encname.cpp @@ -0,0 +1,69 @@ +#include "rar.hpp" + +EncodeFileName::EncodeFileName() +{ + Flags=0; + FlagBits=0; + FlagsPos=0; + DestSize=0; +} + + + + +void EncodeFileName::Decode(char *Name,byte *EncName,size_t EncSize,wchar *NameW, + size_t MaxDecSize) +{ + size_t EncPos=0,DecPos=0; + byte HighByte=EncPos=EncSize) + break; + Flags=EncName[EncPos++]; + FlagBits=8; + } + switch(Flags>>6) + { + case 0: + if (EncPos>=EncSize) + break; + NameW[DecPos++]=EncName[EncPos++]; + break; + case 1: + if (EncPos>=EncSize) + break; + NameW[DecPos++]=EncName[EncPos++]+(HighByte<<8); + break; + case 2: + if (EncPos+1>=EncSize) + break; + NameW[DecPos++]=EncName[EncPos]+(EncName[EncPos+1]<<8); + EncPos+=2; + break; + case 3: + { + if (EncPos>=EncSize) + break; + int Length=EncName[EncPos++]; + if ((Length & 0x80)!=0) + { + if (EncPos>=EncSize) + break; + byte Correction=EncName[EncPos++]; + for (Length=(Length&0x7f)+2;Length>0 && DecPos0 && DecPos1) + exit(RARX_USERBREAK); + // Otherwise return from signal handler and let Wait() function to close + // files and quit. We cannot use the same approach as in Windows, + // because Unix signal handler can block execution of our main code. +#endif + +#if defined(_WIN_ALL) && !defined(_MSC_VER) + // never reached, just to avoid a compiler warning + return TRUE; +#endif +} + + +void ErrorHandler::SetSignalHandlers(bool Enable) +{ + EnableBreak=Enable; +#ifdef _WIN_ALL + SetConsoleCtrlHandler(Enable ? ProcessSignal:NULL,TRUE); +#else + signal(SIGINT,Enable ? ProcessSignal:SIG_IGN); + signal(SIGTERM,Enable ? ProcessSignal:SIG_IGN); +#endif +} + + +void ErrorHandler::Throw(RAR_EXIT Code) +{ + if (Code==RARX_USERBREAK && !EnableBreak) + return; +#if !defined(SILENT) + // Do not write "aborted" when just displaying online help. + if (Code!=RARX_SUCCESS && Code!=RARX_USERERROR) + mprintf(L"\n%s\n",St(MProgAborted)); +#endif + SetErrorCode(Code); + throw Code; +} + + +void ErrorHandler::SysErrMsg() +{ +#if !defined(SFX_MODULE) && !defined(SILENT) +#ifdef _WIN_ALL + wchar *lpMsgBuf=NULL; + int ErrType=GetLastError(); + if (ErrType!=0 && FormatMessage(FORMAT_MESSAGE_ALLOCATE_BUFFER|FORMAT_MESSAGE_FROM_SYSTEM, + NULL,ErrType,MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), + (LPTSTR)&lpMsgBuf,0,NULL)) + { + wchar *CurMsg=lpMsgBuf; + while (CurMsg!=NULL) + { + while (*CurMsg=='\r' || *CurMsg=='\n') + CurMsg++; + if (*CurMsg==0) + break; + wchar *EndMsg=wcschr(CurMsg,'\r'); + if (EndMsg==NULL) + EndMsg=wcschr(CurMsg,'\n'); + if (EndMsg!=NULL) + { + *EndMsg=0; + EndMsg++; + } + uiMsg(UIERROR_SYSERRMSG,CurMsg); + CurMsg=EndMsg; + } + } + LocalFree( lpMsgBuf ); +#endif + +#if defined(_UNIX) || defined(_EMX) + if (errno!=0) + { + char *err=strerror(errno); + if (err!=NULL) + { + wchar Msg[1024]; + CharToWide(err,Msg,ASIZE(Msg)); + uiMsg(UIERROR_SYSERRMSG,Msg); + } + } +#endif + +#endif +} + + +int ErrorHandler::GetSystemErrorCode() +{ +#ifdef _WIN_ALL + return GetLastError(); +#else + return errno; +#endif +} + + +void ErrorHandler::SetSystemErrorCode(int Code) +{ +#ifdef _WIN_ALL + SetLastError(Code); +#else + errno=Code; +#endif +} diff --git a/reference/unrar/errhnd.hpp b/reference/unrar/errhnd.hpp new file mode 100644 index 00000000..26df96d5 --- /dev/null +++ b/reference/unrar/errhnd.hpp @@ -0,0 +1,70 @@ +#ifndef _RAR_ERRHANDLER_ +#define _RAR_ERRHANDLER_ + +enum RAR_EXIT // RAR exit code. +{ + RARX_SUCCESS = 0, + RARX_WARNING = 1, + RARX_FATAL = 2, + RARX_CRC = 3, + RARX_LOCK = 4, + RARX_WRITE = 5, + RARX_OPEN = 6, + RARX_USERERROR = 7, + RARX_MEMORY = 8, + RARX_CREATE = 9, + RARX_NOFILES = 10, + RARX_BADPWD = 11, + RARX_USERBREAK = 255 +}; + +class ErrorHandler +{ + private: + RAR_EXIT ExitCode; + uint ErrCount; + bool EnableBreak; + bool Silent; + bool DisableShutdown; // Shutdown is not suitable after last error. + public: + ErrorHandler(); + void Clean(); + void MemoryError(); + void OpenError(const wchar *FileName); + void CloseError(const wchar *FileName); + void ReadError(const wchar *FileName); + bool AskRepeatRead(const wchar *FileName); + void WriteError(const wchar *ArcName,const wchar *FileName); + void WriteErrorFAT(const wchar *FileName); + bool AskRepeatWrite(const wchar *FileName,bool DiskFull); + void SeekError(const wchar *FileName); + void GeneralErrMsg(const wchar *fmt,...); + void MemoryErrorMsg(); + void OpenErrorMsg(const wchar *FileName); + void OpenErrorMsg(const wchar *ArcName,const wchar *FileName); + void CreateErrorMsg(const wchar *FileName); + void CreateErrorMsg(const wchar *ArcName,const wchar *FileName); + void ReadErrorMsg(const wchar *FileName); + void ReadErrorMsg(const wchar *ArcName,const wchar *FileName); + void WriteErrorMsg(const wchar *ArcName,const wchar *FileName); + void ArcBrokenMsg(const wchar *ArcName); + void ChecksumFailedMsg(const wchar *ArcName,const wchar *FileName); + void UnknownMethodMsg(const wchar *ArcName,const wchar *FileName); + void Exit(RAR_EXIT ExitCode); + void SetErrorCode(RAR_EXIT Code); + RAR_EXIT GetErrorCode() {return ExitCode;} + uint GetErrorCount() {return ErrCount;} + void SetSignalHandlers(bool Enable); + void Throw(RAR_EXIT Code); + void SetSilent(bool Mode) {Silent=Mode;}; + void SysErrMsg(); + int GetSystemErrorCode(); + void SetSystemErrorCode(int Code); + bool IsShutdownEnabled() {return !DisableShutdown;} + + bool UserBreak; // Ctrl+Break is pressed. + bool MainExit; // main() is completed. +}; + + +#endif diff --git a/reference/unrar/extinfo.cpp b/reference/unrar/extinfo.cpp new file mode 100644 index 00000000..5cb90a40 --- /dev/null +++ b/reference/unrar/extinfo.cpp @@ -0,0 +1,178 @@ +#include "rar.hpp" + +#include "hardlinks.cpp" +#include "win32stm.cpp" + +#ifdef _WIN_ALL +#include "win32acl.cpp" +#include "win32lnk.cpp" +#endif + +#ifdef _UNIX +#include "uowners.cpp" +#ifdef SAVE_LINKS +#include "ulinks.cpp" +#endif +#endif + + + +// RAR2 service header extra records. +#ifndef SFX_MODULE +void SetExtraInfo20(CommandData *Cmd,Archive &Arc,wchar *Name) +{ + if (Cmd->Test) + return; + switch(Arc.SubBlockHead.SubType) + { +#ifdef _UNIX + case UO_HEAD: + if (Cmd->ProcessOwners) + ExtractUnixOwner20(Arc,Name); + break; +#endif +#ifdef _WIN_ALL + case NTACL_HEAD: + if (Cmd->ProcessOwners) + ExtractACL20(Arc,Name); + break; + case STREAM_HEAD: + ExtractStreams20(Arc,Name); + break; +#endif + } +} +#endif + + +// RAR3 and RAR5 service header extra records. +void SetExtraInfo(CommandData *Cmd,Archive &Arc,wchar *Name) +{ +#ifdef _UNIX + if (!Cmd->Test && Cmd->ProcessOwners && Arc.Format==RARFMT15 && + Arc.SubHead.CmpName(SUBHEAD_TYPE_UOWNER)) + ExtractUnixOwner30(Arc,Name); +#endif +#ifdef _WIN_ALL + if (!Cmd->Test && Cmd->ProcessOwners && Arc.SubHead.CmpName(SUBHEAD_TYPE_ACL)) + ExtractACL(Arc,Name); + if (Arc.SubHead.CmpName(SUBHEAD_TYPE_STREAM)) + ExtractStreams(Arc,Name,Cmd->Test); +#endif +} + + +// Extra data stored directly in file header. +void SetFileHeaderExtra(CommandData *Cmd,Archive &Arc,wchar *Name) +{ +#ifdef _UNIX + if (Cmd->ProcessOwners && Arc.Format==RARFMT50 && Arc.FileHead.UnixOwnerSet) + SetUnixOwner(Arc,Name); +#endif +} + + + + +// Calculate a number of path components except \. and \.. +static int CalcAllowedDepth(const wchar *Name) +{ + int AllowedDepth=0; + while (*Name!=0) + { + if (IsPathDiv(Name[0]) && Name[1]!=0 && !IsPathDiv(Name[1])) + { + bool Dot=Name[1]=='.' && (IsPathDiv(Name[2]) || Name[2]==0); + bool Dot2=Name[1]=='.' && Name[2]=='.' && (IsPathDiv(Name[3]) || Name[3]==0); + if (!Dot && !Dot2) + AllowedDepth++; + } + Name++; + } + return AllowedDepth; +} + + +// Check if all existing path components are directories and not links. +static bool LinkInPath(const wchar *Name) +{ + wchar Path[NM]; + if (wcslen(Name)>=ASIZE(Path)) + return true; // It should not be that long, skip. + wcsncpyz(Path,Name,ASIZE(Path)); + for (wchar *s=Path+wcslen(Path)-1;s>Path;s--) + if (IsPathDiv(*s)) + { + *s=0; + FindData FD; + if (FindFile::FastFind(Path,&FD,true) && (FD.IsLink || !FD.IsDir)) + return true; + } + return false; +} + + +bool IsRelativeSymlinkSafe(CommandData *Cmd,const wchar *SrcName,const wchar *PrepSrcName,const wchar *TargetName) +{ + // Catch root dir based /path/file paths also as stuff like \\?\. + // Do not check PrepSrcName here, it can be root based if destination path + // is a root based. + if (IsFullRootPath(SrcName) || IsFullRootPath(TargetName)) + return false; + + // Number of ".." in link target. + int UpLevels=0; + for (int Pos=0;*TargetName!=0;Pos++) + { + bool Dot2=TargetName[0]=='.' && TargetName[1]=='.' && + (IsPathDiv(TargetName[2]) || TargetName[2]==0) && + (Pos==0 || IsPathDiv(*(TargetName-1))); + if (Dot2) + UpLevels++; + TargetName++; + } + // If link target includes "..", it must not have another links + // in the path, because they can bypass our safety check. For example, + // suppose we extracted "lnk1" -> "." first and "lnk1/lnk2" -> ".." next + // or "dir/lnk1" -> ".." first and "dir/lnk1/lnk2" -> ".." next. + if (UpLevels>0 && LinkInPath(PrepSrcName)) + return false; + + // We could check just prepared src name, but for extra safety + // we check both original (as from archive header) and prepared + // (after applying the destination path and -ep switches) names. + + int AllowedDepth=CalcAllowedDepth(SrcName); // Original name depth. + + // Remove the destination path from prepared name if any. We should not + // count the destination path depth, because the link target must point + // inside of this path, not outside of it. + size_t ExtrPathLength=wcslen(Cmd->ExtrPath); + if (ExtrPathLength>0 && wcsncmp(PrepSrcName,Cmd->ExtrPath,ExtrPathLength)==0) + { + PrepSrcName+=ExtrPathLength; + while (IsPathDiv(*PrepSrcName)) + PrepSrcName++; + } + int PrepAllowedDepth=CalcAllowedDepth(PrepSrcName); + + return AllowedDepth>=UpLevels && PrepAllowedDepth>=UpLevels; +} + + +bool ExtractSymlink(CommandData *Cmd,ComprDataIO &DataIO,Archive &Arc,const wchar *LinkName) +{ +#if defined(SAVE_LINKS) && defined(_UNIX) + // For RAR 3.x archives we process links even in test mode to skip link data. + if (Arc.Format==RARFMT15) + return ExtractUnixLink30(Cmd,DataIO,Arc,LinkName); + if (Arc.Format==RARFMT50) + return ExtractUnixLink50(Cmd,LinkName,&Arc.FileHead); +#elif defined _WIN_ALL + // RAR 5.0 archives store link information in file header, so there is + // no need to additionally test it if we do not create a file. + if (Arc.Format==RARFMT50) + return CreateReparsePoint(Cmd,LinkName,&Arc.FileHead); +#endif + return false; +} diff --git a/reference/unrar/extinfo.hpp b/reference/unrar/extinfo.hpp new file mode 100644 index 00000000..2b0005da --- /dev/null +++ b/reference/unrar/extinfo.hpp @@ -0,0 +1,23 @@ +#ifndef _RAR_EXTINFO_ +#define _RAR_EXTINFO_ + +bool IsRelativeSymlinkSafe(CommandData *Cmd,const wchar *SrcName,const wchar *PrepSrcName,const wchar *TargetName); +bool ExtractSymlink(CommandData *Cmd,ComprDataIO &DataIO,Archive &Arc,const wchar *LinkName); +#ifdef _UNIX +void SetUnixOwner(Archive &Arc,const wchar *FileName); +#endif + +bool ExtractHardlink(wchar *NameNew,wchar *NameExisting,size_t NameExistingSize); + +void GetStreamNameNTFS(Archive &Arc,wchar *StreamName,size_t MaxSize); + +#ifdef _WIN_ALL +bool SetPrivilege(LPCTSTR PrivName); +#endif + +void SetExtraInfo20(CommandData *Cmd,Archive &Arc,wchar *Name); +void SetExtraInfo(CommandData *Cmd,Archive &Arc,wchar *Name); +void SetFileHeaderExtra(CommandData *Cmd,Archive &Arc,wchar *Name); + + +#endif diff --git a/reference/unrar/extract.cpp b/reference/unrar/extract.cpp new file mode 100644 index 00000000..f9461136 --- /dev/null +++ b/reference/unrar/extract.cpp @@ -0,0 +1,1167 @@ +#include "rar.hpp" + +CmdExtract::CmdExtract(CommandData *Cmd) +{ + CmdExtract::Cmd=Cmd; + + *ArcName=0; + + *DestFileName=0; + + TotalFileCount=0; + Unp=new Unpack(&DataIO); +#ifdef RAR_SMP + Unp->SetThreads(Cmd->Threads); +#endif +} + + +CmdExtract::~CmdExtract() +{ + delete Unp; +} + + +void CmdExtract::DoExtract() +{ +#if defined(_WIN_ALL) && !defined(SFX_MODULE) && !defined(SILENT) + Fat32=NotFat32=false; +#endif + PasswordCancelled=false; + DataIO.SetCurrentCommand(Cmd->Command[0]); + + FindData FD; + while (Cmd->GetArcName(ArcName,ASIZE(ArcName))) + if (FindFile::FastFind(ArcName,&FD)) + DataIO.TotalArcSize+=FD.Size; + + Cmd->ArcNames.Rewind(); + while (Cmd->GetArcName(ArcName,ASIZE(ArcName))) + { + if (Cmd->ManualPassword) + Cmd->Password.Clean(); // Clean user entered password before processing next archive. + while (true) + { + EXTRACT_ARC_CODE Code=ExtractArchive(); + if (Code!=EXTRACT_ARC_REPEAT) + break; + } + if (FindFile::FastFind(ArcName,&FD)) + DataIO.ProcessedArcSize+=FD.Size; + } + + // Clean user entered password. Not really required, just for extra safety. + if (Cmd->ManualPassword) + Cmd->Password.Clean(); + + if (TotalFileCount==0 && Cmd->Command[0]!='I' && + ErrHandler.GetErrorCode()!=RARX_BADPWD) // Not in case of wrong archive password. + { + if (!PasswordCancelled) + uiMsg(UIERROR_NOFILESTOEXTRACT,ArcName); + ErrHandler.SetErrorCode(RARX_NOFILES); + } + else + if (!Cmd->DisableDone) + if (Cmd->Command[0]=='I') + mprintf(St(MDone)); + else + if (ErrHandler.GetErrorCount()==0) + mprintf(St(MExtrAllOk)); + else + mprintf(St(MExtrTotalErr),ErrHandler.GetErrorCount()); +} + + +void CmdExtract::ExtractArchiveInit(Archive &Arc) +{ + DataIO.UnpArcSize=Arc.FileLength(); + + FileCount=0; + MatchedArgs=0; +#ifndef SFX_MODULE + FirstFile=true; +#endif + + PasswordAll=(Cmd->Password.IsSet()); + + DataIO.UnpVolume=false; + + PrevProcessed=false; + AllMatchesExact=true; + ReconstructDone=false; + AnySolidDataUnpackedWell=false; + + StartTime.SetCurrentTime(); +} + + +EXTRACT_ARC_CODE CmdExtract::ExtractArchive() +{ + Archive Arc(Cmd); + if (!Arc.WOpen(ArcName)) + return EXTRACT_ARC_NEXT; + + if (!Arc.IsArchive(true)) + { +#if !defined(SFX_MODULE) && !defined(RARDLL) + if (CmpExt(ArcName,L"rev")) + { + wchar FirstVolName[NM]; + VolNameToFirstName(ArcName,FirstVolName,ASIZE(FirstVolName),true); + + // If several volume names from same volume set are specified + // and current volume is not first in set and first volume is present + // and specified too, let's skip the current volume. + if (wcsicomp(ArcName,FirstVolName)!=0 && FileExist(FirstVolName) && + Cmd->ArcNames.Search(FirstVolName,false)) + return EXTRACT_ARC_NEXT; + RecVolumesTest(Cmd,NULL,ArcName); + TotalFileCount++; // Suppress "No files to extract" message. + return EXTRACT_ARC_NEXT; + } +#endif + + mprintf(St(MNotRAR),ArcName); + +#ifndef SFX_MODULE + if (CmpExt(ArcName,L"rar")) +#endif + ErrHandler.SetErrorCode(RARX_WARNING); + return EXTRACT_ARC_NEXT; + } + + if (Arc.FailedHeaderDecryption) // Bad archive password. + return EXTRACT_ARC_NEXT; + +#ifndef SFX_MODULE + if (Arc.Volume && !Arc.FirstVolume) + { + wchar FirstVolName[NM]; + VolNameToFirstName(ArcName,FirstVolName,ASIZE(FirstVolName),Arc.NewNumbering); + + // If several volume names from same volume set are specified + // and current volume is not first in set and first volume is present + // and specified too, let's skip the current volume. + if (wcsicomp(ArcName,FirstVolName)!=0 && FileExist(FirstVolName) && + Cmd->ArcNames.Search(FirstVolName,false)) + return EXTRACT_ARC_NEXT; + } +#endif + + int64 VolumeSetSize=0; // Total size of volumes after the current volume. + + if (Arc.Volume) + { + // Calculate the total size of all accessible volumes. + // This size is necessary to display the correct total progress indicator. + + wchar NextName[NM]; + wcscpy(NextName,Arc.FileName); + + while (true) + { + // First volume is already added to DataIO.TotalArcSize + // in initial TotalArcSize calculation in DoExtract. + // So we skip it and start from second volume. + NextVolumeName(NextName,ASIZE(NextName),!Arc.NewNumbering); + FindData FD; + if (FindFile::FastFind(NextName,&FD)) + VolumeSetSize+=FD.Size; + else + break; + } + DataIO.TotalArcSize+=VolumeSetSize; + } + + ExtractArchiveInit(Arc); + + if (*Cmd->Command=='T' || *Cmd->Command=='I') + Cmd->Test=true; + + + if (*Cmd->Command=='I') + { + Cmd->DisablePercentage=true; + } + else + uiStartArchiveExtract(!Cmd->Test,ArcName); + + Arc.ViewComment(); + + + while (1) + { + size_t Size=Arc.ReadHeader(); + + + bool Repeat=false; + if (!ExtractCurrentFile(Arc,Size,Repeat)) + if (Repeat) + { + // If we started extraction from not first volume and need to + // restart it from first, we must correct DataIO.TotalArcSize + // for correct total progress display. We subtract the size + // of current volume and all volumes after it and add the size + // of new (first) volume. + FindData OldArc,NewArc; + if (FindFile::FastFind(Arc.FileName,&OldArc) && + FindFile::FastFind(ArcName,&NewArc)) + DataIO.TotalArcSize-=VolumeSetSize+OldArc.Size-NewArc.Size; + return EXTRACT_ARC_REPEAT; + } + else + break; + } + + +#if !defined(SFX_MODULE) && !defined(RARDLL) + if (Cmd->Test && Arc.Volume) + RecVolumesTest(Cmd,&Arc,ArcName); +#endif + + return EXTRACT_ARC_NEXT; +} + + +bool CmdExtract::ExtractCurrentFile(Archive &Arc,size_t HeaderSize,bool &Repeat) +{ + // We can get negative sizes in corrupt archive and it is unacceptable + // for size comparisons in CmdExtract::UnstoreFile and ComprDataIO::UnpRead, + // where we cast sizes to size_t and can exceed another read or available + // size. We could fix it when reading an archive. But we prefer to do it + // here, because this function is called directly in unrar.dll, so we fix + // bad parameters passed to dll. Also we want to see real negative sizes + // in the listing of corrupt archive. + if (Arc.FileHead.PackSize<0) + Arc.FileHead.PackSize=0; + if (Arc.FileHead.UnpSize<0) + Arc.FileHead.UnpSize=0; + + wchar Command=Cmd->Command[0]; + if (HeaderSize==0) + if (DataIO.UnpVolume) + { +#ifdef NOVOLUME + return false; +#else + // Supposing we unpack an old RAR volume without end of archive record + // and last file is not split between volumes. + if (!MergeArchive(Arc,&DataIO,false,Command)) + { + ErrHandler.SetErrorCode(RARX_WARNING); + return false; + } +#endif + } + else + return false; + HEADER_TYPE HeaderType=Arc.GetHeaderType(); + if (HeaderType!=HEAD_FILE) + { +#ifndef SFX_MODULE + if (HeaderType==HEAD3_OLDSERVICE && PrevProcessed) + SetExtraInfo20(Cmd,Arc,DestFileName); +#endif + if (HeaderType==HEAD_SERVICE && PrevProcessed) + SetExtraInfo(Cmd,Arc,DestFileName); + if (HeaderType==HEAD_ENDARC) + if (Arc.EndArcHead.NextVolume) + { +#ifndef NOVOLUME + if (!MergeArchive(Arc,&DataIO,false,Command)) + { + ErrHandler.SetErrorCode(RARX_WARNING); + return false; + } +#endif + Arc.Seek(Arc.CurBlockPos,SEEK_SET); + return true; + } + else + return false; + Arc.SeekToNext(); + return true; + } + PrevProcessed=false; + + if (!Cmd->Recurse && MatchedArgs>=Cmd->FileArgs.ItemsCount() && AllMatchesExact) + return false; + + int MatchType=MATCH_WILDSUBPATH; + + bool EqualNames=false; + wchar MatchedArg[NM]; + int MatchNumber=Cmd->IsProcessFile(Arc.FileHead,&EqualNames,MatchType,MatchedArg,ASIZE(MatchedArg)); + bool MatchFound=MatchNumber!=0; +#ifndef SFX_MODULE + if (Cmd->ExclPath==EXCL_BASEPATH) + { + wcsncpyz(Cmd->ArcPath,MatchedArg,ASIZE(Cmd->ArcPath)); + *PointToName(Cmd->ArcPath)=0; + if (IsWildcard(Cmd->ArcPath)) // Cannot correctly process path*\* masks here. + *Cmd->ArcPath=0; + } +#endif + if (MatchFound && !EqualNames) + AllMatchesExact=false; + + Arc.ConvertAttributes(); + +#if !defined(SFX_MODULE) && !defined(RARDLL) + if (Arc.FileHead.SplitBefore && FirstFile) + { + wchar CurVolName[NM]; + wcsncpyz(CurVolName,ArcName,ASIZE(CurVolName)); + VolNameToFirstName(ArcName,ArcName,ASIZE(ArcName),Arc.NewNumbering); + + if (wcsicomp(ArcName,CurVolName)!=0 && FileExist(ArcName)) + { + // If first volume name does not match the current name and if such + // volume name really exists, let's unpack from this first volume. + Repeat=true; + return false; + } +#ifndef RARDLL + if (!ReconstructDone) + { + ReconstructDone=true; + if (RecVolumesRestore(Cmd,Arc.FileName,true)) + { + Repeat=true; + return false; + } + } +#endif + wcsncpyz(ArcName,CurVolName,ASIZE(ArcName)); + } +#endif + + wchar ArcFileName[NM]; + ConvertPath(Arc.FileHead.FileName,ArcFileName); + + if (Arc.FileHead.Version) + { + if (Cmd->VersionControl!=1 && !EqualNames) + { + if (Cmd->VersionControl==0) + MatchFound=false; + int Version=ParseVersionFileName(ArcFileName,false); + if (Cmd->VersionControl-1==Version) + ParseVersionFileName(ArcFileName,true); + else + MatchFound=false; + } + } + else + if (!Arc.IsArcDir() && Cmd->VersionControl>1) + MatchFound=false; + + DataIO.UnpVolume=Arc.FileHead.SplitAfter; + DataIO.NextVolumeMissing=false; + + Arc.Seek(Arc.NextBlockPos-Arc.FileHead.PackSize,SEEK_SET); + + bool ExtrFile=false; + bool SkipSolid=false; + +#ifndef SFX_MODULE + if (FirstFile && (MatchFound || Arc.Solid) && Arc.FileHead.SplitBefore) + { + if (MatchFound) + { + uiMsg(UIERROR_NEEDPREVVOL,Arc.FileName,ArcFileName); +#ifdef RARDLL + Cmd->DllError=ERAR_BAD_DATA; +#endif + ErrHandler.SetErrorCode(RARX_OPEN); + } + MatchFound=false; + } + + FirstFile=false; +#endif + + if (MatchFound || (SkipSolid=Arc.Solid)!=0) + { + // First common call of uiStartFileExtract. It is done before overwrite + // prompts, so if SkipSolid state is changed below, we'll need to make + // additional uiStartFileExtract calls with updated parameters. + if (!uiStartFileExtract(ArcFileName,!Cmd->Test,Cmd->Test && Command!='I',SkipSolid)) + return false; + + ExtrPrepareName(Arc,ArcFileName,DestFileName,ASIZE(DestFileName)); + + // DestFileName can be set empty in case of excessive -ap switch. + ExtrFile=!SkipSolid && *DestFileName!=0 && !Arc.FileHead.SplitBefore; + + if ((Cmd->FreshFiles || Cmd->UpdateFiles) && (Command=='E' || Command=='X')) + { + FindData FD; + if (FindFile::FastFind(DestFileName,&FD)) + { + if (FD.mtime >= Arc.FileHead.mtime) + { + // If directory already exists and its modification time is newer + // than start of extraction, it is likely it was created + // when creating a path to one of already extracted items. + // In such case we'll better update its time even if archived + // directory is older. + + if (!FD.IsDir || FD.mtimeFreshFiles) + ExtrFile=false; + } + + if (!CheckUnpVer(Arc,ArcFileName)) + { + ErrHandler.SetErrorCode(RARX_FATAL); +#ifdef RARDLL + Cmd->DllError=ERAR_UNKNOWN_FORMAT; +#endif + Arc.SeekToNext(); + return !Arc.Solid; // Can try extracting next file only in non-solid archive. + } + + while (true) // Repeat password prompt in case of wrong password here. + { + if (Arc.FileHead.Encrypted) + { + // Stop archive extracting if user cancelled a password prompt. +#ifdef RARDLL + if (!ExtrDllGetPassword()) + { + Cmd->DllError=ERAR_MISSING_PASSWORD; + return false; + } +#else + if (!ExtrGetPassword(Arc,ArcFileName)) + { + PasswordCancelled=true; + return false; + } +#endif + // Skip only the current encrypted file if empty password is entered. + // Actually our "cancel" code above intercepts empty passwords too now, + // so we keep the code below just in case we'll decide process empty + // and cancelled passwords differently sometimes. + if (!Cmd->Password.IsSet()) + { + ErrHandler.SetErrorCode(RARX_WARNING); +#ifdef RARDLL + Cmd->DllError=ERAR_MISSING_PASSWORD; +#endif + ExtrFile=false; + } + } + + + // Set a password before creating the file, so we can skip creating + // in case of wrong password. + SecPassword FilePassword=Cmd->Password; +#if defined(_WIN_ALL) && !defined(SFX_MODULE) + ConvertDosPassword(Arc,FilePassword); +#endif + + byte PswCheck[SIZE_PSWCHECK]; + DataIO.SetEncryption(false,Arc.FileHead.CryptMethod,&FilePassword, + Arc.FileHead.SaltSet ? Arc.FileHead.Salt:NULL, + Arc.FileHead.InitV,Arc.FileHead.Lg2Count, + Arc.FileHead.HashKey,PswCheck); + + // If header is damaged, we cannot rely on password check value, + // because it can be damaged too. + if (Arc.FileHead.Encrypted && Arc.FileHead.UsePswCheck && + memcmp(Arc.FileHead.PswCheck,PswCheck,SIZE_PSWCHECK)!=0 && + !Arc.BrokenHeader) + { + uiMsg(UIWAIT_BADPSW,ArcFileName); + + if (!PasswordAll) // If entered manually and not through -p. + { + Cmd->Password.Clean(); + continue; // Request a password again. + } +#ifdef RARDLL + // If we already have ERAR_EOPEN as result of missing volume, + // we should not replace it with less precise ERAR_BAD_PASSWORD. + if (Cmd->DllError!=ERAR_EOPEN) + Cmd->DllError=ERAR_BAD_PASSWORD; +#endif + ErrHandler.SetErrorCode(RARX_BADPWD); + ExtrFile=false; + } + break; + } + +#ifdef RARDLL + if (*Cmd->DllDestName!=0) + wcsncpyz(DestFileName,Cmd->DllDestName,ASIZE(DestFileName)); +#endif + + File CurFile; + + bool LinkEntry=Arc.FileHead.RedirType!=FSREDIR_NONE; + if (LinkEntry && Arc.FileHead.RedirType!=FSREDIR_FILECOPY) + { + if (ExtrFile && Command!='P' && !Cmd->Test) + { + // Overwrite prompt for symbolic and hard links. + bool UserReject=false; + if (FileExist(DestFileName) && !UserReject) + FileCreate(Cmd,NULL,DestFileName,ASIZE(DestFileName),&UserReject,Arc.FileHead.UnpSize,&Arc.FileHead.mtime); + if (UserReject) + ExtrFile=false; + } + } + else + if (Arc.IsArcDir()) + { + if (!ExtrFile || Command=='P' || Command=='I' || Command=='E' || Cmd->ExclPath==EXCL_SKIPWHOLEPATH) + return true; + TotalFileCount++; + ExtrCreateDir(Arc,ArcFileName); + // It is important to not increment MatchedArgs here, so we extract + // dir with its entire contents and not dir record only even if + // dir record precedes files. + return true; + } + else + if (ExtrFile) // Create files and file copies (FSREDIR_FILECOPY). + ExtrFile=ExtrCreateFile(Arc,CurFile); + + if (!ExtrFile && Arc.Solid) + { + SkipSolid=true; + ExtrFile=true; + + // We changed SkipSolid, so we need to call uiStartFileExtract + // with "Skip" parameter to change the operation status + // from "extracting" to "skipping". For example, it can be necessary + // if user answered "No" to overwrite prompt when unpacking + // a solid archive. + if (!uiStartFileExtract(ArcFileName,false,false,true)) + return false; + } + if (ExtrFile) + { + // Set it in test mode, so we also test subheaders such as NTFS streams + // after tested file. + if (Cmd->Test) + PrevProcessed=true; + + bool TestMode=Cmd->Test || SkipSolid; // Unpack to memory, not to disk. + + if (!SkipSolid) + { + if (!TestMode && Command!='P' && CurFile.IsDevice()) + { + uiMsg(UIERROR_INVALIDNAME,Arc.FileName,DestFileName); + ErrHandler.WriteError(Arc.FileName,DestFileName); + } + TotalFileCount++; + } + FileCount++; + if (Command!='I') + if (SkipSolid) + mprintf(St(MExtrSkipFile),ArcFileName); + else + switch(Cmd->Test ? 'T':Command) // "Test" can be also enabled by -t switch. + { + case 'T': + mprintf(St(MExtrTestFile),ArcFileName); + break; +#ifndef SFX_MODULE + case 'P': + mprintf(St(MExtrPrinting),ArcFileName); + break; +#endif + case 'X': + case 'E': + mprintf(St(MExtrFile),DestFileName); + break; + } + if (!Cmd->DisablePercentage) + mprintf(L" "); + + DataIO.CurUnpRead=0; + DataIO.CurUnpWrite=0; + DataIO.UnpHash.Init(Arc.FileHead.FileHash.Type,Cmd->Threads); + DataIO.PackedDataHash.Init(Arc.FileHead.FileHash.Type,Cmd->Threads); + DataIO.SetPackedSizeToRead(Arc.FileHead.PackSize); + DataIO.SetFiles(&Arc,&CurFile); + DataIO.SetTestMode(TestMode); + DataIO.SetSkipUnpCRC(SkipSolid); + +#if defined(_WIN_ALL) && !defined(SFX_MODULE) && !defined(SILENT) + if (!TestMode && !Arc.BrokenHeader && + Arc.FileHead.UnpSize>0xffffffff && (Fat32 || !NotFat32)) + { + if (!Fat32) // Not detected yet. + NotFat32=!(Fat32=IsFAT(Cmd->ExtrPath)); + if (Fat32) + uiMsg(UIMSG_FAT32SIZE); // Inform user about FAT32 size limit. + } +#endif + + if (!TestMode && !Arc.BrokenHeader && + (Arc.FileHead.PackSize<<11)>Arc.FileHead.UnpSize && + (Arc.FileHead.UnpSize<100000000 || Arc.FileLength()>Arc.FileHead.PackSize)) + CurFile.Prealloc(Arc.FileHead.UnpSize); + + CurFile.SetAllowDelete(!Cmd->KeepBroken); + + bool FileCreateMode=!TestMode && !SkipSolid && Command!='P'; + bool ShowChecksum=true; // Display checksum verification result. + + bool LinkSuccess=true; // Assume success for test mode. + if (LinkEntry) + { + FILE_SYSTEM_REDIRECT Type=Arc.FileHead.RedirType; + + if (Type==FSREDIR_HARDLINK || Type==FSREDIR_FILECOPY) + { + wchar NameExisting[NM]; + ExtrPrepareName(Arc,Arc.FileHead.RedirName,NameExisting,ASIZE(NameExisting)); + if (FileCreateMode && *NameExisting!=0) // *NameExisting can be 0 in case of excessive -ap switch. + if (Type==FSREDIR_HARDLINK) + LinkSuccess=ExtractHardlink(DestFileName,NameExisting,ASIZE(NameExisting)); + else + LinkSuccess=ExtractFileCopy(CurFile,Arc.FileName,DestFileName,NameExisting,ASIZE(NameExisting)); + } + else + if (Type==FSREDIR_UNIXSYMLINK || Type==FSREDIR_WINSYMLINK || Type==FSREDIR_JUNCTION) + { + if (FileCreateMode) + LinkSuccess=ExtractSymlink(Cmd,DataIO,Arc,DestFileName); + } + else + { + uiMsg(UIERROR_UNKNOWNEXTRA, Arc.FileName, DestFileName); + LinkSuccess=false; + } + + if (!LinkSuccess || Arc.Format==RARFMT15 && !FileCreateMode) + { + // RAR 5.x links have a valid data checksum even in case of + // failure, because they do not store any data. + // We do not want to display "OK" in this case. + // For 4.x symlinks we verify the checksum only when extracting, + // but not when testing an archive. + ShowChecksum=false; + } + PrevProcessed=FileCreateMode && LinkSuccess; + } + else + if (!Arc.FileHead.SplitBefore) + if (Arc.FileHead.Method==0) + UnstoreFile(DataIO,Arc.FileHead.UnpSize); + else + { + Unp->Init(Arc.FileHead.WinSize,Arc.FileHead.Solid); + Unp->SetDestSize(Arc.FileHead.UnpSize); +#ifndef SFX_MODULE + if (Arc.Format!=RARFMT50 && Arc.FileHead.UnpVer<=15) + Unp->DoUnpack(15,FileCount>1 && Arc.Solid); + else +#endif + Unp->DoUnpack(Arc.FileHead.UnpVer,Arc.FileHead.Solid); + } + + Arc.SeekToNext(); + + // We check for "split after" flag to detect partially extracted files + // from incomplete volume sets. For them file header contains packed + // data hash, which must not be compared against unpacked data hash + // to prevent accidental match. Moreover, for -m0 volumes packed data + // hash would match truncated unpacked data hash and lead to fake "OK" + // in incomplete volume set. + bool ValidCRC=!Arc.FileHead.SplitAfter && DataIO.UnpHash.Cmp(&Arc.FileHead.FileHash,Arc.FileHead.UseHashKey ? Arc.FileHead.HashKey:NULL); + + // We set AnySolidDataUnpackedWell to true if we found at least one + // valid non-zero solid file in preceding solid stream. If it is true + // and if current encrypted file is broken, we do not need to hint + // about a wrong password and can report CRC error only. + if (!Arc.FileHead.Solid) + AnySolidDataUnpackedWell=false; // Reset the flag, because non-solid file is found. + else + if (Arc.FileHead.Method!=0 && Arc.FileHead.UnpSize>0 && ValidCRC) + AnySolidDataUnpackedWell=true; + + bool BrokenFile=false; + + // Checksum is not calculated in skip solid mode for performance reason. + if (!SkipSolid && ShowChecksum) + { + if (ValidCRC) + { + if (Command!='P' && Command!='I') + mprintf(L"%s%s ",Cmd->DisablePercentage ? L" ":L"\b\b\b\b\b ", + Arc.FileHead.FileHash.Type==HASH_NONE ? L" ?":St(MOk)); + } + else + { + if (Arc.FileHead.Encrypted && (!Arc.FileHead.UsePswCheck || + Arc.BrokenHeader) && !AnySolidDataUnpackedWell) + uiMsg(UIERROR_CHECKSUMENC,Arc.FileName,ArcFileName); + else + uiMsg(UIERROR_CHECKSUM,Arc.FileName,ArcFileName); + BrokenFile=true; + ErrHandler.SetErrorCode(RARX_CRC); +#ifdef RARDLL + // If we already have ERAR_EOPEN as result of missing volume + // or ERAR_BAD_PASSWORD for RAR5 wrong password, + // we should not replace it with less precise ERAR_BAD_DATA. + if (Cmd->DllError!=ERAR_EOPEN && Cmd->DllError!=ERAR_BAD_PASSWORD) + Cmd->DllError=ERAR_BAD_DATA; +#endif + } + } + else + mprintf(L"\b\b\b\b\b "); + + if (!TestMode && (Command=='X' || Command=='E') && + (!LinkEntry || Arc.FileHead.RedirType==FSREDIR_FILECOPY && LinkSuccess) && + (!BrokenFile || Cmd->KeepBroken)) + { + // We could preallocate more space that really written to broken file. + if (BrokenFile) + CurFile.Truncate(); + +#if defined(_WIN_ALL) || defined(_EMX) + if (Cmd->ClearArc) + Arc.FileHead.FileAttr&=~FILE_ATTRIBUTE_ARCHIVE; +#endif + + + CurFile.SetOpenFileTime( + Cmd->xmtime==EXTTIME_NONE ? NULL:&Arc.FileHead.mtime, + Cmd->xctime==EXTTIME_NONE ? NULL:&Arc.FileHead.ctime, + Cmd->xatime==EXTTIME_NONE ? NULL:&Arc.FileHead.atime); + CurFile.Close(); +#if defined(_WIN_ALL) && !defined(SFX_MODULE) + if (Cmd->SetCompressedAttr && + (Arc.FileHead.FileAttr & FILE_ATTRIBUTE_COMPRESSED)!=0) + SetFileCompression(CurFile.FileName,true); +#endif + SetFileHeaderExtra(Cmd,Arc,CurFile.FileName); + + CurFile.SetCloseFileTime( + Cmd->xmtime==EXTTIME_NONE ? NULL:&Arc.FileHead.mtime, + Cmd->xatime==EXTTIME_NONE ? NULL:&Arc.FileHead.atime); + if (!Cmd->IgnoreGeneralAttr && !SetFileAttr(CurFile.FileName,Arc.FileHead.FileAttr)) + uiMsg(UIERROR_FILEATTR,Arc.FileName,CurFile.FileName); + + PrevProcessed=true; + } + } + } + // It is important to increment it for files, but not dirs. So we extract + // dir with its entire contents, not just dir record only even if dir + // record precedes files. + if (MatchFound) + MatchedArgs++; + if (DataIO.NextVolumeMissing) + return false; + if (!ExtrFile) + if (!Arc.Solid) + Arc.SeekToNext(); + else + if (!SkipSolid) + return false; + return true; +} + + +void CmdExtract::UnstoreFile(ComprDataIO &DataIO,int64 DestUnpSize) +{ + // 512 KB and larger buffer reported to reduce performance on old XP + // computers with WDC WD2000JD HDD. According to test made by user + // 256 KB buffer is optimal. + Array Buffer(0x40000); + while (1) + { + uint Code=DataIO.UnpRead(&Buffer[0],Buffer.Size()); + if (Code==0 || (int)Code==-1) + break; + Code=Code=0) + DestUnpSize-=Code; + } +} + + +bool CmdExtract::ExtractFileCopy(File &New,wchar *ArcName,wchar *NameNew,wchar *NameExisting,size_t NameExistingSize) +{ + SlashToNative(NameExisting,NameExisting,NameExistingSize); // Not needed for RAR 5.1+ archives. + + File Existing; + if (!Existing.WOpen(NameExisting)) + { + uiMsg(UIERROR_FILECOPY,ArcName,NameExisting,NameNew); + uiMsg(UIERROR_FILECOPYHINT,ArcName); +#ifdef RARDLL + Cmd->DllError=ERAR_EREFERENCE; +#endif + return false; + } + + Array Buffer(0x100000); + int64 CopySize=0; + + while (true) + { + Wait(); + int ReadSize=Existing.Read(&Buffer[0],Buffer.Size()); + if (ReadSize==0) + break; + New.Write(&Buffer[0],ReadSize); + CopySize+=ReadSize; + } + + return true; +} + + +void CmdExtract::ExtrPrepareName(Archive &Arc,const wchar *ArcFileName,wchar *DestName,size_t DestSize) +{ + wcsncpyz(DestName,Cmd->ExtrPath,DestSize); + + if (*Cmd->ExtrPath!=0) + { + wchar LastChar=*PointToLastChar(Cmd->ExtrPath); + // We need IsPathDiv check here to correctly handle Unix forward slash + // in the end of destination path in Windows: rar x arc dest/ + // IsDriveDiv is needed for current drive dir: rar x arc d: + if (!IsPathDiv(LastChar) && !IsDriveDiv(LastChar)) + { + // Destination path can be without trailing slash if it come from GUI shell. + AddEndSlash(DestName,DestSize); + } + } + +#ifndef SFX_MODULE + if (Cmd->AppendArcNameToPath) + { + wcsncatz(DestName,PointToName(Arc.FirstVolumeName),DestSize); + SetExt(DestName,NULL,DestSize); + AddEndSlash(DestName,DestSize); + } +#endif + +#ifndef SFX_MODULE + size_t ArcPathLength=wcslen(Cmd->ArcPath); + if (ArcPathLength>0) + { + size_t NameLength=wcslen(ArcFileName); + + // Earlier we compared lengths only here, but then noticed a cosmetic bug + // in WinRAR. When extracting a file reference from subfolder with + // "Extract relative paths", so WinRAR sets ArcPath, if reference target + // is missing, error message removed ArcPath both from reference and target + // names. If target was stored in another folder, its name looked wrong. + if (NameLength>=ArcPathLength && + wcsnicompc(Cmd->ArcPath,ArcFileName,ArcPathLength)==0 && + (IsPathDiv(Cmd->ArcPath[ArcPathLength-1]) || + IsPathDiv(ArcFileName[ArcPathLength]) || ArcFileName[ArcPathLength]==0)) + { + ArcFileName+=Min(ArcPathLength,NameLength); + while (IsPathDiv(*ArcFileName)) + ArcFileName++; + if (*ArcFileName==0) // Excessive -ap switch. + { + *DestName=0; + return; + } + } + } +#endif + + wchar Command=Cmd->Command[0]; + // Use -ep3 only in systems, where disk letters are exist, not in Unix. + bool AbsPaths=Cmd->ExclPath==EXCL_ABSPATH && Command=='X' && IsDriveDiv(':'); + + // We do not use any user specified destination paths when extracting + // absolute paths in -ep3 mode. + if (AbsPaths) + *DestName=0; + + if (Command=='E' || Cmd->ExclPath==EXCL_SKIPWHOLEPATH) + wcsncatz(DestName,PointToName(ArcFileName),DestSize); + else + wcsncatz(DestName,ArcFileName,DestSize); + +#ifdef _WIN_ALL + // Must do after Cmd->ArcPath processing above, so file name and arc path + // trailing spaces are in sync. + if (!Cmd->AllowIncompatNames) + MakeNameCompatible(DestName); +#endif + + wchar DiskLetter=toupperw(DestName[0]); + + if (AbsPaths) + { + if (DestName[1]=='_' && IsPathDiv(DestName[2]) && + DiskLetter>='A' && DiskLetter<='Z') + DestName[1]=':'; + else + if (DestName[0]=='_' && DestName[1]=='_') + { + // Convert __server\share to \\server\share. + DestName[0]=CPATHDIVIDER; + DestName[1]=CPATHDIVIDER; + } + } +} + + +#ifdef RARDLL +bool CmdExtract::ExtrDllGetPassword() +{ + if (!Cmd->Password.IsSet()) + { + if (Cmd->Callback!=NULL) + { + wchar PasswordW[MAXPASSWORD]; + *PasswordW=0; + if (Cmd->Callback(UCM_NEEDPASSWORDW,Cmd->UserData,(LPARAM)PasswordW,ASIZE(PasswordW))==-1) + *PasswordW=0; + if (*PasswordW==0) + { + char PasswordA[MAXPASSWORD]; + *PasswordA=0; + if (Cmd->Callback(UCM_NEEDPASSWORD,Cmd->UserData,(LPARAM)PasswordA,ASIZE(PasswordA))==-1) + *PasswordA=0; + GetWideName(PasswordA,NULL,PasswordW,ASIZE(PasswordW)); + cleandata(PasswordA,sizeof(PasswordA)); + } + Cmd->Password.Set(PasswordW); + cleandata(PasswordW,sizeof(PasswordW)); + Cmd->ManualPassword=true; + } + if (!Cmd->Password.IsSet()) + return false; + } + return true; +} +#endif + + +#ifndef RARDLL +bool CmdExtract::ExtrGetPassword(Archive &Arc,const wchar *ArcFileName) +{ + if (!Cmd->Password.IsSet()) + { + if (!uiGetPassword(UIPASSWORD_FILE,ArcFileName,&Cmd->Password) || !Cmd->Password.IsSet()) + { + // Suppress "test is ok" message in GUI if user entered + // an empty password or cancelled a password prompt. + uiMsg(UIERROR_INCERRCOUNT); + + return false; + } + Cmd->ManualPassword=true; + } +#if !defined(SILENT) + else + if (!PasswordAll && !Arc.FileHead.Solid) + { + eprintf(St(MUseCurPsw),ArcFileName); + switch(Cmd->AllYes ? 1 : Ask(St(MYesNoAll))) + { + case -1: + ErrHandler.Exit(RARX_USERBREAK); + case 2: + if (!uiGetPassword(UIPASSWORD_FILE,ArcFileName,&Cmd->Password)) + return false; + break; + case 3: + PasswordAll=true; + break; + } + } +#endif + return true; +} +#endif + + +#if defined(_WIN_ALL) && !defined(SFX_MODULE) +void CmdExtract::ConvertDosPassword(Archive &Arc,SecPassword &DestPwd) +{ + if (Arc.Format==RARFMT15 && Arc.FileHead.HostOS==HOST_MSDOS) + { + // We need the password in OEM encoding if file was encrypted by + // native RAR/DOS (not extender based). Let's make the conversion. + wchar PlainPsw[MAXPASSWORD]; + Cmd->Password.Get(PlainPsw,ASIZE(PlainPsw)); + char PswA[MAXPASSWORD]; + CharToOemBuffW(PlainPsw,PswA,ASIZE(PswA)); + PswA[ASIZE(PswA)-1]=0; + CharToWide(PswA,PlainPsw,ASIZE(PlainPsw)); + DestPwd.Set(PlainPsw); + cleandata(PlainPsw,sizeof(PlainPsw)); + cleandata(PswA,sizeof(PswA)); + } +} +#endif + + +void CmdExtract::ExtrCreateDir(Archive &Arc,const wchar *ArcFileName) +{ + if (Cmd->Test) + { + mprintf(St(MExtrTestFile),ArcFileName); + mprintf(L" %s",St(MOk)); + return; + } + + MKDIR_CODE MDCode=MakeDir(DestFileName,!Cmd->IgnoreGeneralAttr,Arc.FileHead.FileAttr); + bool DirExist=false; + if (MDCode!=MKDIR_SUCCESS) + { + DirExist=FileExist(DestFileName); + if (DirExist && !IsDir(GetFileAttr(DestFileName))) + { + // File with name same as this directory exists. Propose user + // to overwrite it. + bool UserReject; + FileCreate(Cmd,NULL,DestFileName,ASIZE(DestFileName),&UserReject,Arc.FileHead.UnpSize,&Arc.FileHead.mtime); + DirExist=false; + } + if (!DirExist) + { + CreatePath(DestFileName,true); + MDCode=MakeDir(DestFileName,!Cmd->IgnoreGeneralAttr,Arc.FileHead.FileAttr); + if (MDCode!=MKDIR_SUCCESS) + { + wchar OrigName[ASIZE(DestFileName)]; + wcsncpyz(OrigName,DestFileName,ASIZE(OrigName)); + MakeNameUsable(DestFileName,true); + CreatePath(DestFileName,true); + MDCode=MakeDir(DestFileName,!Cmd->IgnoreGeneralAttr,Arc.FileHead.FileAttr); +#ifndef SFX_MODULE + if (MDCode==MKDIR_SUCCESS) + uiMsg(UIERROR_RENAMING,Arc.FileName,OrigName,DestFileName); +#endif + } + } + } + if (MDCode==MKDIR_SUCCESS) + { + mprintf(St(MCreatDir),DestFileName); + mprintf(L" %s",St(MOk)); + PrevProcessed=true; + } + else + if (DirExist) + { + if (!Cmd->IgnoreGeneralAttr) + SetFileAttr(DestFileName,Arc.FileHead.FileAttr); + PrevProcessed=true; + } + else + { + uiMsg(UIERROR_DIRCREATE,Arc.FileName,DestFileName); + ErrHandler.SysErrMsg(); +#ifdef RARDLL + Cmd->DllError=ERAR_ECREATE; +#endif + ErrHandler.SetErrorCode(RARX_CREATE); + } + if (PrevProcessed) + { +#if defined(_WIN_ALL) && !defined(SFX_MODULE) + if (Cmd->SetCompressedAttr && + (Arc.FileHead.FileAttr & FILE_ATTRIBUTE_COMPRESSED)!=0 && WinNT()) + SetFileCompression(DestFileName,true); +#endif + SetFileHeaderExtra(Cmd,Arc,DestFileName); + SetDirTime(DestFileName, + Cmd->xmtime==EXTTIME_NONE ? NULL:&Arc.FileHead.mtime, + Cmd->xctime==EXTTIME_NONE ? NULL:&Arc.FileHead.ctime, + Cmd->xatime==EXTTIME_NONE ? NULL:&Arc.FileHead.atime); + } +} + + +bool CmdExtract::ExtrCreateFile(Archive &Arc,File &CurFile) +{ + bool Success=true; + wchar Command=Cmd->Command[0]; +#if !defined(SFX_MODULE) + if (Command=='P') + CurFile.SetHandleType(FILE_HANDLESTD); +#endif + if ((Command=='E' || Command=='X') && !Cmd->Test) + { + bool UserReject; + // Specify "write only" mode to avoid OpenIndiana NAS problems + // with SetFileTime and read+write files. + if (!FileCreate(Cmd,&CurFile,DestFileName,ASIZE(DestFileName),&UserReject,Arc.FileHead.UnpSize,&Arc.FileHead.mtime,true)) + { + Success=false; + if (!UserReject) + { + ErrHandler.CreateErrorMsg(Arc.FileName,DestFileName); +#ifdef RARDLL + Cmd->DllError=ERAR_ECREATE; +#endif + if (!IsNameUsable(DestFileName)) + { + uiMsg(UIMSG_CORRECTINGNAME,Arc.FileName); + + wchar OrigName[ASIZE(DestFileName)]; + wcsncpyz(OrigName,DestFileName,ASIZE(OrigName)); + + MakeNameUsable(DestFileName,true); + + CreatePath(DestFileName,true); + if (FileCreate(Cmd,&CurFile,DestFileName,ASIZE(DestFileName),&UserReject,Arc.FileHead.UnpSize,&Arc.FileHead.mtime,true)) + { +#ifndef SFX_MODULE + uiMsg(UIERROR_RENAMING,Arc.FileName,OrigName,DestFileName); +#endif + Success=true; + } + else + ErrHandler.CreateErrorMsg(Arc.FileName,DestFileName); + } + } + } + } + return Success; +} + + +bool CmdExtract::CheckUnpVer(Archive &Arc,const wchar *ArcFileName) +{ + bool WrongVer; + if (Arc.Format==RARFMT50) // Both SFX and RAR can unpack RAR 5.0 archives. + WrongVer=Arc.FileHead.UnpVer>VER_UNPACK5; + else + { +#ifdef SFX_MODULE // SFX can unpack only RAR 2.9 archives. + WrongVer=Arc.FileHead.UnpVer!=VER_UNPACK; +#else // All formats since 1.3 for RAR. + WrongVer=Arc.FileHead.UnpVer<13 || Arc.FileHead.UnpVer>VER_UNPACK; +#endif + } + + // We can unpack stored files regardless of compression version field. + if (Arc.FileHead.Method==0) + WrongVer=false; + + if (WrongVer) + { + ErrHandler.UnknownMethodMsg(Arc.FileName,ArcFileName); + uiMsg(UIERROR_NEWERRAR,Arc.FileName); + } + return !WrongVer; +} diff --git a/reference/unrar/extract.hpp b/reference/unrar/extract.hpp new file mode 100644 index 00000000..85a21f53 --- /dev/null +++ b/reference/unrar/extract.hpp @@ -0,0 +1,62 @@ +#ifndef _RAR_EXTRACT_ +#define _RAR_EXTRACT_ + +enum EXTRACT_ARC_CODE {EXTRACT_ARC_NEXT,EXTRACT_ARC_REPEAT}; + +class CmdExtract +{ + private: + EXTRACT_ARC_CODE ExtractArchive(); + bool ExtractFileCopy(File &New,wchar *ArcName,wchar *NameNew,wchar *NameExisting,size_t NameExistingSize); + void ExtrPrepareName(Archive &Arc,const wchar *ArcFileName,wchar *DestName,size_t DestSize); +#ifdef RARDLL + bool ExtrDllGetPassword(); +#else + bool ExtrGetPassword(Archive &Arc,const wchar *ArcFileName); +#endif +#if defined(_WIN_ALL) && !defined(SFX_MODULE) + void ConvertDosPassword(Archive &Arc,SecPassword &DestPwd); +#endif + void ExtrCreateDir(Archive &Arc,const wchar *ArcFileName); + bool ExtrCreateFile(Archive &Arc,File &CurFile); + bool CheckUnpVer(Archive &Arc,const wchar *ArcFileName); + + RarTime StartTime; // time when extraction started + + CommandData *Cmd; + + ComprDataIO DataIO; + Unpack *Unp; + unsigned long TotalFileCount; + + unsigned long FileCount; + unsigned long MatchedArgs; + bool FirstFile; + bool AllMatchesExact; + bool ReconstructDone; + + // If any non-zero solid file was successfully unpacked before current. + // If true and if current encrypted file is broken, obviously + // the password is correct and we can report broken CRC without + // any wrong password hints. + bool AnySolidDataUnpackedWell; + + wchar ArcName[NM]; + + bool PasswordAll; + bool PrevProcessed; // If previous file was successfully extracted or tested. + wchar DestFileName[NM]; + bool PasswordCancelled; +#if defined(_WIN_ALL) && !defined(SFX_MODULE) && !defined(SILENT) + bool Fat32,NotFat32; +#endif + public: + CmdExtract(CommandData *Cmd); + ~CmdExtract(); + void DoExtract(); + void ExtractArchiveInit(Archive &Arc); + bool ExtractCurrentFile(Archive &Arc,size_t HeaderSize,bool &Repeat); + static void UnstoreFile(ComprDataIO &DataIO,int64 DestUnpSize); +}; + +#endif diff --git a/reference/unrar/filcreat.cpp b/reference/unrar/filcreat.cpp new file mode 100644 index 00000000..a64a7d4d --- /dev/null +++ b/reference/unrar/filcreat.cpp @@ -0,0 +1,163 @@ +#include "rar.hpp" + +// If NewFile==NULL, we delete created file after user confirmation. +// It is useful we we need to overwrite an existing folder or file, +// but need user confirmation for that. +bool FileCreate(RAROptions *Cmd,File *NewFile,wchar *Name,size_t MaxNameSize, + bool *UserReject,int64 FileSize,RarTime *FileTime,bool WriteOnly) +{ + if (UserReject!=NULL) + *UserReject=false; +#ifdef _WIN_ALL + bool ShortNameChanged=false; +#endif + while (FileExist(Name)) + { +#if defined(_WIN_ALL) + if (!ShortNameChanged) + { + // Avoid the infinite loop if UpdateExistingShortName returns + // the same name. + ShortNameChanged=true; + + // Maybe our long name matches the short name of existing file. + // Let's check if we can change the short name. + if (UpdateExistingShortName(Name)) + continue; + } + // Allow short name check again. It is necessary, because rename and + // autorename below can change the name, so we need to check it again. + ShortNameChanged=false; +#endif + UIASKREP_RESULT Choice=uiAskReplaceEx(Cmd,Name,MaxNameSize,FileSize,FileTime,(NewFile==NULL ? UIASKREP_F_NORENAME:0)); + + if (Choice==UIASKREP_R_REPLACE) + break; + if (Choice==UIASKREP_R_SKIP) + { + if (UserReject!=NULL) + *UserReject=true; + return false; + } + if (Choice==UIASKREP_R_CANCEL) + ErrHandler.Exit(RARX_USERBREAK); + } + + // Try to truncate the existing file first instead of delete, + // so we preserve existing file permissions such as NTFS permissions. + uint FileMode=WriteOnly ? FMF_WRITE|FMF_SHAREREAD:FMF_UPDATE|FMF_SHAREREAD; + if (NewFile!=NULL && NewFile->Create(Name,FileMode)) + return true; + + CreatePath(Name,true); + return NewFile!=NULL ? NewFile->Create(Name,FileMode):DelFile(Name); +} + + +bool GetAutoRenamedName(wchar *Name,size_t MaxNameSize) +{ + wchar NewName[NM]; + size_t NameLength=wcslen(Name); + wchar *Ext=GetExt(Name); + if (Ext==NULL) + Ext=Name+NameLength; + for (uint FileVer=1;;FileVer++) + { + swprintf(NewName,ASIZE(NewName),L"%.*ls(%u)%ls",uint(Ext-Name),Name,FileVer,Ext); + if (!FileExist(NewName)) + { + wcsncpyz(Name,NewName,MaxNameSize); + break; + } + if (FileVer>=1000000) + return false; + } + return true; +} + + +#if defined(_WIN_ALL) +// If we find a file, which short name is equal to 'Name', we try to change +// its short name, while preserving the long name. It helps when unpacking +// an archived file, which long name is equal to short name of already +// existing file. Otherwise we would overwrite the already existing file, +// even though its long name does not match the name of unpacking file. +bool UpdateExistingShortName(const wchar *Name) +{ + wchar LongPathName[NM]; + DWORD Res=GetLongPathName(Name,LongPathName,ASIZE(LongPathName)); + if (Res==0 || Res>=ASIZE(LongPathName)) + return false; + wchar ShortPathName[NM]; + Res=GetShortPathName(Name,ShortPathName,ASIZE(ShortPathName)); + if (Res==0 || Res>=ASIZE(ShortPathName)) + return false; + wchar *LongName=PointToName(LongPathName); + wchar *ShortName=PointToName(ShortPathName); + + // We continue only if file has a short name, which does not match its + // long name, and this short name is equal to name of file which we need + // to create. + if (*ShortName==0 || wcsicomp(LongName,ShortName)==0 || + wcsicomp(PointToName(Name),ShortName)!=0) + return false; + + // Generate the temporary new name for existing file. + wchar NewName[NM]; + *NewName=0; + for (int I=0;I<10000 && *NewName==0;I+=123) + { + // Here we copy the path part of file to create. We'll make the temporary + // file in the same folder. + wcsncpyz(NewName,Name,ASIZE(NewName)); + + // Here we set the random name part. + swprintf(PointToName(NewName),ASIZE(NewName),L"rtmp%d",I); + + // If such file is already exist, try next random name. + if (FileExist(NewName)) + *NewName=0; + } + + // If we could not generate the name not used by any other file, we return. + if (*NewName==0) + return false; + + // FastFind returns the name without path, but we need the fully qualified + // name for renaming, so we use the path from file to create and long name + // from existing file. + wchar FullName[NM]; + wcsncpyz(FullName,Name,ASIZE(FullName)); + SetName(FullName,LongName,ASIZE(FullName)); + + // Rename the existing file to randomly generated name. Normally it changes + // the short name too. + if (!MoveFile(FullName,NewName)) + return false; + + // Now we need to create the temporary empty file with same name as + // short name of our already existing file. We do it to occupy its previous + // short name and not allow to use it again when renaming the file back to + // its original long name. + File KeepShortFile; + bool Created=false; + if (!FileExist(Name)) + Created=KeepShortFile.Create(Name,FMF_WRITE|FMF_SHAREREAD); + + // Now we rename the existing file from temporary name to original long name. + // Since its previous short name is occupied by another file, it should + // get another short name. + MoveFile(NewName,FullName); + + if (Created) + { + // Delete the temporary zero length file occupying the short name, + KeepShortFile.Close(); + KeepShortFile.Delete(); + } + // We successfully changed the short name. Maybe sometimes we'll simplify + // this function by use of SetFileShortName Windows API call. + // But SetFileShortName is not available in older Windows. + return true; +} +#endif diff --git a/reference/unrar/filcreat.hpp b/reference/unrar/filcreat.hpp new file mode 100644 index 00000000..44f801d4 --- /dev/null +++ b/reference/unrar/filcreat.hpp @@ -0,0 +1,14 @@ +#ifndef _RAR_FILECREATE_ +#define _RAR_FILECREATE_ + +bool FileCreate(RAROptions *Cmd,File *NewFile,wchar *Name,size_t MaxNameSize, + bool *UserReject,int64 FileSize=INT64NDF, + RarTime *FileTime=NULL,bool WriteOnly=false); + +bool GetAutoRenamedName(wchar *Name,size_t MaxNameSize); + +#if defined(_WIN_ALL) +bool UpdateExistingShortName(const wchar *Name); +#endif + +#endif diff --git a/reference/unrar/file.cpp b/reference/unrar/file.cpp new file mode 100644 index 00000000..5e6235a7 --- /dev/null +++ b/reference/unrar/file.cpp @@ -0,0 +1,729 @@ +#include "rar.hpp" + +File::File() +{ + hFile=FILE_BAD_HANDLE; + *FileName=0; + NewFile=false; + LastWrite=false; + HandleType=FILE_HANDLENORMAL; + SkipClose=false; + IgnoreReadErrors=false; + ErrorType=FILE_SUCCESS; + OpenShared=false; + AllowDelete=true; + AllowExceptions=true; +#ifdef _WIN_ALL + NoSequentialRead=false; + CreateMode=FMF_UNDEFINED; +#endif +} + + +File::~File() +{ + if (hFile!=FILE_BAD_HANDLE && !SkipClose) + if (NewFile) + Delete(); + else + Close(); +} + + +void File::operator = (File &SrcFile) +{ + hFile=SrcFile.hFile; + NewFile=SrcFile.NewFile; + LastWrite=SrcFile.LastWrite; + HandleType=SrcFile.HandleType; + wcsncpyz(FileName,SrcFile.FileName,ASIZE(FileName)); + SrcFile.SkipClose=true; +} + + +bool File::Open(const wchar *Name,uint Mode) +{ + ErrorType=FILE_SUCCESS; + FileHandle hNewFile; + bool OpenShared=File::OpenShared || (Mode & FMF_OPENSHARED)!=0; + bool UpdateMode=(Mode & FMF_UPDATE)!=0; + bool WriteMode=(Mode & FMF_WRITE)!=0; +#ifdef _WIN_ALL + uint Access=WriteMode ? GENERIC_WRITE:GENERIC_READ; + if (UpdateMode) + Access|=GENERIC_WRITE; + uint ShareMode=(Mode & FMF_OPENEXCLUSIVE) ? 0 : FILE_SHARE_READ; + if (OpenShared) + ShareMode|=FILE_SHARE_WRITE; + uint Flags=NoSequentialRead ? 0:FILE_FLAG_SEQUENTIAL_SCAN; + hNewFile=CreateFile(Name,Access,ShareMode,NULL,OPEN_EXISTING,Flags,NULL); + + DWORD LastError; + if (hNewFile==FILE_BAD_HANDLE) + { + LastError=GetLastError(); + + wchar LongName[NM]; + if (GetWinLongPath(Name,LongName,ASIZE(LongName))) + { + hNewFile=CreateFile(LongName,Access,ShareMode,NULL,OPEN_EXISTING,Flags,NULL); + + // For archive names longer than 260 characters first CreateFile + // (without \\?\) fails and sets LastError to 3 (access denied). + // We need the correct "file not found" error code to decide + // if we create a new archive or quit with "cannot create" error. + // So we need to check the error code after \\?\ CreateFile again, + // otherwise we'll fail to create new archives with long names. + // But we cannot simply assign the new code to LastError, + // because it would break "..\arcname.rar" relative names processing. + // First CreateFile returns the correct "file not found" code for such + // names, but "\\?\" CreateFile returns ERROR_INVALID_NAME treating + // dots as a directory name. So we check only for "file not found" + // error here and for other errors use the first CreateFile result. + if (GetLastError()==ERROR_FILE_NOT_FOUND) + LastError=ERROR_FILE_NOT_FOUND; + } + } + if (hNewFile==FILE_BAD_HANDLE && LastError==ERROR_FILE_NOT_FOUND) + ErrorType=FILE_NOTFOUND; + +#else + int flags=UpdateMode ? O_RDWR:(WriteMode ? O_WRONLY:O_RDONLY); +#ifdef O_BINARY + flags|=O_BINARY; +#if defined(_AIX) && defined(_LARGE_FILE_API) + flags|=O_LARGEFILE; +#endif +#endif + char NameA[NM]; + WideToChar(Name,NameA,ASIZE(NameA)); + + int handle=open(NameA,flags); +#ifdef LOCK_EX + +#ifdef _OSF_SOURCE + extern "C" int flock(int, int); +#endif + + if (!OpenShared && UpdateMode && handle>=0 && flock(handle,LOCK_EX|LOCK_NB)==-1) + { + close(handle); + return false; + } +#endif + if (handle==-1) + hNewFile=FILE_BAD_HANDLE; + else + { +#ifdef FILE_USE_OPEN + hNewFile=handle; +#else + hNewFile=fdopen(handle,UpdateMode ? UPDATEBINARY:READBINARY); +#endif + } + if (hNewFile==FILE_BAD_HANDLE && errno==ENOENT) + ErrorType=FILE_NOTFOUND; +#endif + NewFile=false; + HandleType=FILE_HANDLENORMAL; + SkipClose=false; + bool Success=hNewFile!=FILE_BAD_HANDLE; + if (Success) + { + hFile=hNewFile; + wcsncpyz(FileName,Name,ASIZE(FileName)); + } + return Success; +} + + +#if !defined(SFX_MODULE) +void File::TOpen(const wchar *Name) +{ + if (!WOpen(Name)) + ErrHandler.Exit(RARX_OPEN); +} +#endif + + +bool File::WOpen(const wchar *Name) +{ + if (Open(Name)) + return true; + ErrHandler.OpenErrorMsg(Name); + return false; +} + + +bool File::Create(const wchar *Name,uint Mode) +{ + // OpenIndiana based NAS and CIFS shares fail to set the file time if file + // was created in read+write mode and some data was written and not flushed + // before SetFileTime call. So we should use the write only mode if we plan + // SetFileTime call and do not need to read from file. + bool WriteMode=(Mode & FMF_WRITE)!=0; + bool ShareRead=(Mode & FMF_SHAREREAD)!=0 || File::OpenShared; +#ifdef _WIN_ALL + CreateMode=Mode; + uint Access=WriteMode ? GENERIC_WRITE:GENERIC_READ|GENERIC_WRITE; + DWORD ShareMode=ShareRead ? FILE_SHARE_READ:0; + + // Windows automatically removes dots and spaces in the end of file name, + // So we detect such names and process them with \\?\ prefix. + wchar *LastChar=PointToLastChar(Name); + bool Special=*LastChar=='.' || *LastChar==' '; + + if (Special && (Mode & FMF_STANDARDNAMES)==0) + hFile=FILE_BAD_HANDLE; + else + hFile=CreateFile(Name,Access,ShareMode,NULL,CREATE_ALWAYS,0,NULL); + + if (hFile==FILE_BAD_HANDLE) + { + wchar LongName[NM]; + if (GetWinLongPath(Name,LongName,ASIZE(LongName))) + hFile=CreateFile(LongName,Access,ShareMode,NULL,CREATE_ALWAYS,0,NULL); + } + +#else + char NameA[NM]; + WideToChar(Name,NameA,ASIZE(NameA)); +#ifdef FILE_USE_OPEN + hFile=open(NameA,(O_CREAT|O_TRUNC) | (WriteMode ? O_WRONLY : O_RDWR),0666); +#else + hFile=fopen(NameA,WriteMode ? WRITEBINARY:CREATEBINARY); +#endif +#endif + NewFile=true; + HandleType=FILE_HANDLENORMAL; + SkipClose=false; + wcsncpyz(FileName,Name,ASIZE(FileName)); + return hFile!=FILE_BAD_HANDLE; +} + + +#if !defined(SFX_MODULE) +void File::TCreate(const wchar *Name,uint Mode) +{ + if (!WCreate(Name,Mode)) + ErrHandler.Exit(RARX_FATAL); +} +#endif + + +bool File::WCreate(const wchar *Name,uint Mode) +{ + if (Create(Name,Mode)) + return true; + ErrHandler.CreateErrorMsg(Name); + return false; +} + + +bool File::Close() +{ + bool Success=true; + + if (hFile!=FILE_BAD_HANDLE) + { + if (!SkipClose) + { +#ifdef _WIN_ALL + // We use the standard system handle for stdout in Windows + // and it must not be closed here. + if (HandleType==FILE_HANDLENORMAL) + Success=CloseHandle(hFile)==TRUE; +#else +#ifdef FILE_USE_OPEN + Success=close(hFile)!=-1; +#else + Success=fclose(hFile)!=EOF; +#endif +#endif + } + hFile=FILE_BAD_HANDLE; + } + HandleType=FILE_HANDLENORMAL; + if (!Success && AllowExceptions) + ErrHandler.CloseError(FileName); + return Success; +} + + +bool File::Delete() +{ + if (HandleType!=FILE_HANDLENORMAL) + return false; + if (hFile!=FILE_BAD_HANDLE) + Close(); + if (!AllowDelete) + return false; + return DelFile(FileName); +} + + +bool File::Rename(const wchar *NewName) +{ + // No need to rename if names are already same. + bool Success=wcscmp(FileName,NewName)==0; + + if (!Success) + Success=RenameFile(FileName,NewName); + + if (Success) + wcscpy(FileName,NewName); + + return Success; +} + + +bool File::Write(const void *Data,size_t Size) +{ + if (Size==0) + return true; + if (HandleType==FILE_HANDLESTD) + { +#ifdef _WIN_ALL + hFile=GetStdHandle(STD_OUTPUT_HANDLE); +#else + // Cannot use the standard stdout here, because it already has wide orientation. + if (hFile==FILE_BAD_HANDLE) + { +#ifdef FILE_USE_OPEN + hFile=dup(STDOUT_FILENO); // Open new stdout stream. +#else + hFile=fdopen(dup(STDOUT_FILENO),"w"); // Open new stdout stream. +#endif + } +#endif + } + bool Success; + while (1) + { + Success=false; +#ifdef _WIN_ALL + DWORD Written=0; + if (HandleType!=FILE_HANDLENORMAL) + { + // writing to stdout can fail in old Windows if data block is too large + const size_t MaxSize=0x4000; + for (size_t I=0;ISize && FilePos-Size<=0xffffffff && FilePos+Size>0xffffffff) + ErrHandler.WriteErrorFAT(FileName); +#endif + if (ErrHandler.AskRepeatWrite(FileName,false)) + { +#if !defined(_WIN_ALL) && !defined(FILE_USE_OPEN) + clearerr(hFile); +#endif + if (Written0) + Seek(Tell()-Written,SEEK_SET); + continue; + } + ErrHandler.WriteError(NULL,FileName); + } + break; + } + LastWrite=true; + return Success; // It can return false only if AllowExceptions is disabled. +} + + +int File::Read(void *Data,size_t Size) +{ + int64 FilePos=0; // Initialized only to suppress some compilers warning. + + if (IgnoreReadErrors) + FilePos=Tell(); + int ReadSize; + while (true) + { + ReadSize=DirectRead(Data,Size); + if (ReadSize==-1) + { + ErrorType=FILE_READERROR; + if (AllowExceptions) + if (IgnoreReadErrors) + { + ReadSize=0; + for (size_t I=0;IMaxDeviceRead) +// Size=MaxDeviceRead; + hFile=GetStdHandle(STD_INPUT_HANDLE); +#else +#ifdef FILE_USE_OPEN + hFile=STDIN_FILENO; +#else + hFile=stdin; +#endif +#endif + } +#ifdef _WIN_ALL + // For pipes like 'type file.txt | rar -si arcname' ReadFile may return + // data in small ~4KB blocks. It may slightly reduce the compression ratio. + DWORD Read; + if (!ReadFile(hFile,Data,(DWORD)Size,&Read,NULL)) + { + if (IsDevice() && Size>MaxDeviceRead) + return DirectRead(Data,MaxDeviceRead); + if (HandleType==FILE_HANDLESTD && GetLastError()==ERROR_BROKEN_PIPE) + return 0; + + // We had a bug report about failure to archive 1C database lock file + // 1Cv8tmp.1CL, which is a zero length file with a region above 200 KB + // permanently locked. If our first read request uses too large buffer + // and if we are in -dh mode, so we were able to open the file, + // we'll fail with "Read error". So now we use try a smaller buffer size + // in case of lock error. + if (HandleType==FILE_HANDLENORMAL && Size>MaxLockedRead && + GetLastError()==ERROR_LOCK_VIOLATION) + return DirectRead(Data,MaxLockedRead); + + return -1; + } + return Read; +#else +#ifdef FILE_USE_OPEN + ssize_t ReadSize=read(hFile,Data,Size); + if (ReadSize==-1) + return -1; + return (int)ReadSize; +#else + if (LastWrite) + { + fflush(hFile); + LastWrite=false; + } + clearerr(hFile); + size_t ReadSize=fread(Data,1,Size,hFile); + if (ferror(hFile)) + return -1; + return (int)ReadSize; +#endif +#endif +} + + +void File::Seek(int64 Offset,int Method) +{ + if (!RawSeek(Offset,Method) && AllowExceptions) + ErrHandler.SeekError(FileName); +} + + +bool File::RawSeek(int64 Offset,int Method) +{ + if (hFile==FILE_BAD_HANDLE) + return true; + if (Offset<0 && Method!=SEEK_SET) + { + Offset=(Method==SEEK_CUR ? Tell():FileLength())+Offset; + Method=SEEK_SET; + } +#ifdef _WIN_ALL + LONG HighDist=(LONG)(Offset>>32); + if (SetFilePointer(hFile,(LONG)Offset,&HighDist,Method)==0xffffffff && + GetLastError()!=NO_ERROR) + return false; +#else + LastWrite=false; +#ifdef FILE_USE_OPEN + if (lseek(hFile,(off_t)Offset,Method)==-1) + return false; +#elif defined(_LARGEFILE_SOURCE) && !defined(_OSF_SOURCE) && !defined(__VMS) + if (fseeko(hFile,Offset,Method)!=0) + return false; +#else + if (fseek(hFile,(long)Offset,Method)!=0) + return false; +#endif +#endif + return true; +} + + +int64 File::Tell() +{ + if (hFile==FILE_BAD_HANDLE) + if (AllowExceptions) + ErrHandler.SeekError(FileName); + else + return -1; +#ifdef _WIN_ALL + LONG HighDist=0; + uint LowDist=SetFilePointer(hFile,0,&HighDist,FILE_CURRENT); + if (LowDist==0xffffffff && GetLastError()!=NO_ERROR) + if (AllowExceptions) + ErrHandler.SeekError(FileName); + else + return -1; + return INT32TO64(HighDist,LowDist); +#else +#ifdef FILE_USE_OPEN + return lseek(hFile,0,SEEK_CUR); +#elif defined(_LARGEFILE_SOURCE) && !defined(_OSF_SOURCE) + return ftello(hFile); +#else + return ftell(hFile); +#endif +#endif +} + + +void File::Prealloc(int64 Size) +{ +#ifdef _WIN_ALL + if (RawSeek(Size,SEEK_SET)) + { + Truncate(); + Seek(0,SEEK_SET); + } +#endif + +#if defined(_UNIX) && defined(USE_FALLOCATE) + // fallocate is rather new call. Only latest kernels support it. + // So we are not using it by default yet. + int fd = GetFD(); + if (fd >= 0) + fallocate(fd, 0, 0, Size); +#endif +} + + +byte File::GetByte() +{ + byte Byte=0; + Read(&Byte,1); + return Byte; +} + + +void File::PutByte(byte Byte) +{ + Write(&Byte,1); +} + + +bool File::Truncate() +{ +#ifdef _WIN_ALL + return SetEndOfFile(hFile)==TRUE; +#else + return ftruncate(GetFD(),(off_t)Tell())==0; +#endif +} + + +void File::Flush() +{ +#ifdef _WIN_ALL + FlushFileBuffers(hFile); +#else +#ifndef FILE_USE_OPEN + fflush(hFile); +#endif + fsync(GetFD()); +#endif +} + + +void File::SetOpenFileTime(RarTime *ftm,RarTime *ftc,RarTime *fta) +{ +#ifdef _WIN_ALL + // Workaround for OpenIndiana NAS time bug. If we cannot create a file + // in write only mode, we need to flush the write buffer before calling + // SetFileTime or file time will not be changed. + if (CreateMode!=FMF_UNDEFINED && (CreateMode & FMF_WRITE)==0) + FlushFileBuffers(hFile); + + bool sm=ftm!=NULL && ftm->IsSet(); + bool sc=ftc!=NULL && ftc->IsSet(); + bool sa=fta!=NULL && fta->IsSet(); + FILETIME fm,fc,fa; + if (sm) + ftm->GetWinFT(&fm); + if (sc) + ftc->GetWinFT(&fc); + if (sa) + fta->GetWinFT(&fa); + SetFileTime(hFile,sc ? &fc:NULL,sa ? &fa:NULL,sm ? &fm:NULL); +#endif +} + + +void File::SetCloseFileTime(RarTime *ftm,RarTime *fta) +{ +// Android APP_PLATFORM := android-14 does not support futimens and futimes. +// Newer platforms support futimens, but fail on Android 4.2. +// We have to use utime for Android. +// Also we noticed futimens fail to set timestamps on NTFS partition +// mounted to virtual Linux x86 machine, but utimensat worked correctly. +// So we set timestamps for already closed files in Unix. +#ifdef _UNIX + SetCloseFileTimeByName(FileName,ftm,fta); +#endif +} + + +void File::SetCloseFileTimeByName(const wchar *Name,RarTime *ftm,RarTime *fta) +{ +#ifdef _UNIX + bool setm=ftm!=NULL && ftm->IsSet(); + bool seta=fta!=NULL && fta->IsSet(); + if (setm || seta) + { + char NameA[NM]; + WideToChar(Name,NameA,ASIZE(NameA)); + +#ifdef UNIX_TIME_NS + timespec times[2]; + times[0].tv_sec=seta ? fta->GetUnix() : 0; + times[0].tv_nsec=seta ? long(fta->GetUnixNS()%1000000000) : UTIME_NOW; + times[1].tv_sec=setm ? ftm->GetUnix() : 0; + times[1].tv_nsec=setm ? long(ftm->GetUnixNS()%1000000000) : UTIME_NOW; + utimensat(AT_FDCWD,NameA,times,0); +#else + utimbuf ut; + if (setm) + ut.modtime=ftm->GetUnix(); + else + ut.modtime=fta->GetUnix(); // Need to set something, cannot left it 0. + if (seta) + ut.actime=fta->GetUnix(); + else + ut.actime=ut.modtime; // Need to set something, cannot left it 0. + utime(NameA,&ut); +#endif + } +#endif +} + + +void File::GetOpenFileTime(RarTime *ft) +{ +#ifdef _WIN_ALL + FILETIME FileTime; + GetFileTime(hFile,NULL,NULL,&FileTime); + ft->SetWinFT(&FileTime); +#endif +#if defined(_UNIX) || defined(_EMX) + struct stat st; + fstat(GetFD(),&st); + ft->SetUnix(st.st_mtime); +#endif +} + + +int64 File::FileLength() +{ + SaveFilePos SavePos(*this); + Seek(0,SEEK_END); + return Tell(); +} + + +bool File::IsDevice() +{ + if (hFile==FILE_BAD_HANDLE) + return false; +#ifdef _WIN_ALL + uint Type=GetFileType(hFile); + return Type==FILE_TYPE_CHAR || Type==FILE_TYPE_PIPE; +#else + return isatty(GetFD()); +#endif +} + + +#ifndef SFX_MODULE +int64 File::Copy(File &Dest,int64 Length) +{ + Array Buffer(0x40000); + int64 CopySize=0; + bool CopyAll=(Length==INT64NDF); + + while (CopyAll || Length>0) + { + Wait(); + size_t SizeToRead=(!CopyAll && Length<(int64)Buffer.Size()) ? (size_t)Length:Buffer.Size(); + char *Buf=&Buffer[0]; + int ReadSize=Read(Buf,SizeToRead); + if (ReadSize==0) + break; + size_t WriteSize=ReadSize; +#ifdef _WIN_ALL + // For FAT32 USB flash drives in Windows if first write is 4 KB or more, + // write caching is disabled and "write through" is enabled, resulting + // in bad performance, especially for many small files. It happens when + // we create SFX archive on USB drive, because SFX module is written first. + // So we split the first write to small 1 KB followed by rest of data. + if (CopySize==0 && WriteSize>=4096) + { + const size_t FirstWrite=1024; + Dest.Write(Buf,FirstWrite); + Buf+=FirstWrite; + WriteSize-=FirstWrite; + } +#endif + Dest.Write(Buf,WriteSize); + CopySize+=ReadSize; + if (!CopyAll) + Length-=ReadSize; + } + return CopySize; +} +#endif diff --git a/reference/unrar/file.hpp b/reference/unrar/file.hpp new file mode 100644 index 00000000..8a7631e3 --- /dev/null +++ b/reference/unrar/file.hpp @@ -0,0 +1,126 @@ +#ifndef _RAR_FILE_ +#define _RAR_FILE_ + +#define FILE_USE_OPEN + +#ifdef _WIN_ALL + typedef HANDLE FileHandle; + #define FILE_BAD_HANDLE INVALID_HANDLE_VALUE +#elif defined(FILE_USE_OPEN) + typedef off_t FileHandle; + #define FILE_BAD_HANDLE -1 +#else + typedef FILE* FileHandle; + #define FILE_BAD_HANDLE NULL +#endif + +class RAROptions; + +enum FILE_HANDLETYPE {FILE_HANDLENORMAL,FILE_HANDLESTD}; + +enum FILE_ERRORTYPE {FILE_SUCCESS,FILE_NOTFOUND,FILE_READERROR}; + +enum FILE_MODE_FLAGS { + // Request read only access to file. Default for Open. + FMF_READ=0, + + // Request both read and write access to file. Default for Create. + FMF_UPDATE=1, + + // Request write only access to file. + FMF_WRITE=2, + + // Open files which are already opened for write by other programs. + FMF_OPENSHARED=4, + + // Open files only if no other program is opened it even in shared mode. + FMF_OPENEXCLUSIVE=8, + + // Provide read access to created file for other programs. + FMF_SHAREREAD=16, + + // Use standard NTFS names without trailing dots and spaces. + FMF_STANDARDNAMES=32, + + // Mode flags are not defined yet. + FMF_UNDEFINED=256 +}; + + +class File +{ + private: + FileHandle hFile; + bool LastWrite; + FILE_HANDLETYPE HandleType; + bool SkipClose; + bool IgnoreReadErrors; + bool NewFile; + bool AllowDelete; + bool AllowExceptions; +#ifdef _WIN_ALL + bool NoSequentialRead; + uint CreateMode; +#endif + protected: + bool OpenShared; // Set by 'Archive' class. + public: + wchar FileName[NM]; + + FILE_ERRORTYPE ErrorType; + public: + File(); + virtual ~File(); + void operator = (File &SrcFile); + virtual bool Open(const wchar *Name,uint Mode=FMF_READ); + void TOpen(const wchar *Name); + bool WOpen(const wchar *Name); + bool Create(const wchar *Name,uint Mode=FMF_UPDATE|FMF_SHAREREAD); + void TCreate(const wchar *Name,uint Mode=FMF_UPDATE|FMF_SHAREREAD); + bool WCreate(const wchar *Name,uint Mode=FMF_UPDATE|FMF_SHAREREAD); + bool Close(); + bool Delete(); + bool Rename(const wchar *NewName); + bool Write(const void *Data,size_t Size); + virtual int Read(void *Data,size_t Size); + int DirectRead(void *Data,size_t Size); + virtual void Seek(int64 Offset,int Method); + bool RawSeek(int64 Offset,int Method); + virtual int64 Tell(); + void Prealloc(int64 Size); + byte GetByte(); + void PutByte(byte Byte); + bool Truncate(); + void Flush(); + void SetOpenFileTime(RarTime *ftm,RarTime *ftc=NULL,RarTime *fta=NULL); + void SetCloseFileTime(RarTime *ftm,RarTime *fta=NULL); + static void SetCloseFileTimeByName(const wchar *Name,RarTime *ftm,RarTime *fta); + void GetOpenFileTime(RarTime *ft); + bool IsOpened() {return hFile!=FILE_BAD_HANDLE;}; + int64 FileLength(); + void SetHandleType(FILE_HANDLETYPE Type) {HandleType=Type;} + FILE_HANDLETYPE GetHandleType() {return HandleType;} + bool IsDevice(); + static bool RemoveCreated(); + FileHandle GetHandle() {return hFile;} + void SetHandle(FileHandle Handle) {Close();hFile=Handle;} + void SetIgnoreReadErrors(bool Mode) {IgnoreReadErrors=Mode;} + int64 Copy(File &Dest,int64 Length=INT64NDF); + void SetAllowDelete(bool Allow) {AllowDelete=Allow;} + void SetExceptions(bool Allow) {AllowExceptions=Allow;} +#ifdef _WIN_ALL + void RemoveSequentialFlag() {NoSequentialRead=true;} +#endif +#ifdef _UNIX + int GetFD() + { +#ifdef FILE_USE_OPEN + return hFile; +#else + return fileno(hFile); +#endif + } +#endif +}; + +#endif diff --git a/reference/unrar/filefn.cpp b/reference/unrar/filefn.cpp new file mode 100644 index 00000000..4eb7b9b2 --- /dev/null +++ b/reference/unrar/filefn.cpp @@ -0,0 +1,510 @@ +#include "rar.hpp" + +MKDIR_CODE MakeDir(const wchar *Name,bool SetAttr,uint Attr) +{ +#ifdef _WIN_ALL + // Windows automatically removes dots and spaces in the end of directory + // name. So we detect such names and process them with \\?\ prefix. + wchar *LastChar=PointToLastChar(Name); + bool Special=*LastChar=='.' || *LastChar==' '; + BOOL RetCode=Special ? FALSE : CreateDirectory(Name,NULL); + if (RetCode==0 && !FileExist(Name)) + { + wchar LongName[NM]; + if (GetWinLongPath(Name,LongName,ASIZE(LongName))) + RetCode=CreateDirectory(LongName,NULL); + } + if (RetCode!=0) // Non-zero return code means success for CreateDirectory. + { + if (SetAttr) + SetFileAttr(Name,Attr); + return MKDIR_SUCCESS; + } + int ErrCode=GetLastError(); + if (ErrCode==ERROR_FILE_NOT_FOUND || ErrCode==ERROR_PATH_NOT_FOUND) + return MKDIR_BADPATH; + return MKDIR_ERROR; +#elif defined(_UNIX) + char NameA[NM]; + WideToChar(Name,NameA,ASIZE(NameA)); + mode_t uattr=SetAttr ? (mode_t)Attr:0777; + int ErrCode=mkdir(NameA,uattr); + if (ErrCode==-1) + return errno==ENOENT ? MKDIR_BADPATH:MKDIR_ERROR; + return MKDIR_SUCCESS; +#else + return MKDIR_ERROR; +#endif +} + + +bool CreatePath(const wchar *Path,bool SkipLastName) +{ + if (Path==NULL || *Path==0) + return false; + +#if defined(_WIN_ALL) || defined(_EMX) + uint DirAttr=0; +#else + uint DirAttr=0777; +#endif + + bool Success=true; + + for (const wchar *s=Path;*s!=0;s++) + { + wchar DirName[NM]; + if (s-Path>=ASIZE(DirName)) + break; + + // Process all kinds of path separators, so user can enter Unix style + // path in Windows or Windows in Unix. s>Path check avoids attempting + // creating an empty directory for paths starting from path separator. + if (IsPathDiv(*s) && s>Path) + { +#ifdef _WIN_ALL + // We must not attempt to create "D:" directory, because first + // CreateDirectory will fail, so we'll use \\?\D:, which forces Wine + // to create "D:" directory. + if (s==Path+2 && Path[1]==':') + continue; +#endif + wcsncpy(DirName,Path,s-Path); + DirName[s-Path]=0; + + Success=MakeDir(DirName,true,DirAttr)==MKDIR_SUCCESS; + if (Success) + { + mprintf(St(MCreatDir),DirName); + mprintf(L" %s",St(MOk)); + } + } + } + if (!SkipLastName && !IsPathDiv(*PointToLastChar(Path))) + Success=MakeDir(Path,true,DirAttr)==MKDIR_SUCCESS; + return Success; +} + + +void SetDirTime(const wchar *Name,RarTime *ftm,RarTime *ftc,RarTime *fta) +{ +#if defined(_WIN_ALL) + bool sm=ftm!=NULL && ftm->IsSet(); + bool sc=ftc!=NULL && ftc->IsSet(); + bool sa=fta!=NULL && fta->IsSet(); + + uint DirAttr=GetFileAttr(Name); + bool ResetAttr=(DirAttr!=0xffffffff && (DirAttr & FILE_ATTRIBUTE_READONLY)!=0); + if (ResetAttr) + SetFileAttr(Name,0); + + HANDLE hFile=CreateFile(Name,GENERIC_WRITE,FILE_SHARE_READ|FILE_SHARE_WRITE, + NULL,OPEN_EXISTING,FILE_FLAG_BACKUP_SEMANTICS,NULL); + if (hFile==INVALID_HANDLE_VALUE) + { + wchar LongName[NM]; + if (GetWinLongPath(Name,LongName,ASIZE(LongName))) + hFile=CreateFile(LongName,GENERIC_WRITE,FILE_SHARE_READ|FILE_SHARE_WRITE, + NULL,OPEN_EXISTING,FILE_FLAG_BACKUP_SEMANTICS,NULL); + } + + if (hFile==INVALID_HANDLE_VALUE) + return; + FILETIME fm,fc,fa; + if (sm) + ftm->GetWinFT(&fm); + if (sc) + ftc->GetWinFT(&fc); + if (sa) + fta->GetWinFT(&fa); + SetFileTime(hFile,sc ? &fc:NULL,sa ? &fa:NULL,sm ? &fm:NULL); + CloseHandle(hFile); + if (ResetAttr) + SetFileAttr(Name,DirAttr); +#endif +#if defined(_UNIX) || defined(_EMX) + File::SetCloseFileTimeByName(Name,ftm,fta); +#endif +} + + +bool IsRemovable(const wchar *Name) +{ +#if defined(_WIN_ALL) + wchar Root[NM]; + GetPathRoot(Name,Root,ASIZE(Root)); + int Type=GetDriveType(*Root!=0 ? Root:NULL); + return Type==DRIVE_REMOVABLE || Type==DRIVE_CDROM; +#else + return false; +#endif +} + + +#ifndef SFX_MODULE +int64 GetFreeDisk(const wchar *Name) +{ +#ifdef _WIN_ALL + wchar Root[NM]; + GetFilePath(Name,Root,ASIZE(Root)); + + ULARGE_INTEGER uiTotalSize,uiTotalFree,uiUserFree; + uiUserFree.u.LowPart=uiUserFree.u.HighPart=0; + if (GetDiskFreeSpaceEx(*Root!=0 ? Root:NULL,&uiUserFree,&uiTotalSize,&uiTotalFree) && + uiUserFree.u.HighPart<=uiTotalFree.u.HighPart) + return INT32TO64(uiUserFree.u.HighPart,uiUserFree.u.LowPart); + return 0; +#elif defined(_UNIX) + wchar Root[NM]; + GetFilePath(Name,Root,ASIZE(Root)); + char RootA[NM]; + WideToChar(Root,RootA,ASIZE(RootA)); + struct statvfs sfs; + if (statvfs(*RootA!=0 ? RootA:".",&sfs)!=0) + return 0; + int64 FreeSize=sfs.f_bsize; + FreeSize=FreeSize*sfs.f_bavail; + return FreeSize; +#else + return 0; +#endif +} +#endif + + +#if defined(_WIN_ALL) && !defined(SFX_MODULE) && !defined(SILENT) +// Return 'true' for FAT and FAT32, so we can adjust the maximum supported +// file size to 4 GB for these file systems. +bool IsFAT(const wchar *Name) +{ + wchar Root[NM]; + GetPathRoot(Name,Root,ASIZE(Root)); + wchar FileSystem[MAX_PATH+1]; + if (GetVolumeInformation(Root,NULL,0,NULL,NULL,NULL,FileSystem,ASIZE(FileSystem))) + return wcscmp(FileSystem,L"FAT")==0 || wcscmp(FileSystem,L"FAT32")==0; + return false; +} +#endif + + +bool FileExist(const wchar *Name) +{ +#ifdef _WIN_ALL + return GetFileAttr(Name)!=0xffffffff; +#elif defined(ENABLE_ACCESS) + char NameA[NM]; + WideToChar(Name,NameA,ASIZE(NameA)); + return access(NameA,0)==0; +#else + FindData FD; + return FindFile::FastFind(Name,&FD); +#endif +} + + +bool WildFileExist(const wchar *Name) +{ + if (IsWildcard(Name)) + { + FindFile Find; + Find.SetMask(Name); + FindData fd; + return Find.Next(&fd); + } + return FileExist(Name); +} + + +bool IsDir(uint Attr) +{ +#ifdef _WIN_ALL + return Attr!=0xffffffff && (Attr & FILE_ATTRIBUTE_DIRECTORY)!=0; +#endif +#if defined(_UNIX) + return (Attr & 0xF000)==0x4000; +#endif +} + + +bool IsUnreadable(uint Attr) +{ +#if defined(_UNIX) && defined(S_ISFIFO) && defined(S_ISSOCK) && defined(S_ISCHR) + return S_ISFIFO(Attr) || S_ISSOCK(Attr) || S_ISCHR(Attr); +#endif + return false; +} + + +bool IsLink(uint Attr) +{ +#ifdef _UNIX + return (Attr & 0xF000)==0xA000; +#elif defined(_WIN_ALL) + return (Attr & FILE_ATTRIBUTE_REPARSE_POINT)!=0; +#else + return false; +#endif +} + + + + + + +bool IsDeleteAllowed(uint FileAttr) +{ +#ifdef _WIN_ALL + return (FileAttr & (FILE_ATTRIBUTE_READONLY|FILE_ATTRIBUTE_SYSTEM|FILE_ATTRIBUTE_HIDDEN))==0; +#else + return (FileAttr & (S_IRUSR|S_IWUSR))==(S_IRUSR|S_IWUSR); +#endif +} + + +void PrepareToDelete(const wchar *Name) +{ +#if defined(_WIN_ALL) || defined(_EMX) + SetFileAttr(Name,0); +#endif +#ifdef _UNIX + if (Name!=NULL) + { + char NameA[NM]; + WideToChar(Name,NameA,ASIZE(NameA)); + chmod(NameA,S_IRUSR|S_IWUSR|S_IXUSR); + } +#endif +} + + +uint GetFileAttr(const wchar *Name) +{ +#ifdef _WIN_ALL + DWORD Attr=GetFileAttributes(Name); + if (Attr==0xffffffff) + { + wchar LongName[NM]; + if (GetWinLongPath(Name,LongName,ASIZE(LongName))) + Attr=GetFileAttributes(LongName); + } + return Attr; +#else + char NameA[NM]; + WideToChar(Name,NameA,ASIZE(NameA)); + struct stat st; + if (stat(NameA,&st)!=0) + return 0; + return st.st_mode; +#endif +} + + +bool SetFileAttr(const wchar *Name,uint Attr) +{ +#ifdef _WIN_ALL + bool Success=SetFileAttributes(Name,Attr)!=0; + if (!Success) + { + wchar LongName[NM]; + if (GetWinLongPath(Name,LongName,ASIZE(LongName))) + Success=SetFileAttributes(LongName,Attr)!=0; + } + return Success; +#elif defined(_UNIX) + char NameA[NM]; + WideToChar(Name,NameA,ASIZE(NameA)); + return chmod(NameA,(mode_t)Attr)==0; +#else + return false; +#endif +} + + +#if 0 +wchar *MkTemp(wchar *Name,size_t MaxSize) +{ + size_t Length=wcslen(Name); + + RarTime CurTime; + CurTime.SetCurrentTime(); + + // We cannot use CurTime.GetWin() as is, because its lowest bits can + // have low informational value, like being a zero or few fixed numbers. + uint Random=(uint)(CurTime.GetWin()/100000); + + // Using PID we guarantee that different RAR copies use different temp names + // even if started in exactly the same time. + uint PID=0; +#ifdef _WIN_ALL + PID=(uint)GetCurrentProcessId(); +#elif defined(_UNIX) + PID=(uint)getpid(); +#endif + + for (uint Attempt=0;;Attempt++) + { + uint Ext=Random%50000+Attempt; + wchar RndText[50]; + swprintf(RndText,ASIZE(RndText),L"%u.%03u",PID,Ext); + if (Length+wcslen(RndText)>=MaxSize || Attempt==1000) + return NULL; + wcscpy(Name+Length,RndText); + if (!FileExist(Name)) + break; + } + return Name; +} +#endif + + +#if !defined(SFX_MODULE) +void CalcFileSum(File *SrcFile,uint *CRC32,byte *Blake2,uint Threads,int64 Size,uint Flags) +{ + SaveFilePos SavePos(*SrcFile); +#ifndef SILENT + int64 FileLength=Size==INT64NDF ? SrcFile->FileLength() : Size; +#endif + + if ((Flags & (CALCFSUM_SHOWTEXT|CALCFSUM_SHOWPERCENT))!=0) + uiMsg(UIEVENT_FILESUMSTART); + + if ((Flags & CALCFSUM_CURPOS)==0) + SrcFile->Seek(0,SEEK_SET); + + const size_t BufSize=0x100000; + Array Data(BufSize); + + + DataHash HashCRC,HashBlake2; + HashCRC.Init(HASH_CRC32,Threads); + HashBlake2.Init(HASH_BLAKE2,Threads); + + int64 BlockCount=0; + int64 TotalRead=0; + while (true) + { + size_t SizeToRead; + if (Size==INT64NDF) // If we process the entire file. + SizeToRead=BufSize; // Then always attempt to read the entire buffer. + else + SizeToRead=(size_t)Min((int64)BufSize,Size); + int ReadSize=SrcFile->Read(&Data[0],SizeToRead); + if (ReadSize==0) + break; + TotalRead+=ReadSize; + + if ((++BlockCount & 0xf)==0) + { +#ifndef SILENT + if ((Flags & CALCFSUM_SHOWPROGRESS)!=0) + uiExtractProgress(TotalRead,FileLength,TotalRead,FileLength); + else + { + if ((Flags & CALCFSUM_SHOWPERCENT)!=0) + uiMsg(UIEVENT_FILESUMPROGRESS,ToPercent(TotalRead,FileLength)); + } +#endif + Wait(); + } + + if (CRC32!=NULL) + HashCRC.Update(&Data[0],ReadSize); + if (Blake2!=NULL) + HashBlake2.Update(&Data[0],ReadSize); + + if (Size!=INT64NDF) + Size-=ReadSize; + } + if ((Flags & CALCFSUM_SHOWPERCENT)!=0) + uiMsg(UIEVENT_FILESUMEND); + + if (CRC32!=NULL) + *CRC32=HashCRC.GetCRC32(); + if (Blake2!=NULL) + { + HashValue Result; + HashBlake2.Result(&Result); + memcpy(Blake2,Result.Digest,sizeof(Result.Digest)); + } +} +#endif + + +bool RenameFile(const wchar *SrcName,const wchar *DestName) +{ +#ifdef _WIN_ALL + bool Success=MoveFile(SrcName,DestName)!=0; + if (!Success) + { + wchar LongName1[NM],LongName2[NM]; + if (GetWinLongPath(SrcName,LongName1,ASIZE(LongName1)) && + GetWinLongPath(DestName,LongName2,ASIZE(LongName2))) + Success=MoveFile(LongName1,LongName2)!=0; + } + return Success; +#else + char SrcNameA[NM],DestNameA[NM]; + WideToChar(SrcName,SrcNameA,ASIZE(SrcNameA)); + WideToChar(DestName,DestNameA,ASIZE(DestNameA)); + bool Success=rename(SrcNameA,DestNameA)==0; + return Success; +#endif +} + + +bool DelFile(const wchar *Name) +{ +#ifdef _WIN_ALL + bool Success=DeleteFile(Name)!=0; + if (!Success) + { + wchar LongName[NM]; + if (GetWinLongPath(Name,LongName,ASIZE(LongName))) + Success=DeleteFile(LongName)!=0; + } + return Success; +#else + char NameA[NM]; + WideToChar(Name,NameA,ASIZE(NameA)); + bool Success=remove(NameA)==0; + return Success; +#endif +} + + + + +#if defined(_WIN_ALL) && !defined(SFX_MODULE) +bool SetFileCompression(const wchar *Name,bool State) +{ + HANDLE hFile=CreateFile(Name,FILE_READ_DATA|FILE_WRITE_DATA, + FILE_SHARE_READ|FILE_SHARE_WRITE,NULL,OPEN_EXISTING, + FILE_FLAG_BACKUP_SEMANTICS|FILE_FLAG_SEQUENTIAL_SCAN,NULL); + if (hFile==INVALID_HANDLE_VALUE) + { + wchar LongName[NM]; + if (GetWinLongPath(Name,LongName,ASIZE(LongName))) + hFile=CreateFile(LongName,FILE_READ_DATA|FILE_WRITE_DATA, + FILE_SHARE_READ|FILE_SHARE_WRITE,NULL,OPEN_EXISTING, + FILE_FLAG_BACKUP_SEMANTICS|FILE_FLAG_SEQUENTIAL_SCAN,NULL); + } + if (hFile==INVALID_HANDLE_VALUE) + return false; + SHORT NewState=State ? COMPRESSION_FORMAT_DEFAULT:COMPRESSION_FORMAT_NONE; + DWORD Result; + int RetCode=DeviceIoControl(hFile,FSCTL_SET_COMPRESSION,&NewState, + sizeof(NewState),NULL,0,&Result,NULL); + CloseHandle(hFile); + return RetCode!=0; +} +#endif + + + + + + + + + + diff --git a/reference/unrar/filefn.hpp b/reference/unrar/filefn.hpp new file mode 100644 index 00000000..1bda9b58 --- /dev/null +++ b/reference/unrar/filefn.hpp @@ -0,0 +1,50 @@ +#ifndef _RAR_FILEFN_ +#define _RAR_FILEFN_ + +enum MKDIR_CODE {MKDIR_SUCCESS,MKDIR_ERROR,MKDIR_BADPATH}; + +MKDIR_CODE MakeDir(const wchar *Name,bool SetAttr,uint Attr); +bool CreatePath(const wchar *Path,bool SkipLastName); +void SetDirTime(const wchar *Name,RarTime *ftm,RarTime *ftc,RarTime *fta); +bool IsRemovable(const wchar *Name); + +#ifndef SFX_MODULE +int64 GetFreeDisk(const wchar *Name); +#endif + +#if defined(_WIN_ALL) && !defined(SFX_MODULE) && !defined(SILENT) +bool IsFAT(const wchar *Root); +#endif + +bool FileExist(const wchar *Name); +bool WildFileExist(const wchar *Name); +bool IsDir(uint Attr); +bool IsUnreadable(uint Attr); +bool IsLink(uint Attr); +void SetSFXMode(const wchar *FileName); +void EraseDiskContents(const wchar *FileName); +bool IsDeleteAllowed(uint FileAttr); +void PrepareToDelete(const wchar *Name); +uint GetFileAttr(const wchar *Name); +bool SetFileAttr(const wchar *Name,uint Attr); +#if 0 +wchar* MkTemp(wchar *Name,size_t MaxSize); +#endif + +enum CALCFSUM_FLAGS {CALCFSUM_SHOWTEXT=1,CALCFSUM_SHOWPERCENT=2,CALCFSUM_SHOWPROGRESS=4,CALCFSUM_CURPOS=8}; + +void CalcFileSum(File *SrcFile,uint *CRC32,byte *Blake2,uint Threads,int64 Size=INT64NDF,uint Flags=0); + +bool RenameFile(const wchar *SrcName,const wchar *DestName); +bool DelFile(const wchar *Name); +bool DelDir(const wchar *Name); + +#if defined(_WIN_ALL) && !defined(SFX_MODULE) +bool SetFileCompression(const wchar *Name,bool State); +#endif + + + + + +#endif diff --git a/reference/unrar/filestr.cpp b/reference/unrar/filestr.cpp new file mode 100644 index 00000000..4a53132f --- /dev/null +++ b/reference/unrar/filestr.cpp @@ -0,0 +1,162 @@ +#include "rar.hpp" + +bool ReadTextFile( + const wchar *Name, + StringList *List, + bool Config, + bool AbortOnError, + RAR_CHARSET SrcCharset, + bool Unquote, + bool SkipComments, + bool ExpandEnvStr) +{ + wchar FileName[NM]; + *FileName=0; + + if (Name!=NULL) + if (Config) + GetConfigName(Name,FileName,ASIZE(FileName),true,false); + else + wcsncpyz(FileName,Name,ASIZE(FileName)); + + File SrcFile; + if (*FileName!=0) + { + bool OpenCode=AbortOnError ? SrcFile.WOpen(FileName):SrcFile.Open(FileName,0); + + if (!OpenCode) + { + if (AbortOnError) + ErrHandler.Exit(RARX_OPEN); + return false; + } + } + else + SrcFile.SetHandleType(FILE_HANDLESTD); + + uint DataSize=0,ReadSize; + const int ReadBlock=4096; + + Array Data(ReadBlock); + while ((ReadSize=SrcFile.Read(&Data[DataSize],ReadBlock))!=0) + { + DataSize+=ReadSize; + Data.Add(ReadSize); // Always have ReadBlock available for next data. + } + // Set to really read size, so we can zero terminate it correctly. + Data.Alloc(DataSize); + + int LowEndian=DataSize>=2 && Data[0]==255 && Data[1]==254 ? 1:0; + int BigEndian=DataSize>=2 && Data[0]==254 && Data[1]==255 ? 1:0; + bool Utf8=DataSize>=3 && Data[0]==0xef && Data[1]==0xbb && Data[2]==0xbf; + + if (SrcCharset==RCH_DEFAULT) + { + if (LowEndian || BigEndian) + for (size_t I=2;I DataW; + + if (SrcCharset==RCH_DEFAULT || SrcCharset==RCH_OEM || SrcCharset==RCH_ANSI) + { + Data.Push(0); // Zero terminate. +#if defined(_WIN_ALL) + if (SrcCharset==RCH_OEM) + OemToCharA((char *)&Data[0],(char *)&Data[0]); +#endif + DataW.Alloc(Data.Size()); + CharToWide((char *)&Data[0],&DataW[0],DataW.Size()); + } + + if (SrcCharset==RCH_UNICODE) + { + size_t Start=2; // Skip byte order mark. + if (!LowEndian && !BigEndian) // No byte order mask. + { + Start=0; + LowEndian=1; + } + + DataW.Alloc(Data.Size()/2+1); + size_t End=Data.Size() & ~1; // We need even bytes number for UTF-16. + for (size_t I=Start;I=CurStr;SpacePtr--) + { + if (*SpacePtr!=' ' && *SpacePtr!='\t') + break; + *SpacePtr=0; + } + + if (Unquote && *CurStr=='\"') + { + size_t Length=wcslen(CurStr); + if (CurStr[Length-1]=='\"') + { + CurStr[Length-1]=0; + CurStr++; + } + } + + bool Expanded=false; +#if defined(_WIN_ALL) + if (ExpandEnvStr && *CurStr=='%') // Expand environment variables in Windows. + { + wchar ExpName[NM]; + *ExpName=0; + DWORD Result=ExpandEnvironmentStrings(CurStr,ExpName,ASIZE(ExpName)); + Expanded=Result!=0 && ResultAddString(ExpName); + } +#endif + if (!Expanded && *CurStr!=0) + List->AddString(CurStr); + + if (Done) + break; + CurStr=NextStr+1; + while (*CurStr=='\r' || *CurStr=='\n') + CurStr++; + } + return true; +} diff --git a/reference/unrar/filestr.hpp b/reference/unrar/filestr.hpp new file mode 100644 index 00000000..8dfdf64b --- /dev/null +++ b/reference/unrar/filestr.hpp @@ -0,0 +1,15 @@ +#ifndef _RAR_FILESTR_ +#define _RAR_FILESTR_ + +bool ReadTextFile( + const wchar *Name, + StringList *List, + bool Config, + bool AbortOnError=false, + RAR_CHARSET SrcCharset=RCH_DEFAULT, + bool Unquote=false, + bool SkipComments=false, + bool ExpandEnvStr=false +); + +#endif diff --git a/reference/unrar/find.cpp b/reference/unrar/find.cpp new file mode 100644 index 00000000..812af6b3 --- /dev/null +++ b/reference/unrar/find.cpp @@ -0,0 +1,218 @@ +#include "rar.hpp" + +FindFile::FindFile() +{ + *FindMask=0; + FirstCall=true; +#ifdef _WIN_ALL + hFind=INVALID_HANDLE_VALUE; +#else + dirp=NULL; +#endif +} + + +FindFile::~FindFile() +{ +#ifdef _WIN_ALL + if (hFind!=INVALID_HANDLE_VALUE) + FindClose(hFind); +#else + if (dirp!=NULL) + closedir(dirp); +#endif +} + + +void FindFile::SetMask(const wchar *Mask) +{ + wcscpy(FindMask,Mask); + FirstCall=true; +} + + +bool FindFile::Next(FindData *fd,bool GetSymLink) +{ + fd->Error=false; + if (*FindMask==0) + return false; +#ifdef _WIN_ALL + if (FirstCall) + { + if ((hFind=Win32Find(INVALID_HANDLE_VALUE,FindMask,fd))==INVALID_HANDLE_VALUE) + return false; + } + else + if (Win32Find(hFind,FindMask,fd)==INVALID_HANDLE_VALUE) + return false; +#else + if (FirstCall) + { + wchar DirName[NM]; + wcsncpyz(DirName,FindMask,ASIZE(DirName)); + RemoveNameFromPath(DirName); + if (*DirName==0) + wcscpy(DirName,L"."); + char DirNameA[NM]; + WideToChar(DirName,DirNameA,ASIZE(DirNameA)); + if ((dirp=opendir(DirNameA))==NULL) + { + fd->Error=(errno!=ENOENT); + return false; + } + } + while (1) + { + struct dirent *ent=readdir(dirp); + if (ent==NULL) + return false; + if (strcmp(ent->d_name,".")==0 || strcmp(ent->d_name,"..")==0) + continue; + wchar Name[NM]; + if (!CharToWide(ent->d_name,Name,ASIZE(Name))) + uiMsg(UIERROR_INVALIDNAME,UINULL,Name); + + if (CmpName(FindMask,Name,MATCH_NAMES)) + { + wchar FullName[NM]; + wcscpy(FullName,FindMask); + *PointToName(FullName)=0; + if (wcslen(FullName)+wcslen(Name)>=ASIZE(FullName)-1) + { + uiMsg(UIERROR_PATHTOOLONG,FullName,L"",Name); + return false; + } + wcscat(FullName,Name); + if (!FastFind(FullName,fd,GetSymLink)) + { + ErrHandler.OpenErrorMsg(FullName); + continue; + } + wcscpy(fd->Name,FullName); + break; + } + } +#endif + fd->Flags=0; + fd->IsDir=IsDir(fd->FileAttr); + fd->IsLink=IsLink(fd->FileAttr); + + FirstCall=false; + wchar *NameOnly=PointToName(fd->Name); + if (wcscmp(NameOnly,L".")==0 || wcscmp(NameOnly,L"..")==0) + return Next(fd); + return true; +} + + +bool FindFile::FastFind(const wchar *FindMask,FindData *fd,bool GetSymLink) +{ + fd->Error=false; +#ifndef _UNIX + if (IsWildcard(FindMask)) + return false; +#endif +#ifdef _WIN_ALL + HANDLE hFind=Win32Find(INVALID_HANDLE_VALUE,FindMask,fd); + if (hFind==INVALID_HANDLE_VALUE) + return false; + FindClose(hFind); +#else + char FindMaskA[NM]; + WideToChar(FindMask,FindMaskA,ASIZE(FindMaskA)); + + struct stat st; + if (GetSymLink) + { +#ifdef SAVE_LINKS + if (lstat(FindMaskA,&st)!=0) +#else + if (stat(FindMaskA,&st)!=0) +#endif + { + fd->Error=(errno!=ENOENT); + return false; + } + } + else + if (stat(FindMaskA,&st)!=0) + { + fd->Error=(errno!=ENOENT); + return false; + } + fd->FileAttr=st.st_mode; + fd->Size=st.st_size; + +#ifdef UNIX_TIME_NS + fd->mtime.SetUnixNS(st.st_mtim.tv_sec*(uint64)1000000000+st.st_mtim.tv_nsec); + fd->atime.SetUnixNS(st.st_atim.tv_sec*(uint64)1000000000+st.st_atim.tv_nsec); + fd->ctime.SetUnixNS(st.st_ctim.tv_sec*(uint64)1000000000+st.st_ctim.tv_nsec); +#else + fd->mtime.SetUnix(st.st_mtime); + fd->atime.SetUnix(st.st_atime); + fd->ctime.SetUnix(st.st_ctime); +#endif + + wcsncpyz(fd->Name,FindMask,ASIZE(fd->Name)); +#endif + fd->Flags=0; + fd->IsDir=IsDir(fd->FileAttr); + fd->IsLink=IsLink(fd->FileAttr); + + return true; +} + + +#ifdef _WIN_ALL +HANDLE FindFile::Win32Find(HANDLE hFind,const wchar *Mask,FindData *fd) +{ + WIN32_FIND_DATA FindData; + if (hFind==INVALID_HANDLE_VALUE) + { + hFind=FindFirstFile(Mask,&FindData); + if (hFind==INVALID_HANDLE_VALUE) + { + wchar LongMask[NM]; + if (GetWinLongPath(Mask,LongMask,ASIZE(LongMask))) + hFind=FindFirstFile(LongMask,&FindData); + } + if (hFind==INVALID_HANDLE_VALUE) + { + int SysErr=GetLastError(); + // We must not issue an error for "file not found" and "path not found", + // because it is normal to not find anything for wildcard mask when + // archiving. Also searching for non-existent file is normal in some + // other modules, like WinRAR scanning for winrar_theme_description.txt + // to check if any themes are available. + fd->Error=SysErr!=ERROR_FILE_NOT_FOUND && + SysErr!=ERROR_PATH_NOT_FOUND && + SysErr!=ERROR_NO_MORE_FILES; + } + } + else + if (!FindNextFile(hFind,&FindData)) + { + hFind=INVALID_HANDLE_VALUE; + fd->Error=GetLastError()!=ERROR_NO_MORE_FILES; + } + + if (hFind!=INVALID_HANDLE_VALUE) + { + wcsncpyz(fd->Name,Mask,ASIZE(fd->Name)); + SetName(fd->Name,FindData.cFileName,ASIZE(fd->Name)); + fd->Size=INT32TO64(FindData.nFileSizeHigh,FindData.nFileSizeLow); + fd->FileAttr=FindData.dwFileAttributes; + fd->ftCreationTime=FindData.ftCreationTime; + fd->ftLastAccessTime=FindData.ftLastAccessTime; + fd->ftLastWriteTime=FindData.ftLastWriteTime; + fd->mtime.SetWinFT(&FindData.ftLastWriteTime); + fd->ctime.SetWinFT(&FindData.ftCreationTime); + fd->atime.SetWinFT(&FindData.ftLastAccessTime); + + + } + fd->Flags=0; + return hFind; +} +#endif + diff --git a/reference/unrar/find.hpp b/reference/unrar/find.hpp new file mode 100644 index 00000000..250637f8 --- /dev/null +++ b/reference/unrar/find.hpp @@ -0,0 +1,49 @@ +#ifndef _RAR_FINDDATA_ +#define _RAR_FINDDATA_ + +enum FINDDATA_FLAGS { + FDDF_SECONDDIR=1 // Second encounter of same directory in SCAN_GETDIRSTWICE ScanTree mode. +}; + +struct FindData +{ + wchar Name[NM]; + uint64 Size; + uint FileAttr; + bool IsDir; + bool IsLink; + RarTime mtime; + RarTime ctime; + RarTime atime; +#ifdef _WIN_ALL + FILETIME ftCreationTime; + FILETIME ftLastAccessTime; + FILETIME ftLastWriteTime; +#endif + uint Flags; + bool Error; +}; + +class FindFile +{ + private: +#ifdef _WIN_ALL + static HANDLE Win32Find(HANDLE hFind,const wchar *Mask,FindData *fd); +#endif + + wchar FindMask[NM]; + bool FirstCall; +#ifdef _WIN_ALL + HANDLE hFind; +#else + DIR *dirp; +#endif + public: + FindFile(); + ~FindFile(); + void SetMask(const wchar *Mask); + bool Next(FindData *fd,bool GetSymLink=false); + static bool FastFind(const wchar *FindMask,FindData *fd,bool GetSymLink=false); +}; + +#endif diff --git a/reference/unrar/getbits.cpp b/reference/unrar/getbits.cpp new file mode 100644 index 00000000..e4db2695 --- /dev/null +++ b/reference/unrar/getbits.cpp @@ -0,0 +1,52 @@ +#include "rar.hpp" + +BitInput::BitInput(bool AllocBuffer) +{ + ExternalBuffer=false; + if (AllocBuffer) + { + // getbits32 attempts to read data from InAddr, ... InAddr+3 positions. + // So let's allocate 3 additional bytes for situation, when we need to + // read only 1 byte from the last position of buffer and avoid a crash + // from access to next 3 bytes, which contents we do not need. + size_t BufSize=MAX_SIZE+3; + InBuf=new byte[BufSize]; + + // Ensure that we get predictable results when accessing bytes in area + // not filled with read data. + memset(InBuf,0,BufSize); + } + else + InBuf=NULL; +} + + +BitInput::~BitInput() +{ + if (!ExternalBuffer) + delete[] InBuf; +} + + +void BitInput::faddbits(uint Bits) +{ + // Function wrapped version of inline addbits to save code size. + addbits(Bits); +} + + +uint BitInput::fgetbits() +{ + // Function wrapped version of inline getbits to save code size. + return getbits(); +} + + +void BitInput::SetExternalBuffer(byte *Buf) +{ + if (InBuf!=NULL && !ExternalBuffer) + delete[] InBuf; + InBuf=Buf; + ExternalBuffer=true; +} + diff --git a/reference/unrar/getbits.hpp b/reference/unrar/getbits.hpp new file mode 100644 index 00000000..2e151da9 --- /dev/null +++ b/reference/unrar/getbits.hpp @@ -0,0 +1,68 @@ +#ifndef _RAR_GETBITS_ +#define _RAR_GETBITS_ + +class BitInput +{ + public: + enum BufferSize {MAX_SIZE=0x8000}; // Size of input buffer. + + int InAddr; // Curent byte position in the buffer. + int InBit; // Current bit position in the current byte. + + bool ExternalBuffer; + public: + BitInput(bool AllocBuffer); + ~BitInput(); + + byte *InBuf; // Dynamically allocated input buffer. + + void InitBitInput() + { + InAddr=InBit=0; + } + + // Move forward by 'Bits' bits. + void addbits(uint Bits) + { + Bits+=InBit; + InAddr+=Bits>>3; + InBit=Bits&7; + } + + // Return 16 bits from current position in the buffer. + // Bit at (InAddr,InBit) has the highest position in returning data. + uint getbits() + { + uint BitField=(uint)InBuf[InAddr] << 16; + BitField|=(uint)InBuf[InAddr+1] << 8; + BitField|=(uint)InBuf[InAddr+2]; + BitField >>= (8-InBit); + return BitField & 0xffff; + } + + // Return 32 bits from current position in the buffer. + // Bit at (InAddr,InBit) has the highest position in returning data. + uint getbits32() + { + uint BitField=(uint)InBuf[InAddr] << 24; + BitField|=(uint)InBuf[InAddr+1] << 16; + BitField|=(uint)InBuf[InAddr+2] << 8; + BitField|=(uint)InBuf[InAddr+3]; + BitField <<= InBit; + BitField|=(uint)InBuf[InAddr+4] >> (8-InBit); + return BitField & 0xffffffff; + } + + void faddbits(uint Bits); + uint fgetbits(); + + // Check if buffer has enough space for IncPtr bytes. Returns 'true' + // if buffer will be overflown. + bool Overflow(uint IncPtr) + { + return InAddr+IncPtr>=MAX_SIZE; + } + + void SetExternalBuffer(byte *Buf); +}; +#endif diff --git a/reference/unrar/global.cpp b/reference/unrar/global.cpp new file mode 100644 index 00000000..3975813a --- /dev/null +++ b/reference/unrar/global.cpp @@ -0,0 +1,7 @@ +#define INCLUDEGLOBAL + +#if defined(__BORLANDC__) || defined(_MSC_VER) +#pragma hdrstop +#endif + +#include "rar.hpp" diff --git a/reference/unrar/global.hpp b/reference/unrar/global.hpp new file mode 100644 index 00000000..35c6cf91 --- /dev/null +++ b/reference/unrar/global.hpp @@ -0,0 +1,14 @@ +#ifndef _RAR_GLOBAL_ +#define _RAR_GLOBAL_ + +#ifdef INCLUDEGLOBAL + #define EXTVAR +#else + #define EXTVAR extern +#endif + +EXTVAR ErrorHandler ErrHandler; + + + +#endif diff --git a/reference/unrar/hardlinks.cpp b/reference/unrar/hardlinks.cpp new file mode 100644 index 00000000..cf0b25a9 --- /dev/null +++ b/reference/unrar/hardlinks.cpp @@ -0,0 +1,34 @@ +bool ExtractHardlink(wchar *NameNew,wchar *NameExisting,size_t NameExistingSize) +{ + SlashToNative(NameExisting,NameExisting,NameExistingSize); // Not needed for RAR 5.1+ archives. + + if (!FileExist(NameExisting)) + return false; + CreatePath(NameNew,true); + +#ifdef _WIN_ALL + bool Success=CreateHardLink(NameNew,NameExisting,NULL)!=0; + if (!Success) + { + uiMsg(UIERROR_HLINKCREATE,NameNew); + ErrHandler.SysErrMsg(); + ErrHandler.SetErrorCode(RARX_CREATE); + } + return Success; +#elif defined(_UNIX) + char NameExistingA[NM],NameNewA[NM]; + WideToChar(NameExisting,NameExistingA,ASIZE(NameExistingA)); + WideToChar(NameNew,NameNewA,ASIZE(NameNewA)); + bool Success=link(NameExistingA,NameNewA)==0; + if (!Success) + { + uiMsg(UIERROR_HLINKCREATE,NameNew); + ErrHandler.SysErrMsg(); + ErrHandler.SetErrorCode(RARX_CREATE); + } + return Success; +#else + return false; +#endif +} + diff --git a/reference/unrar/hash.cpp b/reference/unrar/hash.cpp new file mode 100644 index 00000000..42791f4f --- /dev/null +++ b/reference/unrar/hash.cpp @@ -0,0 +1,135 @@ +#include "rar.hpp" + +void HashValue::Init(HASH_TYPE Type) +{ + HashValue::Type=Type; + + // Zero length data CRC32 is 0. It is important to set it when creating + // headers with no following data like directories or symlinks. + if (Type==HASH_RAR14 || Type==HASH_CRC32) + CRC32=0; + if (Type==HASH_BLAKE2) + { + // dd0e891776933f43c7d032b08a917e25741f8aa9a12c12e1cac8801500f2ca4f + // is BLAKE2sp hash of empty data. We init the structure to this value, + // so if we create a file or service header with no following data like + // "file copy" or "symlink", we set the checksum to proper value avoiding + // additional header type or size checks when extracting. + static byte EmptyHash[32]={ + 0xdd, 0x0e, 0x89, 0x17, 0x76, 0x93, 0x3f, 0x43, + 0xc7, 0xd0, 0x32, 0xb0, 0x8a, 0x91, 0x7e, 0x25, + 0x74, 0x1f, 0x8a, 0xa9, 0xa1, 0x2c, 0x12, 0xe1, + 0xca, 0xc8, 0x80, 0x15, 0x00, 0xf2, 0xca, 0x4f + }; + memcpy(Digest,EmptyHash,sizeof(Digest)); + } +} + + +bool HashValue::operator == (const HashValue &cmp) +{ + if (Type==HASH_NONE || cmp.Type==HASH_NONE) + return true; + if (Type==HASH_RAR14 && cmp.Type==HASH_RAR14 || + Type==HASH_CRC32 && cmp.Type==HASH_CRC32) + return CRC32==cmp.CRC32; + if (Type==HASH_BLAKE2 && cmp.Type==HASH_BLAKE2) + return memcmp(Digest,cmp.Digest,sizeof(Digest))==0; + return false; +} + + +DataHash::DataHash() +{ + blake2ctx=NULL; + HashType=HASH_NONE; +#ifdef RAR_SMP + ThPool=NULL; + MaxThreads=0; +#endif +} + + +DataHash::~DataHash() +{ +#ifdef RAR_SMP + DestroyThreadPool(ThPool); +#endif + cleandata(&CurCRC32, sizeof(CurCRC32)); + if (blake2ctx!=NULL) + { + cleandata(blake2ctx, sizeof(blake2sp_state)); + delete blake2ctx; + } +} + + +void DataHash::Init(HASH_TYPE Type,uint MaxThreads) +{ + if (blake2ctx==NULL) + blake2ctx=new blake2sp_state; + HashType=Type; + if (Type==HASH_RAR14) + CurCRC32=0; + if (Type==HASH_CRC32) + CurCRC32=0xffffffff; // Initial CRC32 value. + if (Type==HASH_BLAKE2) + blake2sp_init(blake2ctx); +#ifdef RAR_SMP + DataHash::MaxThreads=Min(MaxThreads,MaxHashThreads); +#endif +} + + +void DataHash::Update(const void *Data,size_t DataSize) +{ +#ifndef SFX_MODULE + if (HashType==HASH_RAR14) + CurCRC32=Checksum14((ushort)CurCRC32,Data,DataSize); +#endif + if (HashType==HASH_CRC32) + CurCRC32=CRC32(CurCRC32,Data,DataSize); + + if (HashType==HASH_BLAKE2) + { +#ifdef RAR_SMP + if (MaxThreads>1 && ThPool==NULL) + ThPool=CreateThreadPool(); + blake2ctx->ThPool=ThPool; + blake2ctx->MaxThreads=MaxThreads; +#endif + blake2sp_update( blake2ctx, (byte *)Data, DataSize); + } +} + + +void DataHash::Result(HashValue *Result) +{ + Result->Type=HashType; + if (HashType==HASH_RAR14) + Result->CRC32=CurCRC32; + if (HashType==HASH_CRC32) + Result->CRC32=CurCRC32^0xffffffff; + if (HashType==HASH_BLAKE2) + { + // Preserve the original context, so we can continue hashing if necessary. + blake2sp_state res=*blake2ctx; + blake2sp_final(&res,Result->Digest); + } +} + + +uint DataHash::GetCRC32() +{ + return HashType==HASH_CRC32 ? CurCRC32^0xffffffff : 0; +} + + +bool DataHash::Cmp(HashValue *CmpValue,byte *Key) +{ + HashValue Final; + Result(&Final); + if (Key!=NULL) + ConvertHashToMAC(&Final,Key); + return Final==*CmpValue; +} diff --git a/reference/unrar/hash.hpp b/reference/unrar/hash.hpp new file mode 100644 index 00000000..b7d879f6 --- /dev/null +++ b/reference/unrar/hash.hpp @@ -0,0 +1,52 @@ +#ifndef _RAR_DATAHASH_ +#define _RAR_DATAHASH_ + +enum HASH_TYPE {HASH_NONE,HASH_RAR14,HASH_CRC32,HASH_BLAKE2}; + +struct HashValue +{ + void Init(HASH_TYPE Type); + bool operator == (const HashValue &cmp); + bool operator != (const HashValue &cmp) {return !(*this==cmp);} + + HASH_TYPE Type; + union + { + uint CRC32; + byte Digest[SHA256_DIGEST_SIZE]; + }; +}; + + +#ifdef RAR_SMP +class ThreadPool; +class DataHash; +#endif + + +class DataHash +{ + private: + HASH_TYPE HashType; + uint CurCRC32; + blake2sp_state *blake2ctx; + +#ifdef RAR_SMP + ThreadPool *ThPool; + + uint MaxThreads; + // Upper limit for maximum threads to prevent wasting threads in pool. + static const uint MaxHashThreads=8; +#endif + public: + DataHash(); + ~DataHash(); + void Init(HASH_TYPE Type,uint MaxThreads); + void Update(const void *Data,size_t DataSize); + void Result(HashValue *Result); + uint GetCRC32(); + bool Cmp(HashValue *CmpValue,byte *Key); + HASH_TYPE Type() {return HashType;} +}; + +#endif diff --git a/reference/unrar/headers.cpp b/reference/unrar/headers.cpp new file mode 100644 index 00000000..b042dc39 --- /dev/null +++ b/reference/unrar/headers.cpp @@ -0,0 +1,61 @@ +#include "rar.hpp" + +void FileHeader::Reset(size_t SubDataSize) +{ + SubData.Alloc(SubDataSize); + BaseBlock::Reset(); + FileHash.Init(HASH_NONE); + mtime.Reset(); + atime.Reset(); + ctime.Reset(); + SplitBefore=false; + SplitAfter=false; + + UnknownUnpSize=0; + + SubFlags=0; // Important for RAR 3.0 subhead. + + CryptMethod=CRYPT_NONE; + Encrypted=false; + SaltSet=false; + UsePswCheck=false; + UseHashKey=false; + Lg2Count=0; + + Solid=false; + Dir=false; + WinSize=0; + Inherited=false; + SubBlock=false; + CommentInHeader=false; + Version=false; + LargeFile=false; + + RedirType=FSREDIR_NONE; + DirTarget=false; + UnixOwnerSet=false; +} + + +FileHeader& FileHeader::operator = (FileHeader &hd) +{ + SubData.Reset(); + memcpy(this,&hd,sizeof(*this)); + SubData.CleanData(); + SubData=hd.SubData; + return *this; +} + + +void MainHeader::Reset() +{ + HighPosAV=0; + PosAV=0; + CommentInHeader=false; + PackComment=false; + Locator=false; + QOpenOffset=0; + QOpenMaxSize=0; + RROffset=0; + RRMaxSize=0; +} diff --git a/reference/unrar/headers.hpp b/reference/unrar/headers.hpp new file mode 100644 index 00000000..94864789 --- /dev/null +++ b/reference/unrar/headers.hpp @@ -0,0 +1,380 @@ +#ifndef _RAR_HEADERS_ +#define _RAR_HEADERS_ + +#define SIZEOF_MARKHEAD3 7 // Size of RAR 4.x archive mark header. +#define SIZEOF_MAINHEAD14 7 // Size of RAR 1.4 main archive header. +#define SIZEOF_MAINHEAD3 13 // Size of RAR 4.x main archive header. +#define SIZEOF_FILEHEAD14 21 // Size of RAR 1.4 file header. +#define SIZEOF_FILEHEAD3 32 // Size of RAR 3.0 file header. +#define SIZEOF_SHORTBLOCKHEAD 7 +#define SIZEOF_LONGBLOCKHEAD 11 +#define SIZEOF_SUBBLOCKHEAD 14 +#define SIZEOF_COMMHEAD 13 +#define SIZEOF_PROTECTHEAD 26 +#define SIZEOF_AVHEAD 14 +#define SIZEOF_SIGNHEAD 15 +#define SIZEOF_UOHEAD 18 +#define SIZEOF_MACHEAD 22 +#define SIZEOF_EAHEAD 24 +#define SIZEOF_BEEAHEAD 24 +#define SIZEOF_STREAMHEAD 26 + +#define VER_PACK 29 +#define VER_PACK5 50 // It is stored as 0, but we subtract 50 when saving an archive. +#define VER_UNPACK 29 +#define VER_UNPACK5 50 // It is stored as 0, but we add 50 when reading an archive. + +#define MHD_VOLUME 0x0001U + +// Old style main archive comment embed into main archive header. Must not +// be used in new archives anymore. +#define MHD_COMMENT 0x0002U + +#define MHD_LOCK 0x0004U +#define MHD_SOLID 0x0008U +#define MHD_PACK_COMMENT 0x0010U +#define MHD_NEWNUMBERING 0x0010U +#define MHD_AV 0x0020U +#define MHD_PROTECT 0x0040U +#define MHD_PASSWORD 0x0080U +#define MHD_FIRSTVOLUME 0x0100U + +#define LHD_SPLIT_BEFORE 0x0001U +#define LHD_SPLIT_AFTER 0x0002U +#define LHD_PASSWORD 0x0004U + +// Old style file comment embed into file header. Must not be used +// in new archives anymore. +#define LHD_COMMENT 0x0008U + +// For non-file subheaders it denotes 'subblock having a parent file' flag. +#define LHD_SOLID 0x0010U + + +#define LHD_WINDOWMASK 0x00e0U +#define LHD_WINDOW64 0x0000U +#define LHD_WINDOW128 0x0020U +#define LHD_WINDOW256 0x0040U +#define LHD_WINDOW512 0x0060U +#define LHD_WINDOW1024 0x0080U +#define LHD_WINDOW2048 0x00a0U +#define LHD_WINDOW4096 0x00c0U +#define LHD_DIRECTORY 0x00e0U + +#define LHD_LARGE 0x0100U +#define LHD_UNICODE 0x0200U +#define LHD_SALT 0x0400U +#define LHD_VERSION 0x0800U +#define LHD_EXTTIME 0x1000U + +#define SKIP_IF_UNKNOWN 0x4000U +#define LONG_BLOCK 0x8000U + +#define EARC_NEXT_VOLUME 0x0001U // Not last volume. +#define EARC_DATACRC 0x0002U // Store CRC32 of RAR archive (now is used only in volumes). +#define EARC_REVSPACE 0x0004U // Reserve space for end of REV file 7 byte record. +#define EARC_VOLNUMBER 0x0008U // Store a number of current volume. + +enum HEADER_TYPE { + // RAR 5.0 header types. + HEAD_MARK=0x00, HEAD_MAIN=0x01, HEAD_FILE=0x02, HEAD_SERVICE=0x03, + HEAD_CRYPT=0x04, HEAD_ENDARC=0x05, HEAD_UNKNOWN=0xff, + + // RAR 1.5 - 4.x header types. + HEAD3_MARK=0x72,HEAD3_MAIN=0x73,HEAD3_FILE=0x74,HEAD3_CMT=0x75, + HEAD3_AV=0x76,HEAD3_OLDSERVICE=0x77,HEAD3_PROTECT=0x78,HEAD3_SIGN=0x79, + HEAD3_SERVICE=0x7a,HEAD3_ENDARC=0x7b +}; + +enum { EA_HEAD=0x100,UO_HEAD=0x101,MAC_HEAD=0x102,BEEA_HEAD=0x103, + NTACL_HEAD=0x104,STREAM_HEAD=0x105 }; + + +// Internal implementation, depends on archive format version. +enum HOST_SYSTEM { + // RAR 5.0 host OS + HOST5_WINDOWS=0,HOST5_UNIX=1, + + // RAR 3.0 host OS. + HOST_MSDOS=0,HOST_OS2=1,HOST_WIN32=2,HOST_UNIX=3,HOST_MACOS=4, + HOST_BEOS=5,HOST_MAX +}; + +// Unified archive format independent implementation. +enum HOST_SYSTEM_TYPE { + HSYS_WINDOWS, HSYS_UNIX, HSYS_UNKNOWN +}; + + +// We also use these values in extra field, so do not modify them. +enum FILE_SYSTEM_REDIRECT { + FSREDIR_NONE=0, FSREDIR_UNIXSYMLINK, FSREDIR_WINSYMLINK, FSREDIR_JUNCTION, + FSREDIR_HARDLINK, FSREDIR_FILECOPY +}; + + +#define SUBHEAD_TYPE_CMT L"CMT" +#define SUBHEAD_TYPE_QOPEN L"QO" +#define SUBHEAD_TYPE_ACL L"ACL" +#define SUBHEAD_TYPE_STREAM L"STM" +#define SUBHEAD_TYPE_UOWNER L"UOW" +#define SUBHEAD_TYPE_AV L"AV" +#define SUBHEAD_TYPE_RR L"RR" +#define SUBHEAD_TYPE_OS2EA L"EA2" + +/* new file inherits a subblock when updating a host file */ +#define SUBHEAD_FLAGS_INHERITED 0x80000000 + +#define SUBHEAD_FLAGS_CMT_UNICODE 0x00000001 + + +struct MarkHeader +{ + byte Mark[8]; + + // Following fields are virtual and not present in real blocks. + uint HeadSize; +}; + + +struct BaseBlock +{ + uint HeadCRC; // 'ushort' for RAR 1.5. + HEADER_TYPE HeaderType; // 1 byte for RAR 1.5. + uint Flags; // 'ushort' for RAR 1.5. + uint HeadSize; // 'ushort' for RAR 1.5, up to 2 MB for RAR 5.0. + + bool SkipIfUnknown; + + void Reset() + { + SkipIfUnknown=false; + } +}; + + +struct BlockHeader:BaseBlock +{ + uint DataSize; +}; + + +struct MainHeader:BaseBlock +{ + ushort HighPosAV; + uint PosAV; + bool CommentInHeader; + bool PackComment; // For RAR 1.4 archive format only. + bool Locator; + uint64 QOpenOffset; // Offset of quick list record. + uint64 QOpenMaxSize; // Maximum size of QOpen offset in locator extra field. + uint64 RROffset; // Offset of recovery record. + uint64 RRMaxSize; // Maximum size of RR offset in locator extra field. + void Reset(); +}; + + +struct FileHeader:BlockHeader +{ + byte HostOS; + byte UnpVer; + byte Method; + union { + uint FileAttr; + uint SubFlags; + }; + wchar FileName[NM]; + + Array SubData; + + RarTime mtime; + RarTime ctime; + RarTime atime; + + int64 PackSize; + int64 UnpSize; + int64 MaxSize; // Reserve size bytes for vint of this size. + + HashValue FileHash; + + uint FileFlags; + + bool SplitBefore; + bool SplitAfter; + + bool UnknownUnpSize; + + bool Encrypted; + CRYPT_METHOD CryptMethod; + bool SaltSet; + byte Salt[SIZE_SALT50]; + byte InitV[SIZE_INITV]; + bool UsePswCheck; + byte PswCheck[SIZE_PSWCHECK]; + + // Use HMAC calculated from HashKey and checksum instead of plain checksum. + bool UseHashKey; + + // Key to convert checksum to HMAC. Derived from password with PBKDF2 + // using additional iterations. + byte HashKey[SHA256_DIGEST_SIZE]; + + uint Lg2Count; // Log2 of PBKDF2 repetition count. + + bool Solid; + bool Dir; + bool CommentInHeader; // RAR 2.0 file comment. + bool Version; // name.ext;ver file name containing the version number. + size_t WinSize; + bool Inherited; // New file inherits a subblock when updating a host file (for subblocks only). + + // 'true' if file sizes use 8 bytes instead of 4. Not used in RAR 5.0. + bool LargeFile; + + // 'true' for HEAD_SERVICE block, which is a child of preceding file block. + // RAR 4.x uses 'solid' flag to indicate child subheader blocks in archives. + bool SubBlock; + + HOST_SYSTEM_TYPE HSType; + + FILE_SYSTEM_REDIRECT RedirType; + wchar RedirName[NM]; + bool DirTarget; + + bool UnixOwnerSet,UnixOwnerNumeric,UnixGroupNumeric; + char UnixOwnerName[256],UnixGroupName[256]; +#ifdef _UNIX + uid_t UnixOwnerID; + gid_t UnixGroupID; +#else // Need these Unix fields in Windows too for 'list' command. + uint UnixOwnerID; + uint UnixGroupID; +#endif + + void Reset(size_t SubDataSize=0); + + bool CmpName(const wchar *Name) + { + return(wcscmp(FileName,Name)==0); + } + + FileHeader& operator = (FileHeader &hd); +}; + + +struct EndArcHeader:BaseBlock +{ + // Optional CRC32 of entire archive up to start of EndArcHeader block. + // Present in RAR 4.x archives if EARC_DATACRC flag is set. + uint ArcDataCRC; + + uint VolNumber; // Optional number of current volume. + + // 7 additional zero bytes can be stored here if EARC_REVSPACE is set. + + bool NextVolume; // Not last volume. + bool DataCRC; + bool RevSpace; + bool StoreVolNumber; + void Reset() + { + BaseBlock::Reset(); + NextVolume=false; + DataCRC=false; + RevSpace=false; + StoreVolNumber=false; + } +}; + + +struct CryptHeader:BaseBlock +{ + bool UsePswCheck; + uint Lg2Count; // Log2 of PBKDF2 repetition count. + byte Salt[SIZE_SALT50]; + byte PswCheck[SIZE_PSWCHECK]; +}; + + +// SubBlockHeader and its successors were used in RAR 2.x format. +// RAR 4.x uses FileHeader with HEAD_SERVICE HeaderType for subblocks. +struct SubBlockHeader:BlockHeader +{ + ushort SubType; + byte Level; +}; + + +struct CommentHeader:BaseBlock +{ + ushort UnpSize; + byte UnpVer; + byte Method; + ushort CommCRC; +}; + + +struct ProtectHeader:BlockHeader +{ + byte Version; + ushort RecSectors; + uint TotalBlocks; + byte Mark[8]; +}; + + +struct AVHeader:BaseBlock +{ + byte UnpVer; + byte Method; + byte AVVer; + uint AVInfoCRC; +}; + + +struct SignHeader:BaseBlock +{ + uint CreationTime; + ushort ArcNameSize; + ushort UserNameSize; +}; + + +struct UnixOwnersHeader:SubBlockHeader +{ + ushort OwnerNameSize; + ushort GroupNameSize; +/* dummy */ + char OwnerName[256]; + char GroupName[256]; +}; + + +struct EAHeader:SubBlockHeader +{ + uint UnpSize; + byte UnpVer; + byte Method; + uint EACRC; +}; + + +struct StreamHeader:SubBlockHeader +{ + uint UnpSize; + byte UnpVer; + byte Method; + uint StreamCRC; + ushort StreamNameSize; + char StreamName[260]; +}; + + +struct MacFInfoHeader:SubBlockHeader +{ + uint fileType; + uint fileCreator; +}; + + +#endif diff --git a/reference/unrar/headers5.hpp b/reference/unrar/headers5.hpp new file mode 100644 index 00000000..9ea8d979 --- /dev/null +++ b/reference/unrar/headers5.hpp @@ -0,0 +1,100 @@ +#ifndef _RAR_HEADERS5_ +#define _RAR_HEADERS5_ + +#define SIZEOF_MARKHEAD5 8 // RAR 5.0 signature length. +#define SIZEOF_SHORTBLOCKHEAD5 7 // Smallest RAR 5.0 block size. + +// RAR 5.0 block flags common for all blocks. + +// Additional extra area is present in the end of block header. +#define HFL_EXTRA 0x0001 +// Additional data area is present in the end of block header. +#define HFL_DATA 0x0002 +// Unknown blocks with this flag must be skipped when updating an archive. +#define HFL_SKIPIFUNKNOWN 0x0004 +// Data area of this block is continuing from previous volume. +#define HFL_SPLITBEFORE 0x0008 +// Data area of this block is continuing in next volume. +#define HFL_SPLITAFTER 0x0010 +// Block depends on preceding file block. +#define HFL_CHILD 0x0020 +// Preserve a child block if host is modified. +#define HFL_INHERITED 0x0040 + +// RAR 5.0 main archive header specific flags. +#define MHFL_VOLUME 0x0001 // Volume. +#define MHFL_VOLNUMBER 0x0002 // Volume number field is present. True for all volumes except first. +#define MHFL_SOLID 0x0004 // Solid archive. +#define MHFL_PROTECT 0x0008 // Recovery record is present. +#define MHFL_LOCK 0x0010 // Locked archive. + +// RAR 5.0 file header specific flags. +#define FHFL_DIRECTORY 0x0001 // Directory. +#define FHFL_UTIME 0x0002 // Time field in Unix format is present. +#define FHFL_CRC32 0x0004 // CRC32 field is present. +#define FHFL_UNPUNKNOWN 0x0008 // Unknown unpacked size. + +// RAR 5.0 end of archive header specific flags. +#define EHFL_NEXTVOLUME 0x0001 // Not last volume. + +// RAR 5.0 archive encryption header specific flags. +#define CHFL_CRYPT_PSWCHECK 0x0001 // Password check data is present. + + +// RAR 5.0 file compression flags. +#define FCI_ALGO_BIT0 0x0001 // Version of compression algorithm. +#define FCI_ALGO_BIT1 0x0002 // 0 .. 63. +#define FCI_ALGO_BIT2 0x0004 +#define FCI_ALGO_BIT3 0x0008 +#define FCI_ALGO_BIT4 0x0010 +#define FCI_ALGO_BIT5 0x0020 +#define FCI_SOLID 0x0040 // Solid flag. +#define FCI_METHOD_BIT0 0x0080 // Compression method. +#define FCI_METHOD_BIT1 0x0100 // 0 .. 5 (6 and 7 are not used). +#define FCI_METHOD_BIT2 0x0200 +#define FCI_DICT_BIT0 0x0400 // Dictionary size. +#define FCI_DICT_BIT1 0x0800 // 128 KB .. 4 GB. +#define FCI_DICT_BIT2 0x1000 +#define FCI_DICT_BIT3 0x2000 + +// Main header extra field values. +#define MHEXTRA_LOCATOR 0x01 // Position of quick list and other blocks. + +// Flags for MHEXTRA_LOCATOR. +#define MHEXTRA_LOCATOR_QLIST 0x01 // Quick open offset is present. +#define MHEXTRA_LOCATOR_RR 0x02 // Recovery record offset is present. + +// File and service header extra field values. +#define FHEXTRA_CRYPT 0x01 // Encryption parameters. +#define FHEXTRA_HASH 0x02 // File hash. +#define FHEXTRA_HTIME 0x03 // High precision file time. +#define FHEXTRA_VERSION 0x04 // File version information. +#define FHEXTRA_REDIR 0x05 // File system redirection (links, etc.). +#define FHEXTRA_UOWNER 0x06 // Unix owner and group information. +#define FHEXTRA_SUBDATA 0x07 // Service header subdata array. + + +// Hash type values for FHEXTRA_HASH. +#define FHEXTRA_HASH_BLAKE2 0x00 + +// Flags for FHEXTRA_HTIME. +#define FHEXTRA_HTIME_UNIXTIME 0x01 // Use Unix time_t format. +#define FHEXTRA_HTIME_MTIME 0x02 // mtime is present. +#define FHEXTRA_HTIME_CTIME 0x04 // ctime is present. +#define FHEXTRA_HTIME_ATIME 0x08 // atime is present. +#define FHEXTRA_HTIME_UNIX_NS 0x10 // Unix format with nanosecond precision. + +// Flags for FHEXTRA_CRYPT. +#define FHEXTRA_CRYPT_PSWCHECK 0x01 // Store password check data. +#define FHEXTRA_CRYPT_HASHMAC 0x02 // Use MAC for unpacked data checksums. + +// Flags for FHEXTRA_REDIR. +#define FHEXTRA_REDIR_DIR 0x01 // Link target is directory. + +// Flags for FHEXTRA_UOWNER. +#define FHEXTRA_UOWNER_UNAME 0x01 // User name string is present. +#define FHEXTRA_UOWNER_GNAME 0x02 // Group name string is present. +#define FHEXTRA_UOWNER_NUMUID 0x04 // Numeric user ID is present. +#define FHEXTRA_UOWNER_NUMGID 0x08 // Numeric group ID is present. + +#endif diff --git a/reference/unrar/isnt.cpp b/reference/unrar/isnt.cpp new file mode 100644 index 00000000..6fadec04 --- /dev/null +++ b/reference/unrar/isnt.cpp @@ -0,0 +1,24 @@ +#include "rar.hpp" + +#ifdef _WIN_ALL +DWORD WinNT() +{ + static int dwPlatformId=-1; + static DWORD dwMajorVersion,dwMinorVersion; + if (dwPlatformId==-1) + { + OSVERSIONINFO WinVer; + WinVer.dwOSVersionInfoSize=sizeof(WinVer); + GetVersionEx(&WinVer); + dwPlatformId=WinVer.dwPlatformId; + dwMajorVersion=WinVer.dwMajorVersion; + dwMinorVersion=WinVer.dwMinorVersion; + } + DWORD Result=0; + if (dwPlatformId==VER_PLATFORM_WIN32_NT) + Result=dwMajorVersion*0x100+dwMinorVersion; + + + return Result; +} +#endif diff --git a/reference/unrar/isnt.hpp b/reference/unrar/isnt.hpp new file mode 100644 index 00000000..85790da4 --- /dev/null +++ b/reference/unrar/isnt.hpp @@ -0,0 +1,13 @@ +#ifndef _RAR_ISNT_ +#define _RAR_ISNT_ + +enum WINNT_VERSION { + WNT_NONE=0,WNT_NT351=0x0333,WNT_NT4=0x0400,WNT_W2000=0x0500, + WNT_WXP=0x0501,WNT_W2003=0x0502,WNT_VISTA=0x0600,WNT_W7=0x0601, + WNT_W8=0x0602,WNT_W81=0x0603,WNT_W10=0x0a00 +}; + +DWORD WinNT(); + + +#endif diff --git a/reference/unrar/license.txt b/reference/unrar/license.txt new file mode 100644 index 00000000..0811276a --- /dev/null +++ b/reference/unrar/license.txt @@ -0,0 +1,42 @@ + ****** ***** ****** UnRAR - free utility for RAR archives + ** ** ** ** ** ** ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ****** ******* ****** License for use and distribution of + ** ** ** ** ** ** ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + ** ** ** ** ** ** FREE portable version + ~~~~~~~~~~~~~~~~~~~~~ + + The source code of UnRAR utility is freeware. This means: + + 1. All copyrights to RAR and the utility UnRAR are exclusively + owned by the author - Alexander Roshal. + + 2. UnRAR source code may be used in any software to handle + RAR archives without limitations free of charge, but cannot be + used to develop RAR (WinRAR) compatible archiver and to + re-create RAR compression algorithm, which is proprietary. + Distribution of modified UnRAR source code in separate form + or as a part of other software is permitted, provided that + full text of this paragraph, starting from "UnRAR source code" + words, is included in license, or in documentation if license + is not available, and in source code comments of resulting package. + + 3. The UnRAR utility may be freely distributed. It is allowed + to distribute UnRAR inside of other software packages. + + 4. THE RAR ARCHIVER AND THE UnRAR UTILITY ARE DISTRIBUTED "AS IS". + NO WARRANTY OF ANY KIND IS EXPRESSED OR IMPLIED. YOU USE AT + YOUR OWN RISK. THE AUTHOR WILL NOT BE LIABLE FOR DATA LOSS, + DAMAGES, LOSS OF PROFITS OR ANY OTHER KIND OF LOSS WHILE USING + OR MISUSING THIS SOFTWARE. + + 5. Installing and using the UnRAR utility signifies acceptance of + these terms and conditions of the license. + + 6. If you don't agree with terms of the license you must remove + UnRAR files from your storage devices and cease to use the + utility. + + Thank you for your interest in RAR and UnRAR. + + + Alexander L. Roshal diff --git a/reference/unrar/list.cpp b/reference/unrar/list.cpp new file mode 100644 index 00000000..561122b4 --- /dev/null +++ b/reference/unrar/list.cpp @@ -0,0 +1,472 @@ +#include "rar.hpp" + +static void ListFileHeader(Archive &Arc,FileHeader &hd,bool &TitleShown,bool Verbose,bool Technical,bool Bare); +static void ListSymLink(Archive &Arc); +static void ListFileAttr(uint A,HOST_SYSTEM_TYPE HostType,wchar *AttrStr,size_t AttrSize); +static void ListOldSubHeader(Archive &Arc); +static void ListNewSubHeader(CommandData *Cmd,Archive &Arc); + +void ListArchive(CommandData *Cmd) +{ + int64 SumPackSize=0,SumUnpSize=0; + uint ArcCount=0,SumFileCount=0; + bool Technical=(Cmd->Command[1]=='T'); + bool ShowService=Technical && Cmd->Command[2]=='A'; + bool Bare=(Cmd->Command[1]=='B'); + bool Verbose=(Cmd->Command[0]=='V'); + + wchar ArcName[NM]; + while (Cmd->GetArcName(ArcName,ASIZE(ArcName))) + { + if (Cmd->ManualPassword) + Cmd->Password.Clean(); // Clean user entered password before processing next archive. + + Archive Arc(Cmd); +#ifdef _WIN_ALL + Arc.RemoveSequentialFlag(); +#endif + if (!Arc.WOpen(ArcName)) + continue; + bool FileMatched=true; + while (1) + { + int64 TotalPackSize=0,TotalUnpSize=0; + uint FileCount=0; + if (Arc.IsArchive(true)) + { + bool TitleShown=false; + if (!Bare) + { + Arc.ViewComment(); + mprintf(L"\n%s: %s",St(MListArchive),Arc.FileName); + mprintf(L"\n%s: ",St(MListDetails)); + uint SetCount=0; + const wchar *Fmt=Arc.Format==RARFMT14 ? L"RAR 1.4":(Arc.Format==RARFMT15 ? L"RAR 4":L"RAR 5"); + mprintf(L"%s%s", SetCount++ > 0 ? L", ":L"", Fmt); + if (Arc.Solid) + mprintf(L"%s%s", SetCount++ > 0 ? L", ":L"", St(MListSolid)); + if (Arc.SFXSize>0) + mprintf(L"%s%s", SetCount++ > 0 ? L", ":L"", St(MListSFX)); + if (Arc.Volume) + if (Arc.Format==RARFMT50) + { + // RAR 5.0 archives store the volume number in main header, + // so it is already available now. + if (SetCount++ > 0) + mprintf(L", "); + mprintf(St(MVolumeNumber),Arc.VolNumber+1); + } + else + mprintf(L"%s%s", SetCount++ > 0 ? L", ":L"", St(MListVolume)); + if (Arc.Protected) + mprintf(L"%s%s", SetCount++ > 0 ? L", ":L"", St(MListRR)); + if (Arc.Locked) + mprintf(L"%s%s", SetCount++ > 0 ? L", ":L"", St(MListLock)); + if (Arc.Encrypted) + mprintf(L"%s%s", SetCount++ > 0 ? L", ":L"", St(MListEncHead)); + mprintf(L"\n"); + } + + wchar VolNumText[50]; + *VolNumText=0; + while(Arc.ReadHeader()>0) + { + HEADER_TYPE HeaderType=Arc.GetHeaderType(); + if (HeaderType==HEAD_ENDARC) + { +#ifndef SFX_MODULE + // Only RAR 1.5 archives store the volume number in end record. + if (Arc.EndArcHead.StoreVolNumber && Arc.Format==RARFMT15) + swprintf(VolNumText,ASIZE(VolNumText),L"%.10ls %u",St(MListVolume),Arc.VolNumber+1); +#endif + if (Technical && ShowService) + { + mprintf(L"\n%12ls: %ls",St(MListService),L"EOF"); + if (*VolNumText!=0) + mprintf(L"\n%12ls: %ls",St(MListFlags),VolNumText); + mprintf(L"\n"); + } + break; + } + switch(HeaderType) + { + case HEAD_FILE: + FileMatched=Cmd->IsProcessFile(Arc.FileHead)!=0; + if (FileMatched) + { + ListFileHeader(Arc,Arc.FileHead,TitleShown,Verbose,Technical,Bare); + if (!Arc.FileHead.SplitBefore) + { + TotalUnpSize+=Arc.FileHead.UnpSize; + FileCount++; + } + TotalPackSize+=Arc.FileHead.PackSize; + } + break; + case HEAD_SERVICE: + if (FileMatched && !Bare) + { + if (Technical && ShowService) + ListFileHeader(Arc,Arc.SubHead,TitleShown,Verbose,true,false); + } + break; + } + Arc.SeekToNext(); + } + if (!Bare && !Technical) + if (TitleShown) + { + wchar UnpSizeText[20]; + itoa(TotalUnpSize,UnpSizeText,ASIZE(UnpSizeText)); + + wchar PackSizeText[20]; + itoa(TotalPackSize,PackSizeText,ASIZE(PackSizeText)); + + if (Verbose) + { + mprintf(L"\n----------- --------- -------- ----- ---------- ----- -------- ----"); + mprintf(L"\n%21ls %9ls %3d%% %-27ls %u",UnpSizeText, + PackSizeText,ToPercentUnlim(TotalPackSize,TotalUnpSize), + VolNumText,FileCount); + } + else + { + mprintf(L"\n----------- --------- ---------- ----- ----"); + mprintf(L"\n%21ls %-16ls %u",UnpSizeText,VolNumText,FileCount); + } + + SumFileCount+=FileCount; + SumUnpSize+=TotalUnpSize; + SumPackSize+=TotalPackSize; + mprintf(L"\n"); + } + else + mprintf(St(MListNoFiles)); + + ArcCount++; + +#ifndef NOVOLUME + if (Cmd->VolSize!=0 && (Arc.FileHead.SplitAfter || + Arc.GetHeaderType()==HEAD_ENDARC && Arc.EndArcHead.NextVolume) && + MergeArchive(Arc,NULL,false,Cmd->Command[0])) + Arc.Seek(0,SEEK_SET); + else +#endif + break; + } + else + { + if (Cmd->ArcNames.ItemsCount()<2 && !Bare) + mprintf(St(MNotRAR),Arc.FileName); + break; + } + } + } + + // Clean user entered password. Not really required, just for extra safety. + if (Cmd->ManualPassword) + Cmd->Password.Clean(); + + if (ArcCount>1 && !Bare && !Technical) + { + wchar UnpSizeText[20],PackSizeText[20]; + itoa(SumUnpSize,UnpSizeText,ASIZE(UnpSizeText)); + itoa(SumPackSize,PackSizeText,ASIZE(PackSizeText)); + + if (Verbose) + mprintf(L"%21ls %9ls %3d%% %28ls %u",UnpSizeText,PackSizeText, + ToPercentUnlim(SumPackSize,SumUnpSize),L"",SumFileCount); + else + mprintf(L"%21ls %18s %lu",UnpSizeText,L"",SumFileCount); + } +} + + +enum LISTCOL_TYPE { + LCOL_NAME,LCOL_ATTR,LCOL_SIZE,LCOL_PACKED,LCOL_RATIO,LCOL_CSUM,LCOL_ENCR +}; + + +void ListFileHeader(Archive &Arc,FileHeader &hd,bool &TitleShown,bool Verbose,bool Technical,bool Bare) +{ + wchar *Name=hd.FileName; + RARFORMAT Format=Arc.Format; + + if (Bare) + { + mprintf(L"%s\n",Name); + return; + } + + if (!TitleShown && !Technical) + { + if (Verbose) + { + mprintf(L"\n%ls",St(MListTitleV)); + mprintf(L"\n----------- --------- -------- ----- ---------- ----- -------- ----"); + } + else + { + mprintf(L"\n%ls",St(MListTitleL)); + mprintf(L"\n----------- --------- ---------- ----- ----"); + } + TitleShown=true; + } + + wchar UnpSizeText[30],PackSizeText[30]; + if (hd.UnpSize==INT64NDF) + wcscpy(UnpSizeText,L"?"); + else + itoa(hd.UnpSize,UnpSizeText,ASIZE(UnpSizeText)); + itoa(hd.PackSize,PackSizeText,ASIZE(PackSizeText)); + + wchar AttrStr[30]; + if (hd.HeaderType==HEAD_SERVICE) + swprintf(AttrStr,ASIZE(AttrStr),L"%cB",hd.Inherited ? 'I' : '.'); + else + ListFileAttr(hd.FileAttr,hd.HSType,AttrStr,ASIZE(AttrStr)); + + wchar RatioStr[10]; + + if (hd.SplitBefore && hd.SplitAfter) + wcscpy(RatioStr,L"<->"); + else + if (hd.SplitBefore) + wcscpy(RatioStr,L"<--"); + else + if (hd.SplitAfter) + wcscpy(RatioStr,L"-->"); + else + swprintf(RatioStr,ASIZE(RatioStr),L"%d%%",ToPercentUnlim(hd.PackSize,hd.UnpSize)); + + wchar DateStr[50]; + hd.mtime.GetText(DateStr,ASIZE(DateStr),Technical); + + if (Technical) + { + mprintf(L"\n%12s: %s",St(MListName),Name); + + bool FileBlock=hd.HeaderType==HEAD_FILE; + + if (!FileBlock && Arc.SubHead.CmpName(SUBHEAD_TYPE_STREAM)) + { + mprintf(L"\n%12ls: %ls",St(MListType),St(MListStream)); + wchar StreamName[NM]; + GetStreamNameNTFS(Arc,StreamName,ASIZE(StreamName)); + mprintf(L"\n%12ls: %ls",St(MListTarget),StreamName); + } + else + { + const wchar *Type=St(FileBlock ? (hd.Dir ? MListDir:MListFile):MListService); + + if (hd.RedirType!=FSREDIR_NONE) + switch(hd.RedirType) + { + case FSREDIR_UNIXSYMLINK: + Type=St(MListUSymlink); break; + case FSREDIR_WINSYMLINK: + Type=St(MListWSymlink); break; + case FSREDIR_JUNCTION: + Type=St(MListJunction); break; + case FSREDIR_HARDLINK: + Type=St(MListHardlink); break; + case FSREDIR_FILECOPY: + Type=St(MListCopy); break; + } + mprintf(L"\n%12ls: %ls",St(MListType),Type); + if (hd.RedirType!=FSREDIR_NONE) + if (Format==RARFMT15) + { + char LinkTargetA[NM]; + if (Arc.FileHead.Encrypted) + { + // Link data are encrypted. We would need to ask for password + // and initialize decryption routine to display the link target. + strncpyz(LinkTargetA,"*<-?->",ASIZE(LinkTargetA)); + } + else + { + int DataSize=(int)Min(hd.PackSize,ASIZE(LinkTargetA)-1); + Arc.Read(LinkTargetA,DataSize); + LinkTargetA[DataSize > 0 ? DataSize : 0] = 0; + } + wchar LinkTarget[NM]; + CharToWide(LinkTargetA,LinkTarget,ASIZE(LinkTarget)); + mprintf(L"\n%12ls: %ls",St(MListTarget),LinkTarget); + } + else + mprintf(L"\n%12ls: %ls",St(MListTarget),hd.RedirName); + } + if (!hd.Dir) + { + mprintf(L"\n%12ls: %ls",St(MListSize),UnpSizeText); + mprintf(L"\n%12ls: %ls",St(MListPacked),PackSizeText); + mprintf(L"\n%12ls: %ls",St(MListRatio),RatioStr); + } + if (hd.mtime.IsSet()) + mprintf(L"\n%12ls: %ls",St(MListMtime),DateStr); + if (hd.ctime.IsSet()) + { + hd.ctime.GetText(DateStr,ASIZE(DateStr),true); + mprintf(L"\n%12ls: %ls",St(MListCtime),DateStr); + } + if (hd.atime.IsSet()) + { + hd.atime.GetText(DateStr,ASIZE(DateStr),true); + mprintf(L"\n%12ls: %ls",St(MListAtime),DateStr); + } + mprintf(L"\n%12ls: %ls",St(MListAttr),AttrStr); + if (hd.FileHash.Type==HASH_CRC32) + mprintf(L"\n%12ls: %8.8X", + hd.UseHashKey ? L"CRC32 MAC":hd.SplitAfter ? L"Pack-CRC32":L"CRC32", + hd.FileHash.CRC32); + if (hd.FileHash.Type==HASH_BLAKE2) + { + wchar BlakeStr[BLAKE2_DIGEST_SIZE*2+1]; + BinToHex(hd.FileHash.Digest,BLAKE2_DIGEST_SIZE,NULL,BlakeStr,ASIZE(BlakeStr)); + mprintf(L"\n%12ls: %ls", + hd.UseHashKey ? L"BLAKE2 MAC":hd.SplitAfter ? L"Pack-BLAKE2":L"BLAKE2", + BlakeStr); + } + + const wchar *HostOS=L""; + if (Format==RARFMT50 && hd.HSType!=HSYS_UNKNOWN) + HostOS=hd.HSType==HSYS_WINDOWS ? L"Windows":L"Unix"; + if (Format==RARFMT15) + { + static const wchar *RarOS[]={ + L"DOS",L"OS/2",L"Windows",L"Unix",L"Mac OS",L"BeOS",L"WinCE",L"",L"",L"" + }; + if (hd.HostOS=0x100000 ? hd.WinSize/0x100000:hd.WinSize/0x400, + hd.WinSize>=0x100000 ? L"M":L"K"); + + if (hd.Solid || hd.Encrypted) + { + mprintf(L"\n%12ls: ",St(MListFlags)); + if (hd.Solid) + mprintf(L"%ls ",St(MListSolid)); + if (hd.Encrypted) + mprintf(L"%ls ",St(MListEnc)); + } + + if (hd.Version) + { + uint Version=ParseVersionFileName(Name,false); + if (Version!=0) + mprintf(L"\n%12ls: %u",St(MListFileVer),Version); + } + + if (hd.UnixOwnerSet) + { + mprintf(L"\n%12ls: ",L"Unix owner"); + if (*hd.UnixOwnerName!=0) + mprintf(L"%ls:",GetWide(hd.UnixOwnerName)); + if (*hd.UnixGroupName!=0) + mprintf(L"%ls",GetWide(hd.UnixGroupName)); + if ((*hd.UnixOwnerName!=0 || *hd.UnixGroupName!=0) && (hd.UnixOwnerNumeric || hd.UnixGroupNumeric)) + mprintf(L" "); + if (hd.UnixOwnerNumeric) + mprintf(L"#%d:",hd.UnixOwnerID); + if (hd.UnixGroupNumeric) + mprintf(L"#%d:",hd.UnixGroupID); + } + + mprintf(L"\n"); + return; + } + + mprintf(L"\n%c%10ls %9ls ",hd.Encrypted ? '*' : ' ',AttrStr,UnpSizeText); + + if (Verbose) + mprintf(L"%9ls %4ls ",PackSizeText,RatioStr); + + mprintf(L" %ls ",DateStr); + + if (Verbose) + { + if (hd.FileHash.Type==HASH_CRC32) + mprintf(L"%8.8X ",hd.FileHash.CRC32); + else + if (hd.FileHash.Type==HASH_BLAKE2) + { + byte *S=hd.FileHash.Digest; + mprintf(L"%02x%02x..%02x ",S[0],S[1],S[31]); + } + else + mprintf(L"???????? "); + } + mprintf(L"%ls",Name); +} + +/* +void ListSymLink(Archive &Arc) +{ + if (Arc.FileHead.HSType==HSYS_UNIX && (Arc.FileHead.FileAttr & 0xF000)==0xA000) + if (Arc.FileHead.Encrypted) + { + // Link data are encrypted. We would need to ask for password + // and initialize decryption routine to display the link target. + mprintf(L"\n%22ls %ls",L"-->",L"*<-?->"); + } + else + { + char FileName[NM]; + uint DataSize=(uint)Min(Arc.FileHead.PackSize,sizeof(FileName)-1); + Arc.Read(FileName,DataSize); + FileName[DataSize]=0; + mprintf(L"\n%22ls %ls",L"-->",GetWide(FileName)); + } +} +*/ + +void ListFileAttr(uint A,HOST_SYSTEM_TYPE HostType,wchar *AttrStr,size_t AttrSize) +{ + switch(HostType) + { + case HSYS_WINDOWS: + swprintf(AttrStr,AttrSize,L"%c%c%c%c%c%c%c", + (A & 0x2000)!=0 ? 'I' : '.', // Not content indexed. + (A & 0x0800)!=0 ? 'C' : '.', // Compressed. + (A & 0x0020)!=0 ? 'A' : '.', // Archive. + (A & 0x0010)!=0 ? 'D' : '.', // Directory. + (A & 0x0004)!=0 ? 'S' : '.', // System. + (A & 0x0002)!=0 ? 'H' : '.', // Hidden. + (A & 0x0001)!=0 ? 'R' : '.'); // Read-only. + break; + case HSYS_UNIX: + switch (A & 0xF000) + { + case 0x4000: + AttrStr[0]='d'; + break; + case 0xA000: + AttrStr[0]='l'; + break; + default: + AttrStr[0]='-'; + break; + } + swprintf(AttrStr+1,AttrSize-1,L"%c%c%c%c%c%c%c%c%c", + (A & 0x0100) ? 'r' : '-', + (A & 0x0080) ? 'w' : '-', + (A & 0x0040) ? ((A & 0x0800)!=0 ? 's':'x'):((A & 0x0800)!=0 ? 'S':'-'), + (A & 0x0020) ? 'r' : '-', + (A & 0x0010) ? 'w' : '-', + (A & 0x0008) ? ((A & 0x0400)!=0 ? 's':'x'):((A & 0x0400)!=0 ? 'S':'-'), + (A & 0x0004) ? 'r' : '-', + (A & 0x0002) ? 'w' : '-', + (A & 0x0001) ? ((A & 0x200)!=0 ? 't' : 'x') : '-'); + break; + case HSYS_UNKNOWN: + wcscpy(AttrStr,L"?"); + break; + } +} diff --git a/reference/unrar/list.hpp b/reference/unrar/list.hpp new file mode 100644 index 00000000..7721ae52 --- /dev/null +++ b/reference/unrar/list.hpp @@ -0,0 +1,6 @@ +#ifndef _RAR_LIST_ +#define _RAR_LIST_ + +void ListArchive(CommandData *Cmd); + +#endif diff --git a/reference/unrar/loclang.hpp b/reference/unrar/loclang.hpp new file mode 100644 index 00000000..427128b4 --- /dev/null +++ b/reference/unrar/loclang.hpp @@ -0,0 +1,381 @@ +#define MYesNo L"_Yes_No" +#define MYesNoAll L"_Yes_No_All" +#define MYesNoAllQ L"_Yes_No_All_nEver_Quit" +#define MYesNoAllRenQ L"_Yes_No_All_nEver_Rename_Quit" +#define MContinueQuit L"_Continue_Quit" +#define MRetryAbort L"_Retry_Abort" +#define MCopyright L"\nRAR %s Copyright (c) 1993-%d Alexander Roshal %d %s %d" +#define MRegTo L"\nRegistered to %s\n" +#define MShare L"\nTrial version Type 'rar -?' for help\n" +#define MRegKeyWarning L"\nAvailable license key is valid only for %s\n" +#define MUCopyright L"\nUNRAR %s freeware Copyright (c) 1993-%d Alexander Roshal\n" +#define MBeta L"beta" +#define Mx86 L"x86" +#define Mx64 L"x64" +#define MMonthJan L"Jan" +#define MMonthFeb L"Feb" +#define MMonthMar L"Mar" +#define MMonthApr L"Apr" +#define MMonthMay L"May" +#define MMonthJun L"Jun" +#define MMonthJul L"Jul" +#define MMonthAug L"Aug" +#define MMonthSep L"Sep" +#define MMonthOct L"Oct" +#define MMonthNov L"Nov" +#define MMonthDec L"Dec" +#define MRARTitle1 L"\nUsage: rar - - " +#define MUNRARTitle1 L"\nUsage: unrar - - " +#define MRARTitle2 L"\n <@listfiles...> " +#define MCHelpCmd L"\n\n" +#define MCHelpCmdA L"\n a Add files to archive" +#define MCHelpCmdC L"\n c Add archive comment" +#define MCHelpCmdCH L"\n ch Change archive parameters" +#define MCHelpCmdCW L"\n cw Write archive comment to file" +#define MCHelpCmdD L"\n d Delete files from archive" +#define MCHelpCmdE L"\n e Extract files without archived paths" +#define MCHelpCmdF L"\n f Freshen files in archive" +#define MCHelpCmdI L"\n i[par]= Find string in archives" +#define MCHelpCmdK L"\n k Lock archive" +#define MCHelpCmdL L"\n l[t[a],b] List archive contents [technical[all], bare]" +#define MCHelpCmdM L"\n m[f] Move to archive [files only]" +#define MCHelpCmdP L"\n p Print file to stdout" +#define MCHelpCmdR L"\n r Repair archive" +#define MCHelpCmdRC L"\n rc Reconstruct missing volumes" +#define MCHelpCmdRN L"\n rn Rename archived files" +#define MCHelpCmdRR L"\n rr[N] Add data recovery record" +#define MCHelpCmdRV L"\n rv[N] Create recovery volumes" +#define MCHelpCmdS L"\n s[name|-] Convert archive to or from SFX" +#define MCHelpCmdT L"\n t Test archive files" +#define MCHelpCmdU L"\n u Update files in archive" +#define MCHelpCmdV L"\n v[t[a],b] Verbosely list archive contents [technical[all],bare]" +#define MCHelpCmdX L"\n x Extract files with full path" +#define MCHelpSw L"\n\n" +#define MCHelpSwm L"\n - Stop switches scanning" +#define MCHelpSwAT L"\n @[+] Disable [enable] file lists" +#define MCHelpSwAC L"\n ac Clear Archive attribute after compression or extraction" +#define MCHelpSwAD L"\n ad Append archive name to destination path" +#define MCHelpSwAG L"\n ag[format] Generate archive name using the current date" +#define MCHelpSwAI L"\n ai Ignore file attributes" +#define MCHelpSwAO L"\n ao Add files with Archive attribute set" +#define MCHelpSwAP L"\n ap Set path inside archive" +#define MCHelpSwAS L"\n as Synchronize archive contents" +#define MCHelpSwCm L"\n c- Disable comments show" +#define MCHelpSwCFGm L"\n cfg- Disable read configuration" +#define MCHelpSwCL L"\n cl Convert names to lower case" +#define MCHelpSwCU L"\n cu Convert names to upper case" +#define MCHelpSwDF L"\n df Delete files after archiving" +#define MCHelpSwDH L"\n dh Open shared files" +#define MCHelpSwDR L"\n dr Delete files to Recycle Bin" +#define MCHelpSwDS L"\n ds Disable name sort for solid archive" +#define MCHelpSwDW L"\n dw Wipe files after archiving" +#define MCHelpSwEa L"\n e[+] Set file exclude and include attributes" +#define MCHelpSwED L"\n ed Do not add empty directories" +#define MCHelpSwEN L"\n en Do not put 'end of archive' block" +#define MCHelpSwEP L"\n ep Exclude paths from names" +#define MCHelpSwEP1 L"\n ep1 Exclude base directory from names" +#define MCHelpSwEP2 L"\n ep2 Expand paths to full" +#define MCHelpSwEP3 L"\n ep3 Expand paths to full including the drive letter" +#define MCHelpSwF L"\n f Freshen files" +#define MCHelpSwHP L"\n hp[password] Encrypt both file data and headers" +#define MCHelpSwHT L"\n ht[b|c] Select hash type [BLAKE2,CRC32] for file checksum" +#define MCHelpSwIDP L"\n id[c,d,p,q] Disable messages" +#define MCHelpSwIEML L"\n ieml[addr] Send archive by email" +#define MCHelpSwIERR L"\n ierr Send all messages to stderr" +#define MCHelpSwILOG L"\n ilog[name] Log errors to file (registered versions only)" +#define MCHelpSwINUL L"\n inul Disable all messages" +#define MCHelpSwIOFF L"\n ioff Turn PC off after completing an operation" +#define MCHelpSwISND L"\n isnd Enable sound" +#define MCHelpSwIVER L"\n iver Display the version number" +#define MCHelpSwK L"\n k Lock archive" +#define MCHelpSwKB L"\n kb Keep broken extracted files" +#define MCHelpSwLog L"\n log[f][=name] Write names to log file" +#define MCHelpSwMn L"\n m<0..5> Set compression level (0-store...3-default...5-maximal)" +#define MCHelpSwMA L"\n ma[4|5] Specify a version of archiving format" +#define MCHelpSwMC L"\n mc Set advanced compression parameters" +#define MCHelpSwMD L"\n md[k,m,g] Dictionary size in KB, MB or GB" +#define MCHelpSwMS L"\n ms[ext;ext] Specify file types to store" +#define MCHelpSwMT L"\n mt Set the number of threads" +#define MCHelpSwN L"\n n Additionally filter included files" +#define MCHelpSwNa L"\n n@ Read additional filter masks from stdin" +#define MCHelpSwNal L"\n n@ Read additional filter masks from list file" +#define MCHelpSwO L"\n o[+|-] Set the overwrite mode" +#define MCHelpSwOC L"\n oc Set NTFS Compressed attribute" +#define MCHelpSwOH L"\n oh Save hard links as the link instead of the file" +#define MCHelpSwOI L"\n oi[0-4][:min] Save identical files as references" +#define MCHelpSwOL L"\n ol[a] Process symbolic links as the link [absolute paths]" +#define MCHelpSwONI L"\n oni Allow potentially incompatible names" +#define MCHelpSwOR L"\n or Rename files automatically" +#define MCHelpSwOS L"\n os Save NTFS streams" +#define MCHelpSwOW L"\n ow Save or restore file owner and group" +#define MCHelpSwP L"\n p[password] Set password" +#define MCHelpSwPm L"\n p- Do not query password" +#define MCHelpSwQO L"\n qo[-|+] Add quick open information [none|force]" +#define MCHelpSwR L"\n r Recurse subdirectories" +#define MCHelpSwRm L"\n r- Disable recursion" +#define MCHelpSwR0 L"\n r0 Recurse subdirectories for wildcard names only" +#define MCHelpSwRI L"\n ri

[:] Set priority (0-default,1-min..15-max) and sleep time in ms" +#define MCHelpSwRR L"\n rr[N] Add data recovery record" +#define MCHelpSwRV L"\n rv[N] Create recovery volumes" +#define MCHelpSwS L"\n s[,v[-],e] Create solid archive" +#define MCHelpSwSm L"\n s- Disable solid archiving" +#define MCHelpSwSC L"\n sc[obj] Specify the character set" +#define MCHelpSwSFX L"\n sfx[name] Create SFX archive" +#define MCHelpSwSI L"\n si[name] Read data from standard input (stdin)" +#define MCHelpSwSL L"\n sl Process files with size less than specified" +#define MCHelpSwSM L"\n sm Process files with size more than specified" +#define MCHelpSwT L"\n t Test files after archiving" +#define MCHelpSwTK L"\n tk Keep original archive time" +#define MCHelpSwTL L"\n tl Set archive time to latest file" +#define MCHelpSwTN L"\n tn

+ /// Provides a resource pool that enables reusing instances of type . + /// + /// + /// + /// Renting and returning buffers with an can increase performance + /// in situations where arrays are created and destroyed frequently, resulting in significant + /// memory pressure on the garbage collector. + /// + /// + /// This class is thread-safe. All members may be used by multiple threads concurrently. + /// + /// + internal abstract class ArrayPool + { + /// The lazily-initialized shared pool instance. + private static ArrayPool s_sharedInstance = null; + + /// + /// Retrieves a shared instance. + /// + /// + /// The shared pool provides a default implementation of + /// that's intended for general applicability. It maintains arrays of multiple sizes, and + /// may hand back a larger array than was actually requested, but will never hand back a smaller + /// array than was requested. Renting a buffer from it with will result in an + /// existing buffer being taken from the pool if an appropriate buffer is available or in a new + /// buffer being allocated if one is not available. + /// + public static ArrayPool Shared + { + [MethodImpl(MethodImplOptions.AggressiveInlining)] + get { return Volatile.Read(ref s_sharedInstance) ?? EnsureSharedCreated(); } + } + + /// Ensures that has been initialized to a pool and returns it. + [MethodImpl(MethodImplOptions.NoInlining)] + private static ArrayPool EnsureSharedCreated() + { + Interlocked.CompareExchange(ref s_sharedInstance, Create(), null); + return s_sharedInstance; + } + + /// + /// Creates a new instance using default configuration options. + /// + /// A new instance. + public static ArrayPool Create() + { + return new DefaultArrayPool(); + } + + /// + /// Creates a new instance using custom configuration options. + /// + /// The maximum length of array instances that may be stored in the pool. + /// + /// The maximum number of array instances that may be stored in each bucket in the pool. The pool + /// groups arrays of similar lengths into buckets for faster access. + /// + /// A new instance with the specified configuration options. + /// + /// The created pool will group arrays into buckets, with no more than + /// in each bucket and with those arrays not exceeding in length. + /// + public static ArrayPool Create(int maxArrayLength, int maxArraysPerBucket) + { + return new DefaultArrayPool(maxArrayLength, maxArraysPerBucket); + } + + /// + /// Retrieves a buffer that is at least the requested length. + /// + /// The minimum length of the array needed. + /// + /// An that is at least in length. + /// + /// + /// This buffer is loaned to the caller and should be returned to the same pool via + /// so that it may be reused in subsequent usage of . + /// It is not a fatal error to not return a rented buffer, but failure to do so may lead to + /// decreased application performance, as the pool may need to create a new buffer to replace + /// the one lost. + /// + public abstract T[] Rent(int minimumLength); + + /// + /// Returns to the pool an array that was previously obtained via on the same + /// instance. + /// + /// + /// The buffer previously obtained from to return to the pool. + /// + /// + /// If true and if the pool will store the buffer to enable subsequent reuse, + /// will clear of its contents so that a subsequent consumer via + /// will not see the previous consumer's content. If false or if the pool will release the buffer, + /// the array's contents are left unchanged. + /// + /// + /// Once a buffer has been returned to the pool, the caller gives up all ownership of the buffer + /// and must not use it. The reference returned from a given call to must only be + /// returned via once. The default + /// may hold onto the returned buffer in order to rent it again, or it may release the returned buffer + /// if it's determined that the pool already has enough buffers stored. + /// + public abstract void Return(T[] array, bool clearArray = false); + } +} +#endif \ No newline at end of file diff --git a/src/SharpCompress/Buffers/DefaultArrayPool.cs b/src/SharpCompress/Buffers/DefaultArrayPool.cs new file mode 100644 index 00000000..43cd2c37 --- /dev/null +++ b/src/SharpCompress/Buffers/DefaultArrayPool.cs @@ -0,0 +1,144 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. +#if NETCORE +using System; + +namespace SharpCompress.Buffers +{ + internal sealed partial class DefaultArrayPool : ArrayPool + { + /// The default maximum length of each array in the pool (2^20). + private const int DefaultMaxArrayLength = 1024 * 1024; + /// The default maximum number of arrays per bucket that are available for rent. + private const int DefaultMaxNumberOfArraysPerBucket = 50; + /// Lazily-allocated empty array used when arrays of length 0 are requested. + private static T[] s_emptyArray; // we support contracts earlier than those with Array.Empty() + + private readonly Bucket[] _buckets; + + internal DefaultArrayPool() : this(DefaultMaxArrayLength, DefaultMaxNumberOfArraysPerBucket) + { + } + + internal DefaultArrayPool(int maxArrayLength, int maxArraysPerBucket) + { + if (maxArrayLength <= 0) + { + throw new ArgumentOutOfRangeException(nameof(maxArrayLength)); + } + if (maxArraysPerBucket <= 0) + { + throw new ArgumentOutOfRangeException(nameof(maxArraysPerBucket)); + } + + // Our bucketing algorithm has a min length of 2^4 and a max length of 2^30. + // Constrain the actual max used to those values. + const int MinimumArrayLength = 0x10, MaximumArrayLength = 0x40000000; + if (maxArrayLength > MaximumArrayLength) + { + maxArrayLength = MaximumArrayLength; + } + else if (maxArrayLength < MinimumArrayLength) + { + maxArrayLength = MinimumArrayLength; + } + + // Create the buckets. + int poolId = Id; + int maxBuckets = Utilities.SelectBucketIndex(maxArrayLength); + var buckets = new Bucket[maxBuckets + 1]; + for (int i = 0; i < buckets.Length; i++) + { + buckets[i] = new Bucket(Utilities.GetMaxSizeForBucket(i), maxArraysPerBucket, poolId); + } + _buckets = buckets; + } + + /// Gets an ID for the pool to use with events. + private int Id => GetHashCode(); + + public override T[] Rent(int minimumLength) + { + // Arrays can't be smaller than zero. We allow requesting zero-length arrays (even though + // pooling such an array isn't valuable) as it's a valid length array, and we want the pool + // to be usable in general instead of using `new`, even for computed lengths. + if (minimumLength < 0) + { + throw new ArgumentOutOfRangeException(nameof(minimumLength)); + } + else if (minimumLength == 0) + { + // No need for events with the empty array. Our pool is effectively infinite + // and we'll never allocate for rents and never store for returns. + return s_emptyArray ?? (s_emptyArray = new T[0]); + } + + T[] buffer = null; + + int index = Utilities.SelectBucketIndex(minimumLength); + if (index < _buckets.Length) + { + // Search for an array starting at the 'index' bucket. If the bucket is empty, bump up to the + // next higher bucket and try that one, but only try at most a few buckets. + const int MaxBucketsToTry = 2; + int i = index; + do + { + // Attempt to rent from the bucket. If we get a buffer from it, return it. + buffer = _buckets[i].Rent(); + if (buffer != null) + { + return buffer; + } + } + while (++i < _buckets.Length && i != index + MaxBucketsToTry); + + // The pool was exhausted for this buffer size. Allocate a new buffer with a size corresponding + // to the appropriate bucket. + buffer = new T[_buckets[index]._bufferLength]; + } + else + { + // The request was for a size too large for the pool. Allocate an array of exactly the requested length. + // When it's returned to the pool, we'll simply throw it away. + buffer = new T[minimumLength]; + } + + return buffer; + } + + public override void Return(T[] array, bool clearArray = false) + { + if (array == null) + { + throw new ArgumentNullException(nameof(array)); + } + else if (array.Length == 0) + { + // Ignore empty arrays. When a zero-length array is rented, we return a singleton + // rather than actually taking a buffer out of the lowest bucket. + return; + } + + // Determine with what bucket this array length is associated + int bucket = Utilities.SelectBucketIndex(array.Length); + + // If we can tell that the buffer was allocated, drop it. Otherwise, check if we have space in the pool + if (bucket < _buckets.Length) + { + // Clear the array if the user requests + if (clearArray) + { + Array.Clear(array, 0, array.Length); + } + + // Return the buffer to its bucket. In the future, we might consider having Return return false + // instead of dropping a bucket, in which case we could try to return to a lower-sized bucket, + // just as how in Rent we allow renting from a higher-sized bucket. + _buckets[bucket].Return(array); + } + } + } +} +#endif \ No newline at end of file diff --git a/src/SharpCompress/Buffers/DefaultArrayPoolBucket.cs b/src/SharpCompress/Buffers/DefaultArrayPoolBucket.cs new file mode 100644 index 00000000..30124889 --- /dev/null +++ b/src/SharpCompress/Buffers/DefaultArrayPoolBucket.cs @@ -0,0 +1,111 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +#if NETCORE +using System; +using System.Diagnostics; +using System.Threading; + +namespace SharpCompress.Buffers +{ + internal sealed partial class DefaultArrayPool : ArrayPool + { + /// Provides a thread-safe bucket containing buffers that can be Rent'd and Return'd. + private sealed class Bucket + { + internal readonly int _bufferLength; + private readonly T[][] _buffers; + private readonly int _poolId; + + private SpinLock _lock; // do not make this readonly; it's a mutable struct + private int _index; + + /// + /// Creates the pool with numberOfBuffers arrays where each buffer is of bufferLength length. + /// + internal Bucket(int bufferLength, int numberOfBuffers, int poolId) + { + _lock = new SpinLock(Debugger.IsAttached); // only enable thread tracking if debugger is attached; it adds non-trivial overheads to Enter/Exit + _buffers = new T[numberOfBuffers][]; + _bufferLength = bufferLength; + _poolId = poolId; + } + + /// Gets an ID for the bucket to use with events. + internal int Id => GetHashCode(); + + /// Takes an array from the bucket. If the bucket is empty, returns null. + internal T[] Rent() + { + T[][] buffers = _buffers; + T[] buffer = null; + + // While holding the lock, grab whatever is at the next available index and + // update the index. We do as little work as possible while holding the spin + // lock to minimize contention with other threads. The try/finally is + // necessary to properly handle thread aborts on platforms which have them. + bool lockTaken = false, allocateBuffer = false; + try + { + _lock.Enter(ref lockTaken); + + if (_index < buffers.Length) + { + buffer = buffers[_index]; + buffers[_index++] = null; + allocateBuffer = buffer == null; + } + } + finally + { + if (lockTaken) _lock.Exit(false); + } + + // While we were holding the lock, we grabbed whatever was at the next available index, if + // there was one. If we tried and if we got back null, that means we hadn't yet allocated + // for that slot, in which case we should do so now. + if (allocateBuffer) + { + buffer = new T[_bufferLength]; + } + + return buffer; + } + + /// + /// Attempts to return the buffer to the bucket. If successful, the buffer will be stored + /// in the bucket and true will be returned; otherwise, the buffer won't be stored, and false + /// will be returned. + /// + internal void Return(T[] array) + { + // Check to see if the buffer is the correct size for this bucket + if (array.Length != _bufferLength) + { + throw new ArgumentException("Buffer not from pool", nameof(array)); + } + + // While holding the spin lock, if there's room available in the bucket, + // put the buffer into the next available slot. Otherwise, we just drop it. + // The try/finally is necessary to properly handle thread aborts on platforms + // which have them. + bool lockTaken = false; + try + { + _lock.Enter(ref lockTaken); + + if (_index != 0) + { + _buffers[--_index] = array; + } + } + finally + { + if (lockTaken) _lock.Exit(false); + } + } + } + } +} +#endif \ No newline at end of file diff --git a/src/SharpCompress/Buffers/Utilities.cs b/src/SharpCompress/Buffers/Utilities.cs new file mode 100644 index 00000000..f4100e37 --- /dev/null +++ b/src/SharpCompress/Buffers/Utilities.cs @@ -0,0 +1,38 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. +#if NETCORE +using System.Diagnostics; +using System.Runtime.CompilerServices; + +namespace SharpCompress.Buffers +{ + internal static class Utilities + { + [MethodImpl(MethodImplOptions.AggressiveInlining)] + internal static int SelectBucketIndex(int bufferSize) + { + Debug.Assert(bufferSize > 0); + + uint bitsRemaining = ((uint)bufferSize - 1) >> 4; + + int poolIndex = 0; + if (bitsRemaining > 0xFFFF) { bitsRemaining >>= 16; poolIndex = 16; } + if (bitsRemaining > 0xFF) { bitsRemaining >>= 8; poolIndex += 8; } + if (bitsRemaining > 0xF) { bitsRemaining >>= 4; poolIndex += 4; } + if (bitsRemaining > 0x3) { bitsRemaining >>= 2; poolIndex += 2; } + if (bitsRemaining > 0x1) { bitsRemaining >>= 1; poolIndex += 1; } + + return poolIndex + (int)bitsRemaining; + } + + [MethodImpl(MethodImplOptions.AggressiveInlining)] + internal static int GetMaxSizeForBucket(int binIndex) + { + int maxSize = 16 << binIndex; + Debug.Assert(maxSize >= 0); + return maxSize; + } + } +} +#endif \ No newline at end of file diff --git a/src/SharpCompress/Common/ArchiveEncoding.cs b/src/SharpCompress/Common/ArchiveEncoding.cs index e37da084..c18b65c9 100644 --- a/src/SharpCompress/Common/ArchiveEncoding.cs +++ b/src/SharpCompress/Common/ArchiveEncoding.cs @@ -32,11 +32,33 @@ namespace SharpCompress.Common Password = Encoding.UTF8; } +#if NETSTANDARD1_3 || NETSTANDARD2_0 + static ArchiveEncoding() + { + Encoding.RegisterProvider(CodePagesEncodingProvider.Instance); + } +#endif + public string Decode(byte[] bytes) { return Decode(bytes, 0, bytes.Length); } + public string Decode437(byte[] bytes) + { +#if NETSTANDARD1_0 + return Decode(bytes, 0, bytes.Length); +#else + //allow forced and custom to override this. + if (CustomDecoder != null || Forced != null) + { + return Decode(bytes, 0, bytes.Length); + } + var extendedAsciiEncoding = Encoding.GetEncoding(437); + return extendedAsciiEncoding.GetString(bytes, 0, bytes.Length); +#endif + } + public string Decode(byte[] bytes, int start, int length) { return GetDecoder().Invoke(bytes, start, length); @@ -54,7 +76,7 @@ namespace SharpCompress.Common public Func GetDecoder() { - return CustomDecoder ?? ((bytes, index, count) => (Default ?? Encoding.UTF8).GetString(bytes, index, count)); + return CustomDecoder ?? ((bytes, index, count) => GetEncoding().GetString(bytes, index, count)); } } } \ No newline at end of file diff --git a/src/SharpCompress/Common/CompressionType.cs b/src/SharpCompress/Common/CompressionType.cs index e2774385..23ed354f 100644 --- a/src/SharpCompress/Common/CompressionType.cs +++ b/src/SharpCompress/Common/CompressionType.cs @@ -13,6 +13,7 @@ BCJ2, LZip, Xz, - Unknown + Unknown, + Deflate64 } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Entry.cs b/src/SharpCompress/Common/Entry.cs index 5f07af2f..7855b217 100644 --- a/src/SharpCompress/Common/Entry.cs +++ b/src/SharpCompress/Common/Entry.cs @@ -63,7 +63,13 @@ namespace SharpCompress.Common /// /// Entry is split among multiple volumes /// - public abstract bool IsSplit { get; } + public abstract bool IsSplitAfter { get; } + + /// + public override string ToString() + { + return Key; + } internal abstract IEnumerable Parts { get; } internal bool IsSolid { get; set; } diff --git a/src/SharpCompress/Common/EntryStream.cs b/src/SharpCompress/Common/EntryStream.cs index 0120b782..8d40cabd 100644 --- a/src/SharpCompress/Common/EntryStream.cs +++ b/src/SharpCompress/Common/EntryStream.cs @@ -6,15 +6,15 @@ namespace SharpCompress.Common { public class EntryStream : Stream { - public IReader Reader { get; } - private readonly Stream stream; - private bool completed; - private bool isDisposed; + private readonly IReader _reader; + private readonly Stream _stream; + private bool _completed; + private bool _isDisposed; internal EntryStream(IReader reader, Stream stream) { - Reader = reader; - this.stream = stream; + _reader = reader; + _stream = stream; } /// @@ -22,26 +22,23 @@ namespace SharpCompress.Common /// public void SkipEntry() { - var buffer = new byte[4096]; - while (Read(buffer, 0, buffer.Length) > 0) - { - } - completed = true; + this.Skip(); + _completed = true; } protected override void Dispose(bool disposing) { - if (!(completed || Reader.Cancelled)) + if (!(_completed || _reader.Cancelled)) { SkipEntry(); } - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; + _isDisposed = true; base.Dispose(disposing); - stream.Dispose(); + _stream.Dispose(); } public override bool CanRead => true; @@ -55,20 +52,30 @@ namespace SharpCompress.Common throw new NotSupportedException(); } - public override long Length => throw new NotSupportedException(); + public override long Length => _stream.Length; public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); } public override int Read(byte[] buffer, int offset, int count) { - int read = stream.Read(buffer, offset, count); + int read = _stream.Read(buffer, offset, count); if (read <= 0) { - completed = true; + _completed = true; } return read; } + public override int ReadByte() + { + int value = _stream.ReadByte(); + if (value == -1) + { + _completed = true; + } + return value; + } + public override long Seek(long offset, SeekOrigin origin) { throw new NotSupportedException(); diff --git a/src/SharpCompress/Common/GZip/GZipEntry.cs b/src/SharpCompress/Common/GZip/GZipEntry.cs index 6c80ddff..0b43cd45 100644 --- a/src/SharpCompress/Common/GZip/GZipEntry.cs +++ b/src/SharpCompress/Common/GZip/GZipEntry.cs @@ -7,24 +7,24 @@ namespace SharpCompress.Common.GZip { public class GZipEntry : Entry { - private readonly GZipFilePart filePart; + private readonly GZipFilePart _filePart; internal GZipEntry(GZipFilePart filePart) { - this.filePart = filePart; + _filePart = filePart; } public override CompressionType CompressionType => CompressionType.GZip; public override long Crc => 0; - public override string Key => filePart.FilePartName; + public override string Key => _filePart.FilePartName; public override long CompressedSize => 0; public override long Size => 0; - public override DateTime? LastModifiedTime => filePart.DateModified; + public override DateTime? LastModifiedTime => _filePart.DateModified; public override DateTime? CreatedTime => null; @@ -36,9 +36,9 @@ namespace SharpCompress.Common.GZip public override bool IsDirectory => false; - public override bool IsSplit => false; + public override bool IsSplitAfter => false; - internal override IEnumerable Parts => filePart.AsEnumerable(); + internal override IEnumerable Parts => _filePart.AsEnumerable(); internal static IEnumerable GetEntries(Stream stream, OptionsBase options) { diff --git a/src/SharpCompress/Common/GZip/GZipFilePart.cs b/src/SharpCompress/Common/GZip/GZipFilePart.cs index 30cc20e9..1f942e74 100644 --- a/src/SharpCompress/Common/GZip/GZipFilePart.cs +++ b/src/SharpCompress/Common/GZip/GZipFilePart.cs @@ -19,7 +19,7 @@ namespace SharpCompress.Common.GZip { ReadAndValidateGzipHeader(stream); EntryStartPosition = stream.Position; - this._stream = stream; + _stream = stream; } internal long EntryStartPosition { get; } @@ -30,7 +30,7 @@ namespace SharpCompress.Common.GZip internal override Stream GetCompressedStream() { - return new DeflateStream(_stream, CompressionMode.Decompress, CompressionLevel.Default, false); + return new DeflateStream(_stream, CompressionMode.Decompress, CompressionLevel.Default); } internal override Stream GetRawStream() @@ -61,7 +61,7 @@ namespace SharpCompress.Common.GZip } Int32 timet = DataConverter.LittleEndian.GetInt32(header, 4); - DateModified = TarHeader.Epoch.AddSeconds(timet); + DateModified = TarHeader.EPOCH.AddSeconds(timet); if ((header[3] & 0x04) == 0x04) { // read and discard extra field diff --git a/src/SharpCompress/Common/IEntry.Extensions.cs b/src/SharpCompress/Common/IEntry.Extensions.cs index 70a634ac..76d75a8f 100644 --- a/src/SharpCompress/Common/IEntry.Extensions.cs +++ b/src/SharpCompress/Common/IEntry.Extensions.cs @@ -5,7 +5,7 @@ using SharpCompress.Readers; namespace SharpCompress.Common { - internal static class IEntryExtensions + internal static class EntryExtensions { internal static void PreserveExtractionOptions(this IEntry entry, string destinationFileName, ExtractionOptions options) diff --git a/src/SharpCompress/Common/IEntry.cs b/src/SharpCompress/Common/IEntry.cs index ac15828c..e8f13a30 100644 --- a/src/SharpCompress/Common/IEntry.cs +++ b/src/SharpCompress/Common/IEntry.cs @@ -12,7 +12,7 @@ namespace SharpCompress.Common string Key { get; } bool IsDirectory { get; } bool IsEncrypted { get; } - bool IsSplit { get; } + bool IsSplitAfter { get; } DateTime? LastAccessedTime { get; } DateTime? LastModifiedTime { get; } long Size { get; } diff --git a/src/SharpCompress/Common/Rar/Headers/AVHeader.cs b/src/SharpCompress/Common/Rar/Headers/AVHeader.cs index acb878ba..51dda2cf 100644 --- a/src/SharpCompress/Common/Rar/Headers/AVHeader.cs +++ b/src/SharpCompress/Common/Rar/Headers/AVHeader.cs @@ -2,19 +2,29 @@ namespace SharpCompress.Common.Rar.Headers { - internal class SignHeader : RarHeader + internal class AvHeader : RarHeader { - protected override void ReadFromReader(MarkingBinaryReader reader) + public AvHeader(RarHeader header, RarCrcBinaryReader reader) + : base(header, reader, HeaderType.Av) { - CreationTime = reader.ReadInt32(); - ArcNameSize = reader.ReadInt16(); - UserNameSize = reader.ReadInt16(); + if (IsRar5) + throw new InvalidFormatException("unexpected rar5 record"); } - internal int CreationTime { get; private set; } + protected override void ReadFinish(MarkingBinaryReader reader) + { + UnpackVersion = reader.ReadByte(); + Method = reader.ReadByte(); + AvVersion = reader.ReadByte(); + AvInfoCrc = reader.ReadInt32(); + } - internal short ArcNameSize { get; private set; } + internal int AvInfoCrc { get; private set; } - internal short UserNameSize { get; private set; } + internal byte UnpackVersion { get; private set; } + + internal byte Method { get; private set; } + + internal byte AvVersion { get; private set; } } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/Headers/ArchiveCryptHeader.cs b/src/SharpCompress/Common/Rar/Headers/ArchiveCryptHeader.cs new file mode 100644 index 00000000..389a0697 --- /dev/null +++ b/src/SharpCompress/Common/Rar/Headers/ArchiveCryptHeader.cs @@ -0,0 +1,57 @@ +using SharpCompress.IO; + +namespace SharpCompress.Common.Rar.Headers +{ + internal class ArchiveCryptHeader : RarHeader + { + + private const int CRYPT_VERSION = 0; // Supported encryption version. + private const int SIZE_SALT50 = 16; + private const int SIZE_SALT30 = 8; + private const int SIZE_INITV = 16; + private const int SIZE_PSWCHECK = 8; + private const int SIZE_PSWCHECK_CSUM = 4; + private const int CRYPT5_KDF_LG2_COUNT = 15; // LOG2 of PDKDF2 iteration count. + private const int CRYPT5_KDF_LG2_COUNT_MAX = 24; // LOG2 of maximum accepted iteration count. + + + private bool _usePswCheck; + private uint _lg2Count; // Log2 of PBKDF2 repetition count. + private byte[] _salt; + private byte[] _pswCheck; + private byte[] _pswCheckCsm; + + public ArchiveCryptHeader(RarHeader header, RarCrcBinaryReader reader) + : base(header, reader, HeaderType.Crypt) + { + } + + protected override void ReadFinish(MarkingBinaryReader reader) + { + var cryptVersion = reader.ReadRarVIntUInt32(); + if (cryptVersion > CRYPT_VERSION) + { + //error? + return; + } + var encryptionFlags = reader.ReadRarVIntUInt32(); + _usePswCheck = FlagUtility.HasFlag(encryptionFlags, EncryptionFlagsV5.CHFL_CRYPT_PSWCHECK); + _lg2Count = reader.ReadRarVIntByte(1); + + + //UsePswCheck = HasHeaderFlag(EncryptionFlagsV5.CHFL_CRYPT_PSWCHECK); + if (_lg2Count > CRYPT5_KDF_LG2_COUNT_MAX) + { + //error? + return; + } + + _salt = reader.ReadBytes(SIZE_SALT50); + if (_usePswCheck) + { + _pswCheck = reader.ReadBytes(SIZE_PSWCHECK); + _pswCheckCsm = reader.ReadBytes(SIZE_PSWCHECK_CSUM); + } + } + } +} \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/Headers/ArchiveHeader.cs b/src/SharpCompress/Common/Rar/Headers/ArchiveHeader.cs index 214a25c0..72b4bd4b 100644 --- a/src/SharpCompress/Common/Rar/Headers/ArchiveHeader.cs +++ b/src/SharpCompress/Common/Rar/Headers/ArchiveHeader.cs @@ -4,27 +4,78 @@ namespace SharpCompress.Common.Rar.Headers { internal class ArchiveHeader : RarHeader { - internal const short mainHeaderSizeWithEnc = 7; - internal const short mainHeaderSize = 6; - - protected override void ReadFromReader(MarkingBinaryReader reader) + public ArchiveHeader(RarHeader header, RarCrcBinaryReader reader) + : base(header, reader, HeaderType.Archive) { - HighPosAv = reader.ReadInt16(); - PosAv = reader.ReadInt32(); - if (ArchiveHeaderFlags.HasFlag(ArchiveFlags.ENCRYPTVER)) + } + + protected override void ReadFinish(MarkingBinaryReader reader) + { + if (IsRar5) { - EncryptionVersion = reader.ReadByte(); + Flags = reader.ReadRarVIntUInt16(); + if (HasFlag(ArchiveFlagsV5.HAS_VOLUME_NUMBER)) + { + VolumeNumber = (int)reader.ReadRarVIntUInt32(); + } + // later: we may have a locator record if we need it + //if (ExtraSize != 0) { + // ReadLocator(reader); + //} + } + else + { + Flags = HeaderFlags; + HighPosAv = reader.ReadInt16(); + PosAv = reader.ReadInt32(); + if (HasFlag(ArchiveFlagsV4.ENCRYPT_VER)) + { + EncryptionVersion = reader.ReadByte(); + } } } - internal ArchiveFlags ArchiveHeaderFlags => (ArchiveFlags)Flags; + private void ReadLocator(MarkingBinaryReader reader) { + var size = reader.ReadRarVIntUInt16(); + var type = reader.ReadRarVIntUInt16(); + if (type != 1) throw new InvalidFormatException("expected locator record"); + var flags = reader.ReadRarVIntUInt16(); + const ushort hasQuickOpenOffset = 0x01; + const ushort hasRecoveryOffset = 0x02; + ulong quickOpenOffset = 0; + if ((flags & hasQuickOpenOffset) == hasQuickOpenOffset) { + quickOpenOffset = reader.ReadRarVInt(); + } + ulong recoveryOffset = 0; + if ((flags & hasRecoveryOffset) == hasRecoveryOffset) { + recoveryOffset = reader.ReadRarVInt(); + } + } - internal short HighPosAv { get; private set; } + private ushort Flags { get; set; } - internal int PosAv { get; private set; } + private bool HasFlag(ushort flag) + { + return (Flags & flag) == flag; + } - internal byte EncryptionVersion { get; private set; } + internal int? VolumeNumber { get; private set; } - public bool HasPassword => ArchiveHeaderFlags.HasFlag(ArchiveFlags.PASSWORD); + internal short? HighPosAv { get; private set; } + + internal int? PosAv { get; private set; } + + private byte? EncryptionVersion { get; set; } + + public bool? IsEncrypted => IsRar5 ? (bool?)null : HasFlag(ArchiveFlagsV4.PASSWORD); + + public bool OldNumberingFormat => !IsRar5 && !HasFlag(ArchiveFlagsV4.NEW_NUMBERING); + + public bool IsVolume => HasFlag(IsRar5 ? ArchiveFlagsV5.VOLUME : ArchiveFlagsV4.VOLUME); + + // RAR5: Volume number field is present. True for all volumes except first. + public bool IsFirstVolume => IsRar5 ? VolumeNumber == null : HasFlag(ArchiveFlagsV4.FIRST_VOLUME); + + public bool IsSolid => HasFlag(IsRar5 ? ArchiveFlagsV5.SOLID : ArchiveFlagsV4.SOLID); } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/Headers/CommentHeader.cs b/src/SharpCompress/Common/Rar/Headers/CommentHeader.cs index c9943bde..4845ce2b 100644 --- a/src/SharpCompress/Common/Rar/Headers/CommentHeader.cs +++ b/src/SharpCompress/Common/Rar/Headers/CommentHeader.cs @@ -4,12 +4,18 @@ namespace SharpCompress.Common.Rar.Headers { internal class CommentHeader : RarHeader { - protected override void ReadFromReader(MarkingBinaryReader reader) + protected CommentHeader(RarHeader header, RarCrcBinaryReader reader) + : base(header, reader, HeaderType.Comment) + { + if (IsRar5) throw new InvalidFormatException("unexpected rar5 record"); + } + + protected override void ReadFinish(MarkingBinaryReader reader) { UnpSize = reader.ReadInt16(); UnpVersion = reader.ReadByte(); UnpMethod = reader.ReadByte(); - CommCRC = reader.ReadInt16(); + CommCrc = reader.ReadInt16(); } internal short UnpSize { get; private set; } @@ -17,6 +23,6 @@ namespace SharpCompress.Common.Rar.Headers internal byte UnpVersion { get; private set; } internal byte UnpMethod { get; private set; } - internal short CommCRC { get; private set; } + internal short CommCrc { get; private set; } } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/Headers/EndArchiveHeader.cs b/src/SharpCompress/Common/Rar/Headers/EndArchiveHeader.cs index b2f62039..926a8dfe 100644 --- a/src/SharpCompress/Common/Rar/Headers/EndArchiveHeader.cs +++ b/src/SharpCompress/Common/Rar/Headers/EndArchiveHeader.cs @@ -4,21 +4,39 @@ namespace SharpCompress.Common.Rar.Headers { internal class EndArchiveHeader : RarHeader { - protected override void ReadFromReader(MarkingBinaryReader reader) + public EndArchiveHeader(RarHeader header, RarCrcBinaryReader reader) + : base(header, reader, HeaderType.EndArchive) { - if (EndArchiveFlags.HasFlag(EndArchiveFlags.EARC_DATACRC)) + } + + protected override void ReadFinish(MarkingBinaryReader reader) + { + if (IsRar5) { - ArchiveCRC = reader.ReadInt32(); + Flags = reader.ReadRarVIntUInt16(); } - if (EndArchiveFlags.HasFlag(EndArchiveFlags.EARC_VOLNUMBER)) + else { - VolumeNumber = reader.ReadInt16(); + Flags = HeaderFlags; + if (HasFlag(EndArchiveFlagsV4.DATA_CRC)) + { + ArchiveCrc = reader.ReadInt32(); + } + if (HasFlag(EndArchiveFlagsV4.VOLUME_NUMBER)) + { + VolumeNumber = reader.ReadInt16(); + } } } - internal EndArchiveFlags EndArchiveFlags => (EndArchiveFlags)Flags; + private ushort Flags { get; set; } - internal int? ArchiveCRC { get; private set; } + private bool HasFlag(ushort flag) + { + return (Flags & flag) == flag; + } + + internal int? ArchiveCrc { get; private set; } internal short? VolumeNumber { get; private set; } } diff --git a/src/SharpCompress/Common/Rar/Headers/FileHeader.cs b/src/SharpCompress/Common/Rar/Headers/FileHeader.cs index 1b539eb7..829083e9 100644 --- a/src/SharpCompress/Common/Rar/Headers/FileHeader.cs +++ b/src/SharpCompress/Common/Rar/Headers/FileHeader.cs @@ -1,35 +1,235 @@ +#if !Rar2017_64bit +using nint = System.Int32; +using nuint = System.UInt32; +using size_t = System.UInt32; +#else +using nint = System.Int64; +using nuint = System.UInt64; +using size_t = System.UInt64; +#endif + using SharpCompress.IO; using System; using System.IO; +using System.Text; namespace SharpCompress.Common.Rar.Headers { internal class FileHeader : RarHeader { - private const byte SALT_SIZE = 8; + private uint _fileCrc; - private const byte NEWLHD_SIZE = 32; - - protected override void ReadFromReader(MarkingBinaryReader reader) + public FileHeader(RarHeader header, RarCrcBinaryReader reader, HeaderType headerType) + : base(header, reader, headerType) { + } + + protected override void ReadFinish(MarkingBinaryReader reader) + { + if (IsRar5) + { + ReadFromReaderV5(reader); + } + else + { + ReadFromReaderV4(reader); + } + } + + private void ReadFromReaderV5(MarkingBinaryReader reader) + { + Flags = reader.ReadRarVIntUInt16(); + + var lvalue = checked((long)reader.ReadRarVInt()); + + // long.MaxValue causes the unpack code to finish when the input stream is exhausted + UncompressedSize = HasFlag(FileFlagsV5.UNPACKED_SIZE_UNKNOWN) ? long.MaxValue : lvalue; + + FileAttributes = reader.ReadRarVIntUInt32(); + + if (HasFlag(FileFlagsV5.HAS_MOD_TIME)) { + FileLastModifiedTime = Utility.UnixTimeToDateTime(reader.ReadUInt32()); + } + + if (HasFlag(FileFlagsV5.HAS_CRC32)) { + FileCrc = reader.ReadUInt32(); + } + + var compressionInfo = reader.ReadRarVIntUInt16(); + + // Lower 6 bits (0x003f mask) contain the version of compression algorithm, resulting in possible 0 - 63 values. Current version is 0. + // "+ 50" to not mix with old RAR format algorithms. For example, + // we may need to use the compression algorithm 15 in the future, + // but it was already used in RAR 1.5 and Unpack needs to distinguish + // them. + CompressionAlgorithm = (byte)((compressionInfo & 0x3f) + 50); + + // 7th bit (0x0040) defines the solid flag. If it is set, RAR continues to use the compression dictionary left after processing preceding files. + // It can be set only for file headers and is never set for service headers. + IsSolid = (compressionInfo & 0x40) == 0x40; + + // Bits 8 - 10 (0x0380 mask) define the compression method. Currently only values 0 - 5 are used. 0 means no compression. + CompressionMethod = (byte)((compressionInfo >> 7) & 0x7); + + // Bits 11 - 14 (0x3c00) define the minimum size of dictionary size required to extract data. Value 0 means 128 KB, 1 - 256 KB, ..., 14 - 2048 MB, 15 - 4096 MB. + WindowSize = IsDirectory ? 0 : ((size_t)0x20000) << ((compressionInfo>>10) & 0xf); + + HostOs = reader.ReadRarVIntByte(); + + var nameSize = reader.ReadRarVIntUInt16(); + + // Variable length field containing Name length bytes in UTF-8 format without trailing zero. + // For file header this is a name of archived file. Forward slash character is used as the path separator both for Unix and Windows names. + // Backslashes are treated as a part of name for Unix names and as invalid character for Windows file names. Type of name is defined by Host OS field. + // + // TODO: not sure if anything needs to be done to handle the following: + // If Unix file name contains any high ASCII characters which cannot be correctly converted to Unicode and UTF-8 + // we map such characters to to 0xE080 - 0xE0FF private use Unicode area and insert 0xFFFE Unicode non-character + // to resulting string to indicate that it contains mapped characters, which need to be converted back when extracting. + // Concrete position of 0xFFFE is not defined, we need to search the entire string for it. Such mapped names are not + // portable and can be correctly unpacked only on the same system where they were created. + // + // For service header this field contains a name of service header. Now the following names are used: + // CMT Archive comment + // QO Archive quick open data + // ACL NTFS file permissions + // STM NTFS alternate data stream + // RR Recovery record + var b = reader.ReadBytes(nameSize); + FileName = ConvertPathV5(Encoding.UTF8.GetString(b, 0, b.Length)); + + // extra size seems to be redudant since we know the total header size + if (ExtraSize != RemainingHeaderBytes(reader)) + { + throw new InvalidFormatException("rar5 header size / extra size inconsistency"); + } + + while (RemainingHeaderBytes(reader) > 0) { + var size = reader.ReadRarVIntUInt16(); + int n = RemainingHeaderBytes(reader); + var type = reader.ReadRarVIntUInt16(); + switch (type) { +//TODO +// case 1: // file encryption +// { +// var version = reader.ReadRarVIntByte(); +// if (version != 0) throw new InvalidFormatException("unknown encryption algorithm "+ version); +// +// } +// break; +// case 2: // file hash +// { +// +// } +// break; + case 3: // file time + { + ushort flags = reader.ReadRarVIntUInt16(); + var isWindowsTime = (flags & 1) == 0; + if ((flags & 0x2) == 0x2) { + FileLastModifiedTime = ReadExtendedTimeV5(reader, isWindowsTime); + } + if ((flags & 0x4) == 0x4) { + FileCreatedTime = ReadExtendedTimeV5(reader, isWindowsTime); + } + if ((flags & 0x8) == 0x8) { + FileLastAccessedTime = ReadExtendedTimeV5(reader, isWindowsTime); + } + } + break; +//TODO +// case 4: // file version +// { +// +// } +// break; +// case 5: // file system redirection +// { +// +// } +// break; +// case 6: // unix owner +// { +// +// } +// break; +// case 7: // service data +// { +// +// } +// break; + + default: + // skip unknown record types to allow new record types to be added in the future + break; + } + // drain any trailing bytes of extra record + int did = n - RemainingHeaderBytes(reader); + int drain = size - did; + if (drain > 0) + { + reader.ReadBytes(drain); + } + } + + if (AdditionalDataSize != 0) { + CompressedSize = AdditionalDataSize; + } + } + + + private static DateTime ReadExtendedTimeV5(MarkingBinaryReader reader, bool isWindowsTime) + { + if (isWindowsTime) + { + return DateTime.FromFileTime(reader.ReadInt64()); + } + else + { + return Utility.UnixTimeToDateTime(reader.ReadUInt32()); + } + } + + private static string ConvertPathV5(string path) + { +#if NO_FILE + // not sure what to do here + throw new NotImplementedException("TODO"); +#else + if (Path.DirectorySeparatorChar == '\\') + { + // replace embedded \\ with valid filename char + return path.Replace('\\', '-').Replace('/', '\\'); + } + return path; +#endif + } + + + private void ReadFromReaderV4(MarkingBinaryReader reader) + { + Flags = HeaderFlags; + IsSolid = HasFlag(FileFlagsV4.SOLID); + WindowSize = IsDirectory ? 0U : ((size_t)0x10000) << ((Flags & FileFlagsV4.WINDOW_MASK) >> 5); + uint lowUncompressedSize = reader.ReadUInt32(); - HostOS = (HostOS)reader.ReadByte(); + HostOs = reader.ReadByte(); - FileCRC = reader.ReadUInt32(); + FileCrc = reader.ReadUInt32(); - FileLastModifiedTime = Utility.DosDateToDateTime(reader.ReadInt32()); + FileLastModifiedTime = Utility.DosDateToDateTime(reader.ReadUInt32()); - RarVersion = reader.ReadByte(); - PackingMethod = reader.ReadByte(); + CompressionAlgorithm = reader.ReadByte(); + CompressionMethod = (byte)(reader.ReadByte() - 0x30); short nameSize = reader.ReadInt16(); - FileAttributes = reader.ReadInt32(); + FileAttributes = reader.ReadUInt32(); uint highCompressedSize = 0; uint highUncompressedkSize = 0; - if (FileFlags.HasFlag(FileFlags.LARGE)) + if (HasFlag(FileFlagsV4.LARGE)) { highCompressedSize = reader.ReadUInt32(); highUncompressedkSize = reader.ReadUInt32(); @@ -42,18 +242,21 @@ namespace SharpCompress.Common.Rar.Headers highUncompressedkSize = int.MaxValue; } } - CompressedSize = UInt32To64(highCompressedSize, AdditionalSize); + CompressedSize = UInt32To64(highCompressedSize, checked((uint)AdditionalDataSize)); UncompressedSize = UInt32To64(highUncompressedkSize, lowUncompressedSize); nameSize = nameSize > 4 * 1024 ? (short)(4 * 1024) : nameSize; byte[] fileNameBytes = reader.ReadBytes(nameSize); - switch (HeaderType) + const int saltSize = 8; + const int newLhdSize = 32; + + switch (HeaderCode) { - case HeaderType.FileHeader: + case HeaderCodeV.RAR4_FILE_HEADER: { - if (FileFlags.HasFlag(FileFlags.UNICODE)) + if (HasFlag(FileFlagsV4.UNICODE)) { int length = 0; while (length < fileNameBytes.Length @@ -75,15 +278,15 @@ namespace SharpCompress.Common.Rar.Headers { FileName = ArchiveEncoding.Decode(fileNameBytes); } - FileName = ConvertPath(FileName, HostOS); + FileName = ConvertPathV4(FileName); } break; - case HeaderType.NewSubHeader: + case HeaderCodeV.RAR4_NEW_SUB_HEADER: { - int datasize = HeaderSize - NEWLHD_SIZE - nameSize; - if (FileFlags.HasFlag(FileFlags.SALT)) + int datasize = HeaderSize - newLhdSize - nameSize; + if (HasFlag(FileFlagsV4.SALT)) { - datasize -= SALT_SIZE; + datasize -= saltSize; } if (datasize > 0) { @@ -99,34 +302,33 @@ namespace SharpCompress.Common.Rar.Headers break; } - if (FileFlags.HasFlag(FileFlags.SALT)) + if (HasFlag(FileFlagsV4.SALT)) { - Salt = reader.ReadBytes(SALT_SIZE); + R4Salt = reader.ReadBytes(saltSize); } - if (FileFlags.HasFlag(FileFlags.EXTTIME)) + if (HasFlag(FileFlagsV4.EXT_TIME)) { // verify that the end of the header hasn't been reached before reading the Extended Time. // some tools incorrectly omit Extended Time despite specifying FileFlags.EXTTIME, which most parsers tolerate. - if (ReadBytes + reader.CurrentReadByteCount <= HeaderSize - 2) + if (RemainingHeaderBytes(reader) >= 2) { ushort extendedFlags = reader.ReadUInt16(); - FileLastModifiedTime = ProcessExtendedTime(extendedFlags, FileLastModifiedTime, reader, 0); - FileCreatedTime = ProcessExtendedTime(extendedFlags, null, reader, 1); - FileLastAccessedTime = ProcessExtendedTime(extendedFlags, null, reader, 2); - FileArchivedTime = ProcessExtendedTime(extendedFlags, null, reader, 3); + FileLastModifiedTime = ProcessExtendedTimeV4(extendedFlags, FileLastModifiedTime, reader, 0); + FileCreatedTime = ProcessExtendedTimeV4(extendedFlags, null, reader, 1); + FileLastAccessedTime = ProcessExtendedTimeV4(extendedFlags, null, reader, 2); + FileArchivedTime = ProcessExtendedTimeV4(extendedFlags, null, reader, 3); } } } - private long UInt32To64(uint x, uint y) + private static long UInt32To64(uint x, uint y) { long l = x; l <<= 32; return l + y; } - private static DateTime? ProcessExtendedTime(ushort extendedFlags, DateTime? time, MarkingBinaryReader reader, - int i) + private static DateTime? ProcessExtendedTimeV4(ushort extendedFlags, DateTime? time, MarkingBinaryReader reader, int i) { uint rmode = (uint)extendedFlags >> (3 - i) * 4; if ((rmode & 8) == 0) @@ -135,8 +337,8 @@ namespace SharpCompress.Common.Rar.Headers } if (i != 0) { - uint DosTime = reader.ReadUInt32(); - time = Utility.DosDateToDateTime(DosTime); + uint dosTime = reader.ReadUInt32(); + time = Utility.DosDateToDateTime(dosTime); } if ((rmode & 4) == 0) { @@ -154,7 +356,7 @@ namespace SharpCompress.Common.Rar.Headers return time.Value.AddMilliseconds(nanosecondHundreds * Math.Pow(10, -4)); } - private static string ConvertPath(string path, HostOS os) + private static string ConvertPathV4(string path) { #if NO_FILE return path.Replace('\\', '/'); @@ -171,12 +373,71 @@ namespace SharpCompress.Common.Rar.Headers #endif } + public override string ToString() + { + return FileName; + } + + private ushort Flags { get; set; } + + private bool HasFlag(ushort flag) + { + return (Flags & flag) == flag; + } + + internal uint FileCrc + { + get { + if (IsRar5 && !HasFlag(FileFlagsV5.HAS_CRC32)) { +//!!! rar5: + throw new InvalidOperationException("TODO rar5"); + } + return _fileCrc; + } + private set => _fileCrc = value; + } + + // 0 - storing + // 1 - fastest compression + // 2 - fast compression + // 3 - normal compression + // 4 - good compression + // 5 - best compression + internal byte CompressionMethod { get; private set; } + internal bool IsStored => CompressionMethod == 0; + + // eg (see DoUnpack()) + //case 15: // rar 1.5 compression + //case 20: // rar 2.x compression + //case 26: // files larger than 2GB + //case 29: // rar 3.x compression + //case 50: // RAR 5.0 compression algorithm. + internal byte CompressionAlgorithm { get; private set; } + + public bool IsSolid { get; private set; } + + // unused for UnpackV1 implementation (limitation) + internal size_t WindowSize { get; private set; } + + internal byte[] R4Salt { get; private set; } + + private byte HostOs { get; set; } + internal uint FileAttributes { get; private set; } + internal long CompressedSize { get; private set; } + internal long UncompressedSize { get; private set; } + internal string FileName { get; private set; } + internal byte[] SubData { get; private set; } + internal int RecoverySectors { get; private set; } internal long DataStartPosition { get; set; } + public Stream PackedStream { get; set; } - internal HostOS HostOS { get; private set; } + public bool IsSplitAfter => IsRar5 ? HasHeaderFlag(HeaderFlagsV5.SPLIT_AFTER) : HasFlag(FileFlagsV4.SPLIT_AFTER); - internal uint FileCRC { get; private set; } + public bool IsDirectory => HasFlag(IsRar5 ? FileFlagsV5.DIRECTORY : FileFlagsV4.DIRECTORY); +//!!! TODO rar5 + public bool IsEncrypted => HasFlag(FileFlagsV4.PASSWORD); + internal DateTime? FileLastModifiedTime { get; private set; } internal DateTime? FileCreatedTime { get; private set; } @@ -184,32 +445,5 @@ namespace SharpCompress.Common.Rar.Headers internal DateTime? FileLastAccessedTime { get; private set; } internal DateTime? FileArchivedTime { get; private set; } - - internal byte RarVersion { get; private set; } - - internal byte PackingMethod { get; private set; } - - internal int FileAttributes { get; private set; } - - internal FileFlags FileFlags => (FileFlags)Flags; - - internal long CompressedSize { get; private set; } - - internal long UncompressedSize { get; private set; } - - internal string FileName { get; private set; } - - internal byte[] SubData { get; private set; } - - internal int RecoverySectors { get; private set; } - - internal byte[] Salt { get; private set; } - - public override string ToString() - { - return FileName; - } - - public Stream PackedStream { get; set; } } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/Headers/Flags.cs b/src/SharpCompress/Common/Rar/Headers/Flags.cs index a2dbd550..0c19079b 100644 --- a/src/SharpCompress/Common/Rar/Headers/Flags.cs +++ b/src/SharpCompress/Common/Rar/Headers/Flags.cs @@ -1,84 +1,149 @@ -using System; - namespace SharpCompress.Common.Rar.Headers { - internal enum HeaderType + internal enum HeaderType : byte { - MarkHeader = 0x72, - ArchiveHeader = 0x73, - FileHeader = 0x74, - CommHeader = 0x75, - AvHeader = 0x76, - SubHeader = 0x77, - ProtectHeader = 0x78, - SignHeader = 0x79, - NewSubHeader = 0x7a, - EndArchiveHeader = 0x7b + Null, + Mark, + Archive, + File, + Service, + Comment, + Av, + Protect, + Sign, + NewSub, + EndArchive, + Crypt } - internal enum HeaderFlags : short + internal static class HeaderCodeV { - LONG_BLOCK = -0x8000 + public const byte RAR4_MARK_HEADER = 0x72; + public const byte RAR4_ARCHIVE_HEADER = 0x73; + public const byte RAR4_FILE_HEADER = 0x74; + public const byte RAR4_COMMENT_HEADER = 0x75; + public const byte RAR4_AV_HEADER = 0x76; + public const byte RAR4_SUB_HEADER = 0x77; + public const byte RAR4_PROTECT_HEADER = 0x78; + public const byte RAR4_SIGN_HEADER = 0x79; + public const byte RAR4_NEW_SUB_HEADER = 0x7a; + public const byte RAR4_END_ARCHIVE_HEADER = 0x7b; + + public const byte RAR5_ARCHIVE_HEADER = 0x01; + public const byte RAR5_FILE_HEADER = 0x02; + public const byte RAR5_SERVICE_HEADER = 0x03; + public const byte RAR5_ARCHIVE_ENCRYPTION_HEADER = 0x04; + public const byte RAR5_END_ARCHIVE_HEADER = 0x05; } - [Flags] - internal enum ArchiveFlags + internal static class HeaderFlagsV4 { - VOLUME = 0x0001, - COMMENT = 0x0002, - LOCK = 0x0004, - SOLID = 0x0008, - NEWNUMBERING = 0x0010, - AV = 0x0020, - PROTECT = 0x0040, - PASSWORD = 0x0080, - FIRSTVOLUME = 0x0100, - ENCRYPTVER = 0x0200 + public const ushort HAS_DATA = 0x8000; } - internal enum HostOS - { - MSDOS = 0, - OS2 = 1, - Win32 = 2, - Unix = 3, - MacOS = 4, - BeOS = 5 + internal static class EncryptionFlagsV5 + { + // RAR 5.0 archive encryption header specific flags. + public const uint CHFL_CRYPT_PSWCHECK = 0x01; // Password check data is present. + + public const uint FHEXTRA_CRYPT_PSWCHECK = 0x01; // Password check data is present. + public const uint FHEXTRA_CRYPT_HASHMAC = 0x02; } - [Flags] - internal enum FileFlags : ushort + internal static class HeaderFlagsV5 { - SPLIT_BEFORE = 0x0001, - SPLIT_AFTER = 0x0002, - PASSWORD = 0x0004, - COMMENT = 0x0008, - SOLID = 0x0010, - - WINDOWMASK = 0x00e0, - WINDOW64 = 0x0000, - WINDOW128 = 0x0020, - WINDOW256 = 0x0040, - WINDOW512 = 0x0060, - WINDOW1024 = 0x0080, - WINDOW2048 = 0x00a0, - WINDOW4096 = 0x00c0, - DIRECTORY = 0x00e0, - - LARGE = 0x0100, - UNICODE = 0x0200, - SALT = 0x0400, - VERSION = 0x0800, - EXTTIME = 0x1000, - EXTFLAGS = 0x2000 + public const ushort HAS_EXTRA = 0x0001; + public const ushort HAS_DATA = 0x0002; + public const ushort KEEP = 0x0004; // block must be kept during an update + public const ushort SPLIT_BEFORE = 0x0008; + public const ushort SPLIT_AFTER = 0x0010; + public const ushort CHILD = 0x0020; // ??? Block depends on preceding file block. + public const ushort PRESERVE_CHILD = 0x0040; // ???? Preserve a child block if host block is modified } - [Flags] - internal enum EndArchiveFlags + internal static class ArchiveFlagsV4 { - EARC_NEXT_VOLUME = 0x0001, - EARC_DATACRC = 0x0002, - EARC_REVSPACE = 0x0004, - EARC_VOLNUMBER = 0x0008 + public const ushort VOLUME = 0x0001; + public const ushort COMMENT = 0x0002; + public const ushort LOCK = 0x0004; + public const ushort SOLID = 0x0008; + public const ushort NEW_NUMBERING = 0x0010; + public const ushort AV = 0x0020; + public const ushort PROTECT = 0x0040; + public const ushort PASSWORD = 0x0080; + public const ushort FIRST_VOLUME = 0x0100; + public const ushort ENCRYPT_VER = 0x0200; + } + + internal static class ArchiveFlagsV5 + { + public const ushort VOLUME = 0x0001; + public const ushort HAS_VOLUME_NUMBER = 0x0002; + public const ushort SOLID = 0x0004; + public const ushort PROTECT = 0x0008; + public const ushort LOCK = 0x0010; + } + + internal static class HostOsV4 + { + public const byte MS_DOS = 0; + public const byte OS2 = 1; + public const byte WIN32 = 2; + public const byte UNIX = 3; + public const byte MAC_OS = 4; + public const byte BE_OS = 5; + } + + internal static class HostOsV5 + { + public const byte WINDOWS = 0; + public const byte UNIX = 1; + } + + internal static class FileFlagsV4 + { + public const ushort SPLIT_BEFORE = 0x0001; + public const ushort SPLIT_AFTER = 0x0002; + public const ushort PASSWORD = 0x0004; + public const ushort COMMENT = 0x0008; + public const ushort SOLID = 0x0010; + + public const ushort WINDOW_MASK = 0x00e0; + public const ushort WINDOW64 = 0x0000; + public const ushort WINDOW128 = 0x0020; + public const ushort WINDOW256 = 0x0040; + public const ushort WINDOW512 = 0x0060; + public const ushort WINDOW1024 = 0x0080; + public const ushort WINDOW2048 = 0x00a0; + public const ushort WINDOW4096 = 0x00c0; + public const ushort DIRECTORY = 0x00e0; + + public const ushort LARGE = 0x0100; + public const ushort UNICODE = 0x0200; + public const ushort SALT = 0x0400; + public const ushort VERSION = 0x0800; + public const ushort EXT_TIME = 0x1000; + public const ushort EXT_FLAGS = 0x2000; + } + + internal static class FileFlagsV5 + { + public const ushort DIRECTORY = 0x0001; + public const ushort HAS_MOD_TIME = 0x0002; + public const ushort HAS_CRC32 = 0x0004; + public const ushort UNPACKED_SIZE_UNKNOWN = 0x0008; + } + + internal static class EndArchiveFlagsV4 + { + public const ushort NEXT_VOLUME = 0x0001; + public const ushort DATA_CRC = 0x0002; + public const ushort REV_SPACE = 0x0004; + public const ushort VOLUME_NUMBER = 0x0008; + } + + internal static class EndArchiveFlagsV5 + { + public const ushort HAS_NEXT_VOLUME = 0x0001; } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/Headers/IRarHeader.cs b/src/SharpCompress/Common/Rar/Headers/IRarHeader.cs new file mode 100644 index 00000000..bbc03593 --- /dev/null +++ b/src/SharpCompress/Common/Rar/Headers/IRarHeader.cs @@ -0,0 +1,7 @@ +namespace SharpCompress.Common.Rar.Headers +{ + internal interface IRarHeader + { + HeaderType HeaderType { get; } + } +} \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/Headers/MarkHeader.cs b/src/SharpCompress/Common/Rar/Headers/MarkHeader.cs index 4c4d0b6e..3a27cf69 100644 --- a/src/SharpCompress/Common/Rar/Headers/MarkHeader.cs +++ b/src/SharpCompress/Common/Rar/Headers/MarkHeader.cs @@ -1,26 +1,96 @@ -using SharpCompress.IO; +using System; +using System.IO; namespace SharpCompress.Common.Rar.Headers { - internal class MarkHeader : RarHeader + internal class MarkHeader : IRarHeader { - protected override void ReadFromReader(MarkingBinaryReader reader) - { + private const int MAX_SFX_SIZE = 0x80000 - 16; //archive.cpp line 136 + + internal bool OldNumberingFormat { get; private set; } + + public bool IsRar5 { get; } + + private MarkHeader(bool isRar5) + { + IsRar5 = isRar5; } - internal bool IsValid() + public HeaderType HeaderType => HeaderType.Mark; + + private static byte GetByte(Stream stream) { - // Rar old signature: 52 45 7E 5E (not supported) - - // Rar4 signature: 52 61 72 21 1A 07 00 - return HeadCRC == 0x6152 && - HeaderType == HeaderType.MarkHeader && - Flags == 0x1A21 && - HeaderSize == 0x07; - - // Rar5 signature: 52 61 72 21 1A 07 10 00 (not supported yet) + var b = stream.ReadByte(); + if (b != -1) + { + return (byte)b; + } + throw new EndOfStreamException(); } - internal bool OldFormat { get; private set; } + public static MarkHeader Read(Stream stream, bool leaveStreamOpen, bool lookForHeader) + { + int maxScanIndex = lookForHeader ? MAX_SFX_SIZE : 0; + try + { + int start = -1; + var b = GetByte(stream); start++; + while (start <= maxScanIndex) + { + // Rar old signature: 52 45 7E 5E + // Rar4 signature: 52 61 72 21 1A 07 00 + // Rar5 signature: 52 61 72 21 1A 07 01 00 + if (b == 0x52) + { + b = GetByte(stream); start++; + if (b == 0x61) + { + b = GetByte(stream); start++; + if (b != 0x72) continue; + b = GetByte(stream); start++; + if (b != 0x21) continue; + b = GetByte(stream); start++; + if (b != 0x1a) continue; + b = GetByte(stream); start++; + if (b != 0x07) continue; + + b = GetByte(stream); start++; + if (b == 1) + { + b = GetByte(stream); start++; + if (b != 0) continue; + return new MarkHeader(true); // Rar5 + } + else if (b == 0) + { + return new MarkHeader(false); // Rar4 + } + } + else if (b == 0x45) + { + b = GetByte(stream); start++; + if (b != 0x7e) continue; + b = GetByte(stream); start++; + if (b != 0x5e) continue; + throw new InvalidFormatException("Rar format version pre-4 is unsupported."); + } + } + else + { + b = GetByte(stream); start++; + } + } + } + catch (Exception e) + { + if (!leaveStreamOpen) + { + stream.Dispose(); + } + throw new InvalidFormatException("Error trying to read rar signature.", e); + } + + throw new InvalidFormatException("Rar signature not found"); + } } -} \ No newline at end of file +} diff --git a/src/SharpCompress/Common/Rar/Headers/NewSubHeader.cs b/src/SharpCompress/Common/Rar/Headers/NewSubHeaderType.cs similarity index 82% rename from src/SharpCompress/Common/Rar/Headers/NewSubHeader.cs rename to src/SharpCompress/Common/Rar/Headers/NewSubHeaderType.cs index 6218f123..84d50aee 100644 --- a/src/SharpCompress/Common/Rar/Headers/NewSubHeader.cs +++ b/src/SharpCompress/Common/Rar/Headers/NewSubHeaderType.cs @@ -20,26 +20,26 @@ namespace SharpCompress.Common.Rar.Headers //internal static final NewSubHeaderType SUBHEAD_TYPE_BEOSEA = new NewSubHeaderType(new byte[]{'E','A','B','E'}); - private readonly byte[] bytes; + private readonly byte[] _bytes; private NewSubHeaderType(params char[] chars) { - bytes = new byte[chars.Length]; + _bytes = new byte[chars.Length]; for (int i = 0; i < chars.Length; ++i) { - bytes[i] = (byte)chars[i]; + _bytes[i] = (byte)chars[i]; } } internal bool Equals(byte[] bytes) { - if (this.bytes.Length != bytes.Length) + if (_bytes.Length != bytes.Length) { return false; } for (int i = 0; i < bytes.Length; ++i) { - if (this.bytes[i] != bytes[i]) + if (_bytes[i] != bytes[i]) { return false; } @@ -47,13 +47,9 @@ namespace SharpCompress.Common.Rar.Headers return true; } - #region IEquatable Members - public bool Equals(NewSubHeaderType other) { - return Equals(other.bytes); + return Equals(other._bytes); } - - #endregion } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/Headers/ProtectHeader.cs b/src/SharpCompress/Common/Rar/Headers/ProtectHeader.cs index e2f78c32..f7f0e8ba 100644 --- a/src/SharpCompress/Common/Rar/Headers/ProtectHeader.cs +++ b/src/SharpCompress/Common/Rar/Headers/ProtectHeader.cs @@ -5,7 +5,13 @@ namespace SharpCompress.Common.Rar.Headers // ProtectHeader is part of the Recovery Record feature internal class ProtectHeader : RarHeader { - protected override void ReadFromReader(MarkingBinaryReader reader) + public ProtectHeader(RarHeader header, RarCrcBinaryReader reader) + : base(header, reader, HeaderType.Protect) + { + if (IsRar5) throw new InvalidFormatException("unexpected rar5 record"); + } + + protected override void ReadFinish(MarkingBinaryReader reader) { Version = reader.ReadByte(); RecSectors = reader.ReadUInt16(); @@ -13,7 +19,7 @@ namespace SharpCompress.Common.Rar.Headers Mark = reader.ReadBytes(8); } - internal uint DataSize => AdditionalSize; + internal uint DataSize => checked((uint)AdditionalDataSize); internal byte Version { get; private set; } internal ushort RecSectors { get; private set; } internal uint TotalBlocks { get; private set; } diff --git a/src/SharpCompress/Common/Rar/Headers/RarHeader.cs b/src/SharpCompress/Common/Rar/Headers/RarHeader.cs index 30753aee..b15f6500 100644 --- a/src/SharpCompress/Common/Rar/Headers/RarHeader.cs +++ b/src/SharpCompress/Common/Rar/Headers/RarHeader.cs @@ -1,38 +1,21 @@ using System; using System.IO; using SharpCompress.IO; -using System.Text; namespace SharpCompress.Common.Rar.Headers { - internal class RarHeader + // http://www.forensicswiki.org/w/images/5/5b/RARFileStructure.txt + // https://www.rarlab.com/technote.htm + internal class RarHeader : IRarHeader { - internal const short BaseBlockSize = 7; - internal const short LONG_BLOCK = -0x8000; + private readonly HeaderType _headerType; + private readonly bool _isRar5; - private void FillBase(RarHeader baseHeader) - { - HeadCRC = baseHeader.HeadCRC; - HeaderType = baseHeader.HeaderType; - Flags = baseHeader.Flags; - HeaderSize = baseHeader.HeaderSize; - AdditionalSize = baseHeader.AdditionalSize; - ReadBytes = baseHeader.ReadBytes; - ArchiveEncoding = baseHeader.ArchiveEncoding; - } - - internal static RarHeader Create(RarCrcBinaryReader reader, ArchiveEncoding archiveEncoding) + internal static RarHeader TryReadBase(RarCrcBinaryReader reader, bool isRar5, ArchiveEncoding archiveEncoding) { try { - RarHeader header = new RarHeader(); - - header.ArchiveEncoding = archiveEncoding; - reader.Mark(); - header.ReadStartFromReader(reader); - header.ReadBytes += reader.CurrentReadByteCount; - - return header; + return new RarHeader(reader, isRar5, archiveEncoding); } catch (EndOfStreamException) { @@ -40,82 +23,108 @@ namespace SharpCompress.Common.Rar.Headers } } - private void ReadStartFromReader(RarCrcBinaryReader reader) + private RarHeader(RarCrcBinaryReader reader, bool isRar5, ArchiveEncoding archiveEncoding) { - HeadCRC = reader.ReadUInt16(); - reader.ResetCrc(); - HeaderType = (HeaderType)(reader.ReadByte() & 0xff); - Flags = reader.ReadInt16(); - HeaderSize = reader.ReadInt16(); - if (FlagUtility.HasFlag(Flags, LONG_BLOCK)) + _headerType = HeaderType.Null; + _isRar5 = isRar5; + ArchiveEncoding = archiveEncoding; + if (IsRar5) { - AdditionalSize = reader.ReadUInt32(); - } - } + HeaderCrc = reader.ReadUInt32(); + reader.ResetCrc(); + HeaderSize = (int)reader.ReadRarVIntUInt32(3); + reader.Mark(); + HeaderCode = reader.ReadRarVIntByte(); + HeaderFlags = reader.ReadRarVIntUInt16(2); - protected virtual void ReadFromReader(MarkingBinaryReader reader) - { - throw new NotImplementedException(); - } - - internal T PromoteHeader(RarCrcBinaryReader reader) - where T : RarHeader, new() - { - T header = new T(); - header.FillBase(this); - - reader.Mark(); - header.ReadFromReader(reader); - header.ReadBytes += reader.CurrentReadByteCount; - - int headerSizeDiff = header.HeaderSize - (int)header.ReadBytes; - - if (headerSizeDiff > 0) - { - reader.ReadBytes(headerSizeDiff); - } - - VerifyHeaderCrc(reader.GetCrc()); - - return header; - } - - private void VerifyHeaderCrc(ushort crc) - { - if (HeaderType != HeaderType.MarkHeader) - { - if (crc != HeadCRC) + if (HasHeaderFlag(HeaderFlagsV5.HAS_EXTRA)) { - throw new InvalidFormatException("rar header crc mismatch"); + ExtraSize = reader.ReadRarVIntUInt32(); + } + if (HasHeaderFlag(HeaderFlagsV5.HAS_DATA)) + { + AdditionalDataSize = (long)reader.ReadRarVInt(); + } + } else { + reader.Mark(); + HeaderCrc = reader.ReadUInt16(); + reader.ResetCrc(); + HeaderCode = reader.ReadByte(); + HeaderFlags = reader.ReadUInt16(); + HeaderSize = reader.ReadInt16(); + if (HasHeaderFlag(HeaderFlagsV4.HAS_DATA)) + { + AdditionalDataSize = reader.ReadUInt32(); } } } - protected virtual void PostReadingBytes(MarkingBinaryReader reader) - { + protected RarHeader(RarHeader header, RarCrcBinaryReader reader, HeaderType headerType) { + _headerType = headerType; + _isRar5 = header.IsRar5; + HeaderCrc = header.HeaderCrc; + HeaderCode = header.HeaderCode; + HeaderFlags = header.HeaderFlags; + HeaderSize = header.HeaderSize; + ExtraSize = header.ExtraSize; + AdditionalDataSize = header.AdditionalDataSize; + ArchiveEncoding = header.ArchiveEncoding; + ReadFinish(reader); + + int n = RemainingHeaderBytes(reader); + if (n > 0) + { + reader.ReadBytes(n); + } + + VerifyHeaderCrc(reader.GetCrc32()); } - /// - /// This is the number of bytes read when reading the header - /// - protected long ReadBytes { get; private set; } + protected int RemainingHeaderBytes(MarkingBinaryReader reader) { + return checked(HeaderSize - (int)reader.CurrentReadByteCount); + } - protected ushort HeadCRC { get; private set; } + protected virtual void ReadFinish(MarkingBinaryReader reader) + { + throw new NotImplementedException(); + } - internal HeaderType HeaderType { get; private set; } + private void VerifyHeaderCrc(uint crc32) + { + var b = (IsRar5 ? crc32 : (ushort)crc32) == HeaderCrc; + if (!b) + { + throw new InvalidFormatException("rar header crc mismatch"); + } + } + + public HeaderType HeaderType => _headerType; + + protected bool IsRar5 => _isRar5; + + protected uint HeaderCrc { get; } + + internal byte HeaderCode { get; } + + protected ushort HeaderFlags { get; } + + protected bool HasHeaderFlag(ushort flag) + { + return (HeaderFlags & flag) == flag; + } + + protected int HeaderSize { get; } + + internal ArchiveEncoding ArchiveEncoding { get; } /// - /// Untyped flags. These should be typed when Promoting to another header + /// Extra header size. /// - protected short Flags { get; private set; } - - protected short HeaderSize { get; private set; } - - internal ArchiveEncoding ArchiveEncoding { get; private set; } + protected uint ExtraSize { get; } /// - /// This additional size of the header could be file data + /// Size of additional data (eg file contents) /// - protected uint AdditionalSize { get; private set; } + protected long AdditionalDataSize { get; } } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/Headers/RarHeaderFactory.cs b/src/SharpCompress/Common/Rar/Headers/RarHeaderFactory.cs index d05840ee..8c4f34f7 100644 --- a/src/SharpCompress/Common/Rar/Headers/RarHeaderFactory.cs +++ b/src/SharpCompress/Common/Rar/Headers/RarHeaderFactory.cs @@ -8,7 +8,7 @@ namespace SharpCompress.Common.Rar.Headers { internal class RarHeaderFactory { - private const int MAX_SFX_SIZE = 0x80000 - 16; //archive.cpp line 136 + private bool _isRar5; internal RarHeaderFactory(StreamingMode mode, ReaderOptions options) { @@ -20,141 +20,67 @@ namespace SharpCompress.Common.Rar.Headers internal StreamingMode StreamingMode { get; } internal bool IsEncrypted { get; private set; } - internal IEnumerable ReadHeaders(Stream stream) + internal IEnumerable ReadHeaders(Stream stream) { - if (Options.LookForHeader) - { - stream = CheckSFX(stream); - } + var markHeader = MarkHeader.Read(stream, Options.LeaveStreamOpen, Options.LookForHeader); + _isRar5 = markHeader.IsRar5; + yield return markHeader; RarHeader header; - while ((header = ReadNextHeader(stream)) != null) + while ((header = TryReadNextHeader(stream)) != null) { yield return header; - if (header.HeaderType == HeaderType.EndArchiveHeader) + if (header.HeaderType == HeaderType.EndArchive) { - yield break; // the end? + // End of archive marker. RAR does not read anything after this header letting to use third + // party tools to add extra information such as a digital signature to archive. + yield break; } } } - private Stream CheckSFX(Stream stream) + private RarHeader TryReadNextHeader(Stream stream) { - RewindableStream rewindableStream = GetRewindableStream(stream); - stream = rewindableStream; - BinaryReader reader = new BinaryReader(rewindableStream); - try + RarCrcBinaryReader reader; + if (!IsEncrypted) { - int count = 0; - while (true) - { - byte firstByte = reader.ReadByte(); - if (firstByte == 0x52) - { - MemoryStream buffer = new MemoryStream(); - byte[] nextThreeBytes = reader.ReadBytes(3); - if ((nextThreeBytes[0] == 0x45) - && (nextThreeBytes[1] == 0x7E) - && (nextThreeBytes[2] == 0x5E)) - { - //old format and isvalid - buffer.WriteByte(0x52); - buffer.Write(nextThreeBytes, 0, 3); - rewindableStream.Rewind(buffer); - break; - } - byte[] secondThreeBytes = reader.ReadBytes(3); - if ((nextThreeBytes[0] == 0x61) - && (nextThreeBytes[1] == 0x72) - && (nextThreeBytes[2] == 0x21) - && (secondThreeBytes[0] == 0x1A) - && (secondThreeBytes[1] == 0x07) - && (secondThreeBytes[2] == 0x00)) - { - //new format and isvalid - buffer.WriteByte(0x52); - buffer.Write(nextThreeBytes, 0, 3); - buffer.Write(secondThreeBytes, 0, 3); - rewindableStream.Rewind(buffer); - break; - } - buffer.Write(nextThreeBytes, 0, 3); - buffer.Write(secondThreeBytes, 0, 3); - rewindableStream.Rewind(buffer); - } - if (count > MAX_SFX_SIZE) - { - break; - } - } - } - catch (Exception e) + reader = new RarCrcBinaryReader(stream); + } + else { - if (!Options.LeaveStreamOpen) - { -#if NET35 - reader.Close(); -#else - reader.Dispose(); -#endif - } - throw new InvalidFormatException("Error trying to read rar signature.", e); - } - return stream; - } - - private RewindableStream GetRewindableStream(Stream stream) - { - RewindableStream rewindableStream = stream as RewindableStream; - if (rewindableStream == null) - { - rewindableStream = new RewindableStream(stream); - } - return rewindableStream; - } - - private RarHeader ReadNextHeader(Stream stream) - { #if !NO_CRYPTO - var reader = new RarCryptoBinaryReader(stream, Options.Password); - - if (IsEncrypted) - { if (Options.Password == null) { throw new CryptographicException("Encrypted Rar archive has no password specified."); } - reader.SkipQueue(); - byte[] salt = reader.ReadBytes(8); - reader.InitializeAes(salt); - } + reader = new RarCryptoBinaryReader(stream, Options.Password); #else - var reader = new RarCrcBinaryReader(stream); - + throw new CryptographicException("Rar encryption unsupported on this platform"); #endif + } - RarHeader header = RarHeader.Create(reader, Options.ArchiveEncoding); + var header = RarHeader.TryReadBase(reader, _isRar5, Options.ArchiveEncoding); if (header == null) { return null; } - switch (header.HeaderType) + switch (header.HeaderCode) { - case HeaderType.ArchiveHeader: + case HeaderCodeV.RAR5_ARCHIVE_HEADER: + case HeaderCodeV.RAR4_ARCHIVE_HEADER: { - var ah = header.PromoteHeader(reader); - IsEncrypted = ah.HasPassword; + var ah = new ArchiveHeader(header, reader); + if (ah.IsEncrypted == true) + { + //!!! rar5 we don't know yet + IsEncrypted = true; + } return ah; } - case HeaderType.MarkHeader: - { - return header.PromoteHeader(reader); - } - case HeaderType.ProtectHeader: + case HeaderCodeV.RAR4_PROTECT_HEADER: { - ProtectHeader ph = header.PromoteHeader(reader); - + var ph = new ProtectHeader(header, reader); // skip the recovery record data, we do not use it. switch (StreamingMode) { @@ -177,33 +103,24 @@ namespace SharpCompress.Common.Rar.Headers return ph; } - case HeaderType.NewSubHeader: + case HeaderCodeV.RAR5_SERVICE_HEADER: { - FileHeader fh = header.PromoteHeader(reader); - switch (StreamingMode) - { - case StreamingMode.Seekable: - { - fh.DataStartPosition = reader.BaseStream.Position; - reader.BaseStream.Position += fh.CompressedSize; - } - break; - case StreamingMode.Streaming: - { - //skip the data because it's useless? - reader.BaseStream.Skip(fh.CompressedSize); - } - break; - default: - { - throw new InvalidFormatException("Invalid StreamingMode"); - } - } + var fh = new FileHeader(header, reader, HeaderType.Service); + SkipData(fh, reader); return fh; } - case HeaderType.FileHeader: + + case HeaderCodeV.RAR4_NEW_SUB_HEADER: { - FileHeader fh = header.PromoteHeader(reader); + var fh = new FileHeader(header, reader, HeaderType.NewSub); + SkipData(fh, reader); + return fh; + } + + case HeaderCodeV.RAR5_FILE_HEADER: + case HeaderCodeV.RAR4_FILE_HEADER: + { + var fh = new FileHeader(header, reader, HeaderType.File); switch (StreamingMode) { case StreamingMode.Seekable: @@ -215,16 +132,16 @@ namespace SharpCompress.Common.Rar.Headers case StreamingMode.Streaming: { var ms = new ReadOnlySubStream(reader.BaseStream, fh.CompressedSize); - if (fh.Salt == null) + if (fh.R4Salt == null) { fh.PackedStream = ms; } else { #if !NO_CRYPTO - fh.PackedStream = new RarCryptoWrapper(ms, Options.Password, fh.Salt); + fh.PackedStream = new RarCryptoWrapper(ms, Options.Password, fh.R4Salt); #else - throw new NotSupportedException("RarCrypto not supported"); + throw new NotSupportedException("RarCrypto not supported"); #endif } } @@ -236,15 +153,40 @@ namespace SharpCompress.Common.Rar.Headers } return fh; } - case HeaderType.EndArchiveHeader: + case HeaderCodeV.RAR5_END_ARCHIVE_HEADER: + case HeaderCodeV.RAR4_END_ARCHIVE_HEADER: { - return header.PromoteHeader(reader); + return new EndArchiveHeader(header, reader); } + case HeaderCodeV.RAR5_ARCHIVE_ENCRYPTION_HEADER: + { + var ch = new ArchiveCryptHeader(header, reader); + IsEncrypted = true; + return ch; + } default: { - throw new InvalidFormatException("Invalid Rar Header: " + header.HeaderType); + throw new InvalidFormatException("Unknown Rar Header: " + header.HeaderCode); } } } + + private void SkipData(FileHeader fh, RarCrcBinaryReader reader) { + switch (StreamingMode) { + case StreamingMode.Seekable: { + fh.DataStartPosition = reader.BaseStream.Position; + reader.BaseStream.Position += fh.CompressedSize; + } + break; + case StreamingMode.Streaming: { + //skip the data because it's useless? + reader.BaseStream.Skip(fh.CompressedSize); + } + break; + default: { + throw new InvalidFormatException("Invalid StreamingMode"); + } + } + } } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/Headers/SignHeader.cs b/src/SharpCompress/Common/Rar/Headers/SignHeader.cs index 708241d8..58b3baef 100644 --- a/src/SharpCompress/Common/Rar/Headers/SignHeader.cs +++ b/src/SharpCompress/Common/Rar/Headers/SignHeader.cs @@ -2,22 +2,25 @@ namespace SharpCompress.Common.Rar.Headers { - internal class AVHeader : RarHeader + internal class SignHeader : RarHeader { - protected override void ReadFromReader(MarkingBinaryReader reader) - { - UnpackVersion = reader.ReadByte(); - Method = reader.ReadByte(); - AVVersion = reader.ReadByte(); - AVInfoCRC = reader.ReadInt32(); + protected SignHeader(RarHeader header, RarCrcBinaryReader reader) + : base(header, reader, HeaderType.Sign) + { + if (IsRar5) throw new InvalidFormatException("unexpected rar5 record"); } - internal int AVInfoCRC { get; private set; } + protected override void ReadFinish(MarkingBinaryReader reader) + { + CreationTime = reader.ReadInt32(); + ArcNameSize = reader.ReadInt16(); + UserNameSize = reader.ReadInt16(); + } - internal byte UnpackVersion { get; private set; } + internal int CreationTime { get; private set; } - internal byte Method { get; private set; } + internal short ArcNameSize { get; private set; } - internal byte AVVersion { get; private set; } + internal short UserNameSize { get; private set; } } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Rar/RarCrcBinaryReader.cs b/src/SharpCompress/Common/Rar/RarCrcBinaryReader.cs index 6df617b5..ddfd8b98 100644 --- a/src/SharpCompress/Common/Rar/RarCrcBinaryReader.cs +++ b/src/SharpCompress/Common/Rar/RarCrcBinaryReader.cs @@ -2,27 +2,30 @@ using System.IO; using SharpCompress.Compressors.Rar; using SharpCompress.IO; -namespace SharpCompress.Common.Rar { - internal class RarCrcBinaryReader : MarkingBinaryReader { - private uint currentCrc; +namespace SharpCompress.Common.Rar +{ + internal class RarCrcBinaryReader : MarkingBinaryReader + { + private uint _currentCrc; - public RarCrcBinaryReader(Stream stream) : base(stream) + public RarCrcBinaryReader(Stream stream) + : base(stream) { } - public ushort GetCrc() + public uint GetCrc32() { - return (ushort)~currentCrc; + return ~_currentCrc; } public void ResetCrc() { - currentCrc = 0xffffffff; + _currentCrc = 0xffffffff; } - protected void UpdateCrc(byte b) + protected void UpdateCrc(byte b) { - currentCrc = RarCRC.CheckCrc(currentCrc, b); + _currentCrc = RarCRC.CheckCrc(_currentCrc, b); } protected byte[] ReadBytesNoCrc(int count) @@ -30,10 +33,17 @@ namespace SharpCompress.Common.Rar { return base.ReadBytes(count); } + public override byte ReadByte() + { + var b = base.ReadByte(); + _currentCrc = RarCRC.CheckCrc(_currentCrc, b); + return b; + } + public override byte[] ReadBytes(int count) { var result = base.ReadBytes(count); - currentCrc = RarCRC.CheckCrc(currentCrc, result, 0, result.Length); + _currentCrc = RarCRC.CheckCrc(_currentCrc, result, 0, result.Length); return result; } } diff --git a/src/SharpCompress/Common/Rar/RarCryptoBinaryReader.cs b/src/SharpCompress/Common/Rar/RarCryptoBinaryReader.cs index 9635375b..66c0d606 100644 --- a/src/SharpCompress/Common/Rar/RarCryptoBinaryReader.cs +++ b/src/SharpCompress/Common/Rar/RarCryptoBinaryReader.cs @@ -1,50 +1,60 @@ - -#if !NO_CRYPTO +#if !NO_CRYPTO using System.Collections.Generic; using System.IO; -using SharpCompress.IO; namespace SharpCompress.Common.Rar { internal class RarCryptoBinaryReader : RarCrcBinaryReader { - private RarRijndael rijndael; - private byte[] salt; - private readonly string password; - private readonly Queue data = new Queue(); - private long readCount; + private RarRijndael _rijndael; + private byte[] _salt; + private readonly string _password; + private readonly Queue _data = new Queue(); + private long _readCount; - public RarCryptoBinaryReader(Stream stream, string password ) + public RarCryptoBinaryReader(Stream stream, string password) : base(stream) { - this.password = password; + _password = password; + + // coderb: not sure why this was being done at this logical point + //SkipQueue(); + byte[] salt = ReadBytes(8); + InitializeAes(salt); } // track read count ourselves rather than using the underlying stream since we buffer - public override long CurrentReadByteCount { - get - { - return this.readCount; - } - protected set + public override long CurrentReadByteCount + { + get => _readCount; + protected set { // ignore } } - public override void Mark() { - this.readCount = 0; - } - - protected bool UseEncryption + public override void Mark() { - get { return salt != null; } + _readCount = 0; } + private bool UseEncryption => _salt != null; + internal void InitializeAes(byte[] salt) { - this.salt = salt; - rijndael = RarRijndael.InitializeFrom(password, salt); + _salt = salt; + _rijndael = RarRijndael.InitializeFrom(_password, salt); + } + + public override byte ReadByte() + { + if (UseEncryption) + { + return ReadAndDecryptBytes(1)[0]; + } + + _readCount++; + return base.ReadByte(); } public override byte[] ReadBytes(int count) @@ -53,13 +63,14 @@ namespace SharpCompress.Common.Rar { return ReadAndDecryptBytes(count); } - this.readCount += count; + + _readCount += count; return base.ReadBytes(count); } private byte[] ReadAndDecryptBytes(int count) { - int queueSize = data.Count; + int queueSize = _data.Count; int sizeToRead = count - queueSize; if (sizeToRead > 0) @@ -68,36 +79,35 @@ namespace SharpCompress.Common.Rar for (int i = 0; i < alignedSize / 16; i++) { //long ax = System.currentTimeMillis(); - byte[] cipherText = base.ReadBytesNoCrc(16); - var readBytes = rijndael.ProcessBlock(cipherText); + byte[] cipherText = ReadBytesNoCrc(16); + var readBytes = _rijndael.ProcessBlock(cipherText); foreach (var readByte in readBytes) - data.Enqueue(readByte); - + _data.Enqueue(readByte); } - } var decryptedBytes = new byte[count]; for (int i = 0; i < count; i++) { - var b = data.Dequeue(); + var b = _data.Dequeue(); decryptedBytes[i] = b; UpdateCrc(b); } - this.readCount += count; + + _readCount += count; return decryptedBytes; } public void ClearQueue() { - data.Clear(); + _data.Clear(); } public void SkipQueue() { var position = BaseStream.Position; - BaseStream.Position = position + data.Count; + BaseStream.Position = position + _data.Count; ClearQueue(); } } diff --git a/src/SharpCompress/Common/Rar/RarCryptoWrapper.cs b/src/SharpCompress/Common/Rar/RarCryptoWrapper.cs index 2a14ae1a..50e1520c 100644 --- a/src/SharpCompress/Common/Rar/RarCryptoWrapper.cs +++ b/src/SharpCompress/Common/Rar/RarCryptoWrapper.cs @@ -8,16 +8,16 @@ namespace SharpCompress.Common.Rar { internal class RarCryptoWrapper : Stream { - private readonly Stream actualStream; - private readonly byte[] salt; - private RarRijndael rijndael; - private readonly Queue data = new Queue(); + private readonly Stream _actualStream; + private readonly byte[] _salt; + private RarRijndael _rijndael; + private readonly Queue _data = new Queue(); public RarCryptoWrapper(Stream actualStream, string password, byte[] salt) { - this.actualStream = actualStream; - this.salt = salt; - rijndael = RarRijndael.InitializeFrom(password, salt); + _actualStream = actualStream; + _salt = salt; + _rijndael = RarRijndael.InitializeFrom(password, salt); } public override void Flush() @@ -37,16 +37,16 @@ namespace SharpCompress.Common.Rar public override int Read(byte[] buffer, int offset, int count) { - if (salt == null) + if (_salt == null) { - return actualStream.Read(buffer, offset, count); + return _actualStream.Read(buffer, offset, count); } return ReadAndDecrypt(buffer, offset, count); } public int ReadAndDecrypt(byte[] buffer, int offset, int count) { - int queueSize = data.Count; + int queueSize = _data.Count; int sizeToRead = count - queueSize; if (sizeToRead > 0) @@ -56,16 +56,16 @@ namespace SharpCompress.Common.Rar { //long ax = System.currentTimeMillis(); byte[] cipherText = new byte[RarRijndael.CRYPTO_BLOCK_SIZE]; - actualStream.Read(cipherText, 0, RarRijndael.CRYPTO_BLOCK_SIZE); + _actualStream.Read(cipherText, 0, RarRijndael.CRYPTO_BLOCK_SIZE); - var readBytes = rijndael.ProcessBlock(cipherText); + var readBytes = _rijndael.ProcessBlock(cipherText); foreach (var readByte in readBytes) - data.Enqueue(readByte); + _data.Enqueue(readByte); } for (int i = 0; i < count; i++) - buffer[offset + i] = data.Dequeue(); + buffer[offset + i] = _data.Dequeue(); } return count; } @@ -75,34 +75,22 @@ namespace SharpCompress.Common.Rar throw new NotSupportedException(); } - public override bool CanRead - { - get { return true; } - } + public override bool CanRead => true; - public override bool CanSeek - { - get { return false; } - } + public override bool CanSeek => false; - public override bool CanWrite - { - get { return false; } - } + public override bool CanWrite => false; - public override long Length - { - get { throw new NotSupportedException(); } - } + public override long Length => throw new NotSupportedException(); public override long Position { get; set; } protected override void Dispose(bool disposing) { - if (rijndael != null) + if (_rijndael != null) { - rijndael.Dispose(); - rijndael = null; + _rijndael.Dispose(); + _rijndael = null; } base.Dispose(disposing); } diff --git a/src/SharpCompress/Common/Rar/RarEntry.cs b/src/SharpCompress/Common/Rar/RarEntry.cs index cddfca3e..85be0bb3 100644 --- a/src/SharpCompress/Common/Rar/RarEntry.cs +++ b/src/SharpCompress/Common/Rar/RarEntry.cs @@ -7,10 +7,15 @@ namespace SharpCompress.Common.Rar { internal abstract FileHeader FileHeader { get; } + /// + /// As the V2017 port isn't complete, add this check to use the legacy Rar code. + /// + internal bool IsRarV3 => FileHeader.CompressionAlgorithm == 29 || FileHeader.CompressionAlgorithm == 36; + /// /// The File's 32 bit CRC Hash /// - public override long Crc => FileHeader.FileCRC; + public override long Crc => FileHeader.FileCrc; /// /// The path of the file internal to the Rar Archive. @@ -40,14 +45,14 @@ namespace SharpCompress.Common.Rar /// /// Entry is password protected and encrypted and cannot be extracted. /// - public override bool IsEncrypted => FileHeader.FileFlags.HasFlag(FileFlags.PASSWORD); + public override bool IsEncrypted => FileHeader.IsEncrypted; /// /// Entry is password protected and encrypted and cannot be extracted. /// - public override bool IsDirectory => FileHeader.FileFlags.HasFlag(FileFlags.DIRECTORY); + public override bool IsDirectory => FileHeader.IsDirectory; - public override bool IsSplit => FileHeader.FileFlags.HasFlag(FileFlags.SPLIT_AFTER); + public override bool IsSplitAfter => FileHeader.IsSplitAfter; public override string ToString() { diff --git a/src/SharpCompress/Common/Rar/RarRijndael.cs b/src/SharpCompress/Common/Rar/RarRijndael.cs index b2ce22d6..824cb64e 100644 --- a/src/SharpCompress/Common/Rar/RarRijndael.cs +++ b/src/SharpCompress/Common/Rar/RarRijndael.cs @@ -4,8 +4,7 @@ using System.Collections.Generic; using System.Linq; using System.Security.Cryptography; using System.Text; -using Org.BouncyCastle.Crypto.Engines; -using Org.BouncyCastle.Crypto.Parameters; +using SharpCompress.Crypto; namespace SharpCompress.Common.Rar { @@ -13,15 +12,15 @@ namespace SharpCompress.Common.Rar { internal const int CRYPTO_BLOCK_SIZE = 16; - private readonly string password; - private readonly byte[] salt; - private byte[] aesInitializationVector; - private RijndaelEngine rijndael; + private readonly string _password; + private readonly byte[] _salt; + private byte[] _aesInitializationVector; + private RijndaelEngine _rijndael; private RarRijndael(string password, byte[] salt) { - this.password = password; - this.salt = salt; + _password = password; + _salt = salt; } private byte[] ComputeHash(byte[] input) @@ -33,19 +32,19 @@ namespace SharpCompress.Common.Rar private void Initialize() { - rijndael = new RijndaelEngine(); - aesInitializationVector = new byte[CRYPTO_BLOCK_SIZE]; - int rawLength = 2*password.Length; + _rijndael = new RijndaelEngine(); + _aesInitializationVector = new byte[CRYPTO_BLOCK_SIZE]; + int rawLength = 2*_password.Length; byte[] rawPassword = new byte[rawLength + 8]; - byte[] passwordBytes = Encoding.UTF8.GetBytes(password); - for (int i = 0; i < password.Length; i++) + byte[] passwordBytes = Encoding.UTF8.GetBytes(_password); + for (int i = 0; i < _password.Length; i++) { rawPassword[i*2] = passwordBytes[i]; rawPassword[i*2 + 1] = 0; } - for (int i = 0; i < salt.Length; i++) + for (int i = 0; i < _salt.Length; i++) { - rawPassword[i + rawLength] = salt[i]; + rawPassword[i + rawLength] = _salt[i]; } @@ -65,7 +64,7 @@ namespace SharpCompress.Common.Rar if (i%(noOfRounds/CRYPTO_BLOCK_SIZE) == 0) { digest = ComputeHash(bytes.ToArray()); - aesInitializationVector[i/(noOfRounds/CRYPTO_BLOCK_SIZE)] = digest[19]; + _aesInitializationVector[i/(noOfRounds/CRYPTO_BLOCK_SIZE)] = digest[19]; } } @@ -85,7 +84,7 @@ namespace SharpCompress.Common.Rar } } - rijndael.Init(false, new KeyParameter(aesKey)); + _rijndael.Init(false, new KeyParameter(aesKey)); } @@ -100,16 +99,16 @@ namespace SharpCompress.Common.Rar { var plainText = new byte[CRYPTO_BLOCK_SIZE]; var decryptedBytes = new List(); - rijndael.ProcessBlock(cipherText, 0, plainText, 0); + _rijndael.ProcessBlock(cipherText, 0, plainText, 0); for (int j = 0; j < plainText.Length; j++) { - decryptedBytes.Add((byte) (plainText[j] ^ aesInitializationVector[j%16])); //32:114, 33:101 + decryptedBytes.Add((byte) (plainText[j] ^ _aesInitializationVector[j%16])); //32:114, 33:101 } - for (int j = 0; j < aesInitializationVector.Length; j++) + for (int j = 0; j < _aesInitializationVector.Length; j++) { - aesInitializationVector[j] = cipherText[j]; + _aesInitializationVector[j] = cipherText[j]; } return decryptedBytes.ToArray(); } diff --git a/src/SharpCompress/Common/Rar/RarVolume.cs b/src/SharpCompress/Common/Rar/RarVolume.cs index 89fee688..265db11c 100644 --- a/src/SharpCompress/Common/Rar/RarVolume.cs +++ b/src/SharpCompress/Common/Rar/RarVolume.cs @@ -13,50 +13,49 @@ namespace SharpCompress.Common.Rar /// public abstract class RarVolume : Volume { - private readonly RarHeaderFactory headerFactory; + private readonly RarHeaderFactory _headerFactory; internal RarVolume(StreamingMode mode, Stream stream, ReaderOptions options) : base(stream, options) { - headerFactory = new RarHeaderFactory(mode, options); + _headerFactory = new RarHeaderFactory(mode, options); } - internal StreamingMode Mode => headerFactory.StreamingMode; + internal ArchiveHeader ArchiveHeader { get; private set; } + + internal StreamingMode Mode => _headerFactory.StreamingMode; internal abstract IEnumerable ReadFileParts(); - internal abstract RarFilePart CreateFilePart(FileHeader fileHeader, MarkHeader markHeader); + internal abstract RarFilePart CreateFilePart(MarkHeader markHeader, FileHeader fileHeader); internal IEnumerable GetVolumeFileParts() { - MarkHeader previousMarkHeader = null; - foreach (RarHeader header in headerFactory.ReadHeaders(Stream)) + MarkHeader lastMarkHeader = null; + foreach (var header in _headerFactory.ReadHeaders(Stream)) { switch (header.HeaderType) { - case HeaderType.ArchiveHeader: + case HeaderType.Mark: + { + lastMarkHeader = header as MarkHeader; + } + break; + case HeaderType.Archive: { ArchiveHeader = header as ArchiveHeader; } break; - case HeaderType.MarkHeader: + case HeaderType.File: { - previousMarkHeader = header as MarkHeader; - } - break; - case HeaderType.FileHeader: - { - FileHeader fh = header as FileHeader; - RarFilePart fp = CreateFilePart(fh, previousMarkHeader); - yield return fp; + var fh = header as FileHeader; + yield return CreateFilePart(lastMarkHeader, fh); } break; } } } - internal ArchiveHeader ArchiveHeader { get; private set; } - private void EnsureArchiveHeaderLoaded() { if (ArchiveHeader == null) @@ -66,7 +65,7 @@ namespace SharpCompress.Common.Rar throw new InvalidOperationException("ArchiveHeader should never been null in a streaming read."); } - //we only want to load the archive header to avoid overhead but have to do the nasty thing and reset the stream + // we only want to load the archive header to avoid overhead but have to do the nasty thing and reset the stream GetVolumeFileParts().First(); Stream.Position = 0; } @@ -81,7 +80,7 @@ namespace SharpCompress.Common.Rar get { EnsureArchiveHeaderLoaded(); - return ArchiveHeader.ArchiveHeaderFlags.HasFlag(ArchiveFlags.FIRSTVOLUME); + return ArchiveHeader.IsFirstVolume; } } @@ -93,7 +92,7 @@ namespace SharpCompress.Common.Rar get { EnsureArchiveHeaderLoaded(); - return ArchiveHeader.ArchiveHeaderFlags.HasFlag(ArchiveFlags.VOLUME); + return ArchiveHeader.IsVolume; } } @@ -106,7 +105,7 @@ namespace SharpCompress.Common.Rar get { EnsureArchiveHeaderLoaded(); - return ArchiveHeader.ArchiveHeaderFlags.HasFlag(ArchiveFlags.SOLID); + return ArchiveHeader.IsSolid; } } } diff --git a/src/SharpCompress/Common/SevenZip/ArchiveDatabase.cs b/src/SharpCompress/Common/SevenZip/ArchiveDatabase.cs index fedd7782..e827eb20 100644 --- a/src/SharpCompress/Common/SevenZip/ArchiveDatabase.cs +++ b/src/SharpCompress/Common/SevenZip/ArchiveDatabase.cs @@ -8,71 +8,78 @@ namespace SharpCompress.Common.SevenZip { internal class ArchiveDatabase { - internal byte MajorVersion; - internal byte MinorVersion; - internal long StartPositionAfterHeader; - internal long DataStartPosition; + internal byte _majorVersion; + internal byte _minorVersion; + internal long _startPositionAfterHeader; + internal long _dataStartPosition; - internal List PackSizes = new List(); - internal List PackCRCs = new List(); - internal List Folders = new List(); - internal List NumUnpackStreamsVector; - internal List Files = new List(); + internal List _packSizes = new List(); + internal List _packCrCs = new List(); + internal List _folders = new List(); + internal List _numUnpackStreamsVector; + internal List _files = new List(); - internal List PackStreamStartPositions = new List(); - internal List FolderStartFileIndex = new List(); - internal List FileIndexToFolderIndexMap = new List(); + internal List _packStreamStartPositions = new List(); + internal List _folderStartFileIndex = new List(); + internal List _fileIndexToFolderIndexMap = new List(); + + internal IPasswordProvider PasswordProvider { get; } + + public ArchiveDatabase(IPasswordProvider passwordProvider) + { + PasswordProvider = passwordProvider; + } internal void Clear() { - PackSizes.Clear(); - PackCRCs.Clear(); - Folders.Clear(); - NumUnpackStreamsVector = null; - Files.Clear(); + _packSizes.Clear(); + _packCrCs.Clear(); + _folders.Clear(); + _numUnpackStreamsVector = null; + _files.Clear(); - PackStreamStartPositions.Clear(); - FolderStartFileIndex.Clear(); - FileIndexToFolderIndexMap.Clear(); + _packStreamStartPositions.Clear(); + _folderStartFileIndex.Clear(); + _fileIndexToFolderIndexMap.Clear(); } internal bool IsEmpty() { - return PackSizes.Count == 0 - && PackCRCs.Count == 0 - && Folders.Count == 0 - && NumUnpackStreamsVector.Count == 0 - && Files.Count == 0; + return _packSizes.Count == 0 + && _packCrCs.Count == 0 + && _folders.Count == 0 + && _numUnpackStreamsVector.Count == 0 + && _files.Count == 0; } private void FillStartPos() { - PackStreamStartPositions.Clear(); + _packStreamStartPositions.Clear(); long startPos = 0; - for (int i = 0; i < PackSizes.Count; i++) + for (int i = 0; i < _packSizes.Count; i++) { - PackStreamStartPositions.Add(startPos); - startPos += PackSizes[i]; + _packStreamStartPositions.Add(startPos); + startPos += _packSizes[i]; } } private void FillFolderStartFileIndex() { - FolderStartFileIndex.Clear(); - FileIndexToFolderIndexMap.Clear(); + _folderStartFileIndex.Clear(); + _fileIndexToFolderIndexMap.Clear(); int folderIndex = 0; int indexInFolder = 0; - for (int i = 0; i < Files.Count; i++) + for (int i = 0; i < _files.Count; i++) { - CFileItem file = Files[i]; + CFileItem file = _files[i]; bool emptyStream = !file.HasStream; if (emptyStream && indexInFolder == 0) { - FileIndexToFolderIndexMap.Add(-1); + _fileIndexToFolderIndexMap.Add(-1); continue; } @@ -82,14 +89,14 @@ namespace SharpCompress.Common.SevenZip // v4.07: Loop for skipping empty folders for (;;) { - if (folderIndex >= Folders.Count) + if (folderIndex >= _folders.Count) { throw new InvalidOperationException(); } - FolderStartFileIndex.Add(i); // check it + _folderStartFileIndex.Add(i); // check it - if (NumUnpackStreamsVector[folderIndex] != 0) + if (_numUnpackStreamsVector[folderIndex] != 0) { break; } @@ -98,7 +105,7 @@ namespace SharpCompress.Common.SevenZip } } - FileIndexToFolderIndexMap.Add(folderIndex); + _fileIndexToFolderIndexMap.Add(folderIndex); if (emptyStream) { @@ -107,7 +114,7 @@ namespace SharpCompress.Common.SevenZip indexInFolder++; - if (indexInFolder >= NumUnpackStreamsVector[folderIndex]) + if (indexInFolder >= _numUnpackStreamsVector[folderIndex]) { folderIndex++; indexInFolder = 0; @@ -123,19 +130,19 @@ namespace SharpCompress.Common.SevenZip internal long GetFolderStreamPos(CFolder folder, int indexInFolder) { - int index = folder.FirstPackStreamId + indexInFolder; - return DataStartPosition + PackStreamStartPositions[index]; + int index = folder._firstPackStreamId + indexInFolder; + return _dataStartPosition + _packStreamStartPositions[index]; } internal long GetFolderFullPackSize(int folderIndex) { - int packStreamIndex = Folders[folderIndex].FirstPackStreamId; - CFolder folder = Folders[folderIndex]; + int packStreamIndex = _folders[folderIndex]._firstPackStreamId; + CFolder folder = _folders[folderIndex]; long size = 0; - for (int i = 0; i < folder.PackStreams.Count; i++) + for (int i = 0; i < folder._packStreams.Count; i++) { - size += PackSizes[packStreamIndex + i]; + size += _packSizes[packStreamIndex + i]; } return size; @@ -143,12 +150,12 @@ namespace SharpCompress.Common.SevenZip internal Stream GetFolderStream(Stream stream, CFolder folder, IPasswordProvider pw) { - int packStreamIndex = folder.FirstPackStreamId; + int packStreamIndex = folder._firstPackStreamId; long folderStartPackPos = GetFolderStreamPos(folder, 0); List packSizes = new List(); - for (int j = 0; j < folder.PackStreams.Count; j++) + for (int j = 0; j < folder._packStreams.Count; j++) { - packSizes.Add(PackSizes[packStreamIndex + j]); + packSizes.Add(_packSizes[packStreamIndex + j]); } return DecoderStreamHelper.CreateDecoderStream(stream, folderStartPackPos, packSizes.ToArray(), folder, pw); @@ -156,15 +163,15 @@ namespace SharpCompress.Common.SevenZip private long GetFolderPackStreamSize(int folderIndex, int streamIndex) { - return PackSizes[Folders[folderIndex].FirstPackStreamId + streamIndex]; + return _packSizes[_folders[folderIndex]._firstPackStreamId + streamIndex]; } private long GetFilePackSize(int fileIndex) { - int folderIndex = FileIndexToFolderIndexMap[fileIndex]; + int folderIndex = _fileIndexToFolderIndexMap[fileIndex]; if (folderIndex != -1) { - if (FolderStartFileIndex[folderIndex] == fileIndex) + if (_folderStartFileIndex[folderIndex] == fileIndex) { return GetFolderFullPackSize(folderIndex); } diff --git a/src/SharpCompress/Common/SevenZip/ArchiveReader.cs b/src/SharpCompress/Common/SevenZip/ArchiveReader.cs index 64a18b8b..2093a831 100644 --- a/src/SharpCompress/Common/SevenZip/ArchiveReader.cs +++ b/src/SharpCompress/Common/SevenZip/ArchiveReader.cs @@ -182,7 +182,7 @@ namespace SharpCompress.Common.SevenZip private DateTime? TranslateTime(long? time) { - if (time.HasValue) + if (time.HasValue && time.Value >= 0 && time.Value <= 2650467743999999999) //maximum Windows file time 31.12.9999 { return TranslateTime(time.Value); } @@ -230,7 +230,7 @@ namespace SharpCompress.Common.SevenZip #if DEBUG Log.WriteLine("NumCoders: " + numCoders); #endif - folder.Coders = new List(numCoders); + folder._coders = new List(numCoders); int numInStreams = 0; int numOutStreams = 0; for (int i = 0; i < numCoders; i++) @@ -242,14 +242,14 @@ namespace SharpCompress.Common.SevenZip try { CCoderInfo coder = new CCoderInfo(); - folder.Coders.Add(coder); + folder._coders.Add(coder); byte mainByte = ReadByte(); int idSize = (mainByte & 0xF); - byte[] longID = new byte[idSize]; - ReadBytes(longID, 0, idSize); + byte[] longId = new byte[idSize]; + ReadBytes(longId, 0, idSize); #if DEBUG - Log.WriteLine("MethodId: " + String.Join("", Enumerable.Range(0, idSize).Select(x => longID[x].ToString("x2")).ToArray())); + Log.WriteLine("MethodId: " + String.Join("", Enumerable.Range(0, idSize).Select(x => longId[x].ToString("x2")).ToArray())); #endif if (idSize > 8) { @@ -258,16 +258,16 @@ namespace SharpCompress.Common.SevenZip ulong id = 0; for (int j = 0; j < idSize; j++) { - id |= (ulong)longID[idSize - 1 - j] << (8 * j); + id |= (ulong)longId[idSize - 1 - j] << (8 * j); } - coder.MethodId = new CMethodId(id); + coder._methodId = new CMethodId(id); if ((mainByte & 0x10) != 0) { - coder.NumInStreams = ReadNum(); - coder.NumOutStreams = ReadNum(); + coder._numInStreams = ReadNum(); + coder._numOutStreams = ReadNum(); #if DEBUG - Log.WriteLine("Complex Stream (In: " + coder.NumInStreams + " - Out: " + coder.NumOutStreams + ")"); + Log.WriteLine("Complex Stream (In: " + coder._numInStreams + " - Out: " + coder._numOutStreams + ")"); #endif } else @@ -275,17 +275,17 @@ namespace SharpCompress.Common.SevenZip #if DEBUG Log.WriteLine("Simple Stream (In: 1 - Out: 1)"); #endif - coder.NumInStreams = 1; - coder.NumOutStreams = 1; + coder._numInStreams = 1; + coder._numOutStreams = 1; } if ((mainByte & 0x20) != 0) { int propsSize = ReadNum(); - coder.Props = new byte[propsSize]; - ReadBytes(coder.Props, 0, propsSize); + coder._props = new byte[propsSize]; + ReadBytes(coder._props, 0, propsSize); #if DEBUG - Log.WriteLine("Settings: " + String.Join("", coder.Props.Select(bt => bt.ToString("x2")).ToArray())); + Log.WriteLine("Settings: " + String.Join("", coder._props.Select(bt => bt.ToString("x2")).ToArray())); #endif } @@ -294,8 +294,8 @@ namespace SharpCompress.Common.SevenZip throw new NotSupportedException(); } - numInStreams += coder.NumInStreams; - numOutStreams += coder.NumOutStreams; + numInStreams += coder._numInStreams; + numOutStreams += coder._numOutStreams; } finally { @@ -306,7 +306,7 @@ namespace SharpCompress.Common.SevenZip } int numBindPairs = numOutStreams - 1; - folder.BindPairs = new List(numBindPairs); + folder._bindPairs = new List(numBindPairs); #if DEBUG Log.WriteLine("BindPairs: " + numBindPairs); Log.PushIndent(); @@ -314,11 +314,11 @@ namespace SharpCompress.Common.SevenZip for (int i = 0; i < numBindPairs; i++) { CBindPair bp = new CBindPair(); - bp.InIndex = ReadNum(); - bp.OutIndex = ReadNum(); - folder.BindPairs.Add(bp); + bp._inIndex = ReadNum(); + bp._outIndex = ReadNum(); + folder._bindPairs.Add(bp); #if DEBUG - Log.WriteLine("#" + i + " - In: " + bp.InIndex + " - Out: " + bp.OutIndex); + Log.WriteLine("#" + i + " - In: " + bp._inIndex + " - Out: " + bp._outIndex); #endif } #if DEBUG @@ -342,12 +342,12 @@ namespace SharpCompress.Common.SevenZip #if DEBUG Log.WriteLine("Single PackStream: #" + i); #endif - folder.PackStreams.Add(i); + folder._packStreams.Add(i); break; } } - if (folder.PackStreams.Count != 1) + if (folder._packStreams.Count != 1) { throw new NotSupportedException(); } @@ -364,7 +364,7 @@ namespace SharpCompress.Common.SevenZip #if DEBUG Log.WriteLine("#" + i + " - " + num); #endif - folder.PackStreams.Add(num); + folder._packStreams.Add(num); } #if DEBUG Log.PopIndent(); @@ -412,7 +412,7 @@ namespace SharpCompress.Common.SevenZip return digests; } - private void ReadPackInfo(out long dataOffset, out List packSizes, out List packCRCs) + private void ReadPackInfo(out long dataOffset, out List packSizes, out List packCrCs) { #if DEBUG Log.WriteLine("-- ReadPackInfo --"); @@ -420,7 +420,7 @@ namespace SharpCompress.Common.SevenZip #endif try { - packCRCs = null; + packCrCs = null; dataOffset = checked((long)ReadNumber()); #if DEBUG @@ -457,20 +457,20 @@ namespace SharpCompress.Common.SevenZip { break; } - if (type == BlockType.CRC) + if (type == BlockType.Crc) { - packCRCs = ReadHashDigests(numPackStreams); + packCrCs = ReadHashDigests(numPackStreams); continue; } SkipData(); } - if (packCRCs == null) + if (packCrCs == null) { - packCRCs = new List(numPackStreams); + packCrCs = new List(numPackStreams); for (int i = 0; i < numPackStreams; i++) { - packCRCs.Add(null); + packCrCs.Add(null); } } } @@ -506,10 +506,10 @@ namespace SharpCompress.Common.SevenZip int index = 0; for (int i = 0; i < numFolders; i++) { - var f = new CFolder {FirstPackStreamId = index}; + var f = new CFolder {_firstPackStreamId = index}; folders.Add(f); GetNextFolderItem(f); - index += f.PackStreams.Count; + index += f._packStreams.Count; } } @@ -530,7 +530,7 @@ namespace SharpCompress.Common.SevenZip #if DEBUG Log.Write(" " + size); #endif - folder.UnpackSizes.Add(size); + folder._unpackSizes.Add(size); } #if DEBUG Log.WriteLine(); @@ -545,12 +545,12 @@ namespace SharpCompress.Common.SevenZip return; } - if (type == BlockType.CRC) + if (type == BlockType.Crc) { List crcs = ReadHashDigests(numFolders); for (int i = 0; i < numFolders; i++) { - folders[i].UnpackCRC = crcs[i]; + folders[i]._unpackCrc = crcs[i]; } continue; } @@ -600,7 +600,7 @@ namespace SharpCompress.Common.SevenZip #endif continue; } - if (type == BlockType.CRC || type == BlockType.Size) + if (type == BlockType.Crc || type == BlockType.Size) { break; } @@ -661,7 +661,7 @@ namespace SharpCompress.Common.SevenZip for (int i = 0; i < folders.Count; i++) { int numSubstreams = numUnpackStreamsInFolders[i]; - if (numSubstreams != 1 || !folders[i].UnpackCRCDefined) + if (numSubstreams != 1 || !folders[i].UnpackCrcDefined) { numDigests += numSubstreams; } @@ -672,7 +672,7 @@ namespace SharpCompress.Common.SevenZip for (;;) { - if (type == BlockType.CRC) + if (type == BlockType.Crc) { digests = new List(numDigestsTotal); @@ -683,9 +683,9 @@ namespace SharpCompress.Common.SevenZip { int numSubstreams = numUnpackStreamsInFolders[i]; CFolder folder = folders[i]; - if (numSubstreams == 1 && folder.UnpackCRCDefined) + if (numSubstreams == 1 && folder.UnpackCrcDefined) { - digests.Add(folder.UnpackCRC.Value); + digests.Add(folder._unpackCrc.Value); } else { @@ -733,7 +733,7 @@ namespace SharpCompress.Common.SevenZip List dataVector, out long dataOffset, out List packSizes, - out List packCRCs, + out List packCrCs, out List folders, out List numUnpackStreamsInFolders, out List unpackSizes, @@ -747,7 +747,7 @@ namespace SharpCompress.Common.SevenZip { dataOffset = long.MinValue; packSizes = null; - packCRCs = null; + packCrCs = null; folders = null; numUnpackStreamsInFolders = null; unpackSizes = null; @@ -760,7 +760,7 @@ namespace SharpCompress.Common.SevenZip case BlockType.End: return; case BlockType.PackInfo: - ReadPackInfo(out dataOffset, out packSizes, out packCRCs); + ReadPackInfo(out dataOffset, out packSizes, out packCrCs); break; case BlockType.UnpackInfo: ReadUnpackInfo(dataVector, out folders); @@ -791,7 +791,7 @@ namespace SharpCompress.Common.SevenZip { long dataStartPos; List packSizes; - List packCRCs; + List packCrCs; List folders; List numUnpackStreamsInFolders; List unpackSizes; @@ -800,7 +800,7 @@ namespace SharpCompress.Common.SevenZip ReadStreamsInfo(null, out dataStartPos, out packSizes, - out packCRCs, + out packCrCs, out folders, out numUnpackStreamsInFolders, out unpackSizes, @@ -813,7 +813,7 @@ namespace SharpCompress.Common.SevenZip foreach (var folder in folders) { long oldDataStartPos = dataStartPos; - long[] myPackSizes = new long[folder.PackStreams.Count]; + long[] myPackSizes = new long[folder._packStreams.Count]; for (int i = 0; i < myPackSizes.Length; i++) { long packSize = packSizes[packIndex + i]; @@ -833,9 +833,9 @@ namespace SharpCompress.Common.SevenZip } dataVector.Add(data); - if (folder.UnpackCRCDefined) + if (folder.UnpackCrcDefined) { - if (CRC.Finish(CRC.Update(CRC.kInitCRC, data, 0, unpackSize)) != folder.UnpackCRC) + if (Crc.Finish(Crc.Update(Crc.INIT_CRC, data, 0, unpackSize)) != folder._unpackCrc) { throw new InvalidOperationException("Decoded stream does not match expected CRC."); } @@ -870,7 +870,7 @@ namespace SharpCompress.Common.SevenZip List dataVector = null; if (type == BlockType.AdditionalStreamsInfo) { - dataVector = ReadAndDecodePackedStreams(db.StartPositionAfterHeader, getTextPassword); + dataVector = ReadAndDecodePackedStreams(db._startPositionAfterHeader, getTextPassword); type = ReadId(); } @@ -880,32 +880,32 @@ namespace SharpCompress.Common.SevenZip if (type == BlockType.MainStreamsInfo) { ReadStreamsInfo(dataVector, - out db.DataStartPosition, - out db.PackSizes, - out db.PackCRCs, - out db.Folders, - out db.NumUnpackStreamsVector, + out db._dataStartPosition, + out db._packSizes, + out db._packCrCs, + out db._folders, + out db._numUnpackStreamsVector, out unpackSizes, out digests); - db.DataStartPosition += db.StartPositionAfterHeader; + db._dataStartPosition += db._startPositionAfterHeader; type = ReadId(); } else { - unpackSizes = new List(db.Folders.Count); - digests = new List(db.Folders.Count); - db.NumUnpackStreamsVector = new List(db.Folders.Count); - for (int i = 0; i < db.Folders.Count; i++) + unpackSizes = new List(db._folders.Count); + digests = new List(db._folders.Count); + db._numUnpackStreamsVector = new List(db._folders.Count); + for (int i = 0; i < db._folders.Count; i++) { - var folder = db.Folders[i]; + var folder = db._folders[i]; unpackSizes.Add(folder.GetUnpackSize()); - digests.Add(folder.UnpackCRC); - db.NumUnpackStreamsVector.Add(1); + digests.Add(folder._unpackCrc); + db._numUnpackStreamsVector.Add(1); } } - db.Files.Clear(); + db._files.Clear(); if (type == BlockType.End) { @@ -921,10 +921,10 @@ namespace SharpCompress.Common.SevenZip #if DEBUG Log.WriteLine("NumFiles: " + numFiles); #endif - db.Files = new List(numFiles); + db._files = new List(numFiles); for (int i = 0; i < numFiles; i++) { - db.Files.Add(new CFileItem()); + db._files.Add(new CFileItem()); } BitVector emptyStreamVector = new BitVector(numFiles); @@ -951,11 +951,11 @@ namespace SharpCompress.Common.SevenZip #if DEBUG Log.Write("FileNames:"); #endif - for (int i = 0; i < db.Files.Count; i++) + for (int i = 0; i < db._files.Count; i++) { - db.Files[i].Name = _currentReader.ReadString(); + db._files[i].Name = _currentReader.ReadString(); #if DEBUG - Log.Write(" " + db.Files[i].Name); + Log.Write(" " + db._files[i].Name); #endif } #if DEBUG @@ -991,7 +991,7 @@ namespace SharpCompress.Common.SevenZip attr = attr.Value & 0x7FFFu; } - db.Files[i].Attrib = attr; + db._files[i].Attrib = attr; #if DEBUG Log.Write(" " + (attr.HasValue ? attr.Value.ToString("x8") : "n/a")); #endif @@ -1057,7 +1057,7 @@ namespace SharpCompress.Common.SevenZip #endif ReadNumberVector(dataVector, numFiles, delegate(int i, long? startPos) { - db.Files[i].StartPos = startPos; + db._files[i].StartPos = startPos; #if DEBUG Log.Write(" " + (startPos.HasValue ? startPos.Value.ToString() : "n/a")); #endif @@ -1072,7 +1072,7 @@ namespace SharpCompress.Common.SevenZip #endif ReadDateTimeVector(dataVector, numFiles, delegate(int i, DateTime? time) { - db.Files[i].CTime = time; + db._files[i].CTime = time; #if DEBUG Log.Write(" " + (time.HasValue ? time.Value.ToString() : "n/a")); #endif @@ -1087,7 +1087,7 @@ namespace SharpCompress.Common.SevenZip #endif ReadDateTimeVector(dataVector, numFiles, delegate(int i, DateTime? time) { - db.Files[i].ATime = time; + db._files[i].ATime = time; #if DEBUG Log.Write(" " + (time.HasValue ? time.Value.ToString() : "n/a")); #endif @@ -1102,7 +1102,7 @@ namespace SharpCompress.Common.SevenZip #endif ReadDateTimeVector(dataVector, numFiles, delegate(int i, DateTime? time) { - db.Files[i].MTime = time; + db._files[i].MTime = time; #if DEBUG Log.Write(" " + (time.HasValue ? time.Value.ToString() : "n/a")); #endif @@ -1129,7 +1129,7 @@ namespace SharpCompress.Common.SevenZip } // since 0.3 record sizes must be correct - bool checkRecordsSize = (db.MajorVersion > 0 || db.MinorVersion > 2); + bool checkRecordsSize = (db._majorVersion > 0 || db._minorVersion > 2); if (checkRecordsSize && _currentReader.Offset - oldPos != size) { throw new InvalidOperationException(); @@ -1140,7 +1140,7 @@ namespace SharpCompress.Common.SevenZip int sizeIndex = 0; for (int i = 0; i < numFiles; i++) { - CFileItem file = db.Files[i]; + CFileItem file = db._files[i]; file.HasStream = !emptyStreamVector[i]; if (file.HasStream) { @@ -1211,13 +1211,13 @@ namespace SharpCompress.Common.SevenZip public ArchiveDatabase ReadDatabase(IPasswordProvider pass) { - var db = new ArchiveDatabase(); + var db = new ArchiveDatabase(pass); db.Clear(); - db.MajorVersion = _header[6]; - db.MinorVersion = _header[7]; + db._majorVersion = _header[6]; + db._minorVersion = _header[7]; - if (db.MajorVersion != 0) + if (db._majorVersion != 0) { throw new InvalidOperationException(); } @@ -1227,18 +1227,18 @@ namespace SharpCompress.Common.SevenZip long nextHeaderSize = (long)DataReader.Get64(_header, 0x14); uint nextHeaderCrc = DataReader.Get32(_header, 0x1C); - uint crc = CRC.kInitCRC; - crc = CRC.Update(crc, nextHeaderOffset); - crc = CRC.Update(crc, nextHeaderSize); - crc = CRC.Update(crc, nextHeaderCrc); - crc = CRC.Finish(crc); + uint crc = Crc.INIT_CRC; + crc = Crc.Update(crc, nextHeaderOffset); + crc = Crc.Update(crc, nextHeaderSize); + crc = Crc.Update(crc, nextHeaderCrc); + crc = Crc.Finish(crc); if (crc != crcFromArchive) { throw new InvalidOperationException(); } - db.StartPositionAfterHeader = _streamOrigin + 0x20; + db._startPositionAfterHeader = _streamOrigin + 0x20; // empty header is ok if (nextHeaderSize == 0) @@ -1252,7 +1252,7 @@ namespace SharpCompress.Common.SevenZip throw new InvalidOperationException(); } - if (nextHeaderOffset > _streamEnding - db.StartPositionAfterHeader) + if (nextHeaderOffset > _streamEnding - db._startPositionAfterHeader) { throw new IndexOutOfRangeException(); } @@ -1262,7 +1262,7 @@ namespace SharpCompress.Common.SevenZip byte[] header = new byte[nextHeaderSize]; _stream.ReadExact(header, 0, header.Length); - if (CRC.Finish(CRC.Update(CRC.kInitCRC, header, 0, header.Length)) != nextHeaderCrc) + if (Crc.Finish(Crc.Update(Crc.INIT_CRC, header, 0, header.Length)) != nextHeaderCrc) { throw new InvalidOperationException(); } @@ -1279,7 +1279,7 @@ namespace SharpCompress.Common.SevenZip throw new InvalidOperationException(); } - var dataVector = ReadAndDecodePackedStreams(db.StartPositionAfterHeader, pass); + var dataVector = ReadAndDecodePackedStreams(db._startPositionAfterHeader, db.PasswordProvider); // compressed header without content is odd but ok if (dataVector.Count == 0) @@ -1301,7 +1301,7 @@ namespace SharpCompress.Common.SevenZip } } - ReadHeader(db, pass); + ReadHeader(db, db.PasswordProvider); } db.Fill(); return db; @@ -1309,17 +1309,17 @@ namespace SharpCompress.Common.SevenZip internal class CExtractFolderInfo { - internal int FileIndex; - internal int FolderIndex; - internal List ExtractStatuses = new List(); + internal int _fileIndex; + internal int _folderIndex; + internal List _extractStatuses = new List(); internal CExtractFolderInfo(int fileIndex, int folderIndex) { - FileIndex = fileIndex; - FolderIndex = folderIndex; + _fileIndex = fileIndex; + _folderIndex = folderIndex; if (fileIndex != -1) { - ExtractStatuses.Add(true); + _extractStatuses.Add(true); } } } @@ -1375,7 +1375,7 @@ namespace SharpCompress.Common.SevenZip private void ProcessEmptyFiles() { - while (_currentIndex < _extractStatuses.Count && _db.Files[_startIndex + _currentIndex].Size == 0) + while (_currentIndex < _extractStatuses.Count && _db._files[_startIndex + _currentIndex].Size == 0) { OpenFile(); _stream.Dispose(); @@ -1388,17 +1388,17 @@ namespace SharpCompress.Common.SevenZip { int index = _startIndex + _currentIndex; #if DEBUG - Log.WriteLine(_db.Files[index].Name); + Log.WriteLine(_db._files[index].Name); #endif - if (_db.Files[index].CrcDefined) + if (_db._files[index].CrcDefined) { - _stream = new CrcCheckStream(_db.Files[index].Crc.Value); + _stream = new CrcCheckStream(_db._files[index].Crc.Value); } else { _stream = new MemoryStream(); } - _rem = _db.Files[index].Size; + _rem = _db._files[index].Size; } public override void Write(byte[] buffer, int offset, int count) @@ -1441,32 +1441,32 @@ namespace SharpCompress.Common.SevenZip #endregion } - private Stream GetCachedDecoderStream(ArchiveDatabase _db, int folderIndex, IPasswordProvider pw) + private Stream GetCachedDecoderStream(ArchiveDatabase db, int folderIndex) { Stream s; if (!_cachedStreams.TryGetValue(folderIndex, out s)) { - CFolder folderInfo = _db.Folders[folderIndex]; - int packStreamIndex = _db.Folders[folderIndex].FirstPackStreamId; - long folderStartPackPos = _db.GetFolderStreamPos(folderInfo, 0); + CFolder folderInfo = db._folders[folderIndex]; + int packStreamIndex = db._folders[folderIndex]._firstPackStreamId; + long folderStartPackPos = db.GetFolderStreamPos(folderInfo, 0); List packSizes = new List(); - for (int j = 0; j < folderInfo.PackStreams.Count; j++) + for (int j = 0; j < folderInfo._packStreams.Count; j++) { - packSizes.Add(_db.PackSizes[packStreamIndex + j]); + packSizes.Add(db._packSizes[packStreamIndex + j]); } s = DecoderStreamHelper.CreateDecoderStream(_stream, folderStartPackPos, packSizes.ToArray(), folderInfo, - pw); + db.PasswordProvider); _cachedStreams.Add(folderIndex, s); } return s; } - public Stream OpenStream(ArchiveDatabase _db, int fileIndex, IPasswordProvider pw) + public Stream OpenStream(ArchiveDatabase db, int fileIndex) { - int folderIndex = _db.FileIndexToFolderIndexMap[fileIndex]; - int numFilesInFolder = _db.NumUnpackStreamsVector[folderIndex]; - int firstFileIndex = _db.FolderStartFileIndex[folderIndex]; + int folderIndex = db._fileIndexToFolderIndexMap[fileIndex]; + int numFilesInFolder = db._numUnpackStreamsVector[folderIndex]; + int firstFileIndex = db._folderStartFileIndex[folderIndex]; if (firstFileIndex > fileIndex || fileIndex - firstFileIndex >= numFilesInFolder) { throw new InvalidOperationException(); @@ -1476,21 +1476,21 @@ namespace SharpCompress.Common.SevenZip long skipSize = 0; for (int i = 0; i < skipCount; i++) { - skipSize += _db.Files[firstFileIndex + i].Size; + skipSize += db._files[firstFileIndex + i].Size; } - Stream s = GetCachedDecoderStream(_db, folderIndex, pw); + Stream s = GetCachedDecoderStream(db, folderIndex); s.Position = skipSize; - return new ReadOnlySubStream(s, _db.Files[fileIndex].Size); + return new ReadOnlySubStream(s, db._files[fileIndex].Size); } - public void Extract(ArchiveDatabase _db, int[] indices, IPasswordProvider pw) + public void Extract(ArchiveDatabase db, int[] indices) { int numItems; bool allFilesMode = (indices == null); if (allFilesMode) { - numItems = _db.Files.Count; + numItems = db._files.Count; } else { @@ -1507,62 +1507,62 @@ namespace SharpCompress.Common.SevenZip { int fileIndex = allFilesMode ? i : indices[i]; - int folderIndex = _db.FileIndexToFolderIndexMap[fileIndex]; + int folderIndex = db._fileIndexToFolderIndexMap[fileIndex]; if (folderIndex == -1) { extractFolderInfoVector.Add(new CExtractFolderInfo(fileIndex, -1)); continue; } - if (extractFolderInfoVector.Count == 0 || folderIndex != extractFolderInfoVector.Last().FolderIndex) + if (extractFolderInfoVector.Count == 0 || folderIndex != extractFolderInfoVector.Last()._folderIndex) { extractFolderInfoVector.Add(new CExtractFolderInfo(-1, folderIndex)); } CExtractFolderInfo efi = extractFolderInfoVector.Last(); - int startIndex = _db.FolderStartFileIndex[folderIndex]; - for (int index = efi.ExtractStatuses.Count; index <= fileIndex - startIndex; index++) + int startIndex = db._folderStartFileIndex[folderIndex]; + for (int index = efi._extractStatuses.Count; index <= fileIndex - startIndex; index++) { - efi.ExtractStatuses.Add(index == fileIndex - startIndex); + efi._extractStatuses.Add(index == fileIndex - startIndex); } } foreach (CExtractFolderInfo efi in extractFolderInfoVector) { int startIndex; - if (efi.FileIndex != -1) + if (efi._fileIndex != -1) { - startIndex = efi.FileIndex; + startIndex = efi._fileIndex; } else { - startIndex = _db.FolderStartFileIndex[efi.FolderIndex]; + startIndex = db._folderStartFileIndex[efi._folderIndex]; } - var outStream = new FolderUnpackStream(_db, 0, startIndex, efi.ExtractStatuses); + var outStream = new FolderUnpackStream(db, 0, startIndex, efi._extractStatuses); - if (efi.FileIndex != -1) + if (efi._fileIndex != -1) { continue; } - int folderIndex = efi.FolderIndex; - CFolder folderInfo = _db.Folders[folderIndex]; + int folderIndex = efi._folderIndex; + CFolder folderInfo = db._folders[folderIndex]; - int packStreamIndex = _db.Folders[folderIndex].FirstPackStreamId; - long folderStartPackPos = _db.GetFolderStreamPos(folderInfo, 0); + int packStreamIndex = db._folders[folderIndex]._firstPackStreamId; + long folderStartPackPos = db.GetFolderStreamPos(folderInfo, 0); List packSizes = new List(); - for (int j = 0; j < folderInfo.PackStreams.Count; j++) + for (int j = 0; j < folderInfo._packStreams.Count; j++) { - packSizes.Add(_db.PackSizes[packStreamIndex + j]); + packSizes.Add(db._packSizes[packStreamIndex + j]); } // TODO: If the decoding fails the last file may be extracted incompletely. Delete it? Stream s = DecoderStreamHelper.CreateDecoderStream(_stream, folderStartPackPos, packSizes.ToArray(), - folderInfo, pw); + folderInfo, db.PasswordProvider); byte[] buffer = new byte[4 << 10]; for (;;) { @@ -1578,14 +1578,14 @@ namespace SharpCompress.Common.SevenZip public IEnumerable GetFiles(ArchiveDatabase db) { - return db.Files; + return db._files; } public int GetFileIndex(ArchiveDatabase db, CFileItem item) { - return db.Files.IndexOf(item); + return db._files.IndexOf(item); } #endregion } -} \ No newline at end of file +} diff --git a/src/SharpCompress/Common/SevenZip/CBindPair.cs b/src/SharpCompress/Common/SevenZip/CBindPair.cs index ca52cdbe..ad947967 100644 --- a/src/SharpCompress/Common/SevenZip/CBindPair.cs +++ b/src/SharpCompress/Common/SevenZip/CBindPair.cs @@ -2,7 +2,7 @@ { internal class CBindPair { - internal int InIndex; - internal int OutIndex; + internal int _inIndex; + internal int _outIndex; } } \ No newline at end of file diff --git a/src/SharpCompress/Common/SevenZip/CCoderInfo.cs b/src/SharpCompress/Common/SevenZip/CCoderInfo.cs index 75ae4781..03572299 100644 --- a/src/SharpCompress/Common/SevenZip/CCoderInfo.cs +++ b/src/SharpCompress/Common/SevenZip/CCoderInfo.cs @@ -2,9 +2,9 @@ { internal class CCoderInfo { - internal CMethodId MethodId; - internal byte[] Props; - internal int NumInStreams; - internal int NumOutStreams; + internal CMethodId _methodId; + internal byte[] _props; + internal int _numInStreams; + internal int _numOutStreams; } } \ No newline at end of file diff --git a/src/SharpCompress/Common/SevenZip/CFolder.cs b/src/SharpCompress/Common/SevenZip/CFolder.cs index 4606d9dc..4a68cf50 100644 --- a/src/SharpCompress/Common/SevenZip/CFolder.cs +++ b/src/SharpCompress/Common/SevenZip/CFolder.cs @@ -6,27 +6,27 @@ namespace SharpCompress.Common.SevenZip { internal class CFolder { - internal List Coders = new List(); - internal List BindPairs = new List(); - internal List PackStreams = new List(); - internal int FirstPackStreamId; - internal List UnpackSizes = new List(); - internal uint? UnpackCRC; + internal List _coders = new List(); + internal List _bindPairs = new List(); + internal List _packStreams = new List(); + internal int _firstPackStreamId; + internal List _unpackSizes = new List(); + internal uint? _unpackCrc; - internal bool UnpackCRCDefined => UnpackCRC != null; + internal bool UnpackCrcDefined => _unpackCrc != null; public long GetUnpackSize() { - if (UnpackSizes.Count == 0) + if (_unpackSizes.Count == 0) { return 0; } - for (int i = UnpackSizes.Count - 1; i >= 0; i--) + for (int i = _unpackSizes.Count - 1; i >= 0; i--) { if (FindBindPairForOutStream(i) < 0) { - return UnpackSizes[i]; + return _unpackSizes[i]; } } @@ -36,9 +36,9 @@ namespace SharpCompress.Common.SevenZip public int GetNumOutStreams() { int count = 0; - for (int i = 0; i < Coders.Count; i++) + for (int i = 0; i < _coders.Count; i++) { - count += Coders[i].NumOutStreams; + count += _coders[i]._numOutStreams; } return count; @@ -46,9 +46,9 @@ namespace SharpCompress.Common.SevenZip public int FindBindPairForInStream(int inStreamIndex) { - for (int i = 0; i < BindPairs.Count; i++) + for (int i = 0; i < _bindPairs.Count; i++) { - if (BindPairs[i].InIndex == inStreamIndex) + if (_bindPairs[i]._inIndex == inStreamIndex) { return i; } @@ -59,9 +59,9 @@ namespace SharpCompress.Common.SevenZip public int FindBindPairForOutStream(int outStreamIndex) { - for (int i = 0; i < BindPairs.Count; i++) + for (int i = 0; i < _bindPairs.Count; i++) { - if (BindPairs[i].OutIndex == outStreamIndex) + if (_bindPairs[i]._outIndex == outStreamIndex) { return i; } @@ -72,9 +72,9 @@ namespace SharpCompress.Common.SevenZip public int FindPackStreamArrayIndex(int inStreamIndex) { - for (int i = 0; i < PackStreams.Count; i++) + for (int i = 0; i < _packStreams.Count; i++) { - if (PackStreams[i] == inStreamIndex) + if (_packStreams[i] == inStreamIndex) { return i; } @@ -85,9 +85,9 @@ namespace SharpCompress.Common.SevenZip public bool IsEncrypted() { - for (int i = Coders.Count - 1; i >= 0; i--) + for (int i = _coders.Count - 1; i >= 0; i--) { - if (Coders[i].MethodId == CMethodId.kAES) + if (_coders[i]._methodId == CMethodId.K_AES) { return true; } @@ -102,25 +102,25 @@ namespace SharpCompress.Common.SevenZip const int kMaskSize = 32; // it must be >= kNumCodersMax const int kNumBindsMax = 32; - if (Coders.Count > kNumCodersMax || BindPairs.Count > kNumBindsMax) + if (_coders.Count > kNumCodersMax || _bindPairs.Count > kNumBindsMax) { return false; } { - var v = new BitVector(BindPairs.Count + PackStreams.Count); + var v = new BitVector(_bindPairs.Count + _packStreams.Count); - for (int i = 0; i < BindPairs.Count; i++) + for (int i = 0; i < _bindPairs.Count; i++) { - if (v.GetAndSet(BindPairs[i].InIndex)) + if (v.GetAndSet(_bindPairs[i]._inIndex)) { return false; } } - for (int i = 0; i < PackStreams.Count; i++) + for (int i = 0; i < _packStreams.Count; i++) { - if (v.GetAndSet(PackStreams[i])) + if (v.GetAndSet(_packStreams[i])) { return false; } @@ -128,10 +128,10 @@ namespace SharpCompress.Common.SevenZip } { - var v = new BitVector(UnpackSizes.Count); - for (int i = 0; i < BindPairs.Count; i++) + var v = new BitVector(_unpackSizes.Count); + for (int i = 0; i < _bindPairs.Count; i++) { - if (v.GetAndSet(BindPairs[i].OutIndex)) + if (v.GetAndSet(_bindPairs[i]._outIndex)) { return false; } @@ -143,23 +143,23 @@ namespace SharpCompress.Common.SevenZip { List inStreamToCoder = new List(); List outStreamToCoder = new List(); - for (int i = 0; i < Coders.Count; i++) + for (int i = 0; i < _coders.Count; i++) { - CCoderInfo coder = Coders[i]; - for (int j = 0; j < coder.NumInStreams; j++) + CCoderInfo coder = _coders[i]; + for (int j = 0; j < coder._numInStreams; j++) { inStreamToCoder.Add(i); } - for (int j = 0; j < coder.NumOutStreams; j++) + for (int j = 0; j < coder._numOutStreams; j++) { outStreamToCoder.Add(i); } } - for (int i = 0; i < BindPairs.Count; i++) + for (int i = 0; i < _bindPairs.Count; i++) { - CBindPair bp = BindPairs[i]; - mask[inStreamToCoder[bp.InIndex]] |= (1u << outStreamToCoder[bp.OutIndex]); + CBindPair bp = _bindPairs[i]; + mask[inStreamToCoder[bp._inIndex]] |= (1u << outStreamToCoder[bp._outIndex]); } } diff --git a/src/SharpCompress/Common/SevenZip/CMethodId.cs b/src/SharpCompress/Common/SevenZip/CMethodId.cs index 685d28bb..ee9e3bb3 100644 --- a/src/SharpCompress/Common/SevenZip/CMethodId.cs +++ b/src/SharpCompress/Common/SevenZip/CMethodId.cs @@ -2,26 +2,26 @@ { internal struct CMethodId { - public const ulong kCopyId = 0; - public const ulong kLzmaId = 0x030101; - public const ulong kLzma2Id = 0x21; - public const ulong kAESId = 0x06F10701; + public const ulong K_COPY_ID = 0; + public const ulong K_LZMA_ID = 0x030101; + public const ulong K_LZMA2_ID = 0x21; + public const ulong K_AES_ID = 0x06F10701; - public static readonly CMethodId kCopy = new CMethodId(kCopyId); - public static readonly CMethodId kLzma = new CMethodId(kLzmaId); - public static readonly CMethodId kLzma2 = new CMethodId(kLzma2Id); - public static readonly CMethodId kAES = new CMethodId(kAESId); + public static readonly CMethodId K_COPY = new CMethodId(K_COPY_ID); + public static readonly CMethodId K_LZMA = new CMethodId(K_LZMA_ID); + public static readonly CMethodId K_LZMA2 = new CMethodId(K_LZMA2_ID); + public static readonly CMethodId K_AES = new CMethodId(K_AES_ID); - public readonly ulong Id; + public readonly ulong _id; public CMethodId(ulong id) { - Id = id; + _id = id; } public override int GetHashCode() { - return Id.GetHashCode(); + return _id.GetHashCode(); } public override bool Equals(object obj) @@ -31,23 +31,23 @@ public bool Equals(CMethodId other) { - return Id == other.Id; + return _id == other._id; } public static bool operator ==(CMethodId left, CMethodId right) { - return left.Id == right.Id; + return left._id == right._id; } public static bool operator !=(CMethodId left, CMethodId right) { - return left.Id != right.Id; + return left._id != right._id; } public int GetLength() { int bytes = 0; - for (ulong value = Id; value != 0; value >>= 8) + for (ulong value = _id; value != 0; value >>= 8) { bytes++; } diff --git a/src/SharpCompress/Common/SevenZip/SevenZipEntry.cs b/src/SharpCompress/Common/SevenZip/SevenZipEntry.cs index dd23d629..584bfb95 100644 --- a/src/SharpCompress/Common/SevenZip/SevenZipEntry.cs +++ b/src/SharpCompress/Common/SevenZip/SevenZipEntry.cs @@ -34,7 +34,7 @@ namespace SharpCompress.Common.SevenZip public override bool IsDirectory => FilePart.Header.IsDir; - public override bool IsSplit => false; + public override bool IsSplitAfter => false; public override int? Attrib => (int)FilePart.Header.Attrib; diff --git a/src/SharpCompress/Common/SevenZip/SevenZipFilePart.cs b/src/SharpCompress/Common/SevenZip/SevenZipFilePart.cs index 9dec4aa0..3c5232b2 100644 --- a/src/SharpCompress/Common/SevenZip/SevenZipFilePart.cs +++ b/src/SharpCompress/Common/SevenZip/SevenZipFilePart.cs @@ -14,17 +14,16 @@ namespace SharpCompress.Common.SevenZip internal SevenZipFilePart(Stream stream, ArchiveDatabase database, int index, CFileItem fileEntry, ArchiveEncoding archiveEncoding) : base(archiveEncoding) { - this._stream = stream; - this._database = database; + _stream = stream; + _database = database; Index = index; Header = fileEntry; if (Header.HasStream) { - Folder = database.Folders[database.FileIndexToFolderIndexMap[index]]; + Folder = database._folders[database._fileIndexToFolderIndexMap[index]]; } } - internal Stream BaseStream { get; private set; } internal CFileItem Header { get; } internal CFolder Folder { get; } internal int Index { get; } @@ -42,14 +41,14 @@ namespace SharpCompress.Common.SevenZip { return null; } - var folderStream = _database.GetFolderStream(_stream, Folder, null); + var folderStream = _database.GetFolderStream(_stream, Folder, _database.PasswordProvider); - int firstFileIndex = _database.FolderStartFileIndex[_database.Folders.IndexOf(Folder)]; + int firstFileIndex = _database._folderStartFileIndex[_database._folders.IndexOf(Folder)]; int skipCount = Index - firstFileIndex; long skipSize = 0; for (int i = 0; i < skipCount; i++) { - skipSize += _database.Files[firstFileIndex + i].Size; + skipSize += _database._files[firstFileIndex + i].Size; } if (skipSize > 0) { @@ -71,31 +70,31 @@ namespace SharpCompress.Common.SevenZip } //copied from DecoderRegistry - private const uint k_Copy = 0x0; - private const uint k_Delta = 3; - private const uint k_LZMA2 = 0x21; - private const uint k_LZMA = 0x030101; - private const uint k_PPMD = 0x030401; - private const uint k_BCJ = 0x03030103; - private const uint k_BCJ2 = 0x0303011B; - private const uint k_Deflate = 0x040108; - private const uint k_BZip2 = 0x040202; + private const uint K_COPY = 0x0; + private const uint K_DELTA = 3; + private const uint K_LZMA2 = 0x21; + private const uint K_LZMA = 0x030101; + private const uint K_PPMD = 0x030401; + private const uint K_BCJ = 0x03030103; + private const uint K_BCJ2 = 0x0303011B; + private const uint K_DEFLATE = 0x040108; + private const uint K_B_ZIP2 = 0x040202; internal CompressionType GetCompression() { - var coder = Folder.Coders.First(); - switch (coder.MethodId.Id) + var coder = Folder._coders.First(); + switch (coder._methodId._id) { - case k_LZMA: - case k_LZMA2: + case K_LZMA: + case K_LZMA2: { return CompressionType.LZMA; } - case k_PPMD: + case K_PPMD: { return CompressionType.PPMd; } - case k_BZip2: + case K_B_ZIP2: { return CompressionType.BZip2; } diff --git a/src/SharpCompress/Common/Tar/Headers/TarHeader.cs b/src/SharpCompress/Common/Tar/Headers/TarHeader.cs index a03495f0..9f7f1536 100644 --- a/src/SharpCompress/Common/Tar/Headers/TarHeader.cs +++ b/src/SharpCompress/Common/Tar/Headers/TarHeader.cs @@ -7,7 +7,7 @@ namespace SharpCompress.Common.Tar.Headers { internal class TarHeader { - internal static readonly DateTime Epoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc); + internal static readonly DateTime EPOCH = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc); public TarHeader(ArchiveEncoding archiveEncoding) { @@ -27,11 +27,11 @@ namespace SharpCompress.Common.Tar.Headers internal Stream PackedStream { get; set; } internal ArchiveEncoding ArchiveEncoding { get; } - internal const int BlockSize = 512; + internal const int BLOCK_SIZE = 512; internal void Write(Stream output) { - byte[] buffer = new byte[BlockSize]; + byte[] buffer = new byte[BLOCK_SIZE]; WriteOctalBytes(511, buffer, 100, 8); // file mode WriteOctalBytes(0, buffer, 108, 8); // owner ID @@ -49,7 +49,7 @@ namespace SharpCompress.Common.Tar.Headers { WriteStringBytes(Name, buffer, 0, 100); WriteOctalBytes(Size, buffer, 124, 12); - var time = (long)(LastModifiedTime.ToUniversalTime() - Epoch).TotalSeconds; + var time = (long)(LastModifiedTime.ToUniversalTime() - EPOCH).TotalSeconds; WriteOctalBytes(time, buffer, 136, 12); buffer[156] = (byte)EntryType; @@ -82,10 +82,10 @@ namespace SharpCompress.Common.Tar.Headers output.Write(nameBytes, 0, nameBytes.Length); // pad to multiple of BlockSize bytes, and make sure a terminating null is added - int numPaddingBytes = BlockSize - (nameBytes.Length % BlockSize); + int numPaddingBytes = BLOCK_SIZE - (nameBytes.Length % BLOCK_SIZE); if (numPaddingBytes == 0) { - numPaddingBytes = BlockSize; + numPaddingBytes = BLOCK_SIZE; } output.Write(new byte[numPaddingBytes], 0, numPaddingBytes); } @@ -114,8 +114,8 @@ namespace SharpCompress.Common.Tar.Headers //Mode = ReadASCIIInt32Base8(buffer, 100, 7); //UserId = ReadASCIIInt32Base8(buffer, 108, 7); //GroupId = ReadASCIIInt32Base8(buffer, 116, 7); - long unixTimeStamp = ReadASCIIInt64Base8(buffer, 136, 11); - LastModifiedTime = Epoch.AddSeconds(unixTimeStamp).ToLocalTime(); + long unixTimeStamp = ReadAsciiInt64Base8(buffer, 136, 11); + LastModifiedTime = EPOCH.AddSeconds(unixTimeStamp).ToLocalTime(); Magic = ArchiveEncoding.Decode(buffer, 257, 6).TrimNulls(); @@ -142,10 +142,10 @@ namespace SharpCompress.Common.Tar.Headers var size = ReadSize(buffer); var nameLength = (int)size; var nameBytes = reader.ReadBytes(nameLength); - var remainingBytesToRead = BlockSize - (nameLength % BlockSize); + var remainingBytesToRead = BLOCK_SIZE - (nameLength % BLOCK_SIZE); // Read the rest of the block and discard the data - if (remainingBytesToRead < BlockSize) + if (remainingBytesToRead < BLOCK_SIZE) { reader.ReadBytes(remainingBytesToRead); } @@ -163,14 +163,14 @@ namespace SharpCompress.Common.Tar.Headers { return DataConverter.BigEndian.GetInt64(buffer, 0x80); } - return ReadASCIIInt64Base8(buffer, 124, 11); + return ReadAsciiInt64Base8(buffer, 124, 11); } private static byte[] ReadBlock(BinaryReader reader) { - byte[] buffer = reader.ReadBytes(BlockSize); + byte[] buffer = reader.ReadBytes(BLOCK_SIZE); - if (buffer.Length != 0 && buffer.Length < BlockSize) + if (buffer.Length != 0 && buffer.Length < BLOCK_SIZE) { throw new InvalidOperationException("Buffer is invalid size"); } @@ -206,7 +206,7 @@ namespace SharpCompress.Common.Tar.Headers } } - private static int ReadASCIIInt32Base8(byte[] buffer, int offset, int count) + private static int ReadAsciiInt32Base8(byte[] buffer, int offset, int count) { string s = Encoding.UTF8.GetString(buffer, offset, count).TrimNulls(); if (string.IsNullOrEmpty(s)) @@ -216,7 +216,7 @@ namespace SharpCompress.Common.Tar.Headers return Convert.ToInt32(s, 8); } - private static long ReadASCIIInt64Base8(byte[] buffer, int offset, int count) + private static long ReadAsciiInt64Base8(byte[] buffer, int offset, int count) { string s = Encoding.UTF8.GetString(buffer, offset, count).TrimNulls(); if (string.IsNullOrEmpty(s)) @@ -226,7 +226,7 @@ namespace SharpCompress.Common.Tar.Headers return Convert.ToInt64(s, 8); } - private static long ReadASCIIInt64(byte[] buffer, int offset, int count) + private static long ReadAsciiInt64(byte[] buffer, int offset, int count) { string s = Encoding.UTF8.GetString(buffer, offset, count).TrimNulls(); if (string.IsNullOrEmpty(s)) diff --git a/src/SharpCompress/Common/Tar/TarEntry.cs b/src/SharpCompress/Common/Tar/TarEntry.cs index 5208bce1..82e707d0 100644 --- a/src/SharpCompress/Common/Tar/TarEntry.cs +++ b/src/SharpCompress/Common/Tar/TarEntry.cs @@ -9,11 +9,11 @@ namespace SharpCompress.Common.Tar { public class TarEntry : Entry { - private readonly TarFilePart filePart; + private readonly TarFilePart _filePart; internal TarEntry(TarFilePart filePart, CompressionType type) { - this.filePart = filePart; + this._filePart = filePart; CompressionType = type; } @@ -21,13 +21,13 @@ namespace SharpCompress.Common.Tar public override long Crc => 0; - public override string Key => filePart.Header.Name; + public override string Key => _filePart.Header.Name; - public override long CompressedSize => filePart.Header.Size; + public override long CompressedSize => _filePart.Header.Size; - public override long Size => filePart.Header.Size; + public override long Size => _filePart.Header.Size; - public override DateTime? LastModifiedTime => filePart.Header.LastModifiedTime; + public override DateTime? LastModifiedTime => _filePart.Header.LastModifiedTime; public override DateTime? CreatedTime => null; @@ -37,11 +37,11 @@ namespace SharpCompress.Common.Tar public override bool IsEncrypted => false; - public override bool IsDirectory => filePart.Header.EntryType == EntryType.Directory; + public override bool IsDirectory => _filePart.Header.EntryType == EntryType.Directory; - public override bool IsSplit => false; + public override bool IsSplitAfter => false; - internal override IEnumerable Parts => filePart.AsEnumerable(); + internal override IEnumerable Parts => _filePart.AsEnumerable(); internal static IEnumerable GetEntries(StreamingMode mode, Stream stream, CompressionType compressionType, ArchiveEncoding archiveEncoding) diff --git a/src/SharpCompress/Common/Tar/TarFilePart.cs b/src/SharpCompress/Common/Tar/TarFilePart.cs index 2aec15f4..dafbb867 100644 --- a/src/SharpCompress/Common/Tar/TarFilePart.cs +++ b/src/SharpCompress/Common/Tar/TarFilePart.cs @@ -11,7 +11,7 @@ namespace SharpCompress.Common.Tar internal TarFilePart(TarHeader header, Stream seekableStream) : base(header.ArchiveEncoding) { - this._seekableStream = seekableStream; + _seekableStream = seekableStream; Header = header; } diff --git a/src/SharpCompress/Common/Tar/TarReadOnlySubStream.cs b/src/SharpCompress/Common/Tar/TarReadOnlySubStream.cs index 337dad74..25300840 100644 --- a/src/SharpCompress/Common/Tar/TarReadOnlySubStream.cs +++ b/src/SharpCompress/Common/Tar/TarReadOnlySubStream.cs @@ -5,8 +5,8 @@ namespace SharpCompress.Common.Tar { internal class TarReadOnlySubStream : Stream { - private bool isDisposed; - private long amountRead; + private bool _isDisposed; + private long _amountRead; public TarReadOnlySubStream(Stream stream, long bytesToRead) { @@ -16,14 +16,14 @@ namespace SharpCompress.Common.Tar protected override void Dispose(bool disposing) { - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; + _isDisposed = true; if (disposing) { - long skipBytes = amountRead % 512; + long skipBytes = _amountRead % 512; if (skipBytes == 0) { return; @@ -67,11 +67,27 @@ namespace SharpCompress.Common.Tar if (read > 0) { BytesLeftToRead -= read; - amountRead += read; + _amountRead += read; } return read; } + public override int ReadByte() + { + if (BytesLeftToRead <= 0) + { + return -1; + } + int value = Stream.ReadByte(); + if (value != -1) + { + --BytesLeftToRead; + ++_amountRead; + } + return value; + + } + public override long Seek(long offset, SeekOrigin origin) { throw new NotSupportedException(); diff --git a/src/SharpCompress/Common/Volume.cs b/src/SharpCompress/Common/Volume.cs index 87b0d856..86d968c0 100644 --- a/src/SharpCompress/Common/Volume.cs +++ b/src/SharpCompress/Common/Volume.cs @@ -6,15 +6,19 @@ namespace SharpCompress.Common { public abstract class Volume : IVolume { - private readonly Stream actualStream; + private readonly Stream _actualStream; internal Volume(Stream stream, ReaderOptions readerOptions) { - actualStream = stream; ReaderOptions = readerOptions; + if (readerOptions.LeaveStreamOpen) + { + stream = new NonDisposingStream(stream); + } + _actualStream = stream; } - internal Stream Stream => new NonDisposingStream(actualStream); + internal Stream Stream => _actualStream; protected ReaderOptions ReaderOptions { get; } @@ -29,14 +33,14 @@ namespace SharpCompress.Common /// public virtual bool IsMultiVolume => true; - private bool disposed; + private bool _disposed; public void Dispose() { - if (!ReaderOptions.LeaveStreamOpen && !disposed) + if (!_disposed) { - actualStream.Dispose(); - disposed = true; + _actualStream.Dispose(); + _disposed = true; } } } diff --git a/src/SharpCompress/Common/Zip/Headers/DirectoryEntryHeader.cs b/src/SharpCompress/Common/Zip/Headers/DirectoryEntryHeader.cs index e7502d26..a71d311d 100644 --- a/src/SharpCompress/Common/Zip/Headers/DirectoryEntryHeader.cs +++ b/src/SharpCompress/Common/Zip/Headers/DirectoryEntryHeader.cs @@ -1,6 +1,6 @@ -using System; -using System.IO; +using System.IO; using System.Linq; +using System.Text; namespace SharpCompress.Common.Zip.Headers { @@ -31,10 +31,20 @@ namespace SharpCompress.Common.Zip.Headers RelativeOffsetOfEntryHeader = reader.ReadUInt32(); byte[] name = reader.ReadBytes(nameLength); - Name = ArchiveEncoding.Decode(name); byte[] extra = reader.ReadBytes(extraLength); byte[] comment = reader.ReadBytes(commentLength); - Comment = ArchiveEncoding.Decode(comment); + + if (Flags.HasFlag(HeaderFlags.Efs)) + { + Name = ArchiveEncoding.Decode(name); + Comment = ArchiveEncoding.Decode(comment); + } + else + { + Name = ArchiveEncoding.Decode437(name); + Comment = ArchiveEncoding.Decode437(comment); + } + LoadExtra(extra); var unicodePathExtra = Extra.FirstOrDefault(u => u.Type == ExtraDataType.UnicodePathExtraField); @@ -50,10 +60,12 @@ namespace SharpCompress.Common.Zip.Headers { CompressedSize = zip64ExtraData.CompressedSize; } + if (UncompressedSize == uint.MaxValue) { UncompressedSize = zip64ExtraData.UncompressedSize; } + if (RelativeOffsetOfEntryHeader == uint.MaxValue) { RelativeOffsetOfEntryHeader = zip64ExtraData.RelativeOffsetOfEntryHeader; diff --git a/src/SharpCompress/Common/Zip/Headers/HeaderFlags.cs b/src/SharpCompress/Common/Zip/Headers/HeaderFlags.cs index 6e35191c..92aaaca7 100644 --- a/src/SharpCompress/Common/Zip/Headers/HeaderFlags.cs +++ b/src/SharpCompress/Common/Zip/Headers/HeaderFlags.cs @@ -11,6 +11,7 @@ namespace SharpCompress.Common.Zip.Headers Bit2 = 4, UsePostDataDescriptor = 8, EnhancedDeflate = 16, - UTF8 = 2048 + //Bit 11: Language encoding flag + Efs = 2048 } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Zip/Headers/LocalEntryHeader.cs b/src/SharpCompress/Common/Zip/Headers/LocalEntryHeader.cs index 2b89385b..e56b6102 100644 --- a/src/SharpCompress/Common/Zip/Headers/LocalEntryHeader.cs +++ b/src/SharpCompress/Common/Zip/Headers/LocalEntryHeader.cs @@ -25,7 +25,17 @@ namespace SharpCompress.Common.Zip.Headers ushort extraLength = reader.ReadUInt16(); byte[] name = reader.ReadBytes(nameLength); byte[] extra = reader.ReadBytes(extraLength); - Name = ArchiveEncoding.Decode(name); + + if (Flags.HasFlag(HeaderFlags.Efs)) + { + Name = ArchiveEncoding.Decode(name); + } + else + { + // Use IBM Code Page 437 (IBM PC character encoding set) + Name = ArchiveEncoding.Decode437(name); + } + LoadExtra(extra); var unicodePathExtra = Extra.FirstOrDefault(u => u.Type == ExtraDataType.UnicodePathExtraField); diff --git a/src/SharpCompress/Common/Zip/Headers/LocalEntryHeaderExtraFactory.cs b/src/SharpCompress/Common/Zip/Headers/LocalEntryHeaderExtraFactory.cs index 23ae243a..b86d5052 100644 --- a/src/SharpCompress/Common/Zip/Headers/LocalEntryHeaderExtraFactory.cs +++ b/src/SharpCompress/Common/Zip/Headers/LocalEntryHeaderExtraFactory.cs @@ -27,7 +27,7 @@ namespace SharpCompress.Common.Zip.Headers { internal byte Version => DataBytes[0]; - internal byte[] NameCRC32 + internal byte[] NameCrc32 { get { diff --git a/src/SharpCompress/Common/Zip/Headers/Zip64DirectoryEndHeader.cs b/src/SharpCompress/Common/Zip/Headers/Zip64DirectoryEndHeader.cs index 2f97ad53..443cbd8a 100644 --- a/src/SharpCompress/Common/Zip/Headers/Zip64DirectoryEndHeader.cs +++ b/src/SharpCompress/Common/Zip/Headers/Zip64DirectoryEndHeader.cs @@ -21,10 +21,10 @@ namespace SharpCompress.Common.Zip.Headers TotalNumberOfEntries = (long)reader.ReadUInt64(); DirectorySize = (long)reader.ReadUInt64(); DirectoryStartOffsetRelativeToDisk = (long)reader.ReadUInt64(); - DataSector = reader.ReadBytes((int)(SizeOfDirectoryEndRecord - SizeOfFixedHeaderDataExceptSignatureAndSizeFields)); + DataSector = reader.ReadBytes((int)(SizeOfDirectoryEndRecord - SIZE_OF_FIXED_HEADER_DATA_EXCEPT_SIGNATURE_AND_SIZE_FIELDS)); } - const int SizeOfFixedHeaderDataExceptSignatureAndSizeFields = 44; + private const int SIZE_OF_FIXED_HEADER_DATA_EXCEPT_SIGNATURE_AND_SIZE_FIELDS = 44; public long SizeOfDirectoryEndRecord { get; private set; } diff --git a/src/SharpCompress/Common/Zip/PkwareTraditionalCryptoStream.cs b/src/SharpCompress/Common/Zip/PkwareTraditionalCryptoStream.cs index 2d412242..2c3f1400 100644 --- a/src/SharpCompress/Common/Zip/PkwareTraditionalCryptoStream.cs +++ b/src/SharpCompress/Common/Zip/PkwareTraditionalCryptoStream.cs @@ -11,23 +11,23 @@ namespace SharpCompress.Common.Zip internal class PkwareTraditionalCryptoStream : Stream { - private readonly PkwareTraditionalEncryptionData encryptor; - private readonly CryptoMode mode; - private readonly Stream stream; - private bool isDisposed; + private readonly PkwareTraditionalEncryptionData _encryptor; + private readonly CryptoMode _mode; + private readonly Stream _stream; + private bool _isDisposed; public PkwareTraditionalCryptoStream(Stream stream, PkwareTraditionalEncryptionData encryptor, CryptoMode mode) { - this.encryptor = encryptor; - this.stream = stream; - this.mode = mode; + this._encryptor = encryptor; + this._stream = stream; + this._mode = mode; } - public override bool CanRead => (mode == CryptoMode.Decrypt); + public override bool CanRead => (_mode == CryptoMode.Decrypt); public override bool CanSeek => false; - public override bool CanWrite => (mode == CryptoMode.Encrypt); + public override bool CanWrite => (_mode == CryptoMode.Encrypt); public override long Length => throw new NotSupportedException(); @@ -35,7 +35,7 @@ namespace SharpCompress.Common.Zip public override int Read(byte[] buffer, int offset, int count) { - if (mode == CryptoMode.Encrypt) + if (_mode == CryptoMode.Encrypt) { throw new NotSupportedException("This stream does not encrypt via Read()"); } @@ -46,15 +46,15 @@ namespace SharpCompress.Common.Zip } byte[] temp = new byte[count]; - int readBytes = stream.Read(temp, 0, count); - byte[] decrypted = encryptor.Decrypt(temp, readBytes); + int readBytes = _stream.Read(temp, 0, count); + byte[] decrypted = _encryptor.Decrypt(temp, readBytes); Buffer.BlockCopy(decrypted, 0, buffer, offset, readBytes); return readBytes; } public override void Write(byte[] buffer, int offset, int count) { - if (mode == CryptoMode.Decrypt) + if (_mode == CryptoMode.Decrypt) { throw new NotSupportedException("This stream does not Decrypt via Write()"); } @@ -75,8 +75,8 @@ namespace SharpCompress.Common.Zip plaintext = buffer; } - byte[] encrypted = encryptor.Encrypt(plaintext, count); - stream.Write(encrypted, 0, encrypted.Length); + byte[] encrypted = _encryptor.Encrypt(plaintext, count); + _stream.Write(encrypted, 0, encrypted.Length); } public override void Flush() @@ -96,13 +96,13 @@ namespace SharpCompress.Common.Zip protected override void Dispose(bool disposing) { - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; + _isDisposed = true; base.Dispose(disposing); - stream.Dispose(); + _stream.Dispose(); } } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Zip/PkwareTraditionalEncryptionData.cs b/src/SharpCompress/Common/Zip/PkwareTraditionalEncryptionData.cs index a423176b..2a328003 100644 --- a/src/SharpCompress/Common/Zip/PkwareTraditionalEncryptionData.cs +++ b/src/SharpCompress/Common/Zip/PkwareTraditionalEncryptionData.cs @@ -7,8 +7,8 @@ namespace SharpCompress.Common.Zip { internal class PkwareTraditionalEncryptionData { - private static readonly CRC32 crc32 = new CRC32(); - private readonly UInt32[] _Keys = {0x12345678, 0x23456789, 0x34567890}; + private static readonly CRC32 CRC32 = new CRC32(); + private readonly UInt32[] _keys = {0x12345678, 0x23456789, 0x34567890}; private readonly ArchiveEncoding _archiveEncoding; private PkwareTraditionalEncryptionData(string password, ArchiveEncoding archiveEncoding) @@ -21,7 +21,7 @@ namespace SharpCompress.Common.Zip { get { - ushort t = (ushort)((ushort)(_Keys[2] & 0xFFFF) | 2); + ushort t = (ushort)((ushort)(_keys[2] & 0xFFFF) | 2); return (byte)((t * (t ^ 1)) >> 8); } } @@ -56,9 +56,9 @@ namespace SharpCompress.Common.Zip var plainText = new byte[length]; for (int i = 0; i < length; i++) { - var C = (byte)(cipherText[i] ^ MagicByte); - UpdateKeys(C); - plainText[i] = C; + var c = (byte)(cipherText[i] ^ MagicByte); + UpdateKeys(c); + plainText[i] = c; } return plainText; } @@ -79,9 +79,9 @@ namespace SharpCompress.Common.Zip var cipherText = new byte[length]; for (int i = 0; i < length; i++) { - byte C = plainText[i]; + byte c = plainText[i]; cipherText[i] = (byte)(plainText[i] ^ MagicByte); - UpdateKeys(C); + UpdateKeys(c); } return cipherText; } @@ -103,10 +103,10 @@ namespace SharpCompress.Common.Zip private void UpdateKeys(byte byteValue) { - _Keys[0] = (UInt32)crc32.ComputeCrc32((int)_Keys[0], byteValue); - _Keys[1] = _Keys[1] + (byte)_Keys[0]; - _Keys[1] = _Keys[1] * 0x08088405 + 1; - _Keys[2] = (UInt32)crc32.ComputeCrc32((int)_Keys[2], (byte)(_Keys[1] >> 24)); + _keys[0] = (UInt32)CRC32.ComputeCrc32((int)_keys[0], byteValue); + _keys[1] = _keys[1] + (byte)_keys[0]; + _keys[1] = _keys[1] * 0x08088405 + 1; + _keys[2] = (UInt32)CRC32.ComputeCrc32((int)_keys[2], (byte)(_keys[1] >> 24)); } } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Zip/SeekableZipFilePart.cs b/src/SharpCompress/Common/Zip/SeekableZipFilePart.cs index 2364eb27..d06b34aa 100644 --- a/src/SharpCompress/Common/Zip/SeekableZipFilePart.cs +++ b/src/SharpCompress/Common/Zip/SeekableZipFilePart.cs @@ -11,7 +11,7 @@ namespace SharpCompress.Common.Zip internal SeekableZipFilePart(SeekableZipHeaderFactory headerFactory, DirectoryEntryHeader header, Stream stream) : base(header, stream) { - this._headerFactory = headerFactory; + _headerFactory = headerFactory; } internal override Stream GetCompressedStream() diff --git a/src/SharpCompress/Common/Zip/StreamingZipFilePart.cs b/src/SharpCompress/Common/Zip/StreamingZipFilePart.cs index a528c137..60715413 100644 --- a/src/SharpCompress/Common/Zip/StreamingZipFilePart.cs +++ b/src/SharpCompress/Common/Zip/StreamingZipFilePart.cs @@ -7,7 +7,7 @@ namespace SharpCompress.Common.Zip { internal class StreamingZipFilePart : ZipFilePart { - private Stream decompressionStream; + private Stream _decompressionStream; internal StreamingZipFilePart(ZipFileEntry header, Stream stream) : base(header, stream) @@ -25,12 +25,12 @@ namespace SharpCompress.Common.Zip { return Stream.Null; } - decompressionStream = CreateDecompressionStream(GetCryptoStream(CreateBaseStream()), Header.CompressionMethod); + _decompressionStream = CreateDecompressionStream(GetCryptoStream(CreateBaseStream()), Header.CompressionMethod); if (LeaveStreamOpen) { - return new NonDisposingStream(decompressionStream); + return new NonDisposingStream(_decompressionStream); } - return decompressionStream; + return _decompressionStream; } internal BinaryReader FixStreamedFileLocation(ref RewindableStream rewindableStream) @@ -41,13 +41,13 @@ namespace SharpCompress.Common.Zip } if (Header.HasData && !Skipped) { - if (decompressionStream == null) + if (_decompressionStream == null) { - decompressionStream = GetCompressedStream(); + _decompressionStream = GetCompressedStream(); } - decompressionStream.Skip(); + _decompressionStream.Skip(); - DeflateStream deflateStream = decompressionStream as DeflateStream; + DeflateStream deflateStream = _decompressionStream as DeflateStream; if (deflateStream != null) { rewindableStream.Rewind(deflateStream.InputBuffer); @@ -55,7 +55,7 @@ namespace SharpCompress.Common.Zip Skipped = true; } var reader = new BinaryReader(rewindableStream); - decompressionStream = null; + _decompressionStream = null; return reader; } } diff --git a/src/SharpCompress/Common/Zip/StreamingZipHeaderFactory.cs b/src/SharpCompress/Common/Zip/StreamingZipHeaderFactory.cs index 1f4df93a..1148c53c 100644 --- a/src/SharpCompress/Common/Zip/StreamingZipHeaderFactory.cs +++ b/src/SharpCompress/Common/Zip/StreamingZipHeaderFactory.cs @@ -28,25 +28,25 @@ namespace SharpCompress.Common.Zip { ZipHeader header = null; BinaryReader reader = new BinaryReader(rewindableStream); - if (lastEntryHeader != null && - (FlagUtility.HasFlag(lastEntryHeader.Flags, HeaderFlags.UsePostDataDescriptor) || lastEntryHeader.IsZip64)) + if (_lastEntryHeader != null && + (FlagUtility.HasFlag(_lastEntryHeader.Flags, HeaderFlags.UsePostDataDescriptor) || _lastEntryHeader.IsZip64)) { - reader = (lastEntryHeader.Part as StreamingZipFilePart).FixStreamedFileLocation(ref rewindableStream); + reader = (_lastEntryHeader.Part as StreamingZipFilePart).FixStreamedFileLocation(ref rewindableStream); long? pos = rewindableStream.CanSeek ? (long?)rewindableStream.Position : null; uint crc = reader.ReadUInt32(); if (crc == POST_DATA_DESCRIPTOR) { crc = reader.ReadUInt32(); } - lastEntryHeader.Crc = crc; - lastEntryHeader.CompressedSize = reader.ReadUInt32(); - lastEntryHeader.UncompressedSize = reader.ReadUInt32(); + _lastEntryHeader.Crc = crc; + _lastEntryHeader.CompressedSize = reader.ReadUInt32(); + _lastEntryHeader.UncompressedSize = reader.ReadUInt32(); if (pos.HasValue) { - lastEntryHeader.DataStartPosition = pos - lastEntryHeader.CompressedSize; + _lastEntryHeader.DataStartPosition = pos - _lastEntryHeader.CompressedSize; } } - lastEntryHeader = null; + _lastEntryHeader = null; uint headerBytes = reader.ReadUInt32(); header = ReadHeader(headerBytes, reader); diff --git a/src/SharpCompress/Common/Zip/WinzipAesCryptoStream.cs b/src/SharpCompress/Common/Zip/WinzipAesCryptoStream.cs index e5618827..62e6cff2 100644 --- a/src/SharpCompress/Common/Zip/WinzipAesCryptoStream.cs +++ b/src/SharpCompress/Common/Zip/WinzipAesCryptoStream.cs @@ -10,25 +10,25 @@ namespace SharpCompress.Common.Zip internal class WinzipAesCryptoStream : Stream { private const int BLOCK_SIZE_IN_BYTES = 16; - private readonly SymmetricAlgorithm cipher; - private readonly byte[] counter = new byte[BLOCK_SIZE_IN_BYTES]; - private readonly Stream stream; - private readonly ICryptoTransform transform; - private int nonce = 1; - private byte[] counterOut = new byte[BLOCK_SIZE_IN_BYTES]; - private bool isFinalBlock; - private long totalBytesLeftToRead; - private bool isDisposed; + private readonly SymmetricAlgorithm _cipher; + private readonly byte[] _counter = new byte[BLOCK_SIZE_IN_BYTES]; + private readonly Stream _stream; + private readonly ICryptoTransform _transform; + private int _nonce = 1; + private byte[] _counterOut = new byte[BLOCK_SIZE_IN_BYTES]; + private bool _isFinalBlock; + private long _totalBytesLeftToRead; + private bool _isDisposed; internal WinzipAesCryptoStream(Stream stream, WinzipAesEncryptionData winzipAesEncryptionData, long length) { - this.stream = stream; - totalBytesLeftToRead = length; + this._stream = stream; + _totalBytesLeftToRead = length; - cipher = CreateCipher(winzipAesEncryptionData); + _cipher = CreateCipher(winzipAesEncryptionData); var iv = new byte[BLOCK_SIZE_IN_BYTES]; - transform = cipher.CreateEncryptor(winzipAesEncryptionData.KeyBytes, iv); + _transform = _cipher.CreateEncryptor(winzipAesEncryptionData.KeyBytes, iv); } private SymmetricAlgorithm CreateCipher(WinzipAesEncryptionData winzipAesEncryptionData) @@ -69,17 +69,17 @@ namespace SharpCompress.Common.Zip protected override void Dispose(bool disposing) { - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; + _isDisposed = true; if (disposing) { //read out last 10 auth bytes var ten = new byte[10]; - stream.ReadFully(ten); - stream.Dispose(); + _stream.ReadFully(ten); + _stream.Dispose(); } } @@ -90,17 +90,17 @@ namespace SharpCompress.Common.Zip public override int Read(byte[] buffer, int offset, int count) { - if (totalBytesLeftToRead == 0) + if (_totalBytesLeftToRead == 0) { return 0; } int bytesToRead = count; - if (count > totalBytesLeftToRead) + if (count > _totalBytesLeftToRead) { - bytesToRead = (int)totalBytesLeftToRead; + bytesToRead = (int)_totalBytesLeftToRead; } - int read = stream.Read(buffer, offset, bytesToRead); - totalBytesLeftToRead -= read; + int read = _stream.Read(buffer, offset, bytesToRead); + _totalBytesLeftToRead -= read; ReadTransformBlocks(buffer, offset, read); @@ -109,7 +109,7 @@ namespace SharpCompress.Common.Zip private int ReadTransformOneBlock(byte[] buffer, int offset, int last) { - if (isFinalBlock) + if (_isFinalBlock) { throw new InvalidOperationException(); } @@ -120,22 +120,22 @@ namespace SharpCompress.Common.Zip : bytesRemaining; // update the counter - DataConverter.LittleEndian.PutBytes(counter, 0, nonce++); + DataConverter.LittleEndian.PutBytes(_counter, 0, _nonce++); // Determine if this is the final block - if ((bytesToRead == bytesRemaining) && (totalBytesLeftToRead == 0)) + if ((bytesToRead == bytesRemaining) && (_totalBytesLeftToRead == 0)) { - counterOut = transform.TransformFinalBlock(counter, + _counterOut = _transform.TransformFinalBlock(_counter, 0, BLOCK_SIZE_IN_BYTES); - isFinalBlock = true; + _isFinalBlock = true; } else { - transform.TransformBlock(counter, + _transform.TransformBlock(_counter, 0, // offset BLOCK_SIZE_IN_BYTES, - counterOut, + _counterOut, 0); // offset } @@ -148,7 +148,7 @@ namespace SharpCompress.Common.Zip { for (int i = 0; i < count; i++) { - buffer[offset + i] = (byte)(counterOut[i] ^ buffer[offset + i]); + buffer[offset + i] = (byte)(_counterOut[i] ^ buffer[offset + i]); } } diff --git a/src/SharpCompress/Common/Zip/WinzipAesEncryptionData.cs b/src/SharpCompress/Common/Zip/WinzipAesEncryptionData.cs index 1966704c..49f3dd24 100644 --- a/src/SharpCompress/Common/Zip/WinzipAesEncryptionData.cs +++ b/src/SharpCompress/Common/Zip/WinzipAesEncryptionData.cs @@ -10,20 +10,20 @@ namespace SharpCompress.Common.Zip { private const int RFC2898_ITERATIONS = 1000; - private byte[] salt; - private WinzipAesKeySize keySize; - private byte[] passwordVerifyValue; - private string password; + private readonly byte[] _salt; + private readonly WinzipAesKeySize _keySize; + private readonly byte[] _passwordVerifyValue; + private readonly string _password; - private byte[] generatedVerifyValue; + private byte[] _generatedVerifyValue; internal WinzipAesEncryptionData(WinzipAesKeySize keySize, byte[] salt, byte[] passwordVerifyValue, string password) { - this.keySize = keySize; - this.salt = salt; - this.passwordVerifyValue = passwordVerifyValue; - this.password = password; + this._keySize = keySize; + this._salt = salt; + this._passwordVerifyValue = passwordVerifyValue; + this._password = password; Initialize(); } @@ -38,7 +38,7 @@ namespace SharpCompress.Common.Zip private int KeySizeInBytes { - get { return KeyLengthInBytes(keySize); + get { return KeyLengthInBytes(_keySize); } } @@ -58,16 +58,16 @@ namespace SharpCompress.Common.Zip private void Initialize() { - var rfc2898 = new Rfc2898DeriveBytes(password, salt, RFC2898_ITERATIONS); + var rfc2898 = new Rfc2898DeriveBytes(_password, _salt, RFC2898_ITERATIONS); KeyBytes = rfc2898.GetBytes(KeySizeInBytes); // 16 or 24 or 32 ??? IvBytes = rfc2898.GetBytes(KeySizeInBytes); - generatedVerifyValue = rfc2898.GetBytes(2); + _generatedVerifyValue = rfc2898.GetBytes(2); - short verify = DataConverter.LittleEndian.GetInt16(passwordVerifyValue, 0); - if (password != null) + short verify = DataConverter.LittleEndian.GetInt16(_passwordVerifyValue, 0); + if (_password != null) { - short generated = DataConverter.LittleEndian.GetInt16(generatedVerifyValue, 0); + short generated = DataConverter.LittleEndian.GetInt16(_generatedVerifyValue, 0); if (verify != generated) { throw new InvalidFormatException("bad password"); diff --git a/src/SharpCompress/Common/Zip/ZipEntry.cs b/src/SharpCompress/Common/Zip/ZipEntry.cs index 8b8c7337..379534ff 100644 --- a/src/SharpCompress/Common/Zip/ZipEntry.cs +++ b/src/SharpCompress/Common/Zip/ZipEntry.cs @@ -6,13 +6,13 @@ namespace SharpCompress.Common.Zip { public class ZipEntry : Entry { - private readonly ZipFilePart filePart; + private readonly ZipFilePart _filePart; internal ZipEntry(ZipFilePart filePart) { if (filePart != null) { - this.filePart = filePart; + this._filePart = filePart; LastModifiedTime = Utility.DosDateToDateTime(filePart.Header.LastModifiedDate, filePart.Header.LastModifiedTime); } @@ -22,7 +22,7 @@ namespace SharpCompress.Common.Zip { get { - switch (filePart.Header.CompressionMethod) + switch (_filePart.Header.CompressionMethod) { case ZipCompressionMethod.BZip2: { @@ -32,6 +32,10 @@ namespace SharpCompress.Common.Zip { return CompressionType.Deflate; } + case ZipCompressionMethod.Deflate64: + { + return CompressionType.Deflate64; + } case ZipCompressionMethod.LZMA: { return CompressionType.LZMA; @@ -52,13 +56,13 @@ namespace SharpCompress.Common.Zip } } - public override long Crc => filePart.Header.Crc; + public override long Crc => _filePart.Header.Crc; - public override string Key => filePart.Header.Name; + public override string Key => _filePart.Header.Name; - public override long CompressedSize => filePart.Header.CompressedSize; + public override long CompressedSize => _filePart.Header.CompressedSize; - public override long Size => filePart.Header.UncompressedSize; + public override long Size => _filePart.Header.UncompressedSize; public override DateTime? LastModifiedTime { get; } @@ -68,12 +72,12 @@ namespace SharpCompress.Common.Zip public override DateTime? ArchivedTime => null; - public override bool IsEncrypted => FlagUtility.HasFlag(filePart.Header.Flags, HeaderFlags.Encrypted); + public override bool IsEncrypted => FlagUtility.HasFlag(_filePart.Header.Flags, HeaderFlags.Encrypted); - public override bool IsDirectory => filePart.Header.IsDirectory; + public override bool IsDirectory => _filePart.Header.IsDirectory; - public override bool IsSplit => false; + public override bool IsSplitAfter => false; - internal override IEnumerable Parts => filePart.AsEnumerable(); + internal override IEnumerable Parts => _filePart.AsEnumerable(); } } \ No newline at end of file diff --git a/src/SharpCompress/Common/Zip/ZipFilePart.cs b/src/SharpCompress/Common/Zip/ZipFilePart.cs index 1f626b0a..f68a1f6f 100644 --- a/src/SharpCompress/Common/Zip/ZipFilePart.cs +++ b/src/SharpCompress/Common/Zip/ZipFilePart.cs @@ -5,6 +5,7 @@ using SharpCompress.Common.Zip.Headers; using SharpCompress.Compressors; using SharpCompress.Compressors.BZip2; using SharpCompress.Compressors.Deflate; +using SharpCompress.Compressors.Deflate64; using SharpCompress.Compressors.LZMA; using SharpCompress.Compressors.PPMd; using SharpCompress.Converters; @@ -66,9 +67,13 @@ namespace SharpCompress.Common.Zip { return new DeflateStream(stream, CompressionMode.Decompress); } + case ZipCompressionMethod.Deflate64: + { + return new Deflate64Stream(stream, CompressionMode.Decompress); + } case ZipCompressionMethod.BZip2: { - return new BZip2Stream(stream, CompressionMode.Decompress); + return new BZip2Stream(stream, CompressionMode.Decompress, false); } case ZipCompressionMethod.LZMA: { diff --git a/src/SharpCompress/Common/Zip/ZipHeaderFactory.cs b/src/SharpCompress/Common/Zip/ZipHeaderFactory.cs index 9fcf9348..bc8187c2 100644 --- a/src/SharpCompress/Common/Zip/ZipHeaderFactory.cs +++ b/src/SharpCompress/Common/Zip/ZipHeaderFactory.cs @@ -21,16 +21,16 @@ namespace SharpCompress.Common.Zip internal const uint ZIP64_END_OF_CENTRAL_DIRECTORY = 0x06064b50; internal const uint ZIP64_END_OF_CENTRAL_DIRECTORY_LOCATOR = 0x07064b50; - protected LocalEntryHeader lastEntryHeader; - private readonly string password; - private readonly StreamingMode mode; - private readonly ArchiveEncoding archiveEncoding; + protected LocalEntryHeader _lastEntryHeader; + private readonly string _password; + private readonly StreamingMode _mode; + private readonly ArchiveEncoding _archiveEncoding; protected ZipHeaderFactory(StreamingMode mode, string password, ArchiveEncoding archiveEncoding) { - this.mode = mode; - this.password = password; - this.archiveEncoding = archiveEncoding; + this._mode = mode; + this._password = password; + this._archiveEncoding = archiveEncoding; } protected ZipHeader ReadHeader(uint headerBytes, BinaryReader reader, bool zip64 = false) @@ -39,26 +39,26 @@ namespace SharpCompress.Common.Zip { case ENTRY_HEADER_BYTES: { - var entryHeader = new LocalEntryHeader(archiveEncoding); + var entryHeader = new LocalEntryHeader(_archiveEncoding); entryHeader.Read(reader); LoadHeader(entryHeader, reader.BaseStream); - lastEntryHeader = entryHeader; + _lastEntryHeader = entryHeader; return entryHeader; } case DIRECTORY_START_HEADER_BYTES: { - var entry = new DirectoryEntryHeader(archiveEncoding); + var entry = new DirectoryEntryHeader(_archiveEncoding); entry.Read(reader); return entry; } case POST_DATA_DESCRIPTOR: { - if (FlagUtility.HasFlag(lastEntryHeader.Flags, HeaderFlags.UsePostDataDescriptor)) + if (FlagUtility.HasFlag(_lastEntryHeader.Flags, HeaderFlags.UsePostDataDescriptor)) { - lastEntryHeader.Crc = reader.ReadUInt32(); - lastEntryHeader.CompressedSize = zip64 ? (long)reader.ReadUInt64() : reader.ReadUInt32(); - lastEntryHeader.UncompressedSize = zip64 ? (long)reader.ReadUInt64() : reader.ReadUInt32(); + _lastEntryHeader.Crc = reader.ReadUInt32(); + _lastEntryHeader.CompressedSize = zip64 ? (long)reader.ReadUInt64() : reader.ReadUInt32(); + _lastEntryHeader.UncompressedSize = zip64 ? (long)reader.ReadUInt64() : reader.ReadUInt32(); } else { @@ -123,12 +123,12 @@ namespace SharpCompress.Common.Zip throw new NotSupportedException("SharpCompress cannot currently read non-seekable Zip Streams with encrypted data that has been written in a non-seekable manner."); } - if (password == null) + if (_password == null) { throw new CryptographicException("No password supplied for encrypted zip."); } - entryHeader.Password = password; + entryHeader.Password = _password; if (entryHeader.CompressionMethod == ZipCompressionMethod.WinzipAes) { @@ -146,7 +146,7 @@ namespace SharpCompress.Common.Zip stream.Read(salt, 0, salt.Length); stream.Read(passwordVerifyValue, 0, 2); entryHeader.WinzipAesEncryptionData = - new WinzipAesEncryptionData(keySize, salt, passwordVerifyValue, password); + new WinzipAesEncryptionData(keySize, salt, passwordVerifyValue, _password); entryHeader.CompressedSize -= (uint)(salt.Length + 2); } @@ -165,7 +165,7 @@ namespace SharpCompress.Common.Zip //} //else //{ - switch (mode) + switch (_mode) { case StreamingMode.Seekable: { diff --git a/src/SharpCompress/Compressors/ADC/ADCBase.cs b/src/SharpCompress/Compressors/ADC/ADCBase.cs index 59b42016..dfca000e 100644 --- a/src/SharpCompress/Compressors/ADC/ADCBase.cs +++ b/src/SharpCompress/Compressors/ADC/ADCBase.cs @@ -33,47 +33,47 @@ namespace SharpCompress.Compressors.ADC /// public static class ADCBase { - const int Plain = 1; - const int TwoByte = 2; - const int ThreeByte = 3; + private const int PLAIN = 1; + private const int TWO_BYTE = 2; + private const int THREE_BYTE = 3; - static int GetChunkType(byte byt) + private static int GetChunkType(byte byt) { if ((byt & 0x80) == 0x80) { - return Plain; + return PLAIN; } if ((byt & 0x40) == 0x40) { - return ThreeByte; + return THREE_BYTE; } - return TwoByte; + return TWO_BYTE; } - static int GetChunkSize(byte byt) + private static int GetChunkSize(byte byt) { switch (GetChunkType(byt)) { - case Plain: + case PLAIN: return (byt & 0x7F) + 1; - case TwoByte: + case TWO_BYTE: return ((byt & 0x3F) >> 2) + 3; - case ThreeByte: + case THREE_BYTE: return (byt & 0x3F) + 4; default: return -1; } } - static int GetOffset(byte[] chunk, int position) + private static int GetOffset(byte[] chunk, int position) { switch (GetChunkType(chunk[position])) { - case Plain: + case PLAIN: return 0; - case TwoByte: + case TWO_BYTE: return ((chunk[position] & 0x03) << 8) + chunk[position + 1]; - case ThreeByte: + case THREE_BYTE: return (chunk[position + 1] << 8) + chunk[position + 2]; default: return -1; @@ -130,7 +130,7 @@ namespace SharpCompress.Compressors.ADC switch (chunkType) { - case Plain: + case PLAIN: chunkSize = GetChunkSize((byte)readByte); if (outPosition + chunkSize > bufferSize) { @@ -141,7 +141,7 @@ namespace SharpCompress.Compressors.ADC outPosition += chunkSize; position += chunkSize + 1; break; - case TwoByte: + case TWO_BYTE: tempMs = new MemoryStream(); chunkSize = GetChunkSize((byte)readByte); tempMs.WriteByte((byte)readByte); @@ -172,7 +172,7 @@ namespace SharpCompress.Compressors.ADC position += 2; } break; - case ThreeByte: + case THREE_BYTE: tempMs = new MemoryStream(); chunkSize = GetChunkSize((byte)readByte); tempMs.WriteByte((byte)readByte); diff --git a/src/SharpCompress/Compressors/ADC/ADCStream.cs b/src/SharpCompress/Compressors/ADC/ADCStream.cs index 0db58080..d8799fdd 100644 --- a/src/SharpCompress/Compressors/ADC/ADCStream.cs +++ b/src/SharpCompress/Compressors/ADC/ADCStream.cs @@ -36,27 +36,27 @@ namespace SharpCompress.Compressors.ADC /// /// This stream holds the compressed data /// - private readonly Stream stream; + private readonly Stream _stream; /// /// Is this instance disposed? /// - private bool isDisposed; + private bool _isDisposed; /// /// Position in decompressed data /// - private long position; + private long _position; /// /// Buffer with currently used chunk of decompressed data /// - private byte[] outBuffer; + private byte[] _outBuffer; /// /// Position in buffer of decompressed data /// - private int outPosition; + private int _outPosition; /// /// Initializates a stream that decompresses ADC data on the fly @@ -70,10 +70,10 @@ namespace SharpCompress.Compressors.ADC throw new NotSupportedException(); } - this.stream = stream; + _stream = stream; } - public override bool CanRead => stream.CanRead; + public override bool CanRead => _stream.CanRead; public override bool CanSeek => false; @@ -81,7 +81,7 @@ namespace SharpCompress.Compressors.ADC public override long Length => throw new NotSupportedException(); - public override long Position { get => position; set => throw new NotSupportedException(); } + public override long Position { get => _position; set => throw new NotSupportedException(); } public override void Flush() { @@ -89,11 +89,11 @@ namespace SharpCompress.Compressors.ADC protected override void Dispose(bool disposing) { - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; + _isDisposed = true; base.Dispose(disposing); } @@ -122,35 +122,35 @@ namespace SharpCompress.Compressors.ADC int size = -1; - if (outBuffer == null) + if (_outBuffer == null) { - size = ADCBase.Decompress(stream, out outBuffer); - outPosition = 0; + size = ADCBase.Decompress(_stream, out _outBuffer); + _outPosition = 0; } int inPosition = offset; int toCopy = count; int copied = 0; - while (outPosition + toCopy >= outBuffer.Length) + while (_outPosition + toCopy >= _outBuffer.Length) { - int piece = outBuffer.Length - outPosition; - Array.Copy(outBuffer, outPosition, buffer, inPosition, piece); + int piece = _outBuffer.Length - _outPosition; + Array.Copy(_outBuffer, _outPosition, buffer, inPosition, piece); inPosition += piece; copied += piece; - position += piece; + _position += piece; toCopy -= piece; - size = ADCBase.Decompress(stream, out outBuffer); - outPosition = 0; - if (size == 0 || outBuffer == null || outBuffer.Length == 0) + size = ADCBase.Decompress(_stream, out _outBuffer); + _outPosition = 0; + if (size == 0 || _outBuffer == null || _outBuffer.Length == 0) { return copied; } } - Array.Copy(outBuffer, outPosition, buffer, inPosition, toCopy); - outPosition += toCopy; - position += toCopy; + Array.Copy(_outBuffer, _outPosition, buffer, inPosition, toCopy); + _outPosition += toCopy; + _position += toCopy; copied += toCopy; return copied; } diff --git a/src/SharpCompress/Compressors/BZip2/BZip2Stream.cs b/src/SharpCompress/Compressors/BZip2/BZip2Stream.cs index 590313d3..c0f040a9 100644 --- a/src/SharpCompress/Compressors/BZip2/BZip2Stream.cs +++ b/src/SharpCompress/Compressors/BZip2/BZip2Stream.cs @@ -12,19 +12,18 @@ namespace SharpCompress.Compressors.BZip2 /// /// The stream to read from /// Compression Mode - /// Leave the underlying stream open when disposed. - /// Should the BZip2 stream continue to decompress the stream when the End Marker is found. - public BZip2Stream(Stream stream, CompressionMode compressionMode, bool leaveOpen = false, - bool decompressContacted = false) + /// Decompress Concatenated + public BZip2Stream(Stream stream, CompressionMode compressionMode, + bool decompressConcatenated) { Mode = compressionMode; if (Mode == CompressionMode.Compress) { - this.stream = new CBZip2OutputStream(stream, leaveOpen); + this.stream = new CBZip2OutputStream(stream); } else { - this.stream = new CBZip2InputStream(stream, decompressContacted, leaveOpen); + this.stream = new CBZip2InputStream(stream, decompressConcatenated); } } @@ -68,6 +67,11 @@ namespace SharpCompress.Compressors.BZip2 return stream.Read(buffer, offset, count); } + public override int ReadByte() + { + return stream.ReadByte(); + } + public override long Seek(long offset, SeekOrigin origin) { return stream.Seek(offset, origin); @@ -83,6 +87,11 @@ namespace SharpCompress.Compressors.BZip2 stream.Write(buffer, offset, count); } + public override void WriteByte(byte value) + { + stream.WriteByte(value); + } + /// /// Consumes two bytes to test if there is a BZip2 header /// diff --git a/src/SharpCompress/Compressors/BZip2/CBZip2InputStream.cs b/src/SharpCompress/Compressors/BZip2/CBZip2InputStream.cs index 0656801e..68d924b4 100644 --- a/src/SharpCompress/Compressors/BZip2/CBZip2InputStream.cs +++ b/src/SharpCompress/Compressors/BZip2/CBZip2InputStream.cs @@ -119,7 +119,6 @@ namespace SharpCompress.Compressors.BZip2 private readonly int[] minLens = new int[BZip2Constants.N_GROUPS]; private Stream bsStream; - private bool leaveOpen; private bool streamEnd; @@ -147,12 +146,12 @@ namespace SharpCompress.Compressors.BZip2 private char z; private bool isDisposed; - public CBZip2InputStream(Stream zStream, bool decompressConcatenated, bool leaveOpen) + public CBZip2InputStream(Stream zStream, bool decompressConcatenated) { this.decompressConcatenated = decompressConcatenated; ll8 = null; tt = null; - BsSetStream(zStream, leaveOpen); + BsSetStream(zStream); Initialize(true); InitBlock(); SetupBlock(); @@ -354,29 +353,15 @@ namespace SharpCompress.Compressors.BZip2 private void BsFinishedWithStream() { - try - { - if (bsStream != null) - { - if (!leaveOpen) - { - bsStream.Dispose(); - } - bsStream = null; - } - } - catch - { - //ignore - } + bsStream?.Dispose(); + bsStream = null; } - private void BsSetStream(Stream f, bool leaveOpen) + private void BsSetStream(Stream f) { bsStream = f; bsLive = 0; bsBuff = 0; - this.leaveOpen = leaveOpen; } private int BsR(int n) @@ -1092,6 +1077,10 @@ namespace SharpCompress.Compressors.BZip2 { } + public override void WriteByte(byte value) + { + } + public override bool CanRead => true; public override bool CanSeek => false; diff --git a/src/SharpCompress/Compressors/BZip2/CBZip2OutputStream.cs b/src/SharpCompress/Compressors/BZip2/CBZip2OutputStream.cs index 9f4817cc..e95ddb36 100644 --- a/src/SharpCompress/Compressors/BZip2/CBZip2OutputStream.cs +++ b/src/SharpCompress/Compressors/BZip2/CBZip2OutputStream.cs @@ -303,16 +303,11 @@ namespace SharpCompress.Compressors.BZip2 private int runLength; public CBZip2OutputStream(Stream inStream) - : this(inStream, 9, false) + : this(inStream, 9) { } - public CBZip2OutputStream(Stream inStream, bool leaveOpen) - : this(inStream, 9, leaveOpen) - { - } - - public CBZip2OutputStream(Stream inStream, int inBlockSize, bool leaveOpen) + public CBZip2OutputStream(Stream inStream, int inBlockSize) { block = null; quadrant = null; @@ -322,7 +317,7 @@ namespace SharpCompress.Compressors.BZip2 inStream.WriteByte((byte)'B'); inStream.WriteByte((byte)'Z'); - BsSetStream(inStream, leaveOpen); + BsSetStream(inStream); workFactor = 50; if (inBlockSize > 9) @@ -441,10 +436,7 @@ namespace SharpCompress.Compressors.BZip2 disposed = true; base.Dispose(); - if (!leaveOpen) - { - bsStream.Dispose(); - } + bsStream?.Dispose(); bsStream = null; } } @@ -596,13 +588,12 @@ namespace SharpCompress.Compressors.BZip2 } } - private void BsSetStream(Stream f, bool leaveOpen) + private void BsSetStream(Stream f) { bsStream = f; bsLive = 0; bsBuff = 0; bytesOut = 0; - this.leaveOpen = leaveOpen; } private void BsFinishedWithStream() @@ -1057,7 +1048,6 @@ namespace SharpCompress.Compressors.BZip2 } private Stream bsStream; - private bool leaveOpen; private void SimpleSort(int lo, int hi, int d) { @@ -1939,6 +1929,11 @@ namespace SharpCompress.Compressors.BZip2 return 0; } + public override int ReadByte() + { + return -1; + } + public override long Seek(long offset, SeekOrigin origin) { return 0; diff --git a/src/SharpCompress/Compressors/Deflate/DeflateStream.cs b/src/SharpCompress/Compressors/Deflate/DeflateStream.cs index d041533e..e095f460 100644 --- a/src/SharpCompress/Compressors/Deflate/DeflateStream.cs +++ b/src/SharpCompress/Compressors/Deflate/DeflateStream.cs @@ -37,10 +37,9 @@ namespace SharpCompress.Compressors.Deflate public DeflateStream(Stream stream, CompressionMode mode, CompressionLevel level = CompressionLevel.Default, - bool leaveOpen = false, Encoding forceEncoding = null) { - _baseStream = new ZlibBaseStream(stream, mode, level, ZlibStreamFlavor.DEFLATE, leaveOpen, forceEncoding); + _baseStream = new ZlibBaseStream(stream, mode, level, ZlibStreamFlavor.DEFLATE, forceEncoding); } #region Zlib properties @@ -216,7 +215,6 @@ namespace SharpCompress.Compressors.Deflate /// /// /// This may or may not result in a Close() call on the captive stream. - /// See the constructors that have a leaveOpen parameter for more information. /// protected override void Dispose(bool disposing) { @@ -224,9 +222,9 @@ namespace SharpCompress.Compressors.Deflate { if (!_disposed) { - if (disposing && (_baseStream != null)) + if (disposing) { - _baseStream.Dispose(); + _baseStream?.Dispose(); } _disposed = true; } @@ -284,6 +282,15 @@ namespace SharpCompress.Compressors.Deflate return _baseStream.Read(buffer, offset, count); } + public override int ReadByte() + { + if (_disposed) + { + throw new ObjectDisposedException("DeflateStream"); + } + return _baseStream.ReadByte(); + } + /// /// Calling this method always throws a . /// @@ -342,6 +349,15 @@ namespace SharpCompress.Compressors.Deflate _baseStream.Write(buffer, offset, count); } + public override void WriteByte(byte value) + { + if (_disposed) + { + throw new ObjectDisposedException("DeflateStream"); + } + _baseStream.WriteByte(value); + } + #endregion public MemoryStream InputBuffer => new MemoryStream(_baseStream._z.InputBuffer, _baseStream._z.NextIn, diff --git a/src/SharpCompress/Compressors/Deflate/GZipStream.cs b/src/SharpCompress/Compressors/Deflate/GZipStream.cs index 6784b6d4..decbc8ad 100644 --- a/src/SharpCompress/Compressors/Deflate/GZipStream.cs +++ b/src/SharpCompress/Compressors/Deflate/GZipStream.cs @@ -51,23 +51,18 @@ namespace SharpCompress.Compressors.Deflate private readonly Encoding _encoding; public GZipStream(Stream stream, CompressionMode mode) - : this(stream, mode, CompressionLevel.Default, false, Encoding.UTF8) + : this(stream, mode, CompressionLevel.Default, Encoding.UTF8) { } public GZipStream(Stream stream, CompressionMode mode, CompressionLevel level) - : this(stream, mode, level, false, Encoding.UTF8) + : this(stream, mode, level, Encoding.UTF8) { } - public GZipStream(Stream stream, CompressionMode mode, bool leaveOpen) - : this(stream, mode, CompressionLevel.Default, leaveOpen, Encoding.UTF8) + public GZipStream(Stream stream, CompressionMode mode, CompressionLevel level, Encoding encoding) { - } - - public GZipStream(Stream stream, CompressionMode mode, CompressionLevel level, bool leaveOpen, Encoding encoding) - { - BaseStream = new ZlibBaseStream(stream, mode, level, ZlibStreamFlavor.GZIP, leaveOpen, encoding); + BaseStream = new ZlibBaseStream(stream, mode, level, ZlibStreamFlavor.GZIP, encoding); _encoding = encoding; } @@ -200,7 +195,6 @@ namespace SharpCompress.Compressors.Deflate /// /// /// This may or may not result in a Close() call on the captive stream. - /// See the doc on constructors that take a leaveOpen parameter for more information. /// protected override void Dispose(bool disposing) { diff --git a/src/SharpCompress/Compressors/Deflate/Zlib.cs b/src/SharpCompress/Compressors/Deflate/Zlib.cs index 8cfaeac8..1badf10b 100644 --- a/src/SharpCompress/Compressors/Deflate/Zlib.cs +++ b/src/SharpCompress/Compressors/Deflate/Zlib.cs @@ -418,7 +418,7 @@ namespace SharpCompress.Compressors.Deflate internal sealed class Adler { // largest prime smaller than 65536 - private static readonly int BASE = 65521; + private static readonly uint BASE = 65521U; // NMAX is the largest n such that 255n(n+1)/2 + (n+1)(BASE-1) <= 2^32-1 private static readonly int NMAX = 5552; @@ -430,8 +430,8 @@ namespace SharpCompress.Compressors.Deflate return 1; } - int s1 = (int)(adler & 0xffff); - int s2 = (int)((adler >> 16) & 0xffff); + uint s1 = adler & 0xffffU; + uint s2 = (adler >> 16) & 0xffffU; while (len > 0) { @@ -486,7 +486,7 @@ namespace SharpCompress.Compressors.Deflate s1 %= BASE; s2 %= BASE; } - return (uint)((s2 << 16) | s1); + return (s2 << 16) | s1; } } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/Deflate/ZlibBaseStream.cs b/src/SharpCompress/Compressors/Deflate/ZlibBaseStream.cs index 875717ec..e499eeb7 100644 --- a/src/SharpCompress/Compressors/Deflate/ZlibBaseStream.cs +++ b/src/SharpCompress/Compressors/Deflate/ZlibBaseStream.cs @@ -50,7 +50,6 @@ namespace SharpCompress.Compressors.Deflate protected internal ZlibStreamFlavor _flavor; protected internal CompressionMode _compressionMode; protected internal CompressionLevel _level; - protected internal bool _leaveOpen; protected internal byte[] _workingBuffer; protected internal int _bufferSize = ZlibConstants.WorkingBufferSizeDefault; protected internal byte[] _buf1 = new byte[1]; @@ -83,14 +82,12 @@ namespace SharpCompress.Compressors.Deflate CompressionMode compressionMode, CompressionLevel level, ZlibStreamFlavor flavor, - bool leaveOpen, Encoding encoding) { _flushMode = FlushType.None; //this._workingBuffer = new byte[WORKING_BUFFER_SIZE_DEFAULT]; _stream = stream; - _leaveOpen = leaveOpen; _compressionMode = compressionMode; _flavor = flavor; _level = level; @@ -361,10 +358,7 @@ namespace SharpCompress.Compressors.Deflate finally { end(); - if (!_leaveOpen) - { - _stream.Dispose(); - } + _stream?.Dispose(); _stream = null; } } @@ -453,7 +447,7 @@ namespace SharpCompress.Compressors.Deflate } Int32 timet = DataConverter.LittleEndian.GetInt32(header, 4); - _GzipMtime = TarHeader.Epoch.AddSeconds(timet); + _GzipMtime = TarHeader.EPOCH.AddSeconds(timet); totalBytesRead += n; if ((header[3] & 0x04) == 0x04) { diff --git a/src/SharpCompress/Compressors/Deflate/ZlibStream.cs b/src/SharpCompress/Compressors/Deflate/ZlibStream.cs index b94ceea8..c82040ef 100644 --- a/src/SharpCompress/Compressors/Deflate/ZlibStream.cs +++ b/src/SharpCompress/Compressors/Deflate/ZlibStream.cs @@ -37,23 +37,18 @@ namespace SharpCompress.Compressors.Deflate private bool _disposed; public ZlibStream(Stream stream, CompressionMode mode) - : this(stream, mode, CompressionLevel.Default, false, Encoding.UTF8) + : this(stream, mode, CompressionLevel.Default, Encoding.UTF8) { } public ZlibStream(Stream stream, CompressionMode mode, CompressionLevel level) - : this(stream, mode, level, false, Encoding.UTF8) + : this(stream, mode, level, Encoding.UTF8) { } - public ZlibStream(Stream stream, CompressionMode mode, bool leaveOpen) - : this(stream, mode, CompressionLevel.Default, leaveOpen, Encoding.UTF8) + public ZlibStream(Stream stream, CompressionMode mode, CompressionLevel level, Encoding encoding) { - } - - public ZlibStream(Stream stream, CompressionMode mode, CompressionLevel level, bool leaveOpen, Encoding encoding) - { - _baseStream = new ZlibBaseStream(stream, mode, level, ZlibStreamFlavor.ZLIB, leaveOpen, encoding); + _baseStream = new ZlibBaseStream(stream, mode, level, ZlibStreamFlavor.ZLIB, encoding); } #region Zlib properties @@ -208,7 +203,6 @@ namespace SharpCompress.Compressors.Deflate /// /// /// This may or may not result in a Close() call on the captive stream. - /// See the constructors that have a leaveOpen parameter for more information. /// protected override void Dispose(bool disposing) { @@ -216,9 +210,9 @@ namespace SharpCompress.Compressors.Deflate { if (!_disposed) { - if (disposing && (_baseStream != null)) + if (disposing) { - _baseStream.Dispose(); + _baseStream?.Dispose(); } _disposed = true; } @@ -276,6 +270,15 @@ namespace SharpCompress.Compressors.Deflate return _baseStream.Read(buffer, offset, count); } + public override int ReadByte() + { + if (_disposed) + { + throw new ObjectDisposedException("ZlibStream"); + } + return _baseStream.ReadByte(); + } + /// /// Calling this method always throws a . /// @@ -327,6 +330,15 @@ namespace SharpCompress.Compressors.Deflate _baseStream.Write(buffer, offset, count); } + public override void WriteByte(byte value) + { + if (_disposed) + { + throw new ObjectDisposedException("ZlibStream"); + } + _baseStream.WriteByte(value); + } + #endregion System.IO.Stream methods } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/Deflate64/BlockType.cs b/src/SharpCompress/Compressors/Deflate64/BlockType.cs new file mode 100644 index 00000000..c8938079 --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/BlockType.cs @@ -0,0 +1,13 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +namespace SharpCompress.Compressors.Deflate64 +{ + internal enum BlockType + { + Uncompressed = 0, + Static = 1, + Dynamic = 2 + } +} diff --git a/src/SharpCompress/Compressors/Deflate64/Deflate64Stream.cs b/src/SharpCompress/Compressors/Deflate64/Deflate64Stream.cs new file mode 100644 index 00000000..be7300c2 --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/Deflate64Stream.cs @@ -0,0 +1,255 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +using SharpCompress.Common.Zip; +using SharpCompress.Compressors.Deflate; +using System; +using System.Diagnostics; +using System.IO; +using System.Runtime.CompilerServices; + +namespace SharpCompress.Compressors.Deflate64 +{ + public sealed class Deflate64Stream : Stream + { + private const int DEFAULT_BUFFER_SIZE = 8192; + + private Stream _stream; + private CompressionMode _mode; + private InflaterManaged _inflater; + private byte[] _buffer; + + public Deflate64Stream(Stream stream, CompressionMode mode) + { + if (stream == null) + throw new ArgumentNullException(nameof(stream)); + if (mode != CompressionMode.Decompress) + throw new NotImplementedException("Deflate64: this implementation only supports decompression"); + if (!stream.CanRead) + throw new ArgumentException("Deflate64: input stream is not readable", nameof(stream)); + + InitializeInflater(stream, ZipCompressionMethod.Deflate64); + } + + /// + /// Sets up this DeflateManagedStream to be used for Inflation/Decompression + /// + private void InitializeInflater(Stream stream, ZipCompressionMethod method = ZipCompressionMethod.Deflate) + { + Debug.Assert(stream != null); + Debug.Assert(method == ZipCompressionMethod.Deflate || method == ZipCompressionMethod.Deflate64); + if (!stream.CanRead) + throw new ArgumentException("Deflate64: input stream is not readable", nameof(stream)); + + _inflater = new InflaterManaged(method == ZipCompressionMethod.Deflate64); + + _stream = stream; + _mode = CompressionMode.Decompress; + _buffer = new byte[DEFAULT_BUFFER_SIZE]; + } + + public override bool CanRead + { + get + { + if (_stream == null) + { + return false; + } + + return (_mode == CompressionMode.Decompress && _stream.CanRead); + } + } + + public override bool CanWrite + { + get + { + if (_stream == null) + { + return false; + } + + return (_mode == CompressionMode.Compress && _stream.CanWrite); + } + } + + public override bool CanSeek => false; + + public override long Length + { + get { throw new NotSupportedException("Deflate64: not supported"); } + } + + public override long Position + { + get { throw new NotSupportedException("Deflate64: not supported"); } + set { throw new NotSupportedException("Deflate64: not supported"); } + } + + public override void Flush() + { + EnsureNotDisposed(); + } + + public override long Seek(long offset, SeekOrigin origin) + { + throw new NotSupportedException("Deflate64: not supported"); + } + + public override void SetLength(long value) + { + throw new NotSupportedException("Deflate64: not supported"); + } + + public override int Read(byte[] array, int offset, int count) + { + EnsureDecompressionMode(); + ValidateParameters(array, offset, count); + EnsureNotDisposed(); + + int bytesRead; + int currentOffset = offset; + int remainingCount = count; + + while (true) + { + bytesRead = _inflater.Inflate(array, currentOffset, remainingCount); + currentOffset += bytesRead; + remainingCount -= bytesRead; + + if (remainingCount == 0) + { + break; + } + + if (_inflater.Finished()) + { + // if we finished decompressing, we can't have anything left in the outputwindow. + Debug.Assert(_inflater.AvailableOutput == 0, "We should have copied all stuff out!"); + break; + } + + int bytes = _stream.Read(_buffer, 0, _buffer.Length); + if (bytes <= 0) + { + break; + } + else if (bytes > _buffer.Length) + { + // The stream is either malicious or poorly implemented and returned a number of + // bytes larger than the buffer supplied to it. + throw new InvalidDataException("Deflate64: invalid data"); + } + + _inflater.SetInput(_buffer, 0, bytes); + } + + return count - remainingCount; + } + + private void ValidateParameters(byte[] array, int offset, int count) + { + if (array == null) + throw new ArgumentNullException(nameof(array)); + + if (offset < 0) + throw new ArgumentOutOfRangeException(nameof(offset)); + + if (count < 0) + throw new ArgumentOutOfRangeException(nameof(count)); + + if (array.Length - offset < count) + throw new ArgumentException("Deflate64: invalid offset/count combination"); + } + + private void EnsureNotDisposed() + { + if (_stream == null) + ThrowStreamClosedException(); + } + + [MethodImpl(MethodImplOptions.NoInlining)] + private static void ThrowStreamClosedException() + { + throw new ObjectDisposedException(null, "Deflate64: stream has been disposed"); + } + + private void EnsureDecompressionMode() + { + if (_mode != CompressionMode.Decompress) + ThrowCannotReadFromDeflateManagedStreamException(); + } + + [MethodImpl(MethodImplOptions.NoInlining)] + private static void ThrowCannotReadFromDeflateManagedStreamException() + { + throw new InvalidOperationException("Deflate64: cannot read from this stream"); + } + + private void EnsureCompressionMode() + { + if (_mode != CompressionMode.Compress) + ThrowCannotWriteToDeflateManagedStreamException(); + } + + [MethodImpl(MethodImplOptions.NoInlining)] + private static void ThrowCannotWriteToDeflateManagedStreamException() + { + throw new InvalidOperationException("Deflate64: cannot write to this stream"); + } + + public override void Write(byte[] array, int offset, int count) + { + ThrowCannotWriteToDeflateManagedStreamException(); + } + + // This is called by Dispose: + private void PurgeBuffers(bool disposing) + { + if (!disposing) + return; + + if (_stream == null) + return; + + Flush(); + } + + protected override void Dispose(bool disposing) + { + try + { + PurgeBuffers(disposing); + } + finally + { + // Close the underlying stream even if PurgeBuffers threw. + // Stream.Close() may throw here (may or may not be due to the same error). + // In this case, we still need to clean up internal resources, hence the inner finally blocks. + try + { + if (disposing) + { + _stream?.Dispose(); + } + } + finally + { + _stream = null; + + try + { + _inflater?.Dispose(); + } + finally + { + _inflater = null; + base.Dispose(disposing); + } + } + } + } + } +} diff --git a/src/SharpCompress/Compressors/Deflate64/DeflateInput.cs b/src/SharpCompress/Compressors/Deflate64/DeflateInput.cs new file mode 100644 index 00000000..faf26edb --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/DeflateInput.cs @@ -0,0 +1,43 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +using System.Diagnostics; + +namespace SharpCompress.Compressors.Deflate64 +{ + internal sealed class DeflateInput + { + internal byte[] Buffer { get; set; } + internal int Count { get; set; } + internal int StartIndex { get; set; } + + internal void ConsumeBytes(int n) + { + Debug.Assert(n <= Count, "Should use more bytes than what we have in the buffer"); + StartIndex += n; + Count -= n; + Debug.Assert(StartIndex + Count <= Buffer.Length, "Input buffer is in invalid state!"); + } + + internal InputState DumpState() => new InputState(Count, StartIndex); + + internal void RestoreState(InputState state) + { + Count = state._count; + StartIndex = state._startIndex; + } + + internal /*readonly */struct InputState + { + internal readonly int _count; + internal readonly int _startIndex; + + internal InputState(int count, int startIndex) + { + _count = count; + _startIndex = startIndex; + } + } + } +} diff --git a/src/SharpCompress/Compressors/Deflate64/FastEncoderStatus.cs b/src/SharpCompress/Compressors/Deflate64/FastEncoderStatus.cs new file mode 100644 index 00000000..ecfae42b --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/FastEncoderStatus.cs @@ -0,0 +1,245 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +using System.Diagnostics; + +namespace SharpCompress.Compressors.Deflate64 +{ + internal static class FastEncoderStatics + { + // static information for encoding, DO NOT MODIFY + + internal static readonly byte[] FAST_ENCODER_TREE_STRUCTURE_DATA = + { + 0xec,0xbd,0x07,0x60,0x1c,0x49,0x96,0x25,0x26,0x2f,0x6d,0xca, + 0x7b,0x7f,0x4a,0xf5,0x4a,0xd7,0xe0,0x74,0xa1,0x08,0x80,0x60, + 0x13,0x24,0xd8,0x90,0x40,0x10,0xec,0xc1,0x88,0xcd,0xe6,0x92, + 0xec,0x1d,0x69,0x47,0x23,0x29,0xab,0x2a,0x81,0xca,0x65,0x56, + 0x65,0x5d,0x66,0x16,0x40,0xcc,0xed,0x9d,0xbc,0xf7,0xde,0x7b, + 0xef,0xbd,0xf7,0xde,0x7b,0xef,0xbd,0xf7,0xba,0x3b,0x9d,0x4e, + 0x27,0xf7,0xdf,0xff,0x3f,0x5c,0x66,0x64,0x01,0x6c,0xf6,0xce, + 0x4a,0xda,0xc9,0x9e,0x21,0x80,0xaa,0xc8,0x1f,0x3f,0x7e,0x7c, + 0x1f,0x3f + }; + + internal static readonly byte[] B_FINAL_FAST_ENCODER_TREE_STRUCTURE_DATA = + { + 0xed,0xbd,0x07,0x60,0x1c,0x49,0x96,0x25,0x26,0x2f,0x6d,0xca, + 0x7b,0x7f,0x4a,0xf5,0x4a,0xd7,0xe0,0x74,0xa1,0x08,0x80,0x60, + 0x13,0x24,0xd8,0x90,0x40,0x10,0xec,0xc1,0x88,0xcd,0xe6,0x92, + 0xec,0x1d,0x69,0x47,0x23,0x29,0xab,0x2a,0x81,0xca,0x65,0x56, + 0x65,0x5d,0x66,0x16,0x40,0xcc,0xed,0x9d,0xbc,0xf7,0xde,0x7b, + 0xef,0xbd,0xf7,0xde,0x7b,0xef,0xbd,0xf7,0xba,0x3b,0x9d,0x4e, + 0x27,0xf7,0xdf,0xff,0x3f,0x5c,0x66,0x64,0x01,0x6c,0xf6,0xce, + 0x4a,0xda,0xc9,0x9e,0x21,0x80,0xaa,0xc8,0x1f,0x3f,0x7e,0x7c, + 0x1f,0x3f + }; + + // Output a currentMatch with length matchLen (>= MIN_MATCH) and displacement matchPos + // + // Optimisation: unlike the other encoders, here we have an array of codes for each currentMatch + // length (not just each currentMatch length slot), complete with all the extra bits filled in, in + // a single array element. + // + // There are many advantages to doing this: + // + // 1. A single array lookup on g_FastEncoderLiteralCodeInfo, instead of separate array lookups + // on g_LengthLookup (to get the length slot), g_FastEncoderLiteralTreeLength, + // g_FastEncoderLiteralTreeCode, g_ExtraLengthBits, and g_BitMask + // + // 2. The array is an array of ULONGs, so no access penalty, unlike for accessing those USHORT + // code arrays in the other encoders (although they could be made into ULONGs with some + // modifications to the source). + // + // Note, if we could guarantee that codeLen <= 16 always, then we could skip an if statement here. + // + // A completely different optimisation is used for the distance codes since, obviously, a table for + // all 8192 distances combining their extra bits is not feasible. The distance codeinfo table is + // made up of code[], len[] and # extraBits for this code. + // + // The advantages are similar to the above; a ULONG array instead of a USHORT and BYTE array, better + // cache locality, fewer memory operations. + // + + + // Encoding information for literal and Length. + // The least 5 significant bits are the length + // and the rest is the code bits. + + internal static readonly uint[] FAST_ENCODER_LITERAL_CODE_INFO = + { + 0x0000d7ee,0x0004d7ee,0x0002d7ee,0x0006d7ee,0x0001d7ee,0x0005d7ee,0x0003d7ee, + 0x0007d7ee,0x000037ee,0x0000c7ec,0x00000126,0x000437ee,0x000237ee,0x000637ee, + 0x000137ee,0x000537ee,0x000337ee,0x000737ee,0x0000b7ee,0x0004b7ee,0x0002b7ee, + 0x0006b7ee,0x0001b7ee,0x0005b7ee,0x0003b7ee,0x0007b7ee,0x000077ee,0x000477ee, + 0x000277ee,0x000677ee,0x000017ed,0x000177ee,0x00000526,0x000577ee,0x000023ea, + 0x0001c7ec,0x000377ee,0x000777ee,0x000217ed,0x000063ea,0x00000b68,0x00000ee9, + 0x00005beb,0x000013ea,0x00000467,0x00001b68,0x00000c67,0x00002ee9,0x00000768, + 0x00001768,0x00000f68,0x00001ee9,0x00001f68,0x00003ee9,0x000053ea,0x000001e9, + 0x000000e8,0x000021e9,0x000011e9,0x000010e8,0x000031e9,0x000033ea,0x000008e8, + 0x0000f7ee,0x0004f7ee,0x000018e8,0x000009e9,0x000004e8,0x000029e9,0x000014e8, + 0x000019e9,0x000073ea,0x0000dbeb,0x00000ce8,0x00003beb,0x0002f7ee,0x000039e9, + 0x00000bea,0x000005e9,0x00004bea,0x000025e9,0x000027ec,0x000015e9,0x000035e9, + 0x00000de9,0x00002bea,0x000127ec,0x0000bbeb,0x0006f7ee,0x0001f7ee,0x0000a7ec, + 0x00007beb,0x0005f7ee,0x0000fbeb,0x0003f7ee,0x0007f7ee,0x00000fee,0x00000326, + 0x00000267,0x00000a67,0x00000667,0x00000726,0x00001ce8,0x000002e8,0x00000e67, + 0x000000a6,0x0001a7ec,0x00002de9,0x000004a6,0x00000167,0x00000967,0x000002a6, + 0x00000567,0x000117ed,0x000006a6,0x000001a6,0x000005a6,0x00000d67,0x000012e8, + 0x00000ae8,0x00001de9,0x00001ae8,0x000007eb,0x000317ed,0x000067ec,0x000097ed, + 0x000297ed,0x00040fee,0x00020fee,0x00060fee,0x00010fee,0x00050fee,0x00030fee, + 0x00070fee,0x00008fee,0x00048fee,0x00028fee,0x00068fee,0x00018fee,0x00058fee, + 0x00038fee,0x00078fee,0x00004fee,0x00044fee,0x00024fee,0x00064fee,0x00014fee, + 0x00054fee,0x00034fee,0x00074fee,0x0000cfee,0x0004cfee,0x0002cfee,0x0006cfee, + 0x0001cfee,0x0005cfee,0x0003cfee,0x0007cfee,0x00002fee,0x00042fee,0x00022fee, + 0x00062fee,0x00012fee,0x00052fee,0x00032fee,0x00072fee,0x0000afee,0x0004afee, + 0x0002afee,0x0006afee,0x0001afee,0x0005afee,0x0003afee,0x0007afee,0x00006fee, + 0x00046fee,0x00026fee,0x00066fee,0x00016fee,0x00056fee,0x00036fee,0x00076fee, + 0x0000efee,0x0004efee,0x0002efee,0x0006efee,0x0001efee,0x0005efee,0x0003efee, + 0x0007efee,0x00001fee,0x00041fee,0x00021fee,0x00061fee,0x00011fee,0x00051fee, + 0x00031fee,0x00071fee,0x00009fee,0x00049fee,0x00029fee,0x00069fee,0x00019fee, + 0x00059fee,0x00039fee,0x00079fee,0x00005fee,0x00045fee,0x00025fee,0x00065fee, + 0x00015fee,0x00055fee,0x00035fee,0x00075fee,0x0000dfee,0x0004dfee,0x0002dfee, + 0x0006dfee,0x0001dfee,0x0005dfee,0x0003dfee,0x0007dfee,0x00003fee,0x00043fee, + 0x00023fee,0x00063fee,0x00013fee,0x00053fee,0x00033fee,0x00073fee,0x0000bfee, + 0x0004bfee,0x0002bfee,0x0006bfee,0x0001bfee,0x0005bfee,0x0003bfee,0x0007bfee, + 0x00007fee,0x00047fee,0x00027fee,0x00067fee,0x00017fee,0x000197ed,0x000397ed, + 0x000057ed,0x00057fee,0x000257ed,0x00037fee,0x000157ed,0x00077fee,0x000357ed, + 0x0000ffee,0x0004ffee,0x0002ffee,0x0006ffee,0x0001ffee,0x00000084,0x00000003, + 0x00000184,0x00000044,0x00000144,0x000000c5,0x000002c5,0x000001c5,0x000003c6, + 0x000007c6,0x00000026,0x00000426,0x000003a7,0x00000ba7,0x000007a7,0x00000fa7, + 0x00000227,0x00000627,0x00000a27,0x00000e27,0x00000068,0x00000868,0x00001068, + 0x00001868,0x00000369,0x00001369,0x00002369,0x00003369,0x000006ea,0x000026ea, + 0x000046ea,0x000066ea,0x000016eb,0x000036eb,0x000056eb,0x000076eb,0x000096eb, + 0x0000b6eb,0x0000d6eb,0x0000f6eb,0x00003dec,0x00007dec,0x0000bdec,0x0000fdec, + 0x00013dec,0x00017dec,0x0001bdec,0x0001fdec,0x00006bed,0x0000ebed,0x00016bed, + 0x0001ebed,0x00026bed,0x0002ebed,0x00036bed,0x0003ebed,0x000003ec,0x000043ec, + 0x000083ec,0x0000c3ec,0x000103ec,0x000143ec,0x000183ec,0x0001c3ec,0x00001bee, + 0x00009bee,0x00011bee,0x00019bee,0x00021bee,0x00029bee,0x00031bee,0x00039bee, + 0x00041bee,0x00049bee,0x00051bee,0x00059bee,0x00061bee,0x00069bee,0x00071bee, + 0x00079bee,0x000167f0,0x000367f0,0x000567f0,0x000767f0,0x000967f0,0x000b67f0, + 0x000d67f0,0x000f67f0,0x001167f0,0x001367f0,0x001567f0,0x001767f0,0x001967f0, + 0x001b67f0,0x001d67f0,0x001f67f0,0x000087ef,0x000187ef,0x000287ef,0x000387ef, + 0x000487ef,0x000587ef,0x000687ef,0x000787ef,0x000887ef,0x000987ef,0x000a87ef, + 0x000b87ef,0x000c87ef,0x000d87ef,0x000e87ef,0x000f87ef,0x0000e7f0,0x0002e7f0, + 0x0004e7f0,0x0006e7f0,0x0008e7f0,0x000ae7f0,0x000ce7f0,0x000ee7f0,0x0010e7f0, + 0x0012e7f0,0x0014e7f0,0x0016e7f0,0x0018e7f0,0x001ae7f0,0x001ce7f0,0x001ee7f0, + 0x0005fff3,0x000dfff3,0x0015fff3,0x001dfff3,0x0025fff3,0x002dfff3,0x0035fff3, + 0x003dfff3,0x0045fff3,0x004dfff3,0x0055fff3,0x005dfff3,0x0065fff3,0x006dfff3, + 0x0075fff3,0x007dfff3,0x0085fff3,0x008dfff3,0x0095fff3,0x009dfff3,0x00a5fff3, + 0x00adfff3,0x00b5fff3,0x00bdfff3,0x00c5fff3,0x00cdfff3,0x00d5fff3,0x00ddfff3, + 0x00e5fff3,0x00edfff3,0x00f5fff3,0x00fdfff3,0x0003fff3,0x000bfff3,0x0013fff3, + 0x001bfff3,0x0023fff3,0x002bfff3,0x0033fff3,0x003bfff3,0x0043fff3,0x004bfff3, + 0x0053fff3,0x005bfff3,0x0063fff3,0x006bfff3,0x0073fff3,0x007bfff3,0x0083fff3, + 0x008bfff3,0x0093fff3,0x009bfff3,0x00a3fff3,0x00abfff3,0x00b3fff3,0x00bbfff3, + 0x00c3fff3,0x00cbfff3,0x00d3fff3,0x00dbfff3,0x00e3fff3,0x00ebfff3,0x00f3fff3, + 0x00fbfff3,0x0007fff3,0x000ffff3,0x0017fff3,0x001ffff3,0x0027fff3,0x002ffff3, + 0x0037fff3,0x003ffff3,0x0047fff3,0x004ffff3,0x0057fff3,0x005ffff3,0x0067fff3, + 0x006ffff3,0x0077fff3,0x007ffff3,0x0087fff3,0x008ffff3,0x0097fff3,0x009ffff3, + 0x00a7fff3,0x00affff3,0x00b7fff3,0x00bffff3,0x00c7fff3,0x00cffff3,0x00d7fff3, + 0x00dffff3,0x00e7fff3,0x00effff3,0x00f7fff3,0x00fffff3,0x0001e7f1,0x0003e7f1, + 0x0005e7f1,0x0007e7f1,0x0009e7f1,0x000be7f1,0x000de7f1,0x000fe7f1,0x0011e7f1, + 0x0013e7f1,0x0015e7f1,0x0017e7f1,0x0019e7f1,0x001be7f1,0x001de7f1,0x001fe7f1, + 0x0021e7f1,0x0023e7f1,0x0025e7f1,0x0027e7f1,0x0029e7f1,0x002be7f1,0x002de7f1, + 0x002fe7f1,0x0031e7f1,0x0033e7f1,0x0035e7f1,0x0037e7f1,0x0039e7f1,0x003be7f1, + 0x003de7f1,0x000047eb + }; + + internal static readonly uint[] FAST_ENCODER_DISTANCE_CODE_INFO = + { + 0x00000f06,0x0001ff0a,0x0003ff0b,0x0007ff0b,0x0000ff19,0x00003f18,0x0000bf28, + 0x00007f28,0x00001f37,0x00005f37,0x00000d45,0x00002f46,0x00000054,0x00001d55, + 0x00000864,0x00000365,0x00000474,0x00001375,0x00000c84,0x00000284,0x00000a94, + 0x00000694,0x00000ea4,0x000001a4,0x000009b4,0x00000bb5,0x000005c4,0x00001bc5, + 0x000007d5,0x000017d5,0x00000000,0x00000100 + }; + + internal static readonly uint[] BIT_MASK = { 0, 1, 3, 7, 15, 31, 63, 127, 255, 511, 1023, 2047, 4095, 8191, 16383, 32767 }; + internal static readonly byte[] EXTRA_LENGTH_BITS = { 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0 }; + internal static readonly byte[] EXTRA_DISTANCE_BITS = { 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 0, 0 }; + internal const int NUM_CHARS = 256; + internal const int NUM_LENGTH_BASE_CODES = 29; + internal const int NUM_DIST_BASE_CODES = 30; + + internal const uint FAST_ENCODER_POST_TREE_BIT_BUF = 0x0022; + internal const int FAST_ENCODER_POST_TREE_BIT_COUNT = 9; + + internal const uint NO_COMPRESSION_HEADER = 0x0; + internal const int NO_COMPRESSION_HEADER_BIT_COUNT = 3; + internal const uint B_FINAL_NO_COMPRESSION_HEADER = 0x1; + internal const int B_FINAL_NO_COMPRESSION_HEADER_BIT_COUNT = 3; + internal const int MAX_CODE_LEN = 16; + + private static readonly byte[] S_DIST_LOOKUP = CreateDistanceLookup(); + + private static byte[] CreateDistanceLookup() + { + byte[] result = new byte[512]; + + // Generate the global slot tables which allow us to convert a distance + // (0..32K) to a distance slot (0..29) + // + // Distance table + // Extra Extra Extra + // Code Bits Dist Code Bits Dist Code Bits Distance + // ---- ---- ---- ---- ---- ------ ---- ---- -------- + // 0 0 1 10 4 33-48 20 9 1025-1536 + // 1 0 2 11 4 49-64 21 9 1537-2048 + // 2 0 3 12 5 65-96 22 10 2049-3072 + // 3 0 4 13 5 97-128 23 10 3073-4096 + // 4 1 5,6 14 6 129-192 24 11 4097-6144 + // 5 1 7,8 15 6 193-256 25 11 6145-8192 + // 6 2 9-12 16 7 257-384 26 12 8193-12288 + // 7 2 13-16 17 7 385-512 27 12 12289-16384 + // 8 3 17-24 18 8 513-768 28 13 16385-24576 + // 9 3 25-32 19 8 769-1024 29 13 24577-32768 + + // Initialize the mapping length (0..255) -> length code (0..28) + //int length = 0; + //for (code = 0; code < FastEncoderStatics.NumLengthBaseCodes-1; code++) { + // for (int n = 0; n < (1 << FastEncoderStatics.ExtraLengthBits[code]); n++) + // lengthLookup[length++] = (byte) code; + //} + //lengthLookup[length-1] = (byte) code; + + // Initialize the mapping dist (0..32K) -> dist code (0..29) + int dist = 0; + int code; + for (code = 0; code < 16; code++) + { + for (int n = 0; n < (1 << EXTRA_DISTANCE_BITS[code]); n++) + result[dist++] = (byte)code; + } + + dist >>= 7; // from now on, all distances are divided by 128 + + for (; code < NUM_DIST_BASE_CODES; code++) + { + for (int n = 0; n < (1 << (EXTRA_DISTANCE_BITS[code] - 7)); n++) + result[256 + dist++] = (byte)code; + } + + return result; + } + + // Return the position slot (0...29) of a match offset (0...32767) + internal static int GetSlot(int pos) => + S_DIST_LOOKUP[((pos) < 256) ? (pos) : (256 + ((pos) >> 7))]; + + // Reverse 'length' of the bits in code + public static uint BitReverse(uint code, int length) + { + uint newCode = 0; + + Debug.Assert(length > 0 && length <= 16, "Invalid len"); + do + { + newCode |= (code & 1); + newCode <<= 1; + code >>= 1; + } while (--length > 0); + + return newCode >> 1; + } + } +} diff --git a/src/SharpCompress/Compressors/Deflate64/HuffmanTree.cs b/src/SharpCompress/Compressors/Deflate64/HuffmanTree.cs new file mode 100644 index 00000000..62c24b59 --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/HuffmanTree.cs @@ -0,0 +1,311 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +using System.Diagnostics; +using System.IO; + +namespace SharpCompress.Compressors.Deflate64 +{ + // Strictly speaking this class is not a HuffmanTree, this class is + // a lookup table combined with a HuffmanTree. The idea is to speed up + // the lookup for short symbols (they should appear more frequently ideally.) + // However we don't want to create a huge table since it might take longer to + // build the table than decoding (Deflate usually generates new tables frequently.) + // + // Jean-loup Gailly and Mark Adler gave a very good explanation about this. + // The full text (algorithm.txt) can be found inside + // ftp://ftp.uu.net/pub/archiving/zip/zlib/zlib.zip. + // + // Following paper explains decoding in details: + // Hirschberg and Lelewer, "Efficient decoding of prefix codes," + // Comm. ACM, 33,4, April 1990, pp. 449-459. + // + + internal sealed class HuffmanTree + { + internal const int MAX_LITERAL_TREE_ELEMENTS = 288; + internal const int MAX_DIST_TREE_ELEMENTS = 32; + internal const int END_OF_BLOCK_CODE = 256; + internal const int NUMBER_OF_CODE_LENGTH_TREE_ELEMENTS = 19; + + private readonly int _tableBits; + private readonly short[] _table; + private readonly short[] _left; + private readonly short[] _right; + private readonly byte[] _codeLengthArray; +#if DEBUG + private uint[] _codeArrayDebug; +#endif + + private readonly int _tableMask; + + // huffman tree for static block + public static HuffmanTree StaticLiteralLengthTree { get; } = new HuffmanTree(GetStaticLiteralTreeLength()); + + public static HuffmanTree StaticDistanceTree { get; } = new HuffmanTree(GetStaticDistanceTreeLength()); + + public HuffmanTree(byte[] codeLengths) + { + Debug.Assert( + codeLengths.Length == MAX_LITERAL_TREE_ELEMENTS || + codeLengths.Length == MAX_DIST_TREE_ELEMENTS || + codeLengths.Length == NUMBER_OF_CODE_LENGTH_TREE_ELEMENTS, + "we only expect three kinds of Length here"); + _codeLengthArray = codeLengths; + + if (_codeLengthArray.Length == MAX_LITERAL_TREE_ELEMENTS) + { + // bits for Literal/Length tree table + _tableBits = 9; + } + else + { + // bits for distance tree table and code length tree table + _tableBits = 7; + } + _tableMask = (1 << _tableBits) - 1; + + _table = new short[1 << _tableBits]; + + // I need to find proof that left and right array will always be + // enough. I think they are. + _left = new short[2 * _codeLengthArray.Length]; + _right = new short[2 * _codeLengthArray.Length]; + + CreateTable(); + } + + // Generate the array contains huffman codes lengths for static huffman tree. + // The data is in RFC 1951. + private static byte[] GetStaticLiteralTreeLength() + { + byte[] literalTreeLength = new byte[MAX_LITERAL_TREE_ELEMENTS]; + for (int i = 0; i <= 143; i++) + literalTreeLength[i] = 8; + + for (int i = 144; i <= 255; i++) + literalTreeLength[i] = 9; + + for (int i = 256; i <= 279; i++) + literalTreeLength[i] = 7; + + for (int i = 280; i <= 287; i++) + literalTreeLength[i] = 8; + + return literalTreeLength; + } + + private static byte[] GetStaticDistanceTreeLength() + { + byte[] staticDistanceTreeLength = new byte[MAX_DIST_TREE_ELEMENTS]; + for (int i = 0; i < MAX_DIST_TREE_ELEMENTS; i++) + { + staticDistanceTreeLength[i] = 5; + } + return staticDistanceTreeLength; + } + + // Calculate the huffman code for each character based on the code length for each character. + // This algorithm is described in standard RFC 1951 + private uint[] CalculateHuffmanCode() + { + uint[] bitLengthCount = new uint[17]; + foreach (int codeLength in _codeLengthArray) + { + bitLengthCount[codeLength]++; + } + bitLengthCount[0] = 0; // clear count for length 0 + + uint[] nextCode = new uint[17]; + uint tempCode = 0; + for (int bits = 1; bits <= 16; bits++) + { + tempCode = (tempCode + bitLengthCount[bits - 1]) << 1; + nextCode[bits] = tempCode; + } + + uint[] code = new uint[MAX_LITERAL_TREE_ELEMENTS]; + for (int i = 0; i < _codeLengthArray.Length; i++) + { + int len = _codeLengthArray[i]; + + if (len > 0) + { + code[i] = FastEncoderStatics.BitReverse(nextCode[len], len); + nextCode[len]++; + } + } + return code; + } + + private void CreateTable() + { + uint[] codeArray = CalculateHuffmanCode(); +#if DEBUG + _codeArrayDebug = codeArray; +#endif + + short avail = (short)_codeLengthArray.Length; + + for (int ch = 0; ch < _codeLengthArray.Length; ch++) + { + // length of this code + int len = _codeLengthArray[ch]; + if (len > 0) + { + // start value (bit reversed) + int start = (int)codeArray[ch]; + + if (len <= _tableBits) + { + // If a particular symbol is shorter than nine bits, + // then that symbol's translation is duplicated + // in all those entries that start with that symbol's bits. + // For example, if the symbol is four bits, then it's duplicated + // 32 times in a nine-bit table. If a symbol is nine bits long, + // it appears in the table once. + // + // Make sure that in the loop below, code is always + // less than table_size. + // + // On last iteration we store at array index: + // initial_start_at + (locs-1)*increment + // = initial_start_at + locs*increment - increment + // = initial_start_at + (1 << tableBits) - increment + // = initial_start_at + table_size - increment + // + // Therefore we must ensure: + // initial_start_at + table_size - increment < table_size + // or: initial_start_at < increment + // + int increment = 1 << len; + if (start >= increment) + { + throw new InvalidDataException("Deflate64: invalid Huffman data"); + } + + // Note the bits in the table are reverted. + int locs = 1 << (_tableBits - len); + for (int j = 0; j < locs; j++) + { + _table[start] = (short)ch; + start += increment; + } + } + else + { + // For any code which has length longer than num_elements, + // build a binary tree. + + int overflowBits = len - _tableBits; // the nodes we need to respent the data. + int codeBitMask = 1 << _tableBits; // mask to get current bit (the bits can't fit in the table) + + // the left, right table is used to repesent the + // the rest bits. When we got the first part (number bits.) and look at + // tbe table, we will need to follow the tree to find the real character. + // This is in place to avoid bloating the table if there are + // a few ones with long code. + int index = start & ((1 << _tableBits) - 1); + short[] array = _table; + + do + { + short value = array[index]; + + if (value == 0) + { + // set up next pointer if this node is not used before. + array[index] = (short)-avail; // use next available slot. + value = (short)-avail; + avail++; + } + + if (value > 0) + { + // prevent an IndexOutOfRangeException from array[index] + throw new InvalidDataException("Deflate64: invalid Huffman data"); + } + + Debug.Assert(value < 0, "CreateTable: Only negative numbers are used for tree pointers!"); + + if ((start & codeBitMask) == 0) + { + // if current bit is 0, go change the left array + array = _left; + } + else + { + // if current bit is 1, set value in the right array + array = _right; + } + index = -value; // go to next node + + codeBitMask <<= 1; + overflowBits--; + } while (overflowBits != 0); + + array[index] = (short)ch; + } + } + } + } + + // + // This function will try to get enough bits from input and + // try to decode the bits. + // If there are no enought bits in the input, this function will return -1. + // + public int GetNextSymbol(InputBuffer input) + { + // Try to load 16 bits into input buffer if possible and get the bitBuffer value. + // If there aren't 16 bits available we will return all we have in the + // input buffer. + uint bitBuffer = input.TryLoad16Bits(); + if (input.AvailableBits == 0) + { // running out of input. + return -1; + } + + // decode an element + int symbol = _table[bitBuffer & _tableMask]; + if (symbol < 0) + { // this will be the start of the binary tree + // navigate the tree + uint mask = (uint)1 << _tableBits; + do + { + symbol = -symbol; + if ((bitBuffer & mask) == 0) + symbol = _left[symbol]; + else + symbol = _right[symbol]; + mask <<= 1; + } while (symbol < 0); + } + + int codeLength = _codeLengthArray[symbol]; + + // huffman code lengths must be at least 1 bit long + if (codeLength <= 0) + { + throw new InvalidDataException("Deflate64: invalid Huffman data"); + } + + // + // If this code is longer than the # bits we had in the bit buffer (i.e. + // we read only part of the code), we can hit the entry in the table or the tree + // for another symbol. However the length of another symbol will not match the + // available bits count. + if (codeLength > input.AvailableBits) + { + // We already tried to load 16 bits and maximum length is 15, + // so this means we are running out of input. + return -1; + } + + input.SkipBits(codeLength); + return symbol; + } + } +} diff --git a/src/SharpCompress/Compressors/Deflate64/InflaterManaged.cs b/src/SharpCompress/Compressors/Deflate64/InflaterManaged.cs new file mode 100644 index 00000000..7ce389b7 --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/InflaterManaged.cs @@ -0,0 +1,738 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +// +// zlib.h -- interface of the 'zlib' general purpose compression library +// version 1.2.1, November 17th, 2003 +// +// Copyright (C) 1995-2003 Jean-loup Gailly and Mark Adler +// +// This software is provided 'as-is', without any express or implied +// warranty. In no event will the authors be held liable for any damages +// arising from the use of this software. +// +// Permission is granted to anyone to use this software for any purpose, +// including commercial applications, and to alter it and redistribute it +// freely, subject to the following restrictions: +// +// 1. The origin of this software must not be misrepresented; you must not +// claim that you wrote the original software. If you use this software +// in a product, an acknowledgment in the product documentation would be +// appreciated but is not required. +// 2. Altered source versions must be plainly marked as such, and must not be +// misrepresented as being the original software. +// 3. This notice may not be removed or altered from any source distribution. +// +// + +using System; +using System.Diagnostics; +using System.IO; + +namespace SharpCompress.Compressors.Deflate64 +{ + internal sealed class InflaterManaged + { + // const tables used in decoding: + + // Extra bits for length code 257 - 285. + private static readonly byte[] S_EXTRA_LENGTH_BITS = + { 0,0,0,0,0,0,0,0,1,1,1,1,2,2,2,2,3,3,3,3,4,4,4,4,5,5,5,5,16 }; + + // The base length for length code 257 - 285. + // The formula to get the real length for a length code is lengthBase[code - 257] + (value stored in extraBits) + private static readonly int[] S_LENGTH_BASE = + { 3,4,5,6,7,8,9,10,11,13,15,17,19,23,27,31,35,43,51,59,67,83,99,115,131,163,195,227,3}; + + // The base distance for distance code 0 - 31 + // The real distance for a distance code is distanceBasePosition[code] + (value stored in extraBits) + private static readonly int[] S_DISTANCE_BASE_POSITION = + { 1,2,3,4,5,7,9,13,17,25,33,49,65,97,129,193,257,385,513,769,1025,1537,2049,3073,4097,6145,8193,12289,16385,24577,32769,49153 }; + + // code lengths for code length alphabet is stored in following order + private static readonly byte[] S_CODE_ORDER = { 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15 }; + + private static readonly byte[] S_STATIC_DISTANCE_TREE_TABLE = + { + 0x00,0x10,0x08,0x18,0x04,0x14,0x0c,0x1c,0x02,0x12,0x0a,0x1a, + 0x06,0x16,0x0e,0x1e,0x01,0x11,0x09,0x19,0x05,0x15,0x0d,0x1d, + 0x03,0x13,0x0b,0x1b,0x07,0x17,0x0f,0x1f + }; + + private readonly OutputWindow _output; + private readonly InputBuffer _input; + private HuffmanTree _literalLengthTree; + private HuffmanTree _distanceTree; + + private InflaterState _state; + //private bool _hasFormatReader; + private int _bfinal; + private BlockType _blockType; + + // uncompressed block + private readonly byte[] _blockLengthBuffer = new byte[4]; + private int _blockLength; + + // compressed block + private int _length; + private int _distanceCode; + private int _extraBits; + + private int _loopCounter; + private int _literalLengthCodeCount; + private int _distanceCodeCount; + private int _codeLengthCodeCount; + private int _codeArraySize; + private int _lengthCode; + + private readonly byte[] _codeList; // temporary array to store the code length for literal/Length and distance + private readonly byte[] _codeLengthTreeCodeLength; + private readonly bool _deflate64; + private HuffmanTree _codeLengthTree; + + //private IFileFormatReader _formatReader; // class to decode header and footer (e.g. gzip) + + internal InflaterManaged(/*IFileFormatReader reader, */bool deflate64) + { + _output = new OutputWindow(); + _input = new InputBuffer(); + + _codeList = new byte[HuffmanTree.MAX_LITERAL_TREE_ELEMENTS + HuffmanTree.MAX_DIST_TREE_ELEMENTS]; + _codeLengthTreeCodeLength = new byte[HuffmanTree.NUMBER_OF_CODE_LENGTH_TREE_ELEMENTS]; + _deflate64 = deflate64; + //if (reader != null) + //{ + // _formatReader = reader; + // _hasFormatReader = true; + //} + Reset(); + } + + private void Reset() + { + _state = //_hasFormatReader ? + //InflaterState.ReadingHeader : // start by reading Header info + InflaterState.ReadingBFinal; // start by reading BFinal bit + } + + public void SetInput(byte[] inputBytes, int offset, int length) => + _input.SetInput(inputBytes, offset, length); // append the bytes + + public bool Finished() => _state == InflaterState.Done || _state == InflaterState.VerifyingFooter; + + public int AvailableOutput => _output.AvailableBytes; + + public int Inflate(byte[] bytes, int offset, int length) + { + // copy bytes from output to outputbytes if we have available bytes + // if buffer is not filled up. keep decoding until no input are available + // if decodeBlock returns false. Throw an exception. + int count = 0; + do + { + int copied = _output.CopyTo(bytes, offset, length); + if (copied > 0) + { + //if (_hasFormatReader) + //{ + // _formatReader.UpdateWithBytesRead(bytes, offset, copied); + //} + + offset += copied; + count += copied; + length -= copied; + } + + if (length == 0) + { // filled in the bytes array + break; + } + // Decode will return false when more input is needed + } while (!Finished() && Decode()); + + if (_state == InflaterState.VerifyingFooter) + { // finished reading CRC + // In this case finished is true and output window has all the data. + // But some data in output window might not be copied out. + if (_output.AvailableBytes == 0) + { + //_formatReader.Validate(); + } + } + + return count; + } + + //Each block of compressed data begins with 3 header bits + // containing the following data: + // first bit BFINAL + // next 2 bits BTYPE + // Note that the header bits do not necessarily begin on a byte + // boundary, since a block does not necessarily occupy an integral + // number of bytes. + // BFINAL is set if and only if this is the last block of the data + // set. + // BTYPE specifies how the data are compressed, as follows: + // 00 - no compression + // 01 - compressed with fixed Huffman codes + // 10 - compressed with dynamic Huffman codes + // 11 - reserved (error) + // The only difference between the two compressed cases is how the + // Huffman codes for the literal/length and distance alphabets are + // defined. + // + // This function returns true for success (end of block or output window is full,) + // false if we are short of input + // + private bool Decode() + { + bool eob = false; + bool result = false; + + if (Finished()) + { + return true; + } + + //if (_hasFormatReader) + //{ + // if (_state == InflaterState.ReadingHeader) + // { + // if (!_formatReader.ReadHeader(_input)) + // { + // return false; + // } + // _state = InflaterState.ReadingBFinal; + // } + // else if (_state == InflaterState.StartReadingFooter || _state == InflaterState.ReadingFooter) + // { + // if (!_formatReader.ReadFooter(_input)) + // return false; + + // _state = InflaterState.VerifyingFooter; + // return true; + // } + //} + + if (_state == InflaterState.ReadingBFinal) + { + // reading bfinal bit + // Need 1 bit + if (!_input.EnsureBitsAvailable(1)) + return false; + + _bfinal = _input.GetBits(1); + _state = InflaterState.ReadingBType; + } + + if (_state == InflaterState.ReadingBType) + { + // Need 2 bits + if (!_input.EnsureBitsAvailable(2)) + { + _state = InflaterState.ReadingBType; + return false; + } + + _blockType = (BlockType)_input.GetBits(2); + if (_blockType == BlockType.Dynamic) + { + _state = InflaterState.ReadingNumLitCodes; + } + else if (_blockType == BlockType.Static) + { + _literalLengthTree = HuffmanTree.StaticLiteralLengthTree; + _distanceTree = HuffmanTree.StaticDistanceTree; + _state = InflaterState.DecodeTop; + } + else if (_blockType == BlockType.Uncompressed) + { + _state = InflaterState.UncompressedAligning; + } + else + { + throw new InvalidDataException("Deflate64: unknown block type"); + } + } + + if (_blockType == BlockType.Dynamic) + { + if (_state < InflaterState.DecodeTop) + { + // we are reading the header + result = DecodeDynamicBlockHeader(); + } + else + { + result = DecodeBlock(out eob); // this can returns true when output is full + } + } + else if (_blockType == BlockType.Static) + { + result = DecodeBlock(out eob); + } + else if (_blockType == BlockType.Uncompressed) + { + result = DecodeUncompressedBlock(out eob); + } + else + { + throw new InvalidDataException("Deflate64: unknown block type"); + } + + // + // If we reached the end of the block and the block we were decoding had + // bfinal=1 (final block) + // + if (eob && (_bfinal != 0)) + { + //if (_hasFormatReader) + // _state = InflaterState.StartReadingFooter; + //else + _state = InflaterState.Done; + } + return result; + } + + + // Format of Non-compressed blocks (BTYPE=00): + // + // Any bits of input up to the next byte boundary are ignored. + // The rest of the block consists of the following information: + // + // 0 1 2 3 4... + // +---+---+---+---+================================+ + // | LEN | NLEN |... LEN bytes of literal data...| + // +---+---+---+---+================================+ + // + // LEN is the number of data bytes in the block. NLEN is the + // one's complement of LEN. + private bool DecodeUncompressedBlock(out bool endOfBlock) + { + endOfBlock = false; + while (true) + { + switch (_state) + { + case InflaterState.UncompressedAligning: // initial state when calling this function + // we must skip to a byte boundary + _input.SkipToByteBoundary(); + _state = InflaterState.UncompressedByte1; + goto case InflaterState.UncompressedByte1; + + case InflaterState.UncompressedByte1: // decoding block length + case InflaterState.UncompressedByte2: + case InflaterState.UncompressedByte3: + case InflaterState.UncompressedByte4: + int bits = _input.GetBits(8); + if (bits < 0) + { + return false; + } + + _blockLengthBuffer[_state - InflaterState.UncompressedByte1] = (byte)bits; + if (_state == InflaterState.UncompressedByte4) + { + _blockLength = _blockLengthBuffer[0] + ((int)_blockLengthBuffer[1]) * 256; + int blockLengthComplement = _blockLengthBuffer[2] + ((int)_blockLengthBuffer[3]) * 256; + + // make sure complement matches + if ((ushort)_blockLength != (ushort)(~blockLengthComplement)) + { + throw new InvalidDataException("Deflate64: invalid block length"); + } + } + + _state += 1; + break; + + case InflaterState.DecodingUncompressed: // copying block data + + // Directly copy bytes from input to output. + int bytesCopied = _output.CopyFrom(_input, _blockLength); + _blockLength -= bytesCopied; + + if (_blockLength == 0) + { + // Done with this block, need to re-init bit buffer for next block + _state = InflaterState.ReadingBFinal; + endOfBlock = true; + return true; + } + + // We can fail to copy all bytes for two reasons: + // Running out of Input + // running out of free space in output window + if (_output.FreeBytes == 0) + { + return true; + } + + return false; + + default: + Debug./*Fail*/Assert(false, "check why we are here!"); + throw new InvalidDataException("Deflate64: unknown state"); + } + } + } + + private bool DecodeBlock(out bool endOfBlockCodeSeen) + { + endOfBlockCodeSeen = false; + + int freeBytes = _output.FreeBytes; // it is a little bit faster than frequently accessing the property + while (freeBytes > 65536) + { + // With Deflate64 we can have up to a 64kb length, so we ensure at least that much space is available + // in the OutputWindow to avoid overwriting previous unflushed output data. + + int symbol; + switch (_state) + { + case InflaterState.DecodeTop: + // decode an element from the literal tree + + // TODO: optimize this!!! + symbol = _literalLengthTree.GetNextSymbol(_input); + if (symbol < 0) + { + // running out of input + return false; + } + + if (symbol < 256) + { + // literal + _output.Write((byte)symbol); + --freeBytes; + } + else if (symbol == 256) + { + // end of block + endOfBlockCodeSeen = true; + // Reset state + _state = InflaterState.ReadingBFinal; + return true; + } + else + { + // length/distance pair + symbol -= 257; // length code started at 257 + if (symbol < 8) + { + symbol += 3; // match length = 3,4,5,6,7,8,9,10 + _extraBits = 0; + } + else if (!_deflate64 && symbol == 28) + { + // extra bits for code 285 is 0 + symbol = 258; // code 285 means length 258 + _extraBits = 0; + } + else + { + if (symbol < 0 || symbol >= S_EXTRA_LENGTH_BITS.Length) + { + throw new InvalidDataException("Deflate64: invalid data"); + } + _extraBits = S_EXTRA_LENGTH_BITS[symbol]; + Debug.Assert(_extraBits != 0, "We handle other cases separately!"); + } + _length = symbol; + goto case InflaterState.HaveInitialLength; + } + break; + + case InflaterState.HaveInitialLength: + if (_extraBits > 0) + { + _state = InflaterState.HaveInitialLength; + int bits = _input.GetBits(_extraBits); + if (bits < 0) + { + return false; + } + + if (_length < 0 || _length >= S_LENGTH_BASE.Length) + { + throw new InvalidDataException("Deflate64: invalid data"); + } + _length = S_LENGTH_BASE[_length] + bits; + } + _state = InflaterState.HaveFullLength; + goto case InflaterState.HaveFullLength; + + case InflaterState.HaveFullLength: + if (_blockType == BlockType.Dynamic) + { + _distanceCode = _distanceTree.GetNextSymbol(_input); + } + else + { + // get distance code directly for static block + _distanceCode = _input.GetBits(5); + if (_distanceCode >= 0) + { + _distanceCode = S_STATIC_DISTANCE_TREE_TABLE[_distanceCode]; + } + } + + if (_distanceCode < 0) + { + // running out input + return false; + } + + _state = InflaterState.HaveDistCode; + goto case InflaterState.HaveDistCode; + + case InflaterState.HaveDistCode: + // To avoid a table lookup we note that for distanceCode > 3, + // extra_bits = (distanceCode-2) >> 1 + int offset; + if (_distanceCode > 3) + { + _extraBits = (_distanceCode - 2) >> 1; + int bits = _input.GetBits(_extraBits); + if (bits < 0) + { + return false; + } + offset = S_DISTANCE_BASE_POSITION[_distanceCode] + bits; + } + else + { + offset = _distanceCode + 1; + } + + _output.WriteLengthDistance(_length, offset); + freeBytes -= _length; + _state = InflaterState.DecodeTop; + break; + + default: + Debug./*Fail*/Assert(false, "check why we are here!"); + throw new InvalidDataException("Deflate64: unknown state"); + } + } + + return true; + } + + + // Format of the dynamic block header: + // 5 Bits: HLIT, # of Literal/Length codes - 257 (257 - 286) + // 5 Bits: HDIST, # of Distance codes - 1 (1 - 32) + // 4 Bits: HCLEN, # of Code Length codes - 4 (4 - 19) + // + // (HCLEN + 4) x 3 bits: code lengths for the code length + // alphabet given just above, in the order: 16, 17, 18, + // 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15 + // + // These code lengths are interpreted as 3-bit integers + // (0-7); as above, a code length of 0 means the + // corresponding symbol (literal/length or distance code + // length) is not used. + // + // HLIT + 257 code lengths for the literal/length alphabet, + // encoded using the code length Huffman code + // + // HDIST + 1 code lengths for the distance alphabet, + // encoded using the code length Huffman code + // + // The code length repeat codes can cross from HLIT + 257 to the + // HDIST + 1 code lengths. In other words, all code lengths form + // a single sequence of HLIT + HDIST + 258 values. + private bool DecodeDynamicBlockHeader() + { + switch (_state) + { + case InflaterState.ReadingNumLitCodes: + _literalLengthCodeCount = _input.GetBits(5); + if (_literalLengthCodeCount < 0) + { + return false; + } + _literalLengthCodeCount += 257; + _state = InflaterState.ReadingNumDistCodes; + goto case InflaterState.ReadingNumDistCodes; + + case InflaterState.ReadingNumDistCodes: + _distanceCodeCount = _input.GetBits(5); + if (_distanceCodeCount < 0) + { + return false; + } + _distanceCodeCount += 1; + _state = InflaterState.ReadingNumCodeLengthCodes; + goto case InflaterState.ReadingNumCodeLengthCodes; + + case InflaterState.ReadingNumCodeLengthCodes: + _codeLengthCodeCount = _input.GetBits(4); + if (_codeLengthCodeCount < 0) + { + return false; + } + _codeLengthCodeCount += 4; + _loopCounter = 0; + _state = InflaterState.ReadingCodeLengthCodes; + goto case InflaterState.ReadingCodeLengthCodes; + + case InflaterState.ReadingCodeLengthCodes: + while (_loopCounter < _codeLengthCodeCount) + { + int bits = _input.GetBits(3); + if (bits < 0) + { + return false; + } + _codeLengthTreeCodeLength[S_CODE_ORDER[_loopCounter]] = (byte)bits; + ++_loopCounter; + } + + for (int i = _codeLengthCodeCount; i < S_CODE_ORDER.Length; i++) + { + _codeLengthTreeCodeLength[S_CODE_ORDER[i]] = 0; + } + + // create huffman tree for code length + _codeLengthTree = new HuffmanTree(_codeLengthTreeCodeLength); + _codeArraySize = _literalLengthCodeCount + _distanceCodeCount; + _loopCounter = 0; // reset loop count + + _state = InflaterState.ReadingTreeCodesBefore; + goto case InflaterState.ReadingTreeCodesBefore; + + case InflaterState.ReadingTreeCodesBefore: + case InflaterState.ReadingTreeCodesAfter: + while (_loopCounter < _codeArraySize) + { + if (_state == InflaterState.ReadingTreeCodesBefore) + { + if ((_lengthCode = _codeLengthTree.GetNextSymbol(_input)) < 0) + { + return false; + } + } + + // The alphabet for code lengths is as follows: + // 0 - 15: Represent code lengths of 0 - 15 + // 16: Copy the previous code length 3 - 6 times. + // The next 2 bits indicate repeat length + // (0 = 3, ... , 3 = 6) + // Example: Codes 8, 16 (+2 bits 11), + // 16 (+2 bits 10) will expand to + // 12 code lengths of 8 (1 + 6 + 5) + // 17: Repeat a code length of 0 for 3 - 10 times. + // (3 bits of length) + // 18: Repeat a code length of 0 for 11 - 138 times + // (7 bits of length) + if (_lengthCode <= 15) + { + _codeList[_loopCounter++] = (byte)_lengthCode; + } + else + { + int repeatCount; + if (_lengthCode == 16) + { + if (!_input.EnsureBitsAvailable(2)) + { + _state = InflaterState.ReadingTreeCodesAfter; + return false; + } + + if (_loopCounter == 0) + { + // can't have "prev code" on first code + throw new InvalidDataException(); + } + + byte previousCode = _codeList[_loopCounter - 1]; + repeatCount = _input.GetBits(2) + 3; + + if (_loopCounter + repeatCount > _codeArraySize) + { + throw new InvalidDataException(); + } + + for (int j = 0; j < repeatCount; j++) + { + _codeList[_loopCounter++] = previousCode; + } + } + else if (_lengthCode == 17) + { + if (!_input.EnsureBitsAvailable(3)) + { + _state = InflaterState.ReadingTreeCodesAfter; + return false; + } + + repeatCount = _input.GetBits(3) + 3; + + if (_loopCounter + repeatCount > _codeArraySize) + { + throw new InvalidDataException(); + } + + for (int j = 0; j < repeatCount; j++) + { + _codeList[_loopCounter++] = 0; + } + } + else + { + // code == 18 + if (!_input.EnsureBitsAvailable(7)) + { + _state = InflaterState.ReadingTreeCodesAfter; + return false; + } + + repeatCount = _input.GetBits(7) + 11; + + if (_loopCounter + repeatCount > _codeArraySize) + { + throw new InvalidDataException(); + } + + for (int j = 0; j < repeatCount; j++) + { + _codeList[_loopCounter++] = 0; + } + } + } + _state = InflaterState.ReadingTreeCodesBefore; // we want to read the next code. + } + break; + + default: + Debug./*Fail*/Assert(false, "check why we are here!"); + throw new InvalidDataException("Deflate64: unknown state"); + } + + byte[] literalTreeCodeLength = new byte[HuffmanTree.MAX_LITERAL_TREE_ELEMENTS]; + byte[] distanceTreeCodeLength = new byte[HuffmanTree.MAX_DIST_TREE_ELEMENTS]; + + // Create literal and distance tables + Array.Copy(_codeList, 0, literalTreeCodeLength, 0, _literalLengthCodeCount); + Array.Copy(_codeList, _literalLengthCodeCount, distanceTreeCodeLength, 0, _distanceCodeCount); + + // Make sure there is an end-of-block code, otherwise how could we ever end? + if (literalTreeCodeLength[HuffmanTree.END_OF_BLOCK_CODE] == 0) + { + throw new InvalidDataException(); + } + + _literalLengthTree = new HuffmanTree(literalTreeCodeLength); + _distanceTree = new HuffmanTree(distanceTreeCodeLength); + _state = InflaterState.DecodeTop; + return true; + } + + public void Dispose() { } + } +} diff --git a/src/SharpCompress/Compressors/Deflate64/InflaterState.cs b/src/SharpCompress/Compressors/Deflate64/InflaterState.cs new file mode 100644 index 00000000..356ea88f --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/InflaterState.cs @@ -0,0 +1,42 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +namespace SharpCompress.Compressors.Deflate64 +{ + // Do not rearrange the enum values. + internal enum InflaterState + { + ReadingHeader = 0, // Only applies to GZIP + + ReadingBFinal = 2, // About to read bfinal bit + ReadingBType = 3, // About to read blockType bits + + ReadingNumLitCodes = 4, // About to read # literal codes + ReadingNumDistCodes = 5, // About to read # dist codes + ReadingNumCodeLengthCodes = 6, // About to read # code length codes + ReadingCodeLengthCodes = 7, // In the middle of reading the code length codes + ReadingTreeCodesBefore = 8, // In the middle of reading tree codes (loop top) + ReadingTreeCodesAfter = 9, // In the middle of reading tree codes (extension; code > 15) + + DecodeTop = 10, // About to decode a literal (char/match) in a compressed block + HaveInitialLength = 11, // Decoding a match, have the literal code (base length) + HaveFullLength = 12, // Ditto, now have the full match length (incl. extra length bits) + HaveDistCode = 13, // Ditto, now have the distance code also, need extra dist bits + + /* uncompressed blocks */ + UncompressedAligning = 15, + UncompressedByte1 = 16, + UncompressedByte2 = 17, + UncompressedByte3 = 18, + UncompressedByte4 = 19, + DecodingUncompressed = 20, + + // These three apply only to GZIP + StartReadingFooter = 21, // (Initialisation for reading footer) + ReadingFooter = 22, + VerifyingFooter = 23, + + Done = 24 // Finished + } +} diff --git a/src/SharpCompress/Compressors/Deflate64/InputBuffer.cs b/src/SharpCompress/Compressors/Deflate64/InputBuffer.cs new file mode 100644 index 00000000..0f585388 --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/InputBuffer.cs @@ -0,0 +1,202 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +using System; +using System.Diagnostics; + +namespace SharpCompress.Compressors.Deflate64 +{ + // This class can be used to read bits from an byte array quickly. + // Normally we get bits from 'bitBuffer' field and bitsInBuffer stores + // the number of bits available in 'BitBuffer'. + // When we used up the bits in bitBuffer, we will try to get byte from + // the byte array and copy the byte to appropiate position in bitBuffer. + // + // The byte array is not reused. We will go from 'start' to 'end'. + // When we reach the end, most read operations will return -1, + // which means we are running out of input. + + internal sealed class InputBuffer + { + private byte[] _buffer; // byte array to store input + private int _start; // start poisition of the buffer + private int _end; // end position of the buffer + private uint _bitBuffer = 0; // store the bits here, we can quickly shift in this buffer + private int _bitsInBuffer = 0; // number of bits available in bitBuffer + + /// Total bits available in the input buffer. + public int AvailableBits => _bitsInBuffer; + + /// Total bytes available in the input buffer. + public int AvailableBytes => (_end - _start) + (_bitsInBuffer / 8); + + /// Ensure that count bits are in the bit buffer. + /// Can be up to 16. + /// Returns false if input is not sufficient to make this true. + public bool EnsureBitsAvailable(int count) + { + Debug.Assert(0 < count && count <= 16, "count is invalid."); + + // manual inlining to improve perf + if (_bitsInBuffer < count) + { + if (NeedsInput()) + { + return false; + } + // insert a byte to bitbuffer + _bitBuffer |= (uint)_buffer[_start++] << _bitsInBuffer; + _bitsInBuffer += 8; + + if (_bitsInBuffer < count) + { + if (NeedsInput()) + { + return false; + } + // insert a byte to bitbuffer + _bitBuffer |= (uint)_buffer[_start++] << _bitsInBuffer; + _bitsInBuffer += 8; + } + } + + return true; + } + + /// + /// This function will try to load 16 or more bits into bitBuffer. + /// It returns whatever is contained in bitBuffer after loading. + /// The main difference between this and GetBits is that this will + /// never return -1. So the caller needs to check AvailableBits to + /// see how many bits are available. + /// + public uint TryLoad16Bits() + { + if (_bitsInBuffer < 8) + { + if (_start < _end) + { + _bitBuffer |= (uint)_buffer[_start++] << _bitsInBuffer; + _bitsInBuffer += 8; + } + + if (_start < _end) + { + _bitBuffer |= (uint)_buffer[_start++] << _bitsInBuffer; + _bitsInBuffer += 8; + } + } + else if (_bitsInBuffer < 16) + { + if (_start < _end) + { + _bitBuffer |= (uint)_buffer[_start++] << _bitsInBuffer; + _bitsInBuffer += 8; + } + } + + return _bitBuffer; + } + + private uint GetBitMask(int count) => ((uint)1 << count) - 1; + + /// Gets count bits from the input buffer. Returns -1 if not enough bits available. + public int GetBits(int count) + { + Debug.Assert(0 < count && count <= 16, "count is invalid."); + + if (!EnsureBitsAvailable(count)) + { + return -1; + } + + int result = (int)(_bitBuffer & GetBitMask(count)); + _bitBuffer >>= count; + _bitsInBuffer -= count; + return result; + } + + /// + /// Copies length bytes from input buffer to output buffer starting at output[offset]. + /// You have to make sure, that the buffer is byte aligned. If not enough bytes are + /// available, copies fewer bytes. + /// + /// Returns the number of bytes copied, 0 if no byte is available. + public int CopyTo(byte[] output, int offset, int length) + { + Debug.Assert(output != null); + Debug.Assert(offset >= 0); + Debug.Assert(length >= 0); + Debug.Assert(offset <= output.Length - length); + Debug.Assert((_bitsInBuffer % 8) == 0); + + // Copy the bytes in bitBuffer first. + int bytesFromBitBuffer = 0; + while (_bitsInBuffer > 0 && length > 0) + { + output[offset++] = (byte)_bitBuffer; + _bitBuffer >>= 8; + _bitsInBuffer -= 8; + length--; + bytesFromBitBuffer++; + } + + if (length == 0) + { + return bytesFromBitBuffer; + } + + int avail = _end - _start; + if (length > avail) + { + length = avail; + } + + Array.Copy(_buffer, _start, output, offset, length); + _start += length; + return bytesFromBitBuffer + length; + } + + /// + /// Return true is all input bytes are used. + /// This means the caller can call SetInput to add more input. + /// + public bool NeedsInput() => _start == _end; + + /// + /// Set the byte array to be processed. + /// All the bits remained in bitBuffer will be processed before the new bytes. + /// We don't clone the byte array here since it is expensive. + /// The caller should make sure after a buffer is passed in. + /// It will not be changed before calling this function again. + /// + public void SetInput(byte[] buffer, int offset, int length) + { + Debug.Assert(buffer != null); + Debug.Assert(offset >= 0); + Debug.Assert(length >= 0); + Debug.Assert(offset <= buffer.Length - length); + Debug.Assert(_start == _end); + + _buffer = buffer; + _start = offset; + _end = offset + length; + } + + /// Skip n bits in the buffer. + public void SkipBits(int n) + { + Debug.Assert(_bitsInBuffer >= n, "No enough bits in the buffer, Did you call EnsureBitsAvailable?"); + _bitBuffer >>= n; + _bitsInBuffer -= n; + } + + /// Skips to the next byte boundary. + public void SkipToByteBoundary() + { + _bitBuffer >>= (_bitsInBuffer % 8); + _bitsInBuffer = _bitsInBuffer - (_bitsInBuffer % 8); + } + } +} diff --git a/src/SharpCompress/Compressors/Deflate64/Match.cs b/src/SharpCompress/Compressors/Deflate64/Match.cs new file mode 100644 index 00000000..4d5ce54b --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/Match.cs @@ -0,0 +1,17 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +namespace SharpCompress.Compressors.Deflate64 +{ + /// + /// This class represents a match in the history window. + /// + internal sealed class Match + { + internal MatchState State { get; set; } + internal int Position { get; set; } + internal int Length { get; set; } + internal byte Symbol { get; set; } + } +} diff --git a/src/SharpCompress/Compressors/Deflate64/MatchState.cs b/src/SharpCompress/Compressors/Deflate64/MatchState.cs new file mode 100644 index 00000000..f88913bc --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/MatchState.cs @@ -0,0 +1,13 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +namespace SharpCompress.Compressors.Deflate64 +{ + internal enum MatchState + { + HasSymbol = 1, + HasMatch = 2, + HasSymbolAndMatch = 3 + } +} diff --git a/src/SharpCompress/Compressors/Deflate64/OutputWindow.cs b/src/SharpCompress/Compressors/Deflate64/OutputWindow.cs new file mode 100644 index 00000000..2dbc4442 --- /dev/null +++ b/src/SharpCompress/Compressors/Deflate64/OutputWindow.cs @@ -0,0 +1,151 @@ +// Licensed to the .NET Foundation under one or more agreements. +// The .NET Foundation licenses this file to you under the MIT license. +// See the LICENSE file in the project root for more information. + +using System; +using System.Diagnostics; + +namespace SharpCompress.Compressors.Deflate64 +{ + /// + /// This class maintains a window for decompressed output. + /// We need to keep this because the decompressed information can be + /// a literal or a length/distance pair. For length/distance pair, + /// we need to look back in the output window and copy bytes from there. + /// We use a byte array of WindowSize circularly. + /// + internal sealed class OutputWindow + { + // With Deflate64 we can have up to a 65536 length as well as up to a 65538 distance. This means we need a Window that is at + // least 131074 bytes long so we have space to retrieve up to a full 64kb in lookback and place it in our buffer without + // overwriting existing data. OutputWindow requires that the WindowSize be an exponent of 2, so we round up to 2^18. + private const int WINDOW_SIZE = 262144; + private const int WINDOW_MASK = 262143; + + private readonly byte[] _window = new byte[WINDOW_SIZE]; // The window is 2^18 bytes + private int _end; // this is the position to where we should write next byte + private int _bytesUsed; // The number of bytes in the output window which is not consumed. + + /// Add a byte to output window. + public void Write(byte b) + { + Debug.Assert(_bytesUsed < WINDOW_SIZE, "Can't add byte when window is full!"); + _window[_end++] = b; + _end &= WINDOW_MASK; + ++_bytesUsed; + } + + public void WriteLengthDistance(int length, int distance) + { + Debug.Assert((_bytesUsed + length) <= WINDOW_SIZE, "No Enough space"); + + // move backwards distance bytes in the output stream, + // and copy length bytes from this position to the output stream. + _bytesUsed += length; + int copyStart = (_end - distance) & WINDOW_MASK; // start position for coping. + + int border = WINDOW_SIZE - length; + if (copyStart <= border && _end < border) + { + if (length <= distance) + { + Array.Copy(_window, copyStart, _window, _end, length); + _end += length; + } + else + { + // The referenced string may overlap the current + // position; for example, if the last 2 bytes decoded have values + // X and Y, a string reference with + // adds X,Y,X,Y,X to the output stream. + while (length-- > 0) + { + _window[_end++] = _window[copyStart++]; + } + } + } + else + { + // copy byte by byte + while (length-- > 0) + { + _window[_end++] = _window[copyStart++]; + _end &= WINDOW_MASK; + copyStart &= WINDOW_MASK; + } + } + } + + /// + /// Copy up to length of bytes from input directly. + /// This is used for uncompressed block. + /// + public int CopyFrom(InputBuffer input, int length) + { + length = Math.Min(Math.Min(length, WINDOW_SIZE - _bytesUsed), input.AvailableBytes); + int copied; + + // We might need wrap around to copy all bytes. + int tailLen = WINDOW_SIZE - _end; + if (length > tailLen) + { + // copy the first part + copied = input.CopyTo(_window, _end, tailLen); + if (copied == tailLen) + { + // only try to copy the second part if we have enough bytes in input + copied += input.CopyTo(_window, 0, length - tailLen); + } + } + else + { + // only one copy is needed if there is no wrap around. + copied = input.CopyTo(_window, _end, length); + } + + _end = (_end + copied) & WINDOW_MASK; + _bytesUsed += copied; + return copied; + } + + /// Free space in output window. + public int FreeBytes => WINDOW_SIZE - _bytesUsed; + + /// Bytes not consumed in output window. + public int AvailableBytes => _bytesUsed; + + /// Copy the decompressed bytes to output array. + public int CopyTo(byte[] output, int offset, int length) + { + int copyEnd; + + if (length > _bytesUsed) + { + // we can copy all the decompressed bytes out + copyEnd = _end; + length = _bytesUsed; + } + else + { + copyEnd = (_end - _bytesUsed + length) & WINDOW_MASK; // copy length of bytes + } + + int copied = length; + + int tailLen = length - copyEnd; + if (tailLen > 0) + { + // this means we need to copy two parts separately + // copy tailLen bytes from the end of output window + Array.Copy(_window, WINDOW_SIZE - tailLen, + output, offset, tailLen); + offset += tailLen; + length = copyEnd; + } + Array.Copy(_window, copyEnd - length, output, offset, length); + _bytesUsed -= copied; + Debug.Assert(_bytesUsed >= 0, "check this function and find why we copied more bytes than we have"); + return copied; + } + } +} diff --git a/src/SharpCompress/Compressors/Filters/BCJ2Filter.cs b/src/SharpCompress/Compressors/Filters/BCJ2Filter.cs index 65af71ee..d767cd6c 100644 --- a/src/SharpCompress/Compressors/Filters/BCJ2Filter.cs +++ b/src/SharpCompress/Compressors/Filters/BCJ2Filter.cs @@ -5,36 +5,36 @@ namespace SharpCompress.Compressors.Filters { internal class BCJ2Filter : Stream { - private readonly Stream baseStream; - private readonly byte[] input = new byte[4096]; - private int inputOffset; - private int inputCount; - private bool endReached; + private readonly Stream _baseStream; + private readonly byte[] _input = new byte[4096]; + private int _inputOffset; + private int _inputCount; + private bool _endReached; - private long position; - private readonly byte[] output = new byte[4]; - private int outputOffset; - private int outputCount; + private long _position; + private readonly byte[] _output = new byte[4]; + private int _outputOffset; + private int _outputCount; - private readonly byte[] control; - private readonly byte[] data1; - private readonly byte[] data2; + private readonly byte[] _control; + private readonly byte[] _data1; + private readonly byte[] _data2; - private int controlPos; - private int data1Pos; - private int data2Pos; + private int _controlPos; + private int _data1Pos; + private int _data2Pos; - private readonly ushort[] p = new ushort[256 + 2]; - private uint range, code; - private byte prevByte; - private bool isDisposed; + private readonly ushort[] _p = new ushort[256 + 2]; + private uint _range, _code; + private byte _prevByte; + private bool _isDisposed; - private const int kNumTopBits = 24; - private const int kTopValue = 1 << kNumTopBits; + private const int K_NUM_TOP_BITS = 24; + private const int K_TOP_VALUE = 1 << K_NUM_TOP_BITS; - private const int kNumBitModelTotalBits = 11; - private const int kBitModelTotal = 1 << kNumBitModelTotalBits; - private const int kNumMoveBits = 5; + private const int K_NUM_BIT_MODEL_TOTAL_BITS = 11; + private const int K_BIT_MODEL_TOTAL = 1 << K_NUM_BIT_MODEL_TOTAL_BITS; + private const int K_NUM_MOVE_BITS = 5; private static bool IsJ(byte b0, byte b1) { @@ -48,34 +48,34 @@ namespace SharpCompress.Compressors.Filters public BCJ2Filter(byte[] control, byte[] data1, byte[] data2, Stream baseStream) { - this.control = control; - this.data1 = data1; - this.data2 = data2; - this.baseStream = baseStream; + _control = control; + _data1 = data1; + _data2 = data2; + _baseStream = baseStream; int i; - for (i = 0; i < p.Length; i++) + for (i = 0; i < _p.Length; i++) { - p[i] = kBitModelTotal >> 1; + _p[i] = K_BIT_MODEL_TOTAL >> 1; } - code = 0; - range = 0xFFFFFFFF; + _code = 0; + _range = 0xFFFFFFFF; for (i = 0; i < 5; i++) { - code = (code << 8) | control[controlPos++]; + _code = (_code << 8) | control[_controlPos++]; } } protected override void Dispose(bool disposing) { - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; + _isDisposed = true; base.Dispose(disposing); - baseStream.Dispose(); + _baseStream.Dispose(); } public override bool CanRead => true; @@ -89,57 +89,57 @@ namespace SharpCompress.Compressors.Filters throw new NotSupportedException(); } - public override long Length => baseStream.Length + data1.Length + data2.Length; + public override long Length => _baseStream.Length + _data1.Length + _data2.Length; - public override long Position { get => position; set => throw new NotSupportedException(); } + public override long Position { get => _position; set => throw new NotSupportedException(); } public override int Read(byte[] buffer, int offset, int count) { int size = 0; byte b = 0; - while (!endReached && size < count) + while (!_endReached && size < count) { - while (outputOffset < outputCount) + while (_outputOffset < _outputCount) { - b = output[outputOffset++]; + b = _output[_outputOffset++]; buffer[offset++] = b; size++; - position++; + _position++; - prevByte = b; + _prevByte = b; if (size == count) { return size; } } - if (inputOffset == inputCount) + if (_inputOffset == _inputCount) { - inputOffset = 0; - inputCount = baseStream.Read(input, 0, input.Length); - if (inputCount == 0) + _inputOffset = 0; + _inputCount = _baseStream.Read(_input, 0, _input.Length); + if (_inputCount == 0) { - endReached = true; + _endReached = true; break; } } - b = input[inputOffset++]; + b = _input[_inputOffset++]; buffer[offset++] = b; size++; - position++; + _position++; - if (!IsJ(prevByte, b)) + if (!IsJ(_prevByte, b)) { - prevByte = b; + _prevByte = b; } else { int prob; if (b == 0xE8) { - prob = prevByte; + prob = _prevByte; } else if (b == 0xE9) { @@ -150,27 +150,27 @@ namespace SharpCompress.Compressors.Filters prob = 257; } - uint bound = (range >> kNumBitModelTotalBits) * p[prob]; - if (code < bound) + uint bound = (_range >> K_NUM_BIT_MODEL_TOTAL_BITS) * _p[prob]; + if (_code < bound) { - range = bound; - p[prob] += (ushort)((kBitModelTotal - p[prob]) >> kNumMoveBits); - if (range < kTopValue) + _range = bound; + _p[prob] += (ushort)((K_BIT_MODEL_TOTAL - _p[prob]) >> K_NUM_MOVE_BITS); + if (_range < K_TOP_VALUE) { - range <<= 8; - code = (code << 8) | control[controlPos++]; + _range <<= 8; + _code = (_code << 8) | _control[_controlPos++]; } - prevByte = b; + _prevByte = b; } else { - range -= bound; - code -= bound; - p[prob] -= (ushort)(p[prob] >> kNumMoveBits); - if (range < kTopValue) + _range -= bound; + _code -= bound; + _p[prob] -= (ushort)(_p[prob] >> K_NUM_MOVE_BITS); + if (_range < K_TOP_VALUE) { - range <<= 8; - code = (code << 8) | control[controlPos++]; + _range <<= 8; + _code = (_code << 8) | _control[_controlPos++]; } uint dest; @@ -178,24 +178,24 @@ namespace SharpCompress.Compressors.Filters { dest = (uint) - ((data1[data1Pos++] << 24) | (data1[data1Pos++] << 16) | (data1[data1Pos++] << 8) | - data1[data1Pos++]); + ((_data1[_data1Pos++] << 24) | (_data1[_data1Pos++] << 16) | (_data1[_data1Pos++] << 8) | + _data1[_data1Pos++]); } else { dest = (uint) - ((data2[data2Pos++] << 24) | (data2[data2Pos++] << 16) | (data2[data2Pos++] << 8) | - data2[data2Pos++]); + ((_data2[_data2Pos++] << 24) | (_data2[_data2Pos++] << 16) | (_data2[_data2Pos++] << 8) | + _data2[_data2Pos++]); } - dest -= (uint)(position + 4); + dest -= (uint)(_position + 4); - output[0] = (byte)dest; - output[1] = (byte)(dest >> 8); - output[2] = (byte)(dest >> 16); - output[3] = (byte)(dest >> 24); - outputOffset = 0; - outputCount = 4; + _output[0] = (byte)dest; + _output[1] = (byte)(dest >> 8); + _output[2] = (byte)(dest >> 16); + _output[3] = (byte)(dest >> 24); + _outputOffset = 0; + _outputCount = 4; } } } diff --git a/src/SharpCompress/Compressors/Filters/BCJFilter.cs b/src/SharpCompress/Compressors/Filters/BCJFilter.cs index b782d2fd..3caa492b 100644 --- a/src/SharpCompress/Compressors/Filters/BCJFilter.cs +++ b/src/SharpCompress/Compressors/Filters/BCJFilter.cs @@ -8,16 +8,16 @@ namespace SharpCompress.Compressors.Filters private static readonly int[] MASK_TO_BIT_NUMBER = {0, 1, 2, 2, 3, 3, 3, 3}; - private int pos; - private int prevMask; + private int _pos; + private int _prevMask; public BCJFilter(bool isEncoder, Stream baseStream) : base(isEncoder, baseStream, 5) { - pos = 5; + _pos = 5; } - private static bool test86MSByte(byte b) + private static bool Test86MsByte(byte b) { return b == 0x00 || b == 0xFF; } @@ -39,18 +39,18 @@ namespace SharpCompress.Compressors.Filters if ((prevPos & ~3) != 0) { // (unsigned)prevPos > 3 - prevMask = 0; + _prevMask = 0; } else { - prevMask = (prevMask << (prevPos - 1)) & 7; - if (prevMask != 0) + _prevMask = (_prevMask << (prevPos - 1)) & 7; + if (_prevMask != 0) { - if (!MASK_TO_ALLOWED_STATUS[prevMask] || test86MSByte( - buffer[i + 4 - MASK_TO_BIT_NUMBER[prevMask]])) + if (!MASK_TO_ALLOWED_STATUS[_prevMask] || Test86MsByte( + buffer[i + 4 - MASK_TO_BIT_NUMBER[_prevMask]])) { prevPos = i; - prevMask = (prevMask << 1) | 1; + _prevMask = (_prevMask << 1) | 1; continue; } } @@ -58,7 +58,7 @@ namespace SharpCompress.Compressors.Filters prevPos = i; - if (test86MSByte(buffer[i + 4])) + if (Test86MsByte(buffer[i + 4])) { int src = buffer[i + 1] | (buffer[i + 2] << 8) @@ -67,22 +67,22 @@ namespace SharpCompress.Compressors.Filters int dest; while (true) { - if (isEncoder) + if (_isEncoder) { - dest = src + (pos + i - offset); + dest = src + (_pos + i - offset); } else { - dest = src - (pos + i - offset); + dest = src - (_pos + i - offset); } - if (prevMask == 0) + if (_prevMask == 0) { break; } - int index = MASK_TO_BIT_NUMBER[prevMask] * 8; - if (!test86MSByte((byte)(dest >> (24 - index)))) + int index = MASK_TO_BIT_NUMBER[_prevMask] * 8; + if (!Test86MsByte((byte)(dest >> (24 - index)))) { break; } @@ -98,15 +98,15 @@ namespace SharpCompress.Compressors.Filters } else { - prevMask = (prevMask << 1) | 1; + _prevMask = (_prevMask << 1) | 1; } } prevPos = i - prevPos; - prevMask = ((prevPos & ~3) != 0) ? 0 : prevMask << (prevPos - 1); + _prevMask = ((prevPos & ~3) != 0) ? 0 : _prevMask << (prevPos - 1); i -= offset; - pos += i; + _pos += i; return i; } } diff --git a/src/SharpCompress/Compressors/Filters/Filter.cs b/src/SharpCompress/Compressors/Filters/Filter.cs index c0b23eb1..b7fd8159 100644 --- a/src/SharpCompress/Compressors/Filters/Filter.cs +++ b/src/SharpCompress/Compressors/Filters/Filter.cs @@ -5,109 +5,109 @@ namespace SharpCompress.Compressors.Filters { internal abstract class Filter : Stream { - protected bool isEncoder; - protected Stream baseStream; + protected bool _isEncoder; + protected Stream _baseStream; - private readonly byte[] tail; - private readonly byte[] window; - private int transformed; - private int read; - private bool endReached; - private bool isDisposed; + private readonly byte[] _tail; + private readonly byte[] _window; + private int _transformed; + private int _read; + private bool _endReached; + private bool _isDisposed; protected Filter(bool isEncoder, Stream baseStream, int lookahead) { - this.isEncoder = isEncoder; - this.baseStream = baseStream; - tail = new byte[lookahead - 1]; - window = new byte[tail.Length * 2]; + _isEncoder = isEncoder; + _baseStream = baseStream; + _tail = new byte[lookahead - 1]; + _window = new byte[_tail.Length * 2]; } protected override void Dispose(bool disposing) { - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; + _isDisposed = true; base.Dispose(disposing); - baseStream.Dispose(); + _baseStream.Dispose(); } - public override bool CanRead => !isEncoder; + public override bool CanRead => !_isEncoder; public override bool CanSeek => false; - public override bool CanWrite => isEncoder; + public override bool CanWrite => _isEncoder; public override void Flush() { throw new NotSupportedException(); } - public override long Length => baseStream.Length; + public override long Length => _baseStream.Length; - public override long Position { get => baseStream.Position; set => throw new NotSupportedException(); } + public override long Position { get => _baseStream.Position; set => throw new NotSupportedException(); } public override int Read(byte[] buffer, int offset, int count) { int size = 0; - if (transformed > 0) + if (_transformed > 0) { - int copySize = transformed; + int copySize = _transformed; if (copySize > count) { copySize = count; } - Buffer.BlockCopy(tail, 0, buffer, offset, copySize); - transformed -= copySize; - read -= copySize; + Buffer.BlockCopy(_tail, 0, buffer, offset, copySize); + _transformed -= copySize; + _read -= copySize; offset += copySize; count -= copySize; size += copySize; - Buffer.BlockCopy(tail, copySize, tail, 0, read); + Buffer.BlockCopy(_tail, copySize, _tail, 0, _read); } if (count == 0) { return size; } - int inSize = read; + int inSize = _read; if (inSize > count) { inSize = count; } - Buffer.BlockCopy(tail, 0, buffer, offset, inSize); - read -= inSize; - Buffer.BlockCopy(tail, inSize, tail, 0, read); - while (!endReached && inSize < count) + Buffer.BlockCopy(_tail, 0, buffer, offset, inSize); + _read -= inSize; + Buffer.BlockCopy(_tail, inSize, _tail, 0, _read); + while (!_endReached && inSize < count) { - int baseRead = baseStream.Read(buffer, offset + inSize, count - inSize); + int baseRead = _baseStream.Read(buffer, offset + inSize, count - inSize); inSize += baseRead; if (baseRead == 0) { - endReached = true; + _endReached = true; } } - while (!endReached && read < tail.Length) + while (!_endReached && _read < _tail.Length) { - int baseRead = baseStream.Read(tail, read, tail.Length - read); - read += baseRead; + int baseRead = _baseStream.Read(_tail, _read, _tail.Length - _read); + _read += baseRead; if (baseRead == 0) { - endReached = true; + _endReached = true; } } - if (inSize > tail.Length) + if (inSize > _tail.Length) { - transformed = Transform(buffer, offset, inSize); - offset += transformed; - count -= transformed; - size += transformed; - inSize -= transformed; - transformed = 0; + _transformed = Transform(buffer, offset, inSize); + offset += _transformed; + count -= _transformed; + size += _transformed; + inSize -= _transformed; + _transformed = 0; } if (count == 0) @@ -115,20 +115,20 @@ namespace SharpCompress.Compressors.Filters return size; } - Buffer.BlockCopy(buffer, offset, window, 0, inSize); - Buffer.BlockCopy(tail, 0, window, inSize, read); - if (inSize + read > tail.Length) + Buffer.BlockCopy(buffer, offset, _window, 0, inSize); + Buffer.BlockCopy(_tail, 0, _window, inSize, _read); + if (inSize + _read > _tail.Length) { - transformed = Transform(window, 0, inSize + read); + _transformed = Transform(_window, 0, inSize + _read); } else { - transformed = inSize + read; + _transformed = inSize + _read; } - Buffer.BlockCopy(window, 0, buffer, offset, inSize); - Buffer.BlockCopy(window, inSize, tail, 0, read); + Buffer.BlockCopy(_window, 0, buffer, offset, inSize); + Buffer.BlockCopy(_window, inSize, _tail, 0, _read); size += inSize; - transformed -= inSize; + _transformed -= inSize; return size; } @@ -146,7 +146,7 @@ namespace SharpCompress.Compressors.Filters public override void Write(byte[] buffer, int offset, int count) { Transform(buffer, offset, count); - baseStream.Write(buffer, offset, count); + _baseStream.Write(buffer, offset, count); } protected abstract int Transform(byte[] buffer, int offset, int count); diff --git a/src/SharpCompress/Compressors/LZMA/Bcj2DecoderStream.cs b/src/SharpCompress/Compressors/LZMA/Bcj2DecoderStream.cs index 6b37c53c..4bdac69b 100644 --- a/src/SharpCompress/Compressors/LZMA/Bcj2DecoderStream.cs +++ b/src/SharpCompress/Compressors/LZMA/Bcj2DecoderStream.cs @@ -6,28 +6,28 @@ namespace SharpCompress.Compressors.LZMA { internal class Bcj2DecoderStream : DecoderStream2 { - private const int kNumTopBits = 24; - private const uint kTopValue = (1 << kNumTopBits); + private const int K_NUM_TOP_BITS = 24; + private const uint K_TOP_VALUE = (1 << K_NUM_TOP_BITS); private class RangeDecoder { - internal readonly Stream mStream; - internal uint Range; - internal uint Code; + internal readonly Stream _mStream; + internal uint _range; + internal uint _code; public RangeDecoder(Stream stream) { - mStream = stream; - Range = 0xFFFFFFFF; + _mStream = stream; + _range = 0xFFFFFFFF; for (int i = 0; i < 5; i++) { - Code = (Code << 8) | ReadByte(); + _code = (_code << 8) | ReadByte(); } } public byte ReadByte() { - int bt = mStream.ReadByte(); + int bt = _mStream.ReadByte(); if (bt < 0) { throw new EndOfStreamException(); @@ -38,22 +38,22 @@ namespace SharpCompress.Compressors.LZMA public void Dispose() { - mStream.Dispose(); + _mStream.Dispose(); } } private class StatusDecoder { - private const int numMoveBits = 5; + private const int NUM_MOVE_BITS = 5; - private const int kNumBitModelTotalBits = 11; - private const uint kBitModelTotal = 1u << kNumBitModelTotalBits; + private const int K_NUM_BIT_MODEL_TOTAL_BITS = 11; + private const uint K_BIT_MODEL_TOTAL = 1u << K_NUM_BIT_MODEL_TOTAL_BITS; - private uint Prob; + private uint _prob; public StatusDecoder() { - Prob = kBitModelTotal / 2; + _prob = K_BIT_MODEL_TOTAL / 2; } private void UpdateModel(uint symbol) @@ -64,49 +64,49 @@ namespace SharpCompress.Compressors.LZMA */ if (symbol == 0) { - Prob += (kBitModelTotal - Prob) >> numMoveBits; + _prob += (K_BIT_MODEL_TOTAL - _prob) >> NUM_MOVE_BITS; } else { - Prob -= (Prob) >> numMoveBits; + _prob -= (_prob) >> NUM_MOVE_BITS; } } public uint Decode(RangeDecoder decoder) { - uint newBound = (decoder.Range >> kNumBitModelTotalBits) * Prob; - if (decoder.Code < newBound) + uint newBound = (decoder._range >> K_NUM_BIT_MODEL_TOTAL_BITS) * _prob; + if (decoder._code < newBound) { - decoder.Range = newBound; - Prob += (kBitModelTotal - Prob) >> numMoveBits; - if (decoder.Range < kTopValue) + decoder._range = newBound; + _prob += (K_BIT_MODEL_TOTAL - _prob) >> NUM_MOVE_BITS; + if (decoder._range < K_TOP_VALUE) { - decoder.Code = (decoder.Code << 8) | decoder.ReadByte(); - decoder.Range <<= 8; + decoder._code = (decoder._code << 8) | decoder.ReadByte(); + decoder._range <<= 8; } return 0; } - decoder.Range -= newBound; - decoder.Code -= newBound; - Prob -= Prob >> numMoveBits; - if (decoder.Range < kTopValue) + decoder._range -= newBound; + decoder._code -= newBound; + _prob -= _prob >> NUM_MOVE_BITS; + if (decoder._range < K_TOP_VALUE) { - decoder.Code = (decoder.Code << 8) | decoder.ReadByte(); - decoder.Range <<= 8; + decoder._code = (decoder._code << 8) | decoder.ReadByte(); + decoder._range <<= 8; } return 1; } } - private readonly Stream mMainStream; - private readonly Stream mCallStream; - private readonly Stream mJumpStream; - private readonly RangeDecoder mRangeDecoder; - private readonly StatusDecoder[] mStatusDecoder; - private long mWritten; - private readonly IEnumerator mIter; - private bool mFinished; - private bool isDisposed; + private readonly Stream _mMainStream; + private readonly Stream _mCallStream; + private readonly Stream _mJumpStream; + private readonly RangeDecoder _mRangeDecoder; + private readonly StatusDecoder[] _mStatusDecoder; + private long _mWritten; + private readonly IEnumerator _mIter; + private bool _mFinished; + private bool _isDisposed; public Bcj2DecoderStream(Stream[] streams, byte[] info, long limit) { @@ -120,31 +120,31 @@ namespace SharpCompress.Compressors.LZMA throw new NotSupportedException(); } - mMainStream = streams[0]; - mCallStream = streams[1]; - mJumpStream = streams[2]; - mRangeDecoder = new RangeDecoder(streams[3]); + _mMainStream = streams[0]; + _mCallStream = streams[1]; + _mJumpStream = streams[2]; + _mRangeDecoder = new RangeDecoder(streams[3]); - mStatusDecoder = new StatusDecoder[256 + 2]; - for (int i = 0; i < mStatusDecoder.Length; i++) + _mStatusDecoder = new StatusDecoder[256 + 2]; + for (int i = 0; i < _mStatusDecoder.Length; i++) { - mStatusDecoder[i] = new StatusDecoder(); + _mStatusDecoder[i] = new StatusDecoder(); } - mIter = Run().GetEnumerator(); + _mIter = Run().GetEnumerator(); } protected override void Dispose(bool disposing) { - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; + _isDisposed = true; base.Dispose(disposing); - mMainStream.Dispose(); - mCallStream.Dispose(); - mJumpStream.Dispose(); + _mMainStream.Dispose(); + _mCallStream.Dispose(); + _mJumpStream.Dispose(); } private static bool IsJcc(byte b0, byte b1) @@ -174,25 +174,41 @@ namespace SharpCompress.Compressors.LZMA public override int Read(byte[] buffer, int offset, int count) { - if (count == 0 || mFinished) + if (count == 0 || _mFinished) { return 0; } for (int i = 0; i < count; i++) { - if (!mIter.MoveNext()) + if (!_mIter.MoveNext()) { - mFinished = true; + _mFinished = true; return i; } - buffer[offset + i] = mIter.Current; + buffer[offset + i] = _mIter.Current; } return count; } + public override int ReadByte() + { + if (_mFinished) + { + return -1; + } + + if (!_mIter.MoveNext()) + { + _mFinished = true; + return -1; + } + + return _mIter.Current; + } + public IEnumerable Run() { const uint kBurstSize = (1u << 18); @@ -205,14 +221,14 @@ namespace SharpCompress.Compressors.LZMA uint i; for (i = 0; i < kBurstSize; i++) { - int tmp = mMainStream.ReadByte(); + int tmp = _mMainStream.ReadByte(); if (tmp < 0) { yield break; } b = (byte)tmp; - mWritten++; + _mWritten++; yield return b; if (IsJ(prevByte, b)) { @@ -228,9 +244,9 @@ namespace SharpCompress.Compressors.LZMA continue; } - if (mStatusDecoder[GetIndex(prevByte, b)].Decode(mRangeDecoder) == 1) + if (_mStatusDecoder[GetIndex(prevByte, b)].Decode(_mRangeDecoder) == 1) { - Stream s = (b == 0xE8) ? mCallStream : mJumpStream; + Stream s = (b == 0xE8) ? _mCallStream : _mJumpStream; uint src = 0; for (i = 0; i < 4; i++) @@ -245,14 +261,14 @@ namespace SharpCompress.Compressors.LZMA src |= (uint)b0; } - uint dest = src - (uint)(mWritten + 4); - mWritten++; + uint dest = src - (uint)(_mWritten + 4); + _mWritten++; yield return (byte)dest; - mWritten++; + _mWritten++; yield return (byte)(dest >> 8); - mWritten++; + _mWritten++; yield return (byte)(dest >> 16); - mWritten++; + _mWritten++; yield return (byte)(dest >> 24); prevByte = (byte)(dest >> 24); processedBytes += 4; diff --git a/src/SharpCompress/Compressors/LZMA/BitVector.cs b/src/SharpCompress/Compressors/LZMA/BitVector.cs index f3934c4e..9f267f72 100644 --- a/src/SharpCompress/Compressors/LZMA/BitVector.cs +++ b/src/SharpCompress/Compressors/LZMA/BitVector.cs @@ -6,24 +6,24 @@ namespace SharpCompress.Compressors.LZMA { internal class BitVector { - private readonly uint[] mBits; + private readonly uint[] _mBits; public BitVector(int length) { Length = length; - mBits = new uint[(length + 31) >> 5]; + _mBits = new uint[(length + 31) >> 5]; } public BitVector(int length, bool initValue) { Length = length; - mBits = new uint[(length + 31) >> 5]; + _mBits = new uint[(length + 31) >> 5]; if (initValue) { - for (int i = 0; i < mBits.Length; i++) + for (int i = 0; i < _mBits.Length; i++) { - mBits[i] = ~0u; + _mBits[i] = ~0u; } } } @@ -61,7 +61,7 @@ namespace SharpCompress.Compressors.LZMA throw new ArgumentOutOfRangeException(nameof(index)); } - return (mBits[index >> 5] & (1u << (index & 31))) != 0; + return (_mBits[index >> 5] & (1u << (index & 31))) != 0; } } @@ -72,7 +72,7 @@ namespace SharpCompress.Compressors.LZMA throw new ArgumentOutOfRangeException(nameof(index)); } - mBits[index >> 5] |= 1u << (index & 31); + _mBits[index >> 5] |= 1u << (index & 31); } internal bool GetAndSet(int index) @@ -82,9 +82,9 @@ namespace SharpCompress.Compressors.LZMA throw new ArgumentOutOfRangeException(nameof(index)); } - uint bits = mBits[index >> 5]; + uint bits = _mBits[index >> 5]; uint mask = 1u << (index & 31); - mBits[index >> 5] |= mask; + _mBits[index >> 5] |= mask; return (bits & mask) != 0; } diff --git a/src/SharpCompress/Compressors/LZMA/CRC.cs b/src/SharpCompress/Compressors/LZMA/CRC.cs index 92e5851d..2ee0600a 100644 --- a/src/SharpCompress/Compressors/LZMA/CRC.cs +++ b/src/SharpCompress/Compressors/LZMA/CRC.cs @@ -3,12 +3,12 @@ using System.IO; namespace SharpCompress.Compressors.LZMA { - internal static class CRC + internal static class Crc { - public const uint kInitCRC = 0xFFFFFFFF; - private static readonly uint[] kTable = new uint[4 * 256]; + internal const uint INIT_CRC = 0xFFFFFFFF; + internal static readonly uint[] TABLE = new uint[4 * 256]; - static CRC() + static Crc() { const uint kCrcPoly = 0xEDB88320; @@ -20,19 +20,19 @@ namespace SharpCompress.Compressors.LZMA r = (r >> 1) ^ (kCrcPoly & ~((r & 1) - 1)); } - kTable[i] = r; + TABLE[i] = r; } - for (uint i = 256; i < kTable.Length; i++) + for (uint i = 256; i < TABLE.Length; i++) { - uint r = kTable[i - 256]; - kTable[i] = kTable[r & 0xFF] ^ (r >> 8); + uint r = TABLE[i - 256]; + TABLE[i] = TABLE[r & 0xFF] ^ (r >> 8); } } public static uint From(Stream stream, long length) { - uint crc = kInitCRC; + uint crc = INIT_CRC; byte[] buffer = new byte[Math.Min(length, 4 << 10)]; while (length > 0) { @@ -54,16 +54,16 @@ namespace SharpCompress.Compressors.LZMA public static uint Update(uint crc, byte bt) { - return kTable[(crc & 0xFF) ^ bt] ^ (crc >> 8); + return TABLE[(crc & 0xFF) ^ bt] ^ (crc >> 8); } public static uint Update(uint crc, uint value) { crc ^= value; - return kTable[0x300 + (crc & 0xFF)] - ^ kTable[0x200 + ((crc >> 8) & 0xFF)] - ^ kTable[0x100 + ((crc >> 16) & 0xFF)] - ^ kTable[0x000 + (crc >> 24)]; + return TABLE[0x300 + (crc & 0xFF)] + ^ TABLE[0x200 + ((crc >> 8) & 0xFF)] + ^ TABLE[0x100 + ((crc >> 16) & 0xFF)] + ^ TABLE[0x000 + (crc >> 24)]; } public static uint Update(uint crc, ulong value) diff --git a/src/SharpCompress/Compressors/LZMA/DecoderStream.cs b/src/SharpCompress/Compressors/LZMA/DecoderStream.cs index c9c6fd21..1921ebab 100644 --- a/src/SharpCompress/Compressors/LZMA/DecoderStream.cs +++ b/src/SharpCompress/Compressors/LZMA/DecoderStream.cs @@ -43,10 +43,10 @@ namespace SharpCompress.Compressors.LZMA { private static int FindCoderIndexForOutStreamIndex(CFolder folderInfo, int outStreamIndex) { - for (int coderIndex = 0; coderIndex < folderInfo.Coders.Count; coderIndex++) + for (int coderIndex = 0; coderIndex < folderInfo._coders.Count; coderIndex++) { - var coderInfo = folderInfo.Coders[coderIndex]; - outStreamIndex -= coderInfo.NumOutStreams; + var coderInfo = folderInfo._coders[coderIndex]; + outStreamIndex -= coderInfo._numOutStreams; if (outStreamIndex < 0) { return coderIndex; @@ -64,11 +64,11 @@ namespace SharpCompress.Compressors.LZMA primaryOutStreamIndex = -1; for (int outStreamIndex = 0, coderIndex = 0; - coderIndex < folderInfo.Coders.Count; + coderIndex < folderInfo._coders.Count; coderIndex++) { for (int coderOutStreamIndex = 0; - coderOutStreamIndex < folderInfo.Coders[coderIndex].NumOutStreams; + coderOutStreamIndex < folderInfo._coders[coderIndex]._numOutStreams; coderOutStreamIndex++, outStreamIndex++) { if (folderInfo.FindBindPairForOutStream(outStreamIndex) < 0) @@ -94,8 +94,8 @@ namespace SharpCompress.Compressors.LZMA private static Stream CreateDecoderStream(Stream[] packStreams, long[] packSizes, Stream[] outStreams, CFolder folderInfo, int coderIndex, IPasswordProvider pass) { - var coderInfo = folderInfo.Coders[coderIndex]; - if (coderInfo.NumOutStreams != 1) + var coderInfo = folderInfo._coders[coderIndex]; + if (coderInfo._numOutStreams != 1) { throw new NotSupportedException("Multiple output streams are not supported."); } @@ -103,23 +103,23 @@ namespace SharpCompress.Compressors.LZMA int inStreamId = 0; for (int i = 0; i < coderIndex; i++) { - inStreamId += folderInfo.Coders[i].NumInStreams; + inStreamId += folderInfo._coders[i]._numInStreams; } int outStreamId = 0; for (int i = 0; i < coderIndex; i++) { - outStreamId += folderInfo.Coders[i].NumOutStreams; + outStreamId += folderInfo._coders[i]._numOutStreams; } - Stream[] inStreams = new Stream[coderInfo.NumInStreams]; + Stream[] inStreams = new Stream[coderInfo._numInStreams]; for (int i = 0; i < inStreams.Length; i++, inStreamId++) { int bindPairIndex = folderInfo.FindBindPairForInStream(inStreamId); if (bindPairIndex >= 0) { - int pairedOutIndex = folderInfo.BindPairs[bindPairIndex].OutIndex; + int pairedOutIndex = folderInfo._bindPairs[bindPairIndex]._outIndex; if (outStreams[pairedOutIndex] != null) { @@ -153,8 +153,8 @@ namespace SharpCompress.Compressors.LZMA } } - long unpackSize = folderInfo.UnpackSizes[outStreamId]; - return DecoderRegistry.CreateDecoderStream(coderInfo.MethodId, inStreams, coderInfo.Props, pass, unpackSize); + long unpackSize = folderInfo._unpackSizes[outStreamId]; + return DecoderRegistry.CreateDecoderStream(coderInfo._methodId, inStreams, coderInfo._props, pass, unpackSize); } internal static Stream CreateDecoderStream(Stream inStream, long startPos, long[] packSizes, CFolder folderInfo, @@ -165,14 +165,14 @@ namespace SharpCompress.Compressors.LZMA throw new NotSupportedException("Unsupported stream binding structure."); } - Stream[] inStreams = new Stream[folderInfo.PackStreams.Count]; - for (int j = 0; j < folderInfo.PackStreams.Count; j++) + Stream[] inStreams = new Stream[folderInfo._packStreams.Count]; + for (int j = 0; j < folderInfo._packStreams.Count; j++) { inStreams[j] = new BufferedSubStream(inStream, startPos, packSizes[j]); startPos += packSizes[j]; } - Stream[] outStreams = new Stream[folderInfo.UnpackSizes.Count]; + Stream[] outStreams = new Stream[folderInfo._unpackSizes.Count]; int primaryCoderIndex, primaryOutStreamIndex; FindPrimaryOutStreamIndex(folderInfo, out primaryCoderIndex, out primaryOutStreamIndex); diff --git a/src/SharpCompress/Compressors/LZMA/ICoder.cs b/src/SharpCompress/Compressors/LZMA/ICoder.cs index 6e00503c..f5049dd7 100644 --- a/src/SharpCompress/Compressors/LZMA/ICoder.cs +++ b/src/SharpCompress/Compressors/LZMA/ICoder.cs @@ -77,7 +77,7 @@ namespace SharpCompress.Compressors.LZMA /// /// Provides the fields that represent properties idenitifiers for compressing. /// - internal enum CoderPropID + internal enum CoderPropId { /// /// Specifies default property. @@ -157,7 +157,7 @@ namespace SharpCompress.Compressors.LZMA internal interface ISetCoderProperties { - void SetCoderProperties(CoderPropID[] propIDs, object[] properties); + void SetCoderProperties(CoderPropId[] propIDs, object[] properties); } internal interface IWriteCoderProperties diff --git a/src/SharpCompress/Compressors/LZMA/LZ/CRC.cs b/src/SharpCompress/Compressors/LZMA/LZ/CRC.cs deleted file mode 100644 index a6c560d3..00000000 --- a/src/SharpCompress/Compressors/LZMA/LZ/CRC.cs +++ /dev/null @@ -1,68 +0,0 @@ -namespace SharpCompress.Compressors.LZMA.LZ -{ - internal class CRC - { - public static readonly uint[] Table; - - static CRC() - { - Table = new uint[256]; - const uint kPoly = 0xEDB88320; - for (uint i = 0; i < 256; i++) - { - uint r = i; - for (int j = 0; j < 8; j++) - { - if ((r & 1) != 0) - { - r = (r >> 1) ^ kPoly; - } - else - { - r >>= 1; - } - } - Table[i] = r; - } - } - - private uint _value = 0xFFFFFFFF; - - public void Init() - { - _value = 0xFFFFFFFF; - } - - public void UpdateByte(byte b) - { - _value = Table[(((byte)(_value)) ^ b)] ^ (_value >> 8); - } - - public void Update(byte[] data, uint offset, uint size) - { - for (uint i = 0; i < size; i++) - { - _value = Table[(((byte)(_value)) ^ data[offset + i])] ^ (_value >> 8); - } - } - - public uint GetDigest() - { - return _value ^ 0xFFFFFFFF; - } - - private static uint CalculateDigest(byte[] data, uint offset, uint size) - { - CRC crc = new CRC(); - - // crc.Init(); - crc.Update(data, offset, size); - return crc.GetDigest(); - } - - private static bool VerifyDigest(uint digest, byte[] data, uint offset, uint size) - { - return (CalculateDigest(data, offset, size) == digest); - } - } -} \ No newline at end of file diff --git a/src/SharpCompress/Compressors/LZMA/LZ/LzBinTree.cs b/src/SharpCompress/Compressors/LZMA/LZ/LzBinTree.cs index 79ecceb8..bfa34b84 100644 --- a/src/SharpCompress/Compressors/LZMA/LZ/LzBinTree.cs +++ b/src/SharpCompress/Compressors/LZMA/LZ/LzBinTree.cs @@ -16,34 +16,34 @@ namespace SharpCompress.Compressors.LZMA.LZ private UInt32 _hashMask; private UInt32 _hashSizeSum; - private bool HASH_ARRAY = true; + private bool _hashArray = true; - private const UInt32 kHash2Size = 1 << 10; - private const UInt32 kHash3Size = 1 << 16; - private const UInt32 kBT2HashSize = 1 << 16; - private const UInt32 kStartMaxLen = 1; - private const UInt32 kHash3Offset = kHash2Size; - private const UInt32 kEmptyHashValue = 0; - private const UInt32 kMaxValForNormalize = ((UInt32)1 << 31) - 1; + private const UInt32 K_HASH2_SIZE = 1 << 10; + private const UInt32 K_HASH3_SIZE = 1 << 16; + private const UInt32 K_BT2_HASH_SIZE = 1 << 16; + private const UInt32 K_START_MAX_LEN = 1; + private const UInt32 K_HASH3_OFFSET = K_HASH2_SIZE; + private const UInt32 K_EMPTY_HASH_VALUE = 0; + private const UInt32 K_MAX_VAL_FOR_NORMALIZE = ((UInt32)1 << 31) - 1; - private UInt32 kNumHashDirectBytes; - private UInt32 kMinMatchCheck = 4; - private UInt32 kFixHashSize = kHash2Size + kHash3Size; + private UInt32 _kNumHashDirectBytes; + private UInt32 _kMinMatchCheck = 4; + private UInt32 _kFixHashSize = K_HASH2_SIZE + K_HASH3_SIZE; public void SetType(int numHashBytes) { - HASH_ARRAY = (numHashBytes > 2); - if (HASH_ARRAY) + _hashArray = (numHashBytes > 2); + if (_hashArray) { - kNumHashDirectBytes = 0; - kMinMatchCheck = 4; - kFixHashSize = kHash2Size + kHash3Size; + _kNumHashDirectBytes = 0; + _kMinMatchCheck = 4; + _kFixHashSize = K_HASH2_SIZE + K_HASH3_SIZE; } else { - kNumHashDirectBytes = 2; - kMinMatchCheck = 2 + 1; - kFixHashSize = 0; + _kNumHashDirectBytes = 2; + _kMinMatchCheck = 2 + 1; + _kFixHashSize = 0; } } @@ -62,7 +62,7 @@ namespace SharpCompress.Compressors.LZMA.LZ base.Init(); for (UInt32 i = 0; i < _hashSizeSum; i++) { - _hash[i] = kEmptyHashValue; + _hash[i] = K_EMPTY_HASH_VALUE; } _cyclicBufferPos = 0; ReduceOffsets(-1); @@ -75,7 +75,7 @@ namespace SharpCompress.Compressors.LZMA.LZ _cyclicBufferPos = 0; } base.MovePos(); - if (_pos == kMaxValForNormalize) + if (_pos == K_MAX_VAL_FOR_NORMALIZE) { Normalize(); } @@ -99,7 +99,7 @@ namespace SharpCompress.Compressors.LZMA.LZ public void Create(UInt32 historySize, UInt32 keepAddBufferBefore, UInt32 matchMaxLen, UInt32 keepAddBufferAfter) { - if (historySize > kMaxValForNormalize - 256) + if (historySize > K_MAX_VAL_FOR_NORMALIZE - 256) { throw new Exception(); } @@ -118,9 +118,9 @@ namespace SharpCompress.Compressors.LZMA.LZ _son = new UInt32[(_cyclicBufferSize = cyclicBufferSize) * 2]; } - UInt32 hs = kBT2HashSize; + UInt32 hs = K_BT2_HASH_SIZE; - if (HASH_ARRAY) + if (_hashArray) { hs = historySize - 1; hs |= (hs >> 1); @@ -135,7 +135,7 @@ namespace SharpCompress.Compressors.LZMA.LZ } _hashMask = hs; hs++; - hs += kFixHashSize; + hs += _kFixHashSize; } if (hs != _hashSizeSum) { @@ -153,7 +153,7 @@ namespace SharpCompress.Compressors.LZMA.LZ else { lenLimit = _streamPos - _pos; - if (lenLimit < kMinMatchCheck) + if (lenLimit < _kMinMatchCheck) { MovePos(); return 0; @@ -163,29 +163,29 @@ namespace SharpCompress.Compressors.LZMA.LZ UInt32 offset = 0; UInt32 matchMinPos = (_pos > _cyclicBufferSize) ? (_pos - _cyclicBufferSize) : 0; UInt32 cur = _bufferOffset + _pos; - UInt32 maxLen = kStartMaxLen; // to avoid items for len < hashSize; + UInt32 maxLen = K_START_MAX_LEN; // to avoid items for len < hashSize; UInt32 hashValue, hash2Value = 0, hash3Value = 0; - if (HASH_ARRAY) + if (_hashArray) { - UInt32 temp = CRC.Table[_bufferBase[cur]] ^ _bufferBase[cur + 1]; - hash2Value = temp & (kHash2Size - 1); + UInt32 temp = Crc.TABLE[_bufferBase[cur]] ^ _bufferBase[cur + 1]; + hash2Value = temp & (K_HASH2_SIZE - 1); temp ^= ((UInt32)(_bufferBase[cur + 2]) << 8); - hash3Value = temp & (kHash3Size - 1); - hashValue = (temp ^ (CRC.Table[_bufferBase[cur + 3]] << 5)) & _hashMask; + hash3Value = temp & (K_HASH3_SIZE - 1); + hashValue = (temp ^ (Crc.TABLE[_bufferBase[cur + 3]] << 5)) & _hashMask; } else { hashValue = _bufferBase[cur] ^ ((UInt32)(_bufferBase[cur + 1]) << 8); } - UInt32 curMatch = _hash[kFixHashSize + hashValue]; - if (HASH_ARRAY) + UInt32 curMatch = _hash[_kFixHashSize + hashValue]; + if (_hashArray) { UInt32 curMatch2 = _hash[hash2Value]; - UInt32 curMatch3 = _hash[kHash3Offset + hash3Value]; + UInt32 curMatch3 = _hash[K_HASH3_OFFSET + hash3Value]; _hash[hash2Value] = _pos; - _hash[kHash3Offset + hash3Value] = _pos; + _hash[K_HASH3_OFFSET + hash3Value] = _pos; if (curMatch2 > matchMinPos) { if (_bufferBase[_bufferOffset + curMatch2] == _bufferBase[cur]) @@ -210,26 +210,26 @@ namespace SharpCompress.Compressors.LZMA.LZ if (offset != 0 && curMatch2 == curMatch) { offset -= 2; - maxLen = kStartMaxLen; + maxLen = K_START_MAX_LEN; } } - _hash[kFixHashSize + hashValue] = _pos; + _hash[_kFixHashSize + hashValue] = _pos; UInt32 ptr0 = (_cyclicBufferPos << 1) + 1; UInt32 ptr1 = (_cyclicBufferPos << 1); UInt32 len0, len1; - len0 = len1 = kNumHashDirectBytes; + len0 = len1 = _kNumHashDirectBytes; - if (kNumHashDirectBytes != 0) + if (_kNumHashDirectBytes != 0) { if (curMatch > matchMinPos) { - if (_bufferBase[_bufferOffset + curMatch + kNumHashDirectBytes] != - _bufferBase[cur + kNumHashDirectBytes]) + if (_bufferBase[_bufferOffset + curMatch + _kNumHashDirectBytes] != + _bufferBase[cur + _kNumHashDirectBytes]) { - distances[offset++] = maxLen = kNumHashDirectBytes; + distances[offset++] = maxLen = _kNumHashDirectBytes; distances[offset++] = _pos - curMatch - 1; } } @@ -241,7 +241,7 @@ namespace SharpCompress.Compressors.LZMA.LZ { if (curMatch <= matchMinPos || count-- == 0) { - _son[ptr0] = _son[ptr1] = kEmptyHashValue; + _son[ptr0] = _son[ptr1] = K_EMPTY_HASH_VALUE; break; } UInt32 delta = _pos - curMatch; @@ -303,7 +303,7 @@ namespace SharpCompress.Compressors.LZMA.LZ else { lenLimit = _streamPos - _pos; - if (lenLimit < kMinMatchCheck) + if (lenLimit < _kMinMatchCheck) { MovePos(); continue; @@ -315,36 +315,36 @@ namespace SharpCompress.Compressors.LZMA.LZ UInt32 hashValue; - if (HASH_ARRAY) + if (_hashArray) { - UInt32 temp = CRC.Table[_bufferBase[cur]] ^ _bufferBase[cur + 1]; - UInt32 hash2Value = temp & (kHash2Size - 1); + UInt32 temp = Crc.TABLE[_bufferBase[cur]] ^ _bufferBase[cur + 1]; + UInt32 hash2Value = temp & (K_HASH2_SIZE - 1); _hash[hash2Value] = _pos; temp ^= ((UInt32)(_bufferBase[cur + 2]) << 8); - UInt32 hash3Value = temp & (kHash3Size - 1); - _hash[kHash3Offset + hash3Value] = _pos; - hashValue = (temp ^ (CRC.Table[_bufferBase[cur + 3]] << 5)) & _hashMask; + UInt32 hash3Value = temp & (K_HASH3_SIZE - 1); + _hash[K_HASH3_OFFSET + hash3Value] = _pos; + hashValue = (temp ^ (Crc.TABLE[_bufferBase[cur + 3]] << 5)) & _hashMask; } else { hashValue = _bufferBase[cur] ^ ((UInt32)(_bufferBase[cur + 1]) << 8); } - UInt32 curMatch = _hash[kFixHashSize + hashValue]; - _hash[kFixHashSize + hashValue] = _pos; + UInt32 curMatch = _hash[_kFixHashSize + hashValue]; + _hash[_kFixHashSize + hashValue] = _pos; UInt32 ptr0 = (_cyclicBufferPos << 1) + 1; UInt32 ptr1 = (_cyclicBufferPos << 1); UInt32 len0, len1; - len0 = len1 = kNumHashDirectBytes; + len0 = len1 = _kNumHashDirectBytes; UInt32 count = _cutValue; while (true) { if (curMatch <= matchMinPos || count-- == 0) { - _son[ptr0] = _son[ptr1] = kEmptyHashValue; + _son[ptr0] = _son[ptr1] = K_EMPTY_HASH_VALUE; break; } @@ -398,7 +398,7 @@ namespace SharpCompress.Compressors.LZMA.LZ UInt32 value = items[i]; if (value <= subValue) { - value = kEmptyHashValue; + value = K_EMPTY_HASH_VALUE; } else { diff --git a/src/SharpCompress/Compressors/LZMA/LZ/LzOutWindow.cs b/src/SharpCompress/Compressors/LZMA/LZ/LzOutWindow.cs index 1b230873..6736dc1a 100644 --- a/src/SharpCompress/Compressors/LZMA/LZ/LzOutWindow.cs +++ b/src/SharpCompress/Compressors/LZMA/LZ/LzOutWindow.cs @@ -13,8 +13,8 @@ namespace SharpCompress.Compressors.LZMA.LZ private int _pendingDist; private Stream _stream; - public long Total; - public long Limit; + public long _total; + public long _limit; public void Create(int windowSize) { @@ -30,8 +30,8 @@ namespace SharpCompress.Compressors.LZMA.LZ _pos = 0; _streamPos = 0; _pendingLen = 0; - Total = 0; - Limit = 0; + _total = 0; + _limit = 0; } public void Reset() @@ -50,8 +50,8 @@ namespace SharpCompress.Compressors.LZMA.LZ long len = stream.Length; int size = (len < _windowSize) ? (int)len : _windowSize; stream.Position = len - size; - Total = 0; - Limit = size; + _total = 0; + _limit = size; _pos = _windowSize - size; CopyStream(stream, size); if (_pos == _windowSize) @@ -94,14 +94,14 @@ namespace SharpCompress.Compressors.LZMA.LZ { pos += _windowSize; } - for (; size > 0 && _pos < _windowSize && Total < Limit; size--) + for (; size > 0 && _pos < _windowSize && _total < _limit; size--) { if (pos >= _windowSize) { pos = 0; } _buffer[_pos++] = _buffer[pos++]; - Total++; + _total++; if (_pos >= _windowSize) { Flush(); @@ -114,7 +114,7 @@ namespace SharpCompress.Compressors.LZMA.LZ public void PutByte(byte b) { _buffer[_pos++] = b; - Total++; + _total++; if (_pos >= _windowSize) { Flush(); @@ -134,12 +134,12 @@ namespace SharpCompress.Compressors.LZMA.LZ public int CopyStream(Stream stream, int len) { int size = len; - while (size > 0 && _pos < _windowSize && Total < Limit) + while (size > 0 && _pos < _windowSize && _total < _limit) { int curSize = _windowSize - _pos; - if (curSize > Limit - Total) + if (curSize > _limit - _total) { - curSize = (int)(Limit - Total); + curSize = (int)(_limit - _total); } if (curSize > size) { @@ -152,7 +152,7 @@ namespace SharpCompress.Compressors.LZMA.LZ } size -= numReadBytes; _pos += numReadBytes; - Total += numReadBytes; + _total += numReadBytes; if (_pos >= _windowSize) { Flush(); @@ -163,10 +163,10 @@ namespace SharpCompress.Compressors.LZMA.LZ public void SetLimit(long size) { - Limit = Total + size; + _limit = _total + size; } - public bool HasSpace => _pos < _windowSize && Total < Limit; + public bool HasSpace => _pos < _windowSize && _total < _limit; public bool HasPending => _pendingLen > 0; diff --git a/src/SharpCompress/Compressors/LZMA/LZipStream.cs b/src/SharpCompress/Compressors/LZMA/LZipStream.cs index fbb78f5e..588386f5 100644 --- a/src/SharpCompress/Compressors/LZMA/LZipStream.cs +++ b/src/SharpCompress/Compressors/LZMA/LZipStream.cs @@ -1,4 +1,4 @@ -using System; +using System; using System.IO; using SharpCompress.Converters; using SharpCompress.Crypto; @@ -16,18 +16,16 @@ namespace SharpCompress.Compressors.LZMA /// public class LZipStream : Stream { - private readonly Stream stream; - private readonly CountingWritableSubStream rawStream; - private bool disposed; - private readonly bool leaveOpen; - private bool finished; + private readonly Stream _stream; + private readonly CountingWritableSubStream _countingWritableSubStream; + private bool _disposed; + private bool _finished; - private long writeCount; + private long _writeCount; - public LZipStream(Stream stream, CompressionMode mode, bool leaveOpen = false) + public LZipStream(Stream stream, CompressionMode mode) { Mode = mode; - this.leaveOpen = leaveOpen; if (mode == CompressionMode.Decompress) { @@ -37,7 +35,7 @@ namespace SharpCompress.Compressors.LZMA throw new IOException("Not an LZip stream"); } byte[] properties = GetProperties(dSize); - this.stream = new LzmaStream(properties, stream); + _stream = new LzmaStream(properties, stream); } else { @@ -45,33 +43,33 @@ namespace SharpCompress.Compressors.LZMA int dSize = 104 * 1024; WriteHeaderSize(stream); - rawStream = new CountingWritableSubStream(stream); - this.stream = new Crc32Stream(new LzmaStream(new LzmaEncoderProperties(true, dSize), false, rawStream)); + _countingWritableSubStream = new CountingWritableSubStream(stream); + _stream = new Crc32Stream(new LzmaStream(new LzmaEncoderProperties(true, dSize), false, _countingWritableSubStream)); } } public void Finish() { - if (!finished) + if (!_finished) { if (Mode == CompressionMode.Compress) { - var crc32Stream = (Crc32Stream)stream; + var crc32Stream = (Crc32Stream)_stream; crc32Stream.WrappedStream.Dispose(); crc32Stream.Dispose(); - var compressedCount = rawStream.Count; + var compressedCount = _countingWritableSubStream.Count; var bytes = DataConverter.LittleEndian.GetBytes(crc32Stream.Crc); - rawStream.Write(bytes, 0, bytes.Length); + _countingWritableSubStream.Write(bytes, 0, bytes.Length); - bytes = DataConverter.LittleEndian.GetBytes(writeCount); - rawStream.Write(bytes, 0, bytes.Length); + bytes = DataConverter.LittleEndian.GetBytes(_writeCount); + _countingWritableSubStream.Write(bytes, 0, bytes.Length); //total with headers bytes = DataConverter.LittleEndian.GetBytes(compressedCount + 6 + 20); - rawStream.Write(bytes, 0, bytes.Length); + _countingWritableSubStream.Write(bytes, 0, bytes.Length); } - finished = true; + _finished = true; } } @@ -79,18 +77,15 @@ namespace SharpCompress.Compressors.LZMA protected override void Dispose(bool disposing) { - if (disposed) + if (_disposed) { return; } - disposed = true; + _disposed = true; if (disposing) { Finish(); - if (!leaveOpen) - { - rawStream.Dispose(); - } + _stream.Dispose(); } } @@ -104,7 +99,7 @@ namespace SharpCompress.Compressors.LZMA public override void Flush() { - stream.Flush(); + _stream.Flush(); } // TODO: Both Length and Position are sometimes feasible, but would require @@ -113,7 +108,9 @@ namespace SharpCompress.Compressors.LZMA public override long Position { get => throw new NotImplementedException(); set => throw new NotImplementedException(); } - public override int Read(byte[] buffer, int offset, int count) => stream.Read(buffer, offset, count); + public override int Read(byte[] buffer, int offset, int count) => _stream.Read(buffer, offset, count); + + public override int ReadByte() => _stream.ReadByte(); public override long Seek(long offset, SeekOrigin origin) => throw new NotSupportedException(); @@ -121,8 +118,14 @@ namespace SharpCompress.Compressors.LZMA public override void Write(byte[] buffer, int offset, int count) { - stream.Write(buffer, offset, count); - writeCount += count; + _stream.Write(buffer, offset, count); + _writeCount += count; + } + + public override void WriteByte(byte value) + { + _stream.WriteByte(value); + ++_writeCount; } #endregion diff --git a/src/SharpCompress/Compressors/LZMA/Log.cs b/src/SharpCompress/Compressors/LZMA/Log.cs index ba461215..f3b4eab5 100644 --- a/src/SharpCompress/Compressors/LZMA/Log.cs +++ b/src/SharpCompress/Compressors/LZMA/Log.cs @@ -6,36 +6,36 @@ namespace SharpCompress.Compressors.LZMA { internal static class Log { - private static readonly Stack _indent = new Stack(); - private static bool _needsIndent = true; + private static readonly Stack INDENT = new Stack(); + private static bool NEEDS_INDENT = true; static Log() { - _indent.Push(""); + INDENT.Push(""); } public static void PushIndent(string indent = " ") { - _indent.Push(_indent.Peek() + indent); + INDENT.Push(INDENT.Peek() + indent); } public static void PopIndent() { - if (_indent.Count == 1) + if (INDENT.Count == 1) { throw new InvalidOperationException(); } - _indent.Pop(); + INDENT.Pop(); } private static void EnsureIndent() { - if (_needsIndent) + if (NEEDS_INDENT) { - _needsIndent = false; + NEEDS_INDENT = false; #if !NO_FILE - System.Diagnostics.Debug.Write(_indent.Peek()); + Debug.Write(INDENT.Peek()); #endif } } @@ -44,7 +44,7 @@ namespace SharpCompress.Compressors.LZMA { EnsureIndent(); #if !NO_FILE - System.Diagnostics.Debug.Write(value); + Debug.Write(value); #endif } @@ -52,7 +52,7 @@ namespace SharpCompress.Compressors.LZMA { EnsureIndent(); #if !NO_FILE - System.Diagnostics.Debug.Write(text); + Debug.Write(text); #endif } @@ -60,35 +60,35 @@ namespace SharpCompress.Compressors.LZMA { EnsureIndent(); #if !NO_FILE - System.Diagnostics.Debug.Write(string.Format(format, args)); + Debug.Write(string.Format(format, args)); #endif } public static void WriteLine() { Debug.WriteLine(""); - _needsIndent = true; + NEEDS_INDENT = true; } public static void WriteLine(object value) { EnsureIndent(); Debug.WriteLine(value); - _needsIndent = true; + NEEDS_INDENT = true; } public static void WriteLine(string text) { EnsureIndent(); Debug.WriteLine(text); - _needsIndent = true; + NEEDS_INDENT = true; } public static void WriteLine(string format, params object[] args) { EnsureIndent(); Debug.WriteLine(string.Format(format, args)); - _needsIndent = true; + NEEDS_INDENT = true; } } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/LZMA/LzmaBase.cs b/src/SharpCompress/Compressors/LZMA/LzmaBase.cs index e272607f..6e2b8a5d 100644 --- a/src/SharpCompress/Compressors/LZMA/LzmaBase.cs +++ b/src/SharpCompress/Compressors/LZMA/LzmaBase.cs @@ -2,8 +2,8 @@ namespace SharpCompress.Compressors.LZMA { internal abstract class Base { - public const uint kNumRepDistances = 4; - public const uint kNumStates = 12; + public const uint K_NUM_REP_DISTANCES = 4; + public const uint K_NUM_STATES = 12; // static byte []kLiteralNextStates = {0, 0, 0, 0, 1, 2, 3, 4, 5, 6, 4, 5}; // static byte []kMatchNextStates = {7, 7, 7, 7, 7, 7, 7, 10, 10, 10, 10, 10}; @@ -12,98 +12,98 @@ namespace SharpCompress.Compressors.LZMA public struct State { - public uint Index; + public uint _index; public void Init() { - Index = 0; + _index = 0; } public void UpdateChar() { - if (Index < 4) + if (_index < 4) { - Index = 0; + _index = 0; } - else if (Index < 10) + else if (_index < 10) { - Index -= 3; + _index -= 3; } else { - Index -= 6; + _index -= 6; } } public void UpdateMatch() { - Index = (uint)(Index < 7 ? 7 : 10); + _index = (uint)(_index < 7 ? 7 : 10); } public void UpdateRep() { - Index = (uint)(Index < 7 ? 8 : 11); + _index = (uint)(_index < 7 ? 8 : 11); } public void UpdateShortRep() { - Index = (uint)(Index < 7 ? 9 : 11); + _index = (uint)(_index < 7 ? 9 : 11); } public bool IsCharState() { - return Index < 7; + return _index < 7; } } - public const int kNumPosSlotBits = 6; - public const int kDicLogSizeMin = 0; + public const int K_NUM_POS_SLOT_BITS = 6; + public const int K_DIC_LOG_SIZE_MIN = 0; // public const int kDicLogSizeMax = 30; // public const uint kDistTableSizeMax = kDicLogSizeMax * 2; - public const int kNumLenToPosStatesBits = 2; // it's for speed optimization - public const uint kNumLenToPosStates = 1 << kNumLenToPosStatesBits; + public const int K_NUM_LEN_TO_POS_STATES_BITS = 2; // it's for speed optimization + public const uint K_NUM_LEN_TO_POS_STATES = 1 << K_NUM_LEN_TO_POS_STATES_BITS; - public const uint kMatchMinLen = 2; + public const uint K_MATCH_MIN_LEN = 2; public static uint GetLenToPosState(uint len) { - len -= kMatchMinLen; - if (len < kNumLenToPosStates) + len -= K_MATCH_MIN_LEN; + if (len < K_NUM_LEN_TO_POS_STATES) { return len; } - return kNumLenToPosStates - 1; + return K_NUM_LEN_TO_POS_STATES - 1; } - public const int kNumAlignBits = 4; - public const uint kAlignTableSize = 1 << kNumAlignBits; - public const uint kAlignMask = (kAlignTableSize - 1); + public const int K_NUM_ALIGN_BITS = 4; + public const uint K_ALIGN_TABLE_SIZE = 1 << K_NUM_ALIGN_BITS; + public const uint K_ALIGN_MASK = (K_ALIGN_TABLE_SIZE - 1); - public const uint kStartPosModelIndex = 4; - public const uint kEndPosModelIndex = 14; - public const uint kNumPosModels = kEndPosModelIndex - kStartPosModelIndex; + public const uint K_START_POS_MODEL_INDEX = 4; + public const uint K_END_POS_MODEL_INDEX = 14; + public const uint K_NUM_POS_MODELS = K_END_POS_MODEL_INDEX - K_START_POS_MODEL_INDEX; - public const uint kNumFullDistances = 1 << ((int)kEndPosModelIndex / 2); + public const uint K_NUM_FULL_DISTANCES = 1 << ((int)K_END_POS_MODEL_INDEX / 2); - public const uint kNumLitPosStatesBitsEncodingMax = 4; - public const uint kNumLitContextBitsMax = 8; + public const uint K_NUM_LIT_POS_STATES_BITS_ENCODING_MAX = 4; + public const uint K_NUM_LIT_CONTEXT_BITS_MAX = 8; - public const int kNumPosStatesBitsMax = 4; - public const uint kNumPosStatesMax = (1 << kNumPosStatesBitsMax); - public const int kNumPosStatesBitsEncodingMax = 4; - public const uint kNumPosStatesEncodingMax = (1 << kNumPosStatesBitsEncodingMax); + public const int K_NUM_POS_STATES_BITS_MAX = 4; + public const uint K_NUM_POS_STATES_MAX = (1 << K_NUM_POS_STATES_BITS_MAX); + public const int K_NUM_POS_STATES_BITS_ENCODING_MAX = 4; + public const uint K_NUM_POS_STATES_ENCODING_MAX = (1 << K_NUM_POS_STATES_BITS_ENCODING_MAX); - public const int kNumLowLenBits = 3; - public const int kNumMidLenBits = 3; - public const int kNumHighLenBits = 8; - public const uint kNumLowLenSymbols = 1 << kNumLowLenBits; - public const uint kNumMidLenSymbols = 1 << kNumMidLenBits; + public const int K_NUM_LOW_LEN_BITS = 3; + public const int K_NUM_MID_LEN_BITS = 3; + public const int K_NUM_HIGH_LEN_BITS = 8; + public const uint K_NUM_LOW_LEN_SYMBOLS = 1 << K_NUM_LOW_LEN_BITS; + public const uint K_NUM_MID_LEN_SYMBOLS = 1 << K_NUM_MID_LEN_BITS; - public const uint kNumLenSymbols = kNumLowLenSymbols + kNumMidLenSymbols + - (1 << kNumHighLenBits); + public const uint K_NUM_LEN_SYMBOLS = K_NUM_LOW_LEN_SYMBOLS + K_NUM_MID_LEN_SYMBOLS + + (1 << K_NUM_HIGH_LEN_BITS); - public const uint kMatchMaxLen = kMatchMinLen + kNumLenSymbols - 1; + public const uint K_MATCH_MAX_LEN = K_MATCH_MIN_LEN + K_NUM_LEN_SYMBOLS - 1; } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/LZMA/LzmaDecoder.cs b/src/SharpCompress/Compressors/LZMA/LzmaDecoder.cs index 063b9e3e..896e92cf 100644 --- a/src/SharpCompress/Compressors/LZMA/LzmaDecoder.cs +++ b/src/SharpCompress/Compressors/LZMA/LzmaDecoder.cs @@ -9,50 +9,50 @@ namespace SharpCompress.Compressors.LZMA { private class LenDecoder { - private BitDecoder m_Choice = new BitDecoder(); - private BitDecoder m_Choice2 = new BitDecoder(); - private readonly BitTreeDecoder[] m_LowCoder = new BitTreeDecoder[Base.kNumPosStatesMax]; - private readonly BitTreeDecoder[] m_MidCoder = new BitTreeDecoder[Base.kNumPosStatesMax]; - private BitTreeDecoder m_HighCoder = new BitTreeDecoder(Base.kNumHighLenBits); - private uint m_NumPosStates; + private BitDecoder _choice = new BitDecoder(); + private BitDecoder _choice2 = new BitDecoder(); + private readonly BitTreeDecoder[] _lowCoder = new BitTreeDecoder[Base.K_NUM_POS_STATES_MAX]; + private readonly BitTreeDecoder[] _midCoder = new BitTreeDecoder[Base.K_NUM_POS_STATES_MAX]; + private BitTreeDecoder _highCoder = new BitTreeDecoder(Base.K_NUM_HIGH_LEN_BITS); + private uint _numPosStates; public void Create(uint numPosStates) { - for (uint posState = m_NumPosStates; posState < numPosStates; posState++) + for (uint posState = _numPosStates; posState < numPosStates; posState++) { - m_LowCoder[posState] = new BitTreeDecoder(Base.kNumLowLenBits); - m_MidCoder[posState] = new BitTreeDecoder(Base.kNumMidLenBits); + _lowCoder[posState] = new BitTreeDecoder(Base.K_NUM_LOW_LEN_BITS); + _midCoder[posState] = new BitTreeDecoder(Base.K_NUM_MID_LEN_BITS); } - m_NumPosStates = numPosStates; + _numPosStates = numPosStates; } public void Init() { - m_Choice.Init(); - for (uint posState = 0; posState < m_NumPosStates; posState++) + _choice.Init(); + for (uint posState = 0; posState < _numPosStates; posState++) { - m_LowCoder[posState].Init(); - m_MidCoder[posState].Init(); + _lowCoder[posState].Init(); + _midCoder[posState].Init(); } - m_Choice2.Init(); - m_HighCoder.Init(); + _choice2.Init(); + _highCoder.Init(); } public uint Decode(RangeCoder.Decoder rangeDecoder, uint posState) { - if (m_Choice.Decode(rangeDecoder) == 0) + if (_choice.Decode(rangeDecoder) == 0) { - return m_LowCoder[posState].Decode(rangeDecoder); + return _lowCoder[posState].Decode(rangeDecoder); } - uint symbol = Base.kNumLowLenSymbols; - if (m_Choice2.Decode(rangeDecoder) == 0) + uint symbol = Base.K_NUM_LOW_LEN_SYMBOLS; + if (_choice2.Decode(rangeDecoder) == 0) { - symbol += m_MidCoder[posState].Decode(rangeDecoder); + symbol += _midCoder[posState].Decode(rangeDecoder); } else { - symbol += Base.kNumMidLenSymbols; - symbol += m_HighCoder.Decode(rangeDecoder); + symbol += Base.K_NUM_MID_LEN_SYMBOLS; + symbol += _highCoder.Decode(rangeDecoder); } return symbol; } @@ -62,18 +62,18 @@ namespace SharpCompress.Compressors.LZMA { private struct Decoder2 { - private BitDecoder[] m_Decoders; + private BitDecoder[] _decoders; public void Create() { - m_Decoders = new BitDecoder[0x300]; + _decoders = new BitDecoder[0x300]; } public void Init() { for (int i = 0; i < 0x300; i++) { - m_Decoders[i].Init(); + _decoders[i].Init(); } } @@ -82,7 +82,7 @@ namespace SharpCompress.Compressors.LZMA uint symbol = 1; do { - symbol = (symbol << 1) | m_Decoders[symbol].Decode(rangeDecoder); + symbol = (symbol << 1) | _decoders[symbol].Decode(rangeDecoder); } while (symbol < 0x100); return (byte)symbol; @@ -95,13 +95,13 @@ namespace SharpCompress.Compressors.LZMA { uint matchBit = (uint)(matchByte >> 7) & 1; matchByte <<= 1; - uint bit = m_Decoders[((1 + matchBit) << 8) + symbol].Decode(rangeDecoder); + uint bit = _decoders[((1 + matchBit) << 8) + symbol].Decode(rangeDecoder); symbol = (symbol << 1) | bit; if (matchBit != bit) { while (symbol < 0x100) { - symbol = (symbol << 1) | m_Decoders[symbol].Decode(rangeDecoder); + symbol = (symbol << 1) | _decoders[symbol].Decode(rangeDecoder); } break; } @@ -111,98 +111,98 @@ namespace SharpCompress.Compressors.LZMA } } - private Decoder2[] m_Coders; - private int m_NumPrevBits; - private int m_NumPosBits; - private uint m_PosMask; + private Decoder2[] _coders; + private int _numPrevBits; + private int _numPosBits; + private uint _posMask; public void Create(int numPosBits, int numPrevBits) { - if (m_Coders != null && m_NumPrevBits == numPrevBits && - m_NumPosBits == numPosBits) + if (_coders != null && _numPrevBits == numPrevBits && + _numPosBits == numPosBits) { return; } - m_NumPosBits = numPosBits; - m_PosMask = ((uint)1 << numPosBits) - 1; - m_NumPrevBits = numPrevBits; - uint numStates = (uint)1 << (m_NumPrevBits + m_NumPosBits); - m_Coders = new Decoder2[numStates]; + _numPosBits = numPosBits; + _posMask = ((uint)1 << numPosBits) - 1; + _numPrevBits = numPrevBits; + uint numStates = (uint)1 << (_numPrevBits + _numPosBits); + _coders = new Decoder2[numStates]; for (uint i = 0; i < numStates; i++) { - m_Coders[i].Create(); + _coders[i].Create(); } } public void Init() { - uint numStates = (uint)1 << (m_NumPrevBits + m_NumPosBits); + uint numStates = (uint)1 << (_numPrevBits + _numPosBits); for (uint i = 0; i < numStates; i++) { - m_Coders[i].Init(); + _coders[i].Init(); } } private uint GetState(uint pos, byte prevByte) { - return ((pos & m_PosMask) << m_NumPrevBits) + (uint)(prevByte >> (8 - m_NumPrevBits)); + return ((pos & _posMask) << _numPrevBits) + (uint)(prevByte >> (8 - _numPrevBits)); } public byte DecodeNormal(RangeCoder.Decoder rangeDecoder, uint pos, byte prevByte) { - return m_Coders[GetState(pos, prevByte)].DecodeNormal(rangeDecoder); + return _coders[GetState(pos, prevByte)].DecodeNormal(rangeDecoder); } public byte DecodeWithMatchByte(RangeCoder.Decoder rangeDecoder, uint pos, byte prevByte, byte matchByte) { - return m_Coders[GetState(pos, prevByte)].DecodeWithMatchByte(rangeDecoder, matchByte); + return _coders[GetState(pos, prevByte)].DecodeWithMatchByte(rangeDecoder, matchByte); } } - private OutWindow m_OutWindow; + private OutWindow _outWindow; - private readonly BitDecoder[] m_IsMatchDecoders = new BitDecoder[Base.kNumStates << Base.kNumPosStatesBitsMax]; - private readonly BitDecoder[] m_IsRepDecoders = new BitDecoder[Base.kNumStates]; - private readonly BitDecoder[] m_IsRepG0Decoders = new BitDecoder[Base.kNumStates]; - private readonly BitDecoder[] m_IsRepG1Decoders = new BitDecoder[Base.kNumStates]; - private readonly BitDecoder[] m_IsRepG2Decoders = new BitDecoder[Base.kNumStates]; - private readonly BitDecoder[] m_IsRep0LongDecoders = new BitDecoder[Base.kNumStates << Base.kNumPosStatesBitsMax]; + private readonly BitDecoder[] _isMatchDecoders = new BitDecoder[Base.K_NUM_STATES << Base.K_NUM_POS_STATES_BITS_MAX]; + private readonly BitDecoder[] _isRepDecoders = new BitDecoder[Base.K_NUM_STATES]; + private readonly BitDecoder[] _isRepG0Decoders = new BitDecoder[Base.K_NUM_STATES]; + private readonly BitDecoder[] _isRepG1Decoders = new BitDecoder[Base.K_NUM_STATES]; + private readonly BitDecoder[] _isRepG2Decoders = new BitDecoder[Base.K_NUM_STATES]; + private readonly BitDecoder[] _isRep0LongDecoders = new BitDecoder[Base.K_NUM_STATES << Base.K_NUM_POS_STATES_BITS_MAX]; - private readonly BitTreeDecoder[] m_PosSlotDecoder = new BitTreeDecoder[Base.kNumLenToPosStates]; - private readonly BitDecoder[] m_PosDecoders = new BitDecoder[Base.kNumFullDistances - Base.kEndPosModelIndex]; + private readonly BitTreeDecoder[] _posSlotDecoder = new BitTreeDecoder[Base.K_NUM_LEN_TO_POS_STATES]; + private readonly BitDecoder[] _posDecoders = new BitDecoder[Base.K_NUM_FULL_DISTANCES - Base.K_END_POS_MODEL_INDEX]; - private BitTreeDecoder m_PosAlignDecoder = new BitTreeDecoder(Base.kNumAlignBits); + private BitTreeDecoder _posAlignDecoder = new BitTreeDecoder(Base.K_NUM_ALIGN_BITS); - private readonly LenDecoder m_LenDecoder = new LenDecoder(); - private readonly LenDecoder m_RepLenDecoder = new LenDecoder(); + private readonly LenDecoder _lenDecoder = new LenDecoder(); + private readonly LenDecoder _repLenDecoder = new LenDecoder(); - private readonly LiteralDecoder m_LiteralDecoder = new LiteralDecoder(); + private readonly LiteralDecoder _literalDecoder = new LiteralDecoder(); - private int m_DictionarySize; + private int _dictionarySize; - private uint m_PosStateMask; + private uint _posStateMask; - private Base.State state = new Base.State(); - private uint rep0, rep1, rep2, rep3; + private Base.State _state = new Base.State(); + private uint _rep0, _rep1, _rep2, _rep3; public Decoder() { - m_DictionarySize = -1; - for (int i = 0; i < Base.kNumLenToPosStates; i++) + _dictionarySize = -1; + for (int i = 0; i < Base.K_NUM_LEN_TO_POS_STATES; i++) { - m_PosSlotDecoder[i] = new BitTreeDecoder(Base.kNumPosSlotBits); + _posSlotDecoder[i] = new BitTreeDecoder(Base.K_NUM_POS_SLOT_BITS); } } private void CreateDictionary() { - if (m_DictionarySize < 0) + if (_dictionarySize < 0) { throw new InvalidParamException(); } - m_OutWindow = new OutWindow(); - int blockSize = Math.Max(m_DictionarySize, (1 << 12)); - m_OutWindow.Create(blockSize); + _outWindow = new OutWindow(); + int blockSize = Math.Max(_dictionarySize, (1 << 12)); + _outWindow.Create(blockSize); } private void SetLiteralProperties(int lp, int lc) @@ -215,95 +215,95 @@ namespace SharpCompress.Compressors.LZMA { throw new InvalidParamException(); } - m_LiteralDecoder.Create(lp, lc); + _literalDecoder.Create(lp, lc); } private void SetPosBitsProperties(int pb) { - if (pb > Base.kNumPosStatesBitsMax) + if (pb > Base.K_NUM_POS_STATES_BITS_MAX) { throw new InvalidParamException(); } uint numPosStates = (uint)1 << pb; - m_LenDecoder.Create(numPosStates); - m_RepLenDecoder.Create(numPosStates); - m_PosStateMask = numPosStates - 1; + _lenDecoder.Create(numPosStates); + _repLenDecoder.Create(numPosStates); + _posStateMask = numPosStates - 1; } private void Init() { uint i; - for (i = 0; i < Base.kNumStates; i++) + for (i = 0; i < Base.K_NUM_STATES; i++) { - for (uint j = 0; j <= m_PosStateMask; j++) + for (uint j = 0; j <= _posStateMask; j++) { - uint index = (i << Base.kNumPosStatesBitsMax) + j; - m_IsMatchDecoders[index].Init(); - m_IsRep0LongDecoders[index].Init(); + uint index = (i << Base.K_NUM_POS_STATES_BITS_MAX) + j; + _isMatchDecoders[index].Init(); + _isRep0LongDecoders[index].Init(); } - m_IsRepDecoders[i].Init(); - m_IsRepG0Decoders[i].Init(); - m_IsRepG1Decoders[i].Init(); - m_IsRepG2Decoders[i].Init(); + _isRepDecoders[i].Init(); + _isRepG0Decoders[i].Init(); + _isRepG1Decoders[i].Init(); + _isRepG2Decoders[i].Init(); } - m_LiteralDecoder.Init(); - for (i = 0; i < Base.kNumLenToPosStates; i++) + _literalDecoder.Init(); + for (i = 0; i < Base.K_NUM_LEN_TO_POS_STATES; i++) { - m_PosSlotDecoder[i].Init(); + _posSlotDecoder[i].Init(); } - // m_PosSpecDecoder.Init(); - for (i = 0; i < Base.kNumFullDistances - Base.kEndPosModelIndex; i++) + // _PosSpecDecoder.Init(); + for (i = 0; i < Base.K_NUM_FULL_DISTANCES - Base.K_END_POS_MODEL_INDEX; i++) { - m_PosDecoders[i].Init(); + _posDecoders[i].Init(); } - m_LenDecoder.Init(); - m_RepLenDecoder.Init(); - m_PosAlignDecoder.Init(); + _lenDecoder.Init(); + _repLenDecoder.Init(); + _posAlignDecoder.Init(); - state.Init(); - rep0 = 0; - rep1 = 0; - rep2 = 0; - rep3 = 0; + _state.Init(); + _rep0 = 0; + _rep1 = 0; + _rep2 = 0; + _rep3 = 0; } public void Code(Stream inStream, Stream outStream, Int64 inSize, Int64 outSize, ICodeProgress progress) { - if (m_OutWindow == null) + if (_outWindow == null) { CreateDictionary(); } - m_OutWindow.Init(outStream); + _outWindow.Init(outStream); if (outSize > 0) { - m_OutWindow.SetLimit(outSize); + _outWindow.SetLimit(outSize); } else { - m_OutWindow.SetLimit(Int64.MaxValue - m_OutWindow.Total); + _outWindow.SetLimit(Int64.MaxValue - _outWindow._total); } RangeCoder.Decoder rangeDecoder = new RangeCoder.Decoder(); rangeDecoder.Init(inStream); - Code(m_DictionarySize, m_OutWindow, rangeDecoder); + Code(_dictionarySize, _outWindow, rangeDecoder); - m_OutWindow.ReleaseStream(); + _outWindow.ReleaseStream(); rangeDecoder.ReleaseStream(); - if (!rangeDecoder.IsFinished || (inSize > 0 && rangeDecoder.Total != inSize)) + if (!rangeDecoder.IsFinished || (inSize > 0 && rangeDecoder._total != inSize)) { throw new DataErrorException(); } - if (m_OutWindow.HasPending) + if (_outWindow.HasPending) { throw new DataErrorException(); } - m_OutWindow = null; + _outWindow = null; } internal bool Code(int dictionarySize, OutWindow outWindow, RangeCoder.Decoder rangeDecoder) @@ -314,104 +314,104 @@ namespace SharpCompress.Compressors.LZMA while (outWindow.HasSpace) { - uint posState = (uint)outWindow.Total & m_PosStateMask; - if (m_IsMatchDecoders[(state.Index << Base.kNumPosStatesBitsMax) + posState].Decode(rangeDecoder) == 0) + uint posState = (uint)outWindow._total & _posStateMask; + if (_isMatchDecoders[(_state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState].Decode(rangeDecoder) == 0) { byte b; byte prevByte = outWindow.GetByte(0); - if (!state.IsCharState()) + if (!_state.IsCharState()) { - b = m_LiteralDecoder.DecodeWithMatchByte(rangeDecoder, - (uint)outWindow.Total, prevByte, - outWindow.GetByte((int)rep0)); + b = _literalDecoder.DecodeWithMatchByte(rangeDecoder, + (uint)outWindow._total, prevByte, + outWindow.GetByte((int)_rep0)); } else { - b = m_LiteralDecoder.DecodeNormal(rangeDecoder, (uint)outWindow.Total, prevByte); + b = _literalDecoder.DecodeNormal(rangeDecoder, (uint)outWindow._total, prevByte); } outWindow.PutByte(b); - state.UpdateChar(); + _state.UpdateChar(); } else { uint len; - if (m_IsRepDecoders[state.Index].Decode(rangeDecoder) == 1) + if (_isRepDecoders[_state._index].Decode(rangeDecoder) == 1) { - if (m_IsRepG0Decoders[state.Index].Decode(rangeDecoder) == 0) + if (_isRepG0Decoders[_state._index].Decode(rangeDecoder) == 0) { if ( - m_IsRep0LongDecoders[(state.Index << Base.kNumPosStatesBitsMax) + posState].Decode( + _isRep0LongDecoders[(_state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState].Decode( rangeDecoder) == 0) { - state.UpdateShortRep(); - outWindow.PutByte(outWindow.GetByte((int)rep0)); + _state.UpdateShortRep(); + outWindow.PutByte(outWindow.GetByte((int)_rep0)); continue; } } else { UInt32 distance; - if (m_IsRepG1Decoders[state.Index].Decode(rangeDecoder) == 0) + if (_isRepG1Decoders[_state._index].Decode(rangeDecoder) == 0) { - distance = rep1; + distance = _rep1; } else { - if (m_IsRepG2Decoders[state.Index].Decode(rangeDecoder) == 0) + if (_isRepG2Decoders[_state._index].Decode(rangeDecoder) == 0) { - distance = rep2; + distance = _rep2; } else { - distance = rep3; - rep3 = rep2; + distance = _rep3; + _rep3 = _rep2; } - rep2 = rep1; + _rep2 = _rep1; } - rep1 = rep0; - rep0 = distance; + _rep1 = _rep0; + _rep0 = distance; } - len = m_RepLenDecoder.Decode(rangeDecoder, posState) + Base.kMatchMinLen; - state.UpdateRep(); + len = _repLenDecoder.Decode(rangeDecoder, posState) + Base.K_MATCH_MIN_LEN; + _state.UpdateRep(); } else { - rep3 = rep2; - rep2 = rep1; - rep1 = rep0; - len = Base.kMatchMinLen + m_LenDecoder.Decode(rangeDecoder, posState); - state.UpdateMatch(); - uint posSlot = m_PosSlotDecoder[Base.GetLenToPosState(len)].Decode(rangeDecoder); - if (posSlot >= Base.kStartPosModelIndex) + _rep3 = _rep2; + _rep2 = _rep1; + _rep1 = _rep0; + len = Base.K_MATCH_MIN_LEN + _lenDecoder.Decode(rangeDecoder, posState); + _state.UpdateMatch(); + uint posSlot = _posSlotDecoder[Base.GetLenToPosState(len)].Decode(rangeDecoder); + if (posSlot >= Base.K_START_POS_MODEL_INDEX) { int numDirectBits = (int)((posSlot >> 1) - 1); - rep0 = ((2 | (posSlot & 1)) << numDirectBits); - if (posSlot < Base.kEndPosModelIndex) + _rep0 = ((2 | (posSlot & 1)) << numDirectBits); + if (posSlot < Base.K_END_POS_MODEL_INDEX) { - rep0 += BitTreeDecoder.ReverseDecode(m_PosDecoders, - rep0 - posSlot - 1, rangeDecoder, numDirectBits); + _rep0 += BitTreeDecoder.ReverseDecode(_posDecoders, + _rep0 - posSlot - 1, rangeDecoder, numDirectBits); } else { - rep0 += (rangeDecoder.DecodeDirectBits( - numDirectBits - Base.kNumAlignBits) << Base.kNumAlignBits); - rep0 += m_PosAlignDecoder.ReverseDecode(rangeDecoder); + _rep0 += (rangeDecoder.DecodeDirectBits( + numDirectBits - Base.K_NUM_ALIGN_BITS) << Base.K_NUM_ALIGN_BITS); + _rep0 += _posAlignDecoder.ReverseDecode(rangeDecoder); } } else { - rep0 = posSlot; + _rep0 = posSlot; } } - if (rep0 >= outWindow.Total || rep0 >= dictionarySizeCheck) + if (_rep0 >= outWindow._total || _rep0 >= dictionarySizeCheck) { - if (rep0 == 0xFFFFFFFF) + if (_rep0 == 0xFFFFFFFF) { return true; } throw new DataErrorException(); } - outWindow.CopyBlock((int)rep0, (int)len); + outWindow.CopyBlock((int)_rep0, (int)len); } } return false; @@ -427,7 +427,7 @@ namespace SharpCompress.Compressors.LZMA int remainder = properties[0] / 9; int lp = remainder % 5; int pb = remainder / 5; - if (pb > Base.kNumPosStatesBitsMax) + if (pb > Base.K_NUM_POS_STATES_BITS_MAX) { throw new InvalidParamException(); } @@ -436,21 +436,21 @@ namespace SharpCompress.Compressors.LZMA Init(); if (properties.Length >= 5) { - m_DictionarySize = 0; + _dictionarySize = 0; for (int i = 0; i < 4; i++) { - m_DictionarySize += properties[1 + i] << (i * 8); + _dictionarySize += properties[1 + i] << (i * 8); } } } public void Train(Stream stream) { - if (m_OutWindow == null) + if (_outWindow == null) { CreateDictionary(); } - m_OutWindow.Train(stream); + _outWindow.Train(stream); } /* diff --git a/src/SharpCompress/Compressors/LZMA/LzmaEncoder.cs b/src/SharpCompress/Compressors/LZMA/LzmaEncoder.cs index 6f5b0145..be604d32 100644 --- a/src/SharpCompress/Compressors/LZMA/LzmaEncoder.cs +++ b/src/SharpCompress/Compressors/LZMA/LzmaEncoder.cs @@ -9,26 +9,26 @@ namespace SharpCompress.Compressors.LZMA { private enum EMatchFinderType { - BT2, - BT4 + Bt2, + Bt4 } - private const UInt32 kIfinityPrice = 0xFFFFFFF; + private const UInt32 K_IFINITY_PRICE = 0xFFFFFFF; - private static readonly Byte[] g_FastPos = new Byte[1 << 11]; + private static readonly Byte[] G_FAST_POS = new Byte[1 << 11]; static Encoder() { const Byte kFastSlots = 22; int c = 2; - g_FastPos[0] = 0; - g_FastPos[1] = 1; + G_FAST_POS[0] = 0; + G_FAST_POS[1] = 1; for (Byte slotFast = 2; slotFast < kFastSlots; slotFast++) { UInt32 k = ((UInt32)1 << ((slotFast >> 1) - 1)); for (UInt32 j = 0; j < k; j++, c++) { - g_FastPos[c] = slotFast; + G_FAST_POS[c] = slotFast; } } } @@ -37,61 +37,61 @@ namespace SharpCompress.Compressors.LZMA { if (pos < (1 << 11)) { - return g_FastPos[pos]; + return G_FAST_POS[pos]; } if (pos < (1 << 21)) { - return (UInt32)(g_FastPos[pos >> 10] + 20); + return (UInt32)(G_FAST_POS[pos >> 10] + 20); } - return (UInt32)(g_FastPos[pos >> 20] + 40); + return (UInt32)(G_FAST_POS[pos >> 20] + 40); } private static UInt32 GetPosSlot2(UInt32 pos) { if (pos < (1 << 17)) { - return (UInt32)(g_FastPos[pos >> 6] + 12); + return (UInt32)(G_FAST_POS[pos >> 6] + 12); } if (pos < (1 << 27)) { - return (UInt32)(g_FastPos[pos >> 16] + 32); + return (UInt32)(G_FAST_POS[pos >> 16] + 32); } - return (UInt32)(g_FastPos[pos >> 26] + 52); + return (UInt32)(G_FAST_POS[pos >> 26] + 52); } private Base.State _state = new Base.State(); private Byte _previousByte; - private readonly UInt32[] _repDistances = new UInt32[Base.kNumRepDistances]; + private readonly UInt32[] _repDistances = new UInt32[Base.K_NUM_REP_DISTANCES]; private void BaseInit() { _state.Init(); _previousByte = 0; - for (UInt32 i = 0; i < Base.kNumRepDistances; i++) + for (UInt32 i = 0; i < Base.K_NUM_REP_DISTANCES; i++) { _repDistances[i] = 0; } } - private const int kDefaultDictionaryLogSize = 22; - private const UInt32 kNumFastBytesDefault = 0x20; + private const int K_DEFAULT_DICTIONARY_LOG_SIZE = 22; + private const UInt32 K_NUM_FAST_BYTES_DEFAULT = 0x20; private class LiteralEncoder { public struct Encoder2 { - private BitEncoder[] m_Encoders; + private BitEncoder[] _encoders; public void Create() { - m_Encoders = new BitEncoder[0x300]; + _encoders = new BitEncoder[0x300]; } public void Init() { for (int i = 0; i < 0x300; i++) { - m_Encoders[i].Init(); + _encoders[i].Init(); } } @@ -101,7 +101,7 @@ namespace SharpCompress.Compressors.LZMA for (int i = 7; i >= 0; i--) { uint bit = (uint)((symbol >> i) & 1); - m_Encoders[context].Encode(rangeEncoder, bit); + _encoders[context].Encode(rangeEncoder, bit); context = (context << 1) | bit; } } @@ -120,7 +120,7 @@ namespace SharpCompress.Compressors.LZMA state += ((1 + matchBit) << 8); same = (matchBit == bit); } - m_Encoders[state].Encode(rangeEncoder, bit); + _encoders[state].Encode(rangeEncoder, bit); context = (context << 1) | bit; } } @@ -136,7 +136,7 @@ namespace SharpCompress.Compressors.LZMA { uint matchBit = (uint)(matchByte >> i) & 1; uint bit = (uint)(symbol >> i) & 1; - price += m_Encoders[((1 + matchBit) << 8) + context].GetPrice(bit); + price += _encoders[((1 + matchBit) << 8) + context].GetPrice(bit); context = (context << 1) | bit; if (matchBit != bit) { @@ -148,47 +148,47 @@ namespace SharpCompress.Compressors.LZMA for (; i >= 0; i--) { uint bit = (uint)(symbol >> i) & 1; - price += m_Encoders[context].GetPrice(bit); + price += _encoders[context].GetPrice(bit); context = (context << 1) | bit; } return price; } } - private Encoder2[] m_Coders; - private int m_NumPrevBits; - private int m_NumPosBits; - private uint m_PosMask; + private Encoder2[] _coders; + private int _numPrevBits; + private int _numPosBits; + private uint _posMask; public void Create(int numPosBits, int numPrevBits) { - if (m_Coders != null && m_NumPrevBits == numPrevBits && m_NumPosBits == numPosBits) + if (_coders != null && _numPrevBits == numPrevBits && _numPosBits == numPosBits) { return; } - m_NumPosBits = numPosBits; - m_PosMask = ((uint)1 << numPosBits) - 1; - m_NumPrevBits = numPrevBits; - uint numStates = (uint)1 << (m_NumPrevBits + m_NumPosBits); - m_Coders = new Encoder2[numStates]; + _numPosBits = numPosBits; + _posMask = ((uint)1 << numPosBits) - 1; + _numPrevBits = numPrevBits; + uint numStates = (uint)1 << (_numPrevBits + _numPosBits); + _coders = new Encoder2[numStates]; for (uint i = 0; i < numStates; i++) { - m_Coders[i].Create(); + _coders[i].Create(); } } public void Init() { - uint numStates = (uint)1 << (m_NumPrevBits + m_NumPosBits); + uint numStates = (uint)1 << (_numPrevBits + _numPosBits); for (uint i = 0; i < numStates; i++) { - m_Coders[i].Init(); + _coders[i].Init(); } } public Encoder2 GetSubCoder(UInt32 pos, Byte prevByte) { - return m_Coders[((pos & m_PosMask) << m_NumPrevBits) + (uint)(prevByte >> (8 - m_NumPrevBits))]; + return _coders[((pos & _posMask) << _numPrevBits) + (uint)(prevByte >> (8 - _numPrevBits))]; } } @@ -196,16 +196,16 @@ namespace SharpCompress.Compressors.LZMA { private BitEncoder _choice = new BitEncoder(); private BitEncoder _choice2 = new BitEncoder(); - private readonly BitTreeEncoder[] _lowCoder = new BitTreeEncoder[Base.kNumPosStatesEncodingMax]; - private readonly BitTreeEncoder[] _midCoder = new BitTreeEncoder[Base.kNumPosStatesEncodingMax]; - private BitTreeEncoder _highCoder = new BitTreeEncoder(Base.kNumHighLenBits); + private readonly BitTreeEncoder[] _lowCoder = new BitTreeEncoder[Base.K_NUM_POS_STATES_ENCODING_MAX]; + private readonly BitTreeEncoder[] _midCoder = new BitTreeEncoder[Base.K_NUM_POS_STATES_ENCODING_MAX]; + private BitTreeEncoder _highCoder = new BitTreeEncoder(Base.K_NUM_HIGH_LEN_BITS); public LenEncoder() { - for (UInt32 posState = 0; posState < Base.kNumPosStatesEncodingMax; posState++) + for (UInt32 posState = 0; posState < Base.K_NUM_POS_STATES_ENCODING_MAX; posState++) { - _lowCoder[posState] = new BitTreeEncoder(Base.kNumLowLenBits); - _midCoder[posState] = new BitTreeEncoder(Base.kNumMidLenBits); + _lowCoder[posState] = new BitTreeEncoder(Base.K_NUM_LOW_LEN_BITS); + _midCoder[posState] = new BitTreeEncoder(Base.K_NUM_MID_LEN_BITS); } } @@ -223,16 +223,16 @@ namespace SharpCompress.Compressors.LZMA public void Encode(RangeCoder.Encoder rangeEncoder, UInt32 symbol, UInt32 posState) { - if (symbol < Base.kNumLowLenSymbols) + if (symbol < Base.K_NUM_LOW_LEN_SYMBOLS) { _choice.Encode(rangeEncoder, 0); _lowCoder[posState].Encode(rangeEncoder, symbol); } else { - symbol -= Base.kNumLowLenSymbols; + symbol -= Base.K_NUM_LOW_LEN_SYMBOLS; _choice.Encode(rangeEncoder, 1); - if (symbol < Base.kNumMidLenSymbols) + if (symbol < Base.K_NUM_MID_LEN_SYMBOLS) { _choice2.Encode(rangeEncoder, 0); _midCoder[posState].Encode(rangeEncoder, symbol); @@ -240,7 +240,7 @@ namespace SharpCompress.Compressors.LZMA else { _choice2.Encode(rangeEncoder, 1); - _highCoder.Encode(rangeEncoder, symbol - Base.kNumMidLenSymbols); + _highCoder.Encode(rangeEncoder, symbol - Base.K_NUM_MID_LEN_SYMBOLS); } } } @@ -252,7 +252,7 @@ namespace SharpCompress.Compressors.LZMA UInt32 b0 = a1 + _choice2.GetPrice0(); UInt32 b1 = a1 + _choice2.GetPrice1(); UInt32 i = 0; - for (i = 0; i < Base.kNumLowLenSymbols; i++) + for (i = 0; i < Base.K_NUM_LOW_LEN_SYMBOLS; i++) { if (i >= numSymbols) { @@ -260,28 +260,28 @@ namespace SharpCompress.Compressors.LZMA } prices[st + i] = a0 + _lowCoder[posState].GetPrice(i); } - for (; i < Base.kNumLowLenSymbols + Base.kNumMidLenSymbols; i++) + for (; i < Base.K_NUM_LOW_LEN_SYMBOLS + Base.K_NUM_MID_LEN_SYMBOLS; i++) { if (i >= numSymbols) { return; } - prices[st + i] = b0 + _midCoder[posState].GetPrice(i - Base.kNumLowLenSymbols); + prices[st + i] = b0 + _midCoder[posState].GetPrice(i - Base.K_NUM_LOW_LEN_SYMBOLS); } for (; i < numSymbols; i++) { - prices[st + i] = b1 + _highCoder.GetPrice(i - Base.kNumLowLenSymbols - Base.kNumMidLenSymbols); + prices[st + i] = b1 + _highCoder.GetPrice(i - Base.K_NUM_LOW_LEN_SYMBOLS - Base.K_NUM_MID_LEN_SYMBOLS); } } } - private const UInt32 kNumLenSpecSymbols = Base.kNumLowLenSymbols + Base.kNumMidLenSymbols; + private const UInt32 K_NUM_LEN_SPEC_SYMBOLS = Base.K_NUM_LOW_LEN_SYMBOLS + Base.K_NUM_MID_LEN_SYMBOLS; private class LenPriceTableEncoder : LenEncoder { - private readonly UInt32[] _prices = new UInt32[Base.kNumLenSymbols << Base.kNumPosStatesBitsEncodingMax]; + private readonly UInt32[] _prices = new UInt32[Base.K_NUM_LEN_SYMBOLS << Base.K_NUM_POS_STATES_BITS_ENCODING_MAX]; private UInt32 _tableSize; - private readonly UInt32[] _counters = new UInt32[Base.kNumPosStatesEncodingMax]; + private readonly UInt32[] _counters = new UInt32[Base.K_NUM_POS_STATES_ENCODING_MAX]; public void SetTableSize(UInt32 tableSize) { @@ -290,12 +290,12 @@ namespace SharpCompress.Compressors.LZMA public UInt32 GetPrice(UInt32 symbol, UInt32 posState) { - return _prices[posState * Base.kNumLenSymbols + symbol]; + return _prices[posState * Base.K_NUM_LEN_SYMBOLS + symbol]; } private void UpdateTable(UInt32 posState) { - SetPrices(posState, _tableSize, _prices, posState * Base.kNumLenSymbols); + SetPrices(posState, _tableSize, _prices, posState * Base.K_NUM_LEN_SYMBOLS); _counters[posState] = _tableSize; } @@ -317,76 +317,76 @@ namespace SharpCompress.Compressors.LZMA } } - private const UInt32 kNumOpts = 1 << 12; + private const UInt32 K_NUM_OPTS = 1 << 12; private class Optimal { - public Base.State State; + public Base.State _state; - public bool Prev1IsChar; - public bool Prev2; + public bool _prev1IsChar; + public bool _prev2; - public UInt32 PosPrev2; - public UInt32 BackPrev2; + public UInt32 _posPrev2; + public UInt32 _backPrev2; - public UInt32 Price; - public UInt32 PosPrev; - public UInt32 BackPrev; + public UInt32 _price; + public UInt32 _posPrev; + public UInt32 _backPrev; - public UInt32 Backs0; - public UInt32 Backs1; - public UInt32 Backs2; - public UInt32 Backs3; + public UInt32 _backs0; + public UInt32 _backs1; + public UInt32 _backs2; + public UInt32 _backs3; public void MakeAsChar() { - BackPrev = 0xFFFFFFFF; - Prev1IsChar = false; + _backPrev = 0xFFFFFFFF; + _prev1IsChar = false; } public void MakeAsShortRep() { - BackPrev = 0; + _backPrev = 0; ; - Prev1IsChar = false; + _prev1IsChar = false; } public bool IsShortRep() { - return (BackPrev == 0); + return (_backPrev == 0); } } - private readonly Optimal[] _optimum = new Optimal[kNumOpts]; + private readonly Optimal[] _optimum = new Optimal[K_NUM_OPTS]; private BinTree _matchFinder; private readonly RangeCoder.Encoder _rangeEncoder = new RangeCoder.Encoder(); private readonly BitEncoder[] _isMatch = - new BitEncoder[Base.kNumStates << Base.kNumPosStatesBitsMax]; + new BitEncoder[Base.K_NUM_STATES << Base.K_NUM_POS_STATES_BITS_MAX]; - private readonly BitEncoder[] _isRep = new BitEncoder[Base.kNumStates]; - private readonly BitEncoder[] _isRepG0 = new BitEncoder[Base.kNumStates]; - private readonly BitEncoder[] _isRepG1 = new BitEncoder[Base.kNumStates]; - private readonly BitEncoder[] _isRepG2 = new BitEncoder[Base.kNumStates]; + private readonly BitEncoder[] _isRep = new BitEncoder[Base.K_NUM_STATES]; + private readonly BitEncoder[] _isRepG0 = new BitEncoder[Base.K_NUM_STATES]; + private readonly BitEncoder[] _isRepG1 = new BitEncoder[Base.K_NUM_STATES]; + private readonly BitEncoder[] _isRepG2 = new BitEncoder[Base.K_NUM_STATES]; private readonly BitEncoder[] _isRep0Long = - new BitEncoder[Base.kNumStates << Base.kNumPosStatesBitsMax]; + new BitEncoder[Base.K_NUM_STATES << Base.K_NUM_POS_STATES_BITS_MAX]; - private readonly BitTreeEncoder[] _posSlotEncoder = new BitTreeEncoder[Base.kNumLenToPosStates]; + private readonly BitTreeEncoder[] _posSlotEncoder = new BitTreeEncoder[Base.K_NUM_LEN_TO_POS_STATES]; private readonly BitEncoder[] _posEncoders = - new BitEncoder[Base.kNumFullDistances - Base.kEndPosModelIndex]; + new BitEncoder[Base.K_NUM_FULL_DISTANCES - Base.K_END_POS_MODEL_INDEX]; - private BitTreeEncoder _posAlignEncoder = new BitTreeEncoder(Base.kNumAlignBits); + private BitTreeEncoder _posAlignEncoder = new BitTreeEncoder(Base.K_NUM_ALIGN_BITS); private readonly LenPriceTableEncoder _lenEncoder = new LenPriceTableEncoder(); private readonly LenPriceTableEncoder _repMatchLenEncoder = new LenPriceTableEncoder(); private readonly LiteralEncoder _literalEncoder = new LiteralEncoder(); - private readonly UInt32[] _matchDistances = new UInt32[Base.kMatchMaxLen * 2 + 2]; + private readonly UInt32[] _matchDistances = new UInt32[Base.K_MATCH_MAX_LEN * 2 + 2]; - private UInt32 _numFastBytes = kNumFastBytesDefault; + private UInt32 _numFastBytes = K_NUM_FAST_BYTES_DEFAULT; private UInt32 _longestMatchLength; private UInt32 _numDistancePairs; @@ -397,30 +397,30 @@ namespace SharpCompress.Compressors.LZMA private bool _longestMatchWasFound; - private readonly UInt32[] _posSlotPrices = new UInt32[1 << (Base.kNumPosSlotBits + Base.kNumLenToPosStatesBits)]; - private readonly UInt32[] _distancesPrices = new UInt32[Base.kNumFullDistances << Base.kNumLenToPosStatesBits]; - private readonly UInt32[] _alignPrices = new UInt32[Base.kAlignTableSize]; + private readonly UInt32[] _posSlotPrices = new UInt32[1 << (Base.K_NUM_POS_SLOT_BITS + Base.K_NUM_LEN_TO_POS_STATES_BITS)]; + private readonly UInt32[] _distancesPrices = new UInt32[Base.K_NUM_FULL_DISTANCES << Base.K_NUM_LEN_TO_POS_STATES_BITS]; + private readonly UInt32[] _alignPrices = new UInt32[Base.K_ALIGN_TABLE_SIZE]; private UInt32 _alignPriceCount; - private UInt32 _distTableSize = (kDefaultDictionaryLogSize * 2); + private UInt32 _distTableSize = (K_DEFAULT_DICTIONARY_LOG_SIZE * 2); private int _posStateBits = 2; private UInt32 _posStateMask = (4 - 1); private int _numLiteralPosStateBits; private int _numLiteralContextBits = 3; - private UInt32 _dictionarySize = (1 << kDefaultDictionaryLogSize); + private UInt32 _dictionarySize = (1 << K_DEFAULT_DICTIONARY_LOG_SIZE); private UInt32 _dictionarySizePrev = 0xFFFFFFFF; private UInt32 _numFastBytesPrev = 0xFFFFFFFF; - private Int64 nowPos64; + private Int64 _nowPos64; private bool _finished; private Stream _inStream; - private EMatchFinderType _matchFinderType = EMatchFinderType.BT4; + private EMatchFinderType _matchFinderType = EMatchFinderType.Bt4; private bool _writeEndMark; - private bool _needReleaseMFStream; + private bool _needReleaseMfStream; private bool _processingMode; private void Create() @@ -429,7 +429,7 @@ namespace SharpCompress.Compressors.LZMA { BinTree bt = new BinTree(); int numHashBytes = 4; - if (_matchFinderType == EMatchFinderType.BT2) + if (_matchFinderType == EMatchFinderType.Bt2) { numHashBytes = 2; } @@ -442,20 +442,20 @@ namespace SharpCompress.Compressors.LZMA { return; } - _matchFinder.Create(_dictionarySize, kNumOpts, _numFastBytes, Base.kMatchMaxLen + 1 + kNumOpts); + _matchFinder.Create(_dictionarySize, K_NUM_OPTS, _numFastBytes, Base.K_MATCH_MAX_LEN + 1 + K_NUM_OPTS); _dictionarySizePrev = _dictionarySize; _numFastBytesPrev = _numFastBytes; } public Encoder() { - for (int i = 0; i < kNumOpts; i++) + for (int i = 0; i < K_NUM_OPTS; i++) { _optimum[i] = new Optimal(); } - for (int i = 0; i < Base.kNumLenToPosStates; i++) + for (int i = 0; i < Base.K_NUM_LEN_TO_POS_STATES; i++) { - _posSlotEncoder[i] = new BitTreeEncoder(Base.kNumPosSlotBits); + _posSlotEncoder[i] = new BitTreeEncoder(Base.K_NUM_POS_SLOT_BITS); } } @@ -470,11 +470,11 @@ namespace SharpCompress.Compressors.LZMA _rangeEncoder.Init(); uint i; - for (i = 0; i < Base.kNumStates; i++) + for (i = 0; i < Base.K_NUM_STATES; i++) { for (uint j = 0; j <= _posStateMask; j++) { - uint complexState = (i << Base.kNumPosStatesBitsMax) + j; + uint complexState = (i << Base.K_NUM_POS_STATES_BITS_MAX) + j; _isMatch[complexState].Init(); _isRep0Long[complexState].Init(); } @@ -484,11 +484,11 @@ namespace SharpCompress.Compressors.LZMA _isRepG2[i].Init(); } _literalEncoder.Init(); - for (i = 0; i < Base.kNumLenToPosStates; i++) + for (i = 0; i < Base.K_NUM_LEN_TO_POS_STATES; i++) { _posSlotEncoder[i].Init(); } - for (i = 0; i < Base.kNumFullDistances - Base.kEndPosModelIndex; i++) + for (i = 0; i < Base.K_NUM_FULL_DISTANCES - Base.K_END_POS_MODEL_INDEX; i++) { _posEncoders[i].Init(); } @@ -514,7 +514,7 @@ namespace SharpCompress.Compressors.LZMA if (lenRes == _numFastBytes) { lenRes += _matchFinder.GetMatchLen((int)lenRes - 1, _matchDistances[numDistancePairs - 1], - Base.kMatchMaxLen - lenRes); + Base.K_MATCH_MAX_LEN - lenRes); } } _additionalOffset++; @@ -531,8 +531,8 @@ namespace SharpCompress.Compressors.LZMA private UInt32 GetRepLen1Price(Base.State state, UInt32 posState) { - return _isRepG0[state.Index].GetPrice0() + - _isRep0Long[(state.Index << Base.kNumPosStatesBitsMax) + posState].GetPrice0(); + return _isRepG0[state._index].GetPrice0() + + _isRep0Long[(state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState].GetPrice0(); } private UInt32 GetPureRepPrice(UInt32 repIndex, Base.State state, UInt32 posState) @@ -540,20 +540,20 @@ namespace SharpCompress.Compressors.LZMA UInt32 price; if (repIndex == 0) { - price = _isRepG0[state.Index].GetPrice0(); - price += _isRep0Long[(state.Index << Base.kNumPosStatesBitsMax) + posState].GetPrice1(); + price = _isRepG0[state._index].GetPrice0(); + price += _isRep0Long[(state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState].GetPrice1(); } else { - price = _isRepG0[state.Index].GetPrice1(); + price = _isRepG0[state._index].GetPrice1(); if (repIndex == 1) { - price += _isRepG1[state.Index].GetPrice0(); + price += _isRepG1[state._index].GetPrice0(); } else { - price += _isRepG1[state.Index].GetPrice1(); - price += _isRepG2[state.Index].GetPrice(repIndex - 2); + price += _isRepG1[state._index].GetPrice1(); + price += _isRepG2[state._index].GetPrice(repIndex - 2); } } return price; @@ -561,7 +561,7 @@ namespace SharpCompress.Compressors.LZMA private UInt32 GetRepPrice(UInt32 repIndex, UInt32 len, Base.State state, UInt32 posState) { - UInt32 price = _repMatchLenEncoder.GetPrice(len - Base.kMatchMinLen, posState); + UInt32 price = _repMatchLenEncoder.GetPrice(len - Base.K_MATCH_MIN_LEN, posState); return price + GetPureRepPrice(repIndex, state, posState); } @@ -569,62 +569,62 @@ namespace SharpCompress.Compressors.LZMA { UInt32 price; UInt32 lenToPosState = Base.GetLenToPosState(len); - if (pos < Base.kNumFullDistances) + if (pos < Base.K_NUM_FULL_DISTANCES) { - price = _distancesPrices[(lenToPosState * Base.kNumFullDistances) + pos]; + price = _distancesPrices[(lenToPosState * Base.K_NUM_FULL_DISTANCES) + pos]; } else { - price = _posSlotPrices[(lenToPosState << Base.kNumPosSlotBits) + GetPosSlot2(pos)] + - _alignPrices[pos & Base.kAlignMask]; + price = _posSlotPrices[(lenToPosState << Base.K_NUM_POS_SLOT_BITS) + GetPosSlot2(pos)] + + _alignPrices[pos & Base.K_ALIGN_MASK]; } - return price + _lenEncoder.GetPrice(len - Base.kMatchMinLen, posState); + return price + _lenEncoder.GetPrice(len - Base.K_MATCH_MIN_LEN, posState); } private UInt32 Backward(out UInt32 backRes, UInt32 cur) { _optimumEndIndex = cur; - UInt32 posMem = _optimum[cur].PosPrev; - UInt32 backMem = _optimum[cur].BackPrev; + UInt32 posMem = _optimum[cur]._posPrev; + UInt32 backMem = _optimum[cur]._backPrev; do { - if (_optimum[cur].Prev1IsChar) + if (_optimum[cur]._prev1IsChar) { _optimum[posMem].MakeAsChar(); - _optimum[posMem].PosPrev = posMem - 1; - if (_optimum[cur].Prev2) + _optimum[posMem]._posPrev = posMem - 1; + if (_optimum[cur]._prev2) { - _optimum[posMem - 1].Prev1IsChar = false; - _optimum[posMem - 1].PosPrev = _optimum[cur].PosPrev2; - _optimum[posMem - 1].BackPrev = _optimum[cur].BackPrev2; + _optimum[posMem - 1]._prev1IsChar = false; + _optimum[posMem - 1]._posPrev = _optimum[cur]._posPrev2; + _optimum[posMem - 1]._backPrev = _optimum[cur]._backPrev2; } } UInt32 posPrev = posMem; UInt32 backCur = backMem; - backMem = _optimum[posPrev].BackPrev; - posMem = _optimum[posPrev].PosPrev; + backMem = _optimum[posPrev]._backPrev; + posMem = _optimum[posPrev]._posPrev; - _optimum[posPrev].BackPrev = backCur; - _optimum[posPrev].PosPrev = cur; + _optimum[posPrev]._backPrev = backCur; + _optimum[posPrev]._posPrev = cur; cur = posPrev; } while (cur > 0); - backRes = _optimum[0].BackPrev; - _optimumCurrentIndex = _optimum[0].PosPrev; + backRes = _optimum[0]._backPrev; + _optimumCurrentIndex = _optimum[0]._posPrev; return _optimumCurrentIndex; } - private readonly UInt32[] reps = new UInt32[Base.kNumRepDistances]; - private readonly UInt32[] repLens = new UInt32[Base.kNumRepDistances]; + private readonly UInt32[] _reps = new UInt32[Base.K_NUM_REP_DISTANCES]; + private readonly UInt32[] _repLens = new UInt32[Base.K_NUM_REP_DISTANCES]; private UInt32 GetOptimum(UInt32 position, out UInt32 backRes) { if (_optimumEndIndex != _optimumCurrentIndex) { - UInt32 lenRes = _optimum[_optimumCurrentIndex].PosPrev - _optimumCurrentIndex; - backRes = _optimum[_optimumCurrentIndex].BackPrev; - _optimumCurrentIndex = _optimum[_optimumCurrentIndex].PosPrev; + UInt32 lenRes = _optimum[_optimumCurrentIndex]._posPrev - _optimumCurrentIndex; + backRes = _optimum[_optimumCurrentIndex]._backPrev; + _optimumCurrentIndex = _optimum[_optimumCurrentIndex]._posPrev; return lenRes; } _optimumCurrentIndex = _optimumEndIndex = 0; @@ -647,33 +647,33 @@ namespace SharpCompress.Compressors.LZMA backRes = 0xFFFFFFFF; return 1; } - if (numAvailableBytes > Base.kMatchMaxLen) + if (numAvailableBytes > Base.K_MATCH_MAX_LEN) { - numAvailableBytes = Base.kMatchMaxLen; + numAvailableBytes = Base.K_MATCH_MAX_LEN; } UInt32 repMaxIndex = 0; UInt32 i; - for (i = 0; i < Base.kNumRepDistances; i++) + for (i = 0; i < Base.K_NUM_REP_DISTANCES; i++) { - reps[i] = _repDistances[i]; - repLens[i] = _matchFinder.GetMatchLen(0 - 1, reps[i], Base.kMatchMaxLen); - if (repLens[i] > repLens[repMaxIndex]) + _reps[i] = _repDistances[i]; + _repLens[i] = _matchFinder.GetMatchLen(0 - 1, _reps[i], Base.K_MATCH_MAX_LEN); + if (_repLens[i] > _repLens[repMaxIndex]) { repMaxIndex = i; } } - if (repLens[repMaxIndex] >= _numFastBytes) + if (_repLens[repMaxIndex] >= _numFastBytes) { backRes = repMaxIndex; - UInt32 lenRes = repLens[repMaxIndex]; + UInt32 lenRes = _repLens[repMaxIndex]; MovePos(lenRes - 1); return lenRes; } if (lenMain >= _numFastBytes) { - backRes = _matchDistances[numDistancePairs - 1] + Base.kNumRepDistances; + backRes = _matchDistances[numDistancePairs - 1] + Base.K_NUM_REP_DISTANCES; MovePos(lenMain - 1); return lenMain; } @@ -681,59 +681,59 @@ namespace SharpCompress.Compressors.LZMA Byte currentByte = _matchFinder.GetIndexByte(0 - 1); Byte matchByte = _matchFinder.GetIndexByte((Int32)(0 - _repDistances[0] - 1 - 1)); - if (lenMain < 2 && currentByte != matchByte && repLens[repMaxIndex] < 2) + if (lenMain < 2 && currentByte != matchByte && _repLens[repMaxIndex] < 2) { backRes = 0xFFFFFFFF; return 1; } - _optimum[0].State = _state; + _optimum[0]._state = _state; UInt32 posState = (position & _posStateMask); - _optimum[1].Price = _isMatch[(_state.Index << Base.kNumPosStatesBitsMax) + posState].GetPrice0() + + _optimum[1]._price = _isMatch[(_state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState].GetPrice0() + _literalEncoder.GetSubCoder(position, _previousByte) .GetPrice(!_state.IsCharState(), matchByte, currentByte); _optimum[1].MakeAsChar(); - UInt32 matchPrice = _isMatch[(_state.Index << Base.kNumPosStatesBitsMax) + posState].GetPrice1(); - UInt32 repMatchPrice = matchPrice + _isRep[_state.Index].GetPrice1(); + UInt32 matchPrice = _isMatch[(_state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState].GetPrice1(); + UInt32 repMatchPrice = matchPrice + _isRep[_state._index].GetPrice1(); if (matchByte == currentByte) { UInt32 shortRepPrice = repMatchPrice + GetRepLen1Price(_state, posState); - if (shortRepPrice < _optimum[1].Price) + if (shortRepPrice < _optimum[1]._price) { - _optimum[1].Price = shortRepPrice; + _optimum[1]._price = shortRepPrice; _optimum[1].MakeAsShortRep(); } } - UInt32 lenEnd = ((lenMain >= repLens[repMaxIndex]) ? lenMain : repLens[repMaxIndex]); + UInt32 lenEnd = ((lenMain >= _repLens[repMaxIndex]) ? lenMain : _repLens[repMaxIndex]); if (lenEnd < 2) { - backRes = _optimum[1].BackPrev; + backRes = _optimum[1]._backPrev; return 1; } - _optimum[1].PosPrev = 0; + _optimum[1]._posPrev = 0; - _optimum[0].Backs0 = reps[0]; - _optimum[0].Backs1 = reps[1]; - _optimum[0].Backs2 = reps[2]; - _optimum[0].Backs3 = reps[3]; + _optimum[0]._backs0 = _reps[0]; + _optimum[0]._backs1 = _reps[1]; + _optimum[0]._backs2 = _reps[2]; + _optimum[0]._backs3 = _reps[3]; UInt32 len = lenEnd; do { - _optimum[len--].Price = kIfinityPrice; + _optimum[len--]._price = K_IFINITY_PRICE; } while (len >= 2); - for (i = 0; i < Base.kNumRepDistances; i++) + for (i = 0; i < Base.K_NUM_REP_DISTANCES; i++) { - UInt32 repLen = repLens[i]; + UInt32 repLen = _repLens[i]; if (repLen < 2) { continue; @@ -743,20 +743,20 @@ namespace SharpCompress.Compressors.LZMA { UInt32 curAndLenPrice = price + _repMatchLenEncoder.GetPrice(repLen - 2, posState); Optimal optimum = _optimum[repLen]; - if (curAndLenPrice < optimum.Price) + if (curAndLenPrice < optimum._price) { - optimum.Price = curAndLenPrice; - optimum.PosPrev = 0; - optimum.BackPrev = i; - optimum.Prev1IsChar = false; + optimum._price = curAndLenPrice; + optimum._posPrev = 0; + optimum._backPrev = i; + optimum._prev1IsChar = false; } } while (--repLen >= 2); } - UInt32 normalMatchPrice = matchPrice + _isRep[_state.Index].GetPrice0(); + UInt32 normalMatchPrice = matchPrice + _isRep[_state._index].GetPrice0(); - len = ((repLens[0] >= 2) ? repLens[0] + 1 : 2); + len = ((_repLens[0] >= 2) ? _repLens[0] + 1 : 2); if (len <= lenMain) { UInt32 offs = 0; @@ -769,12 +769,12 @@ namespace SharpCompress.Compressors.LZMA UInt32 distance = _matchDistances[offs + 1]; UInt32 curAndLenPrice = normalMatchPrice + GetPosLenPrice(distance, len, posState); Optimal optimum = _optimum[len]; - if (curAndLenPrice < optimum.Price) + if (curAndLenPrice < optimum._price) { - optimum.Price = curAndLenPrice; - optimum.PosPrev = 0; - optimum.BackPrev = distance + Base.kNumRepDistances; - optimum.Prev1IsChar = false; + optimum._price = curAndLenPrice; + optimum._posPrev = 0; + optimum._backPrev = distance + Base.K_NUM_REP_DISTANCES; + optimum._prev1IsChar = false; } if (len == _matchDistances[offs]) { @@ -806,15 +806,15 @@ namespace SharpCompress.Compressors.LZMA return Backward(out backRes, cur); } position++; - UInt32 posPrev = _optimum[cur].PosPrev; + UInt32 posPrev = _optimum[cur]._posPrev; Base.State state; - if (_optimum[cur].Prev1IsChar) + if (_optimum[cur]._prev1IsChar) { posPrev--; - if (_optimum[cur].Prev2) + if (_optimum[cur]._prev2) { - state = _optimum[_optimum[cur].PosPrev2].State; - if (_optimum[cur].BackPrev2 < Base.kNumRepDistances) + state = _optimum[_optimum[cur]._posPrev2]._state; + if (_optimum[cur]._backPrev2 < Base.K_NUM_REP_DISTANCES) { state.UpdateRep(); } @@ -825,13 +825,13 @@ namespace SharpCompress.Compressors.LZMA } else { - state = _optimum[posPrev].State; + state = _optimum[posPrev]._state; } state.UpdateChar(); } else { - state = _optimum[posPrev].State; + state = _optimum[posPrev]._state; } if (posPrev == cur - 1) { @@ -847,16 +847,16 @@ namespace SharpCompress.Compressors.LZMA else { UInt32 pos; - if (_optimum[cur].Prev1IsChar && _optimum[cur].Prev2) + if (_optimum[cur]._prev1IsChar && _optimum[cur]._prev2) { - posPrev = _optimum[cur].PosPrev2; - pos = _optimum[cur].BackPrev2; + posPrev = _optimum[cur]._posPrev2; + pos = _optimum[cur]._backPrev2; state.UpdateRep(); } else { - pos = _optimum[cur].BackPrev; - if (pos < Base.kNumRepDistances) + pos = _optimum[cur]._backPrev; + if (pos < Base.K_NUM_REP_DISTANCES) { state.UpdateRep(); } @@ -866,91 +866,91 @@ namespace SharpCompress.Compressors.LZMA } } Optimal opt = _optimum[posPrev]; - if (pos < Base.kNumRepDistances) + if (pos < Base.K_NUM_REP_DISTANCES) { if (pos == 0) { - reps[0] = opt.Backs0; - reps[1] = opt.Backs1; - reps[2] = opt.Backs2; - reps[3] = opt.Backs3; + _reps[0] = opt._backs0; + _reps[1] = opt._backs1; + _reps[2] = opt._backs2; + _reps[3] = opt._backs3; } else if (pos == 1) { - reps[0] = opt.Backs1; - reps[1] = opt.Backs0; - reps[2] = opt.Backs2; - reps[3] = opt.Backs3; + _reps[0] = opt._backs1; + _reps[1] = opt._backs0; + _reps[2] = opt._backs2; + _reps[3] = opt._backs3; } else if (pos == 2) { - reps[0] = opt.Backs2; - reps[1] = opt.Backs0; - reps[2] = opt.Backs1; - reps[3] = opt.Backs3; + _reps[0] = opt._backs2; + _reps[1] = opt._backs0; + _reps[2] = opt._backs1; + _reps[3] = opt._backs3; } else { - reps[0] = opt.Backs3; - reps[1] = opt.Backs0; - reps[2] = opt.Backs1; - reps[3] = opt.Backs2; + _reps[0] = opt._backs3; + _reps[1] = opt._backs0; + _reps[2] = opt._backs1; + _reps[3] = opt._backs2; } } else { - reps[0] = (pos - Base.kNumRepDistances); - reps[1] = opt.Backs0; - reps[2] = opt.Backs1; - reps[3] = opt.Backs2; + _reps[0] = (pos - Base.K_NUM_REP_DISTANCES); + _reps[1] = opt._backs0; + _reps[2] = opt._backs1; + _reps[3] = opt._backs2; } } - _optimum[cur].State = state; - _optimum[cur].Backs0 = reps[0]; - _optimum[cur].Backs1 = reps[1]; - _optimum[cur].Backs2 = reps[2]; - _optimum[cur].Backs3 = reps[3]; - UInt32 curPrice = _optimum[cur].Price; + _optimum[cur]._state = state; + _optimum[cur]._backs0 = _reps[0]; + _optimum[cur]._backs1 = _reps[1]; + _optimum[cur]._backs2 = _reps[2]; + _optimum[cur]._backs3 = _reps[3]; + UInt32 curPrice = _optimum[cur]._price; currentByte = _matchFinder.GetIndexByte(0 - 1); - matchByte = _matchFinder.GetIndexByte((Int32)(0 - reps[0] - 1 - 1)); + matchByte = _matchFinder.GetIndexByte((Int32)(0 - _reps[0] - 1 - 1)); posState = (position & _posStateMask); UInt32 curAnd1Price = curPrice + - _isMatch[(state.Index << Base.kNumPosStatesBitsMax) + posState].GetPrice0() + + _isMatch[(state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState].GetPrice0() + _literalEncoder.GetSubCoder(position, _matchFinder.GetIndexByte(0 - 2)). GetPrice(!state.IsCharState(), matchByte, currentByte); Optimal nextOptimum = _optimum[cur + 1]; bool nextIsChar = false; - if (curAnd1Price < nextOptimum.Price) + if (curAnd1Price < nextOptimum._price) { - nextOptimum.Price = curAnd1Price; - nextOptimum.PosPrev = cur; + nextOptimum._price = curAnd1Price; + nextOptimum._posPrev = cur; nextOptimum.MakeAsChar(); nextIsChar = true; } - matchPrice = curPrice + _isMatch[(state.Index << Base.kNumPosStatesBitsMax) + posState].GetPrice1(); - repMatchPrice = matchPrice + _isRep[state.Index].GetPrice1(); + matchPrice = curPrice + _isMatch[(state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState].GetPrice1(); + repMatchPrice = matchPrice + _isRep[state._index].GetPrice1(); if (matchByte == currentByte && - !(nextOptimum.PosPrev < cur && nextOptimum.BackPrev == 0)) + !(nextOptimum._posPrev < cur && nextOptimum._backPrev == 0)) { UInt32 shortRepPrice = repMatchPrice + GetRepLen1Price(state, posState); - if (shortRepPrice <= nextOptimum.Price) + if (shortRepPrice <= nextOptimum._price) { - nextOptimum.Price = shortRepPrice; - nextOptimum.PosPrev = cur; + nextOptimum._price = shortRepPrice; + nextOptimum._posPrev = cur; nextOptimum.MakeAsShortRep(); nextIsChar = true; } } UInt32 numAvailableBytesFull = _matchFinder.GetNumAvailableBytes() + 1; - numAvailableBytesFull = Math.Min(kNumOpts - 1 - cur, numAvailableBytesFull); + numAvailableBytesFull = Math.Min(K_NUM_OPTS - 1 - cur, numAvailableBytesFull); numAvailableBytes = numAvailableBytesFull; if (numAvailableBytes < 2) @@ -965,32 +965,32 @@ namespace SharpCompress.Compressors.LZMA { // try Literal + rep0 UInt32 t = Math.Min(numAvailableBytesFull - 1, _numFastBytes); - UInt32 lenTest2 = _matchFinder.GetMatchLen(0, reps[0], t); + UInt32 lenTest2 = _matchFinder.GetMatchLen(0, _reps[0], t); if (lenTest2 >= 2) { Base.State state2 = state; state2.UpdateChar(); UInt32 posStateNext = (position + 1) & _posStateMask; UInt32 nextRepMatchPrice = curAnd1Price + - _isMatch[(state2.Index << Base.kNumPosStatesBitsMax) + posStateNext] + _isMatch[(state2._index << Base.K_NUM_POS_STATES_BITS_MAX) + posStateNext] .GetPrice1() + - _isRep[state2.Index].GetPrice1(); + _isRep[state2._index].GetPrice1(); { UInt32 offset = cur + 1 + lenTest2; while (lenEnd < offset) { - _optimum[++lenEnd].Price = kIfinityPrice; + _optimum[++lenEnd]._price = K_IFINITY_PRICE; } UInt32 curAndLenPrice = nextRepMatchPrice + GetRepPrice( 0, lenTest2, state2, posStateNext); Optimal optimum = _optimum[offset]; - if (curAndLenPrice < optimum.Price) + if (curAndLenPrice < optimum._price) { - optimum.Price = curAndLenPrice; - optimum.PosPrev = cur + 1; - optimum.BackPrev = 0; - optimum.Prev1IsChar = true; - optimum.Prev2 = false; + optimum._price = curAndLenPrice; + optimum._posPrev = cur + 1; + optimum._backPrev = 0; + optimum._prev1IsChar = true; + optimum._prev2 = false; } } } @@ -998,9 +998,9 @@ namespace SharpCompress.Compressors.LZMA UInt32 startLen = 2; // speed optimization - for (UInt32 repIndex = 0; repIndex < Base.kNumRepDistances; repIndex++) + for (UInt32 repIndex = 0; repIndex < Base.K_NUM_REP_DISTANCES; repIndex++) { - UInt32 lenTest = _matchFinder.GetMatchLen(0 - 1, reps[repIndex], numAvailableBytes); + UInt32 lenTest = _matchFinder.GetMatchLen(0 - 1, _reps[repIndex], numAvailableBytes); if (lenTest < 2) { continue; @@ -1010,16 +1010,16 @@ namespace SharpCompress.Compressors.LZMA { while (lenEnd < cur + lenTest) { - _optimum[++lenEnd].Price = kIfinityPrice; + _optimum[++lenEnd]._price = K_IFINITY_PRICE; } UInt32 curAndLenPrice = repMatchPrice + GetRepPrice(repIndex, lenTest, state, posState); Optimal optimum = _optimum[cur + lenTest]; - if (curAndLenPrice < optimum.Price) + if (curAndLenPrice < optimum._price) { - optimum.Price = curAndLenPrice; - optimum.PosPrev = cur; - optimum.BackPrev = repIndex; - optimum.Prev1IsChar = false; + optimum._price = curAndLenPrice; + optimum._posPrev = cur; + optimum._backPrev = repIndex; + optimum._prev1IsChar = false; } } while (--lenTest >= 2); @@ -1034,7 +1034,7 @@ namespace SharpCompress.Compressors.LZMA if (lenTest < numAvailableBytesFull) { UInt32 t = Math.Min(numAvailableBytesFull - 1 - lenTest, _numFastBytes); - UInt32 lenTest2 = _matchFinder.GetMatchLen((Int32)lenTest, reps[repIndex], t); + UInt32 lenTest2 = _matchFinder.GetMatchLen((Int32)lenTest, _reps[repIndex], t); if (lenTest2 >= 2) { Base.State state2 = state; @@ -1042,39 +1042,39 @@ namespace SharpCompress.Compressors.LZMA UInt32 posStateNext = (position + lenTest) & _posStateMask; UInt32 curAndLenCharPrice = repMatchPrice + GetRepPrice(repIndex, lenTest, state, posState) + - _isMatch[(state2.Index << Base.kNumPosStatesBitsMax) + posStateNext].GetPrice0() + + _isMatch[(state2._index << Base.K_NUM_POS_STATES_BITS_MAX) + posStateNext].GetPrice0() + _literalEncoder.GetSubCoder(position + lenTest, _matchFinder.GetIndexByte((Int32)lenTest - 1 - 1)) .GetPrice(true, _matchFinder.GetIndexByte( - (Int32)lenTest - 1 - (Int32)(reps[repIndex] + 1)), + (Int32)lenTest - 1 - (Int32)(_reps[repIndex] + 1)), _matchFinder.GetIndexByte((Int32)lenTest - 1)); state2.UpdateChar(); posStateNext = (position + lenTest + 1) & _posStateMask; UInt32 nextMatchPrice = curAndLenCharPrice + - _isMatch[(state2.Index << Base.kNumPosStatesBitsMax) + posStateNext] + _isMatch[(state2._index << Base.K_NUM_POS_STATES_BITS_MAX) + posStateNext] .GetPrice1(); - UInt32 nextRepMatchPrice = nextMatchPrice + _isRep[state2.Index].GetPrice1(); + UInt32 nextRepMatchPrice = nextMatchPrice + _isRep[state2._index].GetPrice1(); // for(; lenTest2 >= 2; lenTest2--) { UInt32 offset = lenTest + 1 + lenTest2; while (lenEnd < cur + offset) { - _optimum[++lenEnd].Price = kIfinityPrice; + _optimum[++lenEnd]._price = K_IFINITY_PRICE; } UInt32 curAndLenPrice = nextRepMatchPrice + GetRepPrice(0, lenTest2, state2, posStateNext); Optimal optimum = _optimum[cur + offset]; - if (curAndLenPrice < optimum.Price) + if (curAndLenPrice < optimum._price) { - optimum.Price = curAndLenPrice; - optimum.PosPrev = cur + lenTest + 1; - optimum.BackPrev = 0; - optimum.Prev1IsChar = true; - optimum.Prev2 = true; - optimum.PosPrev2 = cur; - optimum.BackPrev2 = repIndex; + optimum._price = curAndLenPrice; + optimum._posPrev = cur + lenTest + 1; + optimum._backPrev = 0; + optimum._prev1IsChar = true; + optimum._prev2 = true; + optimum._posPrev2 = cur; + optimum._backPrev2 = repIndex; } } } @@ -1093,10 +1093,10 @@ namespace SharpCompress.Compressors.LZMA } if (newLen >= startLen) { - normalMatchPrice = matchPrice + _isRep[state.Index].GetPrice0(); + normalMatchPrice = matchPrice + _isRep[state._index].GetPrice0(); while (lenEnd < cur + newLen) { - _optimum[++lenEnd].Price = kIfinityPrice; + _optimum[++lenEnd]._price = K_IFINITY_PRICE; } UInt32 offs = 0; @@ -1110,12 +1110,12 @@ namespace SharpCompress.Compressors.LZMA UInt32 curBack = _matchDistances[offs + 1]; UInt32 curAndLenPrice = normalMatchPrice + GetPosLenPrice(curBack, lenTest, posState); Optimal optimum = _optimum[cur + lenTest]; - if (curAndLenPrice < optimum.Price) + if (curAndLenPrice < optimum._price) { - optimum.Price = curAndLenPrice; - optimum.PosPrev = cur; - optimum.BackPrev = curBack + Base.kNumRepDistances; - optimum.Prev1IsChar = false; + optimum._price = curAndLenPrice; + optimum._posPrev = cur; + optimum._backPrev = curBack + Base.K_NUM_REP_DISTANCES; + optimum._prev1IsChar = false; } if (lenTest == _matchDistances[offs]) @@ -1131,7 +1131,7 @@ namespace SharpCompress.Compressors.LZMA UInt32 posStateNext = (position + lenTest) & _posStateMask; UInt32 curAndLenCharPrice = curAndLenPrice + _isMatch[ - (state2.Index << Base.kNumPosStatesBitsMax) + + (state2._index << Base.K_NUM_POS_STATES_BITS_MAX) + posStateNext].GetPrice0() + _literalEncoder.GetSubCoder(position + lenTest, _matchFinder.GetIndexByte( @@ -1147,26 +1147,26 @@ namespace SharpCompress.Compressors.LZMA posStateNext = (position + lenTest + 1) & _posStateMask; UInt32 nextMatchPrice = curAndLenCharPrice + _isMatch[ - (state2.Index << Base.kNumPosStatesBitsMax) + + (state2._index << Base.K_NUM_POS_STATES_BITS_MAX) + posStateNext].GetPrice1(); - UInt32 nextRepMatchPrice = nextMatchPrice + _isRep[state2.Index].GetPrice1(); + UInt32 nextRepMatchPrice = nextMatchPrice + _isRep[state2._index].GetPrice1(); UInt32 offset = lenTest + 1 + lenTest2; while (lenEnd < cur + offset) { - _optimum[++lenEnd].Price = kIfinityPrice; + _optimum[++lenEnd]._price = K_IFINITY_PRICE; } curAndLenPrice = nextRepMatchPrice + GetRepPrice(0, lenTest2, state2, posStateNext); optimum = _optimum[cur + offset]; - if (curAndLenPrice < optimum.Price) + if (curAndLenPrice < optimum._price) { - optimum.Price = curAndLenPrice; - optimum.PosPrev = cur + lenTest + 1; - optimum.BackPrev = 0; - optimum.Prev1IsChar = true; - optimum.Prev2 = true; - optimum.PosPrev2 = cur; - optimum.BackPrev2 = curBack + Base.kNumRepDistances; + optimum._price = curAndLenPrice; + optimum._posPrev = cur + lenTest + 1; + optimum._backPrev = 0; + optimum._prev1IsChar = true; + optimum._prev2 = true; + optimum._posPrev2 = cur; + optimum._backPrev2 = curBack + Base.K_NUM_REP_DISTANCES; } } } @@ -1194,23 +1194,23 @@ namespace SharpCompress.Compressors.LZMA return; } - _isMatch[(_state.Index << Base.kNumPosStatesBitsMax) + posState].Encode(_rangeEncoder, 1); - _isRep[_state.Index].Encode(_rangeEncoder, 0); + _isMatch[(_state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState].Encode(_rangeEncoder, 1); + _isRep[_state._index].Encode(_rangeEncoder, 0); _state.UpdateMatch(); - UInt32 len = Base.kMatchMinLen; - _lenEncoder.Encode(_rangeEncoder, len - Base.kMatchMinLen, posState); - UInt32 posSlot = (1 << Base.kNumPosSlotBits) - 1; + UInt32 len = Base.K_MATCH_MIN_LEN; + _lenEncoder.Encode(_rangeEncoder, len - Base.K_MATCH_MIN_LEN, posState); + UInt32 posSlot = (1 << Base.K_NUM_POS_SLOT_BITS) - 1; UInt32 lenToPosState = Base.GetLenToPosState(len); _posSlotEncoder[lenToPosState].Encode(_rangeEncoder, posSlot); int footerBits = 30; UInt32 posReduced = (((UInt32)1) << footerBits) - 1; - _rangeEncoder.EncodeDirectBits(posReduced >> Base.kNumAlignBits, footerBits - Base.kNumAlignBits); - _posAlignEncoder.ReverseEncode(_rangeEncoder, posReduced & Base.kAlignMask); + _rangeEncoder.EncodeDirectBits(posReduced >> Base.K_NUM_ALIGN_BITS, footerBits - Base.K_NUM_ALIGN_BITS); + _posAlignEncoder.ReverseEncode(_rangeEncoder, posReduced & Base.K_ALIGN_MASK); } private void Flush(UInt32 nowPos) { - ReleaseMFStream(); + ReleaseMfStream(); WriteEndMarker(nowPos & _posStateMask); _rangeEncoder.FlushData(); _rangeEncoder.FlushStream(); @@ -1225,7 +1225,7 @@ namespace SharpCompress.Compressors.LZMA if (_inStream != null) { _matchFinder.SetStream(_inStream); - _needReleaseMFStream = true; + _needReleaseMfStream = true; _inStream = null; } @@ -1235,8 +1235,8 @@ namespace SharpCompress.Compressors.LZMA } _finished = true; - Int64 progressPosValuePrev = nowPos64; - if (nowPos64 == 0) + Int64 progressPosValuePrev = _nowPos64; + if (_nowPos64 == 0) { if (_trainSize > 0) { @@ -1256,19 +1256,19 @@ namespace SharpCompress.Compressors.LZMA } if (_matchFinder.GetNumAvailableBytes() == 0) { - Flush((UInt32)nowPos64); + Flush((UInt32)_nowPos64); return; } UInt32 len, numDistancePairs; // it's not used ReadMatchDistances(out len, out numDistancePairs); - UInt32 posState = (UInt32)(nowPos64) & _posStateMask; - _isMatch[(_state.Index << Base.kNumPosStatesBitsMax) + posState].Encode(_rangeEncoder, 0); + UInt32 posState = (UInt32)(_nowPos64) & _posStateMask; + _isMatch[(_state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState].Encode(_rangeEncoder, 0); _state.UpdateChar(); Byte curByte = _matchFinder.GetIndexByte((Int32)(0 - _additionalOffset)); - _literalEncoder.GetSubCoder((UInt32)(nowPos64), _previousByte).Encode(_rangeEncoder, curByte); + _literalEncoder.GetSubCoder((UInt32)(_nowPos64), _previousByte).Encode(_rangeEncoder, curByte); _previousByte = curByte; _additionalOffset--; - nowPos64++; + _nowPos64++; } if (_processingMode && _matchFinder.IsDataStarved) { @@ -1277,7 +1277,7 @@ namespace SharpCompress.Compressors.LZMA } if (_matchFinder.GetNumAvailableBytes() == 0) { - Flush((UInt32)nowPos64); + Flush((UInt32)_nowPos64); return; } while (true) @@ -1289,15 +1289,15 @@ namespace SharpCompress.Compressors.LZMA } UInt32 pos; - UInt32 len = GetOptimum((UInt32)nowPos64, out pos); + UInt32 len = GetOptimum((UInt32)_nowPos64, out pos); - UInt32 posState = ((UInt32)nowPos64) & _posStateMask; - UInt32 complexState = (_state.Index << Base.kNumPosStatesBitsMax) + posState; + UInt32 posState = ((UInt32)_nowPos64) & _posStateMask; + UInt32 complexState = (_state._index << Base.K_NUM_POS_STATES_BITS_MAX) + posState; if (len == 1 && pos == 0xFFFFFFFF) { _isMatch[complexState].Encode(_rangeEncoder, 0); Byte curByte = _matchFinder.GetIndexByte((Int32)(0 - _additionalOffset)); - LiteralEncoder.Encoder2 subCoder = _literalEncoder.GetSubCoder((UInt32)nowPos64, _previousByte); + LiteralEncoder.Encoder2 subCoder = _literalEncoder.GetSubCoder((UInt32)_nowPos64, _previousByte); if (!_state.IsCharState()) { Byte matchByte = @@ -1314,12 +1314,12 @@ namespace SharpCompress.Compressors.LZMA else { _isMatch[complexState].Encode(_rangeEncoder, 1); - if (pos < Base.kNumRepDistances) + if (pos < Base.K_NUM_REP_DISTANCES) { - _isRep[_state.Index].Encode(_rangeEncoder, 1); + _isRep[_state._index].Encode(_rangeEncoder, 1); if (pos == 0) { - _isRepG0[_state.Index].Encode(_rangeEncoder, 0); + _isRepG0[_state._index].Encode(_rangeEncoder, 0); if (len == 1) { _isRep0Long[complexState].Encode(_rangeEncoder, 0); @@ -1331,15 +1331,15 @@ namespace SharpCompress.Compressors.LZMA } else { - _isRepG0[_state.Index].Encode(_rangeEncoder, 1); + _isRepG0[_state._index].Encode(_rangeEncoder, 1); if (pos == 1) { - _isRepG1[_state.Index].Encode(_rangeEncoder, 0); + _isRepG1[_state._index].Encode(_rangeEncoder, 0); } else { - _isRepG1[_state.Index].Encode(_rangeEncoder, 1); - _isRepG2[_state.Index].Encode(_rangeEncoder, pos - 2); + _isRepG1[_state._index].Encode(_rangeEncoder, 1); + _isRepG2[_state._index].Encode(_rangeEncoder, pos - 2); } } if (len == 1) @@ -1348,7 +1348,7 @@ namespace SharpCompress.Compressors.LZMA } else { - _repMatchLenEncoder.Encode(_rangeEncoder, len - Base.kMatchMinLen, posState); + _repMatchLenEncoder.Encode(_rangeEncoder, len - Base.K_MATCH_MIN_LEN, posState); _state.UpdateRep(); } UInt32 distance = _repDistances[pos]; @@ -1363,21 +1363,21 @@ namespace SharpCompress.Compressors.LZMA } else { - _isRep[_state.Index].Encode(_rangeEncoder, 0); + _isRep[_state._index].Encode(_rangeEncoder, 0); _state.UpdateMatch(); - _lenEncoder.Encode(_rangeEncoder, len - Base.kMatchMinLen, posState); - pos -= Base.kNumRepDistances; + _lenEncoder.Encode(_rangeEncoder, len - Base.K_MATCH_MIN_LEN, posState); + pos -= Base.K_NUM_REP_DISTANCES; UInt32 posSlot = GetPosSlot(pos); UInt32 lenToPosState = Base.GetLenToPosState(len); _posSlotEncoder[lenToPosState].Encode(_rangeEncoder, posSlot); - if (posSlot >= Base.kStartPosModelIndex) + if (posSlot >= Base.K_START_POS_MODEL_INDEX) { int footerBits = (int)((posSlot >> 1) - 1); UInt32 baseVal = ((2 | (posSlot & 1)) << footerBits); UInt32 posReduced = pos - baseVal; - if (posSlot < Base.kEndPosModelIndex) + if (posSlot < Base.K_END_POS_MODEL_INDEX) { BitTreeEncoder.ReverseEncode(_posEncoders, baseVal - posSlot - 1, _rangeEncoder, footerBits, @@ -1385,14 +1385,14 @@ namespace SharpCompress.Compressors.LZMA } else { - _rangeEncoder.EncodeDirectBits(posReduced >> Base.kNumAlignBits, - footerBits - Base.kNumAlignBits); - _posAlignEncoder.ReverseEncode(_rangeEncoder, posReduced & Base.kAlignMask); + _rangeEncoder.EncodeDirectBits(posReduced >> Base.K_NUM_ALIGN_BITS, + footerBits - Base.K_NUM_ALIGN_BITS); + _posAlignEncoder.ReverseEncode(_rangeEncoder, posReduced & Base.K_ALIGN_MASK); _alignPriceCount++; } } UInt32 distance = pos; - for (UInt32 i = Base.kNumRepDistances - 1; i >= 1; i--) + for (UInt32 i = Base.K_NUM_REP_DISTANCES - 1; i >= 1; i--) { _repDistances[i] = _repDistances[i - 1]; } @@ -1402,7 +1402,7 @@ namespace SharpCompress.Compressors.LZMA _previousByte = _matchFinder.GetIndexByte((Int32)(len - 1 - _additionalOffset)); } _additionalOffset -= len; - nowPos64 += len; + _nowPos64 += len; if (_additionalOffset == 0) { // if (!_fastMode) @@ -1410,11 +1410,11 @@ namespace SharpCompress.Compressors.LZMA { FillDistancesPrices(); } - if (_alignPriceCount >= Base.kAlignTableSize) + if (_alignPriceCount >= Base.K_ALIGN_TABLE_SIZE) { FillAlignPrices(); } - inSize = nowPos64; + inSize = _nowPos64; outSize = _rangeEncoder.GetProcessedSizeAdd(); if (_processingMode && _matchFinder.IsDataStarved) { @@ -1423,11 +1423,11 @@ namespace SharpCompress.Compressors.LZMA } if (_matchFinder.GetNumAvailableBytes() == 0) { - Flush((UInt32)nowPos64); + Flush((UInt32)_nowPos64); return; } - if (nowPos64 - progressPosValuePrev >= (1 << 12)) + if (_nowPos64 - progressPosValuePrev >= (1 << 12)) { _finished = false; finished = false; @@ -1437,12 +1437,12 @@ namespace SharpCompress.Compressors.LZMA } } - private void ReleaseMFStream() + private void ReleaseMfStream() { - if (_matchFinder != null && _needReleaseMFStream) + if (_matchFinder != null && _needReleaseMfStream) { _matchFinder.ReleaseStream(); - _needReleaseMFStream = false; + _needReleaseMfStream = false; } } @@ -1458,7 +1458,7 @@ namespace SharpCompress.Compressors.LZMA private void ReleaseStreams() { - ReleaseMFStream(); + ReleaseMfStream(); ReleaseOutStream(); } @@ -1478,18 +1478,18 @@ namespace SharpCompress.Compressors.LZMA FillAlignPrices(); } - _lenEncoder.SetTableSize(_numFastBytes + 1 - Base.kMatchMinLen); + _lenEncoder.SetTableSize(_numFastBytes + 1 - Base.K_MATCH_MIN_LEN); _lenEncoder.UpdateTables((UInt32)1 << _posStateBits); - _repMatchLenEncoder.SetTableSize(_numFastBytes + 1 - Base.kMatchMinLen); + _repMatchLenEncoder.SetTableSize(_numFastBytes + 1 - Base.K_MATCH_MIN_LEN); _repMatchLenEncoder.UpdateTables((UInt32)1 << _posStateBits); - nowPos64 = 0; + _nowPos64 = 0; } public void Code(Stream inStream, Stream outStream, Int64 inSize, Int64 outSize, ICodeProgress progress) { - _needReleaseMFStream = false; + _needReleaseMfStream = false; _processingMode = false; try { @@ -1546,7 +1546,7 @@ namespace SharpCompress.Compressors.LZMA public void Train(Stream trainStream) { - if (nowPos64 > 0) + if (_nowPos64 > 0) { throw new InvalidOperationException(); } @@ -1566,58 +1566,58 @@ namespace SharpCompress.Compressors.LZMA } } - private const int kPropSize = 5; - private readonly Byte[] properties = new Byte[kPropSize]; + private const int K_PROP_SIZE = 5; + private readonly Byte[] _properties = new Byte[K_PROP_SIZE]; public void WriteCoderProperties(Stream outStream) { - properties[0] = (Byte)((_posStateBits * 5 + _numLiteralPosStateBits) * 9 + _numLiteralContextBits); + _properties[0] = (Byte)((_posStateBits * 5 + _numLiteralPosStateBits) * 9 + _numLiteralContextBits); for (int i = 0; i < 4; i++) { - properties[1 + i] = (Byte)((_dictionarySize >> (8 * i)) & 0xFF); + _properties[1 + i] = (Byte)((_dictionarySize >> (8 * i)) & 0xFF); } - outStream.Write(properties, 0, kPropSize); + outStream.Write(_properties, 0, K_PROP_SIZE); } - private readonly UInt32[] tempPrices = new UInt32[Base.kNumFullDistances]; + private readonly UInt32[] _tempPrices = new UInt32[Base.K_NUM_FULL_DISTANCES]; private UInt32 _matchPriceCount; private void FillDistancesPrices() { - for (UInt32 i = Base.kStartPosModelIndex; i < Base.kNumFullDistances; i++) + for (UInt32 i = Base.K_START_POS_MODEL_INDEX; i < Base.K_NUM_FULL_DISTANCES; i++) { UInt32 posSlot = GetPosSlot(i); int footerBits = (int)((posSlot >> 1) - 1); UInt32 baseVal = ((2 | (posSlot & 1)) << footerBits); - tempPrices[i] = BitTreeEncoder.ReverseGetPrice(_posEncoders, + _tempPrices[i] = BitTreeEncoder.ReverseGetPrice(_posEncoders, baseVal - posSlot - 1, footerBits, i - baseVal); } - for (UInt32 lenToPosState = 0; lenToPosState < Base.kNumLenToPosStates; lenToPosState++) + for (UInt32 lenToPosState = 0; lenToPosState < Base.K_NUM_LEN_TO_POS_STATES; lenToPosState++) { UInt32 posSlot; BitTreeEncoder encoder = _posSlotEncoder[lenToPosState]; - UInt32 st = (lenToPosState << Base.kNumPosSlotBits); + UInt32 st = (lenToPosState << Base.K_NUM_POS_SLOT_BITS); for (posSlot = 0; posSlot < _distTableSize; posSlot++) { _posSlotPrices[st + posSlot] = encoder.GetPrice(posSlot); } - for (posSlot = Base.kEndPosModelIndex; posSlot < _distTableSize; posSlot++) + for (posSlot = Base.K_END_POS_MODEL_INDEX; posSlot < _distTableSize; posSlot++) { - _posSlotPrices[st + posSlot] += ((((posSlot >> 1) - 1) - Base.kNumAlignBits) << - BitEncoder.kNumBitPriceShiftBits); + _posSlotPrices[st + posSlot] += ((((posSlot >> 1) - 1) - Base.K_NUM_ALIGN_BITS) << + BitEncoder.K_NUM_BIT_PRICE_SHIFT_BITS); } - UInt32 st2 = lenToPosState * Base.kNumFullDistances; + UInt32 st2 = lenToPosState * Base.K_NUM_FULL_DISTANCES; UInt32 i; - for (i = 0; i < Base.kStartPosModelIndex; i++) + for (i = 0; i < Base.K_START_POS_MODEL_INDEX; i++) { _distancesPrices[st2 + i] = _posSlotPrices[st + i]; } - for (; i < Base.kNumFullDistances; i++) + for (; i < Base.K_NUM_FULL_DISTANCES; i++) { - _distancesPrices[st2 + i] = _posSlotPrices[st + GetPosSlot(i)] + tempPrices[i]; + _distancesPrices[st2 + i] = _posSlotPrices[st + GetPosSlot(i)] + _tempPrices[i]; } } _matchPriceCount = 0; @@ -1625,14 +1625,14 @@ namespace SharpCompress.Compressors.LZMA private void FillAlignPrices() { - for (UInt32 i = 0; i < Base.kAlignTableSize; i++) + for (UInt32 i = 0; i < Base.K_ALIGN_TABLE_SIZE; i++) { _alignPrices[i] = _posAlignEncoder.ReverseGetPrice(i); } _alignPriceCount = 0; } - private static readonly string[] kMatchFinderIDs = + private static readonly string[] K_MATCH_FINDER_I_DS = { "BT2", "BT4" @@ -1640,9 +1640,9 @@ namespace SharpCompress.Compressors.LZMA private static int FindMatchFinder(string s) { - for (int m = 0; m < kMatchFinderIDs.Length; m++) + for (int m = 0; m < K_MATCH_FINDER_I_DS.Length; m++) { - if (s == kMatchFinderIDs[m]) + if (s == K_MATCH_FINDER_I_DS[m]) { return m; } @@ -1650,28 +1650,28 @@ namespace SharpCompress.Compressors.LZMA return -1; } - public void SetCoderProperties(CoderPropID[] propIDs, object[] properties) + public void SetCoderProperties(CoderPropId[] propIDs, object[] properties) { for (UInt32 i = 0; i < properties.Length; i++) { object prop = properties[i]; switch (propIDs[i]) { - case CoderPropID.NumFastBytes: + case CoderPropId.NumFastBytes: { if (!(prop is Int32)) { throw new InvalidParamException(); } Int32 numFastBytes = (Int32)prop; - if (numFastBytes < 5 || numFastBytes > Base.kMatchMaxLen) + if (numFastBytes < 5 || numFastBytes > Base.K_MATCH_MAX_LEN) { throw new InvalidParamException(); } _numFastBytes = (UInt32)numFastBytes; break; } - case CoderPropID.Algorithm: + case CoderPropId.Algorithm: { /* if (!(prop is Int32)) @@ -1682,7 +1682,7 @@ namespace SharpCompress.Compressors.LZMA */ break; } - case CoderPropID.MatchFinder: + case CoderPropId.MatchFinder: { if (!(prop is String)) { @@ -1702,7 +1702,7 @@ namespace SharpCompress.Compressors.LZMA } break; } - case CoderPropID.DictionarySize: + case CoderPropId.DictionarySize: { const int kDicLogSizeMaxCompress = 30; if (!(prop is Int32)) @@ -1711,7 +1711,7 @@ namespace SharpCompress.Compressors.LZMA } ; Int32 dictionarySize = (Int32)prop; - if (dictionarySize < (UInt32)(1 << Base.kDicLogSizeMin) || + if (dictionarySize < (UInt32)(1 << Base.K_DIC_LOG_SIZE_MIN) || dictionarySize > (UInt32)(1 << kDicLogSizeMaxCompress)) { throw new InvalidParamException(); @@ -1728,14 +1728,14 @@ namespace SharpCompress.Compressors.LZMA _distTableSize = (UInt32)dicLogSize * 2; break; } - case CoderPropID.PosStateBits: + case CoderPropId.PosStateBits: { if (!(prop is Int32)) { throw new InvalidParamException(); } Int32 v = (Int32)prop; - if (v < 0 || v > (UInt32)Base.kNumPosStatesBitsEncodingMax) + if (v < 0 || v > (UInt32)Base.K_NUM_POS_STATES_BITS_ENCODING_MAX) { throw new InvalidParamException(); } @@ -1743,28 +1743,28 @@ namespace SharpCompress.Compressors.LZMA _posStateMask = (((UInt32)1) << _posStateBits) - 1; break; } - case CoderPropID.LitPosBits: + case CoderPropId.LitPosBits: { if (!(prop is Int32)) { throw new InvalidParamException(); } Int32 v = (Int32)prop; - if (v < 0 || v > Base.kNumLitPosStatesBitsEncodingMax) + if (v < 0 || v > Base.K_NUM_LIT_POS_STATES_BITS_ENCODING_MAX) { throw new InvalidParamException(); } _numLiteralPosStateBits = v; break; } - case CoderPropID.LitContextBits: + case CoderPropId.LitContextBits: { if (!(prop is Int32)) { throw new InvalidParamException(); } Int32 v = (Int32)prop; - if (v < 0 || v > Base.kNumLitContextBitsMax) + if (v < 0 || v > Base.K_NUM_LIT_CONTEXT_BITS_MAX) { throw new InvalidParamException(); } @@ -1772,7 +1772,7 @@ namespace SharpCompress.Compressors.LZMA _numLiteralContextBits = v; break; } - case CoderPropID.EndMarker: + case CoderPropId.EndMarker: { if (!(prop is Boolean)) { diff --git a/src/SharpCompress/Compressors/LZMA/LzmaEncoderProperties.cs b/src/SharpCompress/Compressors/LZMA/LzmaEncoderProperties.cs index dd49e9dd..de7fa6f1 100644 --- a/src/SharpCompress/Compressors/LZMA/LzmaEncoderProperties.cs +++ b/src/SharpCompress/Compressors/LZMA/LzmaEncoderProperties.cs @@ -2,8 +2,8 @@ { public class LzmaEncoderProperties { - internal CoderPropID[] propIDs; - internal object[] properties; + internal CoderPropId[] _propIDs; + internal object[] _properties; public LzmaEncoderProperties() : this(false) @@ -28,18 +28,18 @@ int algorithm = 2; string mf = "bt4"; - propIDs = new[] + _propIDs = new[] { - CoderPropID.DictionarySize, - CoderPropID.PosStateBits, - CoderPropID.LitContextBits, - CoderPropID.LitPosBits, - CoderPropID.Algorithm, - CoderPropID.NumFastBytes, - CoderPropID.MatchFinder, - CoderPropID.EndMarker + CoderPropId.DictionarySize, + CoderPropId.PosStateBits, + CoderPropId.LitContextBits, + CoderPropId.LitPosBits, + CoderPropId.Algorithm, + CoderPropId.NumFastBytes, + CoderPropId.MatchFinder, + CoderPropId.EndMarker }; - properties = new object[] + _properties = new object[] { dictionary, posStateBits, diff --git a/src/SharpCompress/Compressors/LZMA/LzmaStream.cs b/src/SharpCompress/Compressors/LZMA/LzmaStream.cs index 616cc211..c074704f 100644 --- a/src/SharpCompress/Compressors/LZMA/LzmaStream.cs +++ b/src/SharpCompress/Compressors/LZMA/LzmaStream.cs @@ -7,29 +7,29 @@ namespace SharpCompress.Compressors.LZMA { public class LzmaStream : Stream { - private readonly Stream inputStream; - private readonly long inputSize; - private readonly long outputSize; + private readonly Stream _inputStream; + private readonly long _inputSize; + private readonly long _outputSize; - private readonly int dictionarySize; - private readonly OutWindow outWindow = new OutWindow(); - private readonly RangeCoder.Decoder rangeDecoder = new RangeCoder.Decoder(); - private Decoder decoder; + private readonly int _dictionarySize; + private readonly OutWindow _outWindow = new OutWindow(); + private readonly RangeCoder.Decoder _rangeDecoder = new RangeCoder.Decoder(); + private Decoder _decoder; - private long position; - private bool endReached; - private long availableBytes; - private long rangeDecoderLimit; - private long inputPosition; + private long _position; + private bool _endReached; + private long _availableBytes; + private long _rangeDecoderLimit; + private long _inputPosition; // LZMA2 - private readonly bool isLZMA2; - private bool uncompressedChunk; - private bool needDictReset = true; - private bool needProps = true; + private readonly bool _isLzma2; + private bool _uncompressedChunk; + private bool _needDictReset = true; + private bool _needProps = true; - private readonly Encoder encoder; - private bool isDisposed; + private readonly Encoder _encoder; + private bool _isDisposed; public LzmaStream(byte[] properties, Stream inputStream) : this(properties, inputStream, -1, -1, null, properties.Length < 5) @@ -47,82 +47,82 @@ namespace SharpCompress.Compressors.LZMA } public LzmaStream(byte[] properties, Stream inputStream, long inputSize, long outputSize, - Stream presetDictionary, bool isLZMA2) + Stream presetDictionary, bool isLzma2) { - this.inputStream = inputStream; - this.inputSize = inputSize; - this.outputSize = outputSize; - this.isLZMA2 = isLZMA2; + _inputStream = inputStream; + _inputSize = inputSize; + _outputSize = outputSize; + _isLzma2 = isLzma2; - if (!isLZMA2) + if (!isLzma2) { - dictionarySize = DataConverter.LittleEndian.GetInt32(properties, 1); - outWindow.Create(dictionarySize); + _dictionarySize = DataConverter.LittleEndian.GetInt32(properties, 1); + _outWindow.Create(_dictionarySize); if (presetDictionary != null) { - outWindow.Train(presetDictionary); + _outWindow.Train(presetDictionary); } - rangeDecoder.Init(inputStream); + _rangeDecoder.Init(inputStream); - decoder = new Decoder(); - decoder.SetDecoderProperties(properties); + _decoder = new Decoder(); + _decoder.SetDecoderProperties(properties); Properties = properties; - availableBytes = outputSize < 0 ? long.MaxValue : outputSize; - rangeDecoderLimit = inputSize; + _availableBytes = outputSize < 0 ? long.MaxValue : outputSize; + _rangeDecoderLimit = inputSize; } else { - dictionarySize = 2 | (properties[0] & 1); - dictionarySize <<= (properties[0] >> 1) + 11; + _dictionarySize = 2 | (properties[0] & 1); + _dictionarySize <<= (properties[0] >> 1) + 11; - outWindow.Create(dictionarySize); + _outWindow.Create(_dictionarySize); if (presetDictionary != null) { - outWindow.Train(presetDictionary); - needDictReset = false; + _outWindow.Train(presetDictionary); + _needDictReset = false; } Properties = new byte[1]; - availableBytes = 0; + _availableBytes = 0; } } - public LzmaStream(LzmaEncoderProperties properties, bool isLZMA2, Stream outputStream) - : this(properties, isLZMA2, null, outputStream) + public LzmaStream(LzmaEncoderProperties properties, bool isLzma2, Stream outputStream) + : this(properties, isLzma2, null, outputStream) { } - public LzmaStream(LzmaEncoderProperties properties, bool isLZMA2, Stream presetDictionary, Stream outputStream) + public LzmaStream(LzmaEncoderProperties properties, bool isLzma2, Stream presetDictionary, Stream outputStream) { - this.isLZMA2 = isLZMA2; - availableBytes = 0; - endReached = true; + _isLzma2 = isLzma2; + _availableBytes = 0; + _endReached = true; - if (isLZMA2) + if (isLzma2) { throw new NotImplementedException(); } - encoder = new Encoder(); - encoder.SetCoderProperties(properties.propIDs, properties.properties); + _encoder = new Encoder(); + _encoder.SetCoderProperties(properties._propIDs, properties._properties); MemoryStream propStream = new MemoryStream(5); - encoder.WriteCoderProperties(propStream); + _encoder.WriteCoderProperties(propStream); Properties = propStream.ToArray(); - encoder.SetStreams(null, outputStream, -1, -1); + _encoder.SetStreams(null, outputStream, -1, -1); if (presetDictionary != null) { - encoder.Train(presetDictionary); + _encoder.Train(presetDictionary); } } - public override bool CanRead => encoder == null; + public override bool CanRead => _encoder == null; public override bool CanSeek => false; - public override bool CanWrite => encoder != null; + public override bool CanWrite => _encoder != null; public override void Flush() { @@ -130,29 +130,29 @@ namespace SharpCompress.Compressors.LZMA protected override void Dispose(bool disposing) { - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; + _isDisposed = true; if (disposing) { - if (encoder != null) + if (_encoder != null) { - position = encoder.Code(null, true); + _position = _encoder.Code(null, true); } - inputStream?.Dispose(); + _inputStream?.Dispose(); } base.Dispose(disposing); } - public override long Length => position + availableBytes; + public override long Length => _position + _availableBytes; - public override long Position { get => position; set => throw new NotSupportedException(); } + public override long Position { get => _position; set => throw new NotSupportedException(); } public override int Read(byte[] buffer, int offset, int count) { - if (endReached) + if (_endReached) { return 0; } @@ -160,67 +160,67 @@ namespace SharpCompress.Compressors.LZMA int total = 0; while (total < count) { - if (availableBytes == 0) + if (_availableBytes == 0) { - if (isLZMA2) + if (_isLzma2) { - decodeChunkHeader(); + DecodeChunkHeader(); } else { - endReached = true; + _endReached = true; } - if (endReached) + if (_endReached) { break; } } int toProcess = count - total; - if (toProcess > availableBytes) + if (toProcess > _availableBytes) { - toProcess = (int)availableBytes; + toProcess = (int)_availableBytes; } - outWindow.SetLimit(toProcess); - if (uncompressedChunk) + _outWindow.SetLimit(toProcess); + if (_uncompressedChunk) { - inputPosition += outWindow.CopyStream(inputStream, toProcess); + _inputPosition += _outWindow.CopyStream(_inputStream, toProcess); } - else if (decoder.Code(dictionarySize, outWindow, rangeDecoder) - && outputSize < 0) + else if (_decoder.Code(_dictionarySize, _outWindow, _rangeDecoder) + && _outputSize < 0) { - availableBytes = outWindow.AvailableBytes; + _availableBytes = _outWindow.AvailableBytes; } - int read = outWindow.Read(buffer, offset, toProcess); + int read = _outWindow.Read(buffer, offset, toProcess); total += read; offset += read; - position += read; - availableBytes -= read; + _position += read; + _availableBytes -= read; - if (availableBytes == 0 && !uncompressedChunk) + if (_availableBytes == 0 && !_uncompressedChunk) { - rangeDecoder.ReleaseStream(); - if (!rangeDecoder.IsFinished || (rangeDecoderLimit >= 0 && rangeDecoder.Total != rangeDecoderLimit)) + _rangeDecoder.ReleaseStream(); + if (!_rangeDecoder.IsFinished || (_rangeDecoderLimit >= 0 && _rangeDecoder._total != _rangeDecoderLimit)) { throw new DataErrorException(); } - inputPosition += rangeDecoder.Total; - if (outWindow.HasPending) + _inputPosition += _rangeDecoder._total; + if (_outWindow.HasPending) { throw new DataErrorException(); } } } - if (endReached) + if (_endReached) { - if (inputSize >= 0 && inputPosition != inputSize) + if (_inputSize >= 0 && _inputPosition != _inputSize) { throw new DataErrorException(); } - if (outputSize >= 0 && position != outputSize) + if (_outputSize >= 0 && _position != _outputSize) { throw new DataErrorException(); } @@ -229,59 +229,59 @@ namespace SharpCompress.Compressors.LZMA return total; } - private void decodeChunkHeader() + private void DecodeChunkHeader() { - int control = inputStream.ReadByte(); - inputPosition++; + int control = _inputStream.ReadByte(); + _inputPosition++; if (control == 0x00) { - endReached = true; + _endReached = true; return; } if (control >= 0xE0 || control == 0x01) { - needProps = true; - needDictReset = false; - outWindow.Reset(); + _needProps = true; + _needDictReset = false; + _outWindow.Reset(); } - else if (needDictReset) + else if (_needDictReset) { throw new DataErrorException(); } if (control >= 0x80) { - uncompressedChunk = false; + _uncompressedChunk = false; - availableBytes = (control & 0x1F) << 16; - availableBytes += (inputStream.ReadByte() << 8) + inputStream.ReadByte() + 1; - inputPosition += 2; + _availableBytes = (control & 0x1F) << 16; + _availableBytes += (_inputStream.ReadByte() << 8) + _inputStream.ReadByte() + 1; + _inputPosition += 2; - rangeDecoderLimit = (inputStream.ReadByte() << 8) + inputStream.ReadByte() + 1; - inputPosition += 2; + _rangeDecoderLimit = (_inputStream.ReadByte() << 8) + _inputStream.ReadByte() + 1; + _inputPosition += 2; if (control >= 0xC0) { - needProps = false; - Properties[0] = (byte)inputStream.ReadByte(); - inputPosition++; + _needProps = false; + Properties[0] = (byte)_inputStream.ReadByte(); + _inputPosition++; - decoder = new Decoder(); - decoder.SetDecoderProperties(Properties); + _decoder = new Decoder(); + _decoder.SetDecoderProperties(Properties); } - else if (needProps) + else if (_needProps) { throw new DataErrorException(); } else if (control >= 0xA0) { - decoder = new Decoder(); - decoder.SetDecoderProperties(Properties); + _decoder = new Decoder(); + _decoder.SetDecoderProperties(Properties); } - rangeDecoder.Init(inputStream); + _rangeDecoder.Init(_inputStream); } else if (control > 0x02) { @@ -289,9 +289,9 @@ namespace SharpCompress.Compressors.LZMA } else { - uncompressedChunk = true; - availableBytes = (inputStream.ReadByte() << 8) + inputStream.ReadByte() + 1; - inputPosition += 2; + _uncompressedChunk = true; + _availableBytes = (_inputStream.ReadByte() << 8) + _inputStream.ReadByte() + 1; + _inputPosition += 2; } } @@ -307,9 +307,9 @@ namespace SharpCompress.Compressors.LZMA public override void Write(byte[] buffer, int offset, int count) { - if (encoder != null) + if (_encoder != null) { - position = encoder.Code(new MemoryStream(buffer, offset, count), false); + _position = _encoder.Code(new MemoryStream(buffer, offset, count), false); } } diff --git a/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoder.cs b/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoder.cs index 6b6ab41e..f4ee15b3 100644 --- a/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoder.cs +++ b/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoder.cs @@ -5,12 +5,12 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder { internal class Encoder { - public const uint kTopValue = (1 << 24); + public const uint K_TOP_VALUE = (1 << 24); - private Stream Stream; + private Stream _stream; - public UInt64 Low; - public uint Range; + public UInt64 _low; + public uint _range; private uint _cacheSize; private byte _cache; @@ -18,20 +18,20 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder public void SetStream(Stream stream) { - Stream = stream; + _stream = stream; } public void ReleaseStream() { - Stream = null; + _stream = null; } public void Init() { //StartPosition = Stream.Position; - Low = 0; - Range = 0xFFFFFFFF; + _low = 0; + _range = 0xFFFFFFFF; _cacheSize = 1; _cache = 0; } @@ -46,54 +46,54 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder public void FlushStream() { - Stream.Flush(); + _stream.Flush(); } public void CloseStream() { - Stream.Dispose(); + _stream.Dispose(); } public void Encode(uint start, uint size, uint total) { - Low += start * (Range /= total); - Range *= size; - while (Range < kTopValue) + _low += start * (_range /= total); + _range *= size; + while (_range < K_TOP_VALUE) { - Range <<= 8; + _range <<= 8; ShiftLow(); } } public void ShiftLow() { - if ((uint)Low < 0xFF000000 || (uint)(Low >> 32) == 1) + if ((uint)_low < 0xFF000000 || (uint)(_low >> 32) == 1) { byte temp = _cache; do { - Stream.WriteByte((byte)(temp + (Low >> 32))); + _stream.WriteByte((byte)(temp + (_low >> 32))); temp = 0xFF; } while (--_cacheSize != 0); - _cache = (byte)(((uint)Low) >> 24); + _cache = (byte)(((uint)_low) >> 24); } _cacheSize++; - Low = ((uint)Low) << 8; + _low = ((uint)_low) << 8; } public void EncodeDirectBits(uint v, int numTotalBits) { for (int i = numTotalBits - 1; i >= 0; i--) { - Range >>= 1; + _range >>= 1; if (((v >> i) & 1) == 1) { - Low += Range; + _low += _range; } - if (Range < kTopValue) + if (_range < K_TOP_VALUE) { - Range <<= 8; + _range <<= 8; ShiftLow(); } } @@ -101,19 +101,19 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder public void EncodeBit(uint size0, int numTotalBits, uint symbol) { - uint newBound = (Range >> numTotalBits) * size0; + uint newBound = (_range >> numTotalBits) * size0; if (symbol == 0) { - Range = newBound; + _range = newBound; } else { - Low += newBound; - Range -= newBound; + _low += newBound; + _range -= newBound; } - while (Range < kTopValue) + while (_range < K_TOP_VALUE) { - Range <<= 8; + _range <<= 8; ShiftLow(); } } @@ -129,75 +129,75 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder internal class Decoder { - public const uint kTopValue = (1 << 24); - public uint Range; - public uint Code; + public const uint K_TOP_VALUE = (1 << 24); + public uint _range; + public uint _code; // public Buffer.InBuffer Stream = new Buffer.InBuffer(1 << 16); - public Stream Stream; - public long Total; + public Stream _stream; + public long _total; public void Init(Stream stream) { // Stream.Init(stream); - Stream = stream; + _stream = stream; - Code = 0; - Range = 0xFFFFFFFF; + _code = 0; + _range = 0xFFFFFFFF; for (int i = 0; i < 5; i++) { - Code = (Code << 8) | (byte)Stream.ReadByte(); + _code = (_code << 8) | (byte)_stream.ReadByte(); } - Total = 5; + _total = 5; } public void ReleaseStream() { // Stream.ReleaseStream(); - Stream = null; + _stream = null; } public void CloseStream() { - Stream.Dispose(); + _stream.Dispose(); } public void Normalize() { - while (Range < kTopValue) + while (_range < K_TOP_VALUE) { - Code = (Code << 8) | (byte)Stream.ReadByte(); - Range <<= 8; - Total++; + _code = (_code << 8) | (byte)_stream.ReadByte(); + _range <<= 8; + _total++; } } public void Normalize2() { - if (Range < kTopValue) + if (_range < K_TOP_VALUE) { - Code = (Code << 8) | (byte)Stream.ReadByte(); - Range <<= 8; - Total++; + _code = (_code << 8) | (byte)_stream.ReadByte(); + _range <<= 8; + _total++; } } public uint GetThreshold(uint total) { - return Code / (Range /= total); + return _code / (_range /= total); } public void Decode(uint start, uint size) { - Code -= start * Range; - Range *= size; + _code -= start * _range; + _range *= size; Normalize(); } public uint DecodeDirectBits(int numTotalBits) { - uint range = Range; - uint code = Code; + uint range = _range; + uint code = _code; uint result = 0; for (int i = numTotalBits; i > 0; i--) { @@ -214,38 +214,38 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder code -= range & (t - 1); result = (result << 1) | (1 - t); - if (range < kTopValue) + if (range < K_TOP_VALUE) { - code = (code << 8) | (byte)Stream.ReadByte(); + code = (code << 8) | (byte)_stream.ReadByte(); range <<= 8; - Total++; + _total++; } } - Range = range; - Code = code; + _range = range; + _code = code; return result; } public uint DecodeBit(uint size0, int numTotalBits) { - uint newBound = (Range >> numTotalBits) * size0; + uint newBound = (_range >> numTotalBits) * size0; uint symbol; - if (Code < newBound) + if (_code < newBound) { symbol = 0; - Range = newBound; + _range = newBound; } else { symbol = 1; - Code -= newBound; - Range -= newBound; + _code -= newBound; + _range -= newBound; } Normalize(); return symbol; } - public bool IsFinished => Code == 0; + public bool IsFinished => _code == 0; // ulong GetProcessedSize() {return Stream.GetProcessedSize(); } } diff --git a/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoderBit.cs b/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoderBit.cs index cfb312a2..2ab53aff 100644 --- a/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoderBit.cs +++ b/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoderBit.cs @@ -4,28 +4,28 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder { internal struct BitEncoder { - public const int kNumBitModelTotalBits = 11; - public const uint kBitModelTotal = (1 << kNumBitModelTotalBits); - private const int kNumMoveBits = 5; - private const int kNumMoveReducingBits = 2; - public const int kNumBitPriceShiftBits = 6; + public const int K_NUM_BIT_MODEL_TOTAL_BITS = 11; + public const uint K_BIT_MODEL_TOTAL = (1 << K_NUM_BIT_MODEL_TOTAL_BITS); + private const int K_NUM_MOVE_BITS = 5; + private const int K_NUM_MOVE_REDUCING_BITS = 2; + public const int K_NUM_BIT_PRICE_SHIFT_BITS = 6; - private uint Prob; + private uint _prob; public void Init() { - Prob = kBitModelTotal >> 1; + _prob = K_BIT_MODEL_TOTAL >> 1; } public void UpdateModel(uint symbol) { if (symbol == 0) { - Prob += (kBitModelTotal - Prob) >> kNumMoveBits; + _prob += (K_BIT_MODEL_TOTAL - _prob) >> K_NUM_MOVE_BITS; } else { - Prob -= (Prob) >> kNumMoveBits; + _prob -= (_prob) >> K_NUM_MOVE_BITS; } } @@ -33,106 +33,106 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder { // encoder.EncodeBit(Prob, kNumBitModelTotalBits, symbol); // UpdateModel(symbol); - uint newBound = (encoder.Range >> kNumBitModelTotalBits) * Prob; + uint newBound = (encoder._range >> K_NUM_BIT_MODEL_TOTAL_BITS) * _prob; if (symbol == 0) { - encoder.Range = newBound; - Prob += (kBitModelTotal - Prob) >> kNumMoveBits; + encoder._range = newBound; + _prob += (K_BIT_MODEL_TOTAL - _prob) >> K_NUM_MOVE_BITS; } else { - encoder.Low += newBound; - encoder.Range -= newBound; - Prob -= (Prob) >> kNumMoveBits; + encoder._low += newBound; + encoder._range -= newBound; + _prob -= (_prob) >> K_NUM_MOVE_BITS; } - if (encoder.Range < Encoder.kTopValue) + if (encoder._range < Encoder.K_TOP_VALUE) { - encoder.Range <<= 8; + encoder._range <<= 8; encoder.ShiftLow(); } } - private static readonly UInt32[] ProbPrices = new UInt32[kBitModelTotal >> kNumMoveReducingBits]; + private static readonly UInt32[] PROB_PRICES = new UInt32[K_BIT_MODEL_TOTAL >> K_NUM_MOVE_REDUCING_BITS]; static BitEncoder() { - const int kNumBits = (kNumBitModelTotalBits - kNumMoveReducingBits); + const int kNumBits = (K_NUM_BIT_MODEL_TOTAL_BITS - K_NUM_MOVE_REDUCING_BITS); for (int i = kNumBits - 1; i >= 0; i--) { UInt32 start = (UInt32)1 << (kNumBits - i - 1); UInt32 end = (UInt32)1 << (kNumBits - i); for (UInt32 j = start; j < end; j++) { - ProbPrices[j] = ((UInt32)i << kNumBitPriceShiftBits) + - (((end - j) << kNumBitPriceShiftBits) >> (kNumBits - i - 1)); + PROB_PRICES[j] = ((UInt32)i << K_NUM_BIT_PRICE_SHIFT_BITS) + + (((end - j) << K_NUM_BIT_PRICE_SHIFT_BITS) >> (kNumBits - i - 1)); } } } public uint GetPrice(uint symbol) { - return ProbPrices[(((Prob - symbol) ^ ((-(int)symbol))) & (kBitModelTotal - 1)) >> kNumMoveReducingBits]; + return PROB_PRICES[(((_prob - symbol) ^ ((-(int)symbol))) & (K_BIT_MODEL_TOTAL - 1)) >> K_NUM_MOVE_REDUCING_BITS]; } public uint GetPrice0() { - return ProbPrices[Prob >> kNumMoveReducingBits]; + return PROB_PRICES[_prob >> K_NUM_MOVE_REDUCING_BITS]; } public uint GetPrice1() { - return ProbPrices[(kBitModelTotal - Prob) >> kNumMoveReducingBits]; + return PROB_PRICES[(K_BIT_MODEL_TOTAL - _prob) >> K_NUM_MOVE_REDUCING_BITS]; } } internal struct BitDecoder { - public const int kNumBitModelTotalBits = 11; - public const uint kBitModelTotal = (1 << kNumBitModelTotalBits); - private const int kNumMoveBits = 5; + public const int K_NUM_BIT_MODEL_TOTAL_BITS = 11; + public const uint K_BIT_MODEL_TOTAL = (1 << K_NUM_BIT_MODEL_TOTAL_BITS); + private const int K_NUM_MOVE_BITS = 5; - private uint Prob; + private uint _prob; public void UpdateModel(int numMoveBits, uint symbol) { if (symbol == 0) { - Prob += (kBitModelTotal - Prob) >> numMoveBits; + _prob += (K_BIT_MODEL_TOTAL - _prob) >> numMoveBits; } else { - Prob -= (Prob) >> numMoveBits; + _prob -= (_prob) >> numMoveBits; } } public void Init() { - Prob = kBitModelTotal >> 1; + _prob = K_BIT_MODEL_TOTAL >> 1; } public uint Decode(Decoder rangeDecoder) { - uint newBound = (rangeDecoder.Range >> kNumBitModelTotalBits) * Prob; - if (rangeDecoder.Code < newBound) + uint newBound = (rangeDecoder._range >> K_NUM_BIT_MODEL_TOTAL_BITS) * _prob; + if (rangeDecoder._code < newBound) { - rangeDecoder.Range = newBound; - Prob += (kBitModelTotal - Prob) >> kNumMoveBits; - if (rangeDecoder.Range < Decoder.kTopValue) + rangeDecoder._range = newBound; + _prob += (K_BIT_MODEL_TOTAL - _prob) >> K_NUM_MOVE_BITS; + if (rangeDecoder._range < Decoder.K_TOP_VALUE) { - rangeDecoder.Code = (rangeDecoder.Code << 8) | (byte)rangeDecoder.Stream.ReadByte(); - rangeDecoder.Range <<= 8; - rangeDecoder.Total++; + rangeDecoder._code = (rangeDecoder._code << 8) | (byte)rangeDecoder._stream.ReadByte(); + rangeDecoder._range <<= 8; + rangeDecoder._total++; } return 0; } - rangeDecoder.Range -= newBound; - rangeDecoder.Code -= newBound; - Prob -= (Prob) >> kNumMoveBits; - if (rangeDecoder.Range < Decoder.kTopValue) + rangeDecoder._range -= newBound; + rangeDecoder._code -= newBound; + _prob -= (_prob) >> K_NUM_MOVE_BITS; + if (rangeDecoder._range < Decoder.K_TOP_VALUE) { - rangeDecoder.Code = (rangeDecoder.Code << 8) | (byte)rangeDecoder.Stream.ReadByte(); - rangeDecoder.Range <<= 8; - rangeDecoder.Total++; + rangeDecoder._code = (rangeDecoder._code << 8) | (byte)rangeDecoder._stream.ReadByte(); + rangeDecoder._range <<= 8; + rangeDecoder._total++; } return 1; } diff --git a/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoderBitTree.cs b/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoderBitTree.cs index 7196129f..cc511603 100644 --- a/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoderBitTree.cs +++ b/src/SharpCompress/Compressors/LZMA/RangeCoder/RangeCoderBitTree.cs @@ -4,31 +4,31 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder { internal struct BitTreeEncoder { - private readonly BitEncoder[] Models; - private readonly int NumBitLevels; + private readonly BitEncoder[] _models; + private readonly int _numBitLevels; public BitTreeEncoder(int numBitLevels) { - NumBitLevels = numBitLevels; - Models = new BitEncoder[1 << numBitLevels]; + _numBitLevels = numBitLevels; + _models = new BitEncoder[1 << numBitLevels]; } public void Init() { - for (uint i = 1; i < (1 << NumBitLevels); i++) + for (uint i = 1; i < (1 << _numBitLevels); i++) { - Models[i].Init(); + _models[i].Init(); } } public void Encode(Encoder rangeEncoder, UInt32 symbol) { UInt32 m = 1; - for (int bitIndex = NumBitLevels; bitIndex > 0;) + for (int bitIndex = _numBitLevels; bitIndex > 0;) { bitIndex--; UInt32 bit = (symbol >> bitIndex) & 1; - Models[m].Encode(rangeEncoder, bit); + _models[m].Encode(rangeEncoder, bit); m = (m << 1) | bit; } } @@ -36,10 +36,10 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder public void ReverseEncode(Encoder rangeEncoder, UInt32 symbol) { UInt32 m = 1; - for (UInt32 i = 0; i < NumBitLevels; i++) + for (UInt32 i = 0; i < _numBitLevels; i++) { UInt32 bit = symbol & 1; - Models[m].Encode(rangeEncoder, bit); + _models[m].Encode(rangeEncoder, bit); m = (m << 1) | bit; symbol >>= 1; } @@ -49,11 +49,11 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder { UInt32 price = 0; UInt32 m = 1; - for (int bitIndex = NumBitLevels; bitIndex > 0;) + for (int bitIndex = _numBitLevels; bitIndex > 0;) { bitIndex--; UInt32 bit = (symbol >> bitIndex) & 1; - price += Models[m].GetPrice(bit); + price += _models[m].GetPrice(bit); m = (m << 1) + bit; } return price; @@ -63,39 +63,39 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder { UInt32 price = 0; UInt32 m = 1; - for (int i = NumBitLevels; i > 0; i--) + for (int i = _numBitLevels; i > 0; i--) { UInt32 bit = symbol & 1; symbol >>= 1; - price += Models[m].GetPrice(bit); + price += _models[m].GetPrice(bit); m = (m << 1) | bit; } return price; } - public static UInt32 ReverseGetPrice(BitEncoder[] Models, UInt32 startIndex, - int NumBitLevels, UInt32 symbol) + public static UInt32 ReverseGetPrice(BitEncoder[] models, UInt32 startIndex, + int numBitLevels, UInt32 symbol) { UInt32 price = 0; UInt32 m = 1; - for (int i = NumBitLevels; i > 0; i--) + for (int i = numBitLevels; i > 0; i--) { UInt32 bit = symbol & 1; symbol >>= 1; - price += Models[startIndex + m].GetPrice(bit); + price += models[startIndex + m].GetPrice(bit); m = (m << 1) | bit; } return price; } - public static void ReverseEncode(BitEncoder[] Models, UInt32 startIndex, - Encoder rangeEncoder, int NumBitLevels, UInt32 symbol) + public static void ReverseEncode(BitEncoder[] models, UInt32 startIndex, + Encoder rangeEncoder, int numBitLevels, UInt32 symbol) { UInt32 m = 1; - for (int i = 0; i < NumBitLevels; i++) + for (int i = 0; i < numBitLevels; i++) { UInt32 bit = symbol & 1; - Models[startIndex + m].Encode(rangeEncoder, bit); + models[startIndex + m].Encode(rangeEncoder, bit); m = (m << 1) | bit; symbol >>= 1; } @@ -104,40 +104,40 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder internal struct BitTreeDecoder { - private readonly BitDecoder[] Models; - private readonly int NumBitLevels; + private readonly BitDecoder[] _models; + private readonly int _numBitLevels; public BitTreeDecoder(int numBitLevels) { - NumBitLevels = numBitLevels; - Models = new BitDecoder[1 << numBitLevels]; + _numBitLevels = numBitLevels; + _models = new BitDecoder[1 << numBitLevels]; } public void Init() { - for (uint i = 1; i < (1 << NumBitLevels); i++) + for (uint i = 1; i < (1 << _numBitLevels); i++) { - Models[i].Init(); + _models[i].Init(); } } public uint Decode(Decoder rangeDecoder) { uint m = 1; - for (int bitIndex = NumBitLevels; bitIndex > 0; bitIndex--) + for (int bitIndex = _numBitLevels; bitIndex > 0; bitIndex--) { - m = (m << 1) + Models[m].Decode(rangeDecoder); + m = (m << 1) + _models[m].Decode(rangeDecoder); } - return m - ((uint)1 << NumBitLevels); + return m - ((uint)1 << _numBitLevels); } public uint ReverseDecode(Decoder rangeDecoder) { uint m = 1; uint symbol = 0; - for (int bitIndex = 0; bitIndex < NumBitLevels; bitIndex++) + for (int bitIndex = 0; bitIndex < _numBitLevels; bitIndex++) { - uint bit = Models[m].Decode(rangeDecoder); + uint bit = _models[m].Decode(rangeDecoder); m <<= 1; m += bit; symbol |= (bit << bitIndex); @@ -145,14 +145,14 @@ namespace SharpCompress.Compressors.LZMA.RangeCoder return symbol; } - public static uint ReverseDecode(BitDecoder[] Models, UInt32 startIndex, - Decoder rangeDecoder, int NumBitLevels) + public static uint ReverseDecode(BitDecoder[] models, UInt32 startIndex, + Decoder rangeDecoder, int numBitLevels) { uint m = 1; uint symbol = 0; - for (int bitIndex = 0; bitIndex < NumBitLevels; bitIndex++) + for (int bitIndex = 0; bitIndex < numBitLevels; bitIndex++) { - uint bit = Models[startIndex + m].Decode(rangeDecoder); + uint bit = models[startIndex + m].Decode(rangeDecoder); m <<= 1; m += bit; symbol |= (bit << bitIndex); diff --git a/src/SharpCompress/Compressors/LZMA/Registry.cs b/src/SharpCompress/Compressors/LZMA/Registry.cs index 6a42e70c..edc02854 100644 --- a/src/SharpCompress/Compressors/LZMA/Registry.cs +++ b/src/SharpCompress/Compressors/LZMA/Registry.cs @@ -12,43 +12,43 @@ namespace SharpCompress.Compressors.LZMA { internal static class DecoderRegistry { - private const uint k_Copy = 0x0; - private const uint k_Delta = 3; - private const uint k_LZMA2 = 0x21; - private const uint k_LZMA = 0x030101; - private const uint k_PPMD = 0x030401; - private const uint k_BCJ = 0x03030103; - private const uint k_BCJ2 = 0x0303011B; - private const uint k_Deflate = 0x040108; - private const uint k_BZip2 = 0x040202; + private const uint K_COPY = 0x0; + private const uint K_DELTA = 3; + private const uint K_LZMA2 = 0x21; + private const uint K_LZMA = 0x030101; + private const uint K_PPMD = 0x030401; + private const uint K_BCJ = 0x03030103; + private const uint K_BCJ2 = 0x0303011B; + private const uint K_DEFLATE = 0x040108; + private const uint K_B_ZIP2 = 0x040202; internal static Stream CreateDecoderStream(CMethodId id, Stream[] inStreams, byte[] info, IPasswordProvider pass, long limit) { - switch (id.Id) + switch (id._id) { - case k_Copy: + case K_COPY: if (info != null) { throw new NotSupportedException(); } return inStreams.Single(); - case k_LZMA: - case k_LZMA2: + case K_LZMA: + case K_LZMA2: return new LzmaStream(info, inStreams.Single(), -1, limit); #if !NO_CRYPTO - case CMethodId.kAESId: + case CMethodId.K_AES_ID: return new AesDecoderStream(inStreams.Single(), info, pass, limit); #endif - case k_BCJ: + case K_BCJ: return new BCJFilter(false, inStreams.Single()); - case k_BCJ2: + case K_BCJ2: return new Bcj2DecoderStream(inStreams, info, limit); - case k_BZip2: + case K_B_ZIP2: return new BZip2Stream(inStreams.Single(), CompressionMode.Decompress, true); - case k_PPMD: + case K_PPMD: return new PpmdStream(new PpmdProperties(info), inStreams.Single(), false); - case k_Deflate: + case K_DEFLATE: return new DeflateStream(inStreams.Single(), CompressionMode.Decompress); default: throw new NotSupportedException(); diff --git a/src/SharpCompress/Compressors/LZMA/Utilites/CrcBuilderStream.cs b/src/SharpCompress/Compressors/LZMA/Utilites/CrcBuilderStream.cs index 1da7a723..4410dd9a 100644 --- a/src/SharpCompress/Compressors/LZMA/Utilites/CrcBuilderStream.cs +++ b/src/SharpCompress/Compressors/LZMA/Utilites/CrcBuilderStream.cs @@ -5,25 +5,25 @@ namespace SharpCompress.Compressors.LZMA.Utilites { internal class CrcBuilderStream : Stream { - private readonly Stream mTarget; - private uint mCRC; - private bool mFinished; - private bool isDisposed; + private readonly Stream _mTarget; + private uint _mCrc; + private bool _mFinished; + private bool _isDisposed; public CrcBuilderStream(Stream target) { - mTarget = target; - mCRC = CRC.kInitCRC; + _mTarget = target; + _mCrc = Crc.INIT_CRC; } protected override void Dispose(bool disposing) { - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; - mTarget.Dispose(); + _isDisposed = true; + _mTarget.Dispose(); base.Dispose(disposing); } @@ -31,13 +31,13 @@ namespace SharpCompress.Compressors.LZMA.Utilites public uint Finish() { - if (!mFinished) + if (!_mFinished) { - mFinished = true; - mCRC = CRC.Finish(mCRC); + _mFinished = true; + _mCrc = Crc.Finish(_mCrc); } - return mCRC; + return _mCrc; } public override bool CanRead => false; @@ -71,103 +71,14 @@ namespace SharpCompress.Compressors.LZMA.Utilites public override void Write(byte[] buffer, int offset, int count) { - if (mFinished) + if (_mFinished) { throw new InvalidOperationException("CRC calculation has been finished."); } Processed += count; - mCRC = CRC.Update(mCRC, buffer, offset, count); - mTarget.Write(buffer, offset, count); - } - } - - internal class ReadingCrcBuilderStream : Stream - { - private readonly Stream mSource; - private uint mCRC; - private bool mFinished; - - public ReadingCrcBuilderStream(Stream source) - { - mSource = source; - mCRC = CRC.kInitCRC; - } - - protected override void Dispose(bool disposing) - { - try - { - if (disposing) - { - mSource.Dispose(); - } - } - finally - { - base.Dispose(disposing); - } - } - - public long Processed { get; private set; } - - public uint Finish() - { - if (!mFinished) - { - mFinished = true; - mCRC = CRC.Finish(mCRC); - } - - return mCRC; - } - - public override bool CanRead => mSource.CanRead; - - public override bool CanSeek => false; - - public override bool CanWrite => false; - - public override void Flush() - { - throw new NotSupportedException(); - } - - public override long Length => throw new NotSupportedException(); - - public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); } - - public override int Read(byte[] buffer, int offset, int count) - { - if (count > 0 && !mFinished) - { - int read = mSource.Read(buffer, offset, count); - if (read > 0) - { - Processed += read; - mCRC = CRC.Update(mCRC, buffer, offset, read); - return read; - } - - Finish(); - } - - return 0; - } - - public override long Seek(long offset, SeekOrigin origin) - { - throw new NotSupportedException(); - } - - public override void SetLength(long value) - { - throw new NotSupportedException(); - } - - public override void Write(byte[] buffer, int offset, int count) - { - throw new NotSupportedException(); + _mCrc = Crc.Update(_mCrc, buffer, offset, count); + _mTarget.Write(buffer, offset, count); } } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/LZMA/Utilites/CrcCheckStream.cs b/src/SharpCompress/Compressors/LZMA/Utilites/CrcCheckStream.cs index f64887e4..04b00f81 100644 --- a/src/SharpCompress/Compressors/LZMA/Utilites/CrcCheckStream.cs +++ b/src/SharpCompress/Compressors/LZMA/Utilites/CrcCheckStream.cs @@ -6,35 +6,35 @@ namespace SharpCompress.Compressors.LZMA.Utilites { internal class CrcCheckStream : Stream { - private readonly uint mExpectedCRC; - private uint mCurrentCRC; - private bool mClosed; + private readonly uint _mExpectedCrc; + private uint _mCurrentCrc; + private bool _mClosed; - private readonly long[] mBytes = new long[256]; - private long mLength; + private readonly long[] _mBytes = new long[256]; + private long _mLength; public CrcCheckStream(uint crc) { - mExpectedCRC = crc; - mCurrentCRC = CRC.kInitCRC; + _mExpectedCrc = crc; + _mCurrentCrc = Crc.INIT_CRC; } protected override void Dispose(bool disposing) { - if (mCurrentCRC != mExpectedCRC) + if (_mCurrentCrc != _mExpectedCrc) { throw new InvalidOperationException(); } try { - if (disposing && !mClosed) + if (disposing && !_mClosed) { - mClosed = true; - mCurrentCRC = CRC.Finish(mCurrentCRC); + _mClosed = true; + _mCurrentCrc = Crc.Finish(_mCurrentCrc); #if DEBUG - if (mCurrentCRC == mExpectedCRC) + if (_mCurrentCrc == _mExpectedCrc) { - Debug.WriteLine("CRC ok: " + mExpectedCRC.ToString("x8")); + Debug.WriteLine("CRC ok: " + _mExpectedCrc.ToString("x8")); } else { @@ -42,13 +42,13 @@ namespace SharpCompress.Compressors.LZMA.Utilites Debug.WriteLine("bad CRC"); } - double lengthInv = 1.0 / mLength; + double lengthInv = 1.0 / _mLength; double entropy = 0; for (int i = 0; i < 256; i++) { - if (mBytes[i] != 0) + if (_mBytes[i] != 0) { - double p = lengthInv * mBytes[i]; + double p = lengthInv * _mBytes[i]; entropy -= p * Math.Log(p, 256); } } @@ -93,13 +93,13 @@ namespace SharpCompress.Compressors.LZMA.Utilites public override void Write(byte[] buffer, int offset, int count) { - mLength += count; + _mLength += count; for (int i = 0; i < count; i++) { - mBytes[buffer[offset + i]]++; + _mBytes[buffer[offset + i]]++; } - mCurrentCRC = CRC.Update(mCurrentCRC, buffer, offset, count); + _mCurrentCrc = Crc.Update(_mCurrentCrc, buffer, offset, count); } } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/LZMA/Utilites/Utils.cs b/src/SharpCompress/Compressors/LZMA/Utilites/Utils.cs index ad0e54b0..9967e292 100644 --- a/src/SharpCompress/Compressors/LZMA/Utilites/Utils.cs +++ b/src/SharpCompress/Compressors/LZMA/Utilites/Utils.cs @@ -18,7 +18,7 @@ namespace SharpCompress.Compressors.LZMA.Utilites UnpackInfo = 7, SubStreamsInfo = 8, Size = 9, - CRC = 10, + Crc = 10, Folder = 11, CodersUnpackSize = 12, NumUnpackStream = 13, diff --git a/src/SharpCompress/Compressors/PPMd/H/FreqData.cs b/src/SharpCompress/Compressors/PPMd/H/FreqData.cs index 731ed972..1c01404b 100644 --- a/src/SharpCompress/Compressors/PPMd/H/FreqData.cs +++ b/src/SharpCompress/Compressors/PPMd/H/FreqData.cs @@ -6,7 +6,7 @@ namespace SharpCompress.Compressors.PPMd.H { internal class FreqData : Pointer { - internal const int Size = 6; + internal const int SIZE = 6; // struct FreqData // { @@ -14,8 +14,8 @@ namespace SharpCompress.Compressors.PPMd.H // STATE _PACK_ATTR * Stats; // }; - internal FreqData(byte[] Memory) - : base(Memory) + internal FreqData(byte[] memory) + : base(memory) { } @@ -55,7 +55,7 @@ namespace SharpCompress.Compressors.PPMd.H buffer.Append("\n Address="); buffer.Append(Address); buffer.Append("\n size="); - buffer.Append(Size); + buffer.Append(SIZE); buffer.Append("\n summFreq="); buffer.Append(SummFreq); buffer.Append("\n stats="); diff --git a/src/SharpCompress/Compressors/PPMd/H/ModelPPM.cs b/src/SharpCompress/Compressors/PPMd/H/ModelPPM.cs index 8a754333..883aa062 100644 --- a/src/SharpCompress/Compressors/PPMd/H/ModelPPM.cs +++ b/src/SharpCompress/Compressors/PPMd/H/ModelPPM.cs @@ -6,41 +6,41 @@ using Decoder = SharpCompress.Compressors.LZMA.RangeCoder.Decoder; namespace SharpCompress.Compressors.PPMd.H { - internal class ModelPPM + internal class ModelPpm { private void InitBlock() { for (int i = 0; i < 25; i++) { - SEE2Cont[i] = new SEE2Context[16]; + _see2Cont[i] = new See2Context[16]; } for (int i2 = 0; i2 < 128; i2++) { - binSumm[i2] = new int[64]; + _binSumm[i2] = new int[64]; } } public SubAllocator SubAlloc { get; } = new SubAllocator(); - public virtual SEE2Context DummySEE2Cont => dummySEE2Cont; + public virtual See2Context DummySee2Cont => _dummySee2Cont; - public virtual int InitRL => initRL; + public virtual int InitRl => _initRl; - public virtual int EscCount { get => escCount; set => escCount = value & 0xff; } + public virtual int EscCount { get => _escCount; set => _escCount = value & 0xff; } - public virtual int[] CharMask => charMask; + public virtual int[] CharMask => _charMask; - public virtual int NumMasked { get => numMasked; set => numMasked = value; } + public virtual int NumMasked { get => _numMasked; set => _numMasked = value; } - public virtual int PrevSuccess { get => prevSuccess; set => prevSuccess = value & 0xff; } + public virtual int PrevSuccess { get => _prevSuccess; set => _prevSuccess = value & 0xff; } - public virtual int InitEsc { get => initEsc; set => initEsc = value; } + public virtual int InitEsc { get => _initEsc; set => _initEsc = value; } - public virtual int RunLength { get => runLength; set => runLength = value; } + public virtual int RunLength { get => _runLength; set => _runLength = value; } - public virtual int HiBitsFlag { get => hiBitsFlag; set => hiBitsFlag = value & 0xff; } + public virtual int HiBitsFlag { get => _hiBitsFlag; set => _hiBitsFlag = value & 0xff; } - public virtual int[][] BinSumm => binSumm; + public virtual int[][] BinSumm => _binSumm; internal RangeCoder Coder { get; private set; } @@ -48,7 +48,7 @@ namespace SharpCompress.Compressors.PPMd.H public virtual byte[] Heap => SubAlloc.Heap; - public virtual int OrderFall => orderFall; + public virtual int OrderFall => _orderFall; public const int MAX_O = 64; /* maximum allowed model order */ @@ -67,98 +67,98 @@ namespace SharpCompress.Compressors.PPMd.H public const int MAX_FREQ = 124; - private readonly SEE2Context[][] SEE2Cont = new SEE2Context[25][]; + private readonly See2Context[][] _see2Cont = new See2Context[25][]; - private SEE2Context dummySEE2Cont; + private See2Context _dummySee2Cont; - private PPMContext minContext; //medContext + private PpmContext _minContext; //medContext - private PPMContext maxContext; + private PpmContext _maxContext; - private int numMasked, initEsc, orderFall, maxOrder, runLength, initRL; + private int _numMasked, _initEsc, _orderFall, _maxOrder, _runLength, _initRl; - private readonly int[] charMask = new int[256]; + private readonly int[] _charMask = new int[256]; - private readonly int[] NS2Indx = new int[256]; + private readonly int[] _ns2Indx = new int[256]; - private readonly int[] NS2BSIndx = new int[256]; + private readonly int[] _ns2BsIndx = new int[256]; - private readonly int[] HB2Flag = new int[256]; + private readonly int[] _hb2Flag = new int[256]; // byte EscCount, PrevSuccess, HiBitsFlag; - private int escCount, prevSuccess, hiBitsFlag; + private int _escCount, _prevSuccess, _hiBitsFlag; - private readonly int[][] binSumm = new int[128][]; // binary SEE-contexts + private readonly int[][] _binSumm = new int[128][]; // binary SEE-contexts - private static readonly int[] InitBinEsc = {0x3CDD, 0x1F3F, 0x59BF, 0x48F3, 0x64A1, 0x5ABC, 0x6632, 0x6051}; + private static readonly int[] INIT_BIN_ESC = {0x3CDD, 0x1F3F, 0x59BF, 0x48F3, 0x64A1, 0x5ABC, 0x6632, 0x6051}; // Temp fields //UPGRADE_NOTE: Final was removed from the declaration of 'tempState1 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly State tempState1 = new State(null); + private readonly State _tempState1 = new State(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempState2 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly State tempState2 = new State(null); + private readonly State _tempState2 = new State(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempState3 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly State tempState3 = new State(null); + private readonly State _tempState3 = new State(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempState4 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly State tempState4 = new State(null); + private readonly State _tempState4 = new State(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempStateRef1 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly StateRef tempStateRef1 = new StateRef(); + private readonly StateRef _tempStateRef1 = new StateRef(); //UPGRADE_NOTE: Final was removed from the declaration of 'tempStateRef2 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly StateRef tempStateRef2 = new StateRef(); + private readonly StateRef _tempStateRef2 = new StateRef(); //UPGRADE_NOTE: Final was removed from the declaration of 'tempPPMContext1 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly PPMContext tempPPMContext1 = new PPMContext(null); + private readonly PpmContext _tempPpmContext1 = new PpmContext(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempPPMContext2 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly PPMContext tempPPMContext2 = new PPMContext(null); + private readonly PpmContext _tempPpmContext2 = new PpmContext(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempPPMContext3 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly PPMContext tempPPMContext3 = new PPMContext(null); + private readonly PpmContext _tempPpmContext3 = new PpmContext(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempPPMContext4 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly PPMContext tempPPMContext4 = new PPMContext(null); + private readonly PpmContext _tempPpmContext4 = new PpmContext(null); //UPGRADE_NOTE: Final was removed from the declaration of 'ps '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly int[] ps = new int[MAX_O]; + private readonly int[] _ps = new int[MAX_O]; - public ModelPPM() + public ModelPpm() { InitBlock(); - minContext = null; - maxContext = null; + _minContext = null; + _maxContext = null; //medContext = null; } - private void restartModelRare() + private void RestartModelRare() { - Utility.Fill(charMask, 0); - SubAlloc.initSubAllocator(); - initRL = -(maxOrder < 12 ? maxOrder : 12) - 1; - int addr = SubAlloc.allocContext(); - minContext.Address = addr; - maxContext.Address = addr; - minContext.setSuffix(0); - orderFall = maxOrder; - minContext.NumStats = 256; - minContext.FreqData.SummFreq = minContext.NumStats + 1; + Utility.Fill(_charMask, 0); + SubAlloc.InitSubAllocator(); + _initRl = -(_maxOrder < 12 ? _maxOrder : 12) - 1; + int addr = SubAlloc.AllocContext(); + _minContext.Address = addr; + _maxContext.Address = addr; + _minContext.SetSuffix(0); + _orderFall = _maxOrder; + _minContext.NumStats = 256; + _minContext.FreqData.SummFreq = _minContext.NumStats + 1; - addr = SubAlloc.allocUnits(256 / 2); + addr = SubAlloc.AllocUnits(256 / 2); FoundState.Address = addr; - minContext.FreqData.SetStats(addr); + _minContext.FreqData.SetStats(addr); State state = new State(SubAlloc.Heap); - addr = minContext.FreqData.GetStats(); - runLength = initRL; - prevSuccess = 0; + addr = _minContext.FreqData.GetStats(); + _runLength = _initRl; + _prevSuccess = 0; for (int i = 0; i < 256; i++) { - state.Address = addr + i * State.Size; + state.Address = addr + i * State.SIZE; state.Symbol = i; state.Freq = 1; state.SetSuccessor(0); @@ -170,7 +170,7 @@ namespace SharpCompress.Compressors.PPMd.H { for (int m = 0; m < 64; m += 8) { - binSumm[i][k + m] = BIN_SCALE - InitBinEsc[k] / (i + 2); + _binSumm[i][k + m] = BIN_SCALE - INIT_BIN_ESC[k] / (i + 2); } } } @@ -178,68 +178,68 @@ namespace SharpCompress.Compressors.PPMd.H { for (int k = 0; k < 16; k++) { - SEE2Cont[i][k].Initialize(5 * i + 10); + _see2Cont[i][k].Initialize(5 * i + 10); } } } - private void startModelRare(int MaxOrder) + private void StartModelRare(int maxOrder) { - int i, k, m, Step; - escCount = 1; - maxOrder = MaxOrder; - restartModelRare(); + int i, k, m, step; + _escCount = 1; + _maxOrder = maxOrder; + RestartModelRare(); // Bug Fixed - NS2BSIndx[0] = 0; - NS2BSIndx[1] = 2; + _ns2BsIndx[0] = 0; + _ns2BsIndx[1] = 2; for (int j = 0; j < 9; j++) { - NS2BSIndx[2 + j] = 4; + _ns2BsIndx[2 + j] = 4; } for (int j = 0; j < 256 - 11; j++) { - NS2BSIndx[11 + j] = 6; + _ns2BsIndx[11 + j] = 6; } for (i = 0; i < 3; i++) { - NS2Indx[i] = i; + _ns2Indx[i] = i; } - for (m = i, k = 1, Step = 1; i < 256; i++) + for (m = i, k = 1, step = 1; i < 256; i++) { - NS2Indx[i] = m; + _ns2Indx[i] = m; if ((--k) == 0) { - k = ++Step; + k = ++step; m++; } } for (int j = 0; j < 0x40; j++) { - HB2Flag[j] = 0; + _hb2Flag[j] = 0; } for (int j = 0; j < 0x100 - 0x40; j++) { - HB2Flag[0x40 + j] = 0x08; + _hb2Flag[0x40 + j] = 0x08; } - dummySEE2Cont.Shift = PERIOD_BITS; + _dummySee2Cont.Shift = PERIOD_BITS; } - private void clearMask() + private void ClearMask() { - escCount = 1; - Utility.Fill(charMask, 0); + _escCount = 1; + Utility.Fill(_charMask, 0); } - internal bool decodeInit(Unpack unpackRead, int escChar) + internal bool DecodeInit(IRarUnpack unpackRead, int escChar) { - int MaxOrder = unpackRead.Char & 0xff; - bool reset = ((MaxOrder & 0x20) != 0); + int maxOrder = unpackRead.Char & 0xff; + bool reset = ((maxOrder & 0x20) != 0); - int MaxMB = 0; + int maxMb = 0; if (reset) { - MaxMB = unpackRead.Char; + maxMb = unpackRead.Char; } else { @@ -248,7 +248,7 @@ namespace SharpCompress.Compressors.PPMd.H return (false); } } - if ((MaxOrder & 0x40) != 0) + if ((maxOrder & 0x40) != 0) { escChar = unpackRead.Char; unpackRead.PpmEscChar = escChar; @@ -256,60 +256,60 @@ namespace SharpCompress.Compressors.PPMd.H Coder = new RangeCoder(unpackRead); if (reset) { - MaxOrder = (MaxOrder & 0x1f) + 1; - if (MaxOrder > 16) + maxOrder = (maxOrder & 0x1f) + 1; + if (maxOrder > 16) { - MaxOrder = 16 + (MaxOrder - 16) * 3; + maxOrder = 16 + (maxOrder - 16) * 3; } - if (MaxOrder == 1) + if (maxOrder == 1) { - SubAlloc.stopSubAllocator(); + SubAlloc.StopSubAllocator(); return (false); } - SubAlloc.startSubAllocator((MaxMB + 1) << 20); - minContext = new PPMContext(Heap); + SubAlloc.StartSubAllocator((maxMb + 1) << 20); + _minContext = new PpmContext(Heap); //medContext = new PPMContext(Heap); - maxContext = new PPMContext(Heap); + _maxContext = new PpmContext(Heap); FoundState = new State(Heap); - dummySEE2Cont = new SEE2Context(); + _dummySee2Cont = new See2Context(); for (int i = 0; i < 25; i++) { for (int j = 0; j < 16; j++) { - SEE2Cont[i][j] = new SEE2Context(); + _see2Cont[i][j] = new See2Context(); } } - startModelRare(MaxOrder); + StartModelRare(maxOrder); } - return (minContext.Address != 0); + return (_minContext.Address != 0); } - public virtual int decodeChar() + public virtual int DecodeChar() { // Debug //subAlloc.dumpHeap(); - if (minContext.Address <= SubAlloc.PText || minContext.Address > SubAlloc.HeapEnd) + if (_minContext.Address <= SubAlloc.PText || _minContext.Address > SubAlloc.HeapEnd) { return (-1); } - if (minContext.NumStats != 1) + if (_minContext.NumStats != 1) { - if (minContext.FreqData.GetStats() <= SubAlloc.PText || - minContext.FreqData.GetStats() > SubAlloc.HeapEnd) + if (_minContext.FreqData.GetStats() <= SubAlloc.PText || + _minContext.FreqData.GetStats() > SubAlloc.HeapEnd) { return (-1); } - if (!minContext.decodeSymbol1(this)) + if (!_minContext.DecodeSymbol1(this)) { return (-1); } } else { - minContext.decodeBinSymbol(this); + _minContext.DecodeBinSymbol(this); } Coder.Decode(); while (FoundState.Address == 0) @@ -317,94 +317,94 @@ namespace SharpCompress.Compressors.PPMd.H Coder.AriDecNormalize(); do { - orderFall++; - minContext.Address = minContext.getSuffix(); // =MinContext->Suffix; - if (minContext.Address <= SubAlloc.PText || minContext.Address > SubAlloc.HeapEnd) + _orderFall++; + _minContext.Address = _minContext.GetSuffix(); // =MinContext->Suffix; + if (_minContext.Address <= SubAlloc.PText || _minContext.Address > SubAlloc.HeapEnd) { return (-1); } } - while (minContext.NumStats == numMasked); - if (!minContext.decodeSymbol2(this)) + while (_minContext.NumStats == _numMasked); + if (!_minContext.DecodeSymbol2(this)) { return (-1); } Coder.Decode(); } - int Symbol = FoundState.Symbol; - if ((orderFall == 0) && FoundState.GetSuccessor() > SubAlloc.PText) + int symbol = FoundState.Symbol; + if ((_orderFall == 0) && FoundState.GetSuccessor() > SubAlloc.PText) { // MinContext=MaxContext=FoundState->Successor; int addr = FoundState.GetSuccessor(); - minContext.Address = addr; - maxContext.Address = addr; + _minContext.Address = addr; + _maxContext.Address = addr; } else { - updateModel(); + UpdateModel(); //this.foundState.Address=foundState.Address);//TODO just 4 debugging - if (escCount == 0) + if (_escCount == 0) { - clearMask(); + ClearMask(); } } Coder.AriDecNormalize(); // ARI_DEC_NORMALIZE(Coder.code,Coder.low,Coder.range,Coder.UnpackRead); - return (Symbol); + return (symbol); } - public virtual SEE2Context[][] getSEE2Cont() + public virtual See2Context[][] GetSee2Cont() { - return SEE2Cont; + return _see2Cont; } - public virtual void incEscCount(int dEscCount) + public virtual void IncEscCount(int dEscCount) { EscCount = EscCount + dEscCount; } - public virtual void incRunLength(int dRunLength) + public virtual void IncRunLength(int dRunLength) { RunLength = RunLength + dRunLength; } - public virtual int[] getHB2Flag() + public virtual int[] GetHb2Flag() { - return HB2Flag; + return _hb2Flag; } - public virtual int[] getNS2BSIndx() + public virtual int[] GetNs2BsIndx() { - return NS2BSIndx; + return _ns2BsIndx; } - public virtual int[] getNS2Indx() + public virtual int[] GetNs2Indx() { - return NS2Indx; + return _ns2Indx; } - private int createSuccessors(bool Skip, State p1) + private int CreateSuccessors(bool skip, State p1) { //State upState = tempState1.Initialize(null); - StateRef upState = tempStateRef2; - State tempState = tempState1.Initialize(Heap); + StateRef upState = _tempStateRef2; + State tempState = _tempState1.Initialize(Heap); // PPM_CONTEXT* pc=MinContext, * UpBranch=FoundState->Successor; - PPMContext pc = tempPPMContext1.Initialize(Heap); - pc.Address = minContext.Address; - PPMContext upBranch = tempPPMContext2.Initialize(Heap); + PpmContext pc = _tempPpmContext1.Initialize(Heap); + pc.Address = _minContext.Address; + PpmContext upBranch = _tempPpmContext2.Initialize(Heap); upBranch.Address = FoundState.GetSuccessor(); // STATE * p, * ps[MAX_O], ** pps=ps; - State p = tempState2.Initialize(Heap); + State p = _tempState2.Initialize(Heap); int pps = 0; bool noLoop = false; - if (!Skip) + if (!skip) { - ps[pps++] = FoundState.Address; // *pps++ = FoundState; - if (pc.getSuffix() == 0) + _ps[pps++] = FoundState.Address; // *pps++ = FoundState; + if (pc.GetSuffix() == 0) { noLoop = true; } @@ -415,14 +415,14 @@ namespace SharpCompress.Compressors.PPMd.H if (p1.Address != 0) { p.Address = p1.Address; - pc.Address = pc.getSuffix(); // =pc->Suffix; + pc.Address = pc.GetSuffix(); // =pc->Suffix; loopEntry = true; } do { if (!loopEntry) { - pc.Address = pc.getSuffix(); // pc=pc->Suffix; + pc.Address = pc.GetSuffix(); // pc=pc->Suffix; if (pc.NumStats != 1) { p.Address = pc.FreqData.GetStats(); // p=pc->U.Stats @@ -437,7 +437,7 @@ namespace SharpCompress.Compressors.PPMd.H } else { - p.Address = pc.getOneState().Address; // p=&(pc->OneState); + p.Address = pc.GetOneState().Address; // p=&(pc->OneState); } } // LOOP_ENTRY: loopEntry = false; @@ -446,9 +446,9 @@ namespace SharpCompress.Compressors.PPMd.H pc.Address = p.GetSuccessor(); // =p->Successor; break; } - ps[pps++] = p.Address; + _ps[pps++] = p.Address; } - while (pc.getSuffix() != 0); + while (pc.GetSuffix() != 0); } // NO_LOOP: if (pps == 0) { @@ -482,13 +482,13 @@ namespace SharpCompress.Compressors.PPMd.H } else { - upState.Freq = pc.getOneState().Freq; // UpState.Freq=pc->OneState.Freq; + upState.Freq = pc.GetOneState().Freq; // UpState.Freq=pc->OneState.Freq; } do { // pc = pc->createChild(this,*--pps,UpState); - tempState.Address = ps[--pps]; - pc.Address = pc.createChild(this, tempState, upState); + tempState.Address = _ps[--pps]; + pc.Address = pc.CreateChild(this, tempState, upState); if (pc.Address == 0) { return 0; @@ -498,26 +498,26 @@ namespace SharpCompress.Compressors.PPMd.H return pc.Address; } - private void updateModelRestart() + private void UpdateModelRestart() { - restartModelRare(); - escCount = 0; + RestartModelRare(); + _escCount = 0; } - private void updateModel() + private void UpdateModel() { //System.out.println("ModelPPM.updateModel()"); // STATE fs = *FoundState, *p = NULL; - StateRef fs = tempStateRef1; + StateRef fs = _tempStateRef1; fs.Values = FoundState; - State p = tempState3.Initialize(Heap); - State tempState = tempState4.Initialize(Heap); + State p = _tempState3.Initialize(Heap); + State tempState = _tempState4.Initialize(Heap); - PPMContext pc = tempPPMContext3.Initialize(Heap); - PPMContext successor = tempPPMContext4.Initialize(Heap); + PpmContext pc = _tempPpmContext3.Initialize(Heap); + PpmContext successor = _tempPpmContext4.Initialize(Heap); int ns1, ns, cf, sf, s0; - pc.Address = minContext.getSuffix(); + pc.Address = _minContext.GetSuffix(); if (fs.Freq < MAX_FREQ / 4 && pc.Address != 0) { if (pc.NumStats != 1) @@ -530,10 +530,10 @@ namespace SharpCompress.Compressors.PPMd.H p.IncrementAddress(); } while (p.Symbol != fs.Symbol); - tempState.Address = p.Address - State.Size; + tempState.Address = p.Address - State.SIZE; if (p.Freq >= tempState.Freq) { - State.PPMDSwap(p, tempState); + State.PpmdSwap(p, tempState); p.DecrementAddress(); } } @@ -545,31 +545,31 @@ namespace SharpCompress.Compressors.PPMd.H } else { - p.Address = pc.getOneState().Address; + p.Address = pc.GetOneState().Address; if (p.Freq < 32) { p.IncrementFreq(1); } } } - if (orderFall == 0) + if (_orderFall == 0) { - FoundState.SetSuccessor(createSuccessors(true, p)); - minContext.Address = FoundState.GetSuccessor(); - maxContext.Address = FoundState.GetSuccessor(); - if (minContext.Address == 0) + FoundState.SetSuccessor(CreateSuccessors(true, p)); + _minContext.Address = FoundState.GetSuccessor(); + _maxContext.Address = FoundState.GetSuccessor(); + if (_minContext.Address == 0) { - updateModelRestart(); + UpdateModelRestart(); return; } return; } SubAlloc.Heap[SubAlloc.PText] = (byte)fs.Symbol; - SubAlloc.incPText(); + SubAlloc.IncPText(); successor.Address = SubAlloc.PText; if (SubAlloc.PText >= SubAlloc.FakeUnitsStart) { - updateModelRestart(); + UpdateModelRestart(); return; } @@ -579,43 +579,43 @@ namespace SharpCompress.Compressors.PPMd.H { if (fs.GetSuccessor() <= SubAlloc.PText) { - fs.SetSuccessor(createSuccessors(false, p)); + fs.SetSuccessor(CreateSuccessors(false, p)); if (fs.GetSuccessor() == 0) { - updateModelRestart(); + UpdateModelRestart(); return; } } - if (--orderFall == 0) + if (--_orderFall == 0) { successor.Address = fs.GetSuccessor(); - if (maxContext.Address != minContext.Address) + if (_maxContext.Address != _minContext.Address) { - SubAlloc.decPText(1); + SubAlloc.DecPText(1); } } } else { FoundState.SetSuccessor(successor.Address); - fs.SetSuccessor(minContext); + fs.SetSuccessor(_minContext); } // // Debug // subAlloc.dumpHeap(); - ns = minContext.NumStats; - s0 = minContext.FreqData.SummFreq - (ns) - (fs.Freq - 1); - for (pc.Address = maxContext.Address; pc.Address != minContext.Address; pc.Address = pc.getSuffix()) + ns = _minContext.NumStats; + s0 = _minContext.FreqData.SummFreq - (ns) - (fs.Freq - 1); + for (pc.Address = _maxContext.Address; pc.Address != _minContext.Address; pc.Address = pc.GetSuffix()) { if ((ns1 = pc.NumStats) != 1) { if ((ns1 & 1) == 0) { //System.out.println(ns1); - pc.FreqData.SetStats(SubAlloc.expandUnits(pc.FreqData.GetStats(), Utility.URShift(ns1, 1))); + pc.FreqData.SetStats(SubAlloc.ExpandUnits(pc.FreqData.GetStats(), Utility.URShift(ns1, 1))); if (pc.FreqData.GetStats() == 0) { - updateModelRestart(); + UpdateModelRestart(); return; } } @@ -630,13 +630,13 @@ namespace SharpCompress.Compressors.PPMd.H } else { - p.Address = SubAlloc.allocUnits(1); + p.Address = SubAlloc.AllocUnits(1); if (p.Address == 0) { - updateModelRestart(); + UpdateModelRestart(); return; } - p.SetValues(pc.getOneState()); + p.SetValues(pc.GetOneState()); pc.FreqData.SetStats(p); if (p.Freq < MAX_FREQ / 4 - 1) { @@ -646,7 +646,7 @@ namespace SharpCompress.Compressors.PPMd.H { p.Freq = MAX_FREQ - 4; } - pc.FreqData.SummFreq = (p.Freq + initEsc + (ns > 3 ? 1 : 0)); + pc.FreqData.SummFreq = (p.Freq + _initEsc + (ns > 3 ? 1 : 0)); } cf = 2 * fs.Freq * (pc.FreqData.SummFreq + 6); sf = s0 + pc.FreqData.SummFreq; @@ -660,7 +660,7 @@ namespace SharpCompress.Compressors.PPMd.H cf = 4 + (cf >= 9 * sf ? 1 : 0) + (cf >= 12 * sf ? 1 : 0) + (cf >= 15 * sf ? 1 : 0); pc.FreqData.IncrementSummFreq(cf); } - p.Address = pc.FreqData.GetStats() + ns1 * State.Size; + p.Address = pc.FreqData.GetStats() + ns1 * State.SIZE; p.SetSuccessor(successor); p.Symbol = fs.Symbol; p.Freq = cf; @@ -668,8 +668,8 @@ namespace SharpCompress.Compressors.PPMd.H } int address = fs.GetSuccessor(); - maxContext.Address = address; - minContext.Address = address; + _maxContext.Address = address; + _minContext.Address = address; //TODO-----debug // int pos = minContext.getFreqData().getStats(); @@ -686,21 +686,21 @@ namespace SharpCompress.Compressors.PPMd.H StringBuilder buffer = new StringBuilder(); buffer.Append("ModelPPM["); buffer.Append("\n numMasked="); - buffer.Append(numMasked); + buffer.Append(_numMasked); buffer.Append("\n initEsc="); - buffer.Append(initEsc); + buffer.Append(_initEsc); buffer.Append("\n orderFall="); - buffer.Append(orderFall); + buffer.Append(_orderFall); buffer.Append("\n maxOrder="); - buffer.Append(maxOrder); + buffer.Append(_maxOrder); buffer.Append("\n runLength="); - buffer.Append(runLength); + buffer.Append(_runLength); buffer.Append("\n initRL="); - buffer.Append(initRL); + buffer.Append(_initRl); buffer.Append("\n escCount="); - buffer.Append(escCount); + buffer.Append(_escCount); buffer.Append("\n prevSuccess="); - buffer.Append(prevSuccess); + buffer.Append(_prevSuccess); buffer.Append("\n foundState="); buffer.Append(FoundState); buffer.Append("\n coder="); @@ -716,7 +716,7 @@ namespace SharpCompress.Compressors.PPMd.H // subAlloc.dumpHeap(); // } - internal bool decodeInit(Stream stream, int maxOrder, int maxMemory) + internal bool DecodeInit(Stream stream, int maxOrder, int maxMemory) { if (stream != null) { @@ -725,61 +725,61 @@ namespace SharpCompress.Compressors.PPMd.H if (maxOrder == 1) { - SubAlloc.stopSubAllocator(); + SubAlloc.StopSubAllocator(); return (false); } - SubAlloc.startSubAllocator(maxMemory); - minContext = new PPMContext(Heap); + SubAlloc.StartSubAllocator(maxMemory); + _minContext = new PpmContext(Heap); //medContext = new PPMContext(Heap); - maxContext = new PPMContext(Heap); + _maxContext = new PpmContext(Heap); FoundState = new State(Heap); - dummySEE2Cont = new SEE2Context(); + _dummySee2Cont = new See2Context(); for (int i = 0; i < 25; i++) { for (int j = 0; j < 16; j++) { - SEE2Cont[i][j] = new SEE2Context(); + _see2Cont[i][j] = new See2Context(); } } - startModelRare(maxOrder); + StartModelRare(maxOrder); - return (minContext.Address != 0); + return (_minContext.Address != 0); } - internal void nextContext() + internal void NextContext() { int addr = FoundState.GetSuccessor(); - if (orderFall == 0 && addr > SubAlloc.PText) + if (_orderFall == 0 && addr > SubAlloc.PText) { - minContext.Address = addr; - maxContext.Address = addr; + _minContext.Address = addr; + _maxContext.Address = addr; } else { - updateModel(); + UpdateModel(); } } - public int decodeChar(Decoder decoder) + public int DecodeChar(Decoder decoder) { - if (minContext.NumStats != 1) + if (_minContext.NumStats != 1) { - State s = tempState1.Initialize(Heap); - s.Address = minContext.FreqData.GetStats(); + State s = _tempState1.Initialize(Heap); + s.Address = _minContext.FreqData.GetStats(); int i; int count, hiCnt; - if ((count = (int)decoder.GetThreshold((uint)minContext.FreqData.SummFreq)) < (hiCnt = s.Freq)) + if ((count = (int)decoder.GetThreshold((uint)_minContext.FreqData.SummFreq)) < (hiCnt = s.Freq)) { byte symbol; decoder.Decode(0, (uint)s.Freq); symbol = (byte)s.Symbol; - minContext.update1_0(this, s.Address); - nextContext(); + _minContext.update1_0(this, s.Address); + NextContext(); return symbol; } - prevSuccess = 0; - i = minContext.NumStats - 1; + _prevSuccess = 0; + i = _minContext.NumStats - 1; do { s.IncrementAddress(); @@ -788,113 +788,113 @@ namespace SharpCompress.Compressors.PPMd.H byte symbol; decoder.Decode((uint)(hiCnt - s.Freq), (uint)s.Freq); symbol = (byte)s.Symbol; - minContext.update1(this, s.Address); - nextContext(); + _minContext.Update1(this, s.Address); + NextContext(); return symbol; } } while (--i > 0); - if (count >= minContext.FreqData.SummFreq) + if (count >= _minContext.FreqData.SummFreq) { return -2; } - hiBitsFlag = HB2Flag[FoundState.Symbol]; - decoder.Decode((uint)hiCnt, (uint)(minContext.FreqData.SummFreq - hiCnt)); + _hiBitsFlag = _hb2Flag[FoundState.Symbol]; + decoder.Decode((uint)hiCnt, (uint)(_minContext.FreqData.SummFreq - hiCnt)); for (i = 0; i < 256; i++) { - charMask[i] = -1; + _charMask[i] = -1; } - charMask[s.Symbol] = 0; - i = minContext.NumStats - 1; + _charMask[s.Symbol] = 0; + i = _minContext.NumStats - 1; do { s.DecrementAddress(); - charMask[s.Symbol] = 0; + _charMask[s.Symbol] = 0; } while (--i > 0); } else { - State rs = tempState1.Initialize(Heap); - rs.Address = minContext.getOneState().Address; - hiBitsFlag = getHB2Flag()[FoundState.Symbol]; + State rs = _tempState1.Initialize(Heap); + rs.Address = _minContext.GetOneState().Address; + _hiBitsFlag = GetHb2Flag()[FoundState.Symbol]; int off1 = rs.Freq - 1; - int off2 = minContext.getArrayIndex(this, rs); - int bs = binSumm[off1][off2]; + int off2 = _minContext.GetArrayIndex(this, rs); + int bs = _binSumm[off1][off2]; if (decoder.DecodeBit((uint)bs, 14) == 0) { byte symbol; - binSumm[off1][off2] = (bs + INTERVAL - minContext.getMean(bs, PERIOD_BITS, 2)) & 0xFFFF; + _binSumm[off1][off2] = (bs + INTERVAL - _minContext.GetMean(bs, PERIOD_BITS, 2)) & 0xFFFF; FoundState.Address = rs.Address; symbol = (byte)rs.Symbol; rs.IncrementFreq((rs.Freq < 128) ? 1 : 0); - prevSuccess = 1; - incRunLength(1); - nextContext(); + _prevSuccess = 1; + IncRunLength(1); + NextContext(); return symbol; } - bs = (bs - minContext.getMean(bs, PERIOD_BITS, 2)) & 0xFFFF; - binSumm[off1][off2] = bs; - initEsc = PPMContext.ExpEscape[Utility.URShift(bs, 10)]; + bs = (bs - _minContext.GetMean(bs, PERIOD_BITS, 2)) & 0xFFFF; + _binSumm[off1][off2] = bs; + _initEsc = PpmContext.EXP_ESCAPE[Utility.URShift(bs, 10)]; int i; for (i = 0; i < 256; i++) { - charMask[i] = -1; + _charMask[i] = -1; } - charMask[rs.Symbol] = 0; - prevSuccess = 0; + _charMask[rs.Symbol] = 0; + _prevSuccess = 0; } for (;;) { - State s = tempState1.Initialize(Heap); + State s = _tempState1.Initialize(Heap); int i; int freqSum, count, hiCnt; - SEE2Context see; - int num, numMasked = minContext.NumStats; + See2Context see; + int num, numMasked = _minContext.NumStats; do { - orderFall++; - minContext.Address = minContext.getSuffix(); - if (minContext.Address <= SubAlloc.PText || minContext.Address > SubAlloc.HeapEnd) + _orderFall++; + _minContext.Address = _minContext.GetSuffix(); + if (_minContext.Address <= SubAlloc.PText || _minContext.Address > SubAlloc.HeapEnd) { return -1; } } - while (minContext.NumStats == numMasked); + while (_minContext.NumStats == numMasked); hiCnt = 0; - s.Address = minContext.FreqData.GetStats(); + s.Address = _minContext.FreqData.GetStats(); i = 0; - num = minContext.NumStats - numMasked; + num = _minContext.NumStats - numMasked; do { - int k = charMask[s.Symbol]; + int k = _charMask[s.Symbol]; hiCnt += s.Freq & k; - minContext.ps[i] = s.Address; + _minContext._ps[i] = s.Address; s.IncrementAddress(); i -= k; } while (i != num); - see = minContext.makeEscFreq(this, numMasked, out freqSum); + see = _minContext.MakeEscFreq(this, numMasked, out freqSum); freqSum += hiCnt; count = (int)decoder.GetThreshold((uint)freqSum); if (count < hiCnt) { byte symbol; - State ps = tempState2.Initialize(Heap); - for (hiCnt = 0, i = 0, ps.Address = minContext.ps[i]; + State ps = _tempState2.Initialize(Heap); + for (hiCnt = 0, i = 0, ps.Address = _minContext._ps[i]; (hiCnt += ps.Freq) <= count; - i++, ps.Address = minContext.ps[i]) + i++, ps.Address = _minContext._ps[i]) { ; } s.Address = ps.Address; decoder.Decode((uint)(hiCnt - s.Freq), (uint)s.Freq); - see.update(); + see.Update(); symbol = (byte)s.Symbol; - minContext.update2(this, s.Address); - updateModel(); + _minContext.Update2(this, s.Address); + UpdateModel(); return symbol; } if (count >= freqSum) @@ -905,8 +905,8 @@ namespace SharpCompress.Compressors.PPMd.H see.Summ = see.Summ + freqSum; do { - s.Address = minContext.ps[--i]; - charMask[s.Symbol] = 0; + s.Address = _minContext._ps[--i]; + _charMask[s.Symbol] = 0; } while (i != 0); } diff --git a/src/SharpCompress/Compressors/PPMd/H/PPMContext.cs b/src/SharpCompress/Compressors/PPMd/H/PPMContext.cs index e39fa69c..7be31003 100644 --- a/src/SharpCompress/Compressors/PPMd/H/PPMContext.cs +++ b/src/SharpCompress/Compressors/PPMd/H/PPMContext.cs @@ -4,15 +4,15 @@ using SharpCompress.Converters; namespace SharpCompress.Compressors.PPMd.H { - internal class PPMContext : Pointer + internal class PpmContext : Pointer { internal FreqData FreqData { - get => freqData; + get => _freqData; set { - freqData.SummFreq = value.SummFreq; - freqData.SetStats(value.GetStats()); + _freqData.SummFreq = value.SummFreq; + _freqData.SetStats(value.GetStats()); } } @@ -22,14 +22,14 @@ namespace SharpCompress.Compressors.PPMd.H { if (Memory != null) { - numStats = DataConverter.LittleEndian.GetInt16(Memory, Address) & 0xffff; + _numStats = DataConverter.LittleEndian.GetInt16(Memory, Address) & 0xffff; } - return numStats; + return _numStats; } set { - numStats = value & 0xffff; + _numStats = value & 0xffff; if (Memory != null) { DataConverter.LittleEndian.PutBytes(Memory, Address, (short)value); @@ -39,89 +39,89 @@ namespace SharpCompress.Compressors.PPMd.H //UPGRADE_NOTE: Final was removed from the declaration of 'unionSize '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" //UPGRADE_NOTE: The initialization of 'unionSize' was moved to static method 'SharpCompress.Unpack.PPM.PPMContext'. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1005'" - private static readonly int unionSize; + private static readonly int UNION_SIZE; //UPGRADE_NOTE: Final was removed from the declaration of 'size '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - public static readonly int size = 2 + unionSize + 4; // 12 + public static readonly int SIZE = 2 + UNION_SIZE + 4; // 12 // ushort NumStats; - private int numStats; // determines if feqData or onstate is used + private int _numStats; // determines if feqData or onstate is used // (1==onestate) //UPGRADE_NOTE: Final was removed from the declaration of 'freqData '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly FreqData freqData; // -\ + private readonly FreqData _freqData; // -\ // |-> union //UPGRADE_NOTE: Final was removed from the declaration of 'oneState '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly State oneState; // -/ + private readonly State _oneState; // -/ - private int suffix; // pointer ppmcontext + private int _suffix; // pointer ppmcontext //UPGRADE_NOTE: Final was removed from the declaration of 'ExpEscape'. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - public static readonly int[] ExpEscape = {25, 14, 9, 7, 5, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 2}; + public static readonly int[] EXP_ESCAPE = {25, 14, 9, 7, 5, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 2}; // Temp fields //UPGRADE_NOTE: Final was removed from the declaration of 'tempState1 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly State tempState1 = new State(null); + private readonly State _tempState1 = new State(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempState2 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly State tempState2 = new State(null); + private readonly State _tempState2 = new State(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempState3 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly State tempState3 = new State(null); + private readonly State _tempState3 = new State(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempState4 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly State tempState4 = new State(null); + private readonly State _tempState4 = new State(null); //UPGRADE_NOTE: Final was removed from the declaration of 'tempState5 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly State tempState5 = new State(null); - private PPMContext tempPPMContext; + private readonly State _tempState5 = new State(null); + private PpmContext _tempPpmContext; //UPGRADE_NOTE: Final was removed from the declaration of 'ps '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - internal int[] ps = new int[256]; + internal int[] _ps = new int[256]; - public PPMContext(byte[] Memory) - : base(Memory) + public PpmContext(byte[] memory) + : base(memory) { - oneState = new State(Memory); - freqData = new FreqData(Memory); + _oneState = new State(memory); + _freqData = new FreqData(memory); } - internal PPMContext Initialize(byte[] mem) + internal PpmContext Initialize(byte[] mem) { - oneState.Initialize(mem); - freqData.Initialize(mem); - return base.Initialize(mem); + _oneState.Initialize(mem); + _freqData.Initialize(mem); + return base.Initialize(mem); } - internal State getOneState() + internal State GetOneState() { - return oneState; + return _oneState; } - internal void setOneState(StateRef oneState) + internal void SetOneState(StateRef oneState) { - this.oneState.SetValues(oneState); + _oneState.SetValues(oneState); } - internal int getSuffix() + internal int GetSuffix() { if (Memory != null) { - suffix = DataConverter.LittleEndian.GetInt32(Memory, Address + 8); + _suffix = DataConverter.LittleEndian.GetInt32(Memory, Address + 8); } - return suffix; + return _suffix; } - internal void setSuffix(PPMContext suffix) + internal void SetSuffix(PpmContext suffix) { - setSuffix(suffix.Address); + SetSuffix(suffix.Address); } - internal void setSuffix(int suffix) + internal void SetSuffix(int suffix) { - this.suffix = suffix; + _suffix = suffix; if (Memory != null) { DataConverter.LittleEndian.PutBytes(Memory, Address + 8, suffix); @@ -134,62 +134,62 @@ namespace SharpCompress.Compressors.PPMd.H set { base.Address = value; - oneState.Address = value + 2; - freqData.Address = value + 2; + _oneState.Address = value + 2; + _freqData.Address = value + 2; } } - private PPMContext getTempPPMContext(byte[] Memory) + private PpmContext GetTempPpmContext(byte[] memory) { - if (tempPPMContext == null) + if (_tempPpmContext == null) { - tempPPMContext = new PPMContext(null); + _tempPpmContext = new PpmContext(null); } - return tempPPMContext.Initialize(Memory); + return _tempPpmContext.Initialize(memory); } - internal int createChild(ModelPPM model, State pStats, StateRef firstState) + internal int CreateChild(ModelPpm model, State pStats, StateRef firstState) { - PPMContext pc = getTempPPMContext(model.SubAlloc.Heap); - pc.Address = model.SubAlloc.allocContext(); + PpmContext pc = GetTempPpmContext(model.SubAlloc.Heap); + pc.Address = model.SubAlloc.AllocContext(); if (pc != null) { pc.NumStats = 1; - pc.setOneState(firstState); - pc.setSuffix(this); + pc.SetOneState(firstState); + pc.SetSuffix(this); pStats.SetSuccessor(pc); } return pc.Address; } - internal void rescale(ModelPPM model) + internal void Rescale(ModelPpm model) { - int OldNS = NumStats, i = NumStats - 1, Adder, EscFreq; + int oldNs = NumStats, i = NumStats - 1, adder, escFreq; // STATE* p1, * p; State p1 = new State(model.Heap); State p = new State(model.Heap); State temp = new State(model.Heap); - for (p.Address = model.FoundState.Address; p.Address != freqData.GetStats(); p.DecrementAddress()) + for (p.Address = model.FoundState.Address; p.Address != _freqData.GetStats(); p.DecrementAddress()) { - temp.Address = p.Address - State.Size; - State.PPMDSwap(p, temp); + temp.Address = p.Address - State.SIZE; + State.PpmdSwap(p, temp); } - temp.Address = freqData.GetStats(); + temp.Address = _freqData.GetStats(); temp.IncrementFreq(4); - freqData.IncrementSummFreq(4); - EscFreq = freqData.SummFreq - p.Freq; - Adder = (model.OrderFall != 0) ? 1 : 0; - p.Freq = Utility.URShift((p.Freq + Adder), 1); - freqData.SummFreq = p.Freq; + _freqData.IncrementSummFreq(4); + escFreq = _freqData.SummFreq - p.Freq; + adder = (model.OrderFall != 0) ? 1 : 0; + p.Freq = Utility.URShift((p.Freq + adder), 1); + _freqData.SummFreq = p.Freq; do { p.IncrementAddress(); - EscFreq -= p.Freq; - p.Freq = Utility.URShift((p.Freq + Adder), 1); - freqData.IncrementSummFreq(p.Freq); - temp.Address = p.Address - State.Size; + escFreq -= p.Freq; + p.Freq = Utility.URShift((p.Freq + adder), 1); + _freqData.IncrementSummFreq(p.Freq); + temp.Address = p.Address - State.SIZE; if (p.Freq > temp.Freq) { p1.Address = p.Address; @@ -200,12 +200,12 @@ namespace SharpCompress.Compressors.PPMd.H do { // p1[0]=p1[-1]; - temp2.Address = p1.Address - State.Size; + temp2.Address = p1.Address - State.SIZE; p1.SetValues(temp2); p1.DecrementAddress(); - temp3.Address = p1.Address - State.Size; + temp3.Address = p1.Address - State.SIZE; } - while (p1.Address != freqData.GetStats() && tmp.Freq > temp3.Freq); + while (p1.Address != _freqData.GetStats() && tmp.Freq > temp3.Freq); p1.SetValues(tmp); } } @@ -218,12 +218,12 @@ namespace SharpCompress.Compressors.PPMd.H p.DecrementAddress(); } while (p.Freq == 0); - EscFreq += i; + escFreq += i; NumStats = NumStats - i; if (NumStats == 1) { StateRef tmp = new StateRef(); - temp.Address = freqData.GetStats(); + temp.Address = _freqData.GetStats(); tmp.Values = temp; // STATE tmp=*U.Stats; @@ -231,68 +231,68 @@ namespace SharpCompress.Compressors.PPMd.H { // tmp.Freq-=(tmp.Freq >> 1) tmp.DecrementFreq(Utility.URShift(tmp.Freq, 1)); - EscFreq = Utility.URShift(EscFreq, 1); + escFreq = Utility.URShift(escFreq, 1); } - while (EscFreq > 1); - model.SubAlloc.freeUnits(freqData.GetStats(), Utility.URShift((OldNS + 1), 1)); - oneState.SetValues(tmp); - model.FoundState.Address = oneState.Address; + while (escFreq > 1); + model.SubAlloc.FreeUnits(_freqData.GetStats(), Utility.URShift((oldNs + 1), 1)); + _oneState.SetValues(tmp); + model.FoundState.Address = _oneState.Address; return; } } - EscFreq -= Utility.URShift(EscFreq, 1); - freqData.IncrementSummFreq(EscFreq); - int n0 = Utility.URShift((OldNS + 1), 1), n1 = Utility.URShift((NumStats + 1), 1); + escFreq -= Utility.URShift(escFreq, 1); + _freqData.IncrementSummFreq(escFreq); + int n0 = Utility.URShift((oldNs + 1), 1), n1 = Utility.URShift((NumStats + 1), 1); if (n0 != n1) { - freqData.SetStats(model.SubAlloc.shrinkUnits(freqData.GetStats(), n0, n1)); + _freqData.SetStats(model.SubAlloc.ShrinkUnits(_freqData.GetStats(), n0, n1)); } - model.FoundState.Address = freqData.GetStats(); + model.FoundState.Address = _freqData.GetStats(); } - internal int getArrayIndex(ModelPPM Model, State rs) + internal int GetArrayIndex(ModelPpm model, State rs) { - PPMContext tempSuffix = getTempPPMContext(Model.SubAlloc.Heap); - tempSuffix.Address = getSuffix(); + PpmContext tempSuffix = GetTempPpmContext(model.SubAlloc.Heap); + tempSuffix.Address = GetSuffix(); int ret = 0; - ret += Model.PrevSuccess; - ret += Model.getNS2BSIndx()[tempSuffix.NumStats - 1]; - ret += Model.HiBitsFlag + 2 * Model.getHB2Flag()[rs.Symbol]; - ret += ((Utility.URShift(Model.RunLength, 26)) & 0x20); + ret += model.PrevSuccess; + ret += model.GetNs2BsIndx()[tempSuffix.NumStats - 1]; + ret += model.HiBitsFlag + 2 * model.GetHb2Flag()[rs.Symbol]; + ret += ((Utility.URShift(model.RunLength, 26)) & 0x20); return ret; } - internal int getMean(int summ, int shift, int round) + internal int GetMean(int summ, int shift, int round) { return (Utility.URShift((summ + (1 << (shift - round))), (shift))); } - internal void decodeBinSymbol(ModelPPM model) + internal void DecodeBinSymbol(ModelPpm model) { - State rs = tempState1.Initialize(model.Heap); - rs.Address = oneState.Address; // State& - model.HiBitsFlag = model.getHB2Flag()[model.FoundState.Symbol]; + State rs = _tempState1.Initialize(model.Heap); + rs.Address = _oneState.Address; // State& + model.HiBitsFlag = model.GetHb2Flag()[model.FoundState.Symbol]; int off1 = rs.Freq - 1; - int off2 = getArrayIndex(model, rs); + int off2 = GetArrayIndex(model, rs); int bs = model.BinSumm[off1][off2]; - if (model.Coder.GetCurrentShiftCount(ModelPPM.TOT_BITS) < bs) + if (model.Coder.GetCurrentShiftCount(ModelPpm.TOT_BITS) < bs) { model.FoundState.Address = rs.Address; rs.IncrementFreq((rs.Freq < 128) ? 1 : 0); model.Coder.SubRange.LowCount = 0; model.Coder.SubRange.HighCount = bs; - bs = ((bs + ModelPPM.INTERVAL - getMean(bs, ModelPPM.PERIOD_BITS, 2)) & 0xffff); + bs = ((bs + ModelPpm.INTERVAL - GetMean(bs, ModelPpm.PERIOD_BITS, 2)) & 0xffff); model.BinSumm[off1][off2] = bs; model.PrevSuccess = 1; - model.incRunLength(1); + model.IncRunLength(1); } else { model.Coder.SubRange.LowCount = bs; - bs = (bs - getMean(bs, ModelPPM.PERIOD_BITS, 2)) & 0xFFFF; + bs = (bs - GetMean(bs, ModelPpm.PERIOD_BITS, 2)) & 0xFFFF; model.BinSumm[off1][off2] = bs; - model.Coder.SubRange.HighCount = ModelPPM.BIN_SCALE; - model.InitEsc = ExpEscape[Utility.URShift(bs, 10)]; + model.Coder.SubRange.HighCount = ModelPpm.BIN_SCALE; + model.InitEsc = EXP_ESCAPE[Utility.URShift(bs, 10)]; model.NumMasked = 1; model.CharMask[rs.Symbol] = model.EscCount; model.PrevSuccess = 0; @@ -317,50 +317,50 @@ namespace SharpCompress.Compressors.PPMd.H // state2.Address=p2); // } - internal void update1(ModelPPM model, int p) + internal void Update1(ModelPpm model, int p) { model.FoundState.Address = p; model.FoundState.IncrementFreq(4); - freqData.IncrementSummFreq(4); - State p0 = tempState3.Initialize(model.Heap); - State p1 = tempState4.Initialize(model.Heap); + _freqData.IncrementSummFreq(4); + State p0 = _tempState3.Initialize(model.Heap); + State p1 = _tempState4.Initialize(model.Heap); p0.Address = p; - p1.Address = p - State.Size; + p1.Address = p - State.SIZE; if (p0.Freq > p1.Freq) { - State.PPMDSwap(p0, p1); + State.PpmdSwap(p0, p1); model.FoundState.Address = p1.Address; - if (p1.Freq > ModelPPM.MAX_FREQ) + if (p1.Freq > ModelPpm.MAX_FREQ) { - rescale(model); + Rescale(model); } } } - internal void update1_0(ModelPPM model, int p) + internal void update1_0(ModelPpm model, int p) { model.FoundState.Address = p; - model.PrevSuccess = 2 * model.FoundState.Freq > freqData.SummFreq ? 1 : 0; - model.incRunLength(model.PrevSuccess); - freqData.IncrementSummFreq(4); + model.PrevSuccess = 2 * model.FoundState.Freq > _freqData.SummFreq ? 1 : 0; + model.IncRunLength(model.PrevSuccess); + _freqData.IncrementSummFreq(4); model.FoundState.IncrementFreq(4); - if (model.FoundState.Freq > ModelPPM.MAX_FREQ) + if (model.FoundState.Freq > ModelPpm.MAX_FREQ) { - rescale(model); + Rescale(model); } } - internal bool decodeSymbol2(ModelPPM model) + internal bool DecodeSymbol2(ModelPpm model) { long count; int hiCnt, i = NumStats - model.NumMasked; - SEE2Context psee2c = makeEscFreq2(model, i); + See2Context psee2C = MakeEscFreq2(model, i); RangeCoder coder = model.Coder; // STATE* ps[256], ** pps=ps, * p=U.Stats-1; - State p = tempState1.Initialize(model.Heap); - State temp = tempState2.Initialize(model.Heap); - p.Address = freqData.GetStats() - State.Size; + State p = _tempState1.Initialize(model.Heap); + State temp = _tempState2.Initialize(model.Heap); + p.Address = _freqData.GetStats() - State.SIZE; int pps = 0; hiCnt = 0; @@ -372,28 +372,28 @@ namespace SharpCompress.Compressors.PPMd.H } while (model.CharMask[p.Symbol] == model.EscCount); hiCnt += p.Freq; - ps[pps++] = p.Address; + _ps[pps++] = p.Address; } while (--i != 0); - coder.SubRange.incScale(hiCnt); + coder.SubRange.IncScale(hiCnt); count = coder.CurrentCount; if (count >= coder.SubRange.Scale) { return false; } pps = 0; - p.Address = ps[pps]; + p.Address = _ps[pps]; if (count < hiCnt) { hiCnt = 0; while ((hiCnt += p.Freq) <= count) { - p.Address = ps[++pps]; // p=*++pps; + p.Address = _ps[++pps]; // p=*++pps; } coder.SubRange.HighCount = hiCnt; coder.SubRange.LowCount = hiCnt - p.Freq; - psee2c.update(); - update2(model, p.Address); + psee2C.Update(); + Update2(model, p.Address); } else { @@ -403,106 +403,106 @@ namespace SharpCompress.Compressors.PPMd.H pps--; do { - temp.Address = ps[++pps]; // (*++pps) + temp.Address = _ps[++pps]; // (*++pps) model.CharMask[temp.Symbol] = model.EscCount; } while (--i != 0); - psee2c.incSumm((int)coder.SubRange.Scale); + psee2C.IncSumm((int)coder.SubRange.Scale); model.NumMasked = NumStats; } return (true); } - internal void update2(ModelPPM model, int p) + internal void Update2(ModelPpm model, int p) { - State temp = tempState5.Initialize(model.Heap); + State temp = _tempState5.Initialize(model.Heap); temp.Address = p; model.FoundState.Address = p; model.FoundState.IncrementFreq(4); - freqData.IncrementSummFreq(4); - if (temp.Freq > ModelPPM.MAX_FREQ) + _freqData.IncrementSummFreq(4); + if (temp.Freq > ModelPpm.MAX_FREQ) { - rescale(model); + Rescale(model); } - model.incEscCount(1); - model.RunLength = model.InitRL; + model.IncEscCount(1); + model.RunLength = model.InitRl; } - private SEE2Context makeEscFreq2(ModelPPM model, int Diff) + private See2Context MakeEscFreq2(ModelPpm model, int diff) { - SEE2Context psee2c; + See2Context psee2C; int numStats = NumStats; if (numStats != 256) { - PPMContext suff = getTempPPMContext(model.Heap); - suff.Address = getSuffix(); - int idx1 = model.getNS2Indx()[Diff - 1]; + PpmContext suff = GetTempPpmContext(model.Heap); + suff.Address = GetSuffix(); + int idx1 = model.GetNs2Indx()[diff - 1]; int idx2 = 0; - idx2 += ((Diff < suff.NumStats - numStats) ? 1 : 0); - idx2 += 2 * ((freqData.SummFreq < 11 * numStats) ? 1 : 0); - idx2 += 4 * ((model.NumMasked > Diff) ? 1 : 0); + idx2 += ((diff < suff.NumStats - numStats) ? 1 : 0); + idx2 += 2 * ((_freqData.SummFreq < 11 * numStats) ? 1 : 0); + idx2 += 4 * ((model.NumMasked > diff) ? 1 : 0); idx2 += model.HiBitsFlag; - psee2c = model.getSEE2Cont()[idx1][idx2]; - model.Coder.SubRange.Scale = psee2c.Mean; + psee2C = model.GetSee2Cont()[idx1][idx2]; + model.Coder.SubRange.Scale = psee2C.Mean; } else { - psee2c = model.DummySEE2Cont; + psee2C = model.DummySee2Cont; model.Coder.SubRange.Scale = 1; } - return psee2c; + return psee2C; } - internal SEE2Context makeEscFreq(ModelPPM model, int numMasked, out int escFreq) + internal See2Context MakeEscFreq(ModelPpm model, int numMasked, out int escFreq) { - SEE2Context psee2c; + See2Context psee2C; int numStats = NumStats; int nonMasked = numStats - numMasked; if (numStats != 256) { - PPMContext suff = getTempPPMContext(model.Heap); - suff.Address = getSuffix(); - int idx1 = model.getNS2Indx()[nonMasked - 1]; + PpmContext suff = GetTempPpmContext(model.Heap); + suff.Address = GetSuffix(); + int idx1 = model.GetNs2Indx()[nonMasked - 1]; int idx2 = 0; idx2 += ((nonMasked < suff.NumStats - numStats) ? 1 : 0); - idx2 += 2 * ((freqData.SummFreq < 11 * numStats) ? 1 : 0); + idx2 += 2 * ((_freqData.SummFreq < 11 * numStats) ? 1 : 0); idx2 += 4 * ((numMasked > nonMasked) ? 1 : 0); idx2 += model.HiBitsFlag; - psee2c = model.getSEE2Cont()[idx1][idx2]; - escFreq = psee2c.Mean; + psee2C = model.GetSee2Cont()[idx1][idx2]; + escFreq = psee2C.Mean; } else { - psee2c = model.DummySEE2Cont; + psee2C = model.DummySee2Cont; escFreq = 1; } - return psee2c; + return psee2C; } - internal bool decodeSymbol1(ModelPPM model) + internal bool DecodeSymbol1(ModelPpm model) { RangeCoder coder = model.Coder; - coder.SubRange.Scale = freqData.SummFreq; + coder.SubRange.Scale = _freqData.SummFreq; State p = new State(model.Heap); - p.Address = freqData.GetStats(); - int i, HiCnt; + p.Address = _freqData.GetStats(); + int i, hiCnt; long count = coder.CurrentCount; if (count >= coder.SubRange.Scale) { return false; } - if (count < (HiCnt = p.Freq)) + if (count < (hiCnt = p.Freq)) { - coder.SubRange.HighCount = HiCnt; - model.PrevSuccess = (2 * HiCnt > coder.SubRange.Scale) ? 1 : 0; - model.incRunLength(model.PrevSuccess); - HiCnt += 4; + coder.SubRange.HighCount = hiCnt; + model.PrevSuccess = (2 * hiCnt > coder.SubRange.Scale) ? 1 : 0; + model.IncRunLength(model.PrevSuccess); + hiCnt += 4; model.FoundState.Address = p.Address; - model.FoundState.Freq = HiCnt; - freqData.IncrementSummFreq(4); - if (HiCnt > ModelPPM.MAX_FREQ) + model.FoundState.Freq = hiCnt; + _freqData.IncrementSummFreq(4); + if (hiCnt > ModelPpm.MAX_FREQ) { - rescale(model); + Rescale(model); } coder.SubRange.LowCount = 0; return true; @@ -514,12 +514,12 @@ namespace SharpCompress.Compressors.PPMd.H model.PrevSuccess = 0; int numStats = NumStats; i = numStats - 1; - while ((HiCnt += p.IncrementAddress().Freq) <= count) + while ((hiCnt += p.IncrementAddress().Freq) <= count) { if (--i == 0) { - model.HiBitsFlag = model.getHB2Flag()[model.FoundState.Symbol]; - coder.SubRange.LowCount = HiCnt; + model.HiBitsFlag = model.GetHb2Flag()[model.FoundState.Symbol]; + coder.SubRange.LowCount = hiCnt; model.CharMask[p.Symbol] = model.EscCount; model.NumMasked = numStats; i = numStats - 1; @@ -533,9 +533,9 @@ namespace SharpCompress.Compressors.PPMd.H return (true); } } - coder.SubRange.LowCount = HiCnt - p.Freq; - coder.SubRange.HighCount = HiCnt; - update1(model, p.Address); + coder.SubRange.LowCount = hiCnt - p.Freq; + coder.SubRange.HighCount = hiCnt; + Update1(model, p.Address); return (true); } @@ -546,22 +546,22 @@ namespace SharpCompress.Compressors.PPMd.H buffer.Append("\n Address="); buffer.Append(Address); buffer.Append("\n size="); - buffer.Append(size); + buffer.Append(SIZE); buffer.Append("\n numStats="); buffer.Append(NumStats); buffer.Append("\n Suffix="); - buffer.Append(getSuffix()); + buffer.Append(GetSuffix()); buffer.Append("\n freqData="); - buffer.Append(freqData); + buffer.Append(_freqData); buffer.Append("\n oneState="); - buffer.Append(oneState); + buffer.Append(_oneState); buffer.Append("\n]"); return buffer.ToString(); } - static PPMContext() + static PpmContext() { - unionSize = Math.Max(FreqData.Size, State.Size); + UNION_SIZE = Math.Max(FreqData.SIZE, State.SIZE); } } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/PPMd/H/RangeCoder.cs b/src/SharpCompress/Compressors/PPMd/H/RangeCoder.cs index 4c04828a..e1946852 100644 --- a/src/SharpCompress/Compressors/PPMd/H/RangeCoder.cs +++ b/src/SharpCompress/Compressors/PPMd/H/RangeCoder.cs @@ -9,22 +9,22 @@ namespace SharpCompress.Compressors.PPMd.H { internal const int TOP = 1 << 24; internal const int BOT = 1 << 15; - internal const long UintMask = 0xFFFFffffL; + internal const long UINT_MASK = 0xFFFFffffL; // uint low, code, range; - private long low, code, range; - private readonly Unpack unpackRead; - private readonly Stream stream; + private long _low, _code, _range; + private readonly IRarUnpack _unpackRead; + private readonly Stream _stream; - internal RangeCoder(Unpack unpackRead) + internal RangeCoder(IRarUnpack unpackRead) { - this.unpackRead = unpackRead; + _unpackRead = unpackRead; Init(); } internal RangeCoder(Stream stream) { - this.stream = stream; + _stream = stream; Init(); } @@ -32,11 +32,11 @@ namespace SharpCompress.Compressors.PPMd.H { SubRange = new SubRange(); - low = code = 0L; - range = 0xFFFFffffL; + _low = _code = 0L; + _range = 0xFFFFffffL; for (int i = 0; i < 4; i++) { - code = ((code << 8) | Char) & UintMask; + _code = ((_code << 8) | Char) & UINT_MASK; } } @@ -44,8 +44,8 @@ namespace SharpCompress.Compressors.PPMd.H { get { - range = (range / SubRange.Scale) & UintMask; - return (int)((code - low) / (range)); + _range = (_range / SubRange.Scale) & UINT_MASK; + return (int)((_code - _low) / (_range)); } } @@ -53,13 +53,13 @@ namespace SharpCompress.Compressors.PPMd.H { get { - if (unpackRead != null) + if (_unpackRead != null) { - return (unpackRead.Char); + return (_unpackRead.Char); } - if (stream != null) + if (_stream != null) { - return stream.ReadByte(); + return _stream.ReadByte(); } return -1; } @@ -67,16 +67,16 @@ namespace SharpCompress.Compressors.PPMd.H internal SubRange SubRange { get; private set; } - internal long GetCurrentShiftCount(int SHIFT) + internal long GetCurrentShiftCount(int shift) { - range = Utility.URShift(range, SHIFT); - return ((code - low) / (range)) & UintMask; + _range = Utility.URShift(_range, shift); + return ((_code - _low) / (_range)) & UINT_MASK; } internal void Decode() { - low = (low + (range * SubRange.LowCount)) & UintMask; - range = (range * (SubRange.HighCount - SubRange.LowCount)) & UintMask; + _low = (_low + (_range * SubRange.LowCount)) & UINT_MASK; + _range = (_range * (SubRange.HighCount - SubRange.LowCount)) & UINT_MASK; } internal void AriDecNormalize() @@ -90,16 +90,16 @@ namespace SharpCompress.Compressors.PPMd.H // Rewrote for clarity bool c2 = false; - while ((low ^ (low + range)) < TOP || (c2 = range < BOT)) + while ((_low ^ (_low + _range)) < TOP || (c2 = _range < BOT)) { if (c2) { - range = (-low & (BOT - 1)) & UintMask; + _range = (-_low & (BOT - 1)) & UINT_MASK; c2 = false; } - code = ((code << 8) | Char) & UintMask; - range = (range << 8) & UintMask; - low = (low << 8) & UintMask; + _code = ((_code << 8) | Char) & UINT_MASK; + _range = (_range << 8) & UINT_MASK; + _low = (_low << 8) & UINT_MASK; } } @@ -109,11 +109,11 @@ namespace SharpCompress.Compressors.PPMd.H StringBuilder buffer = new StringBuilder(); buffer.Append("RangeCoder["); buffer.Append("\n low="); - buffer.Append(low); + buffer.Append(_low); buffer.Append("\n code="); - buffer.Append(code); + buffer.Append(_code); buffer.Append("\n range="); - buffer.Append(range); + buffer.Append(_range); buffer.Append("\n subrange="); buffer.Append(SubRange); buffer.Append("]"); @@ -124,18 +124,18 @@ namespace SharpCompress.Compressors.PPMd.H internal class SubRange { // uint LowCount, HighCount, scale; - private long lowCount, highCount, scale; + private long _lowCount, _highCount, _scale; - internal void incScale(int dScale) + internal void IncScale(int dScale) { Scale = Scale + dScale; } - internal long HighCount { get => highCount; set => highCount = value & RangeCoder.UintMask; } + internal long HighCount { get => _highCount; set => _highCount = value & RangeCoder.UINT_MASK; } - internal long LowCount { get => lowCount & RangeCoder.UintMask; set => lowCount = value & RangeCoder.UintMask; } + internal long LowCount { get => _lowCount & RangeCoder.UINT_MASK; set => _lowCount = value & RangeCoder.UINT_MASK; } - internal long Scale { get => scale; set => scale = value & RangeCoder.UintMask; } + internal long Scale { get => _scale; set => _scale = value & RangeCoder.UINT_MASK; } // Debug public override String ToString() @@ -143,11 +143,11 @@ namespace SharpCompress.Compressors.PPMd.H StringBuilder buffer = new StringBuilder(); buffer.Append("SubRange["); buffer.Append("\n lowCount="); - buffer.Append(lowCount); + buffer.Append(_lowCount); buffer.Append("\n highCount="); - buffer.Append(highCount); + buffer.Append(_highCount); buffer.Append("\n scale="); - buffer.Append(scale); + buffer.Append(_scale); buffer.Append("]"); return buffer.ToString(); } diff --git a/src/SharpCompress/Compressors/PPMd/H/RarMemBlock.cs b/src/SharpCompress/Compressors/PPMd/H/RarMemBlock.cs index 6f0eda1c..78e6824c 100644 --- a/src/SharpCompress/Compressors/PPMd/H/RarMemBlock.cs +++ b/src/SharpCompress/Compressors/PPMd/H/RarMemBlock.cs @@ -4,14 +4,14 @@ namespace SharpCompress.Compressors.PPMd.H { internal class RarMemBlock : Pointer { - public const int size = 12; + public const int SIZE = 12; - private int stamp, NU; + private int _stamp, _nu; - private int next, prev; // Pointer RarMemBlock + private int _next, _prev; // Pointer RarMemBlock - public RarMemBlock(byte[] Memory) - : base(Memory) + public RarMemBlock(byte[] memory) + : base(memory) { } @@ -21,14 +21,14 @@ namespace SharpCompress.Compressors.PPMd.H { if (Memory != null) { - stamp = DataConverter.LittleEndian.GetInt16(Memory, Address) & 0xffff; + _stamp = DataConverter.LittleEndian.GetInt16(Memory, Address) & 0xffff; } - return stamp; + return _stamp; } set { - stamp = value; + _stamp = value; if (Memory != null) { DataConverter.LittleEndian.PutBytes(Memory, Address, (short)value); @@ -63,9 +63,9 @@ namespace SharpCompress.Compressors.PPMd.H { if (Memory != null) { - next = DataConverter.LittleEndian.GetInt32(Memory, Address + 4); + _next = DataConverter.LittleEndian.GetInt32(Memory, Address + 4); } - return next; + return _next; } internal void SetNext(RarMemBlock next) @@ -75,25 +75,25 @@ namespace SharpCompress.Compressors.PPMd.H internal void SetNext(int next) { - this.next = next; + _next = next; if (Memory != null) { DataConverter.LittleEndian.PutBytes(Memory, Address + 4, next); } } - internal int GetNU() + internal int GetNu() { if (Memory != null) { - NU = DataConverter.LittleEndian.GetInt16(Memory, Address + 2) & 0xffff; + _nu = DataConverter.LittleEndian.GetInt16(Memory, Address + 2) & 0xffff; } - return NU; + return _nu; } - internal void SetNU(int nu) + internal void SetNu(int nu) { - NU = nu & 0xffff; + _nu = nu & 0xffff; if (Memory != null) { DataConverter.LittleEndian.PutBytes(Memory, Address + 2, (short)nu); @@ -104,9 +104,9 @@ namespace SharpCompress.Compressors.PPMd.H { if (Memory != null) { - prev = DataConverter.LittleEndian.GetInt32(Memory, Address + 8); + _prev = DataConverter.LittleEndian.GetInt32(Memory, Address + 8); } - return prev; + return _prev; } internal void SetPrev(RarMemBlock prev) @@ -116,7 +116,7 @@ namespace SharpCompress.Compressors.PPMd.H internal void SetPrev(int prev) { - this.prev = prev; + _prev = prev; if (Memory != null) { DataConverter.LittleEndian.PutBytes(Memory, Address + 8, prev); diff --git a/src/SharpCompress/Compressors/PPMd/H/RarNode.cs b/src/SharpCompress/Compressors/PPMd/H/RarNode.cs index 69af3f35..3cbf2894 100644 --- a/src/SharpCompress/Compressors/PPMd/H/RarNode.cs +++ b/src/SharpCompress/Compressors/PPMd/H/RarNode.cs @@ -5,12 +5,12 @@ namespace SharpCompress.Compressors.PPMd.H { internal class RarNode : Pointer { - private int next; //rarnode pointer + private int _next; //rarnode pointer - public const int size = 4; + public const int SIZE = 4; - public RarNode(byte[] Memory) - : base(Memory) + public RarNode(byte[] memory) + : base(memory) { } @@ -18,9 +18,9 @@ namespace SharpCompress.Compressors.PPMd.H { if (Memory != null) { - next = DataConverter.LittleEndian.GetInt32(Memory, Address); + _next = DataConverter.LittleEndian.GetInt32(Memory, Address); } - return next; + return _next; } internal void SetNext(RarNode next) @@ -30,7 +30,7 @@ namespace SharpCompress.Compressors.PPMd.H internal void SetNext(int next) { - this.next = next; + _next = next; if (Memory != null) { DataConverter.LittleEndian.PutBytes(Memory, Address, next); @@ -44,7 +44,7 @@ namespace SharpCompress.Compressors.PPMd.H buffer.Append("\n Address="); buffer.Append(Address); buffer.Append("\n size="); - buffer.Append(size); + buffer.Append(SIZE); buffer.Append("\n next="); buffer.Append(GetNext()); buffer.Append("\n]"); diff --git a/src/SharpCompress/Compressors/PPMd/H/SEE2Context.cs b/src/SharpCompress/Compressors/PPMd/H/SEE2Context.cs index 66344c00..c97f0836 100644 --- a/src/SharpCompress/Compressors/PPMd/H/SEE2Context.cs +++ b/src/SharpCompress/Compressors/PPMd/H/SEE2Context.cs @@ -3,55 +3,55 @@ using System.Text; namespace SharpCompress.Compressors.PPMd.H { - internal class SEE2Context + internal class See2Context { public virtual int Mean { get { - int retVal = Utility.URShift(summ, shift); - summ -= retVal; + int retVal = Utility.URShift(_summ, _shift); + _summ -= retVal; return retVal + ((retVal == 0) ? 1 : 0); } } - public virtual int Count { get => count; set => count = value & 0xff; } + public virtual int Count { get => _count; set => _count = value & 0xff; } - public virtual int Shift { get => shift; set => shift = value & 0xff; } + public virtual int Shift { get => _shift; set => _shift = value & 0xff; } - public virtual int Summ { get => summ; set => summ = value & 0xffff; } + public virtual int Summ { get => _summ; set => _summ = value & 0xffff; } - public const int size = 4; + public const int SIZE = 4; // ushort Summ; - private int summ; + private int _summ; // byte Shift; - private int shift; + private int _shift; // byte Count; - private int count; + private int _count; public void Initialize(int initVal) { - shift = (ModelPPM.PERIOD_BITS - 4) & 0xff; - summ = (initVal << shift) & 0xffff; - count = 4; + _shift = (ModelPpm.PERIOD_BITS - 4) & 0xff; + _summ = (initVal << _shift) & 0xffff; + _count = 4; } - public virtual void update() + public virtual void Update() { - if (shift < ModelPPM.PERIOD_BITS && --count == 0) + if (_shift < ModelPpm.PERIOD_BITS && --_count == 0) { - summ += summ; - count = (3 << shift++); + _summ += _summ; + _count = (3 << _shift++); } - summ &= 0xffff; - count &= 0xff; - shift &= 0xff; + _summ &= 0xffff; + _count &= 0xff; + _shift &= 0xff; } - public virtual void incSumm(int dSumm) + public virtual void IncSumm(int dSumm) { Summ = Summ + dSumm; } @@ -61,13 +61,13 @@ namespace SharpCompress.Compressors.PPMd.H StringBuilder buffer = new StringBuilder(); buffer.Append("SEE2Context["); buffer.Append("\n size="); - buffer.Append(size); + buffer.Append(SIZE); buffer.Append("\n summ="); - buffer.Append(summ); + buffer.Append(_summ); buffer.Append("\n shift="); - buffer.Append(shift); + buffer.Append(_shift); buffer.Append("\n count="); - buffer.Append(count); + buffer.Append(_count); buffer.Append("\n]"); return buffer.ToString(); } diff --git a/src/SharpCompress/Compressors/PPMd/H/State.cs b/src/SharpCompress/Compressors/PPMd/H/State.cs index 2771e5b3..8062da14 100644 --- a/src/SharpCompress/Compressors/PPMd/H/State.cs +++ b/src/SharpCompress/Compressors/PPMd/H/State.cs @@ -6,10 +6,10 @@ namespace SharpCompress.Compressors.PPMd.H { internal class State : Pointer { - internal const int Size = 6; + internal const int SIZE = 6; - internal State(byte[] Memory) - : base(Memory) + internal State(byte[] memory) + : base(memory) { } @@ -32,7 +32,7 @@ namespace SharpCompress.Compressors.PPMd.H return DataConverter.LittleEndian.GetInt32(Memory, Address + 2); } - internal void SetSuccessor(PPMContext successor) + internal void SetSuccessor(PpmContext successor) { SetSuccessor(successor.Address); } @@ -51,25 +51,25 @@ namespace SharpCompress.Compressors.PPMd.H internal void SetValues(State ptr) { - Array.Copy(ptr.Memory, ptr.Address, Memory, Address, Size); + Array.Copy(ptr.Memory, ptr.Address, Memory, Address, SIZE); } internal State DecrementAddress() { - Address = Address - Size; + Address = Address - SIZE; return this; } internal State IncrementAddress() { - Address = Address + Size; + Address = Address + SIZE; return this; } - internal static void PPMDSwap(State ptr1, State ptr2) + internal static void PpmdSwap(State ptr1, State ptr2) { byte[] mem1 = ptr1.Memory, mem2 = ptr2.Memory; - for (int i = 0, pos1 = ptr1.Address, pos2 = ptr2.Address; i < Size; i++, pos1++, pos2++) + for (int i = 0, pos1 = ptr1.Address, pos2 = ptr2.Address; i < SIZE; i++, pos1++, pos2++) { byte temp = mem1[pos1]; mem1[pos1] = mem2[pos2]; @@ -84,7 +84,7 @@ namespace SharpCompress.Compressors.PPMd.H buffer.Append("\n Address="); buffer.Append(Address); buffer.Append("\n size="); - buffer.Append(Size); + buffer.Append(SIZE); buffer.Append("\n symbol="); buffer.Append(Symbol); buffer.Append("\n freq="); diff --git a/src/SharpCompress/Compressors/PPMd/H/StateRef.cs b/src/SharpCompress/Compressors/PPMd/H/StateRef.cs index 3499bdd8..f1e8e90f 100644 --- a/src/SharpCompress/Compressors/PPMd/H/StateRef.cs +++ b/src/SharpCompress/Compressors/PPMd/H/StateRef.cs @@ -5,15 +5,15 @@ namespace SharpCompress.Compressors.PPMd.H { internal class StateRef { - private int symbol; + private int _symbol; - private int freq; + private int _freq; - private int successor; // pointer ppmcontext + private int _successor; // pointer ppmcontext - internal int Symbol { get => symbol; set => symbol = value & 0xff; } + internal int Symbol { get => _symbol; set => _symbol = value & 0xff; } - internal int Freq { get => freq; set => freq = value & 0xff; } + internal int Freq { get => _freq; set => _freq = value & 0xff; } internal State Values { @@ -27,27 +27,27 @@ namespace SharpCompress.Compressors.PPMd.H public virtual void IncrementFreq(int dFreq) { - freq = (freq + dFreq) & 0xff; + _freq = (_freq + dFreq) & 0xff; } public virtual void DecrementFreq(int dFreq) { - freq = (freq - dFreq) & 0xff; + _freq = (_freq - dFreq) & 0xff; } public virtual int GetSuccessor() { - return successor; + return _successor; } - public virtual void SetSuccessor(PPMContext successor) + public virtual void SetSuccessor(PpmContext successor) { SetSuccessor(successor.Address); } public virtual void SetSuccessor(int successor) { - this.successor = successor; + _successor = successor; } public override String ToString() diff --git a/src/SharpCompress/Compressors/PPMd/H/SubAllocator.cs b/src/SharpCompress/Compressors/PPMd/H/SubAllocator.cs index 0918e9a9..c49eed97 100644 --- a/src/SharpCompress/Compressors/PPMd/H/SubAllocator.cs +++ b/src/SharpCompress/Compressors/PPMd/H/SubAllocator.cs @@ -5,15 +5,15 @@ namespace SharpCompress.Compressors.PPMd.H { internal class SubAllocator { - public virtual int FakeUnitsStart { get => fakeUnitsStart; set => fakeUnitsStart = value; } + public virtual int FakeUnitsStart { get => _fakeUnitsStart; set => _fakeUnitsStart = value; } - public virtual int HeapEnd => heapEnd; + public virtual int HeapEnd => _heapEnd; - public virtual int PText { get => pText; set => pText = value; } + public virtual int PText { get => _pText; set => _pText = value; } - public virtual int UnitsStart { get => unitsStart; set => unitsStart = value; } + public virtual int UnitsStart { get => _unitsStart; set => _unitsStart = value; } - public virtual byte[] Heap => heap; + public virtual byte[] Heap => _heap; //UPGRADE_NOTE: Final was removed from the declaration of 'N4 '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" public const int N1 = 4; @@ -30,123 +30,123 @@ namespace SharpCompress.Compressors.PPMd.H public const int FIXED_UNIT_SIZE = 12; - private int subAllocatorSize; + private int _subAllocatorSize; // byte Indx2Units[N_INDEXES], Units2Indx[128], GlueCount; - private readonly int[] indx2Units = new int[N_INDEXES]; - private readonly int[] units2Indx = new int[128]; - private int glueCount; + private readonly int[] _indx2Units = new int[N_INDEXES]; + private readonly int[] _units2Indx = new int[128]; + private int _glueCount; // byte *HeapStart,*LoUnit, *HiUnit; - private int heapStart, loUnit, hiUnit; + private int _heapStart, _loUnit, _hiUnit; //UPGRADE_NOTE: Final was removed from the declaration of 'freeList '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly RarNode[] freeList = new RarNode[N_INDEXES]; + private readonly RarNode[] _freeList = new RarNode[N_INDEXES]; // byte *pText, *UnitsStart,*HeapEnd,*FakeUnitsStart; - private int pText, unitsStart, heapEnd, fakeUnitsStart; + private int _pText, _unitsStart, _heapEnd, _fakeUnitsStart; - private byte[] heap; + private byte[] _heap; - private int freeListPos; + private int _freeListPos; - private int tempMemBlockPos; + private int _tempMemBlockPos; // Temp fields - private RarNode tempRarNode; - private RarMemBlock tempRarMemBlock1; - private RarMemBlock tempRarMemBlock2; - private RarMemBlock tempRarMemBlock3; + private RarNode _tempRarNode; + private RarMemBlock _tempRarMemBlock1; + private RarMemBlock _tempRarMemBlock2; + private RarMemBlock _tempRarMemBlock3; public SubAllocator() { - clean(); + Clean(); } - public virtual void clean() + public virtual void Clean() { - subAllocatorSize = 0; + _subAllocatorSize = 0; } - private void insertNode(int p, int indx) + private void InsertNode(int p, int indx) { - RarNode temp = tempRarNode; + RarNode temp = _tempRarNode; temp.Address = p; - temp.SetNext(freeList[indx].GetNext()); - freeList[indx].SetNext(temp); + temp.SetNext(_freeList[indx].GetNext()); + _freeList[indx].SetNext(temp); } - public virtual void incPText() + public virtual void IncPText() { - pText++; + _pText++; } - private int removeNode(int indx) + private int RemoveNode(int indx) { - int retVal = freeList[indx].GetNext(); - RarNode temp = tempRarNode; + int retVal = _freeList[indx].GetNext(); + RarNode temp = _tempRarNode; temp.Address = retVal; - freeList[indx].SetNext(temp.GetNext()); + _freeList[indx].SetNext(temp.GetNext()); return retVal; } - private int U2B(int NU) + private int U2B(int nu) { - return UNIT_SIZE * NU; + return UNIT_SIZE * nu; } /* memblockptr */ - private int MBPtr(int BasePtr, int Items) + private int MbPtr(int basePtr, int items) { - return (BasePtr + U2B(Items)); + return (basePtr + U2B(items)); } - private void splitBlock(int pv, int oldIndx, int newIndx) + private void SplitBlock(int pv, int oldIndx, int newIndx) { - int i, uDiff = indx2Units[oldIndx] - indx2Units[newIndx]; - int p = pv + U2B(indx2Units[newIndx]); - if (indx2Units[i = units2Indx[uDiff - 1]] != uDiff) + int i, uDiff = _indx2Units[oldIndx] - _indx2Units[newIndx]; + int p = pv + U2B(_indx2Units[newIndx]); + if (_indx2Units[i = _units2Indx[uDiff - 1]] != uDiff) { - insertNode(p, --i); - p += U2B(i = indx2Units[i]); + InsertNode(p, --i); + p += U2B(i = _indx2Units[i]); uDiff -= i; } - insertNode(p, units2Indx[uDiff - 1]); + InsertNode(p, _units2Indx[uDiff - 1]); } - public virtual void stopSubAllocator() + public virtual void StopSubAllocator() { - if (subAllocatorSize != 0) + if (_subAllocatorSize != 0) { - subAllocatorSize = 0; + _subAllocatorSize = 0; //ArrayFactory.BYTES_FACTORY.recycle(heap); - heap = null; - heapStart = 1; + _heap = null; + _heapStart = 1; // rarfree(HeapStart); // Free temp fields - tempRarNode = null; - tempRarMemBlock1 = null; - tempRarMemBlock2 = null; - tempRarMemBlock3 = null; + _tempRarNode = null; + _tempRarMemBlock1 = null; + _tempRarMemBlock2 = null; + _tempRarMemBlock3 = null; } } public virtual int GetAllocatedMemory() { - return subAllocatorSize; + return _subAllocatorSize; } - public virtual bool startSubAllocator(int SASize) + public virtual bool StartSubAllocator(int saSize) { - int t = SASize; - if (subAllocatorSize == t) + int t = saSize; + if (_subAllocatorSize == t) { return true; } - stopSubAllocator(); + StopSubAllocator(); int allocSize = t / FIXED_UNIT_SIZE * UNIT_SIZE + UNIT_SIZE; // adding space for freelist (needed for poiters) @@ -154,56 +154,56 @@ namespace SharpCompress.Compressors.PPMd.H int realAllocSize = 1 + allocSize + 4 * N_INDEXES; // adding space for an additional memblock - tempMemBlockPos = realAllocSize; - realAllocSize += RarMemBlock.size; + _tempMemBlockPos = realAllocSize; + realAllocSize += RarMemBlock.SIZE; - heap = new byte[realAllocSize]; - heapStart = 1; - heapEnd = heapStart + allocSize - UNIT_SIZE; - subAllocatorSize = t; + _heap = new byte[realAllocSize]; + _heapStart = 1; + _heapEnd = _heapStart + allocSize - UNIT_SIZE; + _subAllocatorSize = t; // Bug fixed - freeListPos = heapStart + allocSize; + _freeListPos = _heapStart + allocSize; //UPGRADE_ISSUE: The following fragment of code could not be parsed and was not converted. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1156'" //assert(realAllocSize - tempMemBlockPos == RarMemBlock.size): realAllocSize //+ + tempMemBlockPos + + RarMemBlock.size; // Init freeList - for (int i = 0, pos = freeListPos; i < freeList.Length; i++, pos += RarNode.size) + for (int i = 0, pos = _freeListPos; i < _freeList.Length; i++, pos += RarNode.SIZE) { - freeList[i] = new RarNode(heap); - freeList[i].Address = pos; + _freeList[i] = new RarNode(_heap); + _freeList[i].Address = pos; } // Init temp fields - tempRarNode = new RarNode(heap); - tempRarMemBlock1 = new RarMemBlock(heap); - tempRarMemBlock2 = new RarMemBlock(heap); - tempRarMemBlock3 = new RarMemBlock(heap); + _tempRarNode = new RarNode(_heap); + _tempRarMemBlock1 = new RarMemBlock(_heap); + _tempRarMemBlock2 = new RarMemBlock(_heap); + _tempRarMemBlock3 = new RarMemBlock(_heap); return true; } - private void glueFreeBlocks() + private void GlueFreeBlocks() { - RarMemBlock s0 = tempRarMemBlock1; - s0.Address = tempMemBlockPos; - RarMemBlock p = tempRarMemBlock2; - RarMemBlock p1 = tempRarMemBlock3; + RarMemBlock s0 = _tempRarMemBlock1; + s0.Address = _tempMemBlockPos; + RarMemBlock p = _tempRarMemBlock2; + RarMemBlock p1 = _tempRarMemBlock3; int i, k, sz; - if (loUnit != hiUnit) + if (_loUnit != _hiUnit) { - heap[loUnit] = 0; + _heap[_loUnit] = 0; } for (i = 0, s0.SetPrev(s0), s0.SetNext(s0); i < N_INDEXES; i++) { - while (freeList[i].GetNext() != 0) + while (_freeList[i].GetNext() != 0) { - p.Address = removeNode(i); // =(RAR_MEM_BLK*)RemoveNode(i); + p.Address = RemoveNode(i); // =(RAR_MEM_BLK*)RemoveNode(i); p.InsertAt(s0); // p->insertAt(&s0); p.Stamp = 0xFFFF; // p->Stamp=0xFFFF; - p.SetNU(indx2Units[i]); // p->NU=Indx2Units[i]; + p.SetNu(_indx2Units[i]); // p->NU=Indx2Units[i]; } } for (p.Address = s0.GetNext(); p.Address != s0.Address; p.Address = p.GetNext()) @@ -211,12 +211,12 @@ namespace SharpCompress.Compressors.PPMd.H // while ((p1=MBPtr(p,p->NU))->Stamp == 0xFFFF && int(p->NU)+p1->NU // < 0x10000) // Bug fixed - p1.Address = MBPtr(p.Address, p.GetNU()); - while (p1.Stamp == 0xFFFF && p.GetNU() + p1.GetNU() < 0x10000) + p1.Address = MbPtr(p.Address, p.GetNu()); + while (p1.Stamp == 0xFFFF && p.GetNu() + p1.GetNu() < 0x10000) { p1.Remove(); - p.SetNU(p.GetNU() + p1.GetNU()); // ->NU += p1->NU; - p1.Address = MBPtr(p.Address, p.GetNU()); + p.SetNu(p.GetNu() + p1.GetNu()); // ->NU += p1->NU; + p1.Address = MbPtr(p.Address, p.GetNu()); } } @@ -225,29 +225,29 @@ namespace SharpCompress.Compressors.PPMd.H p.Address = s0.GetNext(); while (p.Address != s0.Address) { - for (p.Remove(), sz = p.GetNU(); sz > 128; sz -= 128, p.Address = MBPtr(p.Address, 128)) + for (p.Remove(), sz = p.GetNu(); sz > 128; sz -= 128, p.Address = MbPtr(p.Address, 128)) { - insertNode(p.Address, N_INDEXES - 1); + InsertNode(p.Address, N_INDEXES - 1); } - if (indx2Units[i = units2Indx[sz - 1]] != sz) + if (_indx2Units[i = _units2Indx[sz - 1]] != sz) { - k = sz - indx2Units[--i]; - insertNode(MBPtr(p.Address, sz - k), k - 1); + k = sz - _indx2Units[--i]; + InsertNode(MbPtr(p.Address, sz - k), k - 1); } - insertNode(p.Address, i); + InsertNode(p.Address, i); p.Address = s0.GetNext(); } } - private int allocUnitsRare(int indx) + private int AllocUnitsRare(int indx) { - if (glueCount == 0) + if (_glueCount == 0) { - glueCount = 255; - glueFreeBlocks(); - if (freeList[indx].GetNext() != 0) + _glueCount = 255; + GlueFreeBlocks(); + if (_freeList[indx].GetNext() != 0) { - return removeNode(indx); + return RemoveNode(indx); } } int i = indx; @@ -255,151 +255,151 @@ namespace SharpCompress.Compressors.PPMd.H { if (++i == N_INDEXES) { - glueCount--; - i = U2B(indx2Units[indx]); - int j = FIXED_UNIT_SIZE * indx2Units[indx]; - if (fakeUnitsStart - pText > j) + _glueCount--; + i = U2B(_indx2Units[indx]); + int j = FIXED_UNIT_SIZE * _indx2Units[indx]; + if (_fakeUnitsStart - _pText > j) { - fakeUnitsStart -= j; - unitsStart -= i; - return unitsStart; + _fakeUnitsStart -= j; + _unitsStart -= i; + return _unitsStart; } return (0); } } - while (freeList[i].GetNext() == 0); - int retVal = removeNode(i); - splitBlock(retVal, i, indx); + while (_freeList[i].GetNext() == 0); + int retVal = RemoveNode(i); + SplitBlock(retVal, i, indx); return retVal; } - public virtual int allocUnits(int NU) + public virtual int AllocUnits(int nu) { - int indx = units2Indx[NU - 1]; - if (freeList[indx].GetNext() != 0) + int indx = _units2Indx[nu - 1]; + if (_freeList[indx].GetNext() != 0) { - return removeNode(indx); + return RemoveNode(indx); } - int retVal = loUnit; - loUnit += U2B(indx2Units[indx]); - if (loUnit <= hiUnit) + int retVal = _loUnit; + _loUnit += U2B(_indx2Units[indx]); + if (_loUnit <= _hiUnit) { return retVal; } - loUnit -= U2B(indx2Units[indx]); - return allocUnitsRare(indx); + _loUnit -= U2B(_indx2Units[indx]); + return AllocUnitsRare(indx); } - public virtual int allocContext() + public virtual int AllocContext() { - if (hiUnit != loUnit) + if (_hiUnit != _loUnit) { - return (hiUnit -= UNIT_SIZE); + return (_hiUnit -= UNIT_SIZE); } - if (freeList[0].GetNext() != 0) + if (_freeList[0].GetNext() != 0) { - return removeNode(0); + return RemoveNode(0); } - return allocUnitsRare(0); + return AllocUnitsRare(0); } - public virtual int expandUnits(int oldPtr, int OldNU) + public virtual int ExpandUnits(int oldPtr, int oldNu) { - int i0 = units2Indx[OldNU - 1]; - int i1 = units2Indx[OldNU - 1 + 1]; + int i0 = _units2Indx[oldNu - 1]; + int i1 = _units2Indx[oldNu - 1 + 1]; if (i0 == i1) { return oldPtr; } - int ptr = allocUnits(OldNU + 1); + int ptr = AllocUnits(oldNu + 1); if (ptr != 0) { // memcpy(ptr,OldPtr,U2B(OldNU)); - Array.Copy(heap, oldPtr, heap, ptr, U2B(OldNU)); - insertNode(oldPtr, i0); + Array.Copy(_heap, oldPtr, _heap, ptr, U2B(oldNu)); + InsertNode(oldPtr, i0); } return ptr; } - public virtual int shrinkUnits(int oldPtr, int oldNU, int newNU) + public virtual int ShrinkUnits(int oldPtr, int oldNu, int newNu) { // System.out.println("SubAllocator.shrinkUnits(" + OldPtr + ", " + // OldNU + ", " + NewNU + ")"); - int i0 = units2Indx[oldNU - 1]; - int i1 = units2Indx[newNU - 1]; + int i0 = _units2Indx[oldNu - 1]; + int i1 = _units2Indx[newNu - 1]; if (i0 == i1) { return oldPtr; } - if (freeList[i1].GetNext() != 0) + if (_freeList[i1].GetNext() != 0) { - int ptr = removeNode(i1); + int ptr = RemoveNode(i1); // memcpy(ptr,OldPtr,U2B(NewNU)); // for (int i = 0; i < U2B(NewNU); i++) { // heap[ptr + i] = heap[OldPtr + i]; // } - Array.Copy(heap, oldPtr, heap, ptr, U2B(newNU)); - insertNode(oldPtr, i0); + Array.Copy(_heap, oldPtr, _heap, ptr, U2B(newNu)); + InsertNode(oldPtr, i0); return ptr; } - splitBlock(oldPtr, i0, i1); + SplitBlock(oldPtr, i0, i1); return oldPtr; } - public virtual void freeUnits(int ptr, int OldNU) + public virtual void FreeUnits(int ptr, int oldNu) { - insertNode(ptr, units2Indx[OldNU - 1]); + InsertNode(ptr, _units2Indx[oldNu - 1]); } - public virtual void decPText(int dPText) + public virtual void DecPText(int dPText) { PText = PText - dPText; } - public virtual void initSubAllocator() + public virtual void InitSubAllocator() { int i, k; - Utility.Fill(heap, freeListPos, freeListPos + sizeOfFreeList(), (byte)0); + Utility.Fill(_heap, _freeListPos, _freeListPos + SizeOfFreeList(), (byte)0); - pText = heapStart; + _pText = _heapStart; - int size2 = FIXED_UNIT_SIZE * (subAllocatorSize / 8 / FIXED_UNIT_SIZE * 7); + int size2 = FIXED_UNIT_SIZE * (_subAllocatorSize / 8 / FIXED_UNIT_SIZE * 7); int realSize2 = size2 / FIXED_UNIT_SIZE * UNIT_SIZE; - int size1 = subAllocatorSize - size2; + int size1 = _subAllocatorSize - size2; int realSize1 = size1 / FIXED_UNIT_SIZE * UNIT_SIZE + size1 % FIXED_UNIT_SIZE; - hiUnit = heapStart + subAllocatorSize; - loUnit = unitsStart = heapStart + realSize1; - fakeUnitsStart = heapStart + size1; - hiUnit = loUnit + realSize2; + _hiUnit = _heapStart + _subAllocatorSize; + _loUnit = _unitsStart = _heapStart + realSize1; + _fakeUnitsStart = _heapStart + size1; + _hiUnit = _loUnit + realSize2; for (i = 0, k = 1; i < N1; i++, k += 1) { - indx2Units[i] = k & 0xff; + _indx2Units[i] = k & 0xff; } for (k++; i < N1 + N2; i++, k += 2) { - indx2Units[i] = k & 0xff; + _indx2Units[i] = k & 0xff; } for (k++; i < N1 + N2 + N3; i++, k += 3) { - indx2Units[i] = k & 0xff; + _indx2Units[i] = k & 0xff; } for (k++; i < (N1 + N2 + N3 + N4); i++, k += 4) { - indx2Units[i] = k & 0xff; + _indx2Units[i] = k & 0xff; } - for (glueCount = 0, k = 0, i = 0; k < 128; k++) + for (_glueCount = 0, k = 0, i = 0; k < 128; k++) { - i += ((indx2Units[i] < (k + 1)) ? 1 : 0); - units2Indx[k] = i & 0xff; + i += ((_indx2Units[i] < (k + 1)) ? 1 : 0); + _units2Indx[k] = i & 0xff; } } - private int sizeOfFreeList() + private int SizeOfFreeList() { - return freeList.Length * RarNode.size; + return _freeList.Length * RarNode.SIZE; } // Debug @@ -426,26 +426,26 @@ namespace SharpCompress.Compressors.PPMd.H StringBuilder buffer = new StringBuilder(); buffer.Append("SubAllocator["); buffer.Append("\n subAllocatorSize="); - buffer.Append(subAllocatorSize); + buffer.Append(_subAllocatorSize); buffer.Append("\n glueCount="); - buffer.Append(glueCount); + buffer.Append(_glueCount); buffer.Append("\n heapStart="); - buffer.Append(heapStart); + buffer.Append(_heapStart); buffer.Append("\n loUnit="); - buffer.Append(loUnit); + buffer.Append(_loUnit); buffer.Append("\n hiUnit="); - buffer.Append(hiUnit); + buffer.Append(_hiUnit); buffer.Append("\n pText="); - buffer.Append(pText); + buffer.Append(_pText); buffer.Append("\n unitsStart="); - buffer.Append(unitsStart); + buffer.Append(_unitsStart); buffer.Append("\n]"); return buffer.ToString(); } static SubAllocator() { - UNIT_SIZE = Math.Max(PPMContext.size, RarMemBlock.size); + UNIT_SIZE = Math.Max(PpmContext.SIZE, RarMemBlock.SIZE); } } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/PPMd/I1/Allocator.cs b/src/SharpCompress/Compressors/PPMd/I1/Allocator.cs index 5b23c422..1f8849aa 100644 --- a/src/SharpCompress/Compressors/PPMd/I1/Allocator.cs +++ b/src/SharpCompress/Compressors/PPMd/I1/Allocator.cs @@ -6,11 +6,11 @@ namespace SharpCompress.Compressors.PPMd.I1 /// of which also simply contain a single address value): internal class Allocator { - private const uint UnitSize = 12; - private const uint LocalOffset = 4; // reserve the first four bytes for Pointer.Zero - private const uint NodeOffset = LocalOffset + MemoryNode.Size; // reserve space for a single memory node + private const uint UNIT_SIZE = 12; + private const uint LOCAL_OFFSET = 4; // reserve the first four bytes for Pointer.Zero + private const uint NODE_OFFSET = LOCAL_OFFSET + MemoryNode.SIZE; // reserve space for a single memory node - private const uint HeapOffset = NodeOffset + IndexCount * MemoryNode.Size; + private const uint HEAP_OFFSET = NODE_OFFSET + INDEX_COUNT * MemoryNode.SIZE; // reserve space for the array of memory nodes @@ -18,21 +18,21 @@ namespace SharpCompress.Compressors.PPMd.I1 private const uint N2 = 4; private const uint N3 = 4; private const uint N4 = (128 + 3 - 1 * N1 - 2 * N2 - 3 * N3) / 4; - private const uint IndexCount = N1 + N2 + N3 + N4; + private const uint INDEX_COUNT = N1 + N2 + N3 + N4; - private static readonly byte[] indexToUnits; - private static readonly byte[] unitsToIndex; + private static readonly byte[] INDEX_TO_UNITS; + private static readonly byte[] UNITS_TO_INDEX; - public uint AllocatorSize; - public uint GlueCount; - public Pointer BaseUnit; - public Pointer LowUnit; - public Pointer HighUnit; - public Pointer Text; - public Pointer Heap; - public MemoryNode[] MemoryNodes; + public uint _allocatorSize; + public uint _glueCount; + public Pointer _baseUnit; + public Pointer _lowUnit; + public Pointer _highUnit; + public Pointer _text; + public Pointer _heap; + public MemoryNode[] _memoryNodes; - public byte[] Memory; + public byte[] _memory; /// /// Initializes static read-only arrays used by the . @@ -47,26 +47,26 @@ namespace SharpCompress.Compressors.PPMd.I1 uint index; uint unitCount; - indexToUnits = new byte[IndexCount]; + INDEX_TO_UNITS = new byte[INDEX_COUNT]; for (index = 0, unitCount = 1; index < N1; index++, unitCount += 1) { - indexToUnits[index] = (byte)unitCount; + INDEX_TO_UNITS[index] = (byte)unitCount; } for (unitCount++; index < N1 + N2; index++, unitCount += 2) { - indexToUnits[index] = (byte)unitCount; + INDEX_TO_UNITS[index] = (byte)unitCount; } for (unitCount++; index < N1 + N2 + N3; index++, unitCount += 3) { - indexToUnits[index] = (byte)unitCount; + INDEX_TO_UNITS[index] = (byte)unitCount; } for (unitCount++; index < N1 + N2 + N3 + N4; index++, unitCount += 4) { - indexToUnits[index] = (byte)unitCount; + INDEX_TO_UNITS[index] = (byte)unitCount; } // Construct the static units to index lookup array. It will contain the following values. @@ -76,12 +76,12 @@ namespace SharpCompress.Compressors.PPMd.I1 // 22 22 22 22 23 23 23 23 24 24 24 24 25 25 25 25 26 26 26 26 27 27 27 27 28 28 28 28 29 29 29 29 // 30 30 30 30 31 31 31 31 32 32 32 32 33 33 33 33 34 34 34 34 35 35 35 35 36 36 36 36 37 37 37 37 - unitsToIndex = new byte[128]; + UNITS_TO_INDEX = new byte[128]; for (unitCount = index = 0; unitCount < 128; unitCount++) { - index += (uint)((indexToUnits[index] < unitCount + 1) ? 1 : 0); - unitsToIndex[unitCount] = (byte)index; + index += (uint)((INDEX_TO_UNITS[index] < unitCount + 1) ? 1 : 0); + UNITS_TO_INDEX[unitCount] = (byte)index; } } @@ -89,7 +89,7 @@ namespace SharpCompress.Compressors.PPMd.I1 public Allocator() { - MemoryNodes = new MemoryNode[IndexCount]; + _memoryNodes = new MemoryNode[INDEX_COUNT]; } /// @@ -98,23 +98,23 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public void Initialize() { - for (int index = 0; index < IndexCount; index++) + for (int index = 0; index < INDEX_COUNT; index++) { - MemoryNodes[index] = new MemoryNode((uint)(NodeOffset + index * MemoryNode.Size), Memory); - MemoryNodes[index].Stamp = 0; - MemoryNodes[index].Next = MemoryNode.Zero; - MemoryNodes[index].UnitCount = 0; + _memoryNodes[index] = new MemoryNode((uint)(NODE_OFFSET + index * MemoryNode.SIZE), _memory); + _memoryNodes[index].Stamp = 0; + _memoryNodes[index].Next = MemoryNode.ZERO; + _memoryNodes[index].UnitCount = 0; } - Text = Heap; + _text = _heap; - uint difference = UnitSize * (AllocatorSize / 8 / UnitSize * 7); + uint difference = UNIT_SIZE * (_allocatorSize / 8 / UNIT_SIZE * 7); - HighUnit = Heap + AllocatorSize; - LowUnit = HighUnit - difference; - BaseUnit = HighUnit - difference; + _highUnit = _heap + _allocatorSize; + _lowUnit = _highUnit - difference; + _baseUnit = _highUnit - difference; - GlueCount = 0; + _glueCount = 0; } /// @@ -127,12 +127,12 @@ namespace SharpCompress.Compressors.PPMd.I1 public void Start(int allocatorSize) { uint size = (uint)allocatorSize; - if (AllocatorSize != size) + if (_allocatorSize != size) { Stop(); - Memory = new byte[HeapOffset + size]; // the single, large array of bytes - Heap = new Pointer(HeapOffset, Memory); // reserve bytes in the range 0 .. HeapOffset - 1 - AllocatorSize = size; + _memory = new byte[HEAP_OFFSET + size]; // the single, large array of bytes + _heap = new Pointer(HEAP_OFFSET, _memory); // reserve bytes in the range 0 .. HeapOffset - 1 + _allocatorSize = size; } } @@ -145,11 +145,11 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public void Stop() { - if (AllocatorSize != 0) + if (_allocatorSize != 0) { - AllocatorSize = 0; - Memory = null; - Heap = Pointer.Zero; + _allocatorSize = 0; + _memory = null; + _heap = Pointer.ZERO; } } @@ -159,36 +159,36 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public uint GetMemoryUsed() { - uint memoryUsed = AllocatorSize - (HighUnit - LowUnit) - (BaseUnit - Text); - for (uint index = 0; index < IndexCount; index++) + uint memoryUsed = _allocatorSize - (_highUnit - _lowUnit) - (_baseUnit - _text); + for (uint index = 0; index < INDEX_COUNT; index++) { - memoryUsed -= UnitSize * indexToUnits[index] * MemoryNodes[index].Stamp; + memoryUsed -= UNIT_SIZE * INDEX_TO_UNITS[index] * _memoryNodes[index].Stamp; } return memoryUsed; } /// - /// Allocate a given number of units from the single, large array. Each unit is bytes + /// Allocate a given number of units from the single, large array. Each unit is bytes /// in size. /// /// /// public Pointer AllocateUnits(uint unitCount) { - uint index = unitsToIndex[unitCount - 1]; - if (MemoryNodes[index].Available) + uint index = UNITS_TO_INDEX[unitCount - 1]; + if (_memoryNodes[index].Available) { - return MemoryNodes[index].Remove(); + return _memoryNodes[index].Remove(); } - Pointer allocatedBlock = LowUnit; - LowUnit += indexToUnits[index] * UnitSize; - if (LowUnit <= HighUnit) + Pointer allocatedBlock = _lowUnit; + _lowUnit += INDEX_TO_UNITS[index] * UNIT_SIZE; + if (_lowUnit <= _highUnit) { return allocatedBlock; } - LowUnit -= indexToUnits[index] * UnitSize; + _lowUnit -= INDEX_TO_UNITS[index] * UNIT_SIZE; return AllocateUnitsRare(index); } @@ -198,13 +198,13 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public Pointer AllocateContext() { - if (HighUnit != LowUnit) + if (_highUnit != _lowUnit) { - return (HighUnit -= UnitSize); + return (_highUnit -= UNIT_SIZE); } - if (MemoryNodes[0].Available) + if (_memoryNodes[0].Available) { - return MemoryNodes[0].Remove(); + return _memoryNodes[0].Remove(); } return AllocateUnitsRare(0); } @@ -217,8 +217,8 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public Pointer ExpandUnits(Pointer oldPointer, uint oldUnitCount) { - uint oldIndex = unitsToIndex[oldUnitCount - 1]; - uint newIndex = unitsToIndex[oldUnitCount]; + uint oldIndex = UNITS_TO_INDEX[oldUnitCount - 1]; + uint newIndex = UNITS_TO_INDEX[oldUnitCount]; if (oldIndex == newIndex) { @@ -227,10 +227,10 @@ namespace SharpCompress.Compressors.PPMd.I1 Pointer pointer = AllocateUnits(oldUnitCount + 1); - if (pointer != Pointer.Zero) + if (pointer != Pointer.ZERO) { CopyUnits(pointer, oldPointer, oldUnitCount); - MemoryNodes[oldIndex].Insert(oldPointer, oldUnitCount); + _memoryNodes[oldIndex].Insert(oldPointer, oldUnitCount); } return pointer; @@ -245,19 +245,19 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public Pointer ShrinkUnits(Pointer oldPointer, uint oldUnitCount, uint newUnitCount) { - uint oldIndex = unitsToIndex[oldUnitCount - 1]; - uint newIndex = unitsToIndex[newUnitCount - 1]; + uint oldIndex = UNITS_TO_INDEX[oldUnitCount - 1]; + uint newIndex = UNITS_TO_INDEX[newUnitCount - 1]; if (oldIndex == newIndex) { return oldPointer; } - if (MemoryNodes[newIndex].Available) + if (_memoryNodes[newIndex].Available) { - Pointer pointer = MemoryNodes[newIndex].Remove(); + Pointer pointer = _memoryNodes[newIndex].Remove(); CopyUnits(pointer, oldPointer, newUnitCount); - MemoryNodes[oldIndex].Insert(oldPointer, indexToUnits[oldIndex]); + _memoryNodes[oldIndex].Insert(oldPointer, INDEX_TO_UNITS[oldIndex]); return pointer; } SplitBlock(oldPointer, oldIndex, newIndex); @@ -272,44 +272,44 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public void FreeUnits(Pointer pointer, uint unitCount) { - uint index = unitsToIndex[unitCount - 1]; - MemoryNodes[index].Insert(pointer, indexToUnits[index]); + uint index = UNITS_TO_INDEX[unitCount - 1]; + _memoryNodes[index].Insert(pointer, INDEX_TO_UNITS[index]); } public void SpecialFreeUnits(Pointer pointer) { - if (pointer != BaseUnit) + if (pointer != _baseUnit) { - MemoryNodes[0].Insert(pointer, 1); + _memoryNodes[0].Insert(pointer, 1); } else { MemoryNode memoryNode = pointer; memoryNode.Stamp = uint.MaxValue; - BaseUnit += UnitSize; + _baseUnit += UNIT_SIZE; } } public Pointer MoveUnitsUp(Pointer oldPointer, uint unitCount) { - uint index = unitsToIndex[unitCount - 1]; + uint index = UNITS_TO_INDEX[unitCount - 1]; - if (oldPointer > BaseUnit + 16 * 1024 || oldPointer > MemoryNodes[index].Next) + if (oldPointer > _baseUnit + 16 * 1024 || oldPointer > _memoryNodes[index].Next) { return oldPointer; } - Pointer pointer = MemoryNodes[index].Remove(); + Pointer pointer = _memoryNodes[index].Remove(); CopyUnits(pointer, oldPointer, unitCount); - unitCount = indexToUnits[index]; + unitCount = INDEX_TO_UNITS[index]; - if (oldPointer != BaseUnit) + if (oldPointer != _baseUnit) { - MemoryNodes[index].Insert(oldPointer, unitCount); + _memoryNodes[index].Insert(oldPointer, unitCount); } else { - BaseUnit += unitCount * UnitSize; + _baseUnit += unitCount * UNIT_SIZE; } return pointer; @@ -322,23 +322,23 @@ namespace SharpCompress.Compressors.PPMd.I1 public void ExpandText() { MemoryNode memoryNode; - uint[] counts = new uint[IndexCount]; + uint[] counts = new uint[INDEX_COUNT]; - while ((memoryNode = BaseUnit).Stamp == uint.MaxValue) + while ((memoryNode = _baseUnit).Stamp == uint.MaxValue) { - BaseUnit = memoryNode + memoryNode.UnitCount; - counts[unitsToIndex[memoryNode.UnitCount - 1]]++; + _baseUnit = memoryNode + memoryNode.UnitCount; + counts[UNITS_TO_INDEX[memoryNode.UnitCount - 1]]++; memoryNode.Stamp = 0; } - for (uint index = 0; index < IndexCount; index++) + for (uint index = 0; index < INDEX_COUNT; index++) { - for (memoryNode = MemoryNodes[index]; counts[index] != 0; memoryNode = memoryNode.Next) + for (memoryNode = _memoryNodes[index]; counts[index] != 0; memoryNode = memoryNode.Next) { while (memoryNode.Next.Stamp == 0) { memoryNode.Unlink(); - MemoryNodes[index].Stamp--; + _memoryNodes[index].Stamp--; if (--counts[index] == 0) { break; @@ -354,73 +354,73 @@ namespace SharpCompress.Compressors.PPMd.I1 private Pointer AllocateUnitsRare(uint index) { - if (GlueCount == 0) + if (_glueCount == 0) { GlueFreeBlocks(); - if (MemoryNodes[index].Available) + if (_memoryNodes[index].Available) { - return MemoryNodes[index].Remove(); + return _memoryNodes[index].Remove(); } } uint oldIndex = index; do { - if (++oldIndex == IndexCount) + if (++oldIndex == INDEX_COUNT) { - GlueCount--; - oldIndex = indexToUnits[index] * UnitSize; - return (BaseUnit - Text > oldIndex) ? (BaseUnit -= oldIndex) : Pointer.Zero; + _glueCount--; + oldIndex = INDEX_TO_UNITS[index] * UNIT_SIZE; + return (_baseUnit - _text > oldIndex) ? (_baseUnit -= oldIndex) : Pointer.ZERO; } } - while (!MemoryNodes[oldIndex].Available); + while (!_memoryNodes[oldIndex].Available); - Pointer allocatedBlock = MemoryNodes[oldIndex].Remove(); + Pointer allocatedBlock = _memoryNodes[oldIndex].Remove(); SplitBlock(allocatedBlock, oldIndex, index); return allocatedBlock; } private void SplitBlock(Pointer pointer, uint oldIndex, uint newIndex) { - uint unitCountDifference = (uint)(indexToUnits[oldIndex] - indexToUnits[newIndex]); - Pointer newPointer = pointer + indexToUnits[newIndex] * UnitSize; + uint unitCountDifference = (uint)(INDEX_TO_UNITS[oldIndex] - INDEX_TO_UNITS[newIndex]); + Pointer newPointer = pointer + INDEX_TO_UNITS[newIndex] * UNIT_SIZE; - uint index = unitsToIndex[unitCountDifference - 1]; - if (indexToUnits[index] != unitCountDifference) + uint index = UNITS_TO_INDEX[unitCountDifference - 1]; + if (INDEX_TO_UNITS[index] != unitCountDifference) { - uint unitCount = indexToUnits[--index]; - MemoryNodes[index].Insert(newPointer, unitCount); - newPointer += unitCount * UnitSize; + uint unitCount = INDEX_TO_UNITS[--index]; + _memoryNodes[index].Insert(newPointer, unitCount); + newPointer += unitCount * UNIT_SIZE; unitCountDifference -= unitCount; } - MemoryNodes[unitsToIndex[unitCountDifference - 1]].Insert(newPointer, unitCountDifference); + _memoryNodes[UNITS_TO_INDEX[unitCountDifference - 1]].Insert(newPointer, unitCountDifference); } private void GlueFreeBlocks() { - MemoryNode memoryNode = new MemoryNode(LocalOffset, Memory); + MemoryNode memoryNode = new MemoryNode(LOCAL_OFFSET, _memory); memoryNode.Stamp = 0; - memoryNode.Next = MemoryNode.Zero; + memoryNode.Next = MemoryNode.ZERO; memoryNode.UnitCount = 0; MemoryNode memoryNode0; MemoryNode memoryNode1; MemoryNode memoryNode2; - if (LowUnit != HighUnit) + if (_lowUnit != _highUnit) { - LowUnit[0] = 0; + _lowUnit[0] = 0; } // Find all unused memory nodes. memoryNode1 = memoryNode; - for (uint index = 0; index < IndexCount; index++) + for (uint index = 0; index < INDEX_COUNT; index++) { - while (MemoryNodes[index].Available) + while (_memoryNodes[index].Available) { - memoryNode0 = MemoryNodes[index].Remove(); + memoryNode0 = _memoryNodes[index].Remove(); if (memoryNode0.UnitCount != 0) { while ((memoryNode2 = memoryNode0 + memoryNode0.UnitCount).Stamp == uint.MaxValue) @@ -444,22 +444,22 @@ namespace SharpCompress.Compressors.PPMd.I1 { for (; unitCount > 128; unitCount -= 128, memoryNode0 += 128) { - MemoryNodes[IndexCount - 1].Insert(memoryNode0, 128); + _memoryNodes[INDEX_COUNT - 1].Insert(memoryNode0, 128); } - uint index = unitsToIndex[unitCount - 1]; - if (indexToUnits[index] != unitCount) + uint index = UNITS_TO_INDEX[unitCount - 1]; + if (INDEX_TO_UNITS[index] != unitCount) { - uint unitCountDifference = unitCount - indexToUnits[--index]; - MemoryNodes[unitCountDifference - 1].Insert(memoryNode0 + (unitCount - unitCountDifference), + uint unitCountDifference = unitCount - INDEX_TO_UNITS[--index]; + _memoryNodes[unitCountDifference - 1].Insert(memoryNode0 + (unitCount - unitCountDifference), unitCountDifference); } - MemoryNodes[index].Insert(memoryNode0, indexToUnits[index]); + _memoryNodes[index].Insert(memoryNode0, INDEX_TO_UNITS[index]); } } - GlueCount = 1 << 13; + _glueCount = 1 << 13; } private void CopyUnits(Pointer target, Pointer source, uint unitCount) @@ -478,8 +478,8 @@ namespace SharpCompress.Compressors.PPMd.I1 target[9] = source[9]; target[10] = source[10]; target[11] = source[11]; - target += UnitSize; - source += UnitSize; + target += UNIT_SIZE; + source += UNIT_SIZE; } while (--unitCount != 0); } diff --git a/src/SharpCompress/Compressors/PPMd/I1/Coder.cs b/src/SharpCompress/Compressors/PPMd/I1/Coder.cs index 9dbe9cfc..a5c0d862 100644 --- a/src/SharpCompress/Compressors/PPMd/I1/Coder.cs +++ b/src/SharpCompress/Compressors/PPMd/I1/Coder.cs @@ -11,94 +11,94 @@ namespace SharpCompress.Compressors.PPMd.I1 /// /// /// Note that in most cases fields are used rather than properties for performance reasons (for example, - /// is a field rather than a property). + /// is a field rather than a property). /// internal class Coder { - private const uint RangeTop = 1 << 24; - private const uint RangeBottom = 1 << 15; - private uint low; - private uint code; - private uint range; + private const uint RANGE_TOP = 1 << 24; + private const uint RANGE_BOTTOM = 1 << 15; + private uint _low; + private uint _code; + private uint _range; - public uint LowCount; - public uint HighCount; - public uint Scale; + public uint _lowCount; + public uint _highCount; + public uint _scale; public void RangeEncoderInitialize() { - low = 0; - range = uint.MaxValue; + _low = 0; + _range = uint.MaxValue; } public void RangeEncoderNormalize(Stream stream) { - while ((low ^ (low + range)) < RangeTop || - range < RangeBottom && ((range = (uint)-low & (RangeBottom - 1)) != 0 || true)) + while ((_low ^ (_low + _range)) < RANGE_TOP || + _range < RANGE_BOTTOM && ((_range = (uint)-_low & (RANGE_BOTTOM - 1)) != 0 || true)) { - stream.WriteByte((byte)(low >> 24)); - range <<= 8; - low <<= 8; + stream.WriteByte((byte)(_low >> 24)); + _range <<= 8; + _low <<= 8; } } public void RangeEncodeSymbol() { - low += LowCount * (range /= Scale); - range *= HighCount - LowCount; + _low += _lowCount * (_range /= _scale); + _range *= _highCount - _lowCount; } public void RangeShiftEncodeSymbol(int rangeShift) { - low += LowCount * (range >>= rangeShift); - range *= HighCount - LowCount; + _low += _lowCount * (_range >>= rangeShift); + _range *= _highCount - _lowCount; } public void RangeEncoderFlush(Stream stream) { for (uint index = 0; index < 4; index++) { - stream.WriteByte((byte)(low >> 24)); - low <<= 8; + stream.WriteByte((byte)(_low >> 24)); + _low <<= 8; } } public void RangeDecoderInitialize(Stream stream) { - low = 0; - code = 0; - range = uint.MaxValue; + _low = 0; + _code = 0; + _range = uint.MaxValue; for (uint index = 0; index < 4; index++) { - code = (code << 8) | (byte)stream.ReadByte(); + _code = (_code << 8) | (byte)stream.ReadByte(); } } public void RangeDecoderNormalize(Stream stream) { - while ((low ^ (low + range)) < RangeTop || - range < RangeBottom && ((range = (uint)-low & (RangeBottom - 1)) != 0 || true)) + while ((_low ^ (_low + _range)) < RANGE_TOP || + _range < RANGE_BOTTOM && ((_range = (uint)-_low & (RANGE_BOTTOM - 1)) != 0 || true)) { - code = (code << 8) | (byte)stream.ReadByte(); - range <<= 8; - low <<= 8; + _code = (_code << 8) | (byte)stream.ReadByte(); + _range <<= 8; + _low <<= 8; } } public uint RangeGetCurrentCount() { - return (code - low) / (range /= Scale); + return (_code - _low) / (_range /= _scale); } public uint RangeGetCurrentShiftCount(int rangeShift) { - return (code - low) / (range >>= rangeShift); + return (_code - _low) / (_range >>= rangeShift); } public void RangeRemoveSubrange() { - low += range * LowCount; - range *= HighCount - LowCount; + _low += _range * _lowCount; + _range *= _highCount - _lowCount; } } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/PPMd/I1/MemoryNode.cs b/src/SharpCompress/Compressors/PPMd/I1/MemoryNode.cs index 23a08643..cded381e 100644 --- a/src/SharpCompress/Compressors/PPMd/I1/MemoryNode.cs +++ b/src/SharpCompress/Compressors/PPMd/I1/MemoryNode.cs @@ -7,9 +7,9 @@ namespace SharpCompress.Compressors.PPMd.I1 { /// - /// A structure containing a single address. The address represents a location in the - /// array. That location in the array contains information itself describing a section - /// of the array (ie. a block of memory). + /// A structure containing a single address. The address represents a location in the + /// array. That location in the array contains information itself describing a section + /// of the array (ie. a block of memory). /// /// /// @@ -24,23 +24,23 @@ namespace SharpCompress.Compressors.PPMd.I1 /// 4 UnitCount /// /// - /// Note that is a field rather than a property for performance reasons. + /// Note that is a field rather than a property for performance reasons. /// /// internal struct MemoryNode { - public uint Address; - public byte[] Memory; - public static readonly MemoryNode Zero = new MemoryNode(0, null); - public const int Size = 12; + public uint _address; + public byte[] _memory; + public static readonly MemoryNode ZERO = new MemoryNode(0, null); + public const int SIZE = 12; /// /// Initializes a new instance of the structure. /// public MemoryNode(uint address, byte[] memory) { - Address = address; - Memory = memory; + _address = address; + _memory = memory; } /// @@ -48,14 +48,14 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public uint Stamp { - get => Memory[Address] | ((uint)Memory[Address + 1]) << 8 | ((uint)Memory[Address + 2]) << 16 | - ((uint)Memory[Address + 3]) << 24; + get => _memory[_address] | ((uint)_memory[_address + 1]) << 8 | ((uint)_memory[_address + 2]) << 16 | + ((uint)_memory[_address + 3]) << 24; set { - Memory[Address] = (byte)value; - Memory[Address + 1] = (byte)(value >> 8); - Memory[Address + 2] = (byte)(value >> 16); - Memory[Address + 3] = (byte)(value >> 24); + _memory[_address] = (byte)value; + _memory[_address + 1] = (byte)(value >> 8); + _memory[_address + 2] = (byte)(value >> 16); + _memory[_address + 3] = (byte)(value >> 24); } } @@ -65,14 +65,14 @@ namespace SharpCompress.Compressors.PPMd.I1 public MemoryNode Next { get => new MemoryNode( - Memory[Address + 4] | ((uint)Memory[Address + 5]) << 8 | - ((uint)Memory[Address + 6]) << 16 | ((uint)Memory[Address + 7]) << 24, Memory); + _memory[_address + 4] | ((uint)_memory[_address + 5]) << 8 | + ((uint)_memory[_address + 6]) << 16 | ((uint)_memory[_address + 7]) << 24, _memory); set { - Memory[Address + 4] = (byte)value.Address; - Memory[Address + 5] = (byte)(value.Address >> 8); - Memory[Address + 6] = (byte)(value.Address >> 16); - Memory[Address + 7] = (byte)(value.Address >> 24); + _memory[_address + 4] = (byte)value._address; + _memory[_address + 5] = (byte)(value._address >> 8); + _memory[_address + 6] = (byte)(value._address >> 16); + _memory[_address + 7] = (byte)(value._address >> 24); } } @@ -81,21 +81,21 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public uint UnitCount { - get => Memory[Address + 8] | ((uint)Memory[Address + 9]) << 8 | - ((uint)Memory[Address + 10]) << 16 | ((uint)Memory[Address + 11]) << 24; + get => _memory[_address + 8] | ((uint)_memory[_address + 9]) << 8 | + ((uint)_memory[_address + 10]) << 16 | ((uint)_memory[_address + 11]) << 24; set { - Memory[Address + 8] = (byte)value; - Memory[Address + 9] = (byte)(value >> 8); - Memory[Address + 10] = (byte)(value >> 16); - Memory[Address + 11] = (byte)(value >> 24); + _memory[_address + 8] = (byte)value; + _memory[_address + 9] = (byte)(value >> 8); + _memory[_address + 10] = (byte)(value >> 16); + _memory[_address + 11] = (byte)(value >> 24); } } /// /// Gets whether there is a next memory node available. /// - public bool Available => Next.Address != 0; + public bool Available => Next._address != 0; /// /// Link in the provided memory node. @@ -147,7 +147,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static implicit operator MemoryNode(Pointer pointer) { - return new MemoryNode(pointer.Address, pointer.Memory); + return new MemoryNode(pointer._address, pointer._memory); } /// @@ -158,7 +158,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static MemoryNode operator +(MemoryNode memoryNode, int offset) { - memoryNode.Address = (uint)(memoryNode.Address + offset * Size); + memoryNode._address = (uint)(memoryNode._address + offset * SIZE); return memoryNode; } @@ -170,7 +170,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static MemoryNode operator +(MemoryNode memoryNode, uint offset) { - memoryNode.Address += offset * Size; + memoryNode._address += offset * SIZE; return memoryNode; } @@ -182,7 +182,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static MemoryNode operator -(MemoryNode memoryNode, int offset) { - memoryNode.Address = (uint)(memoryNode.Address - offset * Size); + memoryNode._address = (uint)(memoryNode._address - offset * SIZE); return memoryNode; } @@ -194,7 +194,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static MemoryNode operator -(MemoryNode memoryNode, uint offset) { - memoryNode.Address -= offset * Size; + memoryNode._address -= offset * SIZE; return memoryNode; } @@ -206,7 +206,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator ==(MemoryNode memoryNode1, MemoryNode memoryNode2) { - return memoryNode1.Address == memoryNode2.Address; + return memoryNode1._address == memoryNode2._address; } /// @@ -217,7 +217,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator !=(MemoryNode memoryNode1, MemoryNode memoryNode2) { - return memoryNode1.Address != memoryNode2.Address; + return memoryNode1._address != memoryNode2._address; } /// @@ -230,7 +230,7 @@ namespace SharpCompress.Compressors.PPMd.I1 if (obj is MemoryNode) { MemoryNode memoryNode = (MemoryNode)obj; - return memoryNode.Address == Address; + return memoryNode._address == _address; } return base.Equals(obj); } @@ -241,7 +241,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// A 32-bit signed integer that is the hash code for this instance. public override int GetHashCode() { - return Address.GetHashCode(); + return _address.GetHashCode(); } } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/PPMd/I1/Model.cs b/src/SharpCompress/Compressors/PPMd/I1/Model.cs index a71939cd..11d065c1 100644 --- a/src/SharpCompress/Compressors/PPMd/I1/Model.cs +++ b/src/SharpCompress/Compressors/PPMd/I1/Model.cs @@ -15,50 +15,50 @@ namespace SharpCompress.Compressors.PPMd.I1 /// internal partial class Model { - public const uint Signature = 0x84acaf8fU; - public const char Variant = 'I'; - public const int MaximumOrder = 16; // maximum allowed model order + public const uint SIGNATURE = 0x84acaf8fU; + public const char VARIANT = 'I'; + public const int MAXIMUM_ORDER = 16; // maximum allowed model order - private const byte UpperFrequency = 5; - private const byte IntervalBitCount = 7; - private const byte PeriodBitCount = 7; - private const byte TotalBitCount = IntervalBitCount + PeriodBitCount; - private const uint Interval = 1 << IntervalBitCount; - private const uint BinaryScale = 1 << TotalBitCount; - private const uint MaximumFrequency = 124; - private const uint OrderBound = 9; + private const byte UPPER_FREQUENCY = 5; + private const byte INTERVAL_BIT_COUNT = 7; + private const byte PERIOD_BIT_COUNT = 7; + private const byte TOTAL_BIT_COUNT = INTERVAL_BIT_COUNT + PERIOD_BIT_COUNT; + private const uint INTERVAL = 1 << INTERVAL_BIT_COUNT; + private const uint BINARY_SCALE = 1 << TOTAL_BIT_COUNT; + private const uint MAXIMUM_FREQUENCY = 124; + private const uint ORDER_BOUND = 9; - private readonly See2Context[,] see2Contexts; - private readonly See2Context emptySee2Context; - private PpmContext maximumContext; - private readonly ushort[,] binarySummary = new ushort[25, 64]; // binary SEE-contexts - private readonly byte[] numberStatisticsToBinarySummaryIndex = new byte[256]; - private readonly byte[] probabilities = new byte[260]; - private readonly byte[] characterMask = new byte[256]; - private byte escapeCount; - private int modelOrder; - private int orderFall; - private int initialEscape; - private int initialRunLength; - private int runLength; - private byte previousSuccess; - private byte numberMasked; - private ModelRestorationMethod method; - private PpmState foundState; // found next state transition + private readonly See2Context[,] _see2Contexts; + private readonly See2Context _emptySee2Context; + private PpmContext _maximumContext; + private readonly ushort[,] _binarySummary = new ushort[25, 64]; // binary SEE-contexts + private readonly byte[] _numberStatisticsToBinarySummaryIndex = new byte[256]; + private readonly byte[] _probabilities = new byte[260]; + private readonly byte[] _characterMask = new byte[256]; + private byte _escapeCount; + private int _modelOrder; + private int _orderFall; + private int _initialEscape; + private int _initialRunLength; + private int _runLength; + private byte _previousSuccess; + private byte _numberMasked; + private ModelRestorationMethod _method; + private PpmState _foundState; // found next state transition - private Allocator Allocator; - private Coder Coder; - private PpmContext minimumContext; - private byte numberStatistics; - private readonly PpmState[] decodeStates = new PpmState[256]; + private Allocator _allocator; + private Coder _coder; + private PpmContext _minimumContext; + private byte _numberStatistics; + private readonly PpmState[] _decodeStates = new PpmState[256]; - private static readonly ushort[] InitialBinaryEscapes = + private static readonly ushort[] INITIAL_BINARY_ESCAPES = { 0x3CDD, 0x1F3F, 0x59BF, 0x48F3, 0x64A1, 0x5ABC, 0x6632, 0x6051 }; - private static readonly byte[] ExponentialEscapes = {25, 14, 9, 7, 5, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 2}; + private static readonly byte[] EXPONENTIAL_ESCAPES = {25, 14, 9, 7, 5, 5, 4, 4, 4, 3, 3, 3, 2, 2, 2, 2}; #region Public Methods @@ -75,15 +75,15 @@ namespace SharpCompress.Compressors.PPMd.I1 // 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 // 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 - numberStatisticsToBinarySummaryIndex[0] = 2 * 0; - numberStatisticsToBinarySummaryIndex[1] = 2 * 1; + _numberStatisticsToBinarySummaryIndex[0] = 2 * 0; + _numberStatisticsToBinarySummaryIndex[1] = 2 * 1; for (int index = 2; index < 11; index++) { - numberStatisticsToBinarySummaryIndex[index] = 2 * 2; + _numberStatisticsToBinarySummaryIndex[index] = 2 * 2; } for (int index = 11; index < 256; index++) { - numberStatisticsToBinarySummaryIndex[index] = 2 * 3; + _numberStatisticsToBinarySummaryIndex[index] = 2 * 3; } // Construct the probability table. Initially it will contain the following values (depending on the value of @@ -101,16 +101,16 @@ namespace SharpCompress.Compressors.PPMd.I1 uint count = 1; uint step = 1; - uint probability = UpperFrequency; + uint probability = UPPER_FREQUENCY; - for (int index = 0; index < UpperFrequency; index++) + for (int index = 0; index < UPPER_FREQUENCY; index++) { - probabilities[index] = (byte)index; + _probabilities[index] = (byte)index; } - for (int index = UpperFrequency; index < 260; index++) + for (int index = UPPER_FREQUENCY; index < 260; index++) { - probabilities[index] = (byte)probability; + _probabilities[index] = (byte)probability; count--; if (count == 0) { @@ -122,21 +122,21 @@ namespace SharpCompress.Compressors.PPMd.I1 // Create the context array. - see2Contexts = new See2Context[24, 32]; + _see2Contexts = new See2Context[24, 32]; for (int index1 = 0; index1 < 24; index1++) { for (int index2 = 0; index2 < 32; index2++) { - see2Contexts[index1, index2] = new See2Context(); + _see2Contexts[index1, index2] = new See2Context(); } } // Set the signature (identifying the algorithm). - emptySee2Context = new See2Context(); - emptySee2Context.Summary = (ushort)(Signature & 0x0000ffff); - emptySee2Context.Shift = (byte)((Signature >> 16) & 0x000000ff); - emptySee2Context.Count = (byte)(Signature >> 24); + _emptySee2Context = new See2Context(); + _emptySee2Context._summary = (ushort)(SIGNATURE & 0x0000ffff); + _emptySee2Context._shift = (byte)((SIGNATURE >> 16) & 0x000000ff); + _emptySee2Context._count = (byte)(SIGNATURE >> 24); } /// @@ -160,19 +160,19 @@ namespace SharpCompress.Compressors.PPMd.I1 internal Coder EncodeStart(PpmdProperties properties) { - Allocator = properties.Allocator; - Coder = new Coder(); - Coder.RangeEncoderInitialize(); - StartModel(properties.ModelOrder, properties.ModelRestorationMethod); - return Coder; + _allocator = properties._allocator; + _coder = new Coder(); + _coder.RangeEncoderInitialize(); + StartModel(properties.ModelOrder, properties.RestorationMethod); + return _coder; } internal void EncodeBlock(Stream target, Stream source, bool final) { while (true) { - minimumContext = maximumContext; - numberStatistics = minimumContext.NumberStatistics; + _minimumContext = _maximumContext; + _numberStatistics = _minimumContext.NumberStatistics; int c = source.ReadByte(); if (c < 0 && !final) @@ -180,52 +180,52 @@ namespace SharpCompress.Compressors.PPMd.I1 return; } - if (numberStatistics != 0) + if (_numberStatistics != 0) { - EncodeSymbol1(c, minimumContext); - Coder.RangeEncodeSymbol(); + EncodeSymbol1(c, _minimumContext); + _coder.RangeEncodeSymbol(); } else { - EncodeBinarySymbol(c, minimumContext); - Coder.RangeShiftEncodeSymbol(TotalBitCount); + EncodeBinarySymbol(c, _minimumContext); + _coder.RangeShiftEncodeSymbol(TOTAL_BIT_COUNT); } - while (foundState == PpmState.Zero) + while (_foundState == PpmState.ZERO) { - Coder.RangeEncoderNormalize(target); + _coder.RangeEncoderNormalize(target); do { - orderFall++; - minimumContext = minimumContext.Suffix; - if (minimumContext == PpmContext.Zero) + _orderFall++; + _minimumContext = _minimumContext.Suffix; + if (_minimumContext == PpmContext.ZERO) { goto StopEncoding; } } - while (minimumContext.NumberStatistics == numberMasked); - EncodeSymbol2(c, minimumContext); - Coder.RangeEncodeSymbol(); + while (_minimumContext.NumberStatistics == _numberMasked); + EncodeSymbol2(c, _minimumContext); + _coder.RangeEncodeSymbol(); } - if (orderFall == 0 && (Pointer)foundState.Successor >= Allocator.BaseUnit) + if (_orderFall == 0 && (Pointer)_foundState.Successor >= _allocator._baseUnit) { - maximumContext = foundState.Successor; + _maximumContext = _foundState.Successor; } else { - UpdateModel(minimumContext); - if (escapeCount == 0) + UpdateModel(_minimumContext); + if (_escapeCount == 0) { ClearMask(); } } - Coder.RangeEncoderNormalize(target); + _coder.RangeEncoderNormalize(target); } StopEncoding: - Coder.RangeEncoderFlush(target); + _coder.RangeEncoderFlush(target); } /// @@ -254,18 +254,18 @@ namespace SharpCompress.Compressors.PPMd.I1 internal Coder DecodeStart(Stream source, PpmdProperties properties) { - Allocator = properties.Allocator; - Coder = new Coder(); - Coder.RangeDecoderInitialize(source); - StartModel(properties.ModelOrder, properties.ModelRestorationMethod); - minimumContext = maximumContext; - numberStatistics = minimumContext.NumberStatistics; - return Coder; + _allocator = properties._allocator; + _coder = new Coder(); + _coder.RangeDecoderInitialize(source); + StartModel(properties.ModelOrder, properties.RestorationMethod); + _minimumContext = _maximumContext; + _numberStatistics = _minimumContext.NumberStatistics; + return _coder; } internal int DecodeBlock(Stream source, byte[] buffer, int offset, int count) { - if (minimumContext == PpmContext.Zero) + if (_minimumContext == PpmContext.ZERO) { return 0; } @@ -273,54 +273,54 @@ namespace SharpCompress.Compressors.PPMd.I1 int total = 0; while (total < count) { - if (numberStatistics != 0) + if (_numberStatistics != 0) { - DecodeSymbol1(minimumContext); + DecodeSymbol1(_minimumContext); } else { - DecodeBinarySymbol(minimumContext); + DecodeBinarySymbol(_minimumContext); } - Coder.RangeRemoveSubrange(); + _coder.RangeRemoveSubrange(); - while (foundState == PpmState.Zero) + while (_foundState == PpmState.ZERO) { - Coder.RangeDecoderNormalize(source); + _coder.RangeDecoderNormalize(source); do { - orderFall++; - minimumContext = minimumContext.Suffix; - if (minimumContext == PpmContext.Zero) + _orderFall++; + _minimumContext = _minimumContext.Suffix; + if (_minimumContext == PpmContext.ZERO) { goto StopDecoding; } } - while (minimumContext.NumberStatistics == numberMasked); - DecodeSymbol2(minimumContext); - Coder.RangeRemoveSubrange(); + while (_minimumContext.NumberStatistics == _numberMasked); + DecodeSymbol2(_minimumContext); + _coder.RangeRemoveSubrange(); } - buffer[offset] = foundState.Symbol; + buffer[offset] = _foundState.Symbol; offset++; total++; - if (orderFall == 0 && (Pointer)foundState.Successor >= Allocator.BaseUnit) + if (_orderFall == 0 && (Pointer)_foundState.Successor >= _allocator._baseUnit) { - maximumContext = foundState.Successor; + _maximumContext = _foundState.Successor; } else { - UpdateModel(minimumContext); - if (escapeCount == 0) + UpdateModel(_minimumContext); + if (_escapeCount == 0) { ClearMask(); } } - minimumContext = maximumContext; - numberStatistics = minimumContext.NumberStatistics; - Coder.RangeDecoderNormalize(source); + _minimumContext = _maximumContext; + _numberStatistics = _minimumContext.NumberStatistics; + _coder.RangeDecoderNormalize(source); } StopDecoding: @@ -337,65 +337,65 @@ namespace SharpCompress.Compressors.PPMd.I1 /// private void StartModel(int modelOrder, ModelRestorationMethod modelRestorationMethod) { - Array.Clear(characterMask, 0, characterMask.Length); - escapeCount = 1; + Array.Clear(_characterMask, 0, _characterMask.Length); + _escapeCount = 1; // Compress in "solid" mode if the model order value is set to 1 (this will examine the current PPM context // structures to determine the value of orderFall). if (modelOrder < 2) { - orderFall = this.modelOrder; - for (PpmContext context = maximumContext; context.Suffix != PpmContext.Zero; context = context.Suffix) + _orderFall = _modelOrder; + for (PpmContext context = _maximumContext; context.Suffix != PpmContext.ZERO; context = context.Suffix) { - orderFall--; + _orderFall--; } return; } - this.modelOrder = modelOrder; - orderFall = modelOrder; - method = modelRestorationMethod; - Allocator.Initialize(); - initialRunLength = -((modelOrder < 12) ? modelOrder : 12) - 1; - runLength = initialRunLength; + _modelOrder = modelOrder; + _orderFall = modelOrder; + _method = modelRestorationMethod; + _allocator.Initialize(); + _initialRunLength = -((modelOrder < 12) ? modelOrder : 12) - 1; + _runLength = _initialRunLength; // Allocate the context structure. - maximumContext = Allocator.AllocateContext(); - maximumContext.Suffix = PpmContext.Zero; - maximumContext.NumberStatistics = 255; - maximumContext.SummaryFrequency = (ushort)(maximumContext.NumberStatistics + 2); - maximumContext.Statistics = Allocator.AllocateUnits(256 / 2); + _maximumContext = _allocator.AllocateContext(); + _maximumContext.Suffix = PpmContext.ZERO; + _maximumContext.NumberStatistics = 255; + _maximumContext.SummaryFrequency = (ushort)(_maximumContext.NumberStatistics + 2); + _maximumContext.Statistics = _allocator.AllocateUnits(256 / 2); // allocates enough space for 256 PPM states (each is 6 bytes) - previousSuccess = 0; + _previousSuccess = 0; for (int index = 0; index < 256; index++) { - PpmState state = maximumContext.Statistics[index]; + PpmState state = _maximumContext.Statistics[index]; state.Symbol = (byte)index; state.Frequency = 1; - state.Successor = PpmContext.Zero; + state.Successor = PpmContext.ZERO; } uint probability = 0; for (int index1 = 0; probability < 25; probability++) { - while (probabilities[index1] == probability) + while (_probabilities[index1] == probability) { index1++; } for (int index2 = 0; index2 < 8; index2++) { - binarySummary[probability, index2] = - (ushort)(BinaryScale - InitialBinaryEscapes[index2] / (index1 + 1)); + _binarySummary[probability, index2] = + (ushort)(BINARY_SCALE - INITIAL_BINARY_ESCAPES[index2] / (index1 + 1)); } for (int index2 = 8; index2 < 64; index2 += 8) { for (int index3 = 0; index3 < 8; index3++) { - binarySummary[probability, index2 + index3] = binarySummary[probability, index3]; + _binarySummary[probability, index2 + index3] = _binarySummary[probability, index3]; } } } @@ -403,36 +403,36 @@ namespace SharpCompress.Compressors.PPMd.I1 probability = 0; for (uint index1 = 0; probability < 24; probability++) { - while (probabilities[index1 + 3] == probability + 3) + while (_probabilities[index1 + 3] == probability + 3) { index1++; } for (int index2 = 0; index2 < 32; index2++) { - see2Contexts[probability, index2].Initialize(2 * index1 + 5); + _see2Contexts[probability, index2].Initialize(2 * index1 + 5); } } } private void UpdateModel(PpmContext minimumContext) { - PpmState state = PpmState.Zero; - PpmContext Successor; - PpmContext currentContext = maximumContext; + PpmState state = PpmState.ZERO; + PpmContext successor; + PpmContext currentContext = _maximumContext; uint numberStatistics; uint ns1; uint cf; uint sf; uint s0; - uint foundStateFrequency = foundState.Frequency; - byte foundStateSymbol = foundState.Symbol; + uint foundStateFrequency = _foundState.Frequency; + byte foundStateSymbol = _foundState.Symbol; byte symbol; byte flag; - PpmContext foundStateSuccessor = foundState.Successor; + PpmContext foundStateSuccessor = _foundState.Successor; PpmContext context = minimumContext.Suffix; - if ((foundStateFrequency < MaximumFrequency / 4) && (context != PpmContext.Zero)) + if ((foundStateFrequency < MAXIMUM_FREQUENCY / 4) && (context != PpmContext.ZERO)) { if (context.NumberStatistics != 0) { @@ -451,7 +451,7 @@ namespace SharpCompress.Compressors.PPMd.I1 state--; } } - cf = (uint)((state.Frequency < MaximumFrequency - 9) ? 2 : 0); + cf = (uint)((state.Frequency < MAXIMUM_FREQUENCY - 9) ? 2 : 0); state.Frequency += (byte)cf; context.SummaryFrequency += (byte)cf; } @@ -462,29 +462,29 @@ namespace SharpCompress.Compressors.PPMd.I1 } } - if (orderFall == 0 && foundStateSuccessor != PpmContext.Zero) + if (_orderFall == 0 && foundStateSuccessor != PpmContext.ZERO) { - foundState.Successor = CreateSuccessors(true, state, minimumContext); - if (foundState.Successor == PpmContext.Zero) + _foundState.Successor = CreateSuccessors(true, state, minimumContext); + if (_foundState.Successor == PpmContext.ZERO) { goto RestartModel; } - maximumContext = foundState.Successor; + _maximumContext = _foundState.Successor; return; } - Allocator.Text[0] = foundStateSymbol; - Allocator.Text++; - Successor = Allocator.Text; + _allocator._text[0] = foundStateSymbol; + _allocator._text++; + successor = _allocator._text; - if (Allocator.Text >= Allocator.BaseUnit) + if (_allocator._text >= _allocator._baseUnit) { goto RestartModel; } - if (foundStateSuccessor != PpmContext.Zero) + if (foundStateSuccessor != PpmContext.ZERO) { - if (foundStateSuccessor < Allocator.BaseUnit) + if (foundStateSuccessor < _allocator._baseUnit) { foundStateSuccessor = CreateSuccessors(false, state, minimumContext); } @@ -494,21 +494,21 @@ namespace SharpCompress.Compressors.PPMd.I1 foundStateSuccessor = ReduceOrder(state, minimumContext); } - if (foundStateSuccessor == PpmContext.Zero) + if (foundStateSuccessor == PpmContext.ZERO) { goto RestartModel; } - if (--orderFall == 0) + if (--_orderFall == 0) { - Successor = foundStateSuccessor; - Allocator.Text -= (maximumContext != minimumContext) ? 1 : 0; + successor = foundStateSuccessor; + _allocator._text -= (_maximumContext != minimumContext) ? 1 : 0; } - else if (method > ModelRestorationMethod.Freeze) + else if (_method > ModelRestorationMethod.Freeze) { - Successor = foundStateSuccessor; - Allocator.Text = Allocator.Heap; - orderFall = 0; + successor = foundStateSuccessor; + _allocator._text = _allocator._heap; + _orderFall = 0; } numberStatistics = minimumContext.NumberStatistics; @@ -521,8 +521,8 @@ namespace SharpCompress.Compressors.PPMd.I1 { if ((ns1 & 1) != 0) { - state = Allocator.ExpandUnits(currentContext.Statistics, (ns1 + 1) >> 1); - if (state == PpmState.Zero) + state = _allocator.ExpandUnits(currentContext.Statistics, (ns1 + 1) >> 1); + if (state == PpmState.ZERO) { goto RestartModel; } @@ -532,23 +532,23 @@ namespace SharpCompress.Compressors.PPMd.I1 } else { - state = Allocator.AllocateUnits(1); - if (state == PpmState.Zero) + state = _allocator.AllocateUnits(1); + if (state == PpmState.ZERO) { goto RestartModel; } Copy(state, currentContext.FirstState); currentContext.Statistics = state; - if (state.Frequency < MaximumFrequency / 4 - 1) + if (state.Frequency < MAXIMUM_FREQUENCY / 4 - 1) { state.Frequency += state.Frequency; } else { - state.Frequency = (byte)(MaximumFrequency - 4); + state.Frequency = (byte)(MAXIMUM_FREQUENCY - 4); } currentContext.SummaryFrequency = - (ushort)(state.Frequency + initialEscape + ((numberStatistics > 2) ? 1 : 0)); + (ushort)(state.Frequency + _initialEscape + ((numberStatistics > 2) ? 1 : 0)); } cf = (uint)(2 * foundStateFrequency * (currentContext.SummaryFrequency + 6)); @@ -566,13 +566,13 @@ namespace SharpCompress.Compressors.PPMd.I1 } state = currentContext.Statistics + (++currentContext.NumberStatistics); - state.Successor = Successor; + state.Successor = successor; state.Symbol = foundStateSymbol; state.Frequency = (byte)cf; currentContext.Flags |= flag; } - maximumContext = foundStateSuccessor; + _maximumContext = foundStateSuccessor; return; RestartModel: @@ -581,22 +581,22 @@ namespace SharpCompress.Compressors.PPMd.I1 private PpmContext CreateSuccessors(bool skip, PpmState state, PpmContext context) { - PpmContext upBranch = foundState.Successor; - PpmState[] states = new PpmState[MaximumOrder]; + PpmContext upBranch = _foundState.Successor; + PpmState[] states = new PpmState[MAXIMUM_ORDER]; uint stateIndex = 0; - byte symbol = foundState.Symbol; + byte symbol = _foundState.Symbol; if (!skip) { - states[stateIndex++] = foundState; - if (context.Suffix == PpmContext.Zero) + states[stateIndex++] = _foundState; + if (context.Suffix == PpmContext.ZERO) { goto NoLoop; } } bool gotoLoopEntry = false; - if (state != PpmState.Zero) + if (state != PpmState.ZERO) { context = context.Suffix; gotoLoopEntry = true; @@ -624,7 +624,7 @@ namespace SharpCompress.Compressors.PPMd.I1 } while (temporary != symbol); } - temporary = (byte)((state.Frequency < MaximumFrequency - 9) ? 1 : 0); + temporary = (byte)((state.Frequency < MAXIMUM_FREQUENCY - 9) ? 1 : 0); state.Frequency += temporary; context.SummaryFrequency += temporary; } @@ -643,7 +643,7 @@ namespace SharpCompress.Compressors.PPMd.I1 } states[stateIndex++] = state; } - while (context.Suffix != PpmContext.Zero); + while (context.Suffix != PpmContext.ZERO); NoLoop: if (stateIndex == 0) @@ -683,10 +683,10 @@ namespace SharpCompress.Compressors.PPMd.I1 do { - PpmContext currentContext = Allocator.AllocateContext(); - if (currentContext == PpmContext.Zero) + PpmContext currentContext = _allocator.AllocateContext(); + if (currentContext == PpmContext.ZERO) { - return PpmContext.Zero; + return PpmContext.ZERO; } currentContext.NumberStatistics = localNumberStatistics; currentContext.Flags = localFlags; @@ -705,19 +705,19 @@ namespace SharpCompress.Compressors.PPMd.I1 private PpmContext ReduceOrder(PpmState state, PpmContext context) { PpmState currentState; - PpmState[] states = new PpmState[MaximumOrder]; + PpmState[] states = new PpmState[MAXIMUM_ORDER]; uint stateIndex = 0; PpmContext currentContext = context; - PpmContext UpBranch = Allocator.Text; + PpmContext upBranch = _allocator._text; byte temporary; - byte symbol = foundState.Symbol; + byte symbol = _foundState.Symbol; - states[stateIndex++] = foundState; - foundState.Successor = UpBranch; - orderFall++; + states[stateIndex++] = _foundState; + _foundState.Successor = upBranch; + _orderFall++; bool gotoLoopEntry = false; - if (state != PpmState.Zero) + if (state != PpmState.ZERO) { context = context.Suffix; gotoLoopEntry = true; @@ -731,17 +731,17 @@ namespace SharpCompress.Compressors.PPMd.I1 goto LoopEntry; } - if (context.Suffix == PpmContext.Zero) + if (context.Suffix == PpmContext.ZERO) { - if (method > ModelRestorationMethod.Freeze) + if (_method > ModelRestorationMethod.Freeze) { do { states[--stateIndex].Successor = context; } while (stateIndex != 0); - Allocator.Text = Allocator.Heap + 1; - orderFall = 1; + _allocator._text = _allocator._heap + 1; + _orderFall = 1; } return context; } @@ -759,7 +759,7 @@ namespace SharpCompress.Compressors.PPMd.I1 } while (temporary != symbol); } - temporary = (byte)((state.Frequency < MaximumFrequency - 9) ? 2 : 0); + temporary = (byte)((state.Frequency < MAXIMUM_FREQUENCY - 9) ? 2 : 0); state.Frequency += temporary; context.SummaryFrequency += temporary; } @@ -770,16 +770,16 @@ namespace SharpCompress.Compressors.PPMd.I1 } LoopEntry: - if (state.Successor != PpmContext.Zero) + if (state.Successor != PpmContext.ZERO) { break; } states[stateIndex++] = state; - state.Successor = UpBranch; - orderFall++; + state.Successor = upBranch; + _orderFall++; } - if (method > ModelRestorationMethod.Freeze) + if (_method > ModelRestorationMethod.Freeze) { context = state.Successor; do @@ -787,22 +787,22 @@ namespace SharpCompress.Compressors.PPMd.I1 states[--stateIndex].Successor = context; } while (stateIndex != 0); - Allocator.Text = Allocator.Heap + 1; - orderFall = 1; + _allocator._text = _allocator._heap + 1; + _orderFall = 1; return context; } - if (state.Successor <= UpBranch) + if (state.Successor <= upBranch) { - currentState = foundState; - foundState = state; - state.Successor = CreateSuccessors(false, PpmState.Zero, context); - foundState = currentState; + currentState = _foundState; + _foundState = state; + state.Successor = CreateSuccessors(false, PpmState.ZERO, context); + _foundState = currentState; } - if (orderFall == 1 && currentContext == maximumContext) + if (_orderFall == 1 && currentContext == _maximumContext) { - foundState.Successor = state.Successor; - Allocator.Text--; + _foundState.Successor = state.Successor; + _allocator._text--; } return state.Successor; @@ -812,8 +812,8 @@ namespace SharpCompress.Compressors.PPMd.I1 { PpmContext currentContext; - Allocator.Text = Allocator.Heap; - for (currentContext = maximumContext; currentContext != context; currentContext = currentContext.Suffix) + _allocator._text = _allocator._heap; + for (currentContext = _maximumContext; currentContext != context; currentContext = currentContext.Suffix) { if (--currentContext.NumberStatistics == 0) { @@ -822,7 +822,7 @@ namespace SharpCompress.Compressors.PPMd.I1 ((currentContext.Flags & 0x10) + ((currentContext.Statistics.Symbol >= 0x40) ? 0x08 : 0x00)); PpmState state = currentContext.Statistics; Copy(currentContext.FirstState, state); - Allocator.SpecialFreeUnits(state); + _allocator.SpecialFreeUnits(state); currentContext.FirstStateFrequency = (byte)((currentContext.FirstStateFrequency + 11) >> 3); } else @@ -843,45 +843,45 @@ namespace SharpCompress.Compressors.PPMd.I1 } } - if (method > ModelRestorationMethod.Freeze) + if (_method > ModelRestorationMethod.Freeze) { - maximumContext = foundStateSuccessor; - Allocator.GlueCount += (uint)(((Allocator.MemoryNodes[1].Stamp & 1) == 0) ? 1 : 0); + _maximumContext = foundStateSuccessor; + _allocator._glueCount += (uint)(((_allocator._memoryNodes[1].Stamp & 1) == 0) ? 1 : 0); } - else if (method == ModelRestorationMethod.Freeze) + else if (_method == ModelRestorationMethod.Freeze) { - while (maximumContext.Suffix != PpmContext.Zero) + while (_maximumContext.Suffix != PpmContext.ZERO) { - maximumContext = maximumContext.Suffix; + _maximumContext = _maximumContext.Suffix; } - RemoveBinaryContexts(0, maximumContext); - method = method + 1; - Allocator.GlueCount = 0; - orderFall = modelOrder; + RemoveBinaryContexts(0, _maximumContext); + _method = _method + 1; + _allocator._glueCount = 0; + _orderFall = _modelOrder; } - else if (method == ModelRestorationMethod.Restart || - Allocator.GetMemoryUsed() < (Allocator.AllocatorSize >> 1)) + else if (_method == ModelRestorationMethod.Restart || + _allocator.GetMemoryUsed() < (_allocator._allocatorSize >> 1)) { - StartModel(modelOrder, method); - escapeCount = 0; + StartModel(_modelOrder, _method); + _escapeCount = 0; } else { - while (maximumContext.Suffix != PpmContext.Zero) + while (_maximumContext.Suffix != PpmContext.ZERO) { - maximumContext = maximumContext.Suffix; + _maximumContext = _maximumContext.Suffix; } do { - CutOff(0, maximumContext); - Allocator.ExpandText(); + CutOff(0, _maximumContext); + _allocator.ExpandText(); } - while (Allocator.GetMemoryUsed() > 3 * (Allocator.AllocatorSize >> 2)); + while (_allocator.GetMemoryUsed() > 3 * (_allocator._allocatorSize >> 2)); - Allocator.GlueCount = 0; - orderFall = modelOrder; + _allocator._glueCount = 0; + _orderFall = _modelOrder; } } @@ -914,8 +914,8 @@ namespace SharpCompress.Compressors.PPMd.I1 private void ClearMask() { - escapeCount = 1; - Array.Clear(characterMask, 0, characterMask.Length); + _escapeCount = 1; + Array.Clear(_characterMask, 0, _characterMask.Length); } #endregion diff --git a/src/SharpCompress/Compressors/PPMd/I1/Pointer.cs b/src/SharpCompress/Compressors/PPMd/I1/Pointer.cs index d868540a..e95574e7 100644 --- a/src/SharpCompress/Compressors/PPMd/I1/Pointer.cs +++ b/src/SharpCompress/Compressors/PPMd/I1/Pointer.cs @@ -7,7 +7,7 @@ using System; namespace SharpCompress.Compressors.PPMd.I1 { /// - /// A structure containing a single address representing a position in the array. This + /// A structure containing a single address representing a position in the array. This /// is intended to mimic the behaviour of a pointer in C/C++. /// /// @@ -17,23 +17,23 @@ namespace SharpCompress.Compressors.PPMd.I1 /// instance rather than just copying a reference to the same instance). /// /// - /// Note that is a field rather than a property for performance reasons. + /// Note that is a field rather than a property for performance reasons. /// /// internal struct Pointer { - public uint Address; - public byte[] Memory; - public static readonly Pointer Zero = new Pointer(0, null); - public const int Size = 1; + public uint _address; + public byte[] _memory; + public static readonly Pointer ZERO = new Pointer(0, null); + public const int SIZE = 1; /// /// Initializes a new instance of the structure. /// public Pointer(uint address, byte[] memory) { - Address = address; - Memory = memory; + _address = address; + _memory = memory; } /// @@ -46,22 +46,22 @@ namespace SharpCompress.Compressors.PPMd.I1 get { #if DEBUG - if (Address == 0) + if (_address == 0) { throw new InvalidOperationException("The pointer being indexed is a null pointer."); } #endif - return Memory[Address + offset]; + return _memory[_address + offset]; } set { #if DEBUG - if (Address == 0) + if (_address == 0) { throw new InvalidOperationException("The pointer being indexed is a null pointer."); } #endif - Memory[Address + offset] = value; + _memory[_address + offset] = value; } } @@ -72,7 +72,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static implicit operator Pointer(MemoryNode memoryNode) { - return new Pointer(memoryNode.Address, memoryNode.Memory); + return new Pointer(memoryNode._address, memoryNode._memory); } /// @@ -82,7 +82,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static implicit operator Pointer(Model.PpmContext context) { - return new Pointer(context.Address, context.Memory); + return new Pointer(context._address, context._memory); } /// @@ -92,7 +92,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static implicit operator Pointer(PpmState state) { - return new Pointer(state.Address, state.Memory); + return new Pointer(state._address, state._memory); } /// @@ -104,12 +104,12 @@ namespace SharpCompress.Compressors.PPMd.I1 public static Pointer operator +(Pointer pointer, int offset) { #if DEBUG - if (pointer.Address == 0) + if (pointer._address == 0) { throw new InvalidOperationException("The pointer is a null pointer."); } #endif - pointer.Address = (uint)(pointer.Address + offset); + pointer._address = (uint)(pointer._address + offset); return pointer; } @@ -122,12 +122,12 @@ namespace SharpCompress.Compressors.PPMd.I1 public static Pointer operator +(Pointer pointer, uint offset) { #if DEBUG - if (pointer.Address == 0) + if (pointer._address == 0) { throw new InvalidOperationException("The pointer is a null pointer."); } #endif - pointer.Address += offset; + pointer._address += offset; return pointer; } @@ -139,12 +139,12 @@ namespace SharpCompress.Compressors.PPMd.I1 public static Pointer operator ++(Pointer pointer) { #if DEBUG - if (pointer.Address == 0) + if (pointer._address == 0) { throw new InvalidOperationException("The pointer being incremented is a null pointer."); } #endif - pointer.Address++; + pointer._address++; return pointer; } @@ -157,12 +157,12 @@ namespace SharpCompress.Compressors.PPMd.I1 public static Pointer operator -(Pointer pointer, int offset) { #if DEBUG - if (pointer.Address == 0) + if (pointer._address == 0) { throw new InvalidOperationException("The pointer is a null pointer."); } #endif - pointer.Address = (uint)(pointer.Address - offset); + pointer._address = (uint)(pointer._address - offset); return pointer; } @@ -175,12 +175,12 @@ namespace SharpCompress.Compressors.PPMd.I1 public static Pointer operator -(Pointer pointer, uint offset) { #if DEBUG - if (pointer.Address == 0) + if (pointer._address == 0) { throw new InvalidOperationException("The pointer is a null pointer."); } #endif - pointer.Address -= offset; + pointer._address -= offset; return pointer; } @@ -192,12 +192,12 @@ namespace SharpCompress.Compressors.PPMd.I1 public static Pointer operator --(Pointer pointer) { #if DEBUG - if (pointer.Address == 0) + if (pointer._address == 0) { throw new InvalidOperationException("The pointer being decremented is a null pointer."); } #endif - pointer.Address--; + pointer._address--; return pointer; } @@ -210,18 +210,18 @@ namespace SharpCompress.Compressors.PPMd.I1 public static uint operator -(Pointer pointer1, Pointer pointer2) { #if DEBUG - if (pointer1.Address == 0) + if (pointer1._address == 0) { throw new InvalidOperationException( "The pointer to the left of the subtraction operator is a null pointer."); } - if (pointer2.Address == 0) + if (pointer2._address == 0) { throw new InvalidOperationException( "The pointer to the right of the subtraction operator is a null pointer."); } #endif - return pointer1.Address - pointer2.Address; + return pointer1._address - pointer2._address; } /// @@ -233,18 +233,18 @@ namespace SharpCompress.Compressors.PPMd.I1 public static bool operator <(Pointer pointer1, Pointer pointer2) { #if DEBUG - if (pointer1.Address == 0) + if (pointer1._address == 0) { throw new InvalidOperationException( "The pointer to the left of the less than operator is a null pointer."); } - if (pointer2.Address == 0) + if (pointer2._address == 0) { throw new InvalidOperationException( "The pointer to the right of the less than operator is a null pointer."); } #endif - return pointer1.Address < pointer2.Address; + return pointer1._address < pointer2._address; } /// @@ -256,18 +256,18 @@ namespace SharpCompress.Compressors.PPMd.I1 public static bool operator <=(Pointer pointer1, Pointer pointer2) { #if DEBUG - if (pointer1.Address == 0) + if (pointer1._address == 0) { throw new InvalidOperationException( "The pointer to the left of the less than or equal to operator is a null pointer."); } - if (pointer2.Address == 0) + if (pointer2._address == 0) { throw new InvalidOperationException( "The pointer to the right of the less than or equal to operator is a null pointer."); } #endif - return pointer1.Address <= pointer2.Address; + return pointer1._address <= pointer2._address; } /// @@ -279,18 +279,18 @@ namespace SharpCompress.Compressors.PPMd.I1 public static bool operator >(Pointer pointer1, Pointer pointer2) { #if DEBUG - if (pointer1.Address == 0) + if (pointer1._address == 0) { throw new InvalidOperationException( "The pointer to the left of the greater than operator is a null pointer."); } - if (pointer2.Address == 0) + if (pointer2._address == 0) { throw new InvalidOperationException( "The pointer to the right of the greater than operator is a null pointer."); } #endif - return pointer1.Address > pointer2.Address; + return pointer1._address > pointer2._address; } /// @@ -302,18 +302,18 @@ namespace SharpCompress.Compressors.PPMd.I1 public static bool operator >=(Pointer pointer1, Pointer pointer2) { #if DEBUG - if (pointer1.Address == 0) + if (pointer1._address == 0) { throw new InvalidOperationException( "The pointer to the left of the greater than or equal to operator is a null pointer."); } - if (pointer2.Address == 0) + if (pointer2._address == 0) { throw new InvalidOperationException( "The pointer to the right of the greater than or equal to operator is a null pointer."); } #endif - return pointer1.Address >= pointer2.Address; + return pointer1._address >= pointer2._address; } /// @@ -324,7 +324,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator ==(Pointer pointer1, Pointer pointer2) { - return pointer1.Address == pointer2.Address; + return pointer1._address == pointer2._address; } /// @@ -335,7 +335,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator !=(Pointer pointer1, Pointer pointer2) { - return pointer1.Address != pointer2.Address; + return pointer1._address != pointer2._address; } /// @@ -348,7 +348,7 @@ namespace SharpCompress.Compressors.PPMd.I1 if (obj is Pointer) { Pointer pointer = (Pointer)obj; - return pointer.Address == Address; + return pointer._address == _address; } return base.Equals(obj); } @@ -359,7 +359,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// A 32-bit signed integer that is the hash code for this instance. public override int GetHashCode() { - return Address.GetHashCode(); + return _address.GetHashCode(); } } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/PPMd/I1/PpmContext.cs b/src/SharpCompress/Compressors/PPMd/I1/PpmContext.cs index 371385fe..6ec1a7b8 100644 --- a/src/SharpCompress/Compressors/PPMd/I1/PpmContext.cs +++ b/src/SharpCompress/Compressors/PPMd/I1/PpmContext.cs @@ -17,40 +17,40 @@ namespace SharpCompress.Compressors.PPMd.I1 /// internal struct PpmContext { - public uint Address; - public byte[] Memory; - public static readonly PpmContext Zero = new PpmContext(0, null); - public const int Size = 12; + public uint _address; + public byte[] _memory; + public static readonly PpmContext ZERO = new PpmContext(0, null); + public const int SIZE = 12; /// /// Initializes a new instance of the structure. /// public PpmContext(uint address, byte[] memory) { - Address = address; - Memory = memory; + _address = address; + _memory = memory; } /// /// Gets or sets the number statistics. /// - public byte NumberStatistics { get => Memory[Address]; set => Memory[Address] = value; } + public byte NumberStatistics { get => _memory[_address]; set => _memory[_address] = value; } /// /// Gets or sets the flags. /// - public byte Flags { get => Memory[Address + 1]; set => Memory[Address + 1] = value; } + public byte Flags { get => _memory[_address + 1]; set => _memory[_address + 1] = value; } /// /// Gets or sets the summary frequency. /// public ushort SummaryFrequency { - get => (ushort)(Memory[Address + 2] | Memory[Address + 3] << 8); + get => (ushort)(_memory[_address + 2] | _memory[_address + 3] << 8); set { - Memory[Address + 2] = (byte)value; - Memory[Address + 3] = (byte)(value >> 8); + _memory[_address + 2] = (byte)value; + _memory[_address + 3] = (byte)(value >> 8); } } @@ -60,14 +60,14 @@ namespace SharpCompress.Compressors.PPMd.I1 public PpmState Statistics { get => new PpmState( - Memory[Address + 4] | ((uint)Memory[Address + 5]) << 8 | - ((uint)Memory[Address + 6]) << 16 | ((uint)Memory[Address + 7]) << 24, Memory); + _memory[_address + 4] | ((uint)_memory[_address + 5]) << 8 | + ((uint)_memory[_address + 6]) << 16 | ((uint)_memory[_address + 7]) << 24, _memory); set { - Memory[Address + 4] = (byte)value.Address; - Memory[Address + 5] = (byte)(value.Address >> 8); - Memory[Address + 6] = (byte)(value.Address >> 16); - Memory[Address + 7] = (byte)(value.Address >> 24); + _memory[_address + 4] = (byte)value._address; + _memory[_address + 5] = (byte)(value._address >> 8); + _memory[_address + 6] = (byte)(value._address >> 16); + _memory[_address + 7] = (byte)(value._address >> 24); } } @@ -77,14 +77,14 @@ namespace SharpCompress.Compressors.PPMd.I1 public PpmContext Suffix { get => new PpmContext( - Memory[Address + 8] | ((uint)Memory[Address + 9]) << 8 | - ((uint)Memory[Address + 10]) << 16 | ((uint)Memory[Address + 11]) << 24, Memory); + _memory[_address + 8] | ((uint)_memory[_address + 9]) << 8 | + ((uint)_memory[_address + 10]) << 16 | ((uint)_memory[_address + 11]) << 24, _memory); set { - Memory[Address + 8] = (byte)value.Address; - Memory[Address + 9] = (byte)(value.Address >> 8); - Memory[Address + 10] = (byte)(value.Address >> 16); - Memory[Address + 11] = (byte)(value.Address >> 24); + _memory[_address + 8] = (byte)value._address; + _memory[_address + 9] = (byte)(value._address >> 8); + _memory[_address + 10] = (byte)(value._address >> 16); + _memory[_address + 11] = (byte)(value._address >> 24); } } @@ -113,21 +113,21 @@ namespace SharpCompress.Compressors.PPMd.I1 /// /// /// - public PpmState FirstState => new PpmState(Address + 2, Memory); + public PpmState FirstState => new PpmState(_address + 2, _memory); /// /// Gets or sets the symbol of the first PPM state. This is provided for convenience. The same /// information can be obtained using the Symbol property on the PPM state provided by the /// property. /// - public byte FirstStateSymbol { get => Memory[Address + 2]; set => Memory[Address + 2] = value; } + public byte FirstStateSymbol { get => _memory[_address + 2]; set => _memory[_address + 2] = value; } /// /// Gets or sets the frequency of the first PPM state. This is provided for convenience. The same /// information can be obtained using the Frequency property on the PPM state provided by the ///context.FirstState property. /// - public byte FirstStateFrequency { get => Memory[Address + 3]; set => Memory[Address + 3] = value; } + public byte FirstStateFrequency { get => _memory[_address + 3]; set => _memory[_address + 3] = value; } /// /// Gets or sets the successor of the first PPM state. This is provided for convenience. The same @@ -136,14 +136,14 @@ namespace SharpCompress.Compressors.PPMd.I1 public PpmContext FirstStateSuccessor { get => new PpmContext( - Memory[Address + 4] | ((uint)Memory[Address + 5]) << 8 | - ((uint)Memory[Address + 6]) << 16 | ((uint)Memory[Address + 7]) << 24, Memory); + _memory[_address + 4] | ((uint)_memory[_address + 5]) << 8 | + ((uint)_memory[_address + 6]) << 16 | ((uint)_memory[_address + 7]) << 24, _memory); set { - Memory[Address + 4] = (byte)value.Address; - Memory[Address + 5] = (byte)(value.Address >> 8); - Memory[Address + 6] = (byte)(value.Address >> 16); - Memory[Address + 7] = (byte)(value.Address >> 24); + _memory[_address + 4] = (byte)value._address; + _memory[_address + 5] = (byte)(value._address >> 8); + _memory[_address + 6] = (byte)(value._address >> 16); + _memory[_address + 7] = (byte)(value._address >> 24); } } @@ -154,7 +154,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static implicit operator PpmContext(Pointer pointer) { - return new PpmContext(pointer.Address, pointer.Memory); + return new PpmContext(pointer._address, pointer._memory); } /// @@ -165,7 +165,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static PpmContext operator +(PpmContext context, int offset) { - context.Address = (uint)(context.Address + offset * Size); + context._address = (uint)(context._address + offset * SIZE); return context; } @@ -177,7 +177,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static PpmContext operator -(PpmContext context, int offset) { - context.Address = (uint)(context.Address - offset * Size); + context._address = (uint)(context._address - offset * SIZE); return context; } @@ -189,7 +189,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator <=(PpmContext context1, PpmContext context2) { - return context1.Address <= context2.Address; + return context1._address <= context2._address; } /// @@ -200,7 +200,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator >=(PpmContext context1, PpmContext context2) { - return context1.Address >= context2.Address; + return context1._address >= context2._address; } /// @@ -211,7 +211,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator ==(PpmContext context1, PpmContext context2) { - return context1.Address == context2.Address; + return context1._address == context2._address; } /// @@ -222,7 +222,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator !=(PpmContext context1, PpmContext context2) { - return context1.Address != context2.Address; + return context1._address != context2._address; } /// @@ -235,7 +235,7 @@ namespace SharpCompress.Compressors.PPMd.I1 if (obj is PpmContext) { PpmContext context = (PpmContext)obj; - return context.Address == Address; + return context._address == _address; } return base.Equals(obj); } @@ -246,38 +246,38 @@ namespace SharpCompress.Compressors.PPMd.I1 /// A 32-bit signed integer that is the hash code for this instance. public override int GetHashCode() { - return Address.GetHashCode(); + return _address.GetHashCode(); } } private void EncodeBinarySymbol(int symbol, PpmContext context) { PpmState state = context.FirstState; - int index1 = probabilities[state.Frequency - 1]; - int index2 = numberStatisticsToBinarySummaryIndex[context.Suffix.NumberStatistics] + previousSuccess + - context.Flags + ((runLength >> 26) & 0x20); + int index1 = _probabilities[state.Frequency - 1]; + int index2 = _numberStatisticsToBinarySummaryIndex[context.Suffix.NumberStatistics] + _previousSuccess + + context.Flags + ((_runLength >> 26) & 0x20); if (state.Symbol == symbol) { - foundState = state; + _foundState = state; state.Frequency += (byte)((state.Frequency < 196) ? 1 : 0); - Coder.LowCount = 0; - Coder.HighCount = binarySummary[index1, index2]; - binarySummary[index1, index2] += - (ushort)(Interval - Mean(binarySummary[index1, index2], PeriodBitCount, 2)); - previousSuccess = 1; - runLength++; + _coder._lowCount = 0; + _coder._highCount = _binarySummary[index1, index2]; + _binarySummary[index1, index2] += + (ushort)(INTERVAL - Mean(_binarySummary[index1, index2], PERIOD_BIT_COUNT, 2)); + _previousSuccess = 1; + _runLength++; } else { - Coder.LowCount = binarySummary[index1, index2]; - binarySummary[index1, index2] -= (ushort)Mean(binarySummary[index1, index2], PeriodBitCount, 2); - Coder.HighCount = BinaryScale; - initialEscape = ExponentialEscapes[binarySummary[index1, index2] >> 10]; - characterMask[state.Symbol] = escapeCount; - previousSuccess = 0; - numberMasked = 0; - foundState = PpmState.Zero; + _coder._lowCount = _binarySummary[index1, index2]; + _binarySummary[index1, index2] -= (ushort)Mean(_binarySummary[index1, index2], PERIOD_BIT_COUNT, 2); + _coder._highCount = BINARY_SCALE; + _initialEscape = EXPONENTIAL_ESCAPES[_binarySummary[index1, index2] >> 10]; + _characterMask[state.Symbol] = _escapeCount; + _previousSuccess = 0; + _numberMasked = 0; + _foundState = PpmState.ZERO; } } @@ -286,46 +286,46 @@ namespace SharpCompress.Compressors.PPMd.I1 uint lowCount; uint index = context.Statistics.Symbol; PpmState state = context.Statistics; - Coder.Scale = context.SummaryFrequency; + _coder._scale = context.SummaryFrequency; if (index == symbol) { - Coder.HighCount = state.Frequency; - previousSuccess = (byte)((2 * Coder.HighCount >= Coder.Scale) ? 1 : 0); - foundState = state; - foundState.Frequency += 4; + _coder._highCount = state.Frequency; + _previousSuccess = (byte)((2 * _coder._highCount >= _coder._scale) ? 1 : 0); + _foundState = state; + _foundState.Frequency += 4; context.SummaryFrequency += 4; - runLength += previousSuccess; - if (state.Frequency > MaximumFrequency) + _runLength += _previousSuccess; + if (state.Frequency > MAXIMUM_FREQUENCY) { Rescale(context); } - Coder.LowCount = 0; + _coder._lowCount = 0; return; } lowCount = state.Frequency; index = context.NumberStatistics; - previousSuccess = 0; + _previousSuccess = 0; while ((++state).Symbol != symbol) { lowCount += state.Frequency; if (--index == 0) { - Coder.LowCount = lowCount; - characterMask[state.Symbol] = escapeCount; - numberMasked = context.NumberStatistics; + _coder._lowCount = lowCount; + _characterMask[state.Symbol] = _escapeCount; + _numberMasked = context.NumberStatistics; index = context.NumberStatistics; - foundState = PpmState.Zero; + _foundState = PpmState.ZERO; do { - characterMask[(--state).Symbol] = escapeCount; + _characterMask[(--state).Symbol] = _escapeCount; } while (--index != 0); - Coder.HighCount = Coder.Scale; + _coder._highCount = _coder._scale; return; } } - Coder.HighCount = (Coder.LowCount = lowCount) + state.Frequency; + _coder._highCount = (_coder._lowCount = lowCount) + state.Frequency; Update1(state, context); } @@ -334,7 +334,7 @@ namespace SharpCompress.Compressors.PPMd.I1 See2Context see2Context = MakeEscapeFrequency(context); uint currentSymbol; uint lowCount = 0; - uint index = (uint)(context.NumberStatistics - numberMasked); + uint index = (uint)(context.NumberStatistics - _numberMasked); PpmState state = context.Statistics - 1; do @@ -344,8 +344,8 @@ namespace SharpCompress.Compressors.PPMd.I1 currentSymbol = state[1].Symbol; state++; } - while (characterMask[currentSymbol] == escapeCount); - characterMask[currentSymbol] = escapeCount; + while (_characterMask[currentSymbol] == _escapeCount); + _characterMask[currentSymbol] = _escapeCount; if (currentSymbol == symbol) { goto SymbolFound; @@ -354,17 +354,17 @@ namespace SharpCompress.Compressors.PPMd.I1 } while (--index != 0); - Coder.LowCount = lowCount; - Coder.Scale += Coder.LowCount; - Coder.HighCount = Coder.Scale; - see2Context.Summary += (ushort)Coder.Scale; - numberMasked = context.NumberStatistics; + _coder._lowCount = lowCount; + _coder._scale += _coder._lowCount; + _coder._highCount = _coder._scale; + see2Context._summary += (ushort)_coder._scale; + _numberMasked = context.NumberStatistics; return; SymbolFound: - Coder.LowCount = lowCount; + _coder._lowCount = lowCount; lowCount += state.Frequency; - Coder.HighCount = lowCount; + _coder._highCount = lowCount; for (PpmState p1 = state; --index != 0;) { do @@ -372,10 +372,10 @@ namespace SharpCompress.Compressors.PPMd.I1 currentSymbol = p1[1].Symbol; p1++; } - while (characterMask[currentSymbol] == escapeCount); + while (_characterMask[currentSymbol] == _escapeCount); lowCount += p1.Frequency; } - Coder.Scale += lowCount; + _coder._scale += lowCount; see2Context.Update(); Update2(state, context); } @@ -383,31 +383,31 @@ namespace SharpCompress.Compressors.PPMd.I1 private void DecodeBinarySymbol(PpmContext context) { PpmState state = context.FirstState; - int index1 = probabilities[state.Frequency - 1]; - int index2 = numberStatisticsToBinarySummaryIndex[context.Suffix.NumberStatistics] + previousSuccess + - context.Flags + ((runLength >> 26) & 0x20); + int index1 = _probabilities[state.Frequency - 1]; + int index2 = _numberStatisticsToBinarySummaryIndex[context.Suffix.NumberStatistics] + _previousSuccess + + context.Flags + ((_runLength >> 26) & 0x20); - if (Coder.RangeGetCurrentShiftCount(TotalBitCount) < binarySummary[index1, index2]) + if (_coder.RangeGetCurrentShiftCount(TOTAL_BIT_COUNT) < _binarySummary[index1, index2]) { - foundState = state; + _foundState = state; state.Frequency += (byte)((state.Frequency < 196) ? 1 : 0); - Coder.LowCount = 0; - Coder.HighCount = binarySummary[index1, index2]; - binarySummary[index1, index2] += - (ushort)(Interval - Mean(binarySummary[index1, index2], PeriodBitCount, 2)); - previousSuccess = 1; - runLength++; + _coder._lowCount = 0; + _coder._highCount = _binarySummary[index1, index2]; + _binarySummary[index1, index2] += + (ushort)(INTERVAL - Mean(_binarySummary[index1, index2], PERIOD_BIT_COUNT, 2)); + _previousSuccess = 1; + _runLength++; } else { - Coder.LowCount = binarySummary[index1, index2]; - binarySummary[index1, index2] -= (ushort)Mean(binarySummary[index1, index2], PeriodBitCount, 2); - Coder.HighCount = BinaryScale; - initialEscape = ExponentialEscapes[binarySummary[index1, index2] >> 10]; - characterMask[state.Symbol] = escapeCount; - previousSuccess = 0; - numberMasked = 0; - foundState = PpmState.Zero; + _coder._lowCount = _binarySummary[index1, index2]; + _binarySummary[index1, index2] -= (ushort)Mean(_binarySummary[index1, index2], PERIOD_BIT_COUNT, 2); + _coder._highCount = BINARY_SCALE; + _initialEscape = EXPONENTIAL_ESCAPES[_binarySummary[index1, index2] >> 10]; + _characterMask[state.Symbol] = _escapeCount; + _previousSuccess = 0; + _numberMasked = 0; + _foundState = PpmState.ZERO; } } @@ -417,48 +417,48 @@ namespace SharpCompress.Compressors.PPMd.I1 uint count; uint highCount = context.Statistics.Frequency; PpmState state = context.Statistics; - Coder.Scale = context.SummaryFrequency; + _coder._scale = context.SummaryFrequency; - count = Coder.RangeGetCurrentCount(); + count = _coder.RangeGetCurrentCount(); if (count < highCount) { - Coder.HighCount = highCount; - previousSuccess = (byte)((2 * Coder.HighCount >= Coder.Scale) ? 1 : 0); - foundState = state; + _coder._highCount = highCount; + _previousSuccess = (byte)((2 * _coder._highCount >= _coder._scale) ? 1 : 0); + _foundState = state; highCount += 4; - foundState.Frequency = (byte)highCount; + _foundState.Frequency = (byte)highCount; context.SummaryFrequency += 4; - runLength += previousSuccess; - if (highCount > MaximumFrequency) + _runLength += _previousSuccess; + if (highCount > MAXIMUM_FREQUENCY) { Rescale(context); } - Coder.LowCount = 0; + _coder._lowCount = 0; return; } index = context.NumberStatistics; - previousSuccess = 0; + _previousSuccess = 0; while ((highCount += (++state).Frequency) <= count) { if (--index == 0) { - Coder.LowCount = highCount; - characterMask[state.Symbol] = escapeCount; - numberMasked = context.NumberStatistics; + _coder._lowCount = highCount; + _characterMask[state.Symbol] = _escapeCount; + _numberMasked = context.NumberStatistics; index = context.NumberStatistics; - foundState = PpmState.Zero; + _foundState = PpmState.ZERO; do { - characterMask[(--state).Symbol] = escapeCount; + _characterMask[(--state).Symbol] = _escapeCount; } while (--index != 0); - Coder.HighCount = Coder.Scale; + _coder._highCount = _coder._scale; return; } } - Coder.HighCount = highCount; - Coder.LowCount = Coder.HighCount - state.Frequency; + _coder._highCount = highCount; + _coder._lowCount = _coder._highCount - state.Frequency; Update1(state, context); } @@ -468,7 +468,7 @@ namespace SharpCompress.Compressors.PPMd.I1 uint currentSymbol; uint count; uint highCount = 0; - uint index = (uint)(context.NumberStatistics - numberMasked); + uint index = (uint)(context.NumberStatistics - _numberMasked); uint stateIndex = 0; PpmState state = context.Statistics - 1; @@ -479,56 +479,56 @@ namespace SharpCompress.Compressors.PPMd.I1 currentSymbol = state[1].Symbol; state++; } - while (characterMask[currentSymbol] == escapeCount); + while (_characterMask[currentSymbol] == _escapeCount); highCount += state.Frequency; - decodeStates[stateIndex++] = state; + _decodeStates[stateIndex++] = state; // note that decodeStates is a static array that is re-used on each call to this method (for performance reasons) } while (--index != 0); - Coder.Scale += highCount; - count = Coder.RangeGetCurrentCount(); + _coder._scale += highCount; + count = _coder.RangeGetCurrentCount(); stateIndex = 0; - state = decodeStates[stateIndex]; + state = _decodeStates[stateIndex]; if (count < highCount) { highCount = 0; while ((highCount += state.Frequency) <= count) { - state = decodeStates[++stateIndex]; + state = _decodeStates[++stateIndex]; } - Coder.HighCount = highCount; - Coder.LowCount = Coder.HighCount - state.Frequency; + _coder._highCount = highCount; + _coder._lowCount = _coder._highCount - state.Frequency; see2Context.Update(); Update2(state, context); } else { - Coder.LowCount = highCount; - Coder.HighCount = Coder.Scale; - index = (uint)(context.NumberStatistics - numberMasked); - numberMasked = context.NumberStatistics; + _coder._lowCount = highCount; + _coder._highCount = _coder._scale; + index = (uint)(context.NumberStatistics - _numberMasked); + _numberMasked = context.NumberStatistics; do { - characterMask[decodeStates[stateIndex].Symbol] = escapeCount; + _characterMask[_decodeStates[stateIndex].Symbol] = _escapeCount; stateIndex++; } while (--index != 0); - see2Context.Summary += (ushort)Coder.Scale; + see2Context._summary += (ushort)_coder._scale; } } private void Update1(PpmState state, PpmContext context) { - foundState = state; - foundState.Frequency += 4; + _foundState = state; + _foundState.Frequency += 4; context.SummaryFrequency += 4; if (state[0].Frequency > state[-1].Frequency) { Swap(state[0], state[-1]); - foundState = --state; - if (state.Frequency > MaximumFrequency) + _foundState = --state; + if (state.Frequency > MAXIMUM_FREQUENCY) { Rescale(context); } @@ -537,15 +537,15 @@ namespace SharpCompress.Compressors.PPMd.I1 private void Update2(PpmState state, PpmContext context) { - foundState = state; - foundState.Frequency += 4; + _foundState = state; + _foundState.Frequency += 4; context.SummaryFrequency += 4; - if (state.Frequency > MaximumFrequency) + if (state.Frequency > MAXIMUM_FREQUENCY) { Rescale(context); } - escapeCount++; - runLength = initialRunLength; + _escapeCount++; + _runLength = _initialRunLength; } private See2Context MakeEscapeFrequency(PpmContext context) @@ -559,16 +559,16 @@ namespace SharpCompress.Compressors.PPMd.I1 // dimension of the see2Contexts array is always in the range 0 .. 31). numberStatistics = context.Suffix.NumberStatistics; - int index1 = probabilities[context.NumberStatistics + 2] - 3; + int index1 = _probabilities[context.NumberStatistics + 2] - 3; int index2 = ((context.SummaryFrequency > 11 * (context.NumberStatistics + 1)) ? 1 : 0) + - ((2 * context.NumberStatistics < numberStatistics + numberMasked) ? 2 : 0) + context.Flags; - see2Context = see2Contexts[index1, index2]; - Coder.Scale = see2Context.Mean(); + ((2 * context.NumberStatistics < numberStatistics + _numberMasked) ? 2 : 0) + context.Flags; + see2Context = _see2Contexts[index1, index2]; + _coder._scale = see2Context.Mean(); } else { - see2Context = emptySee2Context; - Coder.Scale = 1; + see2Context = _emptySee2Context; + _coder._scale = 1; } return see2Context; @@ -587,7 +587,7 @@ namespace SharpCompress.Compressors.PPMd.I1 PpmState p1; PpmState state; - for (state = foundState; state != context.Statistics; state--) + for (state = _foundState; state != context.Statistics; state--) { Swap(state[0], state[-1]); } @@ -595,7 +595,7 @@ namespace SharpCompress.Compressors.PPMd.I1 state.Frequency += 4; context.SummaryFrequency += 4; escapeFrequency = (uint)(context.SummaryFrequency - state.Frequency); - adder = (orderFall != 0 || method > ModelRestorationMethod.Freeze) ? 1 : 0; + adder = (_orderFall != 0 || _method > ModelRestorationMethod.Freeze) ? 1 : 0; state.Frequency = (byte)((state.Frequency + adder) >> 1); context.SummaryFrequency = state.Frequency; @@ -639,20 +639,20 @@ namespace SharpCompress.Compressors.PPMd.I1 localFrequency = context.Statistics.Frequency; localSuccessor = context.Statistics.Successor; localFrequency = (byte)((2 * localFrequency + escapeFrequency - 1) / escapeFrequency); - if (localFrequency > MaximumFrequency / 3) + if (localFrequency > MAXIMUM_FREQUENCY / 3) { - localFrequency = (byte)(MaximumFrequency / 3); + localFrequency = (byte)(MAXIMUM_FREQUENCY / 3); } - Allocator.FreeUnits(context.Statistics, oldUnitCount); + _allocator.FreeUnits(context.Statistics, oldUnitCount); context.FirstStateSymbol = localSymbol; context.FirstStateFrequency = localFrequency; context.FirstStateSuccessor = localSuccessor; context.Flags = (byte)((context.Flags & 0x10) + ((localSymbol >= 0x40) ? 0x08 : 0x00)); - foundState = context.FirstState; + _foundState = context.FirstState; return; } - context.Statistics = Allocator.ShrinkUnits(context.Statistics, oldUnitCount, + context.Statistics = _allocator.ShrinkUnits(context.Statistics, oldUnitCount, (uint)((context.NumberStatistics + 2) >> 1)); context.Flags &= 0xf7; index = context.NumberStatistics; @@ -668,7 +668,7 @@ namespace SharpCompress.Compressors.PPMd.I1 escapeFrequency -= (escapeFrequency >> 1); context.SummaryFrequency += (ushort)escapeFrequency; context.Flags |= 0x04; - foundState = context.Statistics; + _foundState = context.Statistics; } private void Refresh(uint oldUnitCount, bool scale, PpmContext context) @@ -677,7 +677,7 @@ namespace SharpCompress.Compressors.PPMd.I1 int escapeFrequency; int scaleValue = (scale ? 1 : 0); - context.Statistics = Allocator.ShrinkUnits(context.Statistics, oldUnitCount, (uint)((index + 2) >> 1)); + context.Statistics = _allocator.ShrinkUnits(context.Statistics, oldUnitCount, (uint)((index + 2) >> 1)); PpmState statistics = context.Statistics; context.Flags = (byte)((context.Flags & (0x10 + (scale ? 0x04 : 0x00))) + ((statistics.Symbol >= 0x40) ? 0x08 : 0x00)); @@ -706,46 +706,46 @@ namespace SharpCompress.Compressors.PPMd.I1 if (context.NumberStatistics == 0) { state = context.FirstState; - if ((Pointer)state.Successor >= Allocator.BaseUnit) + if ((Pointer)state.Successor >= _allocator._baseUnit) { - if (order < modelOrder) + if (order < _modelOrder) { state.Successor = CutOff(order + 1, state.Successor); } else { - state.Successor = PpmContext.Zero; + state.Successor = PpmContext.ZERO; } - if (state.Successor == PpmContext.Zero && order > OrderBound) + if (state.Successor == PpmContext.ZERO && order > ORDER_BOUND) { - Allocator.SpecialFreeUnits(context); - return PpmContext.Zero; + _allocator.SpecialFreeUnits(context); + return PpmContext.ZERO; } return context; } - Allocator.SpecialFreeUnits(context); - return PpmContext.Zero; + _allocator.SpecialFreeUnits(context); + return PpmContext.ZERO; } uint unitCount = (uint)((context.NumberStatistics + 2) >> 1); - context.Statistics = Allocator.MoveUnitsUp(context.Statistics, unitCount); + context.Statistics = _allocator.MoveUnitsUp(context.Statistics, unitCount); index = context.NumberStatistics; for (state = context.Statistics + index; state >= context.Statistics; state--) { - if (state.Successor < Allocator.BaseUnit) + if (state.Successor < _allocator._baseUnit) { - state.Successor = PpmContext.Zero; + state.Successor = PpmContext.ZERO; Swap(state, context.Statistics[index--]); } - else if (order < modelOrder) + else if (order < _modelOrder) { state.Successor = CutOff(order + 1, state.Successor); } else { - state.Successor = PpmContext.Zero; + state.Successor = PpmContext.ZERO; } } @@ -755,15 +755,15 @@ namespace SharpCompress.Compressors.PPMd.I1 state = context.Statistics; if (index < 0) { - Allocator.FreeUnits(state, unitCount); - Allocator.SpecialFreeUnits(context); - return PpmContext.Zero; + _allocator.FreeUnits(state, unitCount); + _allocator.SpecialFreeUnits(context); + return PpmContext.ZERO; } if (index == 0) { context.Flags = (byte)((context.Flags & 0x10) + ((state.Symbol >= 0x40) ? 0x08 : 0x00)); Copy(context.FirstState, state); - Allocator.FreeUnits(state, unitCount); + _allocator.FreeUnits(state, unitCount); context.FirstStateFrequency = (byte)((context.FirstStateFrequency + 11) >> 3); } else @@ -780,32 +780,32 @@ namespace SharpCompress.Compressors.PPMd.I1 if (context.NumberStatistics == 0) { PpmState state = context.FirstState; - if ((Pointer)state.Successor >= Allocator.BaseUnit && order < modelOrder) + if ((Pointer)state.Successor >= _allocator._baseUnit && order < _modelOrder) { state.Successor = RemoveBinaryContexts(order + 1, state.Successor); } else { - state.Successor = PpmContext.Zero; + state.Successor = PpmContext.ZERO; } - if ((state.Successor == PpmContext.Zero) && + if ((state.Successor == PpmContext.ZERO) && (context.Suffix.NumberStatistics == 0 || context.Suffix.Flags == 0xff)) { - Allocator.FreeUnits(context, 1); - return PpmContext.Zero; + _allocator.FreeUnits(context, 1); + return PpmContext.ZERO; } return context; } for (PpmState state = context.Statistics + context.NumberStatistics; state >= context.Statistics; state--) { - if ((Pointer)state.Successor >= Allocator.BaseUnit && order < modelOrder) + if ((Pointer)state.Successor >= _allocator._baseUnit && order < _modelOrder) { state.Successor = RemoveBinaryContexts(order + 1, state.Successor); } else { - state.Successor = PpmContext.Zero; + state.Successor = PpmContext.ZERO; } } diff --git a/src/SharpCompress/Compressors/PPMd/I1/PpmState.cs b/src/SharpCompress/Compressors/PPMd/I1/PpmState.cs index d6c2d39f..be5f752b 100644 --- a/src/SharpCompress/Compressors/PPMd/I1/PpmState.cs +++ b/src/SharpCompress/Compressors/PPMd/I1/PpmState.cs @@ -16,34 +16,34 @@ namespace SharpCompress.Compressors.PPMd.I1 /// instance rather than just copying a reference to the same instance). /// /// - /// Note that is a field rather than a property for performance reasons. + /// Note that is a field rather than a property for performance reasons. /// /// internal struct PpmState { - public uint Address; - public byte[] Memory; - public static readonly PpmState Zero = new PpmState(0, null); - public const int Size = 6; + public uint _address; + public byte[] _memory; + public static readonly PpmState ZERO = new PpmState(0, null); + public const int SIZE = 6; /// /// Initializes a new instance of the structure. /// public PpmState(uint address, byte[] memory) { - Address = address; - Memory = memory; + _address = address; + _memory = memory; } /// /// Gets or sets the symbol. /// - public byte Symbol { get => Memory[Address]; set => Memory[Address] = value; } + public byte Symbol { get => _memory[_address]; set => _memory[_address] = value; } /// /// Gets or sets the frequency. /// - public byte Frequency { get => Memory[Address + 1]; set => Memory[Address + 1] = value; } + public byte Frequency { get => _memory[_address + 1]; set => _memory[_address + 1] = value; } /// /// Gets or sets the successor. @@ -51,14 +51,14 @@ namespace SharpCompress.Compressors.PPMd.I1 public Model.PpmContext Successor { get => new Model.PpmContext( - Memory[Address + 2] | ((uint)Memory[Address + 3]) << 8 | - ((uint)Memory[Address + 4]) << 16 | ((uint)Memory[Address + 5]) << 24, Memory); + _memory[_address + 2] | ((uint)_memory[_address + 3]) << 8 | + ((uint)_memory[_address + 4]) << 16 | ((uint)_memory[_address + 5]) << 24, _memory); set { - Memory[Address + 2] = (byte)value.Address; - Memory[Address + 3] = (byte)(value.Address >> 8); - Memory[Address + 4] = (byte)(value.Address >> 16); - Memory[Address + 5] = (byte)(value.Address >> 24); + _memory[_address + 2] = (byte)value._address; + _memory[_address + 3] = (byte)(value._address >> 8); + _memory[_address + 4] = (byte)(value._address >> 16); + _memory[_address + 5] = (byte)(value._address >> 24); } } @@ -68,7 +68,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// /// /// - public PpmState this[int offset] => new PpmState((uint)(Address + offset * Size), Memory); + public PpmState this[int offset] => new PpmState((uint)(_address + offset * SIZE), _memory); /// /// Allow a pointer to be implicitly converted to a PPM state. @@ -77,7 +77,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static implicit operator PpmState(Pointer pointer) { - return new PpmState(pointer.Address, pointer.Memory); + return new PpmState(pointer._address, pointer._memory); } /// @@ -88,7 +88,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static PpmState operator +(PpmState state, int offset) { - state.Address = (uint)(state.Address + offset * Size); + state._address = (uint)(state._address + offset * SIZE); return state; } @@ -99,7 +99,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static PpmState operator ++(PpmState state) { - state.Address += Size; + state._address += SIZE; return state; } @@ -111,7 +111,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static PpmState operator -(PpmState state, int offset) { - state.Address = (uint)(state.Address - offset * Size); + state._address = (uint)(state._address - offset * SIZE); return state; } @@ -122,7 +122,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static PpmState operator --(PpmState state) { - state.Address -= Size; + state._address -= SIZE; return state; } @@ -134,7 +134,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator <=(PpmState state1, PpmState state2) { - return state1.Address <= state2.Address; + return state1._address <= state2._address; } /// @@ -145,7 +145,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator >=(PpmState state1, PpmState state2) { - return state1.Address >= state2.Address; + return state1._address >= state2._address; } /// @@ -156,7 +156,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator ==(PpmState state1, PpmState state2) { - return state1.Address == state2.Address; + return state1._address == state2._address; } /// @@ -167,7 +167,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// public static bool operator !=(PpmState state1, PpmState state2) { - return state1.Address != state2.Address; + return state1._address != state2._address; } /// @@ -180,7 +180,7 @@ namespace SharpCompress.Compressors.PPMd.I1 if (obj is PpmState) { PpmState state = (PpmState)obj; - return state.Address == Address; + return state._address == _address; } return base.Equals(obj); } @@ -191,7 +191,7 @@ namespace SharpCompress.Compressors.PPMd.I1 /// A 32-bit signed integer that is the hash code for this instance. public override int GetHashCode() { - return Address.GetHashCode(); + return _address.GetHashCode(); } } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/PPMd/I1/See2Context.cs b/src/SharpCompress/Compressors/PPMd/I1/See2Context.cs index cc59a123..3029d85b 100644 --- a/src/SharpCompress/Compressors/PPMd/I1/See2Context.cs +++ b/src/SharpCompress/Compressors/PPMd/I1/See2Context.cs @@ -18,37 +18,37 @@ namespace SharpCompress.Compressors.PPMd.I1 /// /// /// Note that in most cases fields are used rather than properties for performance reasons (for example, - /// is a field rather than a property). + /// is a field rather than a property). /// /// internal class See2Context { - private const byte PeriodBitCount = 7; + private const byte PERIOD_BIT_COUNT = 7; - public ushort Summary; - public byte Shift; - public byte Count; + public ushort _summary; + public byte _shift; + public byte _count; public void Initialize(uint initialValue) { - Shift = PeriodBitCount - 4; - Summary = (ushort)(initialValue << Shift); - Count = 7; + _shift = PERIOD_BIT_COUNT - 4; + _summary = (ushort)(initialValue << _shift); + _count = 7; } public uint Mean() { - uint value = (uint)(Summary >> Shift); - Summary = (ushort)(Summary - value); + uint value = (uint)(_summary >> _shift); + _summary = (ushort)(_summary - value); return (uint)(value + ((value == 0) ? 1 : 0)); } public void Update() { - if (Shift < PeriodBitCount && --Count == 0) + if (_shift < PERIOD_BIT_COUNT && --_count == 0) { - Summary += Summary; - Count = (byte)(3 << Shift++); + _summary += _summary; + _count = (byte)(3 << _shift++); } } } diff --git a/src/SharpCompress/Compressors/PPMd/PpmdProperties.cs b/src/SharpCompress/Compressors/PPMd/PpmdProperties.cs index bd8b43bc..2ad8d4d9 100644 --- a/src/SharpCompress/Compressors/PPMd/PpmdProperties.cs +++ b/src/SharpCompress/Compressors/PPMd/PpmdProperties.cs @@ -5,12 +5,9 @@ namespace SharpCompress.Compressors.PPMd { public class PpmdProperties { - public PpmdVersion Version = PpmdVersion.I1; - public int ModelOrder; - internal ModelRestorationMethod ModelRestorationMethod; - private int allocatorSize; - internal Allocator Allocator; + private int _allocatorSize; + internal Allocator _allocator; public PpmdProperties() : this(16 << 20, 6) @@ -26,8 +23,12 @@ namespace SharpCompress.Compressors.PPMd { AllocatorSize = allocatorSize; ModelOrder = modelOrder; - ModelRestorationMethod = modelRestorationMethod; + RestorationMethod = modelRestorationMethod; } + + public int ModelOrder { get; } + public PpmdVersion Version { get; } = PpmdVersion.I1; + internal ModelRestorationMethod RestorationMethod { get; } public PpmdProperties(byte[] properties) { @@ -36,11 +37,11 @@ namespace SharpCompress.Compressors.PPMd ushort props = DataConverter.LittleEndian.GetUInt16(properties, 0); AllocatorSize = (((props >> 4) & 0xff) + 1) << 20; ModelOrder = (props & 0x0f) + 1; - ModelRestorationMethod = (ModelRestorationMethod)(props >> 12); + RestorationMethod = (ModelRestorationMethod)(props >> 12); } else if (properties.Length == 5) { - Version = PpmdVersion.H7z; + Version = PpmdVersion.H7Z; AllocatorSize = DataConverter.LittleEndian.GetInt32(properties, 1); ModelOrder = properties[0]; } @@ -48,23 +49,23 @@ namespace SharpCompress.Compressors.PPMd public int AllocatorSize { - get => allocatorSize; + get => _allocatorSize; set { - allocatorSize = value; + _allocatorSize = value; if (Version == PpmdVersion.I1) { - if (Allocator == null) + if (_allocator == null) { - Allocator = new Allocator(); + _allocator = new Allocator(); } - Allocator.Start(allocatorSize); + _allocator.Start(_allocatorSize); } } } public byte[] Properties => DataConverter.LittleEndian.GetBytes( (ushort) - ((ModelOrder - 1) + (((AllocatorSize >> 20) - 1) << 4) + ((ushort)ModelRestorationMethod << 12))); + ((ModelOrder - 1) + (((AllocatorSize >> 20) - 1) << 4) + ((ushort)RestorationMethod << 12))); } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/PPMd/PpmdStream.cs b/src/SharpCompress/Compressors/PPMd/PpmdStream.cs index 1169ed52..b9629d58 100644 --- a/src/SharpCompress/Compressors/PPMd/PpmdStream.cs +++ b/src/SharpCompress/Compressors/PPMd/PpmdStream.cs @@ -8,60 +8,60 @@ namespace SharpCompress.Compressors.PPMd { public class PpmdStream : Stream { - private readonly PpmdProperties properties; - private readonly Stream stream; - private readonly bool compress; - private readonly Model model; - private readonly ModelPPM modelH; - private readonly Decoder decoder; - private long position; - private bool isDisposed; + private readonly PpmdProperties _properties; + private readonly Stream _stream; + private readonly bool _compress; + private readonly Model _model; + private readonly ModelPpm _modelH; + private readonly Decoder _decoder; + private long _position; + private bool _isDisposed; public PpmdStream(PpmdProperties properties, Stream stream, bool compress) { - this.properties = properties; - this.stream = stream; - this.compress = compress; + _properties = properties; + _stream = stream; + _compress = compress; if (properties.Version == PpmdVersion.I1) { - model = new Model(); + _model = new Model(); if (compress) { - model.EncodeStart(properties); + _model.EncodeStart(properties); } else { - model.DecodeStart(stream, properties); + _model.DecodeStart(stream, properties); } } if (properties.Version == PpmdVersion.H) { - modelH = new ModelPPM(); + _modelH = new ModelPpm(); if (compress) { throw new NotImplementedException(); } - modelH.decodeInit(stream, properties.ModelOrder, properties.AllocatorSize); + _modelH.DecodeInit(stream, properties.ModelOrder, properties.AllocatorSize); } - if (properties.Version == PpmdVersion.H7z) + if (properties.Version == PpmdVersion.H7Z) { - modelH = new ModelPPM(); + _modelH = new ModelPpm(); if (compress) { throw new NotImplementedException(); } - modelH.decodeInit(null, properties.ModelOrder, properties.AllocatorSize); - decoder = new Decoder(); - decoder.Init(stream); + _modelH.DecodeInit(null, properties.ModelOrder, properties.AllocatorSize); + _decoder = new Decoder(); + _decoder.Init(stream); } } - public override bool CanRead => !compress; + public override bool CanRead => !_compress; public override bool CanSeek => false; - public override bool CanWrite => compress; + public override bool CanWrite => _compress; public override void Flush() { @@ -69,16 +69,16 @@ namespace SharpCompress.Compressors.PPMd protected override void Dispose(bool isDisposing) { - if (isDisposed) + if (_isDisposed) { return; } - isDisposed = true; + _isDisposed = true; if (isDisposing) { - if (compress) + if (_compress) { - model.EncodeBlock(stream, new MemoryStream(), true); + _model.EncodeBlock(_stream, new MemoryStream(), true); } } base.Dispose(isDisposing); @@ -86,38 +86,38 @@ namespace SharpCompress.Compressors.PPMd public override long Length => throw new NotSupportedException(); - public override long Position { get => position; set => throw new NotSupportedException(); } + public override long Position { get => _position; set => throw new NotSupportedException(); } public override int Read(byte[] buffer, int offset, int count) { - if (compress) + if (_compress) { return 0; } int size = 0; - if (properties.Version == PpmdVersion.I1) + if (_properties.Version == PpmdVersion.I1) { - size = model.DecodeBlock(stream, buffer, offset, count); + size = _model.DecodeBlock(_stream, buffer, offset, count); } - if (properties.Version == PpmdVersion.H) + if (_properties.Version == PpmdVersion.H) { int c; - while (size < count && (c = modelH.decodeChar()) >= 0) + while (size < count && (c = _modelH.DecodeChar()) >= 0) { buffer[offset++] = (byte)c; size++; } } - if (properties.Version == PpmdVersion.H7z) + if (_properties.Version == PpmdVersion.H7Z) { int c; - while (size < count && (c = modelH.decodeChar(decoder)) >= 0) + while (size < count && (c = _modelH.DecodeChar(_decoder)) >= 0) { buffer[offset++] = (byte)c; size++; } } - position += size; + _position += size; return size; } @@ -133,9 +133,9 @@ namespace SharpCompress.Compressors.PPMd public override void Write(byte[] buffer, int offset, int count) { - if (compress) + if (_compress) { - model.EncodeBlock(stream, new MemoryStream(buffer, offset, count), false); + _model.EncodeBlock(_stream, new MemoryStream(buffer, offset, count), false); } } } diff --git a/src/SharpCompress/Compressors/PPMd/PpmdVersion.cs b/src/SharpCompress/Compressors/PPMd/PpmdVersion.cs index c4d17015..8ab1aa38 100644 --- a/src/SharpCompress/Compressors/PPMd/PpmdVersion.cs +++ b/src/SharpCompress/Compressors/PPMd/PpmdVersion.cs @@ -3,7 +3,7 @@ public enum PpmdVersion { H, - H7z, + H7Z, I1 } } \ No newline at end of file diff --git a/src/SharpCompress/Compressors/Rar/Decode/Compress.cs b/src/SharpCompress/Compressors/Rar/Decode/Compress.cs deleted file mode 100644 index 3689daf0..00000000 --- a/src/SharpCompress/Compressors/Rar/Decode/Compress.cs +++ /dev/null @@ -1,28 +0,0 @@ -namespace SharpCompress.Compressors.Rar.Decode -{ - internal class Compress - { - public const int CODEBUFSIZE = 0x4000; - public const int MAXWINSIZE = 0x400000; - - //UPGRADE_NOTE: Final was removed from the declaration of 'MAXWINMASK '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - public static readonly int MAXWINMASK = (MAXWINSIZE - 1); - - public const int LOW_DIST_REP_COUNT = 16; - - public const int NC = 299; /* alphabet = {0, 1, 2, ..., NC - 1} */ - public const int DC = 60; - public const int LDC = 17; - public const int RC = 28; - - //UPGRADE_NOTE: Final was removed from the declaration of 'HUFF_TABLE_SIZE '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - public static readonly int HUFF_TABLE_SIZE = (NC + DC + RC + LDC); - public const int BC = 20; - - public const int NC20 = 298; /* alphabet = {0, 1, 2, ..., NC - 1} */ - public const int DC20 = 48; - public const int RC20 = 28; - public const int BC20 = 19; - public const int MC20 = 257; - } -} \ No newline at end of file diff --git a/src/SharpCompress/Compressors/Rar/Decode/FilterType.cs b/src/SharpCompress/Compressors/Rar/Decode/FilterType.cs deleted file mode 100644 index 24e2b0b5..00000000 --- a/src/SharpCompress/Compressors/Rar/Decode/FilterType.cs +++ /dev/null @@ -1,16 +0,0 @@ -namespace SharpCompress.Compressors.Rar.Decode -{ - internal enum FilterType - { - FILTER_NONE, - FILTER_PPM, - FILTER_E8, - FILTER_E8E9, - FILTER_UPCASETOLOW, - FILTER_AUDIO, - FILTER_RGB, - FILTER_DELTA, - FILTER_ITANIUM, - FILTER_E8E9V2 - } -} \ No newline at end of file diff --git a/src/SharpCompress/Compressors/Rar/IRarUnpack.cs b/src/SharpCompress/Compressors/Rar/IRarUnpack.cs new file mode 100644 index 00000000..0cc86a78 --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/IRarUnpack.cs @@ -0,0 +1,18 @@ +using System.IO; +using SharpCompress.Common.Rar.Headers; + +namespace SharpCompress.Compressors.Rar +{ + internal interface IRarUnpack + { + void DoUnpack(FileHeader fileHeader, Stream readStream, Stream writeStream); + void DoUnpack(); + + // eg u/i pause/resume button + bool Suspended { get; set; } + + long DestSize { get; } + int Char { get; } + int PpmEscChar { get; set; } + } +} diff --git a/src/SharpCompress/Compressors/Rar/MultiVolumeReadOnlyStream.cs b/src/SharpCompress/Compressors/Rar/MultiVolumeReadOnlyStream.cs index d5618d5f..8bdfd64b 100644 --- a/src/SharpCompress/Compressors/Rar/MultiVolumeReadOnlyStream.cs +++ b/src/SharpCompress/Compressors/Rar/MultiVolumeReadOnlyStream.cs @@ -3,7 +3,6 @@ using System.Collections.Generic; using System.IO; using SharpCompress.Common; using SharpCompress.Common.Rar; -using SharpCompress.Common.Rar.Headers; namespace SharpCompress.Compressors.Rar { @@ -39,11 +38,7 @@ namespace SharpCompress.Compressors.Rar filePartEnumerator.Dispose(); filePartEnumerator = null; } - if (currentStream != null) - { - currentStream.Dispose(); - currentStream = null; - } + currentStream = null; } } @@ -51,15 +46,11 @@ namespace SharpCompress.Compressors.Rar { maxPosition = filePartEnumerator.Current.FileHeader.CompressedSize; currentPosition = 0; - if (currentStream != null) - { - currentStream.Dispose(); - } currentStream = filePartEnumerator.Current.GetCompressedStream(); currentPartTotalReadBytes = 0; - CurrentCrc = filePartEnumerator.Current.FileHeader.FileCRC; + CurrentCrc = filePartEnumerator.Current.FileHeader.FileCrc; streamListener.FireFilePartExtractionBegin(filePartEnumerator.Current.FilePartName, filePartEnumerator.Current.FileHeader.CompressedSize, @@ -90,9 +81,9 @@ namespace SharpCompress.Compressors.Rar currentCount -= read; totalRead += read; if (((maxPosition - currentPosition) == 0) - && filePartEnumerator.Current.FileHeader.FileFlags.HasFlag(FileFlags.SPLIT_AFTER)) + && filePartEnumerator.Current.FileHeader.IsSplitAfter) { - if (filePartEnumerator.Current.FileHeader.Salt != null) + if (filePartEnumerator.Current.FileHeader.R4Salt != null) { throw new InvalidFormatException("Sharpcompress currently does not support multi-volume decryption."); } diff --git a/src/SharpCompress/Compressors/Rar/RarCrcStream.cs b/src/SharpCompress/Compressors/Rar/RarCrcStream.cs index 3760d252..329587e7 100644 --- a/src/SharpCompress/Compressors/Rar/RarCrcStream.cs +++ b/src/SharpCompress/Compressors/Rar/RarCrcStream.cs @@ -2,12 +2,15 @@ using System.IO; using SharpCompress.Common; using SharpCompress.Common.Rar.Headers; -namespace SharpCompress.Compressors.Rar { - internal class RarCrcStream : RarStream { +namespace SharpCompress.Compressors.Rar +{ + internal class RarCrcStream : RarStream + { private readonly MultiVolumeReadOnlyStream readStream; private uint currentCrc; - public RarCrcStream(Unpack unpack, FileHeader fileHeader, MultiVolumeReadOnlyStream readStream) : base(unpack, fileHeader, readStream) + public RarCrcStream(IRarUnpack unpack, FileHeader fileHeader, MultiVolumeReadOnlyStream readStream) + : base(unpack, fileHeader, readStream) { this.readStream = readStream; ResetCrc(); @@ -23,19 +26,19 @@ namespace SharpCompress.Compressors.Rar { currentCrc = 0xffffffff; } - public override int Read(byte[] buffer, int offset, int count) { var result = base.Read(buffer, offset, count); - if (result != 0) + if (result != 0) { currentCrc = RarCRC.CheckCrc(currentCrc, buffer, offset, result); - } - else if (GetCrc() != readStream.CurrentCrc) + } + else if (GetCrc() != readStream.CurrentCrc && count != 0) { // NOTE: we use the last FileHeader in a multipart volume to check CRC throw new InvalidFormatException("file crc mismatch"); } + return result; } } diff --git a/src/SharpCompress/Compressors/Rar/RarStream.cs b/src/SharpCompress/Compressors/Rar/RarStream.cs index c36f261c..5d4f13fa 100644 --- a/src/SharpCompress/Compressors/Rar/RarStream.cs +++ b/src/SharpCompress/Compressors/Rar/RarStream.cs @@ -6,7 +6,7 @@ namespace SharpCompress.Compressors.Rar { internal class RarStream : Stream { - private readonly Unpack unpack; + private readonly IRarUnpack unpack; private readonly FileHeader fileHeader; private readonly Stream readStream; @@ -22,25 +22,23 @@ namespace SharpCompress.Compressors.Rar private int outTotal; private bool isDisposed; - public RarStream(Unpack unpack, FileHeader fileHeader, Stream readStream) + public RarStream(IRarUnpack unpack, FileHeader fileHeader, Stream readStream) { this.unpack = unpack; this.fileHeader = fileHeader; this.readStream = readStream; fetch = true; - unpack.doUnpack(fileHeader, readStream, this); + unpack.DoUnpack(fileHeader, readStream, this); fetch = false; } protected override void Dispose(bool disposing) { - if (isDisposed) - { - return; + if (!isDisposed) { + isDisposed = true; + base.Dispose(disposing); + readStream.Dispose(); } - isDisposed = true; - base.Dispose(disposing); - readStream.Dispose(); } public override bool CanRead => true; @@ -76,7 +74,7 @@ namespace SharpCompress.Compressors.Rar outOffset = offset; outCount = count; fetch = true; - unpack.doUnpack(); + unpack.DoUnpack(); fetch = false; } return outTotal; diff --git a/src/SharpCompress/Compressors/Rar/Decode/AudioVariables.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/AudioVariables.cs similarity index 92% rename from src/SharpCompress/Compressors/Rar/Decode/AudioVariables.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Decode/AudioVariables.cs index cb70b1fd..c2510617 100644 --- a/src/SharpCompress/Compressors/Rar/Decode/AudioVariables.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/AudioVariables.cs @@ -1,4 +1,4 @@ -namespace SharpCompress.Compressors.Rar.Decode +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode { internal class AudioVariables { diff --git a/src/SharpCompress/Compressors/Rar/Decode/BitDecode.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/BitDecode.cs similarity index 51% rename from src/SharpCompress/Compressors/Rar/Decode/BitDecode.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Decode/BitDecode.cs index ba13c9b8..a8d499b5 100644 --- a/src/SharpCompress/Compressors/Rar/Decode/BitDecode.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/BitDecode.cs @@ -1,9 +1,9 @@ -namespace SharpCompress.Compressors.Rar.Decode +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode { internal class BitDecode : Decode { internal BitDecode() - : base(new int[Compress.BC]) + : base(new int[PackDef.BC]) { } } diff --git a/src/SharpCompress/Compressors/Rar/Decode/CodeType.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/CodeType.cs similarity index 80% rename from src/SharpCompress/Compressors/Rar/Decode/CodeType.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Decode/CodeType.cs index acbd4aac..e9703ff1 100644 --- a/src/SharpCompress/Compressors/Rar/Decode/CodeType.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/CodeType.cs @@ -1,4 +1,4 @@ -namespace SharpCompress.Compressors.Rar.Decode +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode { internal enum CodeType { diff --git a/src/SharpCompress/Compressors/Rar/Decode/Decode.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/Decode.cs similarity index 93% rename from src/SharpCompress/Compressors/Rar/Decode/Decode.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Decode/Decode.cs index dc5dacee..ceed74de 100644 --- a/src/SharpCompress/Compressors/Rar/Decode/Decode.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/Decode.cs @@ -1,4 +1,4 @@ -namespace SharpCompress.Compressors.Rar.Decode +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode { internal class Decode { diff --git a/src/SharpCompress/Compressors/Rar/Decode/DistDecode.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/DistDecode.cs similarity index 52% rename from src/SharpCompress/Compressors/Rar/Decode/DistDecode.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Decode/DistDecode.cs index 0c26c391..1a0b666b 100644 --- a/src/SharpCompress/Compressors/Rar/Decode/DistDecode.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/DistDecode.cs @@ -1,9 +1,9 @@ -namespace SharpCompress.Compressors.Rar.Decode +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode { internal class DistDecode : Decode { internal DistDecode() - : base(new int[Compress.DC]) + : base(new int[PackDef.DC]) { } } diff --git a/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/FilterType.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/FilterType.cs new file mode 100644 index 00000000..6d915e44 --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/FilterType.cs @@ -0,0 +1,10 @@ +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode +{ + internal enum FilterType : byte + { + // These values must not be changed, because we use them directly + // in RAR5 compression and decompression code. + FILTER_DELTA=0, FILTER_E8, FILTER_E8E9, FILTER_ARM, + FILTER_AUDIO, FILTER_RGB, FILTER_ITANIUM, FILTER_PPM, FILTER_NONE + } +} \ No newline at end of file diff --git a/src/SharpCompress/Compressors/Rar/Decode/LitDecode.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/LitDecode.cs similarity index 51% rename from src/SharpCompress/Compressors/Rar/Decode/LitDecode.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Decode/LitDecode.cs index 6084d0fe..8316c261 100644 --- a/src/SharpCompress/Compressors/Rar/Decode/LitDecode.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/LitDecode.cs @@ -1,9 +1,9 @@ -namespace SharpCompress.Compressors.Rar.Decode +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode { internal class LitDecode : Decode { internal LitDecode() - : base(new int[Compress.NC]) + : base(new int[PackDef.NC]) { } } diff --git a/src/SharpCompress/Compressors/Rar/Decode/LowDistDecode.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/LowDistDecode.cs similarity index 53% rename from src/SharpCompress/Compressors/Rar/Decode/LowDistDecode.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Decode/LowDistDecode.cs index 266bf9e1..a2095bd4 100644 --- a/src/SharpCompress/Compressors/Rar/Decode/LowDistDecode.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/LowDistDecode.cs @@ -1,9 +1,9 @@ -namespace SharpCompress.Compressors.Rar.Decode +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode { internal class LowDistDecode : Decode { internal LowDistDecode() - : base(new int[Compress.LDC]) + : base(new int[PackDef.LDC]) { } } diff --git a/src/SharpCompress/Compressors/Rar/Decode/MultDecode.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/MultDecode.cs similarity index 51% rename from src/SharpCompress/Compressors/Rar/Decode/MultDecode.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Decode/MultDecode.cs index d42a83eb..e6df223c 100644 --- a/src/SharpCompress/Compressors/Rar/Decode/MultDecode.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/MultDecode.cs @@ -1,9 +1,9 @@ -namespace SharpCompress.Compressors.Rar.Decode +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode { internal class MultDecode : Decode { internal MultDecode() - : base(new int[Compress.MC20]) + : base(new int[PackDef.MC20]) { } } diff --git a/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/PackDef.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/PackDef.cs new file mode 100644 index 00000000..87f1977e --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/PackDef.cs @@ -0,0 +1,48 @@ +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode +{ + internal static class PackDef + { + // 20171217 NOTE: these contants are gone from unrar src code + // seems to be more dynamic + public const int MAXWINSIZE = 0x400000; + public const int MAXWINMASK = MAXWINSIZE - 1; + + public const uint MAX_LZ_MATCH = 0x1001; + public const uint MAX3_LZ_MATCH = 0x101; // Maximum match length for RAR v3. + public const int LOW_DIST_REP_COUNT = 16; + + public const int NC = 299; /* alphabet = {0, 1, 2, ..., NC - 1} */ + public const int DC = 60; + public const int LDC = 17; + public const int RC = 28; +// 20171217: NOTE: these constants seem to have been updated in the unrar src code +// at some unknown point. updating causes decompression failure, not sure why. +// public const int NC = 306; /* alphabet = {0, 1, 2, ..., NC - 1} */ +// public const int DC = 64; +// public const int LDC = 16; +// public const int RC = 44; + public const int HUFF_TABLE_SIZE = NC + DC + RC + LDC; + public const int BC = 20; + + public const uint NC30 = 299; /* alphabet = {0, 1, 2, ..., NC - 1} */ + public const uint DC30 = 60; + public const uint LDC30 = 17; + public const uint RC30 = 28; + public const uint BC30 = 20; + public const uint HUFF_TABLE_SIZE30 = NC30 + DC30 + RC30 + LDC30; + + public const int NC20 = 298; /* alphabet = {0, 1, 2, ..., NC - 1} */ + public const int DC20 = 48; + public const int RC20 = 28; + public const int BC20 = 19; + public const int MC20 = 257; + + // Largest alphabet size among all values listed above. + public const uint LARGEST_TABLE_SIZE = 306; + + //public enum { + // CODE_HUFFMAN, CODE_LZ, CODE_REPEATLZ, CODE_CACHELZ, CODE_STARTFILE, + // CODE_ENDFILE, CODE_FILTER, CODE_FILTERDATA + //} + } +} \ No newline at end of file diff --git a/src/SharpCompress/Compressors/Rar/Decode/RepDecode.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/RepDecode.cs similarity index 51% rename from src/SharpCompress/Compressors/Rar/Decode/RepDecode.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Decode/RepDecode.cs index 3a59587f..24d72d16 100644 --- a/src/SharpCompress/Compressors/Rar/Decode/RepDecode.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Decode/RepDecode.cs @@ -1,9 +1,9 @@ -namespace SharpCompress.Compressors.Rar.Decode +namespace SharpCompress.Compressors.Rar.UnpackV1.Decode { internal class RepDecode : Decode { internal RepDecode() - : base(new int[Compress.RC]) + : base(new int[PackDef.RC]) { } } diff --git a/src/SharpCompress/Compressors/Rar/PPM/BlockTypes.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/PPM/BlockTypes.cs similarity index 62% rename from src/SharpCompress/Compressors/Rar/PPM/BlockTypes.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/PPM/BlockTypes.cs index 2550c47a..b3f79fdc 100644 --- a/src/SharpCompress/Compressors/Rar/PPM/BlockTypes.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/PPM/BlockTypes.cs @@ -1,4 +1,4 @@ -namespace SharpCompress.Compressors.Rar.PPM +namespace SharpCompress.Compressors.Rar.UnpackV1.PPM { internal enum BlockTypes { diff --git a/src/SharpCompress/Compressors/Rar/Unpack.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Unpack.cs similarity index 82% rename from src/SharpCompress/Compressors/Rar/Unpack.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Unpack.cs index 197d6e51..ca81b905 100644 --- a/src/SharpCompress/Compressors/Rar/Unpack.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Unpack.cs @@ -4,31 +4,23 @@ using System.IO; using SharpCompress.Common; using SharpCompress.Common.Rar.Headers; using SharpCompress.Compressors.PPMd.H; -using SharpCompress.Compressors.Rar.Decode; -using SharpCompress.Compressors.Rar.PPM; +using SharpCompress.Compressors.Rar.UnpackV1.Decode; +using SharpCompress.Compressors.Rar.UnpackV1.PPM; using SharpCompress.Compressors.Rar.VM; -namespace SharpCompress.Compressors.Rar +namespace SharpCompress.Compressors.Rar.UnpackV1 { - internal sealed class Unpack : Unpack20 + internal sealed partial class Unpack : BitInput, IRarUnpack { - public bool FileExtracted { // Duplicate method - // private boolean ReadEndOfBlock() throws IOException, RarException - // { - // int BitField = getbits(); - // boolean NewTable, NewFile = false; - // if ((BitField & 0x8000) != 0) { - // NewTable = true; - // addbits(1); - // } else { - // NewFile = true; - // NewTable = (BitField & 0x4000) != 0; - // addbits(2); - // } - // tablesRead = !NewTable; - // return !(NewFile || NewTable && !readTables()); - // } - get; private set; } + private readonly BitInput Inp; + + public Unpack() { + // to ease in porting Unpack50.cs + Inp = this; + } + + + public bool FileExtracted { get; private set; } public long DestSize { @@ -40,7 +32,10 @@ namespace SharpCompress.Compressors.Rar } } - public bool Suspended { set => suspended = value; } + public bool Suspended { + get => suspended; + set => suspended = value; + } public int Char { @@ -56,33 +51,25 @@ namespace SharpCompress.Compressors.Rar public int PpmEscChar { get; set; } - //UPGRADE_NOTE: Final was removed from the declaration of 'ppm '. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - private readonly ModelPPM ppm = new ModelPPM(); + private readonly ModelPpm ppm = new ModelPpm(); private readonly RarVM rarVM = new RarVM(); - /* Filters code, one entry per filter */ - //UPGRADE_ISSUE: The following fragment of code could not be parsed and was not converted. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1156'" + // Filters code, one entry per filter private readonly List filters = new List(); - /* Filters stack, several entrances of same filter are possible */ - //UPGRADE_ISSUE: The following fragment of code could not be parsed and was not converted. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1156'" + // Filters stack, several entrances of same filter are possible private readonly List prgStack = new List(); - /* - * lengths of preceding blocks, one length per filter. Used to reduce size - * required to write block length if lengths are repeating - */ - //UPGRADE_ISSUE: The following fragment of code could not be parsed and was not converted. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1156'" + // lengths of preceding blocks, one length per filter. Used to reduce size + // required to write block length if lengths are repeating private readonly List oldFilterLengths = new List(); private int lastFilter; private bool tablesRead; - //UPGRADE_NOTE: The initialization of 'unpOldTable' was moved to method 'InitBlock'. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1005'" - - private readonly byte[] unpOldTable = new byte[Compress.HUFF_TABLE_SIZE]; + private readonly byte[] unpOldTable = new byte[PackDef.HUFF_TABLE_SIZE]; private BlockTypes unpBlockType; @@ -96,25 +83,15 @@ namespace SharpCompress.Compressors.Rar private int lowDistRepCount; - public static int[] DBitLengthCounts = {4, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 14, 0, 12}; + private static readonly int[] DBitLengthCounts = {4, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 14, 0, 12}; private FileHeader fileHeader; - public Unpack() - { - window = null; - - //externalWindow = false; - suspended = false; - unpAllBuf = false; - unpSomeRead = false; - } - - public void init(byte[] window) + private void Init(byte[] window) { if (window == null) { - this.window = new byte[Compress.MAXWINSIZE]; + this.window = new byte[PackDef.MAXWINSIZE]; } else { @@ -123,55 +100,56 @@ namespace SharpCompress.Compressors.Rar //externalWindow = true; } inAddr = 0; - unpInitData(false); + UnpInitData(false); } - public void doUnpack(FileHeader fileHeader, Stream readStream, Stream writeStream) + public void DoUnpack(FileHeader fileHeader, Stream readStream, Stream writeStream) { destUnpSize = fileHeader.UncompressedSize; this.fileHeader = fileHeader; - this.writeStream = writeStream; this.readStream = readStream; - bool solid = FlagUtility.HasFlag(fileHeader.FileFlags, FileFlags.SOLID); - if (!solid) + this.writeStream = writeStream; + if (!fileHeader.IsSolid) { - init(null); + Init(null); } suspended = false; - doUnpack(); + DoUnpack(); } - public void doUnpack() + public void DoUnpack() { - bool solid = FlagUtility.HasFlag(fileHeader.FileFlags, FileFlags.SOLID); - if (fileHeader.PackingMethod == 0x30) + if (fileHeader.CompressionMethod == 0) { - unstoreFile(); + UnstoreFile(); return; } - switch (fileHeader.RarVersion) + switch (fileHeader.CompressionAlgorithm) { case 15: // rar 1.5 compression - unpack15(solid); + unpack15(fileHeader.IsSolid); break; - case 20: - - // rar 2.x compression + case 20: // rar 2.x compression case 26: // files larger than 2GB - unpack20(solid); + unpack20(fileHeader.IsSolid); break; - case 29: - - // rar 3.x compression + case 29: // rar 3.x compression case 36: // alternative hash - unpack29(solid); + Unpack29(fileHeader.IsSolid); break; + + case 50: // rar 5.x compression + Unpack5(fileHeader.IsSolid); + break; + + default: + throw new InvalidFormatException("unknown rar compression version " + fileHeader.CompressionAlgorithm); } } - private void unstoreFile() + private void UnstoreFile() { byte[] buffer = new byte[0x10000]; while (true) @@ -194,10 +172,10 @@ namespace SharpCompress.Compressors.Rar } } - private void unpack29(bool solid) + private void Unpack29(bool solid) { - int[] DDecode = new int[Compress.DC]; - byte[] DBits = new byte[Compress.DC]; + int[] DDecode = new int[PackDef.DC]; + byte[] DBits = new byte[PackDef.DC]; int Bits; @@ -219,12 +197,12 @@ namespace SharpCompress.Compressors.Rar if (!suspended) { - unpInitData(solid); + UnpInitData(solid); if (!unpReadBuf()) { return; } - if ((!solid || !tablesRead) && !readTables()) + if ((!solid || !tablesRead) && !ReadTables()) { return; } @@ -237,7 +215,7 @@ namespace SharpCompress.Compressors.Rar while (true) { - unpPtr &= Compress.MAXWINMASK; + unpPtr &= PackDef.MAXWINMASK; if (inAddr > readBorder) { @@ -249,7 +227,7 @@ namespace SharpCompress.Compressors.Rar // System.out.println(((wrPtr - unpPtr) & // Compress.MAXWINMASK)+":"+wrPtr+":"+unpPtr); - if (((wrPtr - unpPtr) & Compress.MAXWINMASK) < 260 && wrPtr != unpPtr) + if (((wrPtr - unpPtr) & PackDef.MAXWINMASK) < 260 && wrPtr != unpPtr) { UnpWriteBuf(); if (destUnpSize <= 0) @@ -264,7 +242,7 @@ namespace SharpCompress.Compressors.Rar } if (unpBlockType == BlockTypes.BLOCK_PPM) { - int Ch = ppm.decodeChar(); + int Ch = ppm.DecodeChar(); if (Ch == -1) { ppmError = true; @@ -272,10 +250,10 @@ namespace SharpCompress.Compressors.Rar } if (Ch == PpmEscChar) { - int NextCh = ppm.decodeChar(); + int NextCh = ppm.DecodeChar(); if (NextCh == 0) { - if (!readTables()) + if (!ReadTables()) { break; } @@ -287,7 +265,7 @@ namespace SharpCompress.Compressors.Rar } if (NextCh == 3) { - if (!readVMCodePPM()) + if (!ReadVMCodePPM()) { break; } @@ -299,7 +277,7 @@ namespace SharpCompress.Compressors.Rar bool failed = false; for (int I = 0; I < 4 && !failed; I++) { - int ch = ppm.decodeChar(); + int ch = ppm.DecodeChar(); if (ch == -1) { failed = true; @@ -322,17 +300,17 @@ namespace SharpCompress.Compressors.Rar { break; } - copyString(Length + 32, Distance + 2); + CopyString(Length + 32, Distance + 2); continue; } if (NextCh == 5) { - int Length = ppm.decodeChar(); + int Length = ppm.DecodeChar(); if (Length == -1) { break; } - copyString(Length + 4, 1); + CopyString(Length + 4, 1); continue; } } @@ -376,7 +354,7 @@ namespace SharpCompress.Compressors.Rar int LowDist = this.decodeNumber(LDD); if (LowDist == 16) { - lowDistRepCount = Compress.LOW_DIST_REP_COUNT - 1; + lowDistRepCount = PackDef.LOW_DIST_REP_COUNT - 1; Distance += prevLowDist; } else @@ -402,15 +380,15 @@ namespace SharpCompress.Compressors.Rar } } - insertOldDist(Distance); - insertLastMatch(Length, Distance); + InsertOldDist(Distance); + InsertLastMatch(Length, Distance); - copyString(Length, Distance); + CopyString(Length, Distance); continue; } if (Number == 256) { - if (!readEndOfBlock()) + if (!ReadEndOfBlock()) { break; } @@ -418,7 +396,7 @@ namespace SharpCompress.Compressors.Rar } if (Number == 257) { - if (!readVMCode()) + if (!ReadVMCode()) { break; } @@ -428,7 +406,7 @@ namespace SharpCompress.Compressors.Rar { if (lastLength != 0) { - copyString(lastLength, lastDist); + CopyString(lastLength, lastDist); } continue; } @@ -449,8 +427,8 @@ namespace SharpCompress.Compressors.Rar Length += Utility.URShift(GetBits(), (16 - Bits)); AddBits(Bits); } - insertLastMatch(Length, Distance); - copyString(Length, Distance); + InsertLastMatch(Length, Distance); + CopyString(Length, Distance); continue; } if (Number < 272) @@ -461,9 +439,9 @@ namespace SharpCompress.Compressors.Rar Distance += Utility.URShift(GetBits(), (16 - Bits)); AddBits(Bits); } - insertOldDist(Distance); - insertLastMatch(2, Distance); - copyString(2, Distance); + InsertOldDist(Distance); + InsertLastMatch(2, Distance); + CopyString(2, Distance); } } UnpWriteBuf(); @@ -472,7 +450,7 @@ namespace SharpCompress.Compressors.Rar private void UnpWriteBuf() { int WrittenBorder = wrPtr; - int WriteSize = (unpPtr - WrittenBorder) & Compress.MAXWINMASK; + int WriteSize = (unpPtr - WrittenBorder) & PackDef.MAXWINMASK; for (int I = 0; I < prgStack.Count; I++) { UnpackFilter flt = prgStack[I]; @@ -487,17 +465,17 @@ namespace SharpCompress.Compressors.Rar } int BlockStart = flt.BlockStart; // ->BlockStart; int BlockLength = flt.BlockLength; // ->BlockLength; - if (((BlockStart - WrittenBorder) & Compress.MAXWINMASK) < WriteSize) + if (((BlockStart - WrittenBorder) & PackDef.MAXWINMASK) < WriteSize) { if (WrittenBorder != BlockStart) { UnpWriteArea(WrittenBorder, BlockStart); WrittenBorder = BlockStart; - WriteSize = (unpPtr - WrittenBorder) & Compress.MAXWINMASK; + WriteSize = (unpPtr - WrittenBorder) & PackDef.MAXWINMASK; } if (BlockLength <= WriteSize) { - int BlockEnd = (BlockStart + BlockLength) & Compress.MAXWINMASK; + int BlockEnd = (BlockStart + BlockLength) & PackDef.MAXWINMASK; if (BlockStart < BlockEnd || BlockEnd == 0) { // VM.SetMemory(0,Window+BlockStart,BlockLength); @@ -505,7 +483,7 @@ namespace SharpCompress.Compressors.Rar } else { - int FirstPartLength = Compress.MAXWINSIZE - BlockStart; + int FirstPartLength = PackDef.MAXWINSIZE - BlockStart; // VM.SetMemory(0,Window+BlockStart,FirstPartLength); rarVM.setMemory(0, window, BlockStart, FirstPartLength); @@ -639,7 +617,7 @@ namespace SharpCompress.Compressors.Rar writtenFileSize += FilteredDataSize; destUnpSize -= FilteredDataSize; WrittenBorder = BlockEnd; - WriteSize = (unpPtr - WrittenBorder) & Compress.MAXWINMASK; + WriteSize = (unpPtr - WrittenBorder) & PackDef.MAXWINMASK; } else { @@ -669,7 +647,7 @@ namespace SharpCompress.Compressors.Rar } if (endPtr < startPtr) { - UnpWriteData(window, startPtr, -startPtr & Compress.MAXWINMASK); + UnpWriteData(window, startPtr, -startPtr & PackDef.MAXWINMASK); UnpWriteData(window, 0, endPtr); unpAllBuf = true; } @@ -696,7 +674,12 @@ namespace SharpCompress.Compressors.Rar destUnpSize -= size; } - private void insertOldDist(int distance) + private void InsertOldDist(uint distance) { + // TODO uint + InsertOldDist((int)distance); + } + + private void InsertOldDist(int distance) { oldDist[3] = oldDist[2]; oldDist[2] = oldDist[1]; @@ -704,20 +687,25 @@ namespace SharpCompress.Compressors.Rar oldDist[0] = distance; } - private void insertLastMatch(int length, int distance) + private void InsertLastMatch(int length, int distance) { lastDist = distance; lastLength = length; } - private void copyString(int length, int distance) + private void CopyString(uint length, uint distance) { + // TODO uint + CopyString((int)length, (int)distance) ; + } + + private void CopyString(int length, int distance) { // System.out.println("copyString(" + length + ", " + distance + ")"); int destPtr = unpPtr - distance; // System.out.println(unpPtr+":"+distance); - if (destPtr >= 0 && destPtr < Compress.MAXWINSIZE - 260 && unpPtr < Compress.MAXWINSIZE - 260) + if (destPtr >= 0 && destPtr < PackDef.MAXWINSIZE - 260 && unpPtr < PackDef.MAXWINSIZE - 260) { window[unpPtr++] = window[destPtr++]; @@ -730,13 +718,13 @@ namespace SharpCompress.Compressors.Rar { while (length-- != 0) { - window[unpPtr] = window[destPtr++ & Compress.MAXWINMASK]; - unpPtr = (unpPtr + 1) & Compress.MAXWINMASK; + window[unpPtr] = window[destPtr++ & PackDef.MAXWINMASK]; + unpPtr = (unpPtr + 1) & PackDef.MAXWINMASK; } } } - protected internal override void unpInitData(bool solid) + private void UnpInitData(bool solid) { if (!solid) { @@ -752,8 +740,9 @@ namespace SharpCompress.Compressors.Rar unpPtr = 0; wrPtr = 0; PpmEscChar = 2; +WriteBorder=Math.Min(MaxWinSize,UNPACK_MAX_WRITE)&MaxWinMask; - initFilters(); + InitFilters(); } InitBitInput(); ppmError = false; @@ -763,7 +752,37 @@ namespace SharpCompress.Compressors.Rar unpInitData20(solid); } - private void initFilters() +//void Unpack::UnpInitData(bool Solid) +//{ +// if (!Solid) +// { +// memset(OldDist,0,sizeof(OldDist)); +// OldDistPtr=0; +// LastDist=LastLength=0; +//// memset(Window,0,MaxWinSize); +// memset(&BlockTables,0,sizeof(BlockTables)); +// UnpPtr=WrPtr=0; +// WriteBorder=Min(MaxWinSize,UNPACK_MAX_WRITE)&MaxWinMask; +// } +// // Filters never share several solid files, so we can safely reset them +// // even in solid archive. +// InitFilters(); +// +// Inp.InitBitInput(); +// WrittenFileSize=0; +// ReadTop=0; +// ReadBorder=0; +// +// memset(&BlockHeader,0,sizeof(BlockHeader)); +// BlockHeader.BlockSize=-1; // '-1' means not defined yet. +//#ifndef SFX_MODULE +// UnpInitData20(Solid); +//#endif +// UnpInitData30(Solid); +// UnpInitData50(Solid); +//} + + private void InitFilters() { oldFilterLengths.Clear(); lastFilter = 0; @@ -773,7 +792,7 @@ namespace SharpCompress.Compressors.Rar prgStack.Clear(); } - private bool readEndOfBlock() + private bool ReadEndOfBlock() { int BitField = GetBits(); bool NewTable, NewFile = false; @@ -785,18 +804,18 @@ namespace SharpCompress.Compressors.Rar else { NewFile = true; - NewTable = (BitField & 0x4000) != 0 ? true : false; + NewTable = (BitField & 0x4000) != 0; AddBits(2); } tablesRead = !NewTable; - return !(NewFile || NewTable && !readTables()); + return !(NewFile || NewTable && !ReadTables()); } - private bool readTables() + private bool ReadTables() { - byte[] bitLength = new byte[Compress.BC]; + byte[] bitLength = new byte[PackDef.BC]; - byte[] table = new byte[Compress.HUFF_TABLE_SIZE]; + byte[] table = new byte[PackDef.HUFF_TABLE_SIZE]; if (inAddr > readTop - 25) { if (!unpReadBuf()) @@ -809,7 +828,7 @@ namespace SharpCompress.Compressors.Rar if ((bitField & 0x8000) != 0) { unpBlockType = BlockTypes.BLOCK_PPM; - return (ppm.decodeInit(this, PpmEscChar)); + return (ppm.DecodeInit(this, PpmEscChar)); } unpBlockType = BlockTypes.BLOCK_LZ; @@ -822,7 +841,7 @@ namespace SharpCompress.Compressors.Rar } AddBits(2); - for (int i = 0; i < Compress.BC; i++) + for (int i = 0; i < PackDef.BC; i++) { int length = (Utility.URShift(GetBits(), 12)) & 0xFF; AddBits(4); @@ -850,9 +869,9 @@ namespace SharpCompress.Compressors.Rar } } - UnpackUtility.makeDecodeTables(bitLength, 0, BD, Compress.BC); + UnpackUtility.makeDecodeTables(bitLength, 0, BD, PackDef.BC); - int TableSize = Compress.HUFF_TABLE_SIZE; + int TableSize = PackDef.HUFF_TABLE_SIZE; for (int i = 0; i < TableSize;) { @@ -912,10 +931,10 @@ namespace SharpCompress.Compressors.Rar { return (false); } - UnpackUtility.makeDecodeTables(table, 0, LD, Compress.NC); - UnpackUtility.makeDecodeTables(table, Compress.NC, DD, Compress.DC); - UnpackUtility.makeDecodeTables(table, Compress.NC + Compress.DC, LDD, Compress.LDC); - UnpackUtility.makeDecodeTables(table, Compress.NC + Compress.DC + Compress.LDC, RD, Compress.RC); + UnpackUtility.makeDecodeTables(table, 0, LD, PackDef.NC); + UnpackUtility.makeDecodeTables(table, PackDef.NC, DD, PackDef.DC); + UnpackUtility.makeDecodeTables(table, PackDef.NC + PackDef.DC, LDD, PackDef.LDC); + UnpackUtility.makeDecodeTables(table, PackDef.NC + PackDef.DC + PackDef.LDC, RD, PackDef.RC); // memcpy(unpOldTable,table,sizeof(unpOldTable)); @@ -923,7 +942,7 @@ namespace SharpCompress.Compressors.Rar return (true); } - private bool readVMCode() + private bool ReadVMCode() { int FirstByte = GetBits() >> 8; AddBits(8); @@ -939,7 +958,6 @@ namespace SharpCompress.Compressors.Rar AddBits(16); } - //UPGRADE_ISSUE: The following fragment of code could not be parsed and was not converted. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1156'" List vmCode = new List(); for (int I = 0; I < Length; I++) { @@ -950,12 +968,12 @@ namespace SharpCompress.Compressors.Rar vmCode.Add((byte)(GetBits() >> 8)); AddBits(8); } - return (addVMCode(FirstByte, vmCode, Length)); + return (AddVMCode(FirstByte, vmCode, Length)); } - private bool readVMCodePPM() + private bool ReadVMCodePPM() { - int FirstByte = ppm.decodeChar(); + int FirstByte = ppm.DecodeChar(); if (FirstByte == -1) { return (false); @@ -963,7 +981,7 @@ namespace SharpCompress.Compressors.Rar int Length = (FirstByte & 7) + 1; if (Length == 7) { - int B1 = ppm.decodeChar(); + int B1 = ppm.DecodeChar(); if (B1 == -1) { return (false); @@ -972,12 +990,12 @@ namespace SharpCompress.Compressors.Rar } else if (Length == 8) { - int B1 = ppm.decodeChar(); + int B1 = ppm.DecodeChar(); if (B1 == -1) { return (false); } - int B2 = ppm.decodeChar(); + int B2 = ppm.DecodeChar(); if (B2 == -1) { return (false); @@ -985,21 +1003,20 @@ namespace SharpCompress.Compressors.Rar Length = B1 * 256 + B2; } - //UPGRADE_ISSUE: The following fragment of code could not be parsed and was not converted. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1156'" List vmCode = new List(); for (int I = 0; I < Length; I++) { - int Ch = ppm.decodeChar(); + int Ch = ppm.DecodeChar(); if (Ch == -1) { return (false); } vmCode.Add((byte)Ch); // VMCode[I]=Ch; } - return (addVMCode(FirstByte, vmCode, Length)); + return (AddVMCode(FirstByte, vmCode, Length)); } - private bool addVMCode(int firstByte, List vmCode, int length) + private bool AddVMCode(int firstByte, List vmCode, int length) { BitInput Inp = new BitInput(); Inp.InitBitInput(); @@ -1017,7 +1034,7 @@ namespace SharpCompress.Compressors.Rar FiltPos = RarVM.ReadData(Inp); if (FiltPos == 0) { - initFilters(); + InitFilters(); } else { @@ -1075,7 +1092,7 @@ namespace SharpCompress.Compressors.Rar { BlockStart += 258; } - StackFilter.BlockStart = ((BlockStart + unpPtr) & Compress.MAXWINMASK); + StackFilter.BlockStart = ((BlockStart + unpPtr) & PackDef.MAXWINMASK); if ((firstByte & 0x20) != 0) { StackFilter.BlockLength = RarVM.ReadData(Inp); @@ -1084,7 +1101,7 @@ namespace SharpCompress.Compressors.Rar { StackFilter.BlockLength = FiltPos < oldFilterLengths.Count ? oldFilterLengths[FiltPos] : 0; } - StackFilter.NextWindow = (wrPtr != unpPtr) && ((wrPtr - unpPtr) & Compress.MAXWINMASK) <= BlockStart; + StackFilter.NextWindow = (wrPtr != unpPtr) && ((wrPtr - unpPtr) & PackDef.MAXWINMASK) <= BlockStart; // DebugLog("\nNextWindow: UnpPtr=%08x WrPtr=%08x // BlockStart=%08x",UnpPtr,WrPtr,BlockStart); @@ -1238,14 +1255,14 @@ namespace SharpCompress.Compressors.Rar } } - public void cleanUp() + private void CleanUp() { if (ppm != null) { SubAllocator allocator = ppm.SubAlloc; if (allocator != null) { - allocator.stopSubAllocator(); + allocator.StopSubAllocator(); } } } diff --git a/src/SharpCompress/Compressors/Rar/Unpack15.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Unpack15.cs similarity index 89% rename from src/SharpCompress/Compressors/Rar/Unpack15.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Unpack15.cs index c7c1c877..6c0e09f3 100644 --- a/src/SharpCompress/Compressors/Rar/Unpack15.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Unpack15.cs @@ -3,79 +3,61 @@ * Original author: Edmund Wagner * Creation date: 21.06.2007 * -* Source: $HeadURL$ -* Last changed: $LastChangedDate$ -* * the unrar licence applies to all junrar source and binary distributions * you are not allowed to use this source to re-create the RAR compression algorithm -* -* Here some html entities which can be used for escaping javadoc tags: -* "&": "&" or "&" -* "<": "<" or "<" -* ">": ">" or ">" -* "@": "@" */ using System; using System.IO; -using SharpCompress.Compressors.Rar.Decode; -using SharpCompress.Compressors.Rar.VM; +using SharpCompress.Compressors.Rar.UnpackV1.Decode; -namespace SharpCompress.Compressors.Rar +namespace SharpCompress.Compressors.Rar.UnpackV1 { - /// DOCUMENT ME - /// - /// - /// $LastChangedBy$ - /// - /// $LastChangedRevision$ - /// - internal abstract class Unpack15 : BitInput + internal partial class Unpack { - protected internal int readBorder; + private int readBorder; - protected internal bool suspended; + private bool suspended; - protected internal bool unpAllBuf; + internal bool unpAllBuf; - //protected internal ComprDataIO unpIO; + //private ComprDataIO unpIO; + private Stream readStream; + private Stream writeStream; - protected Stream readStream; - protected Stream writeStream; + internal bool unpSomeRead; - protected internal bool unpSomeRead; + private int readTop; - protected internal int readTop; + private long destUnpSize; - protected internal long destUnpSize; + private byte[] window; - protected internal byte[] window; + private readonly int[] oldDist = new int[4]; - protected internal int[] oldDist = new int[4]; + private int unpPtr, wrPtr; - protected internal int unpPtr, wrPtr; + private int oldDistPtr; - protected internal int oldDistPtr; - - protected internal int[] ChSet = new int[256], + private readonly int[] ChSet = new int[256], ChSetA = new int[256], ChSetB = new int[256], ChSetC = new int[256]; - protected internal int[] Place = new int[256], + private readonly int[] Place = new int[256], PlaceA = new int[256], PlaceB = new int[256], PlaceC = new int[256]; - protected internal int[] NToPl = new int[256], NToPlB = new int[256], NToPlC = new int[256]; + private readonly int[] NToPl = new int[256], NToPlB = new int[256], NToPlC = new int[256]; - protected internal int FlagBuf, AvrPlc, AvrPlcB, AvrLn1, AvrLn2, AvrLn3; + private int FlagBuf, AvrPlc, AvrPlcB, AvrLn1, AvrLn2, AvrLn3; - protected internal int Buf60, NumHuf, StMode, LCount, FlagsCnt; + private int Buf60, NumHuf, StMode, LCount, FlagsCnt; - protected internal int Nhfb, Nlzb, MaxDist3; + private int Nhfb, Nlzb, MaxDist3; - protected internal int lastDist, lastLength; + private int lastDist, lastLength; private const int STARTL1 = 2; @@ -126,25 +108,23 @@ namespace SharpCompress.Compressors.Rar private static readonly int[] PosHf4 = {0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 0, 0, 0}; - internal static int[] ShortLen1 = {1, 3, 4, 4, 5, 6, 7, 8, 8, 4, 4, 5, 6, 6, 4, 0}; + private static readonly int[] ShortLen1 = {1, 3, 4, 4, 5, 6, 7, 8, 8, 4, 4, 5, 6, 6, 4, 0}; - internal static int[] ShortXor1 = + private static readonly int[] ShortXor1 = { 0, 0xa0, 0xd0, 0xe0, 0xf0, 0xf8, 0xfc, 0xfe, 0xff, 0xc0, 0x80, 0x90, 0x98 , 0x9c, 0xb0 }; - internal static int[] ShortLen2 = {2, 3, 3, 3, 4, 4, 5, 6, 6, 4, 4, 5, 6, 6, 4, 0}; + private static readonly int[] ShortLen2 = {2, 3, 3, 3, 4, 4, 5, 6, 6, 4, 4, 5, 6, 6, 4, 0}; - internal static int[] ShortXor2 = + private static readonly int[] ShortXor2 = { 0, 0x40, 0x60, 0xa0, 0xd0, 0xe0, 0xf0, 0xf8, 0xfc, 0xc0, 0x80, 0x90, 0x98 , 0x9c, 0xb0 }; - protected internal abstract void unpInitData(bool solid); - - protected void unpack15(bool solid) + private void unpack15(bool solid) { if (suspended) { @@ -152,7 +132,7 @@ namespace SharpCompress.Compressors.Rar } else { - unpInitData(solid); + UnpInitData(solid); oldUnpInitData(solid); unpReadBuf(); if (!solid) @@ -174,13 +154,13 @@ namespace SharpCompress.Compressors.Rar while (destUnpSize >= 0) { - unpPtr &= Compress.MAXWINMASK; + unpPtr &= PackDef.MAXWINMASK; if (inAddr > readTop - 30 && !unpReadBuf()) { break; } - if (((wrPtr - unpPtr) & Compress.MAXWINMASK) < 270 && wrPtr != unpPtr) + if (((wrPtr - unpPtr) & PackDef.MAXWINMASK) < 270 && wrPtr != unpPtr) { oldUnpWriteBuf(); if (suspended) @@ -242,7 +222,7 @@ namespace SharpCompress.Compressors.Rar oldUnpWriteBuf(); } - protected bool unpReadBuf() + private bool unpReadBuf() { int dataSize = readTop - inAddr; if (dataSize < 0) @@ -702,8 +682,8 @@ namespace SharpCompress.Compressors.Rar destUnpSize -= Length; while ((Length--) != 0) { - window[unpPtr] = window[(unpPtr - Distance) & Compress.MAXWINMASK]; - unpPtr = (unpPtr + 1) & Compress.MAXWINMASK; + window[unpPtr] = window[(unpPtr - Distance) & PackDef.MAXWINMASK]; + unpPtr = (unpPtr + 1) & PackDef.MAXWINMASK; } } @@ -718,7 +698,7 @@ namespace SharpCompress.Compressors.Rar return ((Utility.URShift((Num - (I != 0 ? DecTab[I - 1] : 0)), (16 - StartPos))) + PosTab[StartPos]); } - protected void oldUnpWriteBuf() + private void oldUnpWriteBuf() { if (unpPtr != wrPtr) { @@ -726,7 +706,7 @@ namespace SharpCompress.Compressors.Rar } if (unpPtr < wrPtr) { - writeStream.Write(window, wrPtr, -wrPtr & Compress.MAXWINMASK); + writeStream.Write(window, wrPtr, -wrPtr & PackDef.MAXWINMASK); writeStream.Write(window, 0, unpPtr); unpAllBuf = true; } diff --git a/src/SharpCompress/Compressors/Rar/Unpack20.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Unpack20.cs similarity index 78% rename from src/SharpCompress/Compressors/Rar/Unpack20.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/Unpack20.cs index 608442c1..db8e8cd7 100644 --- a/src/SharpCompress/Compressors/Rar/Unpack20.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Unpack20.cs @@ -3,78 +3,48 @@ * Original author: Edmund Wagner * Creation date: 21.06.2007 * -* Source: $HeadURL$ -* Last changed: $LastChangedDate$ -* * the unrar licence applies to all junrar source and binary distributions * you are not allowed to use this source to re-create the RAR compression algorithm -* -* Here some html entities which can be used for escaping javadoc tags: -* "&": "&" or "&" -* "<": "<" or "<" -* ">": ">" or ">" -* "@": "@" */ using System; -using SharpCompress.Compressors.Rar.Decode; +using SharpCompress.Compressors.Rar.UnpackV1.Decode; -namespace SharpCompress.Compressors.Rar +namespace SharpCompress.Compressors.Rar.UnpackV1 { - /// DOCUMENT ME - /// - /// - /// $LastChangedBy$ - /// - /// $LastChangedRevision$ - /// - internal abstract class Unpack20 : Unpack15 + internal partial class Unpack { - public Unpack20() - { - InitBlock(); - } + private readonly MultDecode[] MD = new MultDecode[4]; - private void InitBlock() - { - UnpOldTable20 = new byte[Compress.MC20 * 4]; - } + private readonly byte[] UnpOldTable20 = new byte[PackDef.MC20 * 4]; - protected internal MultDecode[] MD = new MultDecode[4]; - - //UPGRADE_NOTE: The initialization of 'UnpOldTable20' was moved to method 'InitBlock'. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1005'" - protected internal byte[] UnpOldTable20; - - protected internal int UnpAudioBlock, UnpChannels, UnpCurChannel, UnpChannelDelta; + private int UnpAudioBlock, UnpChannels, UnpCurChannel, UnpChannelDelta; private readonly AudioVariables[] AudV = new AudioVariables[4]; - protected internal LitDecode LD = new LitDecode(); + private readonly LitDecode LD = new LitDecode(); - protected internal DistDecode DD = new DistDecode(); + private readonly DistDecode DD = new DistDecode(); - protected internal LowDistDecode LDD = new LowDistDecode(); + private readonly LowDistDecode LDD = new LowDistDecode(); - protected internal RepDecode RD = new RepDecode(); + private readonly RepDecode RD = new RepDecode(); - protected internal BitDecode BD = new BitDecode(); + private readonly BitDecode BD = new BitDecode(); - //UPGRADE_NOTE: Final was removed from the declaration of 'LDecode'. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - public static readonly int[] LDecode = + private static readonly int[] LDecode = { 0, 1, 2, 3, 4, 5, 6, 7, 8, 10, 12, 14, 16, 20, 24, 28, 32, 40, 48, 56, 64, 80, 96, 112, 128, 160, 192, 224 }; - //UPGRADE_NOTE: Final was removed from the declaration of 'LBits'. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - public static readonly byte[] LBits = + private static readonly byte[] LBits = { 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5 }; - //UPGRADE_NOTE: Final was removed from the declaration of 'DDecode'. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - public static readonly int[] DDecode = + private static readonly int[] DDecode = { 0, 1, 2, 3, 4, 6, 8, 12, 16, 24, 32, 48, 64, 96, 128, 192, 256, 384, 512, 768, 1024, 1536, 2048, 3072, 4096, 6144, 8192, 12288, 16384, @@ -83,21 +53,18 @@ namespace SharpCompress.Compressors.Rar 917504, 983040 }; - //UPGRADE_NOTE: Final was removed from the declaration of 'DBits'. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - public static readonly int[] DBits = + private static readonly int[] DBits = { 0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16, 16, 16, 16, 16, 16 , 16, 16, 16, 16, 16, 16, 16 }; - //UPGRADE_NOTE: Final was removed from the declaration of 'SDDecode'. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - public static readonly int[] SDDecode = {0, 4, 8, 16, 32, 64, 128, 192}; + private static readonly int[] SDDecode = {0, 4, 8, 16, 32, 64, 128, 192}; - //UPGRADE_NOTE: Final was removed from the declaration of 'SDBits'. "ms-help://MS.VSCC.v80/dv_commoner/local/redirect.htm?index='!DefaultContextWindowIndex'&keyword='jlca1003'" - public static readonly int[] SDBits = {2, 2, 3, 4, 5, 6, 6, 6}; + private static readonly int[] SDBits = {2, 2, 3, 4, 5, 6, 6, 6}; - internal void unpack20(bool solid) + private void unpack20(bool solid) { int Bits; @@ -107,7 +74,7 @@ namespace SharpCompress.Compressors.Rar } else { - unpInitData(solid); + UnpInitData(solid); if (!unpReadBuf()) { return; @@ -124,7 +91,7 @@ namespace SharpCompress.Compressors.Rar while (destUnpSize >= 0) { - unpPtr &= Compress.MAXWINMASK; + unpPtr &= PackDef.MAXWINMASK; if (inAddr > readTop - 30) { @@ -133,7 +100,7 @@ namespace SharpCompress.Compressors.Rar break; } } - if (((wrPtr - unpPtr) & Compress.MAXWINMASK) < 270 && wrPtr != unpPtr) + if (((wrPtr - unpPtr) & PackDef.MAXWINMASK) < 270 && wrPtr != unpPtr) { oldUnpWriteBuf(); if (suspended) @@ -258,7 +225,7 @@ namespace SharpCompress.Compressors.Rar destUnpSize -= Length; int DestPtr = unpPtr - Distance; - if (DestPtr < Compress.MAXWINSIZE - 300 && unpPtr < Compress.MAXWINSIZE - 300) + if (DestPtr < PackDef.MAXWINSIZE - 300 && unpPtr < PackDef.MAXWINSIZE - 300) { window[unpPtr++] = window[DestPtr++]; window[unpPtr++] = window[DestPtr++]; @@ -272,16 +239,16 @@ namespace SharpCompress.Compressors.Rar { while ((Length--) != 0) { - window[unpPtr] = window[DestPtr++ & Compress.MAXWINMASK]; - unpPtr = (unpPtr + 1) & Compress.MAXWINMASK; + window[unpPtr] = window[DestPtr++ & PackDef.MAXWINMASK]; + unpPtr = (unpPtr + 1) & PackDef.MAXWINMASK; } } } private bool ReadTables20() { - byte[] BitLength = new byte[Compress.BC20]; - byte[] Table = new byte[Compress.MC20 * 4]; + byte[] BitLength = new byte[PackDef.BC20]; + byte[] Table = new byte[PackDef.MC20 * 4]; int TableSize, N, I; if (inAddr > readTop - 25) { @@ -308,18 +275,18 @@ namespace SharpCompress.Compressors.Rar UnpCurChannel = 0; } AddBits(2); - TableSize = Compress.MC20 * UnpChannels; + TableSize = PackDef.MC20 * UnpChannels; } else { - TableSize = Compress.NC20 + Compress.DC20 + Compress.RC20; + TableSize = PackDef.NC20 + PackDef.DC20 + PackDef.RC20; } - for (I = 0; I < Compress.BC20; I++) + for (I = 0; I < PackDef.BC20; I++) { BitLength[I] = (byte)(Utility.URShift(GetBits(), 12)); AddBits(4); } - UnpackUtility.makeDecodeTables(BitLength, 0, BD, Compress.BC20); + UnpackUtility.makeDecodeTables(BitLength, 0, BD, PackDef.BC20); I = 0; while (I < TableSize) { @@ -372,14 +339,14 @@ namespace SharpCompress.Compressors.Rar { for (I = 0; I < UnpChannels; I++) { - UnpackUtility.makeDecodeTables(Table, I * Compress.MC20, MD[I], Compress.MC20); + UnpackUtility.makeDecodeTables(Table, I * PackDef.MC20, MD[I], PackDef.MC20); } } else { - UnpackUtility.makeDecodeTables(Table, 0, LD, Compress.NC20); - UnpackUtility.makeDecodeTables(Table, Compress.NC20, DD, Compress.DC20); - UnpackUtility.makeDecodeTables(Table, Compress.NC20 + Compress.DC20, RD, Compress.RC20); + UnpackUtility.makeDecodeTables(Table, 0, LD, PackDef.NC20); + UnpackUtility.makeDecodeTables(Table, PackDef.NC20, DD, PackDef.DC20); + UnpackUtility.makeDecodeTables(Table, PackDef.NC20 + PackDef.DC20, RD, PackDef.RC20); } // memcpy(UnpOldTable20,Table,sizeof(UnpOldTable20)); @@ -390,7 +357,7 @@ namespace SharpCompress.Compressors.Rar return (true); } - protected void unpInitData20(bool Solid) + private void unpInitData20(bool Solid) { if (!Solid) { diff --git a/src/SharpCompress/Compressors/Rar/UnpackV1/Unpack50.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/Unpack50.cs new file mode 100644 index 00000000..274692cf --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/Unpack50.cs @@ -0,0 +1,841 @@ +#if true +using System; +using System.Collections.Generic; +using SharpCompress.Compressors.Rar.UnpackV1.Decode; +using SharpCompress.Compressors.Rar.VM; + +using size_t=System.UInt32; +using UnpackBlockHeader = SharpCompress.Compressors.Rar.UnpackV1; + +namespace SharpCompress.Compressors.Rar.UnpackV1 +{ + internal partial class Unpack + { + + // Maximum allowed number of compressed bits processed in quick mode. + private const int MAX_QUICK_DECODE_BITS = 10; + + // Maximum number of filters per entire data block. Must be at least + // twice more than MAX_PACK_FILTERS to store filters from two data blocks. + private const int MAX_UNPACK_FILTERS = 8192; + + // Maximum number of filters per entire data block for RAR3 unpack. + // Must be at least twice more than v3_MAX_PACK_FILTERS to store filters + // from two data blocks. + private const int MAX3_UNPACK_FILTERS = 8192; + + // Limit maximum number of channels in RAR3 delta filter to some reasonable + // value to prevent too slow processing of corrupt archives with invalid + // channels number. Must be equal or larger than v3_MAX_FILTER_CHANNELS. + // No need to provide it for RAR5, which uses only 5 bits to store channels. + private const int MAX3_UNPACK_CHANNELS = 1024; + + // Maximum size of single filter block. We restrict it to limit memory + // allocation. Must be equal or larger than MAX_ANALYZE_SIZE. + private const int MAX_FILTER_BLOCK_SIZE = 0x400000; + + // Write data in 4 MB or smaller blocks. Must not exceed PACK_MAX_WRITE, + // so we keep number of buffered filter in unpacker reasonable. + private const int UNPACK_MAX_WRITE = 0x400000; + + // Decode compressed bit fields to alphabet numbers. +// struct DecodeTable +// { +// // Real size of DecodeNum table. +// public uint MaxNum; +// +// // Left aligned start and upper limit codes defining code space +// // ranges for bit lengths. DecodeLen[BitLength-1] defines the start of +// // range for bit length and DecodeLen[BitLength] defines next code +// // after the end of range or in other words the upper limit code +// // for specified bit length. +// //uint DecodeLen[16]; +// public uint [] DecodeLen = new uint[16]; +// +// // Every item of this array contains the sum of all preceding items. +// // So it contains the start position in code list for every bit length. +// public uint DecodePos[16]; +// +// // Number of compressed bits processed in quick mode. +// // Must not exceed MAX_QUICK_DECODE_BITS. +// public uint QuickBits; +// +// // Translates compressed bits (up to QuickBits length) +// // to bit length in quick mode. +// public byte QuickLen[1< Filters { get { return filters; } } + +// TODO: make sure these aren't already somewhere else +public int BlockSize; +public int BlockBitSize; +public int BlockStart; +public int HeaderSize; +public bool LastBlockInFile; +public bool TablePresent; + + public void Unpack5(bool Solid) { + FileExtracted=true; + + if (!Suspended) + { + UnpInitData(Solid); + if (!UnpReadBuf()) + return; + + // Check TablesRead5 to be sure that we read tables at least once + // regardless of current block header TablePresent flag. + // So we can safefly use these tables below. + if (!ReadBlockHeader() || + !ReadTables() || !TablesRead5) + return; + } + + while (true) + { + UnpPtr &= MaxWinMask; + + if (Inp.InAddr>=ReadBorder) + { + bool FileDone=false; + + // We use 'while', because for empty block containing only Huffman table, + // we'll be on the block border once again just after reading the table. + while (Inp.InAddr>BlockHeader.BlockStart+BlockHeader.BlockSize-1 || + Inp.InAddr==BlockHeader.BlockStart+BlockHeader.BlockSize-1 && + Inp.InBit>=BlockHeader.BlockBitSize) + { + if (BlockHeader.LastBlockInFile) + { + FileDone=true; + break; + } + if (!ReadBlockHeader() || !ReadTables()) + return; + } + if (FileDone || !UnpReadBuf()) + break; + } + + if (((WriteBorder-UnpPtr) & MaxWinMask)DestUnpSize) + return; + if (Suspended) + { + FileExtracted=false; + return; + } + } + + //uint MainSlot=DecodeNumber(Inp,LD); + uint MainSlot= this.DecodeNumber(LD); + if (MainSlot<256) + { +// if (Fragmented) +// FragWindow[UnpPtr++]=(byte)MainSlot; +// else + Window[UnpPtr++]=(byte)MainSlot; + continue; + } + if (MainSlot>=262) + { + uint Length=SlotToLength(MainSlot-262); + + //uint DBits,Distance=1,DistSlot=DecodeNumber(Inp,&BlockTables.DD); + int DBits; + uint Distance=1,DistSlot=this.DecodeNumber(DD); + if (DistSlot<4) + { + DBits=0; + Distance+=DistSlot; + } + else + { + //DBits=DistSlot/2 - 1; + DBits=(int)(DistSlot/2 - 1); + Distance+=(2 | (DistSlot & 1)) << DBits; + } + + if (DBits>0) + { + if (DBits>=4) + { + if (DBits>4) + { + Distance+=((Inp.getbits()>>(36-DBits))<<4); + Inp.AddBits(DBits-4); + } + //uint LowDist=DecodeNumber(Inp,&BlockTables.LDD); + uint LowDist=this.DecodeNumber(LDD); + Distance+=LowDist; + } + else + { + Distance+=Inp.getbits()>>(32-DBits); + Inp.AddBits(DBits); + } + } + + if (Distance>0x100) + { + Length++; + if (Distance>0x2000) + { + Length++; + if (Distance>0x40000) + Length++; + } + } + + InsertOldDist(Distance); + LastLength=Length; +// if (Fragmented) +// FragWindow.CopyString(Length,Distance,UnpPtr,MaxWinMask); +// else + CopyString(Length,Distance); + continue; + } + if (MainSlot==256) + { + UnpackFilter Filter = new UnpackFilter(); + if (!ReadFilter(Filter) || !AddFilter(Filter)) + break; + continue; + } + if (MainSlot==257) + { + if (LastLength!=0) +// if (Fragmented) +// FragWindow.CopyString(LastLength,OldDist[0],UnpPtr,MaxWinMask); +// else + //CopyString(LastLength,OldDist[0]); + CopyString(LastLength,OldDistN(0)); + continue; + } + if (MainSlot<262) + { + //uint DistNum=MainSlot-258; + int DistNum=(int)(MainSlot-258); + //uint Distance=OldDist[DistNum]; + uint Distance=OldDistN(DistNum); + //for (uint I=DistNum;I>0;I--) + for (int I=DistNum;I>0;I--) + //OldDistN[I]=OldDistN(I-1); + SetOldDistN(I, OldDistN(I-1)); + //OldDistN[0]=Distance; + SetOldDistN(0, Distance); + + uint LengthSlot=this.DecodeNumber(RD); + uint Length=SlotToLength(LengthSlot); + LastLength=Length; +// if (Fragmented) +// FragWindow.CopyString(Length,Distance,UnpPtr,MaxWinMask); +// else + CopyString(Length,Distance); + continue; + } + } + UnpWriteBuf(); + } + + private uint ReadFilterData() + { + uint ByteCount=(Inp.fgetbits()>>14)+1; + Inp.AddBits(2); + + uint Data=0; + //for (uint I=0;I>8)<<(I*8); + Inp.AddBits(8); + } + return Data; + } + + private bool ReadFilter(UnpackFilter Filter) + { + if (!Inp.ExternalBuffer && Inp.InAddr>ReadTop-16) + if (!UnpReadBuf()) + return false; + + Filter.uBlockStart=ReadFilterData(); + Filter.uBlockLength=ReadFilterData(); + if (Filter.BlockLength>MAX_FILTER_BLOCK_SIZE) + Filter.BlockLength=0; + + //Filter.Type=Inp.fgetbits()>>13; + Filter.Type=(byte)(Inp.fgetbits()>>13); + Inp.faddbits(3); + + if (Filter.Type==(byte)FilterType.FILTER_DELTA) + { + //Filter.Channels=(Inp.fgetbits()>>11)+1; + Filter.Channels=(byte)((Inp.fgetbits()>>11)+1); + Inp.faddbits(5); + } + + return true; + } + + private bool AddFilter(UnpackFilter Filter) + { + if (Filters.Count>=MAX_UNPACK_FILTERS) + { + UnpWriteBuf(); // Write data, apply and flush filters. + if (Filters.Count>=MAX_UNPACK_FILTERS) + InitFilters(); // Still too many filters, prevent excessive memory use. + } + + // If distance to filter start is that large that due to circular dictionary + // mode now it points to old not written yet data, then we set 'NextWindow' + // flag and process this filter only after processing that older data. + Filter.NextWindow=WrPtr!=UnpPtr && ((WrPtr-UnpPtr)&MaxWinMask)<=Filter.BlockStart; + + Filter.uBlockStart=(uint)((Filter.BlockStart+UnpPtr)&MaxWinMask); + Filters.Add(Filter); + return true; + } + + private bool UnpReadBuf() + { + int DataSize=ReadTop-Inp.InAddr; // Data left to process. + if (DataSize<0) + return false; + BlockHeader.BlockSize-=Inp.InAddr-BlockHeader.BlockStart; + if (Inp.InAddr>MAX_SIZE/2) + { + // If we already processed more than half of buffer, let's move + // remaining data into beginning to free more space for new data + // and ensure that calling function does not cross the buffer border + // even if we did not read anything here. Also it ensures that read size + // is not less than CRYPT_BLOCK_SIZE, so we can align it without risk + // to make it zero. + if (DataSize>0) + //memmove(Inp.InBuf,Inp.InBuf+Inp.InAddr,DataSize); + Array.Copy(InBuf, inAddr, InBuf, 0, DataSize); + // TODO: perf + //Buffer.BlockCopy(InBuf, inAddr, InBuf, 0, DataSize); + + Inp.InAddr=0; + ReadTop=DataSize; + } + else + DataSize=ReadTop; + int ReadCode=0; + if (MAX_SIZE!=DataSize) + //ReadCode=UnpIO->UnpRead(Inp.InBuf+DataSize,BitInput.MAX_SIZE-DataSize); + ReadCode = readStream.Read(InBuf, DataSize, MAX_SIZE-DataSize); + if (ReadCode>0) // Can be also -1. + ReadTop+=ReadCode; + ReadBorder=ReadTop-30; + BlockHeader.BlockStart=Inp.InAddr; + if (BlockHeader.BlockSize!=-1) // '-1' means not defined yet. + { + // We may need to quit from main extraction loop and read new block header + // and trees earlier than data in input buffer ends. + ReadBorder=Math.Min(ReadBorder,BlockHeader.BlockStart+BlockHeader.BlockSize-1); + } + return ReadCode!=-1; + } + +//? +// void UnpWriteBuf() +// { +// size_t WrittenBorder=WrPtr; +// size_t FullWriteSize=(UnpPtr-WrittenBorder)&MaxWinMask; +// size_t WriteSizeLeft=FullWriteSize; +// bool NotAllFiltersProcessed=false; +// for (size_t I=0;IType==FilterType.FILTER_NONE) +// continue; +// if (flt->NextWindow) +// { +// // Here we skip filters which have block start in current data range +// // due to address wrap around in circular dictionary, but actually +// // belong to next dictionary block. If such filter start position +// // is included to current write range, then we reset 'NextWindow' flag. +// // In fact we can reset it even without such check, because current +// // implementation seems to guarantee 'NextWindow' flag reset after +// // buffer writing for all existing filters. But let's keep this check +// // just in case. Compressor guarantees that distance between +// // filter block start and filter storing position cannot exceed +// // the dictionary size. So if we covered the filter block start with +// // our write here, we can safely assume that filter is applicable +// // to next block on no further wrap arounds is possible. +// if (((flt->BlockStart-WrPtr)&MaxWinMask)<=FullWriteSize) +// flt->NextWindow=false; +// continue; +// } +// uint BlockStart=flt->BlockStart; +// uint BlockLength=flt->BlockLength; +// if (((BlockStart-WrittenBorder)&MaxWinMask)0) // We set it to 0 also for invalid filters. +// { +// uint BlockEnd=(BlockStart+BlockLength)&MaxWinMask; +// +// FilterSrcMemory.Alloc(BlockLength); +// byte *Mem=&FilterSrcMemory[0]; +// if (BlockStartUnpWrite(OutMem,BlockLength); +// +// UnpSomeRead=true; +// WrittenFileSize+=BlockLength; +// WrittenBorder=BlockEnd; +// WriteSizeLeft=(UnpPtr-WrittenBorder)&MaxWinMask; +// } +// } +// else +// { +// // Current filter intersects the window write border, so we adjust +// // the window border to process this filter next time, not now. +// WrPtr=WrittenBorder; +// +// // Since Filter start position can only increase, we quit processing +// // all following filters for this data block and reset 'NextWindow' +// // flag for them. +// for (size_t J=I;JType!=FilterType.FILTER_NONE) +// flt->NextWindow=false; +// } +// +// // Do not write data left after current filter now. +// NotAllFiltersProcessed=true; +// break; +// } +// } +// } +// +// // Remove processed filters from queue. +// size_t EmptyCount=0; +// for (size_t I=0;I0) +// Filters[I-EmptyCount]=Filters[I]; +// if (Filters[I].Type==FilterType.FILTER_NONE) +// EmptyCount++; +// } +// if (EmptyCount>0) +// Filters.Alloc(Filters.Size()-EmptyCount); +// +// if (!NotAllFiltersProcessed) // Only if all filters are processed. +// { +// // Write data left after last filter. +// UnpWriteArea(WrittenBorder,UnpPtr); +// WrPtr=UnpPtr; +// } +// +// // We prefer to write data in blocks not exceeding UNPACK_MAX_WRITE +// // instead of potentially huge MaxWinSize blocks. It also allows us +// // to keep the size of Filters array reasonable. +// WriteBorder=(UnpPtr+Min(MaxWinSize,UNPACK_MAX_WRITE))&MaxWinMask; +// +// // Choose the nearest among WriteBorder and WrPtr actual written border. +// // If border is equal to UnpPtr, it means that we have MaxWinSize data ahead. +// if (WriteBorder==UnpPtr || +// WrPtr!=UnpPtr && ((WrPtr-UnpPtr)&MaxWinMask)<((WriteBorder-UnpPtr)&MaxWinMask)) +// WriteBorder=WrPtr; +// } + + +// unused + //x byte* ApplyFilter(byte *Data,uint DataSize,UnpackFilter *Flt) +// byte[] ApplyFilter(byte []Data, uint DataSize, UnpackFilter Flt) +// { +// //x byte *SrcData=Data; +// byte []SrcData=Data; +// switch(Flt.Type) +// { +// case (byte)FilterType.FILTER_E8: +// case (byte)FilterType.FILTER_E8E9: +// { +// uint FileOffset=(uint)WrittenFileSize; +// +// const uint FileSize=0x1000000; +// byte CmpByte2=Flt.Type==(byte)FilterType.FILTER_E8E9 ? (byte)0xe9 : (byte)0xe8; +// // DataSize is unsigned, so we use "CurPos+4" and not "DataSize-4" +// // to avoid overflow for DataSize<4. +// for (uint CurPos=0;CurPos+4=0 +// RawPut4(Addr+FileSize,Data); +// } +// else +// if (((Addr-FileSize) & 0x80000000)!=0) // Addr>8); +// D[2]=(byte)(Offset>>16); +// } +// } +// } +// return SrcData; +// case (byte)FilterType.FILTER_DELTA: +// { +// // Unlike RAR3, we do not need to reject excessive channel +// // values here, since RAR5 uses only 5 bits to store channel. +// uint Channels=Flt->Channels,SrcPos=0; +// +// FilterDstMemory.Alloc(DataSize); +// byte *DstData=&FilterDstMemory[0]; +// +// // Bytes from same channels are grouped to continual data blocks, +// // so we need to place them back to their interleaving positions. +// for (uint CurChannel=0;CurChannel0) +//// { +//// size_t BlockSize=FragWindow.GetBlockSize(StartPtr,SizeToWrite); +//// UnpWriteData(&FragWindow[StartPtr],BlockSize); +//// SizeToWrite-=BlockSize; +//// StartPtr=(StartPtr+BlockSize) & MaxWinMask; +//// } +//// } +//// else +// if (EndPtr=DestUnpSize) +// return; +// size_t WriteSize=Size; +// long LeftToWrite=DestUnpSize-WrittenFileSize; +// if ((long)WriteSize>LeftToWrite) +// WriteSize=(size_t)LeftToWrite; +// UnpIO->UnpWrite(Data,WriteSize); +// WrittenFileSize+=Size; +// } + + private void UnpInitData50(bool Solid) + { + if (!Solid) + TablesRead5=false; + } + + private bool ReadBlockHeader() + { + Header.HeaderSize=0; + + if (!Inp.ExternalBuffer && Inp.InAddr>ReadTop-7) + if (!UnpReadBuf()) + return false; + //Inp.faddbits((8-Inp.InBit)&7); + Inp.faddbits((uint)((8-Inp.InBit)&7)); + + byte BlockFlags=(byte)(Inp.fgetbits()>>8); + Inp.faddbits(8); + //uint ByteCount=((BlockFlags>>3)&3)+1; // Block size byte count. + uint ByteCount=(uint)(((BlockFlags>>3)&3)+1); // Block size byte count. + + if (ByteCount==4) + return false; + + //Header.HeaderSize=2+ByteCount; + Header.HeaderSize=(int)(2+ByteCount); + + Header.BlockBitSize=(BlockFlags&7)+1; + + byte SavedCheckSum=(byte)(Inp.fgetbits()>>8); + Inp.faddbits(8); + + int BlockSize=0; + //for (uint I=0;I>8)<<(I*8); + BlockSize+=(int)(Inp.fgetbits()>>8)<<(I*8); + Inp.AddBits(8); + } + + Header.BlockSize=BlockSize; + byte CheckSum=(byte)(0x5a^BlockFlags^BlockSize^(BlockSize>>8)^(BlockSize>>16)); + if (CheckSum!=SavedCheckSum) + return false; + + Header.BlockStart=Inp.InAddr; + ReadBorder=Math.Min(ReadBorder,Header.BlockStart+Header.BlockSize-1); + + Header.LastBlockInFile=(BlockFlags & 0x40)!=0; + Header.TablePresent=(BlockFlags & 0x80)!=0; + return true; + } + +//? +// bool ReadTables(BitInput Inp, ref UnpackBlockHeader Header, ref UnpackBlockTables Tables) +// { +// if (!Header.TablePresent) +// return true; +// +// if (!Inp.ExternalBuffer && Inp.InAddr>ReadTop-25) +// if (!UnpReadBuf()) +// return false; +// +// byte BitLength[BC]; +// for (uint I=0;I> 12); +// Inp.faddbits(4); +// if (Length==15) +// { +// uint ZeroCount=(byte)(Inp.fgetbits() >> 12); +// Inp.faddbits(4); +// if (ZeroCount==0) +// BitLength[I]=15; +// else +// { +// ZeroCount+=2; +// while (ZeroCount-- > 0 && IReadTop-5) +// if (!UnpReadBuf()) +// return false; +// uint Number=DecodeNumber(Inp,&Tables.BD); +// if (Number<16) +// { +// Table[I]=Number; +// I++; +// } +// else +// if (Number<18) +// { +// uint N; +// if (Number==16) +// { +// N=(Inp.fgetbits() >> 13)+3; +// Inp.faddbits(3); +// } +// else +// { +// N=(Inp.fgetbits() >> 9)+11; +// Inp.faddbits(7); +// } +// if (I==0) +// { +// // We cannot have "repeat previous" code at the first position. +// // Multiple such codes would shift Inp position without changing I, +// // which can lead to reading beyond of Inp boundary in mutithreading +// // mode, where Inp.ExternalBuffer disables bounds check and we just +// // reserve a lot of buffer space to not need such check normally. +// return false; +// } +// else +// while (N-- > 0 && I> 13)+3; +// Inp.faddbits(3); +// } +// else +// { +// N=(Inp.fgetbits() >> 9)+11; +// Inp.faddbits(7); +// } +// while (N-- > 0 && IReadTop) +// return false; +// MakeDecodeTables(&Table[0],&Tables.LD,NC); +// MakeDecodeTables(&Table[NC],&Tables.DD,DC); +// MakeDecodeTables(&Table[NC+DC],&Tables.LDD,LDC); +// MakeDecodeTables(&Table[NC+DC+LDC],&Tables.RD,RC); +// return true; +// } + +//? +// void InitFilters() +// { +// Filters.SoftReset(); +// } + + } +} +#endif \ No newline at end of file diff --git a/src/SharpCompress/Compressors/Rar/UnpackFilter.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/UnpackFilter.cs similarity index 65% rename from src/SharpCompress/Compressors/Rar/UnpackFilter.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/UnpackFilter.cs index d199a7e6..66d5d887 100644 --- a/src/SharpCompress/Compressors/Rar/UnpackFilter.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/UnpackFilter.cs @@ -1,14 +1,21 @@ using SharpCompress.Compressors.Rar.VM; -namespace SharpCompress.Compressors.Rar +namespace SharpCompress.Compressors.Rar.UnpackV1 { internal class UnpackFilter { + public byte Type; + public byte Channels; + internal UnpackFilter() { Program = new VMPreparedProgram(); } + // TODO uint + internal uint uBlockStart { get { return (uint)BlockStart; } set { BlockStart = (int)value; } } + internal uint uBlockLength { get { return (uint)BlockLength; } set { BlockLength = (int)value; } } + internal int BlockStart { get; set; } internal int BlockLength { get; set; } diff --git a/src/SharpCompress/Compressors/Rar/UnpackV1/UnpackInline.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/UnpackInline.cs new file mode 100644 index 00000000..1bb2893d --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/UnpackInline.cs @@ -0,0 +1,31 @@ + +namespace SharpCompress.Compressors.Rar.UnpackV1 +{ + internal partial class Unpack + { + private uint SlotToLength(uint Slot) + { + //uint LBits,Length=2; + int LBits; + uint Length=2; + if (Slot<8) + { + LBits=0; + Length+=Slot; + } + else + { + //LBits=Slot/4-1; + LBits=(int)(Slot/4-1); + Length+=(4 | (Slot & 3)) << LBits; + } + + if (LBits>0) + { + Length+=getbits()>>(16-LBits); + AddBits(LBits); + } + return Length; + } + } +} diff --git a/src/SharpCompress/Compressors/Rar/UnpackUtility.cs b/src/SharpCompress/Compressors/Rar/UnpackV1/UnpackUtility.cs similarity index 96% rename from src/SharpCompress/Compressors/Rar/UnpackUtility.cs rename to src/SharpCompress/Compressors/Rar/UnpackV1/UnpackUtility.cs index 8f9cfada..46261496 100644 --- a/src/SharpCompress/Compressors/Rar/UnpackUtility.cs +++ b/src/SharpCompress/Compressors/Rar/UnpackV1/UnpackUtility.cs @@ -1,9 +1,14 @@ using SharpCompress.Compressors.Rar.VM; -namespace SharpCompress.Compressors.Rar +namespace SharpCompress.Compressors.Rar.UnpackV1 { internal static class UnpackUtility { +//!!! TODO rename methods + internal static uint DecodeNumber(this BitInput input, Decode.Decode dec) { + return (uint)input.decodeNumber(dec); + } + internal static int decodeNumber(this BitInput input, Decode.Decode dec) { int bits; diff --git a/src/SharpCompress/Compressors/Rar/UnpackV2017/BitInput.getbits_cpp.cs b/src/SharpCompress/Compressors/Rar/UnpackV2017/BitInput.getbits_cpp.cs new file mode 100644 index 00000000..ff7b70d7 --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV2017/BitInput.getbits_cpp.cs @@ -0,0 +1,67 @@ +#if !Rar2017_64bit +using nint = System.Int32; +using nuint = System.UInt32; +using size_t = System.UInt32; +#else +using nint = System.Int64; +using nuint = System.UInt64; +using size_t = System.UInt64; +#endif + +namespace SharpCompress.Compressors.Rar.UnpackV2017 +{ + internal partial class BitInput + { + +public BitInput(bool AllocBuffer) +{ + ExternalBuffer=false; + if (AllocBuffer) + { + // getbits32 attempts to read data from InAddr, ... InAddr+3 positions. + // So let's allocate 3 additional bytes for situation, when we need to + // read only 1 byte from the last position of buffer and avoid a crash + // from access to next 3 bytes, which contents we do not need. + size_t BufSize=MAX_SIZE+3; + InBuf=new byte[BufSize]; + + // Ensure that we get predictable results when accessing bytes in area + // not filled with read data. + //memset(InBuf,0,BufSize); + } + else + InBuf=null; +} + + +//BitInput::~BitInput() +//{ +// if (!ExternalBuffer) +// delete[] InBuf; +//} +// + +public +void faddbits(uint Bits) +{ + // Function wrapped version of inline addbits to save code size. + addbits(Bits); +} + +public +uint fgetbits() +{ + // Function wrapped version of inline getbits to save code size. + return getbits(); +} + + private void SetExternalBuffer(byte []Buf) +{ + //if (InBuf!=NULL && !ExternalBuffer) + // delete[] InBuf; + InBuf=Buf; + ExternalBuffer=true; +} + + } +} diff --git a/src/SharpCompress/Compressors/Rar/UnpackV2017/BitInput.getbits_hpp.cs b/src/SharpCompress/Compressors/Rar/UnpackV2017/BitInput.getbits_hpp.cs new file mode 100644 index 00000000..4f16592e --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV2017/BitInput.getbits_hpp.cs @@ -0,0 +1,70 @@ +namespace SharpCompress.Compressors.Rar.UnpackV2017 +{ + +internal partial class BitInput +{ + public const int MAX_SIZE=0x8000; // Size of input buffer. + + public int InAddr; // Curent byte position in the buffer. + public int InBit; // Current bit position in the current byte. + + public bool ExternalBuffer; + + //BitInput(bool AllocBuffer); + //~BitInput(); + + public byte[] InBuf; // Dynamically allocated input buffer. + + public + void InitBitInput() + { + InAddr=InBit=0; + } + + // Move forward by 'Bits' bits. + public void addbits(uint _Bits) + { + var Bits = checked((int)_Bits); + Bits+=InBit; + InAddr+=Bits>>3; + InBit=Bits&7; + } + + // Return 16 bits from current position in the buffer. + // Bit at (InAddr,InBit) has the highest position in returning data. + public uint getbits() + { + uint BitField=(uint)InBuf[InAddr] << 16; + BitField|=(uint)InBuf[InAddr+1] << 8; + BitField|=(uint)InBuf[InAddr+2]; + BitField >>= (8-InBit); + return BitField & 0xffff; + } + + // Return 32 bits from current position in the buffer. + // Bit at (InAddr,InBit) has the highest position in returning data. + public uint getbits32() + { + uint BitField=(uint)InBuf[InAddr] << 24; + BitField|=(uint)InBuf[InAddr+1] << 16; + BitField|=(uint)InBuf[InAddr+2] << 8; + BitField|=(uint)InBuf[InAddr+3]; + BitField <<= InBit; + BitField|=(uint)InBuf[InAddr+4] >> (8-InBit); + return BitField & 0xffffffff; + } + + //void faddbits(uint Bits); + //uint fgetbits(); + + // Check if buffer has enough space for IncPtr bytes. Returns 'true' + // if buffer will be overflown. + private bool Overflow(uint IncPtr) + { + return InAddr+IncPtr>=MAX_SIZE; + } + + //void SetExternalBuffer(byte *Buf); +} + +} diff --git a/src/SharpCompress/Compressors/Rar/UnpackV2017/FragmentedWindow.unpack50frag_cpp.cs b/src/SharpCompress/Compressors/Rar/UnpackV2017/FragmentedWindow.unpack50frag_cpp.cs new file mode 100644 index 00000000..e3614217 --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV2017/FragmentedWindow.unpack50frag_cpp.cs @@ -0,0 +1,154 @@ +#if !Rar2017_64bit +using nint = System.Int32; +using nuint = System.UInt32; +using size_t = System.UInt32; +#else +using nint = System.Int64; +using nuint = System.UInt64; +using size_t = System.UInt64; +#endif +using int64 = System.Int64; + +using System; + +namespace SharpCompress.Compressors.Rar.UnpackV2017 +{ + internal partial class FragmentedWindow + { + +public FragmentedWindow() +{ + //memset(Mem,0,sizeof(Mem)); + //memset(MemSize,0,sizeof(MemSize)); +} + + +//FragmentedWindow::~FragmentedWindow() +//{ +// Reset(); +//} + + private void Reset() +{ + for (uint I=0;I=MinSize) + { + NewMem=new byte[Size]; + if (NewMem!=null) + break; + Size-=Size/32; + } + if (NewMem==null) + //throw std::bad_alloc(); + throw new InvalidOperationException(); + + // Clean the window to generate the same output when unpacking corrupt + // RAR files, which may access to unused areas of sliding dictionary. + // sharpcompress: don't need this, freshly allocated above + //Utility.Memset(NewMem,0,Size); + + Mem[BlockNum]=NewMem; + TotalSize+=Size; + MemSize[BlockNum]=TotalSize; + BlockNum++; + } + if (TotalSize 0) + { + this[UnpPtr]=this[SrcPtr++ & MaxWinMask]; + // We need to have masked UnpPtr after quit from loop, so it must not + // be replaced with '(*this)[UnpPtr++ & MaxWinMask]' + UnpPtr=(UnpPtr+1) & MaxWinMask; + } +} + + +public void CopyData(byte[] Dest, size_t destOffset, size_t WinPos,size_t Size) +{ + for (size_t I=0;I DestUnpSize; } + + public int Char + { + get + { + // TODO: coderb: not sure where the "MAXSIZE-30" comes from, ported from V1 code + if (InAddr > MAX_SIZE - 30) + { + UnpReadBuf(); + } + return InBuf[InAddr++]; + } + } + + public int PpmEscChar { get => PPMEscChar; set => PPMEscChar = value; } + + public static byte[] EnsureCapacity(byte[] array, int length) { + return array.Length < length ? new byte[length] : array; + } + + } +} diff --git a/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.rawint_hpp.cs b/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.rawint_hpp.cs new file mode 100644 index 00000000..3a34f9d0 --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.rawint_hpp.cs @@ -0,0 +1,126 @@ +#if !Rar2017_64bit +using nint = System.Int32; +using nuint = System.UInt32; +using size_t = System.UInt32; +#else +using nint = System.Int64; +using nuint = System.UInt64; +using size_t = System.UInt64; +#endif +using int64 = System.Int64; +using uint32 = System.UInt32; + +using System; +using System.Collections.Generic; +using System.Text; + +namespace SharpCompress.Compressors.Rar.UnpackV2017 +{ + internal partial class Unpack + { +//#define rotls(x,n,xsize) (((x)<<(n)) | ((x)>>(xsize-(n)))) +//#define rotrs(x,n,xsize) (((x)>>(n)) | ((x)<<(xsize-(n)))) +//#define rotl32(x,n) rotls(x,n,32) +//#define rotr32(x,n) rotrs(x,n,32) +// +//inline uint RawGet2(const void *Data) +//{ +// byte *D=(byte *)Data; +// return D[0]+(D[1]<<8); +//} + + private uint32 RawGet4(byte[] D, int offset) +{ + return (uint)(D[offset]+(D[offset+1]<<8)+(D[offset+2]<<16)+(D[offset+3]<<24)); +} + +//inline uint64 RawGet8(const void *Data) +//{ +//#if defined(BIG_ENDIAN) || !defined(ALLOW_MISALIGNED) +// byte *D=(byte *)Data; +// return INT32TO64(RawGet4(D+4),RawGet4(D)); +//#else +// return *(uint64 *)Data; +//#endif +//} +// +// +//inline void RawPut2(uint Field,void *Data) +//{ +// byte *D=(byte *)Data; +// D[0]=(byte)(Field); +// D[1]=(byte)(Field>>8); +//} + + private void RawPut4(uint32 Field,byte[] D, int offset) +{ + D[offset]=(byte)(Field); + D[offset+1]=(byte)(Field>>8); + D[offset+2]=(byte)(Field>>16); + D[offset+3]=(byte)(Field>>24); +} + +//inline void RawPut8(uint64 Field,void *Data) +//{ +//#if defined(BIG_ENDIAN) || !defined(ALLOW_MISALIGNED) +// byte *D=(byte *)Data; +// D[0]=(byte)(Field); +// D[1]=(byte)(Field>>8); +// D[2]=(byte)(Field>>16); +// D[3]=(byte)(Field>>24); +// D[4]=(byte)(Field>>32); +// D[5]=(byte)(Field>>40); +// D[6]=(byte)(Field>>48); +// D[7]=(byte)(Field>>56); +//#else +// *(uint64 *)Data=Field; +//#endif +//} + + +//#if defined(LITTLE_ENDIAN) && defined(ALLOW_MISALIGNED) +//#define USE_MEM_BYTESWAP +//#endif + +// Load 4 big endian bytes from memory and return uint32. +//inline uint32 RawGetBE4(const byte *m) +//{ +//#if defined(USE_MEM_BYTESWAP) && defined(_MSC_VER) +// return _byteswap_ulong(*(uint32 *)m); +//#elif defined(USE_MEM_BYTESWAP) && (__GNUC__ > 3) && (__GNUC_MINOR__ > 2) +// return __builtin_bswap32(*(uint32 *)m); +//#else +// return uint32(m[0]<<24) | uint32(m[1]<<16) | uint32(m[2]<<8) | m[3]; +//#endif +//} + + +// Save integer to memory as big endian. +//inline void RawPutBE4(uint32 i,byte *mem) +//{ +//#if defined(USE_MEM_BYTESWAP) && defined(_MSC_VER) +// *(uint32*)mem = _byteswap_ulong(i); +//#elif defined(USE_MEM_BYTESWAP) && (__GNUC__ > 3) && (__GNUC_MINOR__ > 2) +// *(uint32*)mem = __builtin_bswap32(i); +//#else +// mem[0]=byte(i>>24); +// mem[1]=byte(i>>16); +// mem[2]=byte(i>>8); +// mem[3]=byte(i); +//#endif +//} + + +//inline uint32 ByteSwap32(uint32 i) +//{ +//#ifdef _MSC_VER +// return _byteswap_ulong(i); +//#elif (__GNUC__ > 3) && (__GNUC_MINOR__ > 2) +// return __builtin_bswap32(i); +//#else +// return (rotl32(i,24)&0xFF00FF00)|(rotl32(i,8)&0x00FF00FF); +//#endif +//} + + } +} diff --git a/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack15_cpp.cs b/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack15_cpp.cs new file mode 100644 index 00000000..2b71751e --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack15_cpp.cs @@ -0,0 +1,501 @@ +using static SharpCompress.Compressors.Rar.UnpackV2017.Unpack.Unpack15Local; + +namespace SharpCompress.Compressors.Rar.UnpackV2017 +{ + internal partial class Unpack + { + private const int STARTL1 =2; + + private static readonly uint[] DecL1={0x8000,0xa000,0xc000,0xd000,0xe000,0xea00, + 0xee00,0xf000,0xf200,0xf200,0xffff}; + + private static readonly uint[] PosL1={0,0,0,2,3,5,7,11,16,20,24,32,32}; + + private const int STARTL2 =3; + + private static readonly uint[] DecL2={0xa000,0xc000,0xd000,0xe000,0xea00,0xee00, + 0xf000,0xf200,0xf240,0xffff}; + + private static readonly uint[] PosL2={0,0,0,0,5,7,9,13,18,22,26,34,36}; + + private const int STARTHF0 =4; + + private static readonly uint[] DecHf0={0x8000,0xc000,0xe000,0xf200,0xf200,0xf200, + 0xf200,0xf200,0xffff}; + + private static readonly uint[] PosHf0={0,0,0,0,0,8,16,24,33,33,33,33,33}; + + private const int STARTHF1 =5; + + private static readonly uint[] DecHf1={0x2000,0xc000,0xe000,0xf000,0xf200,0xf200, + 0xf7e0,0xffff}; + + private static readonly uint[] PosHf1={0,0,0,0,0,0,4,44,60,76,80,80,127}; + + private const int STARTHF2 =5; + + private static readonly uint[] DecHf2={0x1000,0x2400,0x8000,0xc000,0xfa00,0xffff, + 0xffff,0xffff}; + + private static readonly uint[] PosHf2={0,0,0,0,0,0,2,7,53,117,233,0,0}; + + private const int STARTHF3 =6; + + private static readonly uint[] DecHf3={0x800,0x2400,0xee00,0xfe80,0xffff,0xffff, + 0xffff}; + + private static readonly uint[] PosHf3={0,0,0,0,0,0,0,2,16,218,251,0,0}; + + private const int STARTHF4 =8; + private static readonly uint[] DecHf4={0xff00,0xffff,0xffff,0xffff,0xffff,0xffff}; + private static readonly uint[] PosHf4={0,0,0,0,0,0,0,0,0,255,0,0,0}; + + private void Unpack15(bool Solid) +{ + UnpInitData(Solid); + UnpInitData15(Solid); + UnpReadBuf(); + if (!Solid) + { + InitHuff(); + UnpPtr=0; + } + else + UnpPtr=WrPtr; + --DestUnpSize; + if (DestUnpSize>=0) + { + GetFlagsBuf(); + FlagsCnt=8; + } + + while (DestUnpSize>=0) + { + UnpPtr&=MaxWinMask; + + if (Inp.InAddr>ReadTop-30 && !UnpReadBuf()) + break; + if (((WrPtr-UnpPtr) & MaxWinMask)<270 && WrPtr!=UnpPtr) + UnpWriteBuf20(); + if (StMode != 0) + { + HuffDecode(); + continue; + } + + if (--FlagsCnt < 0) + { + GetFlagsBuf(); + FlagsCnt=7; + } + + if ((FlagBuf & 0x80) != 0) + { + FlagBuf<<=1; + if (Nlzb > Nhfb) + LongLZ(); + else + HuffDecode(); + } + else + { + FlagBuf<<=1; + if (--FlagsCnt < 0) + { + GetFlagsBuf(); + FlagsCnt=7; + } + if ((FlagBuf & 0x80) != 0) + { + FlagBuf<<=1; + if (Nlzb > Nhfb) + HuffDecode(); + else + LongLZ(); + } + else + { + FlagBuf<<=1; + ShortLZ(); + } + } + } + UnpWriteBuf20(); +} + + +//#define GetShortLen1(pos) ((pos)==1 ? Buf60+3:ShortLen1[pos]) + private uint GetShortLen1(uint pos) { return ((pos)==1 ? (uint)(Buf60+3):ShortLen1[pos]); } +//#define GetShortLen2(pos) ((pos)==3 ? Buf60+3:ShortLen2[pos]) + private uint GetShortLen2(uint pos) { return ((pos)==3 ? (uint)(Buf60+3):ShortLen2[pos]); } + +internal static class Unpack15Local { + public static readonly uint[] ShortLen1={1,3,4,4,5,6,7,8,8,4,4,5,6,6,4,0}; + public static readonly uint[] ShortXor1={0,0xa0,0xd0,0xe0,0xf0,0xf8,0xfc,0xfe, + 0xff,0xc0,0x80,0x90,0x98,0x9c,0xb0}; + public static readonly uint[] ShortLen2={2,3,3,3,4,4,5,6,6,4,4,5,6,6,4,0}; + public static readonly uint[] ShortXor2={0,0x40,0x60,0xa0,0xd0,0xe0,0xf0,0xf8, + 0xfc,0xc0,0x80,0x90,0x98,0x9c,0xb0}; +} + + private void ShortLZ() +{ + uint Length,SaveLength; + uint LastDistance; + uint Distance; + int DistancePlace; + NumHuf=0; + + uint BitField=Inp.fgetbits(); + if (LCount==2) + { + Inp.faddbits(1); + if (BitField >= 0x8000) + { + CopyString15((uint)LastDist,LastLength); + return; + } + BitField <<= 1; + LCount=0; + } + + BitField>>=8; + +// not thread safe, replaced by GetShortLen1 and GetShortLen2 macro +// ShortLen1[1]=ShortLen2[3]=Buf60+3; + + if (AvrLn1<37) + { + for (Length=0;;Length++) + if (((BitField^ShortXor1[Length]) & (~(0xff>>(int)GetShortLen1(Length))))==0) + break; + Inp.faddbits(GetShortLen1(Length)); + } + else + { + for (Length=0;;Length++) + if (((BitField^ShortXor2[Length]) & (~(0xff>>(int)GetShortLen2(Length))))==0) + break; + Inp.faddbits(GetShortLen2(Length)); + } + + if (Length >= 9) + { + if (Length == 9) + { + LCount++; + CopyString15((uint)LastDist,LastLength); + return; + } + if (Length == 14) + { + LCount=0; + Length=DecodeNum(Inp.fgetbits(),STARTL2,DecL2,PosL2)+5; + Distance=(Inp.fgetbits()>>1) | 0x8000; + Inp.faddbits(15); + LastLength=Length; + LastDist=Distance; + CopyString15(Distance,Length); + return; + } + + LCount=0; + SaveLength=Length; + Distance=OldDist[(OldDistPtr-(Length-9)) & 3]; + Length=DecodeNum(Inp.fgetbits(),STARTL1,DecL1,PosL1)+2; + if (Length==0x101 && SaveLength==10) + { + Buf60 ^= 1; + return; + } + if (Distance > 256) + Length++; + if (Distance >= MaxDist3) + Length++; + + OldDist[OldDistPtr++]=Distance; + OldDistPtr = OldDistPtr & 3; + LastLength=Length; + LastDist=Distance; + CopyString15(Distance,Length); + return; + } + + LCount=0; + AvrLn1 += Length; + AvrLn1 -= AvrLn1 >> 4; + + DistancePlace=(int)(DecodeNum(Inp.fgetbits(),STARTHF2,DecHf2,PosHf2) & 0xff); + Distance=ChSetA[DistancePlace]; + if (--DistancePlace != -1) + { + LastDistance=ChSetA[DistancePlace]; + ChSetA[DistancePlace+1]=(ushort)LastDistance; + ChSetA[DistancePlace]=(ushort)Distance; + } + Length+=2; + OldDist[OldDistPtr++] = ++Distance; + OldDistPtr = OldDistPtr & 3; + LastLength=Length; + LastDist=Distance; + CopyString15(Distance,Length); +} + + private void LongLZ() +{ + uint Length; + uint Distance; + uint DistancePlace,NewDistancePlace; + uint OldAvr2,OldAvr3; + + NumHuf=0; + Nlzb+=16; + if (Nlzb > 0xff) + { + Nlzb=0x90; + Nhfb >>= 1; + } + OldAvr2=AvrLn2; + + uint BitField=Inp.fgetbits(); + if (AvrLn2 >= 122) + Length=DecodeNum(BitField,STARTL2,DecL2,PosL2); + else + if (AvrLn2 >= 64) + Length=DecodeNum(BitField,STARTL1,DecL1,PosL1); + else + if (BitField < 0x100) + { + Length=BitField; + Inp.faddbits(16); + } + else + { + for (Length=0;((BitField<<(int)Length)&0x8000)==0;Length++) + ; + Inp.faddbits(Length+1); + } + + AvrLn2 += Length; + AvrLn2 -= AvrLn2 >> 5; + + BitField=Inp.fgetbits(); + if (AvrPlcB > 0x28ff) + DistancePlace=DecodeNum(BitField,STARTHF2,DecHf2,PosHf2); + else + if (AvrPlcB > 0x6ff) + DistancePlace=DecodeNum(BitField,STARTHF1,DecHf1,PosHf1); + else + DistancePlace=DecodeNum(BitField,STARTHF0,DecHf0,PosHf0); + + AvrPlcB += DistancePlace; + AvrPlcB -= AvrPlcB >> 8; + while (true) + { + Distance = ChSetB[DistancePlace & 0xff]; + NewDistancePlace = NToPlB[Distance++ & 0xff]++; + if ((Distance & 0xff) != 0) + CorrHuff(ChSetB,NToPlB); + else + break; + } + + ChSetB[DistancePlace & 0xff]=ChSetB[NewDistancePlace]; + ChSetB[NewDistancePlace]=(ushort)Distance; + + Distance=((Distance & 0xff00) | (Inp.fgetbits() >> 8)) >> 1; + Inp.faddbits(7); + + OldAvr3=AvrLn3; + if (Length!=1 && Length!=4) + if (Length==0 && Distance <= MaxDist3) + { + AvrLn3++; + AvrLn3 -= AvrLn3 >> 8; + } + else + if (AvrLn3 > 0) + AvrLn3--; + Length+=3; + if (Distance >= MaxDist3) + Length++; + if (Distance <= 256) + Length+=8; + if (OldAvr3 > 0xb0 || AvrPlc >= 0x2a00 && OldAvr2 < 0x40) + MaxDist3=0x7f00; + else + MaxDist3=0x2001; + OldDist[OldDistPtr++]=Distance; + OldDistPtr = OldDistPtr & 3; + LastLength=Length; + LastDist=Distance; + CopyString15(Distance,Length); +} + + private void HuffDecode() +{ + uint CurByte,NewBytePlace; + uint Length; + uint Distance; + int BytePlace; + + uint BitField=Inp.fgetbits(); + + if (AvrPlc > 0x75ff) + BytePlace=(int)DecodeNum(BitField,STARTHF4,DecHf4,PosHf4); + else + if (AvrPlc > 0x5dff) + BytePlace=(int)DecodeNum(BitField,STARTHF3,DecHf3,PosHf3); + else + if (AvrPlc > 0x35ff) + BytePlace=(int)DecodeNum(BitField,STARTHF2,DecHf2,PosHf2); + else + if (AvrPlc > 0x0dff) + BytePlace=(int)DecodeNum(BitField,STARTHF1,DecHf1,PosHf1); + else + BytePlace=(int)DecodeNum(BitField,STARTHF0,DecHf0,PosHf0); + BytePlace&=0xff; + if (StMode != 0) + { + if (BytePlace==0 && BitField > 0xfff) + BytePlace=0x100; + if (--BytePlace==-1) + { + BitField=Inp.fgetbits(); + Inp.faddbits(1); + if ((BitField & 0x8000) != 0) + { + NumHuf=StMode=0; + return; + } + else + { + Length = (BitField & 0x4000) != 0 ? 4U : 3; + Inp.faddbits(1); + Distance=DecodeNum(Inp.fgetbits(),STARTHF2,DecHf2,PosHf2); + Distance = (Distance << 5) | (Inp.fgetbits() >> 11); + Inp.faddbits(5); + CopyString15(Distance,Length); + return; + } + } + } + else + if (NumHuf++ >= 16 && FlagsCnt==0) + StMode=1; + AvrPlc += (uint)BytePlace; + AvrPlc -= AvrPlc >> 8; + Nhfb+=16; + if (Nhfb > 0xff) + { + Nhfb=0x90; + Nlzb >>= 1; + } + + Window[UnpPtr++]=(byte)(ChSet[BytePlace]>>8); + --DestUnpSize; + + while (true) + { + CurByte=ChSet[BytePlace]; + NewBytePlace=NToPl[CurByte++ & 0xff]++; + if ((CurByte & 0xff) > 0xa1) + CorrHuff(ChSet,NToPl); + else + break; + } + + ChSet[BytePlace]=ChSet[NewBytePlace]; + ChSet[NewBytePlace]=(ushort)CurByte; +} + + private void GetFlagsBuf() +{ + uint Flags,NewFlagsPlace; + uint FlagsPlace=DecodeNum(Inp.fgetbits(),STARTHF2,DecHf2,PosHf2); + + // Our Huffman table stores 257 items and needs all them in other parts + // of code such as when StMode is on, so the first item is control item. + // While normally we do not use the last item to code the flags byte here, + // we need to check for value 256 when unpacking in case we unpack + // a corrupt archive. + if (FlagsPlace>=ChSetC.Length) + return; + + while (true) + { + Flags=ChSetC[FlagsPlace]; + FlagBuf=Flags>>8; + NewFlagsPlace=NToPlC[Flags++ & 0xff]++; + if ((Flags & 0xff) != 0) + break; + CorrHuff(ChSetC,NToPlC); + } + + ChSetC[FlagsPlace]=ChSetC[NewFlagsPlace]; + ChSetC[NewFlagsPlace]=(ushort)Flags; +} + + private void UnpInitData15(bool Solid) +{ + if (!Solid) + { + AvrPlcB=AvrLn1=AvrLn2=AvrLn3=0; + NumHuf=Buf60=0; + AvrPlc=0x3500; + MaxDist3=0x2001; + Nhfb=Nlzb=0x80; + } + FlagsCnt=0; + FlagBuf=0; + StMode=0; + LCount=0; + ReadTop=0; +} + + private void InitHuff() +{ + for (uint I=0;I<256;I++) + { + ChSet[I]=ChSetB[I]=(ushort)(I<<8); + ChSetA[I]=(ushort)I; + ChSetC[I]=(ushort)(((~I+1) & 0xff)<<8); + } + Utility.Memset(NToPl,0,NToPl.Length); + Utility.Memset(NToPlB,0,NToPlB.Length); + Utility.Memset(NToPlC,0,NToPlC.Length); + CorrHuff(ChSetB,NToPlB); +} + + private void CorrHuff(ushort[] CharSet,byte[] NumToPlace) +{ + int I,J; + for (I=7;I>=0;I--) + for (J=0;J<32;J++) + CharSet[J]=(ushort)((CharSet[J] & ~0xff) | I); + Utility.Memset(NumToPlace,0,NToPl.Length); + for (I=6;I>=0;I--) + NumToPlace[I]=(byte)((7-I)*32); +} + + private void CopyString15(uint Distance,uint Length) +{ + DestUnpSize-=Length; + while (Length-- != 0) + { + Window[UnpPtr]=Window[(UnpPtr-Distance) & MaxWinMask]; + UnpPtr=(UnpPtr+1) & MaxWinMask; + } +} + + private uint DecodeNum(uint Num,uint StartPos,uint[] DecTab,uint[] PosTab) +{ + int I; + for (Num&=0xfff0,I=0;DecTab[I]<=Num;I++) + StartPos++; + Inp.faddbits(StartPos); + return(((Num-(I != 0 ? DecTab[I-1]:0))>>(int)(16-StartPos))+PosTab[StartPos]); +} + + } +} diff --git a/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack20_cpp.cs b/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack20_cpp.cs new file mode 100644 index 00000000..1a78f9af --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack20_cpp.cs @@ -0,0 +1,399 @@ +#if !Rar2017_64bit +using nint = System.Int32; +using nuint = System.UInt32; +using size_t = System.UInt32; +#else +using nint = System.Int64; +using nuint = System.UInt64; +using size_t = System.UInt64; +#endif +using int64 = System.Int64; + +using System; +using static SharpCompress.Compressors.Rar.UnpackV2017.PackDef; +using static SharpCompress.Compressors.Rar.UnpackV2017.Unpack.Unpack20Local; + +namespace SharpCompress.Compressors.Rar.UnpackV2017 +{ + internal partial class Unpack + { + private void CopyString20(uint Length,uint Distance) +{ + LastDist=OldDist[OldDistPtr++ & 3]=Distance; + LastLength=Length; + DestUnpSize-=Length; + CopyString(Length,Distance); +} + + +internal static class Unpack20Local { + public static readonly byte[] LDecode={0,1,2,3,4,5,6,7,8,10,12,14,16,20,24,28,32,40,48,56,64,80,96,112,128,160,192,224}; + public static readonly byte[] LBits= {0,0,0,0,0,0,0,0,1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5}; + public static readonly uint[] DDecode={0,1,2,3,4,6,8,12,16,24,32,48,64,96,128,192,256,384,512,768,1024,1536,2048,3072,4096,6144,8192,12288,16384,24576,32768U,49152U,65536,98304,131072,196608,262144,327680,393216,458752,524288,589824,655360,720896,786432,851968,917504,983040}; + public static readonly byte[] DBits= {0,0,0,0,1,1,2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13, 14, 14, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16}; + public static readonly byte[] SDDecode={0,4,8,16,32,64,128,192}; + public static readonly byte[] SDBits= {2,2,3, 4, 5, 6, 6, 6}; +} + + private void Unpack20(bool Solid) +{ + uint Bits; + + if (Suspended) + UnpPtr=WrPtr; + else + { + UnpInitData(Solid); + if (!UnpReadBuf()) + return; + if ((!Solid || !TablesRead2) && !ReadTables20()) + return; + --DestUnpSize; + } + + while (DestUnpSize>=0) + { + UnpPtr&=MaxWinMask; + + if (Inp.InAddr>ReadTop-30) + if (!UnpReadBuf()) + break; + if (((WrPtr-UnpPtr) & MaxWinMask)<270 && WrPtr!=UnpPtr) + { + UnpWriteBuf20(); + if (Suspended) + return; + } + if (UnpAudioBlock) + { + uint AudioNumber=DecodeNumber(Inp,MD[UnpCurChannel]); + + if (AudioNumber==256) + { + if (!ReadTables20()) + break; + continue; + } + Window[UnpPtr++]=DecodeAudio((int)AudioNumber); + if (++UnpCurChannel==UnpChannels) + UnpCurChannel=0; + --DestUnpSize; + continue; + } + + uint Number=DecodeNumber(Inp,BlockTables.LD); + if (Number<256) + { + Window[UnpPtr++]=(byte)Number; + --DestUnpSize; + continue; + } + if (Number>269) + { + uint Length=(uint)(LDecode[Number-=270]+3); + if ((Bits=LBits[Number])>0) + { + Length+=Inp.getbits()>>(int)(16-Bits); + Inp.addbits(Bits); + } + + uint DistNumber=DecodeNumber(Inp,BlockTables.DD); + uint Distance=DDecode[DistNumber]+1; + if ((Bits=DBits[DistNumber])>0) + { + Distance+=Inp.getbits()>>(int)(16-Bits); + Inp.addbits(Bits); + } + + if (Distance>=0x2000) + { + Length++; + if (Distance>=0x40000L) + Length++; + } + + CopyString20(Length,Distance); + continue; + } + if (Number==269) + { + if (!ReadTables20()) + break; + continue; + } + if (Number==256) + { + CopyString20(LastLength,LastDist); + continue; + } + if (Number<261) + { + uint Distance=OldDist[(OldDistPtr-(Number-256)) & 3]; + uint LengthNumber=DecodeNumber(Inp,BlockTables.RD); + uint Length=(uint)(LDecode[LengthNumber]+2); + if ((Bits=LBits[LengthNumber])>0) + { + Length+=Inp.getbits()>>(int)(16-Bits); + Inp.addbits(Bits); + } + if (Distance>=0x101) + { + Length++; + if (Distance>=0x2000) + { + Length++; + if (Distance>=0x40000) + Length++; + } + } + CopyString20(Length,Distance); + continue; + } + if (Number<270) + { + uint Distance=(uint)(SDDecode[Number-=261]+1); + if ((Bits=SDBits[Number])>0) + { + Distance+=Inp.getbits()>>(int)(16-Bits); + Inp.addbits(Bits); + } + CopyString20(2,Distance); + continue; + } + } + ReadLastTables(); + UnpWriteBuf20(); +} + + private void UnpWriteBuf20() +{ + if (UnpPtr!=WrPtr) + UnpSomeRead=true; + if (UnpPtrReadTop-25) + if (!UnpReadBuf()) + return false; + uint BitField=Inp.getbits(); + UnpAudioBlock=(BitField & 0x8000)!=0; + + if ((BitField & 0x4000) != 0) + Utility.Memset(UnpOldTable20,0,UnpOldTable20.Length); + Inp.addbits(2); + + uint TableSize; + if (UnpAudioBlock) + { + UnpChannels=((BitField>>12) & 3)+1; + if (UnpCurChannel>=UnpChannels) + UnpCurChannel=0; + Inp.addbits(2); + TableSize=MC20*UnpChannels; + } + else + TableSize=NC20+DC20+RC20; + + for (uint I=0;I> 12); + Inp.addbits(4); + } + MakeDecodeTables(BitLength,0,BlockTables.BD,BC20); + for (uint I=0;IReadTop-5) + if (!UnpReadBuf()) + return false; + uint Number=DecodeNumber(Inp,BlockTables.BD); + if (Number<16) + { + Table[I]=(byte)((Number+UnpOldTable20[I]) & 0xf); + I++; + } + else + if (Number==16) + { + uint N=(Inp.getbits() >> 14)+3; + Inp.addbits(2); + if (I==0) + return false; // We cannot have "repeat previous" code at the first position. + else + while (N-- > 0 && I> 13)+3; + Inp.addbits(3); + } + else + { + N=(Inp.getbits() >> 9)+11; + Inp.addbits(7); + } + while (N-- > 0 && IReadTop) + return true; + if (UnpAudioBlock) + for (uint I=0;I=Inp.InAddr+5) + if (UnpAudioBlock) + { + if (DecodeNumber(Inp,MD[UnpCurChannel])==256) + ReadTables20(); + } + else + if (DecodeNumber(Inp,BlockTables.LD)==269) + ReadTables20(); +} + + private void UnpInitData20(bool Solid) +{ + if (!Solid) + { + TablesRead2=false; + UnpAudioBlock=false; + UnpChannelDelta=0; + UnpCurChannel=0; + UnpChannels=1; + + //memset(AudV,0,sizeof(AudV)); + AudV = new AudioVariables[4]; + Utility.Memset(UnpOldTable20, 0, UnpOldTable20.Length); + //memset(MD,0,sizeof(MD)); + MD = new DecodeTable[4]; + } +} + + private byte DecodeAudio(int Delta) +{ + AudioVariables V=AudV[UnpCurChannel]; + V.ByteCount++; + V.D4=V.D3; + V.D3=V.D2; + V.D2=V.LastDelta-V.D1; + V.D1=V.LastDelta; + int PCh=8*V.LastChar+V.K1*V.D1+V.K2*V.D2+V.K3*V.D3+V.K4*V.D4+V.K5*UnpChannelDelta; + PCh=(PCh>>3) & 0xFF; + + uint Ch=(uint)(PCh-Delta); + + int D=(sbyte)Delta; + // Left shift of negative value is undefined behavior in C++, + // so we cast it to unsigned to follow the standard. + D=(int)((uint)D<<3); + + V.Dif[0]+=(uint)Math.Abs(D); + V.Dif[1]+=(uint)Math.Abs(D-V.D1); + V.Dif[2]+=(uint)Math.Abs(D+V.D1); + V.Dif[3]+=(uint)Math.Abs(D-V.D2); + V.Dif[4]+=(uint)Math.Abs(D+V.D2); + V.Dif[5]+=(uint)Math.Abs(D-V.D3); + V.Dif[6]+=(uint)Math.Abs(D+V.D3); + V.Dif[7]+=(uint)Math.Abs(D-V.D4); + V.Dif[8]+=(uint)Math.Abs(D+V.D4); + V.Dif[9]+=(uint)Math.Abs(D-UnpChannelDelta); + V.Dif[10]+=(uint)Math.Abs(D+UnpChannelDelta); + + UnpChannelDelta=V.LastDelta=(sbyte)(Ch-V.LastChar); + V.LastChar=(int)Ch; + + if ((V.ByteCount & 0x1F)==0) + { + uint MinDif=V.Dif[0],NumMinDif=0; + V.Dif[0]=0; + for (uint I=1;I=-16) + V.K1--; + break; + case 2: + if (V.K1<16) + V.K1++; + break; + case 3: + if (V.K2>=-16) + V.K2--; + break; + case 4: + if (V.K2<16) + V.K2++; + break; + case 5: + if (V.K3>=-16) + V.K3--; + break; + case 6: + if (V.K3<16) + V.K3++; + break; + case 7: + if (V.K4>=-16) + V.K4--; + break; + case 8: + if (V.K4<16) + V.K4++; + break; + case 9: + if (V.K5>=-16) + V.K5--; + break; + case 10: + if (V.K5<16) + V.K5++; + break; + } + } + return (byte)Ch; +} + + } +} diff --git a/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack30_cpp.cs b/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack30_cpp.cs new file mode 100644 index 00000000..7e77c7d4 --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack30_cpp.cs @@ -0,0 +1,800 @@ +#if !Rar2017_64bit +using nint = System.Int32; +using nuint = System.UInt32; +using size_t = System.UInt32; +#else +using nint = System.Int64; +using nuint = System.UInt64; +using size_t = System.UInt64; +#endif +using int64 = System.Int64; + +using System; +using static SharpCompress.Compressors.Rar.UnpackV2017.PackDef; +using static SharpCompress.Compressors.Rar.UnpackV2017.UnpackGlobal; +//using static SharpCompress.Compressors.Rar.UnpackV2017.Unpack.Unpack30Local; +/* +namespace SharpCompress.Compressors.Rar.UnpackV2017 +{ + internal partial class Unpack + { + +#if !RarV2017_RAR5ONLY +// We use it instead of direct PPM.DecodeChar call to be sure that +// we reset PPM structures in case of corrupt data. It is important, +// because these structures can be invalid after PPM.DecodeChar returned -1. +int SafePPMDecodeChar() +{ + int Ch=PPM.DecodeChar(); + if (Ch==-1) // Corrupt PPM data found. + { + PPM.CleanUp(); // Reset possibly corrupt PPM data structures. + UnpBlockType=BLOCK_LZ; // Set faster and more fail proof LZ mode. + } + return(Ch); +} + +internal static class Unpack30Local { + public static readonly byte[] LDecode={0,1,2,3,4,5,6,7,8,10,12,14,16,20,24,28,32,40,48,56,64,80,96,112,128,160,192,224}; + public static readonly byte[] LBits= {0,0,0,0,0,0,0,0,1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5}; + public static readonly int[] DDecode = new int[DC]; + public static readonly byte[] DBits = new byte[DC]; + public static readonly int[] DBitLengthCounts= {4,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,14,0,12}; + public static readonly byte[] SDDecode={0,4,8,16,32,64,128,192}; + public static readonly byte[] SDBits= {2,2,3, 4, 5, 6, 6, 6}; +} +void Unpack29(bool Solid) +{ + uint Bits; + + if (DDecode[1]==0) + { + int Dist=0,BitLength=0,Slot=0; + for (int I=0;IReadBorder) + { + if (!UnpReadBuf30()) + break; + } + if (((WrPtr-UnpPtr) & MaxWinMask)<260 && WrPtr!=UnpPtr) + { + UnpWriteBuf30(); + if (WrittenFileSize>DestUnpSize) + return; + if (Suspended) + { + FileExtracted=false; + return; + } + } + if (UnpBlockType==BLOCK_PPM) + { + // Here speed is critical, so we do not use SafePPMDecodeChar, + // because sometimes even the inline function can introduce + // some additional penalty. + int Ch=PPM.DecodeChar(); + if (Ch==-1) // Corrupt PPM data found. + { + PPM.CleanUp(); // Reset possibly corrupt PPM data structures. + UnpBlockType=BLOCK_LZ; // Set faster and more fail proof LZ mode. + break; + } + if (Ch==PPMEscChar) + { + int NextCh=SafePPMDecodeChar(); + if (NextCh==0) // End of PPM encoding. + { + if (!ReadTables30()) + break; + continue; + } + if (NextCh==-1) // Corrupt PPM data found. + break; + if (NextCh==2) // End of file in PPM mode. + break; + if (NextCh==3) // Read VM code. + { + if (!ReadVMCodePPM()) + break; + continue; + } + if (NextCh==4) // LZ inside of PPM. + { + uint Distance=0,Length; + bool Failed=false; + for (int I=0;I<4 && !Failed;I++) + { + int _Ch=SafePPMDecodeChar(); + if (_Ch==-1) + Failed=true; + else + if (I==3) + Length=(byte)_Ch; + else + Distance=(Distance<<8)+(byte)_Ch; + } + if (Failed) + break; + + CopyString(Length+32,Distance+2); + continue; + } + if (NextCh==5) // One byte distance match (RLE) inside of PPM. + { + int Length=SafePPMDecodeChar(); + if (Length==-1) + break; + CopyString((uint)(Length+4),1); + continue; + } + // If we are here, NextCh must be 1, what means that current byte + // is equal to our 'escape' byte, so we just store it to Window. + } + Window[UnpPtr++]=(byte)Ch; + continue; + } + + uint Number=DecodeNumber(Inp,BlockTables.LD); + if (Number<256) + { + Window[UnpPtr++]=(byte)Number; + continue; + } + if (Number>=271) + { + uint Length=(uint)(LDecode[Number-=271]+3); + if ((Bits=LBits[Number])>0) + { + Length+=Inp.getbits()>>(int)(16-Bits); + Inp.addbits(Bits); + } + + uint DistNumber=DecodeNumber(Inp,BlockTables.DD); + uint Distance=(uint)(DDecode[DistNumber]+1); + if ((Bits=DBits[DistNumber])>0) + { + if (DistNumber>9) + { + if (Bits>4) + { + Distance+=((Inp.getbits()>>(int)(20-Bits))<<4); + Inp.addbits(Bits-4); + } + if (LowDistRepCount>0) + { + LowDistRepCount--; + Distance+=(uint)PrevLowDist; + } + else + { + uint LowDist=DecodeNumber(Inp,BlockTables.LDD); + if (LowDist==16) + { + LowDistRepCount=(int)(LOW_DIST_REP_COUNT-1); + Distance+=(uint)PrevLowDist; + } + else + { + Distance+=LowDist; + PrevLowDist=(int)LowDist; + } + } + } + else + { + Distance+=Inp.getbits()>>(int)(16-Bits); + Inp.addbits(Bits); + } + } + + if (Distance>=0x2000) + { + Length++; + if (Distance>=0x40000) + Length++; + } + + InsertOldDist(Distance); + LastLength=Length; + CopyString(Length,Distance); + continue; + } + if (Number==256) + { + if (!ReadEndOfBlock()) + break; + continue; + } + if (Number==257) + { + if (!ReadVMCode()) + break; + continue; + } + if (Number==258) + { + if (LastLength!=0) + CopyString(LastLength,OldDist[0]); + continue; + } + if (Number<263) + { + uint DistNum=Number-259; + uint Distance=OldDist[DistNum]; + for (uint I=DistNum;I>0;I--) + OldDist[I]=OldDist[I-1]; + OldDist[0]=Distance; + + uint LengthNumber=DecodeNumber(Inp,BlockTables.RD); + int Length=LDecode[LengthNumber]+2; + if ((Bits=LBits[LengthNumber])>0) + { + Length+=(int)(Inp.getbits()>>(int)(16-Bits)); + Inp.addbits(Bits); + } + LastLength=(uint)Length; + CopyString((uint)Length,Distance); + continue; + } + if (Number<272) + { + uint Distance=(uint)(SDDecode[Number-=263]+1); + if ((Bits=SDBits[Number])>0) + { + Distance+=Inp.getbits()>>(int)(16-Bits); + Inp.addbits(Bits); + } + InsertOldDist(Distance); + LastLength=2; + CopyString(2,Distance); + continue; + } + } + UnpWriteBuf30(); +} + + +// Return 'false' to quit unpacking the current file or 'true' to continue. +bool ReadEndOfBlock() +{ + uint BitField=Inp.getbits(); + bool NewTable,NewFile=false; + + // "1" - no new file, new table just here. + // "00" - new file, no new table. + // "01" - new file, new table (in beginning of next file). + + if ((BitField & 0x8000)!=0) + { + NewTable=true; + Inp.addbits(1); + } + else + { + NewFile=true; + NewTable=(BitField & 0x4000)!=0; + Inp.addbits(2); + } + TablesRead3=!NewTable; + + // Quit immediately if "new file" flag is set. If "new table" flag + // is present, we'll read the table in beginning of next file + // based on 'TablesRead3' 'false' value. + if (NewFile) + return false; + return ReadTables30(); // Quit only if we failed to read tables. +} + + +bool ReadVMCode() +{ + // Entire VM code is guaranteed to fully present in block defined + // by current Huffman table. Compressor checks that VM code does not cross + // Huffman block boundaries. + uint FirstByte=Inp.getbits()>>8; + Inp.addbits(8); + uint Length=(FirstByte & 7)+1; + if (Length==7) + { + Length=(Inp.getbits()>>8)+7; + Inp.addbits(8); + } + else + if (Length==8) + { + Length=Inp.getbits(); + Inp.addbits(16); + } + if (Length==0) + return false; + Array VMCode(Length); + for (uint I=0;I=ReadTop-1 && !UnpReadBuf30() && I>8; + Inp.addbits(8); + } + return AddVMCode(FirstByte,&VMCode[0],Length); +} + + +bool ReadVMCodePPM() +{ + uint FirstByte=(uint)SafePPMDecodeChar(); + if ((int)FirstByte==-1) + return false; + uint Length=(FirstByte & 7)+1; + if (Length==7) + { + int B1=SafePPMDecodeChar(); + if (B1==-1) + return false; + Length=B1+7; + } + else + if (Length==8) + { + int B1=SafePPMDecodeChar(); + if (B1==-1) + return false; + int B2=SafePPMDecodeChar(); + if (B2==-1) + return false; + Length=B1*256+B2; + } + if (Length==0) + return false; + Array VMCode(Length); + for (uint I=0;IFilters30.Count || FiltPos>OldFilterLengths.Count) + return false; + LastFilter=(int)FiltPos; + bool NewFilter=(FiltPos==Filters30.Count); + + UnpackFilter30 StackFilter=new UnpackFilter30(); // New filter for PrgStack. + + UnpackFilter30 Filter; + if (NewFilter) // New filter code, never used before since VM reset. + { + if (FiltPos>MAX3_UNPACK_FILTERS) + { + // Too many different filters, corrupt archive. + //delete StackFilter; + return false; + } + + Filters30.Add(1); + Filters30[Filters30.Count-1]=Filter=new UnpackFilter30(); + StackFilter.ParentFilter=(uint)(Filters30.Count-1); + + // Reserve one item to store the data block length of our new filter + // entry. We'll set it to real block length below, after reading it. + // But we need to initialize it now, because when processing corrupt + // data, we can access this item even before we set it to real value. + OldFilterLengths.Add(0); + } + else // Filter was used in the past. + { + Filter=Filters30[(int)FiltPos]; + StackFilter.ParentFilter=FiltPos; + } + + int EmptyCount=0; + for (int I=0;I0) + PrgStack[I]=null; + } + if (EmptyCount==0) + { + if (PrgStack.Count>MAX3_UNPACK_FILTERS) + { + //delete StackFilter; + return false; + } + PrgStack.Add(1); + EmptyCount=1; + } + size_t StackPos=(uint)(this.PrgStack.Count-EmptyCount); + PrgStack[(int)StackPos]=StackFilter; + + uint BlockStart=RarVM.ReadData(VMCodeInp); + if ((FirstByte & 0x40)!=0) + BlockStart+=258; + StackFilter.BlockStart=(uint)((BlockStart+UnpPtr)&MaxWinMask); + if ((FirstByte & 0x20)!=0) + { + StackFilter.BlockLength=RarVM.ReadData(VMCodeInp); + + // Store the last data block length for current filter. + OldFilterLengths[(int)FiltPos]=(int)StackFilter.BlockLength; + } + else + { + // Set the data block size to same value as the previous block size + // for same filter. It is possible for corrupt data to access a new + // and not filled yet item of OldFilterLengths array here. This is why + // we set new OldFilterLengths items to zero above. + StackFilter.BlockLength=FiltPos>9; + VMCodeInp.faddbits(7); + for (int I=0;I<7;I++) + if ((InitMask & (1<=0x10000 || VMCodeSize==0) + return false; + Array VMCode(VMCodeSize); + for (uint I=0;I>8; + VMCodeInp.faddbits(8); + } + VM.Prepare(&VMCode[0],VMCodeSize,&Filter->Prg); + } + StackFilter.Prg.Type=Filter.Prg.Type; + + return true; +} + + +bool UnpReadBuf30() +{ + int DataSize=ReadTop-Inp.InAddr; // Data left to process. + if (DataSize<0) + return false; + if (Inp.InAddr>BitInput.MAX_SIZE/2) + { + // If we already processed more than half of buffer, let's move + // remaining data into beginning to free more space for new data + // and ensure that calling function does not cross the buffer border + // even if we did not read anything here. Also it ensures that read size + // is not less than CRYPT_BLOCK_SIZE, so we can align it without risk + // to make it zero. + if (DataSize>0) + //x memmove(Inp.InBuf,Inp.InBuf+Inp.InAddr,DataSize); + Array.Copy(Inp.InBuf,Inp.InAddr,Inp.InBuf,0,DataSize); + Inp.InAddr=0; + ReadTop=DataSize; + } + else + DataSize=ReadTop; + int ReadCode=UnpIO_UnpRead(Inp.InBuf,DataSize,BitInput.MAX_SIZE-DataSize); + if (ReadCode>0) + ReadTop+=ReadCode; + ReadBorder=ReadTop-30; + return ReadCode!=-1; +} + + +void UnpWriteBuf30() +{ + uint WrittenBorder=(uint)WrPtr; + uint WriteSize=(uint)((UnpPtr-WrittenBorder)&MaxWinMask); + for (int I=0;IParentFilter]->Prg; + VM_PreparedProgram *Prg=&flt->Prg; + + ExecuteCode(Prg); + + byte[] FilteredData=Prg.FilteredData; + uint FilteredDataSize=Prg.FilteredDataSize; + + delete PrgStack[I]; + PrgStack[I]=null; + while (I+1Prg; + VM_PreparedProgram *NextPrg=&NextFilter->Prg; + + ExecuteCode(NextPrg); + + FilteredData=NextPrg.FilteredData; + FilteredDataSize=NextPrg.FilteredDataSize; + I++; + delete PrgStack[I]; + PrgStack[I]=null; + } + UnpIO_UnpWrite(FilteredData,0,FilteredDataSize); + UnpSomeRead=true; + WrittenFileSize+=FilteredDataSize; + WrittenBorder=BlockEnd; + WriteSize=(uint)((UnpPtr-WrittenBorder)&MaxWinMask); + } + else + { + // Current filter intersects the window write border, so we adjust + // the window border to process this filter next time, not now. + for (size_t J=I;JInitR[6]=(uint)WrittenFileSize; + VM.Execute(Prg); +} + + +bool ReadTables30() +{ + byte[] BitLength = new byte[BC]; + byte[] Table = new byte[HUFF_TABLE_SIZE30]; + if (Inp.InAddr>ReadTop-25) + if (!UnpReadBuf30()) + return(false); + Inp.faddbits((uint)((8-Inp.InBit)&7)); + uint BitField=Inp.fgetbits(); + if ((BitField & 0x8000) != 0) + { + UnpBlockType=BLOCK_PPM; + return(PPM.DecodeInit(this,PPMEscChar)); + } + UnpBlockType=BLOCK_LZ; + + PrevLowDist=0; + LowDistRepCount=0; + + if ((BitField & 0x4000) == 0) + Utility.Memset(UnpOldTable,0,UnpOldTable.Length); + Inp.faddbits(2); + + for (uint I=0;I> 12); + Inp.faddbits(4); + if (Length==15) + { + uint ZeroCount=(byte)(Inp.fgetbits() >> 12); + Inp.faddbits(4); + if (ZeroCount==0) + BitLength[I]=15; + else + { + ZeroCount+=2; + while (ZeroCount-- > 0 && IReadTop-5) + if (!UnpReadBuf30()) + return(false); + uint Number=DecodeNumber(Inp,BlockTables.BD); + if (Number<16) + { + Table[I]=(byte)((Number+this.UnpOldTable[I]) & 0xf); + I++; + } + else + if (Number<18) + { + uint N; + if (Number==16) + { + N=(Inp.fgetbits() >> 13)+3; + Inp.faddbits(3); + } + else + { + N=(Inp.fgetbits() >> 9)+11; + Inp.faddbits(7); + } + if (I==0) + return false; // We cannot have "repeat previous" code at the first position. + else + while (N-- > 0 && I> 13)+3; + Inp.faddbits(3); + } + else + { + N=(Inp.fgetbits() >> 9)+11; + Inp.faddbits(7); + } + while (N-- > 0 && IReadTop) + return false; + MakeDecodeTables(Table,0,BlockTables.LD,NC30); + MakeDecodeTables(Table,(int)NC30,BlockTables.DD,DC30); + MakeDecodeTables(Table,(int)(NC30+DC30),BlockTables.LDD,LDC30); + MakeDecodeTables(Table,(int)(NC30+DC30+LDC30),BlockTables.RD,RC30); + //x memcpy(UnpOldTable,Table,sizeof(UnpOldTable)); + Array.Copy(Table,0,UnpOldTable,0,UnpOldTable.Length); + return true; +} + +#endif + +void UnpInitData30(bool Solid) +{ + if (!Solid) + { + TablesRead3=false; + Utility.Memset(UnpOldTable, 0, UnpOldTable.Length); + PPMEscChar=2; + UnpBlockType=BLOCK_LZ; + } + InitFilters30(Solid); +} + + +void InitFilters30(bool Solid) +{ + if (!Solid) + { + //OldFilterLengths.SoftReset(); + OldFilterLengths.Clear(); + LastFilter=0; + + //for (size_t I=0;I=ReadBorder) + { + bool FileDone=false; + + // We use 'while', because for empty block containing only Huffman table, + // we'll be on the block border once again just after reading the table. + while (Inp.InAddr>BlockHeader.BlockStart+BlockHeader.BlockSize-1 || + Inp.InAddr==BlockHeader.BlockStart+BlockHeader.BlockSize-1 && + Inp.InBit>=BlockHeader.BlockBitSize) + { + if (BlockHeader.LastBlockInFile) + { + FileDone=true; + break; + } + if (!ReadBlockHeader(Inp,ref BlockHeader) || !ReadTables(Inp, ref BlockHeader, ref BlockTables)) + return; + } + if (FileDone || !UnpReadBuf()) + break; + } + + if (((WriteBorder-UnpPtr) & MaxWinMask)DestUnpSize) + return; + if (Suspended) + { + FileExtracted=false; + return; + } + } + + uint MainSlot=DecodeNumber(Inp,BlockTables.LD); + if (MainSlot<256) + { + if (Fragmented) + FragWindow[UnpPtr++]=(byte)MainSlot; + else + Window[UnpPtr++]=(byte)MainSlot; + continue; + } + if (MainSlot>=262) + { + uint Length=SlotToLength(Inp,MainSlot-262); + + uint DBits,Distance=1,DistSlot=DecodeNumber(Inp,BlockTables.DD); + if (DistSlot<4) + { + DBits=0; + Distance+=DistSlot; + } + else + { + DBits=DistSlot/2 - 1; + Distance+=(2 | (DistSlot & 1)) << (int)DBits; + } + + if (DBits>0) + { + if (DBits>=4) + { + if (DBits>4) + { + Distance+=((Inp.getbits32()>>(int)(36-DBits))<<4); + Inp.addbits(DBits-4); + } + uint LowDist=DecodeNumber(Inp,BlockTables.LDD); + Distance+=LowDist; + } + else + { + Distance+=Inp.getbits32()>>(int)(32-DBits); + Inp.addbits(DBits); + } + } + + if (Distance>0x100) + { + Length++; + if (Distance>0x2000) + { + Length++; + if (Distance>0x40000) + Length++; + } + } + + InsertOldDist(Distance); + LastLength=Length; + if (Fragmented) + FragWindow.CopyString(Length,Distance,ref UnpPtr,MaxWinMask); + else + CopyString(Length,Distance); + continue; + } + if (MainSlot==256) + { + UnpackFilter Filter = new UnpackFilter(); + if (!ReadFilter(Inp,Filter) || !AddFilter(Filter)) + break; + continue; + } + if (MainSlot==257) + { + if (LastLength!=0) + if (Fragmented) + FragWindow.CopyString(LastLength,OldDist[0],ref UnpPtr,MaxWinMask); + else + CopyString(LastLength,OldDist[0]); + continue; + } + if (MainSlot<262) + { + uint DistNum=MainSlot-258; + uint Distance=OldDist[DistNum]; + for (uint I=DistNum;I>0;I--) + OldDist[I]=OldDist[I-1]; + OldDist[0]=Distance; + + uint LengthSlot=DecodeNumber(Inp,BlockTables.RD); + uint Length=SlotToLength(Inp,LengthSlot); + LastLength=Length; + if (Fragmented) + FragWindow.CopyString(Length,Distance,ref UnpPtr,MaxWinMask); + else + CopyString(Length,Distance); + continue; + } + } + UnpWriteBuf(); +} + + private uint ReadFilterData(BitInput Inp) +{ + uint ByteCount=(Inp.fgetbits()>>14)+1; + Inp.addbits(2); + + uint Data=0; + for (uint I=0;I>8)<<(int)(I*8); + Inp.addbits(8); + } + return Data; +} + + private bool ReadFilter(BitInput Inp,UnpackFilter Filter) +{ + if (!Inp.ExternalBuffer && Inp.InAddr>ReadTop-16) + if (!UnpReadBuf()) + return false; + + Filter.BlockStart=ReadFilterData(Inp); + Filter.BlockLength=ReadFilterData(Inp); + if (Filter.BlockLength>MAX_FILTER_BLOCK_SIZE) + Filter.BlockLength=0; + + Filter.Type=(byte)(Inp.fgetbits()>>13); + Inp.faddbits(3); + + if (Filter.Type==FILTER_DELTA) + { + Filter.Channels=(byte)((Inp.fgetbits()>>11)+1); + Inp.faddbits(5); + } + + return true; +} + + private bool AddFilter(UnpackFilter Filter) +{ + if (Filters.Count>=MAX_UNPACK_FILTERS) + { + UnpWriteBuf(); // Write data, apply and flush filters. + if (Filters.Count>=MAX_UNPACK_FILTERS) + InitFilters(); // Still too many filters, prevent excessive memory use. + } + + // If distance to filter start is that large that due to circular dictionary + // mode now it points to old not written yet data, then we set 'NextWindow' + // flag and process this filter only after processing that older data. + Filter.NextWindow=WrPtr!=UnpPtr && ((WrPtr-UnpPtr)&MaxWinMask)<=Filter.BlockStart; + + Filter.BlockStart=(uint)((Filter.BlockStart+UnpPtr)&MaxWinMask); + Filters.Add(Filter); + return true; +} + + private bool UnpReadBuf() +{ + int DataSize=ReadTop-Inp.InAddr; // Data left to process. + if (DataSize<0) + return false; + BlockHeader.BlockSize-=Inp.InAddr-BlockHeader.BlockStart; + if (Inp.InAddr>MAX_SIZE/2) + { + // If we already processed more than half of buffer, let's move + // remaining data into beginning to free more space for new data + // and ensure that calling function does not cross the buffer border + // even if we did not read anything here. Also it ensures that read size + // is not less than CRYPT_BLOCK_SIZE, so we can align it without risk + // to make it zero. + if (DataSize>0) + //x memmove(Inp.InBuf,Inp.InBuf+Inp.InAddr,DataSize); + Buffer.BlockCopy(Inp.InBuf, Inp.InAddr, Inp.InBuf, 0, DataSize); + Inp.InAddr=0; + ReadTop=DataSize; + } + else + DataSize=ReadTop; + int ReadCode=0; + if (MAX_SIZE!=DataSize) + ReadCode=UnpIO_UnpRead(Inp.InBuf,DataSize,MAX_SIZE-DataSize); + if (ReadCode>0) // Can be also -1. + ReadTop+=ReadCode; + ReadBorder=ReadTop-30; + BlockHeader.BlockStart=Inp.InAddr; + if (BlockHeader.BlockSize!=-1) // '-1' means not defined yet. + { + // We may need to quit from main extraction loop and read new block header + // and trees earlier than data in input buffer ends. + ReadBorder=Math.Min(ReadBorder,BlockHeader.BlockStart+BlockHeader.BlockSize-1); + } + return ReadCode!=-1; +} + + private void UnpWriteBuf() +{ + size_t WrittenBorder=WrPtr; + size_t FullWriteSize=(UnpPtr-WrittenBorder)&MaxWinMask; + size_t WriteSizeLeft=FullWriteSize; + bool NotAllFiltersProcessed=false; + //for (size_t I=0;I int + for (int I=0;I0) // We set it to 0 also for invalid filters. + { + uint BlockEnd=(BlockStart+BlockLength)&MaxWinMask; + + //x FilterSrcMemory.Alloc(BlockLength); + FilterSrcMemory = EnsureCapacity(FilterSrcMemory, checked((int)BlockLength)); + byte[] Mem= FilterSrcMemory; + if (BlockStart int + for (int J=I;J int + int EmptyCount=0; + // sharpcompress: size_t -> int + for (int I=0;I0) + Filters[I-EmptyCount]=Filters[I]; + if (Filters[I].Type==FILTER_NONE) + EmptyCount++; + } + if (EmptyCount>0) + //Filters.Alloc(Filters.Count-EmptyCount); + Filters.RemoveRange(Filters.Count-EmptyCount, EmptyCount); + + if (!NotAllFiltersProcessed) // Only if all filters are processed. + { + // Write data left after last filter. + UnpWriteArea(WrittenBorder,UnpPtr); + WrPtr=UnpPtr; + } + + // We prefer to write data in blocks not exceeding UNPACK_MAX_WRITE + // instead of potentially huge MaxWinSize blocks. It also allows us + // to keep the size of Filters array reasonable. + WriteBorder=(UnpPtr+Math.Min(MaxWinSize,UNPACK_MAX_WRITE))&MaxWinMask; + + // Choose the nearest among WriteBorder and WrPtr actual written border. + // If border is equal to UnpPtr, it means that we have MaxWinSize data ahead. + if (WriteBorder==UnpPtr || + WrPtr!=UnpPtr && ((WrPtr-UnpPtr)&MaxWinMask)<((WriteBorder-UnpPtr)&MaxWinMask)) + WriteBorder=WrPtr; +} + + private byte[] ApplyFilter(byte[] __d,uint DataSize,UnpackFilter Flt) +{ + int Data = 0; + byte[] SrcData=__d; + switch(Flt.Type) + { + case FILTER_E8: + case FILTER_E8E9: + { + uint FileOffset=(uint)WrittenFileSize; + + const uint FileSize=0x1000000; + byte CmpByte2=Flt.Type==FILTER_E8E9 ? (byte)0xe9 : (byte)0xe8; + // DataSize is unsigned, so we use "CurPos+4" and not "DataSize-4" + // to avoid overflow for DataSize<4. + for (uint CurPos=0;CurPos+4=0 + RawPut4(Addr+FileSize,__d,Data); + } + else + if (((Addr-FileSize) & 0x80000000)!=0) // Addr>8); + __d[D+2]=(byte)(Offset>>16); + } + } + } + return SrcData; + case FILTER_DELTA: + { + // Unlike RAR3, we do not need to reject excessive channel + // values here, since RAR5 uses only 5 bits to store channel. + uint Channels=Flt.Channels,SrcPos=0; + + //x FilterDstMemory.Alloc(DataSize); + FilterDstMemory = EnsureCapacity(FilterDstMemory, checked((int)DataSize)); + + byte[] DstData=FilterDstMemory; + + // Bytes from same channels are grouped to continual data blocks, + // so we need to place them back to their interleaving positions. + for (uint CurChannel=0;CurChannel0) + { + size_t BlockSize=FragWindow.GetBlockSize(StartPtr,SizeToWrite); + //UnpWriteData(&FragWindow[StartPtr],BlockSize); + FragWindow.GetBuffer(StartPtr, out var __buffer, out var __offset); + UnpWriteData(__buffer, __offset, BlockSize); + SizeToWrite-=BlockSize; + StartPtr=(StartPtr+BlockSize) & MaxWinMask; + } + } + else + if (EndPtr=DestUnpSize) + return; + size_t WriteSize=Size; + int64 LeftToWrite=DestUnpSize-WrittenFileSize; + if ((int64)WriteSize>LeftToWrite) + WriteSize=(size_t)LeftToWrite; + UnpIO_UnpWrite(Data, offset, WriteSize); + WrittenFileSize+=Size; +} + + private void UnpInitData50(bool Solid) +{ + if (!Solid) + TablesRead5=false; +} + + private bool ReadBlockHeader(BitInput Inp,ref UnpackBlockHeader Header) +{ + Header.HeaderSize=0; + + if (!Inp.ExternalBuffer && Inp.InAddr>ReadTop-7) + if (!UnpReadBuf()) + return false; + Inp.faddbits((uint)((8-Inp.InBit)&7)); + + byte BlockFlags=(byte)(Inp.fgetbits()>>8); + Inp.faddbits(8); + uint ByteCount=(uint)(((BlockFlags>>3)&3)+1); // Block size byte count. + + if (ByteCount==4) + return false; + + Header.HeaderSize=(int)(2+ByteCount); + + Header.BlockBitSize=(BlockFlags&7)+1; + + byte SavedCheckSum=(byte)(Inp.fgetbits()>>8); + Inp.faddbits(8); + + int BlockSize=0; + for (uint I=0;I>8)<<(int)(I*8)); + Inp.addbits(8); + } + + Header.BlockSize=BlockSize; + byte CheckSum=(byte)(0x5a^BlockFlags^BlockSize^(BlockSize>>8)^(BlockSize>>16)); + if (CheckSum!=SavedCheckSum) + return false; + + Header.BlockStart=Inp.InAddr; + ReadBorder=Math.Min(ReadBorder,Header.BlockStart+Header.BlockSize-1); + + Header.LastBlockInFile=(BlockFlags & 0x40)!=0; + Header.TablePresent=(BlockFlags & 0x80)!=0; + return true; +} + + private bool ReadTables(BitInput Inp,ref UnpackBlockHeader Header, ref UnpackBlockTables Tables) +{ + if (!Header.TablePresent) + return true; + + if (!Inp.ExternalBuffer && Inp.InAddr>ReadTop-25) + if (!UnpReadBuf()) + return false; + + byte[] BitLength = new byte[BC]; + for (uint I=0;I> 12); + Inp.faddbits(4); + if (Length==15) + { + uint ZeroCount=(byte)(Inp.fgetbits() >> 12); + Inp.faddbits(4); + if (ZeroCount==0) + BitLength[I]=15; + else + { + ZeroCount+=2; + while (ZeroCount-- > 0 && IReadTop-5) + if (!UnpReadBuf()) + return false; + uint Number=DecodeNumber(Inp,Tables.BD); + if (Number<16) + { + Table[I]=(byte)Number; + I++; + } + else + if (Number<18) + { + uint N; + if (Number==16) + { + N=(Inp.fgetbits() >> 13)+3; + Inp.faddbits(3); + } + else + { + N=(Inp.fgetbits() >> 9)+11; + Inp.faddbits(7); + } + if (I==0) + { + // We cannot have "repeat previous" code at the first position. + // Multiple such codes would shift Inp position without changing I, + // which can lead to reading beyond of Inp boundary in mutithreading + // mode, where Inp.ExternalBuffer disables bounds check and we just + // reserve a lot of buffer space to not need such check normally. + return false; + } + else + while (N-- > 0 && I> 13)+3; + Inp.faddbits(3); + } + else + { + N=(Inp.fgetbits() >> 9)+11; + Inp.faddbits(7); + } + while (N-- > 0 && IReadTop) + return false; + MakeDecodeTables(Table, 0, Tables.LD,NC); + MakeDecodeTables(Table, (int)NC,Tables.DD,DC); + MakeDecodeTables(Table, (int)(NC+DC),Tables.LDD,LDC); + MakeDecodeTables(Table, (int)(NC+DC+LDC),Tables.RD,RC); + return true; +} + + private void InitFilters() +{ + //Filters.SoftReset(); + Filters.Clear(); +} + + } +} diff --git a/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack_cpp.cs b/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack_cpp.cs new file mode 100644 index 00000000..5b76ef4c --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV2017/Unpack.unpack_cpp.cs @@ -0,0 +1,377 @@ +#if !Rar2017_64bit +using nint = System.Int32; +using nuint = System.UInt32; +using size_t = System.UInt32; +#else +using nint = System.Int64; +using nuint = System.UInt64; +using size_t = System.UInt64; +#endif + +using System; +using SharpCompress.Common; +using static SharpCompress.Compressors.Rar.UnpackV2017.UnpackGlobal; +using static SharpCompress.Compressors.Rar.UnpackV2017.PackDef; + +namespace SharpCompress.Compressors.Rar.UnpackV2017 +{ + internal sealed partial class Unpack : BitInput + { + +public Unpack(/* ComprDataIO *DataIO */) +//:Inp(true),VMCodeInp(true) +: base(true) +{ + _UnpackCtor(); + + //UnpIO=DataIO; + Window=null; + Fragmented=false; + Suspended=false; + UnpAllBuf=false; + UnpSomeRead=false; +#if RarV2017_RAR_SMP + MaxUserThreads=1; + UnpThreadPool=CreateThreadPool(); + ReadBufMT=null; + UnpThreadData=null; +#endif + MaxWinSize=0; + MaxWinMask=0; + + // Perform initialization, which should be done only once for all files. + // It prevents crash if first DoUnpack call is later made with wrong + // (true) 'Solid' value. + UnpInitData(false); +#if !RarV2017_SFX_MODULE + // RAR 1.5 decompression initialization + UnpInitData15(false); + InitHuff(); +#endif +} + +// later: may need Dispose() if we support thread pool +//Unpack::~Unpack() +//{ +// InitFilters30(false); +// +// if (Window!=null) +// free(Window); +//#if RarV2017_RAR_SMP +// DestroyThreadPool(UnpThreadPool); +// delete[] ReadBufMT; +// delete[] UnpThreadData; +//#endif +//} + + private void Init(size_t WinSize,bool Solid) +{ + // If 32-bit RAR unpacks an archive with 4 GB dictionary, the window size + // will be 0 because of size_t overflow. Let's issue the memory error. + if (WinSize==0) + //ErrHandler.MemoryError(); + throw new InvalidFormatException("invalid window size (possibly due to a rar file with a 4GB being unpacked on a 32-bit platform)"); + + // Minimum window size must be at least twice more than maximum possible + // size of filter block, which is 0x10000 in RAR now. If window size is + // smaller, we can have a block with never cleared flt->NextWindow flag + // in UnpWriteBuf(). Minimum window size 0x20000 would be enough, but let's + // use 0x40000 for extra safety and possible filter area size expansion. + const size_t MinAllocSize=0x40000; + if (WinSize>16)>0x10000) // Window size must not exceed 4 GB. + return; + + // Archiving code guarantees that window size does not grow in the same + // solid stream. So if we are here, we are either creating a new window + // or increasing the size of non-solid window. So we could safely reject + // current window data without copying them to a new window, though being + // extra cautious, we still handle the solid window grow case below. + bool Grow=Solid && (Window!=null || Fragmented); + + // We do not handle growth for existing fragmented window. + if (Grow && Fragmented) + //throw std::bad_alloc(); + throw new InvalidFormatException("Grow && Fragmented"); + + byte[] NewWindow=Fragmented ? null : new byte[WinSize]; + + if (NewWindow==null) + if (Grow || WinSize<0x1000000) + { + // We do not support growth for new fragmented window. + // Also exclude RAR4 and small dictionaries. + //throw std::bad_alloc(); + throw new InvalidFormatException("Grow || WinSize<0x1000000"); + } + else + { + if (Window!=null) // If allocated by preceding files. + { + //free(Window); + Window=null; + } + FragWindow.Init(WinSize); + Fragmented=true; + } + + if (!Fragmented) + { + // Clean the window to generate the same output when unpacking corrupt + // RAR files, which may access unused areas of sliding dictionary. + // sharpcompress: don't need this, freshly allocated above + //memset(NewWindow,0,WinSize); + + + // If Window is not NULL, it means that window size has grown. + // In solid streams we need to copy data to a new window in such case. + // RAR archiving code does not allow it in solid streams now, + // but let's implement it anyway just in case we'll change it sometimes. + if (Grow) + for (size_t I=1;I<=MaxWinSize;I++) + NewWindow[(UnpPtr-I)&(WinSize-1)]=Window[(UnpPtr-I)&(MaxWinSize-1)]; + + //if (Window!=null) + // free(Window); + Window=NewWindow; + } + + MaxWinSize=WinSize; + MaxWinMask=MaxWinSize-1; +} + + private void DoUnpack(uint Method,bool Solid) +{ + // Methods <50 will crash in Fragmented mode when accessing NULL Window. + // They cannot be called in such mode now, but we check it below anyway + // just for extra safety. + switch(Method) + { +#if !RarV2017_SFX_MODULE + case 15: // rar 1.5 compression + if (!Fragmented) + Unpack15(Solid); + break; + case 20: // rar 2.x compression + case 26: // files larger than 2GB + if (!Fragmented) + Unpack20(Solid); + break; +#endif +#if !RarV2017_RAR5ONLY + case 29: // rar 3.x compression + if (!Fragmented) + throw new NotImplementedException(); + break; +#endif + case 50: // RAR 5.0 compression algorithm. +#if RarV2017_RAR_SMP + if (MaxUserThreads>1) + { +// We do not use the multithreaded unpack routine to repack RAR archives +// in 'suspended' mode, because unlike the single threaded code it can +// write more than one dictionary for same loop pass. So we would need +// larger buffers of unknown size. Also we do not support multithreading +// in fragmented window mode. + if (!Fragmented) + { + Unpack5MT(Solid); + break; + } + } +#endif + Unpack5(Solid); + break; +#if !Rar2017_NOSTRICT + default: throw new InvalidFormatException("unknown compression method " + Method); +#endif + } +} + + private void UnpInitData(bool Solid) +{ + if (!Solid) + { + Utility.Memset(OldDist, 0, OldDist.Length); + OldDistPtr=0; + LastDist=LastLength=0; +// memset(Window,0,MaxWinSize); + //memset(&BlockTables,0,sizeof(BlockTables)); + BlockTables = new UnpackBlockTables(); + // sharpcompress: no default ctor for struct + BlockTables.Init(); + UnpPtr=WrPtr=0; + WriteBorder=Math.Min(MaxWinSize,UNPACK_MAX_WRITE)&MaxWinMask; + } + // Filters never share several solid files, so we can safely reset them + // even in solid archive. + InitFilters(); + + Inp.InitBitInput(); + WrittenFileSize=0; + ReadTop=0; + ReadBorder=0; + + //memset(&BlockHeader,0,sizeof(BlockHeader)); + BlockHeader = new UnpackBlockHeader(); + BlockHeader.BlockSize=-1; // '-1' means not defined yet. +#if !RarV2017_SFX_MODULE + UnpInitData20(Solid); +#endif + //UnpInitData30(Solid); + UnpInitData50(Solid); +} + + +// LengthTable contains the length in bits for every element of alphabet. +// Dec is the structure to decode Huffman code/ +// Size is size of length table and DecodeNum field in Dec structure, + private void MakeDecodeTables(byte[] LengthTable, int offset, DecodeTable Dec,uint Size) +{ + // Size of alphabet and DecodePos array. + Dec.MaxNum=Size; + + // Calculate how many entries for every bit length in LengthTable we have. + uint[] LengthCount = new uint[16]; + //memset(LengthCount,0,sizeof(LengthCount)); + for (size_t I=0;IDecodeNum,0,Size*sizeof(*Dec->DecodeNum)); + Utility.FillFast(Dec.DecodeNum, 0); + + // Initialize not really used entry for zero length code. + Dec.DecodePos[0]=0; + + // Start code for bit length 1 is 0. + Dec.DecodeLen[0]=0; + + // Right aligned upper limit code for current bit length. + uint UpperLimit=0; + + for (int I=1;I<16;I++) + { + // Adjust the upper limit code. + UpperLimit+=LengthCount[I]; + + // Left aligned upper limit code. + uint LeftAligned=UpperLimit<<(16-I); + + // Prepare the upper limit code for next bit length. + UpperLimit*=2; + + // Store the left aligned upper limit code. + Dec.DecodeLen[I]=(uint)LeftAligned; + + // Every item of this array contains the sum of all preceding items. + // So it contains the start position in code list for every bit length. + Dec.DecodePos[I]=Dec.DecodePos[I-1]+LengthCount[I-1]; + } + + // Prepare the copy of DecodePos. We'll modify this copy below, + // so we cannot use the original DecodePos. + uint[] CopyDecodePos = new uint[Dec.DecodePos.Length]; + //memcpy(CopyDecodePos,Dec->DecodePos,sizeof(CopyDecodePos)); + Array.Copy(Dec.DecodePos, 0, CopyDecodePos, 0, CopyDecodePos.Length); + + // For every bit length in the bit length table and so for every item + // of alphabet. + for (uint I=0;I=Dec.DecodeLen[CurBitLength]) + CurBitLength++; + + // Translation of right aligned bit string to bit length. + Dec.QuickLen[Code]=CurBitLength; + + // Prepare the table for quick translation of position in code list + // to position in alphabet. + + // Calculate the distance from the start code for current bit length. + uint Dist=BitField-Dec.DecodeLen[CurBitLength-1]; + + // Right align the distance. + Dist>>=(16-CurBitLength); + + // Now we can calculate the position in the code list. It is the sum + // of first position for current bit length and right aligned distance + // between our bit field and start code for current bit length. + uint Pos; + if (CurBitLength 0) + { + Window[UnpPtr++] = Window[SrcPtr++]; + } + +// byte *Src=Window+SrcPtr; +// byte *Dest=Window+UnpPtr; +// UnpPtr+=Length; +// +//#if FAST_MEMCPY +// if (Distance=8) +// { +// Dest[0]=Src[0]; +// Dest[1]=Src[1]; +// Dest[2]=Src[2]; +// Dest[3]=Src[3]; +// Dest[4]=Src[4]; +// Dest[5]=Src[5]; +// Dest[6]=Src[6]; +// Dest[7]=Src[7]; +// +// Src+=8; +// Dest+=8; +// Length-=8; +// } +//#if FAST_MEMCPY +// else +// while (Length>=8) +// { +// // In theory we still could overlap here. +// // Supposing Distance == MaxWinSize - 1 we have memcpy(Src, Src + 1, 8). +// // But for real RAR archives Distance <= MaxWinSize - MAX_LZ_MATCH +// // always, so overlap here is impossible. +// +// // This memcpy expanded inline by MSVC. We could also use uint64 +// // assignment, which seems to provide about the same speed. +// memcpy(Dest,Src,8); +// +// Src+=8; +// Dest+=8; +// Length-=8; +// } +//#endif +// +// // Unroll the loop for 0 - 7 bytes left. Note that we use nested "if"s. +// if (Length>0) { Dest[0]=Src[0]; +// if (Length>1) { Dest[1]=Src[1]; +// if (Length>2) { Dest[2]=Src[2]; +// if (Length>3) { Dest[3]=Src[3]; +// if (Length>4) { Dest[4]=Src[4]; +// if (Length>5) { Dest[5]=Src[5]; +// if (Length>6) { Dest[6]=Src[6]; } } } } } } } // Close all nested "if"s. + } + else + while (Length-- > 0) // Slow copying with all possible precautions. + { + Window[UnpPtr]=Window[SrcPtr++ & MaxWinMask]; + // We need to have masked UnpPtr after quit from loop, so it must not + // be replaced with 'Window[UnpPtr++ & MaxWinMask]' + UnpPtr=(UnpPtr+1) & MaxWinMask; + } +} + + private uint DecodeNumber(BitInput Inp,DecodeTable Dec) +{ + // Left aligned 15 bit length raw bit field. + uint BitField=Inp.getbits() & 0xfffe; + + if (BitField>(int)(16-Dec.QuickBits); + Inp.addbits(Dec.QuickLen[Code]); + return Dec.QuickNum[Code]; + } + + // Detect the real bit length for current code. + uint Bits=15; + for (uint I=Dec.QuickBits+1;I<15;I++) + if (BitField>=(int)(16-Bits); + + // Now we can calculate the position in the code list. It is the sum + // of first position for current bit length and right aligned distance + // between our bit field and start code for current bit length. + uint Pos=Dec.DecodePos[Bits]+Dist; + + // Out of bounds safety check required for damaged archives. + if (Pos>=Dec.MaxNum) + Pos=0; + + // Convert the position in the code list to position in alphabet + // and return it. + return Dec.DecodeNum[Pos]; +} + + private uint SlotToLength(BitInput Inp,uint Slot) +{ + uint LBits,Length=2; + if (Slot<8) + { + LBits=0; + Length+=Slot; + } + else + { + LBits=Slot/4-1; + Length+=(4 | (Slot & 3)) << (int)LBits; + } + + if (LBits>0) + { + Length+=Inp.getbits()>>(int)(16-LBits); + Inp.addbits(LBits); + } + return Length; +} + + } +} diff --git a/src/SharpCompress/Compressors/Rar/UnpackV2017/notes.txt b/src/SharpCompress/Compressors/Rar/UnpackV2017/notes.txt new file mode 100644 index 00000000..d56277c2 --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV2017/notes.txt @@ -0,0 +1,50 @@ +#if !Rar2017_64bit +using nint = System.Int32; +using nuint = System.UInt32; +using size_t = System.UInt32; +#else +using nint = System.Int64; +using nuint = System.UInt64; +using size_t = System.UInt64; +#endif +using int64 = System.Int64; + + +notes on C->C# primitive mappings: + +nint := native integer +nuint := native unsigned integer + +type 32b 64b mapping CAREFUL! +char 8 bit 8 bit short +unsigned char 8 bit 8 bit ushort +short int 16 bit 16 bit short + +short int 16 bit 16 bit short +unsigned short int 16 bit 16 bit ushort +int 32 bit 32 bit int +unsigned int 32 bit 32 bit uint +long int 32 bit 64 bit nint *** +unsigned long int 32 bit 64 bit nuint *** +long long int 64 bit 64 bit long +unsigned long long int 64 bit 64 bit ulong +size_t 32 bit 64 bit nuint + +The size_t type is the unsigned integer type that is the result of the sizeof operator (and the offsetof operator), +so it is guaranteed to be big enough to contain the size of the biggest object your system can handle (e.g., a static array of 8Gb). +[size_t] -> ulong (x64) +[size_t] -> uint (x86) + +size_t is an unsigned data type defined by several C/C++ standards, e.g. the C99 ISO/IEC 9899 standard, that is defined +in stddef.h.1 It can be further imported by inclusion of stdlib.h as this file internally sub includes stddef.h. +This type is used to represent the size of an object. Library functions that take or return sizes expect them to be of type or +have the return type of size_t. Further, the most frequently used compiler-based operator sizeof should evaluate to a constant +value that is compatible with size_t. + + +20171218 +urggh, this allows things like new int[int.MaxValue] but NOT new byte[uint.MaxValue] +currently arrays are limited to being indexed by an int hence int.MaxValue entries. weak. +To get arrays > 2GB on x64 we need to configure + +https://docs.microsoft.com/en-us/dotnet/framework/configure-apps/file-schema/runtime/gcallowverylargeobjects-element \ No newline at end of file diff --git a/src/SharpCompress/Compressors/Rar/UnpackV2017/unpack_hpp.cs b/src/SharpCompress/Compressors/Rar/UnpackV2017/unpack_hpp.cs new file mode 100644 index 00000000..8586c98b --- /dev/null +++ b/src/SharpCompress/Compressors/Rar/UnpackV2017/unpack_hpp.cs @@ -0,0 +1,441 @@ +#if !Rar2017_64bit +using nint = System.Int32; +using nuint = System.UInt32; +using size_t = System.UInt32; +#else +using nint = System.Int64; +using nuint = System.UInt64; +using size_t = System.UInt64; +#endif +using int64 = System.Int64; + +using System.Collections.Generic; +using static SharpCompress.Compressors.Rar.UnpackV2017.PackDef; +using static SharpCompress.Compressors.Rar.UnpackV2017.UnpackGlobal; + +// TODO: REMOVE THIS... WIP +#pragma warning disable 169 +#pragma warning disable 414 + +namespace SharpCompress.Compressors.Rar.UnpackV2017 +{ + internal static class UnpackGlobal + { + + +// Maximum allowed number of compressed bits processed in quick mode. +public const int MAX_QUICK_DECODE_BITS =10; + +// Maximum number of filters per entire data block. Must be at least +// twice more than MAX_PACK_FILTERS to store filters from two data blocks. +public const int MAX_UNPACK_FILTERS =8192; + +// Maximum number of filters per entire data block for RAR3 unpack. +// Must be at least twice more than v3_MAX_PACK_FILTERS to store filters +// from two data blocks. +public const int MAX3_UNPACK_FILTERS =8192; + +// Limit maximum number of channels in RAR3 delta filter to some reasonable +// value to prevent too slow processing of corrupt archives with invalid +// channels number. Must be equal or larger than v3_MAX_FILTER_CHANNELS. +// No need to provide it for RAR5, which uses only 5 bits to store channels. +private const int MAX3_UNPACK_CHANNELS =1024; + +// Maximum size of single filter block. We restrict it to limit memory +// allocation. Must be equal or larger than MAX_ANALYZE_SIZE. +public const int MAX_FILTER_BLOCK_SIZE =0x400000; + +// Write data in 4 MB or smaller blocks. Must not exceed PACK_MAX_WRITE, +// so we keep number of buffered filter in unpacker reasonable. +public const int UNPACK_MAX_WRITE =0x400000; + } + +// Decode compressed bit fields to alphabet numbers. + internal sealed class DecodeTable +{ + // Real size of DecodeNum table. + public uint MaxNum; + + // Left aligned start and upper limit codes defining code space + // ranges for bit lengths. DecodeLen[BitLength-1] defines the start of + // range for bit length and DecodeLen[BitLength] defines next code + // after the end of range or in other words the upper limit code + // for specified bit length. + public readonly uint[] DecodeLen = new uint[16]; + + // Every item of this array contains the sum of all preceding items. + // So it contains the start position in code list for every bit length. + public readonly uint[] DecodePos = new uint[16]; + + // Number of compressed bits processed in quick mode. + // Must not exceed MAX_QUICK_DECODE_BITS. + public uint QuickBits; + + // Translates compressed bits (up to QuickBits length) + // to bit length in quick mode. + public readonly byte[] QuickLen = new byte[1< Filters = new List(); + + private readonly uint[] OldDist = new uint[4]; + private uint OldDistPtr; + private uint LastLength; + + // LastDist is necessary only for RAR2 and older with circular OldDist + // array. In RAR3 last distance is always stored in OldDist[0]. + private uint LastDist; + + private size_t UnpPtr,WrPtr; + + // Top border of read packed data. + private int ReadTop; + + // Border to call UnpReadBuf. We use it instead of (ReadTop-C) + // for optimization reasons. Ensures that we have C bytes in buffer + // unless we are at the end of file. + private int ReadBorder; + + private UnpackBlockHeader BlockHeader; + private UnpackBlockTables BlockTables; + + private size_t WriteBorder; + + private byte[] Window; + + private readonly FragmentedWindow FragWindow = new FragmentedWindow(); + private bool Fragmented; + + private int64 DestUnpSize; + + //bool Suspended; + private bool UnpAllBuf; + private bool UnpSomeRead; + private int64 WrittenFileSize; + private bool FileExtracted; + + +/***************************** Unpack v 1.5 *********************************/ + //void Unpack15(bool Solid); + //void ShortLZ(); + //void LongLZ(); + //void HuffDecode(); + //void GetFlagsBuf(); + //void UnpInitData15(int Solid); + //void InitHuff(); + //void CorrHuff(ushort *CharSet,byte *NumToPlace); + //void CopyString15(uint Distance,uint Length); + //uint DecodeNum(uint Num,uint StartPos,uint *DecTab,uint *PosTab); + + private readonly ushort[] ChSet = new ushort[256],ChSetA = new ushort[256],ChSetB = new ushort[256],ChSetC = new ushort[256]; + private readonly byte[] NToPl = new byte[256],NToPlB = new byte[256],NToPlC = new byte[256]; + private uint FlagBuf,AvrPlc,AvrPlcB,AvrLn1,AvrLn2,AvrLn3; + private int Buf60,NumHuf,StMode,LCount,FlagsCnt; + + private uint Nhfb,Nlzb,MaxDist3; +/***************************** Unpack v 1.5 *********************************/ + +/***************************** Unpack v 2.0 *********************************/ + //void Unpack20(bool Solid); + + private DecodeTable[] MD = new DecodeTable[4]; // Decode multimedia data, up to 4 channels. + + private readonly byte[] UnpOldTable20 = new byte[MC20*4]; + private bool UnpAudioBlock; + private uint UnpChannels,UnpCurChannel; + + private int UnpChannelDelta; + //void CopyString20(uint Length,uint Distance); + //bool ReadTables20(); + //void UnpWriteBuf20(); + //void UnpInitData20(int Solid); + //void ReadLastTables(); + //byte DecodeAudio(int Delta); + private AudioVariables[] AudV = new AudioVariables[4]; +/***************************** Unpack v 2.0 *********************************/ + +/***************************** Unpack v 3.0 *********************************/ + public const int BLOCK_LZ = 0; + public const int BLOCK_PPM = 1; + + //void UnpInitData30(bool Solid); + //void Unpack29(bool Solid); + //void InitFilters30(bool Solid); + //bool ReadEndOfBlock(); + //bool ReadVMCode(); + //bool ReadVMCodePPM(); + //bool AddVMCode(uint FirstByte,byte *Code,int CodeSize); + //int SafePPMDecodeChar(); + //bool ReadTables30(); + //bool UnpReadBuf30(); + //void UnpWriteBuf30(); + //void ExecuteCode(VM_PreparedProgram *Prg); + + private int PrevLowDist,LowDistRepCount; + +/*#if !RarV2017_RAR5ONLY + ModelPPM PPM; +#endif*/ + private int PPMEscChar; + + private readonly byte [] UnpOldTable = new byte[HUFF_TABLE_SIZE30]; + private int UnpBlockType; + + // If we already read decoding tables for Unpack v2,v3,v5. + // We should not use a single variable for all algorithm versions, + // because we can have a corrupt archive with one algorithm file + // followed by another algorithm file with "solid" flag and we do not + // want to reuse tables from one algorithm in another. + private bool TablesRead2,TablesRead3,TablesRead5; + + // Virtual machine to execute filters code. +/*#if !RarV2017_RAR5ONLY + RarVM VM; +#endif*/ + + // Buffer to read VM filters code. We moved it here from AddVMCode + // function to reduce time spent in BitInput constructor. + private readonly BitInput VMCodeInp = new BitInput(true); + + // Filters code, one entry per filter. + private readonly List Filters30 = new List(); + + // Filters stack, several entrances of same filter are possible. + private readonly List PrgStack = new List(); + + // Lengths of preceding data blocks, one length of one last block + // for every filter. Used to reduce the size required to write + // the data block length if lengths are repeating. + private readonly List OldFilterLengths = new List(); + + private int LastFilter; +/***************************** Unpack v 3.0 *********************************/ + + //Unpack(ComprDataIO *DataIO); + //~Unpack(); + //void Init(size_t WinSize,bool Solid); + //void DoUnpack(uint Method,bool Solid); + private bool IsFileExtracted() {return(FileExtracted);} + private void SetDestSize(int64 DestSize) {DestUnpSize=DestSize;FileExtracted=false;} + private void SetSuspended(bool Suspended) {this.Suspended=Suspended;} + +#if RarV2017_RAR_SMP + // More than 8 threads are unlikely to provide a noticeable gain + // for unpacking, but would use the additional memory. + void SetThreads(uint Threads) {MaxUserThreads=Min(Threads,8);} + + void UnpackDecode(UnpackThreadData &D); +#endif + + private size_t MaxWinSize; + private size_t MaxWinMask; + + private uint GetChar() + { + if (Inp.InAddr>MAX_SIZE-30) + UnpReadBuf(); + return(Inp.InBuf[Inp.InAddr++]); + } + + + } +} diff --git a/src/SharpCompress/Compressors/Rar/VM/BitInput.cs b/src/SharpCompress/Compressors/Rar/VM/BitInput.cs index 74c525b2..82575912 100644 --- a/src/SharpCompress/Compressors/Rar/VM/BitInput.cs +++ b/src/SharpCompress/Compressors/Rar/VM/BitInput.cs @@ -5,8 +5,14 @@ namespace SharpCompress.Compressors.Rar.VM /// the max size of the input internal const int MAX_SIZE = 0x8000; - protected int inAddr; - protected int inBit; + public int inAddr; + public int inBit; + +// TODO: rename var + public int InAddr { get { return inAddr; } set { inAddr = value; } } + public int InBit { get { return inBit; } set { inBit = value; } } + public bool ExternalBuffer; + /// internal BitInput() @@ -22,6 +28,11 @@ namespace SharpCompress.Compressors.Rar.VM inBit = 0; } + internal void faddbits(uint bits) { + // TODO uint + AddBits((int)bits); + } + /// /// also named faddbits /// @@ -33,6 +44,16 @@ namespace SharpCompress.Compressors.Rar.VM inBit = bits & 7; } + internal uint fgetbits() { + // TODO uint + return (uint)GetBits(); + } + + internal uint getbits() { + // TODO uint + return (uint)GetBits(); + } + /// /// (also named fgetbits) /// diff --git a/src/SharpCompress/Compressors/Xz/Crc32.cs b/src/SharpCompress/Compressors/Xz/Crc32.cs index cb65fec9..72c37ba1 100644 --- a/src/SharpCompress/Compressors/Xz/Crc32.cs +++ b/src/SharpCompress/Compressors/Xz/Crc32.cs @@ -8,7 +8,7 @@ namespace SharpCompress.Compressors.Xz public const UInt32 DefaultPolynomial = 0xedb88320u; public const UInt32 DefaultSeed = 0xffffffffu; - static UInt32[] defaultTable; + private static UInt32[] defaultTable; public static UInt32 Compute(byte[] buffer) { @@ -25,7 +25,7 @@ namespace SharpCompress.Compressors.Xz return ~CalculateHash(InitializeTable(polynomial), seed, buffer, 0, buffer.Length); } - static UInt32[] InitializeTable(UInt32 polynomial) + private static UInt32[] InitializeTable(UInt32 polynomial) { if (polynomial == DefaultPolynomial && defaultTable != null) return defaultTable; @@ -48,7 +48,7 @@ namespace SharpCompress.Compressors.Xz return createTable; } - static UInt32 CalculateHash(UInt32[] table, UInt32 seed, IList buffer, int start, int size) + private static UInt32 CalculateHash(UInt32[] table, UInt32 seed, IList buffer, int start, int size) { var crc = seed; for (var i = start; i < size - start; i++) diff --git a/src/SharpCompress/Compressors/Xz/Filters/BlockFilter.cs b/src/SharpCompress/Compressors/Xz/Filters/BlockFilter.cs index e260a9cd..8a13ec07 100644 --- a/src/SharpCompress/Compressors/Xz/Filters/BlockFilter.cs +++ b/src/SharpCompress/Compressors/Xz/Filters/BlockFilter.cs @@ -15,11 +15,11 @@ namespace SharpCompress.Compressors.Xz.Filters ARCH_ARM_FILTER = 0x07, ARCH_ARMTHUMB_FILTER = 0x08, ARCH_SPARC_FILTER = 0x09, - LZMA2 = 0x21, + LZMA2 = 0x21 } - static Dictionary FilterMap = new Dictionary() - { + private static readonly Dictionary FilterMap = new Dictionary + { {FilterTypes.LZMA2, typeof(Lzma2Filter) } }; @@ -27,8 +27,6 @@ namespace SharpCompress.Compressors.Xz.Filters public abstract bool AllowAsNonLast { get; } public abstract bool ChangesDataSize { get; } - public BlockFilter() { } - public abstract void Init(byte[] properties); public abstract void ValidateFilter(); diff --git a/src/SharpCompress/Compressors/Xz/Filters/Lzma2Filter.cs b/src/SharpCompress/Compressors/Xz/Filters/Lzma2Filter.cs index 7d0020e8..cd2869c9 100644 --- a/src/SharpCompress/Compressors/Xz/Filters/Lzma2Filter.cs +++ b/src/SharpCompress/Compressors/Xz/Filters/Lzma2Filter.cs @@ -1,5 +1,6 @@ using System; using System.IO; +using SharpCompress.Compressors.LZMA; namespace SharpCompress.Compressors.Xz.Filters { @@ -9,7 +10,7 @@ namespace SharpCompress.Compressors.Xz.Filters public override bool AllowAsNonLast => false; public override bool ChangesDataSize => true; - byte _dictionarySize; + private byte _dictionarySize; public uint DictionarySize { get @@ -43,12 +44,17 @@ namespace SharpCompress.Compressors.Xz.Filters public override void SetBaseStream(Stream stream) { - BaseStream = new SharpCompress.Compressors.LZMA.LzmaStream(new[] { _dictionarySize }, stream); + BaseStream = new LzmaStream(new[] { _dictionarySize }, stream); } public override int Read(byte[] buffer, int offset, int count) { return BaseStream.Read(buffer, offset, count); } + + public override int ReadByte() + { + return BaseStream.ReadByte(); + } } } diff --git a/src/SharpCompress/Compressors/Xz/MultiByteIntegers.cs b/src/SharpCompress/Compressors/Xz/MultiByteIntegers.cs index 27c9a6fd..24795128 100644 --- a/src/SharpCompress/Compressors/Xz/MultiByteIntegers.cs +++ b/src/SharpCompress/Compressors/Xz/MultiByteIntegers.cs @@ -18,7 +18,7 @@ namespace SharpCompress.Compressors.Xz int i = 0; while ((LastByte & 0x80) != 0) { - if (i >= MaxBytes) + if (++i >= MaxBytes) throw new InvalidDataException(); LastByte = reader.ReadByte(); if (LastByte == 0) diff --git a/src/SharpCompress/Compressors/Xz/XZBlock.cs b/src/SharpCompress/Compressors/Xz/XZBlock.cs index 118bdc54..96ecef17 100644 --- a/src/SharpCompress/Compressors/Xz/XZBlock.cs +++ b/src/SharpCompress/Compressors/Xz/XZBlock.cs @@ -13,7 +13,7 @@ namespace SharpCompress.Compressors.Xz public Stack Filters { get; private set; } = new Stack(); public bool HeaderIsLoaded { get; private set; } private CheckType _checkType; - private int _checkSize; + private readonly int _checkSize; private bool _streamConnected; private int _numFilters; private byte _blockHeaderSizeByte; @@ -50,11 +50,11 @@ namespace SharpCompress.Compressors.Xz private void SkipPadding() { - int padding = (int)(_bytesRead % 4); - if (padding > 0) + int bytes = (int)(BaseStream.Position % 4); + if (bytes > 0) { - byte[] paddingBytes = new byte[padding]; - BaseStream.Read(paddingBytes, 0, padding); + byte[] paddingBytes = new byte[4 - bytes]; + BaseStream.Read(paddingBytes, 0, paddingBytes.Length); if (paddingBytes.Any(b => b != 0)) throw new InvalidDataException("Padding bytes were non-null"); } diff --git a/src/SharpCompress/Compressors/Xz/XZFooter.cs b/src/SharpCompress/Compressors/Xz/XZFooter.cs index 73d89be4..8339bc65 100644 --- a/src/SharpCompress/Compressors/Xz/XZFooter.cs +++ b/src/SharpCompress/Compressors/Xz/XZFooter.cs @@ -8,7 +8,7 @@ namespace SharpCompress.Compressors.Xz public class XZFooter { private readonly BinaryReader _reader; - private readonly byte[] _magicBytes = new byte[] { 0x59, 0x5A }; + private readonly byte[] _magicBytes = { 0x59, 0x5A }; public long StreamStartPosition { get; private set; } public long BackwardSize { get; private set; } public byte[] StreamFlags { get; private set; } @@ -40,7 +40,7 @@ namespace SharpCompress.Compressors.Xz StreamFlags = reader.ReadBytes(2); } byte[] magBy = _reader.ReadBytes(2); - if (!Enumerable.SequenceEqual(magBy, _magicBytes)) + if (!magBy.SequenceEqual(_magicBytes)) { throw new InvalidDataException("Magic footer missing"); } diff --git a/src/SharpCompress/Compressors/Xz/XZHeader.cs b/src/SharpCompress/Compressors/Xz/XZHeader.cs index c74b888c..16588d6e 100644 --- a/src/SharpCompress/Compressors/Xz/XZHeader.cs +++ b/src/SharpCompress/Compressors/Xz/XZHeader.cs @@ -48,7 +48,7 @@ namespace SharpCompress.Compressors.Xz private void CheckMagicBytes(byte[] header) { - if (!Enumerable.SequenceEqual(header, MagicHeader)) + if (!header.SequenceEqual(MagicHeader)) throw new InvalidDataException("Invalid XZ Stream"); } } diff --git a/src/SharpCompress/Compressors/Xz/XZIndex.cs b/src/SharpCompress/Compressors/Xz/XZIndex.cs index 5e223284..736f978a 100644 --- a/src/SharpCompress/Compressors/Xz/XZIndex.cs +++ b/src/SharpCompress/Compressors/Xz/XZIndex.cs @@ -15,7 +15,7 @@ namespace SharpCompress.Compressors.Xz public ulong NumberOfRecords { get; private set; } public List Records { get; } = new List(); - private bool _indexMarkerAlreadyVerified; + private readonly bool _indexMarkerAlreadyVerified; public XZIndex(BinaryReader reader, bool indexMarkerAlreadyVerified) { @@ -55,10 +55,10 @@ namespace SharpCompress.Compressors.Xz private void SkipPadding() { - int padding = (int)(_reader.BaseStream.Position - StreamStartPosition) % 4; - if (padding > 0) + int bytes = (int)(_reader.BaseStream.Position - StreamStartPosition) % 4; + if (bytes > 0) { - byte[] paddingBytes = _reader.ReadBytes(padding); + byte[] paddingBytes = _reader.ReadBytes(4 - bytes); if (paddingBytes.Any(b => b != 0)) throw new InvalidDataException("Padding bytes were non-null"); } diff --git a/src/SharpCompress/Compressors/Xz/XZStream.cs b/src/SharpCompress/Compressors/Xz/XZStream.cs index d1de55a1..df755c26 100644 --- a/src/SharpCompress/Compressors/Xz/XZStream.cs +++ b/src/SharpCompress/Compressors/Xz/XZStream.cs @@ -40,7 +40,7 @@ namespace SharpCompress.Compressors.Xz public bool HeaderIsRead { get; private set; } private XZBlock _currentBlock; - bool _endOfStream; + private bool _endOfStream; public XZStream(Stream stream) : base(stream) { diff --git a/src/SharpCompress/Converters/DataConverter.cs b/src/SharpCompress/Converters/DataConverter.cs index 2ea5c5c9..6be04586 100644 --- a/src/SharpCompress/Converters/DataConverter.cs +++ b/src/SharpCompress/Converters/DataConverter.cs @@ -47,12 +47,12 @@ namespace SharpCompress.Converters public #endif - unsafe abstract class DataConverter + internal unsafe abstract class DataConverter { // Disables the warning: CLS compliance checking will not be performed on // `XXXX' because it is not visible from outside this assembly #pragma warning disable 3019 - static readonly DataConverter SwapConv = new SwapConverter(); + private static readonly DataConverter SwapConv = new SwapConverter(); public static readonly bool IsLittleEndian = BitConverter.IsLittleEndian; @@ -164,7 +164,7 @@ namespace SharpCompress.Converters } } - class CopyConverter : DataConverter + private class CopyConverter : DataConverter { public override double GetDouble(byte[] data, int index) { @@ -462,7 +462,7 @@ namespace SharpCompress.Converters } } - class SwapConverter : DataConverter + private class SwapConverter : DataConverter { public override double GetDouble(byte[] data, int index) { diff --git a/src/SharpCompress/Crypto/Crc32Stream.cs b/src/SharpCompress/Crypto/Crc32Stream.cs index 13cbc6e2..35dd85bd 100644 --- a/src/SharpCompress/Crypto/Crc32Stream.cs +++ b/src/SharpCompress/Crypto/Crc32Stream.cs @@ -47,6 +47,12 @@ namespace SharpCompress.Crypto hash = CalculateCrc(table, hash, buffer, offset, count); } + public override void WriteByte(byte value) + { + stream.WriteByte(value); + hash = CalculateCrc(table, hash, value); + } + public override bool CanRead => stream.CanRead; public override bool CanSeek => false; public override bool CanWrite => stream.CanWrite; @@ -98,9 +104,16 @@ namespace SharpCompress.Crypto unchecked { for (int i = offset, end = offset + count; i < end; i++) - crc = (crc >> 8) ^ table[(crc ^ buffer[i]) & 0xFF]; + { + crc = CalculateCrc(table, crc, buffer[i]); + } } return crc; } + + private static uint CalculateCrc(uint[] table, uint crc, byte b) + { + return (crc >> 8) ^ table[(crc ^ b) & 0xFF]; + } } } diff --git a/src/SharpCompress/Crypto/CryptoException.cs b/src/SharpCompress/Crypto/CryptoException.cs index 7eb42a74..6e8b9f14 100644 --- a/src/SharpCompress/Crypto/CryptoException.cs +++ b/src/SharpCompress/Crypto/CryptoException.cs @@ -1,6 +1,6 @@ using System; -namespace Org.BouncyCastle.Crypto +namespace SharpCompress.Crypto { public class CryptoException : Exception diff --git a/src/SharpCompress/Crypto/DataLengthException.cs b/src/SharpCompress/Crypto/DataLengthException.cs index 8a3e1c47..b82a9e9c 100644 --- a/src/SharpCompress/Crypto/DataLengthException.cs +++ b/src/SharpCompress/Crypto/DataLengthException.cs @@ -1,6 +1,6 @@ using System; -namespace Org.BouncyCastle.Crypto +namespace SharpCompress.Crypto { public class DataLengthException : CryptoException diff --git a/src/SharpCompress/Crypto/IBlockCipher.cs b/src/SharpCompress/Crypto/IBlockCipher.cs index a4f30012..6f6542e3 100644 --- a/src/SharpCompress/Crypto/IBlockCipher.cs +++ b/src/SharpCompress/Crypto/IBlockCipher.cs @@ -1,4 +1,4 @@ -namespace Org.BouncyCastle.Crypto +namespace SharpCompress.Crypto { /// Base interface for a symmetric key block cipher. public interface IBlockCipher diff --git a/src/SharpCompress/Crypto/ICipherParameters.cs b/src/SharpCompress/Crypto/ICipherParameters.cs index 16439a25..d6e4af8b 100644 --- a/src/SharpCompress/Crypto/ICipherParameters.cs +++ b/src/SharpCompress/Crypto/ICipherParameters.cs @@ -1,4 +1,4 @@ -namespace Org.BouncyCastle.Crypto +namespace SharpCompress.Crypto { public interface ICipherParameters { diff --git a/src/SharpCompress/Crypto/KeyParameter.cs b/src/SharpCompress/Crypto/KeyParameter.cs index b69ce4e1..8658fc9c 100644 --- a/src/SharpCompress/Crypto/KeyParameter.cs +++ b/src/SharpCompress/Crypto/KeyParameter.cs @@ -1,6 +1,6 @@ using System; -namespace Org.BouncyCastle.Crypto.Parameters +namespace SharpCompress.Crypto { public class KeyParameter : ICipherParameters diff --git a/src/SharpCompress/Crypto/RijndaelEngine.cs b/src/SharpCompress/Crypto/RijndaelEngine.cs index 6fa73b4b..06e2bbdd 100644 --- a/src/SharpCompress/Crypto/RijndaelEngine.cs +++ b/src/SharpCompress/Crypto/RijndaelEngine.cs @@ -1,7 +1,6 @@ using System; -using Org.BouncyCastle.Crypto.Parameters; -namespace Org.BouncyCastle.Crypto.Engines +namespace SharpCompress.Crypto { public class RijndaelEngine : IBlockCipher @@ -128,7 +127,7 @@ namespace Org.BouncyCastle.Crypto.Engines 0x2f, 0x5e, 0xbc, 0x63, 0xc6, 0x97, 0x35, 0x6a, 0xd4, 0xb3, 0x7d, 0xfa, 0xef, 0xc5, 0x91 }; - static readonly byte[][] shifts0 = + private static readonly byte[][] shifts0 = { new byte[] {0, 8, 16, 24}, new byte[] {0, 8, 16, 24}, @@ -137,7 +136,7 @@ namespace Org.BouncyCastle.Crypto.Engines new byte[] {0, 8, 24, 32} }; - static readonly byte[][] shifts1 = + private static readonly byte[][] shifts1 = { new byte[] {0, 24, 16, 8}, new byte[] {0, 32, 24, 16}, diff --git a/src/SharpCompress/IO/BufferedSubStream.cs b/src/SharpCompress/IO/BufferedSubStream.cs index dd913d49..f08be705 100644 --- a/src/SharpCompress/IO/BufferedSubStream.cs +++ b/src/SharpCompress/IO/BufferedSubStream.cs @@ -41,7 +41,7 @@ namespace SharpCompress.IO throw new NotSupportedException(); } - public override long Length => throw new NotSupportedException(); + public override long Length => BytesLeftToRead; public override long Position { get => throw new NotSupportedException(); set => throw new NotSupportedException(); } diff --git a/src/SharpCompress/IO/CountingWritableSubStream.cs b/src/SharpCompress/IO/CountingWritableSubStream.cs index c08235c6..17ddf6bb 100644 --- a/src/SharpCompress/IO/CountingWritableSubStream.cs +++ b/src/SharpCompress/IO/CountingWritableSubStream.cs @@ -1,4 +1,4 @@ -using System; +using System; using System.IO; namespace SharpCompress.IO @@ -49,5 +49,11 @@ namespace SharpCompress.IO writableStream.Write(buffer, offset, count); Count += (uint)count; } + + public override void WriteByte(byte value) + { + writableStream.WriteByte(value); + ++Count; + } } } \ No newline at end of file diff --git a/src/SharpCompress/IO/ListeningStream.cs b/src/SharpCompress/IO/ListeningStream.cs index 1cc694e0..a5a76dd1 100644 --- a/src/SharpCompress/IO/ListeningStream.cs +++ b/src/SharpCompress/IO/ListeningStream.cs @@ -47,6 +47,19 @@ namespace SharpCompress.IO return read; } + public override int ReadByte() + { + int value = Stream.ReadByte(); + if (value == -1) + { + return -1; + } + + ++currentEntryTotalReadBytes; + listener.FireCompressedBytesRead(currentEntryTotalReadBytes, currentEntryTotalReadBytes); + return value; + } + public override long Seek(long offset, SeekOrigin origin) { return Stream.Seek(offset, origin); @@ -61,5 +74,10 @@ namespace SharpCompress.IO { Stream.Write(buffer, offset, count); } + + public override void WriteByte(byte value) + { + Stream.WriteByte(value); + } } } \ No newline at end of file diff --git a/src/SharpCompress/IO/MarkingBinaryReader.cs b/src/SharpCompress/IO/MarkingBinaryReader.cs index aa7c820b..e1037995 100644 --- a/src/SharpCompress/IO/MarkingBinaryReader.cs +++ b/src/SharpCompress/IO/MarkingBinaryReader.cs @@ -1,6 +1,5 @@ using System; using System.IO; -using System.Linq; using SharpCompress.Converters; namespace SharpCompress.IO @@ -36,12 +35,22 @@ namespace SharpCompress.IO public override bool ReadBoolean() { - return ReadBytes(1).Single() != 0; + return ReadByte() != 0; } + // NOTE: there is a somewhat fragile dependency on the internals of this class + // with RarCrcBinaryReader and RarCryptoBinaryReader. + // + // RarCrcBinaryReader/RarCryptoBinaryReader need to override any specific methods + // that call directly to the base BinaryReader and do not delegate to other methods + // in this class so that it can track the each byte being read. + // + // if altering this class in a way that changes the implementation be sure to + // update RarCrcBinaryReader/RarCryptoBinaryReader. public override byte ReadByte() { - return ReadBytes(1).Single(); + CurrentReadByteCount++; + return base.ReadByte(); } public override byte[] ReadBytes(int count) @@ -121,5 +130,77 @@ namespace SharpCompress.IO { return DataConverter.LittleEndian.GetUInt64(ReadBytes(8), 0); } + + // RAR5 style variable length encoded value + // maximum value of 0xffffffffffffffff (64 bits) + // technote: "implies max 10 bytes consumed" -- but not really because we could extend indefinitely using 0x80 0x80 ... 0x80 00 + // + // Variable length integer. Can include one or more bytes, where lower 7 bits of every byte contain integer data + // and highest bit in every byte is the continuation flag. If highest bit is 0, this is the last byte in sequence. + // So first byte contains 7 least significant bits of integer and continuation flag. Second byte, if present, + // contains next 7 bits and so on. + public ulong ReadRarVInt(int maxBytes = 10) { + // hopefully this gets inlined + return DoReadRarVInt((maxBytes - 1) * 7); + } + + private ulong DoReadRarVInt(int maxShift) { + int shift = 0; + ulong result = 0; + do { + byte b0 = ReadByte(); + uint b1 = ((uint)b0) & 0x7f; + ulong n = b1; + ulong shifted = n << shift; + if (n != shifted >> shift) { + // overflow + break; + } + result |= shifted; + if (b0 == b1) { + return result; + } + shift += 7; + } while (shift <= maxShift); + + throw new FormatException("malformed vint"); + } + + public uint ReadRarVIntUInt32(int maxBytes = 5) { + // hopefully this gets inlined + return DoReadRarVIntUInt32((maxBytes - 1) * 7); + } + + public ushort ReadRarVIntUInt16(int maxBytes = 3) { + // hopefully this gets inlined + return checked((ushort)DoReadRarVIntUInt32((maxBytes - 1) * 7)); + } + + public byte ReadRarVIntByte(int maxBytes = 2) { + // hopefully this gets inlined + return checked((byte)DoReadRarVIntUInt32((maxBytes - 1) * 7)); + } + + private uint DoReadRarVIntUInt32(int maxShift) { + int shift = 0; + uint result = 0; + do { + byte b0 = ReadByte(); + uint b1 = ((uint)b0) & 0x7f; + uint n = b1; + uint shifted = n << shift; + if (n != shifted >> shift) { + // overflow + break; + } + result |= shifted; + if (b0 == b1) { + return result; + } + shift += 7; + } while (shift <= maxShift); + + throw new FormatException("malformed vint"); + } } } \ No newline at end of file diff --git a/src/SharpCompress/IO/NonDisposingStream.cs b/src/SharpCompress/IO/NonDisposingStream.cs index 0c5363d1..2c9e72c4 100644 --- a/src/SharpCompress/IO/NonDisposingStream.cs +++ b/src/SharpCompress/IO/NonDisposingStream.cs @@ -1,20 +1,28 @@ -using System.IO; +using System; +using System.IO; namespace SharpCompress.IO { - internal class NonDisposingStream : Stream + public class NonDisposingStream : Stream { - public NonDisposingStream(Stream stream) + public NonDisposingStream(Stream stream, bool throwOnDispose = false) { Stream = stream; + ThrowOnDispose = throwOnDispose; } + public bool ThrowOnDispose { get; set; } + protected override void Dispose(bool disposing) { - //don't dispose anything + GC.SuppressFinalize(this); + if (ThrowOnDispose) + { + throw new InvalidOperationException(); + } } - public Stream Stream { get; } + protected Stream Stream { get; } public override bool CanRead => Stream.CanRead; @@ -36,6 +44,11 @@ namespace SharpCompress.IO return Stream.Read(buffer, offset, count); } + public override int ReadByte() + { + return Stream.ReadByte(); + } + public override long Seek(long offset, SeekOrigin origin) { return Stream.Seek(offset, origin); @@ -50,5 +63,10 @@ namespace SharpCompress.IO { Stream.Write(buffer, offset, count); } + + public override void WriteByte(byte value) + { + Stream.WriteByte(value); + } } } \ No newline at end of file diff --git a/src/SharpCompress/IO/ReadOnlySubStream.cs b/src/SharpCompress/IO/ReadOnlySubStream.cs index 48609526..8747019a 100644 --- a/src/SharpCompress/IO/ReadOnlySubStream.cs +++ b/src/SharpCompress/IO/ReadOnlySubStream.cs @@ -3,7 +3,7 @@ using System.IO; namespace SharpCompress.IO { - internal class ReadOnlySubStream : Stream + internal class ReadOnlySubStream : NonDisposingStream { public ReadOnlySubStream(Stream stream, long bytesToRead) : this(stream, null, bytesToRead) @@ -11,8 +11,8 @@ namespace SharpCompress.IO } public ReadOnlySubStream(Stream stream, long? origin, long bytesToRead) + : base(stream, false) { - Stream = stream; if (origin != null) { stream.Position = origin.Value; @@ -20,18 +20,8 @@ namespace SharpCompress.IO BytesLeftToRead = bytesToRead; } - protected override void Dispose(bool disposing) - { - if (disposing) - { - //Stream.Dispose(); - } - } - private long BytesLeftToRead { get; set; } - public Stream Stream { get; } - public override bool CanRead => true; public override bool CanSeek => false; @@ -61,6 +51,20 @@ namespace SharpCompress.IO return read; } + public override int ReadByte() + { + if (BytesLeftToRead <= 0) + { + return -1; + } + int value = Stream.ReadByte(); + if (value != -1) + { + --BytesLeftToRead; + } + return value; + } + public override long Seek(long offset, SeekOrigin origin) { throw new NotSupportedException(); diff --git a/src/SharpCompress/IO/RewindableStream.cs b/src/SharpCompress/IO/RewindableStream.cs index c4b77659..3cfc035c 100644 --- a/src/SharpCompress/IO/RewindableStream.cs +++ b/src/SharpCompress/IO/RewindableStream.cs @@ -1,5 +1,6 @@ using System; using System.IO; +using SharpCompress.Compressors.Filters; namespace SharpCompress.IO { @@ -46,8 +47,13 @@ namespace SharpCompress.IO } else { + bufferStream.TransferTo(buffer); - bufferStream = buffer; + //create new memorystream to allow proper resizing as memorystream could be a user provided buffer + //https://github.com/adamhathcock/sharpcompress/issues/306 + bufferStream = new MemoryStream(); + buffer.Position = 0; + buffer.TransferTo(bufferStream); bufferStream.Position = 0; } isRewound = true; diff --git a/src/SharpCompress/Lazy.cs b/src/SharpCompress/Lazy.cs new file mode 100644 index 00000000..893e3bb1 --- /dev/null +++ b/src/SharpCompress/Lazy.cs @@ -0,0 +1,29 @@ +using System; + +namespace SharpCompress +{ + public class Lazy + { + private readonly Func _lazyFunc; + private bool _evaluated; + private T _value; + + public Lazy(Func lazyFunc) + { + _lazyFunc = lazyFunc; + } + + public T Value + { + get + { + if (!_evaluated) + { + _value = _lazyFunc(); + _evaluated = true; + } + return _value; + } + } + } +} \ No newline at end of file diff --git a/src/SharpCompress/Readers/AbstractReader.cs b/src/SharpCompress/Readers/AbstractReader.cs index a5cc157e..b09497f1 100644 --- a/src/SharpCompress/Readers/AbstractReader.cs +++ b/src/SharpCompress/Readers/AbstractReader.cs @@ -46,11 +46,8 @@ namespace SharpCompress.Readers public void Dispose() { - if (entriesForCurrentReadStream != null) - { - entriesForCurrentReadStream.Dispose(); - } - Volume.Dispose(); + entriesForCurrentReadStream?.Dispose(); + Volume?.Dispose(); } #endregion @@ -97,12 +94,9 @@ namespace SharpCompress.Readers return false; } - internal bool LoadStreamForReading(Stream stream) + protected bool LoadStreamForReading(Stream stream) { - if (entriesForCurrentReadStream != null) - { - entriesForCurrentReadStream.Dispose(); - } + entriesForCurrentReadStream?.Dispose(); if ((stream == null) || (!stream.CanRead)) { throw new MultipartStreamRequiredException("File is split into multiple archives: '" @@ -110,14 +104,10 @@ namespace SharpCompress.Readers "'. A new readable stream is required. Use Cancel if it was intended."); } entriesForCurrentReadStream = GetEntries(stream).GetEnumerator(); - if (entriesForCurrentReadStream.MoveNext()) - { - return true; - } - return false; + return entriesForCurrentReadStream.MoveNext(); } - internal virtual Stream RequestInitialStream() + protected virtual Stream RequestInitialStream() { return Volume.Stream; } @@ -127,7 +117,7 @@ namespace SharpCompress.Readers return entriesForCurrentReadStream.MoveNext(); } - internal abstract IEnumerable GetEntries(Stream stream); + protected abstract IEnumerable GetEntries(Stream stream); #region Entry Skip/Write @@ -172,8 +162,7 @@ namespace SharpCompress.Readers } if ((writableStream == null) || (!writableStream.CanWrite)) { - throw new ArgumentNullException( - "A writable Stream was required. Use Cancel if that was intended."); + throw new ArgumentNullException("A writable Stream was required. Use Cancel if that was intended."); } Write(writableStream); diff --git a/src/SharpCompress/Readers/GZip/GZipReader.cs b/src/SharpCompress/Readers/GZip/GZipReader.cs index b8265409..ecee69ff 100644 --- a/src/SharpCompress/Readers/GZip/GZipReader.cs +++ b/src/SharpCompress/Readers/GZip/GZipReader.cs @@ -31,7 +31,7 @@ namespace SharpCompress.Readers.GZip #endregion Open - internal override IEnumerable GetEntries(Stream stream) + protected override IEnumerable GetEntries(Stream stream) { return GZipEntry.GetEntries(stream, Options); } diff --git a/src/SharpCompress/Readers/IReaderExtensions.cs b/src/SharpCompress/Readers/IReaderExtensions.cs index 4100a607..65ef90ed 100644 --- a/src/SharpCompress/Readers/IReaderExtensions.cs +++ b/src/SharpCompress/Readers/IReaderExtensions.cs @@ -93,10 +93,6 @@ namespace SharpCompress.Readers using (FileStream fs = File.Open(destinationFileName, fm)) { reader.WriteEntryTo(fs); - //using (Stream s = reader.OpenEntryStream()) - //{ - // s.TransferTo(fs); - //} } reader.Entry.PreserveExtractionOptions(destinationFileName, options); } diff --git a/src/SharpCompress/Readers/Rar/MultiVolumeRarReader.cs b/src/SharpCompress/Readers/Rar/MultiVolumeRarReader.cs index 7bc61e2e..db30985a 100644 --- a/src/SharpCompress/Readers/Rar/MultiVolumeRarReader.cs +++ b/src/SharpCompress/Readers/Rar/MultiVolumeRarReader.cs @@ -22,7 +22,7 @@ namespace SharpCompress.Readers.Rar { } - internal override Stream RequestInitialStream() + protected override Stream RequestInitialStream() { if (streams.MoveNext()) { @@ -33,14 +33,9 @@ namespace SharpCompress.Readers.Rar internal override bool NextEntryForCurrentStream() { - if (!base.NextEntryForCurrentStream()) - { - //if we're got another stream to try to process then do so - if (streams.MoveNext() && LoadStreamForReading(streams.Current)) - { - return true; - } - return false; + if (!base.NextEntryForCurrentStream()) { + // if we're got another stream to try to process then do so + return streams.MoveNext() && LoadStreamForReading(streams.Current); } return true; } @@ -94,7 +89,7 @@ namespace SharpCompress.Readers.Rar return true; } - if (!reader.Entry.IsSplit) + if (!reader.Entry.IsSplitAfter) { return false; } diff --git a/src/SharpCompress/Readers/Rar/RarReader.cs b/src/SharpCompress/Readers/Rar/RarReader.cs index 92e4d737..a40dca44 100644 --- a/src/SharpCompress/Readers/Rar/RarReader.cs +++ b/src/SharpCompress/Readers/Rar/RarReader.cs @@ -1,4 +1,4 @@ -using System.Collections.Generic; +using System.Collections.Generic; using System.IO; using System.Linq; using SharpCompress.Common; @@ -13,7 +13,8 @@ namespace SharpCompress.Readers.Rar public abstract class RarReader : AbstractReader { private RarVolume volume; - private readonly Unpack pack = new Unpack(); + internal Lazy UnpackV2017 { get; } = new Lazy(() => new SharpCompress.Compressors.Rar.UnpackV2017.Unpack()); + internal Lazy UnpackV1 { get; } = new Lazy(() => new SharpCompress.Compressors.Rar.UnpackV1.Unpack()); internal RarReader(ReaderOptions options) : base(options, ArchiveType.Rar) @@ -24,8 +25,6 @@ namespace SharpCompress.Readers.Rar public override RarVolume Volume => volume; - #region Open - /// /// Opens a RarReader for Non-seeking usage with a single volume /// @@ -50,9 +49,7 @@ namespace SharpCompress.Readers.Rar return new MultiVolumeRarReader(streams, options ?? new ReaderOptions()); } - #endregion - - internal override IEnumerable GetEntries(Stream stream) + protected override IEnumerable GetEntries(Stream stream) { volume = new RarReaderVolume(stream, Options); foreach (RarFilePart fp in volume.ReadFileParts()) @@ -67,11 +64,14 @@ namespace SharpCompress.Readers.Rar return Entry.Parts; } - protected override EntryStream GetEntryStream() + protected override EntryStream GetEntryStream() { - return CreateEntryStream(new RarCrcStream(pack, Entry.FileHeader, - new MultiVolumeReadOnlyStream( - CreateFilePartEnumerableForCurrentEntry().Cast(), this))); + var stream = new MultiVolumeReadOnlyStream(CreateFilePartEnumerableForCurrentEntry().Cast(), this); + if (Entry.IsRarV3) + { + return CreateEntryStream(new RarCrcStream(UnpackV1.Value, Entry.FileHeader, stream)); + } + return CreateEntryStream(new RarCrcStream(UnpackV2017.Value, Entry.FileHeader, stream)); } } } diff --git a/src/SharpCompress/Readers/Rar/RarReaderVolume.cs b/src/SharpCompress/Readers/Rar/RarReaderVolume.cs index bf68043f..98a8c5c0 100644 --- a/src/SharpCompress/Readers/Rar/RarReaderVolume.cs +++ b/src/SharpCompress/Readers/Rar/RarReaderVolume.cs @@ -13,7 +13,7 @@ namespace SharpCompress.Readers.Rar { } - internal override RarFilePart CreateFilePart(FileHeader fileHeader, MarkHeader markHeader) + internal override RarFilePart CreateFilePart(MarkHeader markHeader, FileHeader fileHeader) { return new NonSeekableStreamFilePart(markHeader, fileHeader); } diff --git a/src/SharpCompress/Readers/Rar/SingleVolumeRarReader.cs b/src/SharpCompress/Readers/Rar/SingleVolumeRarReader.cs index 3ffede69..b8d64f8d 100644 --- a/src/SharpCompress/Readers/Rar/SingleVolumeRarReader.cs +++ b/src/SharpCompress/Readers/Rar/SingleVolumeRarReader.cs @@ -16,14 +16,13 @@ namespace SharpCompress.Readers.Rar internal override void ValidateArchive(RarVolume archive) { - if (archive.IsMultiVolume) - { - throw new MultiVolumeExtractionException( - "Streamed archive is a Multi-volume archive. Use different RarReader method to extract."); + if (archive.IsMultiVolume) { + var msg = "Streamed archive is a Multi-volume archive. Use different RarReader method to extract."; + throw new MultiVolumeExtractionException(msg); } } - internal override Stream RequestInitialStream() + protected override Stream RequestInitialStream() { return stream; } diff --git a/src/SharpCompress/Readers/ReaderFactory.cs b/src/SharpCompress/Readers/ReaderFactory.cs index cf22fcc0..ec7d15c9 100644 --- a/src/SharpCompress/Readers/ReaderFactory.cs +++ b/src/SharpCompress/Readers/ReaderFactory.cs @@ -58,7 +58,7 @@ namespace SharpCompress.Readers if (BZip2Stream.IsBZip2(rewindableStream)) { rewindableStream.Rewind(false); - BZip2Stream testStream = new BZip2Stream(rewindableStream, CompressionMode.Decompress, true); + BZip2Stream testStream = new BZip2Stream(new NonDisposingStream(rewindableStream), CompressionMode.Decompress, false); if (TarArchive.IsTarFile(testStream)) { rewindableStream.Rewind(true); @@ -70,7 +70,7 @@ namespace SharpCompress.Readers if (LZipStream.IsLZipFile(rewindableStream)) { rewindableStream.Rewind(false); - LZipStream testStream = new LZipStream(rewindableStream, CompressionMode.Decompress, true); + LZipStream testStream = new LZipStream(new NonDisposingStream(rewindableStream), CompressionMode.Decompress); if (TarArchive.IsTarFile(testStream)) { rewindableStream.Rewind(true); diff --git a/src/SharpCompress/Readers/Tar/TarReader.cs b/src/SharpCompress/Readers/Tar/TarReader.cs index 809ea3b5..e6c67a92 100644 --- a/src/SharpCompress/Readers/Tar/TarReader.cs +++ b/src/SharpCompress/Readers/Tar/TarReader.cs @@ -27,7 +27,7 @@ namespace SharpCompress.Readers.Tar public override TarVolume Volume { get; } - internal override Stream RequestInitialStream() + protected override Stream RequestInitialStream() { var stream = base.RequestInitialStream(); switch (compressionType) @@ -102,7 +102,7 @@ namespace SharpCompress.Readers.Tar if (LZipStream.IsLZipFile(rewindableStream)) { rewindableStream.Rewind(false); - LZipStream testStream = new LZipStream(rewindableStream, CompressionMode.Decompress, false); + LZipStream testStream = new LZipStream(rewindableStream, CompressionMode.Decompress); if (TarArchive.IsTarFile(testStream)) { rewindableStream.Rewind(true); @@ -116,7 +116,7 @@ namespace SharpCompress.Readers.Tar #endregion Open - internal override IEnumerable GetEntries(Stream stream) + protected override IEnumerable GetEntries(Stream stream) { return TarEntry.GetEntries(StreamingMode.Streaming, stream, compressionType, Options.ArchiveEncoding); } diff --git a/src/SharpCompress/Readers/Zip/ZipReader.cs b/src/SharpCompress/Readers/Zip/ZipReader.cs index 6dfbefb6..871f8283 100644 --- a/src/SharpCompress/Readers/Zip/ZipReader.cs +++ b/src/SharpCompress/Readers/Zip/ZipReader.cs @@ -10,7 +10,7 @@ namespace SharpCompress.Readers.Zip { private readonly StreamingZipHeaderFactory _headerFactory; - internal ZipReader(Stream stream, ReaderOptions options) + private ZipReader(Stream stream, ReaderOptions options) : base(options, ArchiveType.Zip) { Volume = new ZipVolume(stream, options); @@ -35,7 +35,7 @@ namespace SharpCompress.Readers.Zip #endregion Open - internal override IEnumerable GetEntries(Stream stream) + protected override IEnumerable GetEntries(Stream stream) { foreach (ZipHeader h in _headerFactory.ReadStreamHeader(stream)) { diff --git a/src/SharpCompress/SharpCompress.csproj b/src/SharpCompress/SharpCompress.csproj index 97c8e326..b331798c 100644 --- a/src/SharpCompress/SharpCompress.csproj +++ b/src/SharpCompress/SharpCompress.csproj @@ -2,12 +2,11 @@ SharpCompress - Pure C# Decompression/Compression en-US - 0.18 - 0.18.0.0 - 0.18.0.0 + 0.21.1 + 0.21.1 + 0.21.1 Adam Hathcock net45;net35;netstandard1.0;netstandard1.3;netstandard2.0 - $(LibraryFrameworks) true true SharpCompress @@ -20,9 +19,21 @@ https://github.com/adamhathcock/sharpcompress/blob/master/LICENSE.txt false false - SharpCompress is a compression library for NET Standard 1.0 that can unrar, decompress 7zip, decompress xz, zip/unzip, tar/untar lzip/unlzip, bzip2/unbzip2 and gzip/ungzip with forward-only reading and file random access APIs. Write support for zip/tar/bzip2/gzip is implemented. + SharpCompress is a compression library for NET Standard 1.0 that can unrar, decompress 7zip, decompress xz, zip/unzip, tar/untar lzip/unlzip, bzip2/unbzip2 and gzip/ungzip with forward-only reading and file random access APIs. Write support for zip/tar/bzip2/gzip is implemented. $(DefineConstants);NO_FILE;NO_CRYPTO;SILVERLIGHT + + $(DefineConstants);NETCORE + + + + + + $(DefineConstants);NETCORE + + + + \ No newline at end of file diff --git a/src/SharpCompress/Utility.cs b/src/SharpCompress/Utility.cs index 9da09c64..e97d093d 100644 --- a/src/SharpCompress/Utility.cs +++ b/src/SharpCompress/Utility.cs @@ -2,12 +2,18 @@ using System; using System.Collections.Generic; using System.IO; using System.Linq; +using System.Reflection; +using System.Reflection.Emit; +using System.Runtime.InteropServices; +#if NETCORE +using SharpCompress.Buffers; +#endif using SharpCompress.Readers; namespace SharpCompress { internal static class Utility - { + { public static ReadOnlyCollection ToReadOnly(this IEnumerable items) { return new ReadOnlyCollection(items.ToList()); @@ -70,6 +76,71 @@ namespace SharpCompress } } +#if NET45 + // super fast memset, up to 40x faster than for loop on large arrays + // see https://stackoverflow.com/questions/1897555/what-is-the-equivalent-of-memset-in-c + private static readonly Action MemsetDelegate = CreateMemsetDelegate(); + + private static Action CreateMemsetDelegate() { + var dynamicMethod = new DynamicMethod( + "Memset", + MethodAttributes.Public | MethodAttributes.Static, + CallingConventions.Standard, + null, + new[] { typeof(IntPtr), typeof(byte), typeof(uint) }, + typeof(Utility), + true); + var generator = dynamicMethod.GetILGenerator(); + generator.Emit(OpCodes.Ldarg_0); + generator.Emit(OpCodes.Ldarg_1); + generator.Emit(OpCodes.Ldarg_2); + generator.Emit(OpCodes.Initblk); + generator.Emit(OpCodes.Ret); + return (Action)dynamicMethod.CreateDelegate(typeof(Action)); + } + + public static void Memset(byte[] array, byte what, int length) + { + var gcHandle = GCHandle.Alloc(array, GCHandleType.Pinned); + MemsetDelegate(gcHandle.AddrOfPinnedObject(), what, (uint)length); + gcHandle.Free(); + } +#else + public static void Memset(byte[] array, byte what, int length) + { + for(var i = 0; i < length; i++) + { + array[i] = what; + } + } +#endif + + public static void Memset(T[] array, T what, int length) + { + for(var i = 0; i < length; i++) + { + array[i] = what; + } + } + + public static void FillFast(T[] array, T val) where T : struct + { + for (int i = 0; i < array.Length; i++) + { + array[i] = val; + } + } + + public static void FillFast(T[] array, int start, int length, T val) where T : struct + { + int toIndex = start + length; + for (int i = start; i < toIndex; i++) + { + array[i] = val; + } + } + + /// /// Fills the array with an specific value. /// @@ -113,6 +184,18 @@ namespace SharpCompress action(item); } } + + public static void Copy(Array sourceArray, long sourceIndex, Array destinationArray, long destinationIndex, long length) + { + if (sourceIndex > Int32.MaxValue || sourceIndex < Int32.MinValue) + throw new ArgumentOutOfRangeException(); + if (destinationIndex > Int32.MaxValue || destinationIndex < Int32.MinValue) + throw new ArgumentOutOfRangeException(); + if (length > Int32.MaxValue || length < Int32.MinValue) + throw new ArgumentOutOfRangeException(); + + Array.Copy(sourceArray, (int)sourceIndex, destinationArray, (int)destinationIndex, (int)length); + } public static IEnumerable AsEnumerable(this T item) { @@ -138,37 +221,61 @@ namespace SharpCompress public static void Skip(this Stream source, long advanceAmount) { - byte[] buffer = GetTransferByteArray(); - int read = 0; - int readCount = 0; - do + if (source.CanSeek) { - readCount = buffer.Length; - if (readCount > advanceAmount) - { - readCount = (int)advanceAmount; - } - read = source.Read(buffer, 0, readCount); - if (read <= 0) - { - break; - } - advanceAmount -= read; - if (advanceAmount == 0) - { - break; - } + source.Position += advanceAmount; + return; + } + + byte[] buffer = GetTransferByteArray(); + try + { + int read = 0; + int readCount = 0; + do + { + readCount = buffer.Length; + if (readCount > advanceAmount) + { + readCount = (int)advanceAmount; + } + read = source.Read(buffer, 0, readCount); + if (read <= 0) + { + break; + } + advanceAmount -= read; + if (advanceAmount == 0) + { + break; + } + } + while (true); + } + finally + { +#if NETCORE + ArrayPool.Shared.Return(buffer); +#endif } - while (true); } public static void Skip(this Stream source) { byte[] buffer = GetTransferByteArray(); - do + try { + do + { + } + while (source.Read(buffer, 0, buffer.Length) == buffer.Length); + } + finally + { +#if NETCORE + ArrayPool.Shared.Return(buffer); +#endif } - while (source.Read(buffer, 0, buffer.Length) == buffer.Length); } public static DateTime DosDateToDateTime(UInt16 iDate, UInt16 iTime) @@ -225,38 +332,62 @@ namespace SharpCompress (UInt16)(iTime % 65536)); } - public static DateTime DosDateToDateTime(Int32 iTime) + /// + /// Convert Unix time value to a DateTime object. + /// + /// The Unix time stamp you want to convert to DateTime. + /// Returns a DateTime object that represents value of the Unix time. + public static DateTime UnixTimeToDateTime(long unixtime) { - return DosDateToDateTime((UInt32)iTime); + DateTime sTime = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc); + return sTime.AddSeconds(unixtime); } public static long TransferTo(this Stream source, Stream destination) { byte[] array = GetTransferByteArray(); - int count; - long total = 0; - while (ReadTransferBlock(source, array, out count)) + try { - total += count; - destination.Write(array, 0, count); + int count; + long total = 0; + while (ReadTransferBlock(source, array, out count)) + { + total += count; + destination.Write(array, 0, count); + } + return total; + } + finally + { +#if NETCORE + ArrayPool.Shared.Return(array); +#endif } - return total; } public static long TransferTo(this Stream source, Stream destination, Common.Entry entry, IReaderExtractionListener readerExtractionListener) { byte[] array = GetTransferByteArray(); - int count; - var iterations = 0; - long total = 0; - while (ReadTransferBlock(source, array, out count)) + try { - total += count; - destination.Write(array, 0, count); - iterations++; - readerExtractionListener.FireEntryExtractionProgress(entry, total, iterations); + int count; + var iterations = 0; + long total = 0; + while (ReadTransferBlock(source, array, out count)) + { + total += count; + destination.Write(array, 0, count); + iterations++; + readerExtractionListener.FireEntryExtractionProgress(entry, total, iterations); + } + return total; + } + finally + { +#if NETCORE + ArrayPool.Shared.Return(array); +#endif } - return total; } private static bool ReadTransferBlock(Stream source, byte[] array, out int count) @@ -266,7 +397,11 @@ namespace SharpCompress private static byte[] GetTransferByteArray() { +#if NETCORE + return ArrayPool.Shared.Rent(81920); +#else return new byte[81920]; +#endif } public static bool ReadFully(this Stream stream, byte[] buffer) @@ -304,10 +439,5 @@ namespace SharpCompress } return true; } - - public static void CopyTo(this byte[] array, byte[] destination, int index) - { - Array.Copy(array, 0, destination, index, array.Length); - } } } \ No newline at end of file diff --git a/src/SharpCompress/Writers/AbstractWriter.cs b/src/SharpCompress/Writers/AbstractWriter.cs index d331f034..f6656a8b 100644 --- a/src/SharpCompress/Writers/AbstractWriter.cs +++ b/src/SharpCompress/Writers/AbstractWriter.cs @@ -6,7 +6,7 @@ namespace SharpCompress.Writers { public abstract class AbstractWriter : IWriter { - private bool isDisposed; + private bool _isDisposed; protected AbstractWriter(ArchiveType type, WriterOptions writerOptions) { @@ -29,7 +29,7 @@ namespace SharpCompress.Writers protected virtual void Dispose(bool isDisposing) { - if (isDisposing && !WriterOptions.LeaveStreamOpen) + if (isDisposing) { OutputStream.Dispose(); } @@ -37,20 +37,20 @@ namespace SharpCompress.Writers public void Dispose() { - if (!isDisposed) + if (!_isDisposed) { GC.SuppressFinalize(this); Dispose(true); - isDisposed = true; + _isDisposed = true; } } ~AbstractWriter() { - if (!isDisposed) + if (!_isDisposed) { Dispose(false); - isDisposed = true; + _isDisposed = true; } } } diff --git a/src/SharpCompress/Writers/GZip/GZipWriter.cs b/src/SharpCompress/Writers/GZip/GZipWriter.cs index 978fcf60..735d89f5 100644 --- a/src/SharpCompress/Writers/GZip/GZipWriter.cs +++ b/src/SharpCompress/Writers/GZip/GZipWriter.cs @@ -3,6 +3,7 @@ using System.IO; using SharpCompress.Common; using SharpCompress.Compressors; using SharpCompress.Compressors.Deflate; +using SharpCompress.IO; namespace SharpCompress.Writers.GZip { @@ -13,9 +14,12 @@ namespace SharpCompress.Writers.GZip public GZipWriter(Stream destination, GZipWriterOptions options = null) : base(ArchiveType.GZip, options ?? new GZipWriterOptions()) { + if (WriterOptions.LeaveStreamOpen) + { + destination = new NonDisposingStream(destination); + } InitalizeStream(new GZipStream(destination, CompressionMode.Compress, - options?.CompressionLevel ?? CompressionLevel.Default, - WriterOptions.LeaveStreamOpen, + options?.CompressionLevel ?? CompressionLevel.Default, WriterOptions.ArchiveEncoding.GetEncoding())); } diff --git a/src/SharpCompress/Writers/Tar/TarWriter.cs b/src/SharpCompress/Writers/Tar/TarWriter.cs index db7bc97a..8c782366 100644 --- a/src/SharpCompress/Writers/Tar/TarWriter.cs +++ b/src/SharpCompress/Writers/Tar/TarWriter.cs @@ -6,35 +6,44 @@ using SharpCompress.Compressors; using SharpCompress.Compressors.BZip2; using SharpCompress.Compressors.Deflate; using SharpCompress.Compressors.LZMA; +using SharpCompress.IO; namespace SharpCompress.Writers.Tar { public class TarWriter : AbstractWriter { - public TarWriter(Stream destination, WriterOptions options) + private readonly bool finalizeArchiveOnClose; + + public TarWriter(Stream destination, TarWriterOptions options) : base(ArchiveType.Tar, options) { + finalizeArchiveOnClose = options.FinalizeArchiveOnClose; + if (!destination.CanWrite) { throw new ArgumentException("Tars require writable streams."); } + if (WriterOptions.LeaveStreamOpen) + { + destination = new NonDisposingStream(destination); + } switch (options.CompressionType) { case CompressionType.None: break; case CompressionType.BZip2: { - destination = new BZip2Stream(destination, CompressionMode.Compress, true); + destination = new BZip2Stream(destination, CompressionMode.Compress, false); } break; case CompressionType.GZip: { - destination = new GZipStream(destination, CompressionMode.Compress, true); + destination = new GZipStream(destination, CompressionMode.Compress); } break; case CompressionType.LZip: { - destination = new LZipStream(destination, CompressionMode.Compress, true); + destination = new LZipStream(destination, CompressionMode.Compress); } break; default: @@ -74,7 +83,7 @@ namespace SharpCompress.Writers.Tar TarHeader header = new TarHeader(WriterOptions.ArchiveEncoding); - header.LastModifiedTime = modificationTime ?? TarHeader.Epoch; + header.LastModifiedTime = modificationTime ?? TarHeader.EPOCH; header.Name = NormalizeFilename(filename); header.Size = realSize; header.Write(OutputStream); @@ -97,8 +106,10 @@ namespace SharpCompress.Writers.Tar { if (isDisposing) { - PadTo512(0, true); - PadTo512(0, true); + if (finalizeArchiveOnClose) { + PadTo512(0, true); + PadTo512(0, true); + } switch (OutputStream) { case BZip2Stream b: diff --git a/src/SharpCompress/Writers/Tar/TarWriterOptions.cs b/src/SharpCompress/Writers/Tar/TarWriterOptions.cs new file mode 100755 index 00000000..9174b23f --- /dev/null +++ b/src/SharpCompress/Writers/Tar/TarWriterOptions.cs @@ -0,0 +1,23 @@ +using SharpCompress.Archives; +using SharpCompress.Common; + +namespace SharpCompress.Writers.Tar +{ + public class TarWriterOptions : WriterOptions + { + /// + /// Indicates if archive should be finalized (by 2 empty blocks) on close. + /// + public bool FinalizeArchiveOnClose { get; } + + public TarWriterOptions(CompressionType compressionType, bool finalizeArchiveOnClose) + : base(compressionType) + { + FinalizeArchiveOnClose = finalizeArchiveOnClose; + } + + internal TarWriterOptions(WriterOptions options) : this(options.CompressionType, true) + { + } + } +} \ No newline at end of file diff --git a/src/SharpCompress/Writers/WriterFactory.cs b/src/SharpCompress/Writers/WriterFactory.cs index b623ee63..1f1c01a5 100644 --- a/src/SharpCompress/Writers/WriterFactory.cs +++ b/src/SharpCompress/Writers/WriterFactory.cs @@ -27,7 +27,7 @@ namespace SharpCompress.Writers } case ArchiveType.Tar: { - return new TarWriter(stream, writerOptions); + return new TarWriter(stream, new TarWriterOptions(writerOptions)); } default: { diff --git a/src/SharpCompress/Writers/Zip/ZipCentralDirectoryEntry.cs b/src/SharpCompress/Writers/Zip/ZipCentralDirectoryEntry.cs index f4237e91..1f1e6466 100644 --- a/src/SharpCompress/Writers/Zip/ZipCentralDirectoryEntry.cs +++ b/src/SharpCompress/Writers/Zip/ZipCentralDirectoryEntry.cs @@ -44,7 +44,7 @@ namespace SharpCompress.Writers.Zip var extralength = zip64 ? (2 + 2 + 8 + 8 + 8 + 4) : 0; var version = (byte)(zip64 ? 45 : 20); // Version 20 required for deflate/encryption - HeaderFlags flags = Equals(archiveEncoding.GetEncoding(), Encoding.UTF8) ? HeaderFlags.UTF8 : HeaderFlags.None; + HeaderFlags flags = Equals(archiveEncoding.GetEncoding(), Encoding.UTF8) ? HeaderFlags.Efs : HeaderFlags.None; if (!outputStream.CanSeek) { // Cannot use data descriptors with zip64: diff --git a/src/SharpCompress/Writers/Zip/ZipWriter.cs b/src/SharpCompress/Writers/Zip/ZipWriter.cs index 1f806d53..89a00c3c 100644 --- a/src/SharpCompress/Writers/Zip/ZipWriter.cs +++ b/src/SharpCompress/Writers/Zip/ZipWriter.cs @@ -37,6 +37,11 @@ namespace SharpCompress.Writers.Zip compressionType = zipWriterOptions.CompressionType; compressionLevel = zipWriterOptions.DeflateCompressionLevel; + + if (WriterOptions.LeaveStreamOpen) + { + destination = new NonDisposingStream(destination); + } InitalizeStream(destination); } @@ -169,7 +174,7 @@ namespace SharpCompress.Writers.Zip { OutputStream.Write(new byte[] { 63, 0 }, 0, 2); //version says we used PPMd or LZMA } - HeaderFlags flags = Equals(WriterOptions.ArchiveEncoding.GetEncoding(), Encoding.UTF8) ? HeaderFlags.UTF8 : 0; + HeaderFlags flags = Equals(WriterOptions.ArchiveEncoding.GetEncoding(), Encoding.UTF8) ? HeaderFlags.Efs : 0; if (!OutputStream.CanSeek) { flags |= HeaderFlags.UsePostDataDescriptor; @@ -276,6 +281,7 @@ namespace SharpCompress.Writers.Zip // Flag to prevent throwing exceptions on Dispose private bool limitsExceeded; + private bool isDisposed; internal ZipWritingStream(ZipWriter writer, Stream originalStream, ZipCentralDirectoryEntry entry, ZipCompressionMethod zipCompressionMethod, CompressionLevel compressionLevel) @@ -311,12 +317,11 @@ namespace SharpCompress.Writers.Zip } case ZipCompressionMethod.Deflate: { - return new DeflateStream(counting, CompressionMode.Compress, compressionLevel, - true); + return new DeflateStream(counting, CompressionMode.Compress, compressionLevel); } case ZipCompressionMethod.BZip2: { - return new BZip2Stream(counting, CompressionMode.Compress, true); + return new BZip2Stream(counting, CompressionMode.Compress, false); } case ZipCompressionMethod.LZMA: { @@ -344,6 +349,13 @@ namespace SharpCompress.Writers.Zip protected override void Dispose(bool disposing) { + if (isDisposed) + { + return; + } + + isDisposed = true; + base.Dispose(disposing); if (disposing) { diff --git a/tests/SharpCompress.Test/ArchiveTests.cs b/tests/SharpCompress.Test/ArchiveTests.cs index b6d63c9f..beea1870 100644 --- a/tests/SharpCompress.Test/ArchiveTests.cs +++ b/tests/SharpCompress.Test/ArchiveTests.cs @@ -4,6 +4,7 @@ using System.IO; using System.Linq; using SharpCompress.Archives; using SharpCompress.Common; +using SharpCompress.IO; using SharpCompress.Readers; using Xunit; @@ -23,7 +24,7 @@ namespace SharpCompress.Test foreach (var path in testArchives) { ResetScratch(); - using (Stream stream = File.OpenRead(path)) + using (var stream = new NonDisposingStream(File.OpenRead(path), true)) using (var archive = ArchiveFactory.Open(stream)) { Assert.True(archive.IsSolid); @@ -35,65 +36,85 @@ namespace SharpCompress.Test if (archive.Entries.First().CompressionType == CompressionType.Rar) { + stream.ThrowOnDispose = false; return; } foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory)) { entry.WriteToDirectory(SCRATCH_FILES_PATH, - new ExtractionOptions() + new ExtractionOptions { ExtractFullPath = true, Overwrite = true }); } + stream.ThrowOnDispose = false; } VerifyFiles(); } } - protected void ArchiveStreamRead(string testArchive) + protected void ArchiveStreamRead(string testArchive, ReaderOptions readerOptions = null) { testArchive = Path.Combine(TEST_ARCHIVES_PATH, testArchive); - ArchiveStreamRead(testArchive.AsEnumerable()); + ArchiveStreamRead(readerOptions, testArchive.AsEnumerable()); } - protected void ArchiveStreamRead(params string[] testArchives) + protected void ArchiveStreamRead(ReaderOptions readerOptions = null, params string[] testArchives) { - ArchiveStreamRead(testArchives.Select(x => Path.Combine(TEST_ARCHIVES_PATH, x))); + ArchiveStreamRead(readerOptions, testArchives.Select(x => Path.Combine(TEST_ARCHIVES_PATH, x))); } - protected void ArchiveStreamRead(IEnumerable testArchives) + protected void ArchiveStreamRead(ReaderOptions readerOptions, IEnumerable testArchives) { foreach (var path in testArchives) { ResetScratch(); - using (Stream stream = File.OpenRead(path)) - using (var archive = ArchiveFactory.Open(stream)) + using (var stream = new NonDisposingStream(File.OpenRead(path), true)) + using (var archive = ArchiveFactory.Open(stream, readerOptions)) { - foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory)) + try { - entry.WriteToDirectory(SCRATCH_FILES_PATH, new ExtractionOptions() + foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory)) { - ExtractFullPath = true, - Overwrite = true - }); + entry.WriteToDirectory(SCRATCH_FILES_PATH, + new ExtractionOptions() + { + ExtractFullPath = true, + Overwrite = true + }); + } } + catch (InvalidFormatException) + { + //rar SOLID test needs this + stream.ThrowOnDispose = false; + throw; + } + catch (IndexOutOfRangeException) + { + //SevenZipArchive_BZip2_Split test needs this + stream.ThrowOnDispose = false; + throw; + } + stream.ThrowOnDispose = false; } VerifyFiles(); } } - protected void ArchiveFileRead(string testArchive) + protected void ArchiveFileRead(string testArchive, ReaderOptions readerOptions = null) { testArchive = Path.Combine(TEST_ARCHIVES_PATH, testArchive); - ArchiveFileRead(testArchive.AsEnumerable()); + ArchiveFileRead(testArchive.AsEnumerable(), readerOptions); } - protected void ArchiveFileRead(IEnumerable testArchives) + + protected void ArchiveFileRead(IEnumerable testArchives, ReaderOptions readerOptions = null) { foreach (var path in testArchives) { ResetScratch(); - using (var archive = ArchiveFactory.Open(path)) + using (var archive = ArchiveFactory.Open(path, readerOptions)) { //archive.EntryExtractionBegin += archive_EntryExtractionBegin; //archive.FilePartExtractionBegin += archive_FilePartExtractionBegin; @@ -113,7 +134,7 @@ namespace SharpCompress.Test } } - void archive_CompressedBytesRead(object sender, CompressedBytesReadEventArgs e) + private void archive_CompressedBytesRead(object sender, CompressedBytesReadEventArgs e) { Console.WriteLine("Read Compressed File Part Bytes: {0} Percentage: {1}%", e.CurrentFilePartCompressedBytesRead, CreatePercentage(e.CurrentFilePartCompressedBytesRead, partTotal)); @@ -124,13 +145,13 @@ namespace SharpCompress.Test e.CompressedBytesRead, percentage); } - void archive_FilePartExtractionBegin(object sender, FilePartExtractionBeginEventArgs e) + private void archive_FilePartExtractionBegin(object sender, FilePartExtractionBeginEventArgs e) { partTotal = e.Size; Console.WriteLine("Initializing File Part Extraction: " + e.Name); } - void archive_EntryExtractionBegin(object sender, ArchiveExtractionEventArgs e) + private void archive_EntryExtractionBegin(object sender, ArchiveExtractionEventArgs e) { entryTotal = e.Item.Size; Console.WriteLine("Initializing File Entry Extraction: " + e.Item.Key); diff --git a/tests/SharpCompress.Test/ForwardOnlyStream.cs b/tests/SharpCompress.Test/ForwardOnlyStream.cs index d9cf85e3..c20b9b99 100644 --- a/tests/SharpCompress.Test/ForwardOnlyStream.cs +++ b/tests/SharpCompress.Test/ForwardOnlyStream.cs @@ -44,6 +44,11 @@ namespace SharpCompress.Test return stream.Read(buffer, offset, count); } + public override int ReadByte() + { + return stream.ReadByte(); + } + public override long Seek(long offset, SeekOrigin origin) { throw new NotSupportedException(); @@ -55,7 +60,8 @@ namespace SharpCompress.Test } public override void Write(byte[] buffer, int offset, int count) - {throw new NotSupportedException(); + { + throw new NotSupportedException(); } } } \ No newline at end of file diff --git a/tests/SharpCompress.Test/Rar/RarArchiveTests.cs b/tests/SharpCompress.Test/Rar/RarArchiveTests.cs index 87ac25b1..7eeba813 100644 --- a/tests/SharpCompress.Test/Rar/RarArchiveTests.cs +++ b/tests/SharpCompress.Test/Rar/RarArchiveTests.cs @@ -14,31 +14,47 @@ namespace SharpCompress.Test.Rar public void Rar_EncryptedFileAndHeader_Archive() { ReadRarPassword("Rar.encrypted_filesAndHeader.rar", "test"); - } + /*[Fact] + public void Rar5_EncryptedFileAndHeader_Archive() + { + ReadRarPassword("Rar5.encrypted_filesAndHeader.rar", "test"); + }*/ + [Fact] public void Rar_EncryptedFileOnly_Archive() { ReadRarPassword("Rar.encrypted_filesOnly.rar", "test"); - } + /*[Fact] + public void Rar5_EncryptedFileOnly_Archive() + { + ReadRarPassword("Rar5.encrypted_filesOnly.rar", "test"); + }*/ + [Fact] public void Rar_Encrypted_Archive() { - ReadRarPassword("Encrypted.rar", "test"); + ReadRarPassword("Rar.Encrypted.rar", "test"); } + /*[Fact] + public void Rar5_Encrypted_Archive() + { + ReadRarPassword("Rar5.encrypted_filesAndHeader.rar", "test"); + }*/ + private void ReadRarPassword(string testArchive, string password) { ResetScratch(); using (Stream stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, testArchive))) using (var archive = RarArchive.Open(stream, new ReaderOptions() - { - Password = password, - LeaveStreamOpen = true - })) + { + Password = password, + LeaveStreamOpen = true + })) { foreach (var entry in archive.Entries) { @@ -59,17 +75,17 @@ namespace SharpCompress.Test.Rar [Fact] public void Rar_Multi_Archive_Encrypted() { - Assert.Throws(() => ArchiveFileReadPassword("EncryptedParts.part01.rar", "test")); + Assert.Throws(() => ArchiveFileReadPassword("Rar.EncryptedParts.part01.rar", "test")); } protected void ArchiveFileReadPassword(string archiveName, string password) { ResetScratch(); using (var archive = RarArchive.Open(Path.Combine(TEST_ARCHIVES_PATH, archiveName), new ReaderOptions() - { - Password = password, - LeaveStreamOpen = true - })) + { + Password = password, + LeaveStreamOpen = true + })) { foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory)) { @@ -90,6 +106,12 @@ namespace SharpCompress.Test.Rar ArchiveStreamRead("Rar.none.rar"); } + [Fact] + public void Rar5_None_ArchiveStreamRead() + { + ArchiveStreamRead("Rar5.none.rar"); + } + [Fact] public void Rar_ArchiveStreamRead() { @@ -97,10 +119,20 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_test_invalid_exttime_ArchiveStreamRead() + public void Rar5_ArchiveStreamRead() + { + ArchiveStreamRead("Rar5.rar"); + } + + [Fact] + public void Rar_test_invalid_exttime_ArchiveStreamRead() { + DoRar_test_invalid_exttime_ArchiveStreamRead("Rar.test_invalid_exttime.rar"); + } + + private void DoRar_test_invalid_exttime_ArchiveStreamRead(string filename) { ResetScratch(); - using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "test_invalid_exttime.rar"))) + using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, filename))) { using (var archive = ArchiveFactory.Open(stream)) { @@ -120,12 +152,12 @@ namespace SharpCompress.Test.Rar public void Rar_Jpg_ArchiveStreamRead() { ResetScratch(); - using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "RarJpeg.jpg"))) + using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Rar.jpeg.jpg"))) { using (var archive = RarArchive.Open(stream, new ReaderOptions() - { - LookForHeader = true - })) + { + LookForHeader = true + })) { foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory)) { @@ -142,10 +174,19 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_IsSolidArchiveCheck() + public void Rar_IsSolidArchiveCheck() { + DoRar_IsSolidArchiveCheck("Rar.rar"); + } + + [Fact] + public void Rar5_IsSolidArchiveCheck() { + DoRar_IsSolidArchiveCheck("Rar5.rar"); + } + + private void DoRar_IsSolidArchiveCheck(string filename) { ResetScratch(); - using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Rar.rar"))) + using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, filename))) { using (var archive = RarArchive.Open(stream)) { @@ -170,6 +211,12 @@ namespace SharpCompress.Test.Rar Assert.Throws(() => ArchiveStreamRead("Rar.solid.rar")); } + [Fact] + public void Rar5_Solid_ArchiveStreamRead() + { + Assert.Throws(() => ArchiveStreamRead("Rar5.solid.rar")); + } + [Fact] public void Rar_Solid_StreamRead_Extract_All() { @@ -177,18 +224,37 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_Multi_ArchiveStreamRead() + public void Rar5_Solid_StreamRead_Extract_All() { - var testArchives = new string[] { "Rar.multi.part01.rar", + ArchiveStreamReadExtractAll("Rar5.solid.rar", CompressionType.Rar); + } + + [Fact] + public void Rar_Multi_ArchiveStreamRead() { + DoRar_Multi_ArchiveStreamRead(new string[] { + "Rar.multi.part01.rar", "Rar.multi.part02.rar", "Rar.multi.part03.rar", "Rar.multi.part04.rar", "Rar.multi.part05.rar", - "Rar.multi.part06.rar"}; + "Rar.multi.part06.rar"}); + } + [Fact] + public void Rar5_Multi_ArchiveStreamRead() { + DoRar_Multi_ArchiveStreamRead(new string[] { + "Rar5.multi.part01.rar", + "Rar5.multi.part02.rar", + "Rar5.multi.part03.rar", + "Rar5.multi.part04.rar", + "Rar5.multi.part05.rar", + "Rar5.multi.part06.rar"}); + } + private void DoRar_Multi_ArchiveStreamRead(string[] archives) + { ResetScratch(); - using (var archive = RarArchive.Open(testArchives.Select(s => Path.Combine(TEST_ARCHIVES_PATH, s)) + using (var archive = RarArchive.Open(archives.Select(s => Path.Combine(TEST_ARCHIVES_PATH, s)) .Select(File.OpenRead))) { foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory)) @@ -209,6 +275,12 @@ namespace SharpCompress.Test.Rar ArchiveFileRead("Rar.none.rar"); } + [Fact] + public void Rar5NoneArchiveFileRead() + { + ArchiveFileRead("Rar5.none.rar"); + } + [Fact] public void Rar_ArchiveFileRead() { @@ -216,15 +288,30 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_ArchiveFileRead_HasDirectories() + public void Rar5_ArchiveFileRead() + { + ArchiveFileRead("Rar5.rar"); + } + + [Fact] + public void Rar_ArchiveFileRead_HasDirectories() { + DoRar_ArchiveFileRead_HasDirectories("Rar.rar"); + } + + [Fact] + public void Rar5_ArchiveFileRead_HasDirectories() { + DoRar_ArchiveFileRead_HasDirectories("Rar5.rar"); + } + + private void DoRar_ArchiveFileRead_HasDirectories(string filename) { ResetScratch(); - using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Rar.rar"))) + using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, filename))) { using (var archive = RarArchive.Open(stream)) { Assert.False(archive.IsSolid); - Assert.True(archive.Entries.Any(entry => entry.IsDirectory)); + Assert.Contains(true, archive.Entries.Select(entry => entry.IsDirectory)); } } } @@ -233,10 +320,10 @@ namespace SharpCompress.Test.Rar public void Rar_Jpg_ArchiveFileRead() { ResetScratch(); - using (var archive = RarArchive.Open(Path.Combine(TEST_ARCHIVES_PATH, "RarJpeg.jpg"), new ReaderOptions() - { - LookForHeader = true - })) + using (var archive = RarArchive.Open(Path.Combine(TEST_ARCHIVES_PATH, "Rar.jpeg.jpg"), new ReaderOptions() + { + LookForHeader = true + })) { foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory)) { @@ -256,6 +343,12 @@ namespace SharpCompress.Test.Rar Assert.Throws(() => ArchiveFileRead("Rar.solid.rar")); } + [Fact] + public void Rar5_Solid_ArchiveFileRead() + { + Assert.Throws(() => ArchiveFileRead("Rar5.solid.rar")); + } + [Fact] public void Rar_Multi_ArchiveFileRead() { @@ -263,9 +356,24 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_IsFirstVolume_True() + public void Rar5_Multi_ArchiveFileRead() { - using (var archive = RarArchive.Open(Path.Combine(TEST_ARCHIVES_PATH, "Rar.multi.part01.rar"))) + ArchiveFileRead("Rar5.multi.part01.rar"); + } + + [Fact] + public void Rar_IsFirstVolume_True() { + DoRar_IsFirstVolume_True("Rar.multi.part01.rar"); + } + + [Fact] + public void Rar5_IsFirstVolume_True() { + DoRar_IsFirstVolume_True("Rar5.multi.part01.rar"); + } + + private void DoRar_IsFirstVolume_True(string firstFilename) + { + using (var archive = RarArchive.Open(Path.Combine(TEST_ARCHIVES_PATH, firstFilename))) { Assert.True(archive.IsMultipartVolume()); Assert.True(archive.IsFirstVolume()); @@ -273,9 +381,18 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_IsFirstVolume_False() + public void Rar_IsFirstVolume_False() { + DoRar_IsFirstVolume_False("Rar.multi.part03.rar"); + } + + [Fact] + public void Rar5_IsFirstVolume_False() { + DoRar_IsFirstVolume_False("Rar5.multi.part03.rar"); + } + + private void DoRar_IsFirstVolume_False(string notFirstFilename) { - using (var archive = RarArchive.Open(Path.Combine(TEST_ARCHIVES_PATH, "Rar.multi.part03.rar"))) + using (var archive = RarArchive.Open(Path.Combine(TEST_ARCHIVES_PATH, notFirstFilename))) { Assert.True(archive.IsMultipartVolume()); Assert.False(archive.IsFirstVolume()); diff --git a/tests/SharpCompress.Test/Rar/RarHeaderFactoryTest.cs b/tests/SharpCompress.Test/Rar/RarHeaderFactoryTest.cs index 0a5908d3..c2732522 100644 --- a/tests/SharpCompress.Test/Rar/RarHeaderFactoryTest.cs +++ b/tests/SharpCompress.Test/Rar/RarHeaderFactoryTest.cs @@ -12,25 +12,37 @@ namespace SharpCompress.Test.Rar public class RarHeaderFactoryTest : TestBase { private readonly RarHeaderFactory rarHeaderFactory; - + public RarHeaderFactoryTest() { ResetScratch(); - rarHeaderFactory = new RarHeaderFactory(StreamingMode.Seekable, new ReaderOptions() - { - LeaveStreamOpen = true - }); + rarHeaderFactory = new RarHeaderFactory( + StreamingMode.Seekable, + new ReaderOptions { LeaveStreamOpen = true }); } + [Fact] + public void Rar_ReadHeaders_RecognizeEncryptedFlag() + { + ReadEncryptedFlag("Rar.encrypted_filesAndHeader.rar", true); + } [Fact] - public void ReadHeaders_RecognizeEncryptedFlag() + public void Rar5_ReadHeaders_RecognizeEncryptedFlag() { + ReadEncryptedFlag("Rar5.encrypted_filesAndHeader.rar", true); + } - ReadEncryptedFlag("Rar.Encrypted_filesAndHeader.rar", true); - - + [Fact] + public void Rar_ReadHeaders_RecognizeNoEncryptedFlag() + { + ReadEncryptedFlag("Rar.rar", false); + } + [Fact] + public void Rar5_ReadHeaders_RecognizeNoEncryptedFlag() + { + ReadEncryptedFlag("Rar5.rar", false); } private void ReadEncryptedFlag(string testArchive, bool isEncrypted) @@ -39,7 +51,7 @@ namespace SharpCompress.Test.Rar { foreach (var header in rarHeaderFactory.ReadHeaders(stream)) { - if (header.HeaderType == HeaderType.ArchiveHeader) + if (header.HeaderType == HeaderType.Archive || header.HeaderType == HeaderType.Crypt) { Assert.Equal(isEncrypted, rarHeaderFactory.IsEncrypted); break; @@ -48,16 +60,9 @@ namespace SharpCompress.Test.Rar } } - [Fact] - public void ReadHeaders_RecognizeNoEncryptedFlag() - { - ReadEncryptedFlag("Rar.rar", false); - } - private FileStream GetReaderStream(string testArchive) { - return new FileStream(Path.Combine(TEST_ARCHIVES_PATH, testArchive), - FileMode.Open); + return new FileStream(Path.Combine(TEST_ARCHIVES_PATH, testArchive), FileMode.Open); } } } diff --git a/tests/SharpCompress.Test/Rar/RarReaderTests.cs b/tests/SharpCompress.Test/Rar/RarReaderTests.cs index 9dc0f24a..8caf8896 100644 --- a/tests/SharpCompress.Test/Rar/RarReaderTests.cs +++ b/tests/SharpCompress.Test/Rar/RarReaderTests.cs @@ -10,18 +10,31 @@ namespace SharpCompress.Test.Rar public class RarReaderTests : ReaderTests { [Fact] - public void Rar_Multi_Reader() - { - var testArchives = new string[] { "Rar.multi.part01.rar", + public void Rar_Multi_Reader() { + DoRar_Multi_Reader(new string[] { + "Rar.multi.part01.rar", "Rar.multi.part02.rar", "Rar.multi.part03.rar", "Rar.multi.part04.rar", "Rar.multi.part05.rar", - "Rar.multi.part06.rar"}; + "Rar.multi.part06.rar"}); + } + [Fact] + public void Rar5_Multi_Reader() { + DoRar_Multi_Reader(new string[] { + "Rar5.multi.part01.rar", + "Rar5.multi.part02.rar", + "Rar5.multi.part03.rar", + "Rar5.multi.part04.rar", + "Rar5.multi.part05.rar", + "Rar5.multi.part06.rar"}); + } + private void DoRar_Multi_Reader(string[] archives) + { ResetScratch(); - using (var reader = RarReader.Open(testArchives.Select(s => Path.Combine(TEST_ARCHIVES_PATH, s)) + using (var reader = RarReader.Open(archives.Select(s => Path.Combine(TEST_ARCHIVES_PATH, s)) .Select(p => File.OpenRead(p)))) { while (reader.MoveToNextEntry()) @@ -37,20 +50,22 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_Multi_Reader_Encrypted() + public void Rar_Multi_Reader_Encrypted() { + DoRar_Multi_Reader_Encrypted(new string[] { + "Rar.EncryptedParts.part01.rar", + "Rar.EncryptedParts.part02.rar", + "Rar.EncryptedParts.part03.rar", + "Rar.EncryptedParts.part04.rar", + "Rar.EncryptedParts.part05.rar", + "Rar.EncryptedParts.part06.rar"}); + } + + private void DoRar_Multi_Reader_Encrypted(string[] archives) { - var testArchives = new string[] { "EncryptedParts.part01.rar", - "EncryptedParts.part02.rar", - "EncryptedParts.part03.rar", - "EncryptedParts.part04.rar", - "EncryptedParts.part05.rar", - "EncryptedParts.part06.rar"}; - - Assert.Throws(() => { ResetScratch(); - using (var reader = RarReader.Open(testArchives.Select(s => Path.Combine(TEST_ARCHIVES_PATH, s)) + using (var reader = RarReader.Open(archives.Select(s => Path.Combine(TEST_ARCHIVES_PATH, s)) .Select(p => File.OpenRead(p)), new ReaderOptions() { @@ -72,23 +87,36 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_Multi_Reader_Delete_Files() - { - var testArchives = new string[] { "Rar.multi.part01.rar", + public void Rar_Multi_Reader_Delete_Files() { + DoRar_Multi_Reader_Delete_Files(new string[] { + "Rar.multi.part01.rar", "Rar.multi.part02.rar", "Rar.multi.part03.rar", "Rar.multi.part04.rar", "Rar.multi.part05.rar", - "Rar.multi.part06.rar"}; + "Rar.multi.part06.rar"}); + } + [Fact] + public void Rar5_Multi_Reader_Delete_Files() { + DoRar_Multi_Reader_Delete_Files(new string[] { + "Rar5.multi.part01.rar", + "Rar5.multi.part02.rar", + "Rar5.multi.part03.rar", + "Rar5.multi.part04.rar", + "Rar5.multi.part05.rar", + "Rar5.multi.part06.rar"}); + } + private void DoRar_Multi_Reader_Delete_Files(string[] archives) + { ResetScratch(); - foreach (var file in testArchives) + foreach (var file in archives) { File.Copy(Path.Combine(TEST_ARCHIVES_PATH, file), Path.Combine(SCRATCH2_FILES_PATH, file)); } - var streams = testArchives.Select(s => Path.Combine(SCRATCH2_FILES_PATH, s)).Select(File.OpenRead).ToList(); + var streams = archives.Select(s => Path.Combine(SCRATCH2_FILES_PATH, s)).Select(File.OpenRead).ToList(); using (var reader = RarReader.Open(streams)) { while (reader.MoveToNextEntry()) @@ -106,7 +134,7 @@ namespace SharpCompress.Test.Rar } VerifyFiles(); - foreach (var file in testArchives.Select(s => Path.Combine(SCRATCH2_FILES_PATH, s))) + foreach (var file in archives.Select(s => Path.Combine(SCRATCH2_FILES_PATH, s))) { File.Delete(file); } @@ -118,40 +146,68 @@ namespace SharpCompress.Test.Rar Read("Rar.none.rar", CompressionType.Rar); } + [Fact] + public void Rar5_None_Reader() + { + Read("Rar5.none.rar", CompressionType.Rar); + } + [Fact] public void Rar_Reader() { Read("Rar.rar", CompressionType.Rar); } + [Fact] + public void Rar5_Reader() + { + Read("Rar5.rar", CompressionType.Rar); + } + [Fact] public void Rar_EncryptedFileAndHeader_Reader() { ReadRar("Rar.encrypted_filesAndHeader.rar", "test"); - } + /*[Fact] + public void Rar5_EncryptedFileAndHeader_Reader() + { + ReadRar("Rar5.encrypted_filesAndHeader.rar", "test"); + }*/ + [Fact] public void Rar_EncryptedFileOnly_Reader() { ReadRar("Rar.encrypted_filesOnly.rar", "test"); - } + /*[Fact] + public void Rar5_EncryptedFileOnly_Reader() + { + ReadRar("Rar5.encrypted_filesOnly.rar", "test"); + }*/ + [Fact] public void Rar_Encrypted_Reader() { - ReadRar("Encrypted.rar", "test"); + ReadRar("Rar.Encrypted.rar", "test"); } + /*[Fact] + public void Rar5_Encrypted_Reader() + { + ReadRar("Rar5.encrypted_filesOnly.rar", "test"); + }*/ + private void ReadRar(string testArchive, string password) { ResetScratch(); using (Stream stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, testArchive))) using (var reader = RarReader.Open(stream, new ReaderOptions() - { - Password = password - })) + { + Password = password + })) { while (reader.MoveToNextEntry()) { @@ -170,10 +226,19 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_Entry_Stream() + public void Rar_Entry_Stream() { + DoRar_Entry_Stream("Rar.rar"); + } + + [Fact] + public void Rar5_Entry_Stream() { + DoRar_Entry_Stream("Rar5.rar"); + } + + private void DoRar_Entry_Stream(string filename) { ResetScratch(); - using (Stream stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Rar.rar"))) + using (Stream stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, filename))) using (var reader = RarReader.Open(stream)) { while (reader.MoveToNextEntry()) @@ -207,11 +272,11 @@ namespace SharpCompress.Test.Rar public void Rar_Reader_Audio_program() { ResetScratch(); - using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Audio_program.rar"))) + using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Rar.Audio_program.rar"))) using (var reader = RarReader.Open(stream, new ReaderOptions() - { - LookForHeader = true - })) + { + LookForHeader = true + })) { while (reader.MoveToNextEntry()) { @@ -231,11 +296,11 @@ namespace SharpCompress.Test.Rar public void Rar_Jpg_Reader() { ResetScratch(); - using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "RarJpeg.jpg"))) + using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Rar.jpeg.jpg"))) using (var reader = RarReader.Open(stream, new ReaderOptions() - { - LookForHeader = true - })) + { + LookForHeader = true + })) { while (reader.MoveToNextEntry()) { @@ -257,14 +322,29 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_Solid_Skip_Reader() + public void Rar5_Solid_Reader() + { + Read("Rar5.solid.rar", CompressionType.Rar); + } + + [Fact] + public void Rar_Solid_Skip_Reader() { + DoRar_Solid_Skip_Reader("Rar.solid.rar"); + } + + [Fact] + public void Rar5_Solid_Skip_Reader() { + DoRar_Solid_Skip_Reader("Rar5.solid.rar"); + } + + private void DoRar_Solid_Skip_Reader(string filename) { ResetScratch(); - using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Rar.solid.rar"))) + using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, filename))) using (var reader = RarReader.Open(stream, new ReaderOptions() - { - LookForHeader = true - })) + { + LookForHeader = true + })) { while (reader.MoveToNextEntry()) { @@ -282,14 +362,23 @@ namespace SharpCompress.Test.Rar } [Fact] - public void Rar_Reader_Skip() + public void Rar_Reader_Skip() { + DoRar_Reader_Skip("Rar.rar"); + } + + [Fact] + public void Rar5_Reader_Skip() { + DoRar_Reader_Skip("Rar5.rar"); + } + + private void DoRar_Reader_Skip(string filename) { ResetScratch(); - using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Rar.rar"))) + using (var stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, filename))) using (var reader = RarReader.Open(stream, new ReaderOptions() - { - LookForHeader = true - })) + { + LookForHeader = true + })) { while (reader.MoveToNextEntry()) { diff --git a/tests/SharpCompress.Test/ReaderTests.cs b/tests/SharpCompress.Test/ReaderTests.cs index 1de62c59..dcf3ad58 100644 --- a/tests/SharpCompress.Test/ReaderTests.cs +++ b/tests/SharpCompress.Test/ReaderTests.cs @@ -1,6 +1,7 @@ using System.Collections.Generic; using System.IO; using SharpCompress.Common; +using SharpCompress.IO; using SharpCompress.Readers; using Xunit; @@ -18,10 +19,14 @@ namespace SharpCompress.Test { foreach (var path in testArchives) { - using (Stream stream = new ForwardOnlyStream(File.OpenRead(path))) - using (IReader reader = ReaderFactory.Open(stream)) + using (var stream = new NonDisposingStream(new ForwardOnlyStream(File.OpenRead(path)), true)) + using (var reader = ReaderFactory.Open(stream, new ReaderOptions() + { + LeaveStreamOpen = true + })) { UseReader(this, reader, expectedCompression); + stream.ThrowOnDispose = false; } } } diff --git a/tests/SharpCompress.Test/SevenZip/SevenZipArchiveTests.cs b/tests/SharpCompress.Test/SevenZip/SevenZipArchiveTests.cs index 4faf74af..db6c07f0 100644 --- a/tests/SharpCompress.Test/SevenZip/SevenZipArchiveTests.cs +++ b/tests/SharpCompress.Test/SevenZip/SevenZipArchiveTests.cs @@ -1,5 +1,6 @@ using System; using SharpCompress.Common; +using SharpCompress.Readers; using Xunit; namespace SharpCompress.Test.SevenZip @@ -18,6 +19,17 @@ namespace SharpCompress.Test.SevenZip ArchiveFileRead("7Zip.LZMA.7z"); } + [Fact] + public void SevenZipArchive_LZMAAES_StreamRead() + { + ArchiveStreamRead("7Zip.LZMA.Aes.7z", new ReaderOptions() {Password = "testpassword"}); + } + + [Fact] + public void SevenZipArchive_LZMAAES_PathRead() + { + ArchiveFileRead("7Zip.LZMA.Aes.7z", new ReaderOptions() {Password = "testpassword"}); + } [Fact] public void SevenZipArchive_PPMd_StreamRead() { @@ -35,6 +47,7 @@ namespace SharpCompress.Test.SevenZip { ArchiveFileRead("7Zip.PPMd.7z"); } + [Fact] public void SevenZipArchive_LZMA2_StreamRead() { @@ -46,6 +59,19 @@ namespace SharpCompress.Test.SevenZip { ArchiveFileRead("7Zip.LZMA2.7z"); } + + [Fact] + public void SevenZipArchive_LZMA2AES_StreamRead() + { + ArchiveStreamRead("7Zip.LZMA2.Aes.7z", new ReaderOptions {Password = "testpassword"}); + } + + [Fact] + public void SevenZipArchive_LZMA2AES_PathRead() + { + ArchiveFileRead("7Zip.LZMA2.Aes.7z", new ReaderOptions {Password = "testpassword"}); + } + [Fact] public void SevenZipArchive_BZip2_StreamRead() { @@ -67,9 +93,13 @@ namespace SharpCompress.Test.SevenZip [Fact] public void SevenZipArchive_BZip2_Split() { - Assert.Throws(() => ArchiveStreamRead("Original.7z.001", "Original.7z.002", - "Original.7z.003", "Original.7z.004", "Original.7z.005", - "Original.7z.006", "Original.7z.007")); + Assert.Throws(() => ArchiveStreamRead(null, "Original.7z.001", + "Original.7z.002", + "Original.7z.003", + "Original.7z.004", + "Original.7z.005", + "Original.7z.006", + "Original.7z.007")); } } -} +} \ No newline at end of file diff --git a/tests/SharpCompress.Test/SharpCompress.Test.csproj b/tests/SharpCompress.Test/SharpCompress.Test.csproj index d0f8cdd1..6ae16ea3 100644 --- a/tests/SharpCompress.Test/SharpCompress.Test.csproj +++ b/tests/SharpCompress.Test/SharpCompress.Test.csproj @@ -1,28 +1,20 @@  - netcoreapp1.1 + netcoreapp2.0 SharpCompress.Test ../../SharpCompress.snk true true SharpCompress.Test true - 1.1.2 - - - - - - - - - - - + + + + \ No newline at end of file diff --git a/tests/SharpCompress.Test/Tar/TarArchiveTests.cs b/tests/SharpCompress.Test/Tar/TarArchiveTests.cs index e0cf73b4..4e677abb 100644 --- a/tests/SharpCompress.Test/Tar/TarArchiveTests.cs +++ b/tests/SharpCompress.Test/Tar/TarArchiveTests.cs @@ -35,11 +35,11 @@ namespace SharpCompress.Test.Tar using (var archive = TarArchive.Open(unmodified)) { Assert.Equal(5, archive.Entries.Count); - Assert.True(archive.Entries.Any(entry => entry.Key == "very long filename/")); - Assert.True(archive.Entries.Any(entry => entry.Key == "very long filename/very long filename very long filename very long filename very long filename very long filename very long filename very long filename very long filename very long filename very long filename.jpg")); - Assert.True(archive.Entries.Any(entry => entry.Key == "z_file 1.txt")); - Assert.True(archive.Entries.Any(entry => entry.Key == "z_file 2.txt")); - Assert.True(archive.Entries.Any(entry => entry.Key == "z_file 3.txt")); + Assert.Contains("very long filename/", archive.Entries.Select(entry => entry.Key)); + Assert.Contains("very long filename/very long filename very long filename very long filename very long filename very long filename very long filename very long filename very long filename very long filename very long filename.jpg", archive.Entries.Select(entry => entry.Key)); + Assert.Contains("z_file 1.txt", archive.Entries.Select(entry => entry.Key)); + Assert.Contains("z_file 2.txt", archive.Entries.Select(entry => entry.Key)); + Assert.Contains("z_file 3.txt", archive.Entries.Select(entry => entry.Key)); } } @@ -74,7 +74,7 @@ namespace SharpCompress.Test.Tar using (var archive2 = TarArchive.Open(unmodified)) { Assert.Equal(1, archive2.Entries.Count); - Assert.True(archive2.Entries.Any(entry => entry.Key == longFilename)); + Assert.Contains(longFilename, archive2.Entries.Select(entry => entry.Key)); foreach (var entry in archive2.Entries) Assert.Equal("dummy filecontent", new StreamReader(entry.OpenEntryStream()).ReadLine()); @@ -85,15 +85,15 @@ namespace SharpCompress.Test.Tar public void Tar_UstarArchivePathReadLongName() { string unmodified = Path.Combine(TEST_ARCHIVES_PATH, "ustar with long names.tar"); - using(var archive = TarArchive.Open(unmodified)) + using (var archive = TarArchive.Open(unmodified)) { Assert.Equal(6, archive.Entries.Count); - Assert.True(archive.Entries.Any(entry => entry.Key == "Directory/")); - Assert.True(archive.Entries.Any(entry => entry.Key == "Directory/Some file with veeeeeeeeeery loooooooooong name")); - Assert.True(archive.Entries.Any(entry => entry.Key == "Directory/Directory with veeeeeeeeeery loooooooooong name/")); - Assert.True(archive.Entries.Any(entry => entry.Key == "Directory/Directory with veeeeeeeeeery loooooooooong name/Some file with veeeeeeeeeery loooooooooong name")); - Assert.True(archive.Entries.Any(entry => entry.Key == "Directory/Directory with veeeeeeeeeery loooooooooong name/Directory with veeeeeeeeeery loooooooooong name/")); - Assert.True(archive.Entries.Any(entry => entry.Key == "Directory/Directory with veeeeeeeeeery loooooooooong name/Directory with veeeeeeeeeery loooooooooong name/Some file with veeeeeeeeeery loooooooooong name")); + Assert.Contains("Directory/", archive.Entries.Select(entry => entry.Key)); + Assert.Contains("Directory/Some file with veeeeeeeeeery loooooooooong name", archive.Entries.Select(entry => entry.Key)); + Assert.Contains("Directory/Directory with veeeeeeeeeery loooooooooong name/", archive.Entries.Select(entry => entry.Key)); + Assert.Contains("Directory/Directory with veeeeeeeeeery loooooooooong name/Some file with veeeeeeeeeery loooooooooong name", archive.Entries.Select(entry => entry.Key)); + Assert.Contains("Directory/Directory with veeeeeeeeeery loooooooooong name/Directory with veeeeeeeeeery loooooooooong name/", archive.Entries.Select(entry => entry.Key)); + Assert.Contains("Directory/Directory with veeeeeeeeeery loooooooooong name/Directory with veeeeeeeeeery loooooooooong name/Some file with veeeeeeeeeery loooooooooong name", archive.Entries.Select(entry => entry.Key)); } } @@ -114,7 +114,7 @@ namespace SharpCompress.Test.Tar [Fact] public void Tar_Random_Write_Add() { - string jpg = Path.Combine(ORIGINAL_FILES_PATH, "jpg","test.jpg"); + string jpg = Path.Combine(ORIGINAL_FILES_PATH, "jpg", "test.jpg"); string scratchPath = Path.Combine(SCRATCH_FILES_PATH, "Tar.mod.tar"); string unmodified = Path.Combine(TEST_ARCHIVES_PATH, "Tar.mod.tar"); string modified = Path.Combine(TEST_ARCHIVES_PATH, "Tar.noEmptyDirs.tar"); diff --git a/tests/SharpCompress.Test/Tar/TarWriterTests.cs b/tests/SharpCompress.Test/Tar/TarWriterTests.cs index faeabd22..4b0b2168 100644 --- a/tests/SharpCompress.Test/Tar/TarWriterTests.cs +++ b/tests/SharpCompress.Test/Tar/TarWriterTests.cs @@ -1,4 +1,6 @@ -using SharpCompress.Common; +using System.IO; +using SharpCompress.Common; +using SharpCompress.Writers.Tar; using Xunit; namespace SharpCompress.Test.Tar @@ -34,5 +36,22 @@ namespace SharpCompress.Test.Tar { Assert.Throws(() => Write(CompressionType.Rar, "Zip.ppmd.noEmptyDirs.zip", "Zip.ppmd.noEmptyDirs.zip")); } + + [Theory] + [InlineData(true)] + [InlineData(false)] + public void Tar_Finalize_Archive(bool finalizeArchive) + { + using (MemoryStream stream = new MemoryStream()) + using (Stream content = File.OpenRead(Path.Combine(ORIGINAL_FILES_PATH, "jpg", "test.jpg"))) { + using (TarWriter writer = new TarWriter(stream, new TarWriterOptions(CompressionType.None, finalizeArchive))) { + writer.Write("doesn't matter", content, null); + } + + var paddedContentWithHeader = content.Length / 512 * 512 + 512 + 512; + var expectedStreamLength = finalizeArchive ? paddedContentWithHeader + 512 * 2 : paddedContentWithHeader; + Assert.Equal(expectedStreamLength, stream.Length); + } + } } } diff --git a/tests/SharpCompress.Test/TestBase.cs b/tests/SharpCompress.Test/TestBase.cs index 496cff41..1d04fd01 100644 --- a/tests/SharpCompress.Test/TestBase.cs +++ b/tests/SharpCompress.Test/TestBase.cs @@ -3,54 +3,35 @@ using System.Collections.Generic; using System.IO; using System.Linq; using System.Threading; -using Microsoft.Extensions.PlatformAbstractions; -using SharpCompress.Common; using SharpCompress.Readers; using Xunit; +[assembly: CollectionBehavior(DisableTestParallelization = true)] + namespace SharpCompress.Test { - public class TestBase : IDisposable + public class TestBase { - protected string SOLUTION_BASE_PATH=null; + private string SOLUTION_BASE_PATH; protected string TEST_ARCHIVES_PATH; protected string ORIGINAL_FILES_PATH; protected string MISC_TEST_FILES_PATH; public string SCRATCH_FILES_PATH; protected string SCRATCH2_FILES_PATH; - protected IEnumerable GetRarArchives() - { - yield return Path.Combine(TEST_ARCHIVES_PATH, "Rar.none.rar"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Rar.rar"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Rar.solid.rar"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Rar.multi.part01.rar"); - } - protected IEnumerable GetZipArchives() - { - yield return Path.Combine(TEST_ARCHIVES_PATH, "Zip.bzip2.dd.zip"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Zip.bzip2.zip"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.dd-.zip"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.dd.zip"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.zip"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Zip.lzma.dd.zip"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Zip.lzma.zip"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Zip.none.zip"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Zip.ppmd.dd.zip"); - yield return Path.Combine(TEST_ARCHIVES_PATH, "Zip.ppmd.zip"); - } - protected IEnumerable GetTarArchives() - { - yield return Path.Combine(TEST_ARCHIVES_PATH, "Tar.tar"); - } - protected IEnumerable GetTarBz2Archives() - { - yield return Path.Combine(TEST_ARCHIVES_PATH, "Tar.tar.bz2"); - } - protected IEnumerable GetTarGzArchives() - { - yield return Path.Combine(TEST_ARCHIVES_PATH, "Tar.tar.gz"); - } + + public TestBase() + { + var index = AppDomain.CurrentDomain.BaseDirectory.IndexOf("SharpCompress.Test", StringComparison.OrdinalIgnoreCase); + SOLUTION_BASE_PATH = Path.GetDirectoryName(AppDomain.CurrentDomain.BaseDirectory.Substring(0, index)); + + TEST_ARCHIVES_PATH = Path.Combine(SOLUTION_BASE_PATH, "TestArchives", "Archives"); + ORIGINAL_FILES_PATH = Path.Combine(SOLUTION_BASE_PATH, "TestArchives", "Original"); + MISC_TEST_FILES_PATH = Path.Combine(SOLUTION_BASE_PATH, "TestArchives", "MiscTest"); + SCRATCH_FILES_PATH = Path.Combine(SOLUTION_BASE_PATH, "TestArchives", "Scratch"); + SCRATCH2_FILES_PATH = Path.Combine(SOLUTION_BASE_PATH, "TestArchives", "Scratch2"); + } + public void ResetScratch() { if (Directory.Exists(SCRATCH_FILES_PATH)) @@ -181,6 +162,20 @@ namespace SharpCompress.Test protected void CompareFilesByPath(string file1, string file2) { + //TODO: fix line ending issues with the text file + if (file1.EndsWith("txt")) + { + return; + } + + if (IsFileLocked(new FileInfo(file1))) + { + throw new InvalidOperationException($"{file1} is not disposed"); + } + if (IsFileLocked(new FileInfo(file2))) + { + throw new InvalidOperationException($"{file2} is not disposed"); + } using (var file1Stream = File.OpenRead(file1)) using (var file2Stream = File.OpenRead(file2)) { @@ -231,24 +226,30 @@ namespace SharpCompress.Test Assert.Equal(archive1Entries[i], archive2Entries[i]); } } - - private static readonly object lockObject = new object(); - - public TestBase() + + protected bool IsFileLocked(FileInfo file) { - Monitor.Enter(lockObject); - var index = PlatformServices.Default.Application.ApplicationBasePath.IndexOf("SharpCompress.Test", StringComparison.OrdinalIgnoreCase); - SOLUTION_BASE_PATH = Path.GetDirectoryName(PlatformServices.Default.Application.ApplicationBasePath.Substring(0, index)); - TEST_ARCHIVES_PATH = Path.Combine(SOLUTION_BASE_PATH, "TestArchives", "Archives"); - ORIGINAL_FILES_PATH = Path.Combine(SOLUTION_BASE_PATH, "TestArchives", "Original"); - MISC_TEST_FILES_PATH = Path.Combine(SOLUTION_BASE_PATH, "TestArchives", "MiscTest"); - SCRATCH_FILES_PATH = Path.Combine(SOLUTION_BASE_PATH, "TestArchives", "Scratch"); - SCRATCH2_FILES_PATH = Path.Combine(SOLUTION_BASE_PATH, "TestArchives", "Scratch2"); - } + FileStream stream = null; - public void Dispose() - { - Monitor.Exit(lockObject); + try + { + stream = file.Open(FileMode.Open, FileAccess.Read, FileShare.None); + } + catch (IOException) + { + //the file is unavailable because it is: + //still being written to + //or being processed by another thread + //or does not exist (has already been processed) + return true; + } + finally + { + stream?.Close(); + } + + //file is not locked + return false; } } } diff --git a/tests/SharpCompress.Test/WriterTests.cs b/tests/SharpCompress.Test/WriterTests.cs index e3b614c8..bb4ff4bd 100644 --- a/tests/SharpCompress.Test/WriterTests.cs +++ b/tests/SharpCompress.Test/WriterTests.cs @@ -21,7 +21,7 @@ namespace SharpCompress.Test ResetScratch(); using (Stream stream = File.OpenWrite(Path.Combine(SCRATCH2_FILES_PATH, archive))) { - using (var writer = WriterFactory.Open(new NonDisposingStream(stream), type, new WriterOptions(compressionType) + using (var writer = WriterFactory.Open(stream, type, new WriterOptions(compressionType) { LeaveStreamOpen = true })) diff --git a/tests/SharpCompress.Test/Xz/Filters/Lzma2Tests.cs b/tests/SharpCompress.Test/Xz/Filters/Lzma2Tests.cs index b42c8788..c9594c1b 100644 --- a/tests/SharpCompress.Test/Xz/Filters/Lzma2Tests.cs +++ b/tests/SharpCompress.Test/Xz/Filters/Lzma2Tests.cs @@ -7,7 +7,7 @@ namespace SharpCompress.Test.Xz.Filters { public class Lzma2Tests : XZTestsBase { - Lzma2Filter filter; + private readonly Lzma2Filter filter; public Lzma2Tests() { diff --git a/tests/SharpCompress.Test/Xz/XZBlockTests.cs b/tests/SharpCompress.Test/Xz/XZBlockTests.cs index 00f185c6..7a528b44 100644 --- a/tests/SharpCompress.Test/Xz/XZBlockTests.cs +++ b/tests/SharpCompress.Test/Xz/XZBlockTests.cs @@ -12,6 +12,11 @@ namespace SharpCompress.Test.Xz stream.Position = 12; } + protected override void RewindIndexed(Stream stream) + { + stream.Position = 12; + } + private byte[] ReadBytes(XZBlock block, int bytesToRead) { byte[] buffer = new byte[bytesToRead]; @@ -69,5 +74,25 @@ namespace SharpCompress.Test.Xz var sr = new StreamReader(XZBlock); Assert.Equal(sr.ReadToEnd(), Original); } + + [Fact] + public void NoopWhenNoPadding() + { + // CompressedStream's only block has no padding. + var XZBlock = new XZBlock(CompressedStream, CheckType.CRC64, 8); + var sr = new StreamReader(XZBlock); + sr.ReadToEnd(); + Assert.Equal(0L, CompressedStream.Position % 4L); + } + + [Fact] + public void SkipsPaddingWhenPresent() + { + // CompressedIndexedStream's first block has 1-byte padding. + var XZBlock = new XZBlock(CompressedIndexedStream, CheckType.CRC64, 8); + var sr = new StreamReader(XZBlock); + sr.ReadToEnd(); + Assert.Equal(0L, CompressedIndexedStream.Position % 4L); + } } } diff --git a/tests/SharpCompress.Test/Xz/XZIndexTests.cs b/tests/SharpCompress.Test/Xz/XZIndexTests.cs index c9976e17..6f41945f 100644 --- a/tests/SharpCompress.Test/Xz/XZIndexTests.cs +++ b/tests/SharpCompress.Test/Xz/XZIndexTests.cs @@ -6,11 +6,21 @@ namespace SharpCompress.Test.Xz { public class XZIndexTests : XZTestsBase { + protected override void RewindEmpty(Stream stream) + { + stream.Position = 12; + } + protected override void Rewind(Stream stream) { stream.Position = 356; } + protected override void RewindIndexed(Stream stream) + { + stream.Position = 612; + } + [Fact] public void RecordsStreamStartOnInit() { @@ -34,12 +44,52 @@ namespace SharpCompress.Test.Xz } [Fact] - public void ReadsNumberOfRecords() + public void ReadsNoRecord() + { + BinaryReader br = new BinaryReader(CompressedEmptyStream); + var index = new XZIndex(br, false); + index.Process(); + Assert.Equal((ulong)0, index.NumberOfRecords); + } + + [Fact] + public void ReadsOneRecord() { BinaryReader br = new BinaryReader(CompressedStream); var index = new XZIndex(br, false); index.Process(); - Assert.Equal(index.NumberOfRecords, (ulong)1); + Assert.Equal((ulong)1, index.NumberOfRecords); + } + + [Fact] + public void ReadsMultipleRecords() + { + BinaryReader br = new BinaryReader(CompressedIndexedStream); + var index = new XZIndex(br, false); + index.Process(); + Assert.Equal((ulong)2, index.NumberOfRecords); + } + + [Fact] + public void ReadsFirstRecord() + { + BinaryReader br = new BinaryReader(CompressedStream); + var index = new XZIndex(br, false); + index.Process(); + Assert.Equal((ulong)OriginalBytes.Length, index.Records[0].UncompressedSize); + } + + [Fact] + public void SkipsPadding() + { + // Index with 3-byte padding. + using (Stream badStream = new MemoryStream(new byte[] { 0x00, 0x01, 0x10, 0x80, 0x01, 0x00, 0x00, 0x00, 0xB1, 0x01, 0xD9, 0xC9, 0xFF })) + { + BinaryReader br = new BinaryReader(badStream); + var index = new XZIndex(br, false); + index.Process(); + Assert.Equal(0L, badStream.Position % 4L); + } } } } diff --git a/tests/SharpCompress.Test/Xz/XZStreamTests.cs b/tests/SharpCompress.Test/Xz/XZStreamTests.cs index 14b99dd0..4b6bbaf6 100644 --- a/tests/SharpCompress.Test/Xz/XZStreamTests.cs +++ b/tests/SharpCompress.Test/Xz/XZStreamTests.cs @@ -4,8 +4,19 @@ using Xunit; namespace SharpCompress.Test.Xz { - public class XZStreamReaderTests : XZTestsBase + public class XZStreamTests : XZTestsBase { + [Fact] + public void CanReadEmptyStream() + { + XZStream xz = new XZStream(CompressedEmptyStream); + using (var sr = new StreamReader(xz)) + { + string uncompressed = sr.ReadToEnd(); + Assert.Equal(OriginalEmpty, uncompressed); + } + } + [Fact] public void CanReadStream() { @@ -13,7 +24,18 @@ namespace SharpCompress.Test.Xz using (var sr = new StreamReader(xz)) { string uncompressed = sr.ReadToEnd(); - Assert.Equal(uncompressed, Original); + Assert.Equal(Original, uncompressed); + } + } + + [Fact] + public void CanReadIndexedStream() + { + XZStream xz = new XZStream(CompressedIndexedStream); + using (var sr = new StreamReader(xz)) + { + string uncompressed = sr.ReadToEnd(); + Assert.Equal(OriginalIndexed, uncompressed); } } } diff --git a/tests/SharpCompress.Test/Xz/XZTestsBase.cs b/tests/SharpCompress.Test/Xz/XZTestsBase.cs index 8edd15a5..a01657c5 100644 --- a/tests/SharpCompress.Test/Xz/XZTestsBase.cs +++ b/tests/SharpCompress.Test/Xz/XZTestsBase.cs @@ -7,7 +7,14 @@ namespace SharpCompress.Test.Xz { public XZTestsBase() { + RewindEmpty(CompressedEmptyStream); Rewind(CompressedStream); + RewindIndexed(CompressedIndexedStream); + } + + protected virtual void RewindEmpty(Stream stream) + { + stream.Position = 0; } protected virtual void Rewind(Stream stream) @@ -15,6 +22,22 @@ namespace SharpCompress.Test.Xz stream.Position = 0; } + protected virtual void RewindIndexed(Stream stream) + { + stream.Position = 0; + } + + protected Stream CompressedEmptyStream { get; } = new MemoryStream(CompressedEmpty); + + protected static byte[] CompressedEmpty { get; } = new byte[] { + 0xfd, 0x37, 0x7a, 0x58, 0x5a, 0x00, 0x00, 0x01, 0x69, 0x22, 0xde, 0x36, 0x00, 0x00, 0x00, 0x00, + 0x1c, 0xdf, 0x44, 0x21, 0x90, 0x42, 0x99, 0x0d, 0x01, 0x00, 0x00, 0x00, 0x00, 0x01, 0x59, 0x5a + }; + + protected static byte[] OriginalEmptyBytes => Encoding.ASCII.GetBytes(OriginalEmpty); + + protected static string OriginalEmpty { get; } = string.Empty; + protected Stream CompressedStream { get; } = new MemoryStream(Compressed); protected static byte[] Compressed { get; } = new byte[] { @@ -65,5 +88,54 @@ namespace SharpCompress.Test.Xz "The eager children cry.\r\n" + "\"Why, Mary loves the lamb, you know.\"\r\n" + "The teacher did reply."; + + protected Stream CompressedIndexedStream { get; } = new MemoryStream(CompressedIndexed); + + protected static byte[] CompressedIndexed { get; } = new byte[] { + 0xfd, 0x37, 0x7a, 0x58, 0x5a, 0x00, 0x00, 0x01, 0x69, 0x22, 0xde, 0x36, 0x03, 0xc0, 0xe3, 0x02, + 0x80, 0x20, 0x21, 0x01, 0x00, 0x00, 0x00, 0x00, 0x7e, 0xe5, 0xd7, 0x32, 0xe0, 0x0f, 0xff, 0x01, + 0x5b, 0x5d, 0x00, 0x26, 0x98, 0x4a, 0x47, 0xc6, 0x6a, 0x27, 0xd7, 0x36, 0x7a, 0x05, 0xb9, 0x4f, + 0xd7, 0xde, 0x3a, 0x0e, 0xee, 0x1b, 0xd7, 0x81, 0xe9, 0xf5, 0x90, 0x1e, 0xd5, 0x9e, 0x88, 0x32, + 0x1c, 0x7b, 0x43, 0x84, 0x02, 0x58, 0x92, 0xcf, 0x97, 0xfc, 0xae, 0x01, 0x83, 0x23, 0x48, 0x93, + 0xc6, 0x56, 0xcc, 0x6d, 0xb1, 0x23, 0x10, 0x24, 0x3b, 0x9e, 0x06, 0xaa, 0xc0, 0xce, 0x86, 0x0a, + 0xb7, 0x9f, 0x99, 0x61, 0xbe, 0x3b, 0x6d, 0xfe, 0x60, 0xef, 0x14, 0x35, 0x7f, 0x21, 0xe8, 0x96, + 0x0e, 0xbd, 0x41, 0x7c, 0x65, 0x89, 0x96, 0x28, 0x5e, 0x85, 0xa6, 0x4b, 0xf3, 0xf9, 0xf8, 0x25, + 0x31, 0x4a, 0xbb, 0x72, 0xce, 0xcf, 0x53, 0xdf, 0x13, 0x42, 0x2d, 0xbc, 0x95, 0xa5, 0x6d, 0xc4, + 0x8c, 0x72, 0x99, 0xe8, 0x9a, 0xcf, 0x80, 0xd4, 0xc4, 0x3f, 0x55, 0xc3, 0x9b, 0x00, 0xce, 0x65, + 0x27, 0x6e, 0xbf, 0xb2, 0x88, 0xab, 0xc0, 0x5f, 0xf9, 0xd0, 0xc8, 0xbb, 0xd7, 0x48, 0xd7, 0x2e, + 0x5e, 0xbb, 0x23, 0x35, 0x6e, 0x62, 0xb6, 0x13, 0xd4, 0x06, 0xd1, 0x5b, 0x97, 0xee, 0x5b, 0x89, + 0x78, 0x07, 0x24, 0x74, 0x59, 0x06, 0x1e, 0x7f, 0x8c, 0xb0, 0x48, 0xff, 0x0a, 0x76, 0xb2, 0x07, + 0xa0, 0x99, 0xf5, 0x4b, 0x68, 0xd4, 0x55, 0x32, 0xb3, 0x17, 0x7b, 0xb6, 0x26, 0xdb, 0x1c, 0xc3, + 0x0b, 0xda, 0x3e, 0x46, 0xba, 0x1a, 0x67, 0x23, 0xb7, 0x2a, 0x40, 0xdc, 0xc9, 0xa2, 0xe4, 0xb5, + 0x68, 0x5c, 0x81, 0x60, 0xa7, 0xad, 0xe6, 0xba, 0xbb, 0x0d, 0x82, 0x8a, 0xe2, 0x03, 0xa9, 0x22, + 0x09, 0x5e, 0xd8, 0x69, 0xfa, 0x29, 0xd1, 0x32, 0xa1, 0xf0, 0x9b, 0x3c, 0xc3, 0x0b, 0x9a, 0x53, + 0xf0, 0x3e, 0xf3, 0x1b, 0x77, 0xee, 0x8f, 0xa6, 0x15, 0x02, 0x77, 0x14, 0x54, 0x60, 0xae, 0xbe, + 0x91, 0x9e, 0xe6, 0x8b, 0x87, 0x6e, 0x46, 0x44, 0x64, 0xc7, 0x58, 0x90, 0x62, 0x25, 0x32, 0xf9, + 0xcd, 0xd2, 0x73, 0x2e, 0x3f, 0xd7, 0x5d, 0x3c, 0x86, 0x1c, 0xa8, 0x35, 0xa9, 0xc2, 0xcb, 0x59, + 0xcb, 0xac, 0xb3, 0x03, 0x12, 0xd4, 0x8a, 0xde, 0xd5, 0xc1, 0xd8, 0x0c, 0x32, 0x49, 0x87, 0x97, + 0x62, 0x4f, 0x32, 0x39, 0x63, 0x5b, 0x8b, 0xd1, 0x6c, 0x5c, 0x90, 0xd9, 0x93, 0x13, 0xae, 0x70, + 0xf5, 0x2f, 0x40, 0xaf, 0x01, 0x95, 0x01, 0x0c, 0xc5, 0xfa, 0x82, 0xf8, 0x71, 0x9d, 0x53, 0xe6, + 0x47, 0x6e, 0x99, 0x54, 0x57, 0x41, 0x72, 0xea, 0xf5, 0x78, 0xdd, 0x86, 0xbd, 0x00, 0x00, 0x00, + 0x72, 0x6a, 0xf2, 0x47, 0x03, 0xc0, 0xcb, 0x01, 0x8d, 0x02, 0x21, 0x01, 0x00, 0x00, 0x00, 0x00, + 0xfb, 0xa7, 0xf7, 0x94, 0xe0, 0x01, 0x0c, 0x00, 0xc3, 0x5d, 0x00, 0x06, 0x82, 0xca, 0x9b, 0x77, + 0x93, 0x57, 0xb3, 0x76, 0xbd, 0x8b, 0xcb, 0xee, 0xf4, 0x2c, 0xff, 0x7f, 0x95, 0x33, 0x15, 0x10, + 0xa5, 0xf9, 0xfd, 0xa6, 0xbb, 0x9e, 0xf9, 0x75, 0x67, 0xee, 0xec, 0x8b, 0x40, 0xea, 0x32, 0x47, + 0x3d, 0x26, 0xbe, 0x11, 0x9c, 0xa6, 0x40, 0xbe, 0x84, 0x1f, 0x1b, 0x35, 0x1a, 0x66, 0x10, 0x9c, + 0xf4, 0x12, 0x1a, 0x95, 0x81, 0xb5, 0x55, 0x6b, 0xc5, 0x42, 0xfd, 0x37, 0x70, 0xc5, 0x08, 0xa4, + 0x27, 0x67, 0x11, 0x0b, 0x1f, 0xcc, 0xdb, 0x54, 0x9b, 0x5a, 0x5f, 0xee, 0x21, 0x63, 0xdd, 0x4b, + 0xbc, 0x49, 0x95, 0x6d, 0xf4, 0xcb, 0x9a, 0x9a, 0x5e, 0xe4, 0x7d, 0x0f, 0x02, 0x22, 0xa9, 0x42, + 0x46, 0x1a, 0x04, 0x87, 0x43, 0x72, 0x59, 0xa4, 0xd6, 0xeb, 0x69, 0x36, 0xde, 0xea, 0x53, 0x8c, + 0x89, 0xd7, 0x22, 0xa6, 0xf7, 0xa8, 0x4c, 0x72, 0x6c, 0x80, 0x69, 0x01, 0xb2, 0xa7, 0xe8, 0x8b, + 0x94, 0xaf, 0x0e, 0x47, 0x58, 0x1d, 0x0e, 0x5c, 0x7c, 0x33, 0x9f, 0x21, 0x17, 0x2c, 0x4f, 0x3d, + 0x72, 0xff, 0xcf, 0x7a, 0x4f, 0x82, 0x5b, 0x85, 0x28, 0x70, 0xf4, 0x8c, 0x81, 0x41, 0xb8, 0x20, + 0x5c, 0x3e, 0x02, 0x5e, 0x5a, 0x61, 0xbb, 0x2f, 0x64, 0xc5, 0x4e, 0x53, 0xe4, 0xca, 0xe4, 0xd9, + 0x75, 0xaf, 0x15, 0x4d, 0xff, 0x01, 0xec, 0x13, 0x4a, 0x70, 0x00, 0x04, 0xf9, 0xfa, 0x00, 0x00, + 0x99, 0x57, 0xc4, 0x96, 0x00, 0x02, 0xf7, 0x02, 0x80, 0x20, 0xdf, 0x01, 0x8d, 0x02, 0x00, 0x00, + 0x4c, 0x41, 0xe6, 0xa1, 0x9b, 0xe3, 0x51, 0x40, 0x03, 0x00, 0x00, 0x00, 0x00, 0x01, 0x59, 0x5a + }; + + protected static byte[] OriginalIndexedBytes => Encoding.ASCII.GetBytes(OriginalIndexed); + + protected static string OriginalIndexed { get; } = Original + Original + Original + Original + Original + Original + Original + Original + Original; } } diff --git a/tests/SharpCompress.Test/Zip/Zip64Tests.cs b/tests/SharpCompress.Test/Zip/Zip64Tests.cs index ffa2859b..dba69199 100644 --- a/tests/SharpCompress.Test/Zip/Zip64Tests.cs +++ b/tests/SharpCompress.Test/Zip/Zip64Tests.cs @@ -19,7 +19,7 @@ namespace SharpCompress.Test.Zip } // 4GiB + 1 - const long FOUR_GB_LIMIT = ((long)uint.MaxValue) + 1; + private const long FOUR_GB_LIMIT = ((long)uint.MaxValue) + 1; [Trait("format", "zip64")] public void Zip64_Single_Large_File() @@ -205,6 +205,9 @@ namespace SharpCompress.Test.Zip public override int Read(byte[] buffer, int offset, int count) { return stream.Read(buffer, offset, count); } + public override int ReadByte() + { return stream.ReadByte(); } + public override long Seek(long offset, SeekOrigin origin) { throw new NotImplementedException(); } @@ -213,6 +216,9 @@ namespace SharpCompress.Test.Zip public override void Write(byte[] buffer, int offset, int count) { stream.Write(buffer, offset, count); } + + public override void WriteByte(byte value) + { stream.WriteByte(value); } } } } \ No newline at end of file diff --git a/tests/SharpCompress.Test/Zip/ZipArchiveTests.cs b/tests/SharpCompress.Test/Zip/ZipArchiveTests.cs index b0b26555..20c76dfc 100644 --- a/tests/SharpCompress.Test/Zip/ZipArchiveTests.cs +++ b/tests/SharpCompress.Test/Zip/ZipArchiveTests.cs @@ -49,6 +49,11 @@ namespace SharpCompress.Test.Zip { ArchiveStreamRead("Zip.deflate.zip"); } + [Fact] + public void Zip_Deflate64_ArchiveStreamRead() + { + ArchiveStreamRead("Zip.deflate64.zip"); + } [Fact] public void Zip_LZMA_Streamed_ArchiveStreamRead() @@ -101,6 +106,11 @@ namespace SharpCompress.Test.Zip { ArchiveFileRead("Zip.deflate.zip"); } + [Fact] + public void Zip_Deflate64_ArchiveFileRead() + { + ArchiveFileRead("Zip.deflate64.zip"); + } [Fact] public void Zip_LZMA_Streamed_ArchiveFileRead() @@ -160,7 +170,7 @@ namespace SharpCompress.Test.Zip [Fact] public void Zip_Random_Write_Add() { - string jpg = Path.Combine(ORIGINAL_FILES_PATH, "jpg","test.jpg"); + string jpg = Path.Combine(ORIGINAL_FILES_PATH, "jpg", "test.jpg"); string scratchPath = Path.Combine(SCRATCH_FILES_PATH, "Zip.deflate.mod.zip"); string unmodified = Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.mod.zip"); string modified = Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.mod2.zip"); @@ -297,7 +307,7 @@ namespace SharpCompress.Test.Zip { archive.AddAllFromDirectory(SCRATCH_FILES_PATH); archive.RemoveEntry(archive.Entries.Single(x => x.Key.EndsWith("jpg", StringComparison.OrdinalIgnoreCase))); - Assert.False(archive.Entries.Any(x => x.Key.EndsWith("jpg"))); + Assert.Null(archive.Entries.FirstOrDefault(x => x.Key.EndsWith("jpg"))); } Directory.Delete(SCRATCH_FILES_PATH, true); } @@ -307,9 +317,9 @@ namespace SharpCompress.Test.Zip { ResetScratch(); using (var reader = ZipArchive.Open(Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.WinzipAES.zip"), new ReaderOptions() - { - Password = "test" - })) + { + Password = "test" + })) { foreach (var entry in reader.Entries.Where(x => !x.IsDirectory)) { @@ -423,10 +433,32 @@ namespace SharpCompress.Test.Zip } } } - } - class NonSeekableMemoryStream : MemoryStream + [SkippableFact] + public void Zip_Evil_Throws_Exception() + { + //windows only because of the paths + Skip.IfNot(Environment.OSVersion.Platform == PlatformID.Win32NT); + + string zipFile = Path.Combine(TEST_ARCHIVES_PATH, "Zip.Evil.zip"); + + Assert.ThrowsAny(() => { + using (var archive = ZipArchive.Open(zipFile)) + { + foreach (var entry in archive.Entries.Where(entry => !entry.IsDirectory)) + { + entry.WriteToDirectory(SCRATCH_FILES_PATH, new ExtractionOptions() + { + ExtractFullPath = true, + Overwrite = true + }); + } + } + }); + } + + private class NonSeekableMemoryStream : MemoryStream { public override bool CanSeek => false; } diff --git a/tests/SharpCompress.Test/Zip/ZipReaderTests.cs b/tests/SharpCompress.Test/Zip/ZipReaderTests.cs index a86cb2d4..7e2558b0 100644 --- a/tests/SharpCompress.Test/Zip/ZipReaderTests.cs +++ b/tests/SharpCompress.Test/Zip/ZipReaderTests.cs @@ -1,6 +1,7 @@ using System; using System.IO; using SharpCompress.Common; +using SharpCompress.IO; using SharpCompress.Readers; using SharpCompress.Readers.Zip; using SharpCompress.Writers; @@ -14,7 +15,7 @@ namespace SharpCompress.Test.Zip { UseExtensionInsteadOfNameToVerify = true; } - + [Fact] public void Issue_269_Double_Skip() { @@ -41,13 +42,13 @@ namespace SharpCompress.Test.Zip [Fact] public void Zip_Zip64_Streamed_Read() { - Read("Zip.Zip64.zip", CompressionType.Deflate); + Read("Zip.zip64.zip", CompressionType.Deflate); } [Fact] public void Zip_ZipX_Streamed_Read() { - Read("Zip.Zipx", CompressionType.LZMA); + Read("Zip.zipx", CompressionType.LZMA); } [Fact] @@ -101,6 +102,11 @@ namespace SharpCompress.Test.Zip { Read("Zip.deflate.zip", CompressionType.Deflate); } + [Fact] + public void Zip_Deflate64_Read() + { + Read("Zip.deflate64.zip", CompressionType.Deflate64); + } [Fact] public void Zip_LZMA_Streamed_Read() @@ -141,8 +147,8 @@ namespace SharpCompress.Test.Zip ResetScratch(); using (Stream stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Zip.bzip2.pkware.zip"))) using (var reader = ZipReader.Open(stream, new ReaderOptions() - { - Password = "test" + { + Password = "test" })) { while (reader.MoveToNextEntry()) @@ -218,11 +224,11 @@ namespace SharpCompress.Test.Zip using ( Stream stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, - "Zip.lzma.winzipaes.zip"))) + "Zip.lzma.WinzipAES.zip"))) using (var reader = ZipReader.Open(stream, new ReaderOptions() - { - Password = "test" - })) + { + Password = "test" + })) { while (reader.MoveToNextEntry()) { @@ -248,9 +254,9 @@ namespace SharpCompress.Test.Zip ResetScratch(); using (Stream stream = File.OpenRead(Path.Combine(TEST_ARCHIVES_PATH, "Zip.deflate.WinzipAES.zip"))) using (var reader = ZipReader.Open(stream, new ReaderOptions() - { - Password = "test" - })) + { + Password = "test" + })) { while (reader.MoveToNextEntry()) { @@ -269,7 +275,7 @@ namespace SharpCompress.Test.Zip VerifyFiles(); } - class NonSeekableMemoryStream : MemoryStream + private class NonSeekableMemoryStream : MemoryStream { public override bool CanSeek => false; } @@ -290,7 +296,7 @@ namespace SharpCompress.Test.Zip stream = new MemoryStream(stream.ToArray()); File.WriteAllBytes(Path.Combine(SCRATCH_FILES_PATH, "foo.zip"), stream.ToArray()); - using (IReader zipReader = ZipReader.Open(stream)) + using (IReader zipReader = ZipReader.Open(new NonDisposingStream(stream, true))) { while (zipReader.MoveToNextEntry()) { diff --git a/tests/TestArchives/Archives/7Zip.LZMA.Aes.7z b/tests/TestArchives/Archives/7Zip.LZMA.Aes.7z new file mode 100644 index 00000000..21a07017 Binary files /dev/null and b/tests/TestArchives/Archives/7Zip.LZMA.Aes.7z differ diff --git a/tests/TestArchives/Archives/7Zip.LZMA2.Aes.7z b/tests/TestArchives/Archives/7Zip.LZMA2.Aes.7z new file mode 100644 index 00000000..94cf0ab1 Binary files /dev/null and b/tests/TestArchives/Archives/7Zip.LZMA2.Aes.7z differ diff --git a/tests/TestArchives/Archives/Audio_program.rar b/tests/TestArchives/Archives/Rar.Audio_program.rar similarity index 100% rename from tests/TestArchives/Archives/Audio_program.rar rename to tests/TestArchives/Archives/Rar.Audio_program.rar diff --git a/tests/TestArchives/Archives/Encrypted.rar b/tests/TestArchives/Archives/Rar.Encrypted.rar similarity index 100% rename from tests/TestArchives/Archives/Encrypted.rar rename to tests/TestArchives/Archives/Rar.Encrypted.rar diff --git a/tests/TestArchives/Archives/EncryptedParts.part01.rar b/tests/TestArchives/Archives/Rar.EncryptedParts.part01.rar similarity index 100% rename from tests/TestArchives/Archives/EncryptedParts.part01.rar rename to tests/TestArchives/Archives/Rar.EncryptedParts.part01.rar diff --git a/tests/TestArchives/Archives/EncryptedParts.part02.rar b/tests/TestArchives/Archives/Rar.EncryptedParts.part02.rar similarity index 100% rename from tests/TestArchives/Archives/EncryptedParts.part02.rar rename to tests/TestArchives/Archives/Rar.EncryptedParts.part02.rar diff --git a/tests/TestArchives/Archives/EncryptedParts.part03.rar b/tests/TestArchives/Archives/Rar.EncryptedParts.part03.rar similarity index 100% rename from tests/TestArchives/Archives/EncryptedParts.part03.rar rename to tests/TestArchives/Archives/Rar.EncryptedParts.part03.rar diff --git a/tests/TestArchives/Archives/EncryptedParts.part04.rar b/tests/TestArchives/Archives/Rar.EncryptedParts.part04.rar similarity index 100% rename from tests/TestArchives/Archives/EncryptedParts.part04.rar rename to tests/TestArchives/Archives/Rar.EncryptedParts.part04.rar diff --git a/tests/TestArchives/Archives/EncryptedParts.part05.rar b/tests/TestArchives/Archives/Rar.EncryptedParts.part05.rar similarity index 100% rename from tests/TestArchives/Archives/EncryptedParts.part05.rar rename to tests/TestArchives/Archives/Rar.EncryptedParts.part05.rar diff --git a/tests/TestArchives/Archives/EncryptedParts.part06.rar b/tests/TestArchives/Archives/Rar.EncryptedParts.part06.rar similarity index 100% rename from tests/TestArchives/Archives/EncryptedParts.part06.rar rename to tests/TestArchives/Archives/Rar.EncryptedParts.part06.rar diff --git a/tests/TestArchives/Archives/Rarjpeg.jpg b/tests/TestArchives/Archives/Rar.jpeg.jpg similarity index 100% rename from tests/TestArchives/Archives/Rarjpeg.jpg rename to tests/TestArchives/Archives/Rar.jpeg.jpg diff --git a/tests/TestArchives/Archives/test_invalid_exttime.rar b/tests/TestArchives/Archives/Rar.test_invalid_exttime.rar similarity index 100% rename from tests/TestArchives/Archives/test_invalid_exttime.rar rename to tests/TestArchives/Archives/Rar.test_invalid_exttime.rar diff --git a/tests/TestArchives/Archives/Rar5.crc_blake2.rar b/tests/TestArchives/Archives/Rar5.crc_blake2.rar new file mode 100644 index 00000000..2e89e107 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.crc_blake2.rar differ diff --git a/tests/TestArchives/Archives/Rar5.encrypted_filesAndHeader.rar b/tests/TestArchives/Archives/Rar5.encrypted_filesAndHeader.rar new file mode 100644 index 00000000..d0bad6d4 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.encrypted_filesAndHeader.rar differ diff --git a/tests/TestArchives/Archives/Rar5.encrypted_filesOnly.rar b/tests/TestArchives/Archives/Rar5.encrypted_filesOnly.rar new file mode 100644 index 00000000..cb18a286 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.encrypted_filesOnly.rar differ diff --git a/tests/TestArchives/Archives/Rar5.multi.part01.rar b/tests/TestArchives/Archives/Rar5.multi.part01.rar new file mode 100644 index 00000000..89731875 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.multi.part01.rar differ diff --git a/tests/TestArchives/Archives/Rar5.multi.part02.rar b/tests/TestArchives/Archives/Rar5.multi.part02.rar new file mode 100644 index 00000000..d23d82e4 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.multi.part02.rar differ diff --git a/tests/TestArchives/Archives/Rar5.multi.part03.rar b/tests/TestArchives/Archives/Rar5.multi.part03.rar new file mode 100644 index 00000000..159c55d8 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.multi.part03.rar differ diff --git a/tests/TestArchives/Archives/Rar5.multi.part04.rar b/tests/TestArchives/Archives/Rar5.multi.part04.rar new file mode 100644 index 00000000..cb0ed618 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.multi.part04.rar differ diff --git a/tests/TestArchives/Archives/Rar5.multi.part05.rar b/tests/TestArchives/Archives/Rar5.multi.part05.rar new file mode 100644 index 00000000..ec132ab8 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.multi.part05.rar differ diff --git a/tests/TestArchives/Archives/Rar5.multi.part06.rar b/tests/TestArchives/Archives/Rar5.multi.part06.rar new file mode 100644 index 00000000..8b908610 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.multi.part06.rar differ diff --git a/tests/TestArchives/Archives/Rar5.none.rar b/tests/TestArchives/Archives/Rar5.none.rar new file mode 100644 index 00000000..276234c5 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.none.rar differ diff --git a/tests/TestArchives/Archives/Rar5.rar b/tests/TestArchives/Archives/Rar5.rar new file mode 100644 index 00000000..0a05d520 Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.rar differ diff --git a/tests/TestArchives/Archives/Rar5.solid.rar b/tests/TestArchives/Archives/Rar5.solid.rar new file mode 100644 index 00000000..e16a01da Binary files /dev/null and b/tests/TestArchives/Archives/Rar5.solid.rar differ diff --git a/tests/TestArchives/Archives/Zip.Evil.zip b/tests/TestArchives/Archives/Zip.Evil.zip new file mode 100644 index 00000000..3474c88b Binary files /dev/null and b/tests/TestArchives/Archives/Zip.Evil.zip differ diff --git a/tests/TestArchives/Archives/Zip.deflate64.zip b/tests/TestArchives/Archives/Zip.deflate64.zip new file mode 100644 index 00000000..2c2e8905 Binary files /dev/null and b/tests/TestArchives/Archives/Zip.deflate64.zip differ