mirror of
https://github.com/google/brotli.git
synced 2026-07-08 17:56:58 +00:00
Incompilable with Intel, few tweaks and options miissing #34
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Sanmayce on GitHub (Sep 25, 2015).
Hi Brotli team,
despite me being one of Google "haters" let me share my 2 cents on current Brotli status.
In next several months I intend to juxtapose several high-performance textual compressors with one goal in mind - showing most balanced ones for high-ratio/high-decompression-speed scenario.
Yesterday I downloaded your 'master' zip and compiled (with several syntactic changes) with Intel v15.0 optimizer.
In my incoming showdown I want to include Brotli wanting to see how it performs in its best environment, I speak textual (mostly English texts) files.
[Question #1:]
Since my goal is to show tightness&decompression-speed top-performers, are following enforced defaults best?
struct BrotliParams {
BrotliParams()
// : mode(MODE_GENERIC),
// quality(11),
// lgwin(22),
// lgblock(0),
// enable_dictionary(true),
// enable_transforms(false),
// greedy_block_split(false),
// enable_context_modeling(true) {}
It would be very good to make these command line toggleable, no?
[Question #2:]
Your little announcement makes the impression Brotli is something special on text, what do I miss to see that? My quick test shows goodness but not greatness?
The below stats are for your yesterday commit compiled with Intel v15.0 (/O3 used), Brotli outperforms Shifune, but in decompression-speed department 3x is no joke, don't tell me if I use a browser or some English texts full-text browser/searcher Brotli will load 'dickens' faster than Zstd or even Shifune.
D:>bro_Intel15.exe -i dickens -o dickens.brotli -v
Brotli compression speed: 0.200944 MB/s
D:>bro_Intel15.exe -i dickens.brotli -o dickens -v -d -f
Brotli decompression speed: 142.945 MB/s
D:>bro_Intel15.exe -i dickens.brotli -o dickens -v -d -f
Brotli decompression speed: 138.861 MB/s
D:>bro_Intel15.exe -i dickens.brotli -o dickens -v -d -f
Brotli decompression speed: 145.079 MB/s
D:>bro_Intel15.exe -i dickens.brotli -o dickens -v -d -f -r 5
Brotli decompression speed: 144.647 MB/s
D:>bro_Intel15.exe -i dickens.brotli -o dickens -v -d -f -r 5
Brotli decompression speed: 145.513 MB/s
D:>bro_Intel15.exe -i dickens.brotli -o dickens -v -d -f -r 20
Brotli decompression speed: 145.841 MB/s
D:>bro_Intel15.exe -i dickens.brotli -o dickens -v -d -f -r 40
Brotli decompression speed: 144.701 MB/s
D:>Nakamichi_Shifune_branchfull.exe dickens.Nakamichi /report
Nakamichi 'Shifune-Totenschiff', written by Kaze, based on Nobuo Ito's LZSS source, babealicious suggestion by m^2 enforced, muffinesque suggestion by Jim Dempsey enforced.
Note: This compile can handle files up to 1711MB.
Decompressing 3740418 bytes ...
RAM-to-RAM performance: 512 MB/s.
Compression Ratio (bigger-the-better): 2.72:1
D:>dir dic*
09/25/2015 03:32 AM 10,192,446 dickens
09/25/2015 03:29 AM 2,962,118 dickens.brotli
09/08/2015 02:33 AM 3,740,418 dickens.Nakamichi
D:>
The above quick run was done on my Core 2 laptop, on Haswell the 3x may jump up to 5x hands down, hate that I don't have Haswell or alike to share the actual stats.
[Question #3:]
Don't you think that your defaults (encode.h) are too low, I do, my big test shows worse ratio than gzip?
D:>zpaq64 add _Deathship_textual_corpus.tar.method58.zpaq _Deathship_textual_corpus.tar -method 58 -threads 1
D:>bsc e _Deathship_textual_corpus.tar _Deathship_textual_corpus.tar.ST6Block256.bsc -b256 -m6 -cp -Tt
D:>xz -z -k -f -9 -e -v -v --threads=1 _Deathship_textual_corpus.tar
D:>lzturbo.exe -39 -b256 -p0 _Deathship_textual_corpus.tar _Deathship_textual_corpus.tar.256MB.lzturbo12-39.lzt
D:>zpaq64 add _Deathship_textual_corpus.tar.method28.zpaq _Deathship_textual_corpus.tar -method 28 -threads 1
D:>7za a -tgzip -mx9 _Deathship_textual_corpus.tar.zip _Deathship_textual_corpus.tar
D:>bro_Intel15.exe -i _Deathship_textual_corpus.tar -o _Deathship_textual_corpus.tar.brotli -v
D:>zstd.exe _Deathship_textual_corpus.tar
D:>LZ4.exe -9 _Deathship_textual_corpus.tar
09/12/2015 12:59 PM 1,125,281,882 _Deathship_textual_corpus.tar.method58.zpaq
09/12/2015 02:34 AM 1,342,098,184 _Deathship_textual_corpus.tar.ST6Block256.bsc
09/11/2015 11:56 AM 1,471,795,768 _Deathship_textual_corpus.tar.xz
09/13/2015 07:31 PM 1,484,820,599 _Deathship_textual_corpus.tar.256MB.lzturbo12-39.lzt
09/14/2015 09:18 AM 1,800,083,824 _Deathship_textual_corpus.tar.method28.zpaq
Here comes Nakamichi 'Shifune' ...
09/13/2015 06:29 AM 2,181,159,237 _Deathship_textual_corpus.tar.zip
09/24/2015 11:36 PM 2,382,646,308 _Deathship_textual_corpus.tar.brotli
09/13/2015 03:04 AM 2,491,454,533 _Deathship_textual_corpus.tar.zst
09/13/2015 07:50 AM 2,626,828,543 _Deathship_textual_corpus.tar.lz4
09/11/2015 06:41 AM 8,090,119,168 _Deathship_textual_corpus.tar
A glimpse at my unfinished latest benchmark:
www.sanmayce.com/Hayabusa/Deathship_showdown.pdf
www.sanmayce.com/Hayabusa/Nakamichi_Shifune.pdf
[Suggestion #1:]
Your time reports seem problematic, I receive 0 MB/s for big files. Please make Brotli with '-b' benchmark or '-t' test (decompression without dump) ability, Zstd&Z4 have very good report.
Your current speed report includes 'fwrite()' time, I want Brotli's pure RAM-2-RAM performance.
[Suggestion #2:]
Make it compileable with Intel C/C++ optimizer, this will be appreciated by me for one. Current changes in bro.cc (I made) to run it:
#1:
#2:
And the actual console dump of how the compilation went:
// The next log/source is modified (for Windows compatibility) Brotli:
And a final note, a byte angry, in your promoting paper you say "Decompresses much faster than current LZMA implementations", usually amateurs like me use 2x, 3x or 15x, your much is not good, one would think from 2x to 20x.
Also why don't you mention the current best (IMO) decompressor on INTERNET?! Not mentioning it (LzTurbo) is like disrespecting not only the man behind it but the BEST as a general notion, yes?
Hope you will refine Brotli and make it usable hi-performance console tool.
Regards,
Kaze