mirror of
https://github.com/google/brotli.git
synced 2026-09-22 06:35:52 +00:00
"failed to determine uncompressed file size" #100
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @Sanmayce on GitHub (Mar 9, 2016).
Hi,
wanted to see how your Brotli fares again with big files, it turns out an issue occurred:
The current side-by-side dump:
The superstrong BSC despite using 256MB blocks, surprisingly, is not that strong when compared to Zstd, as for decompression speed, BSC can't stand a chance.
If you fix that issue I will attempt a new run. By the way, I find no executable within release section, why so?
@jyrkialakuijala commented on GitHub (Mar 9, 2016):
Would you be willing repeat the test so that every algorithm is limited to roughly 16 MB of decoding context?
@Sanmayce commented on GitHub (Mar 10, 2016):
No problema, the idea is this torture to present a clearer picture of mainstream textual [de]compression, yet my desire is to go beyond the funless 4GB of RAM and see what future holds. With higher densities of DDR4 sticks we will have soon 4x32GB in desktop PCs, what about using 1GB blocks as well?!
But first, I need a new compile, not complaining about decompression size.
Also, my intention is to add:
XZ uses 64MB, by default, in its max level i.e. 9.
Can't wait to see also the Planet-Spitting Performance of LZSSE2 level 16, have to wait for its console version... yet, thanks to Hamid's TurboBench I made one quite interesting benchmark of 80 DSLs (dictionaries with minimalistic html-like tagging), the very same that form the:
For the full 80 testdatafiles log:
http://www.sanmayce.com/Downloads/log-Laptop-Toshiba_DSLs.txt
Just one, the Merriam brothers' 4 inches thick one with 2,662 pages defining 450,000+ words:
Just look what LZSSE2 is doing, I AM SHOCKED!
Where is the law? 64KB vs 4MB vs 16MB vs 256MB vs 1GB...
Guru Josh - Whose Law (Is It Anyway)
https://www.youtube.com/watch?v=x7ytf5T1fY8
“I’m living in a future. A future of law. I want to know. Whose law can it be? A future for you and I. I’m living in a world. In a world of law. Whose law can it be?”
Paraphrasing the awesome Guru's question, whose dictionary length can it be?
@Sanmayce commented on GitHub (Mar 10, 2016):
Should I had had more computational power I would have been exploring deeply, anyway:
That's how the DSL looks like:
The add-on roster:
The respective command line invocations:
In UFC (Ultimate-Fighting-Championship), they use the term P4P to describe most balanced and skilled Mixed Martial Artist.
Ilya Grebnov's BSC excels here, however, in my view the best pound-for-pound decompressor is the Conor Stokes' LZSSE2 level 17, still not available within a console tool...
Second best, to me, is Hamid's LzTurbo mode 39, 475MB/s on a weak laptop with 2nd gen mobile i5 and RAM @666MHz - tell me, what compressor can trump it, someone?
Third best, Zstd is closing in fast.
Nakamichi 'Goldenboy' decompresses at 576MB/s but is fatty - with 7MB overweight. LZSSE2 smokes Goldenboy being 4x faster, PURE CRAZINESS!
Bottom-line:
My current TOP 4 of speedy decompressors: