mirror of
https://github.com/google/brotli.git
synced 2026-07-09 02:08:06 +00:00
Abysmal compression speed for x-ray from Silesa #5
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @nemequ on GitHub (Mar 27, 2015).
Brotli's compression speed for the x-ray file from the Silesa compression corpus is unreasonably poor. In the benchmark I'm working on it takes about 60x as long as the next slowest codec, which is ZPAQ at its highest setting.
@svgeesus commented on GitHub (Mar 27, 2015):
As Brotli is deliberately designed to be asymmetric (favouring decompression speed over compression speed) it would be interesting to know where it ranks on decompression time on that image.
@nemequ commented on GitHub (Mar 27, 2015):
Decompression speed is about right. Look at the benchmark I linked to. The results table can be sorted by decompression speed; just click on the "Decompression Speed" header of the results table.
The problem is that this is a huge outlier for brotli's compression speed. It's so slow that it basically amounts to a DoS vulnerability. It takes an hour to compress an 8.1 MiB file (on a Xeon E3-1225 v3), which is 5.62 KiB/s. The next slowest file (ooffice, which is a 5.87 MiB DLL from OpenOffice.org 1.01) is 59.9 KiB/s.
I think a lot of the compression speeds are pretty bad when you look at Brotli's competition (LZHAM is probably the closest competitor—it offers pretty similar ratios and decompression speed but much more stable, and generally faster compression). But 5.62 KiB/s is just unacceptable.
@nemequ commented on GitHub (Mar 29, 2015):
It's probably not unrelated that brotli also allocates a lot of memory for this file. One of my ARM boards (with 1 GiB memory) mercifully terminates with a std::bad_alloc exception (hopefully the others will, as well). An old laptop with 1 GiB memory and plenty of swap space, OTOH, is still happily thrashing after 50 hours. Peak memory usage for that process right now is 1790520 KiB, current is 1266228. For an 8.1 MiB file.
@eustas commented on GitHub (Aug 12, 2015):
I believe this has been fixed.
I've measured compression of x-ray on odroid-c1 with broltli (level 11) - it took 5 min 41 sec, so the speed is about 24 KiB/s. On rpi-2 it would take 10-15 minutes to compress.
With lower level it works much faster. E.g. with level 1 it takes less than 2 sec; on level 9 it works 1 min 24 sec.
Probably it is time to reevaluate brotli compression...
@eustas commented on GitHub (Aug 12, 2015):
FWIW, time -f "%M" shows that brotli uses ~310MB of memory when compress x-ray (on level 11)
@nemequ commented on GitHub (Aug 22, 2015):
I just updated the benchmark, and this issue does seem to be fixed.
Brotli has made a lot of progress in the last few months.