Performance measurements and feedback #70

New Issue

claunia · 2026-01-29T20:31:27Z

claunia commented

2026-01-29 20:31:27 +00:00

Originally created by @nitram509 on GitHub (Nov 28, 2015).

About

I'm working on a project called jbrotli to make Brotli compression available to Java developers.
My goal is to enable Java based HTTP server to use Brotli compression on the fly.

I've did some performance benchmarks with the cp.html (from canterbury corpus).
It turns out, that gzip is little more than twice as fast as brotli, when compressing this HTML.
This fails may expectations - I thought the Brotli outnumbers gzip.

Issue

Most other benchmarks make Brotli shine, compared to gzip.
I'm looking forward to get feedback from the experts,
why these numbers are so "bad" - or where the code could be improved.

Testcases

All these test cases use a) the 24kB HTML from canterbury corpus and b) 2xcp.html, all read from memory.

brotli_compression_with_BrotliCompressor_using_ByteBuffer

uses brotli::BrotliCompressBuffer()
Intended to use simple stateless interface of brotli.
Second, it also is intended to show differences between byte[] and direct ByteBuffer in Java.
Remark: Most likely this direct ByteBuffer thingy is faster than using byte[], as many other projects already experienced that. For comparison to brotli - this can be treated equal to using byte[]

brotli quality parameter = 5

source https://github.com/nitram509/jbrotli/blob/master/jbrotli-native/src/main/cpp/de_bitkings_jbrotli_BrotliCompressor.cxx

brotli_compression_with_BrotliStreamCompressor_using_byte_array

Instantiates only one object of brotli::BrotliCompressor
and only does CopyInputToRingBuffer() followed by WriteBrotliData().
Intended to have pure compression and reuse of the pre-calculated dictionary.

brotli quality parameter = 5

source https://github.com/nitram509/jbrotli/blob/master/jbrotli-native/src/main/cpp/de_bitkings_jbrotli_BrotliStreamCompressor.cxx

gzip_compression

Uses default GZIPOutputStream with default compression level.

compression level default (expect 5 or 6)

JNI_doing_simple_memcpy_using_byte_array

This test uses the same Java method signature as the compressor, but only does 'memcpy'.
This is intended to show how much overhead ther is for doing JNI calls.

Results

## using cp.html 24kB HTML
Benchmark                                                         Mode  Cnt       Score      Error  Units
brotli_compression_with_BrotliCompressor_using_ByteBuffer        thrpt    5          65 ±        2  ops/s
brotli_compression_with_BrotliStreamCompressor_using_byte_array  thrpt    5         761 ±        6  ops/s
gzip_compression                                                 thrpt    5        1992 ±       47  ops/s
JNI_doing_simple_memcpy_using_byte_array                         thrpt    5      837511 ±     4273  ops/s

## using 2xcp.html 48kB HTML
Benchmark                                                         Mode  Cnt       Score      Error  Units
brotli_compression_with_BrotliCompressor_using_ByteBuffer        thrpt    5          60 ±        0  ops/s
brotli_compression_with_BrotliStreamCompressor_using_byte_array  thrpt    5         384 ±       58  ops/s
gzip_compression                                                 thrpt    5        1613 ±       52  ops/s
JNI_doing_simple_memcpy_using_byte_array                         thrpt    5      461217 ±     2856  ops/s

Setup:

using brotli 0.2.0 release
using Visual Studio 2010 compiler 64bit
using custom cmake file (maybe some compiler options are missing)
using Java Micro Benchmark (jmh) for measurement
using Java 1.8.0_u65, 64bit
source https://github.com/nitram509/jbrotli ... consider experimental ;-)

Originally created by @nitram509 on GitHub (Nov 28, 2015). ### About I'm working on a project called [jbrotli](https://github.com/nitram509/jbrotli) to make Brotli compression available to Java developers. My goal is to enable Java based HTTP server to use Brotli compression on the fly. I've did some performance benchmarks with the cp.html (from canterbury corpus). It turns out, that gzip is little more than twice as fast as brotli, when compressing this HTML. This fails may expectations - I thought the Brotli outnumbers gzip. ### Issue Most other benchmarks make Brotli shine, compared to gzip. I'm looking forward to get feedback from the experts, why these numbers are so "bad" - or where the code could be improved. ### Testcases All these test cases use a) the 24kB HTML from canterbury corpus and b) 2xcp.html, all read from memory. ##### brotli_compression_with_BrotliCompressor_using_ByteBuffer uses brotli::BrotliCompressBuffer() Intended to use simple stateless interface of brotli. Second, it also is intended to show differences between byte[] and direct ByteBuffer in Java. Remark: Most likely this direct ByteBuffer thingy is faster than using byte[], as many other projects already experienced that. For comparison to brotli - this can be treated equal to using byte[] - brotli quality parameter = 5 source https://github.com/nitram509/jbrotli/blob/master/jbrotli-native/src/main/cpp/de_bitkings_jbrotli_BrotliCompressor.cxx ##### brotli_compression_with_BrotliStreamCompressor_using_byte_array Instantiates only one object of brotli::BrotliCompressor and only does CopyInputToRingBuffer() followed by WriteBrotliData(). Intended to have pure compression and reuse of the pre-calculated dictionary. - brotli quality parameter = 5 source https://github.com/nitram509/jbrotli/blob/master/jbrotli-native/src/main/cpp/de_bitkings_jbrotli_BrotliStreamCompressor.cxx ##### gzip_compression Uses default GZIPOutputStream with default compression level. - compression level default (expect 5 or 6) ##### JNI_doing_simple_memcpy_using_byte_array This test uses the same Java method signature as the compressor, but only does 'memcpy'. This is intended to show how much overhead ther is for doing JNI calls. ### Results ``` ## using cp.html 24kB HTML Benchmark Mode Cnt Score Error Units brotli_compression_with_BrotliCompressor_using_ByteBuffer thrpt 5 65 ± 2 ops/s brotli_compression_with_BrotliStreamCompressor_using_byte_array thrpt 5 761 ± 6 ops/s gzip_compression thrpt 5 1992 ± 47 ops/s JNI_doing_simple_memcpy_using_byte_array thrpt 5 837511 ± 4273 ops/s ## using 2xcp.html 48kB HTML Benchmark Mode Cnt Score Error Units brotli_compression_with_BrotliCompressor_using_ByteBuffer thrpt 5 60 ± 0 ops/s brotli_compression_with_BrotliStreamCompressor_using_byte_array thrpt 5 384 ± 58 ops/s gzip_compression thrpt 5 1613 ± 52 ops/s JNI_doing_simple_memcpy_using_byte_array thrpt 5 461217 ± 2856 ops/s ``` ### Setup: - using brotli 0.2.0 release - using Visual Studio 2010 compiler 64bit - using custom cmake file (maybe some compiler options are missing) - using Java Micro Benchmark (jmh) for measurement - using Java 1.8.0_u65, 64bit - source https://github.com/nitram509/jbrotli ... consider _experimental_ ;-)

claunia closed this issue

2026-01-29 20:31:28 +00:00

claunia commented

2026-01-29 20:31:30 +00:00

@nemequ commented on GitHub (Dec 2, 2015):

It turns out, that gzip is little more than twice as fast as brotli, when compressing this HTML.
This fails may expectations - I thought the Brotli outnumbers gzip.

Brotli is generally slower than gzip for compression. It is usually a bit faster at decompression, and the compression ratio is much higher. See https://quixdb.github.io/squash-benchmark/?dataset=cp.html&machine=peltast&visible-plugins=brotli,zlib#ratio-vs-decompression for results for cp.html; results are probably similar for most datasets. If you turn on some of the other codecs squash offers (click the label in the legend) you'll get a more complete picture of where brotli fits in with the competition.

This is a pretty good place for brotli to be. Compressing content once and decompressing many times is a pretty common use case, so it's usually better to emphasize decompressor speed and ratio at the expense of compressor speed.

@nemequ commented on GitHub (Dec 2, 2015): > It turns out, that gzip is little more than twice as fast as brotli, when compressing this HTML. > This fails may expectations - I thought the Brotli outnumbers gzip. Brotli is generally slower than gzip for compression. It is usually a bit faster at **de**compression, and the compression ratio is much higher. See https://quixdb.github.io/squash-benchmark/?dataset=cp.html&machine=peltast&visible-plugins=brotli,zlib#ratio-vs-decompression for results for cp.html; results are probably similar for most datasets. If you turn on some of the other codecs squash offers (click the label in the legend) you'll get a more complete picture of where brotli fits in with the competition. This is a pretty good place for brotli to be. Compressing content once and decompressing many times is a pretty common use case, so it's usually better to emphasize decompressor speed and ratio at the expense of compressor speed.

claunia commented

2026-01-29 20:31:32 +00:00

@eustas commented on GitHub (Aug 22, 2016):

We are working on our own Java wrappers. Soon it will be possible to make measurements and compare results / investigate bottlenecks.

@eustas commented on GitHub (Aug 22, 2016): We are working on our own Java wrappers. Soon it will be possible to make measurements and compare results / investigate bottlenecks.

claunia commented

2026-01-29 20:31:35 +00:00

@nitram509 commented on GitHub (Aug 22, 2016):

Wow cool, that you plan to provide Java wrappers. I'm interested in your outcomes.
So we might join forces and can further improve brotli support in the Java World.

@nitram509 commented on GitHub (Aug 22, 2016): Wow cool, that you plan to provide Java wrappers. I'm interested in your outcomes. So we might join forces and can further improve brotli support in the Java World.

claunia commented

2026-01-29 20:31:37 +00:00

@chrisco484 commented on GitHub (Jan 31, 2017):

@nemequ said

Compressing content once and decompressing many times is a pretty common use case

This would apply back in the static HTML days.

Nowadays most websites are data driven and the HTML is dynamically generated on the fly by most Java UI frameworks (and frameworks in other languages) so compressing once and reusing cached compressed versions of the HTML just doesn't really work naturally in that scenario.

To make it work you would need to add a serious amount of code to work out everything that could change on a page - content, theme selection changes (which CSS files to use), user permissions, available JS powered controls etc.,) and when something does change blow away the cached compressed version of the HTML and recompress.

Just looking around the main sites I couldn't find any actually using brotli - weird!

At time of writing this, not Microsoft, not IBM, not even www.google.com uses it and they developed it!

Assuming pre-compression isn't an option for dynamically generated HTML pages then, maybe the trade off between slower compression time (i.e. extra server CPU cycles consumed) vs the savings in the size of downloaded artifacts (consumes slightly less bandwidth) just doesn't hit the sweet spot? Well not yet at least - maybe the compression algorithm implementations have some optimizations coming? Maybe faster to run in native machine code perhaps like the Apache/Tomcat APR?

@chrisco484 commented on GitHub (Jan 31, 2017): @nemequ said > Compressing content once and decompressing many times is a pretty common use case This would apply back in the static HTML days. Nowadays most websites are data driven and the HTML is dynamically generated on the fly by most Java UI frameworks (and frameworks in other languages) so compressing once and reusing cached compressed versions of the HTML just doesn't really work naturally in that scenario. To make it work you would need to add a serious amount of code to work out everything that could change on a page - content, theme selection changes (which CSS files to use), user permissions, available JS powered controls etc.,) and when something does change blow away the cached compressed version of the HTML and recompress. Just looking around the main sites I couldn't find any actually using brotli - weird! At time of writing this, not Microsoft, not IBM, not even www.google.com uses it and they developed it! Assuming pre-compression isn't an option for dynamically generated HTML pages then, maybe the trade off between slower compression time (i.e. extra server CPU cycles consumed) vs the savings in the size of downloaded artifacts (consumes slightly less bandwidth) just doesn't hit the sweet spot? Well not yet at least - maybe the compression algorithm implementations have some optimizations coming? Maybe faster to run in native machine code perhaps like the Apache/Tomcat APR?

claunia commented

2026-01-29 20:31:38 +00:00

@eustas commented on GitHub (Aug 4, 2017):

Hello, Martin.

Now, we have our JNI wrappers published. Could you test their performance, and see if the problem is still here, please.

Thanks in advance,
Eugene.

@eustas commented on GitHub (Aug 4, 2017): Hello, Martin. Now, we have our JNI wrappers published. Could you test their performance, and see if the problem is still here, please. Thanks in advance, Eugene.

claunia commented

2026-01-29 20:31:40 +00:00

@eustas commented on GitHub (Nov 27, 2017):

Ping?

Feel free to reopen...

@eustas commented on GitHub (Nov 27, 2017): Ping? Feel free to reopen...

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/brotli#70