Dynamically generate the compression dictionary? #516

Closed
opened 2026-01-29 20:45:04 +00:00 by claunia · 2 comments
Owner

Originally created by @Timmmm on GitHub (Sep 10, 2024).

Hi, the Chrome guys have quite stringent binary size requirements and are worried about the size of the dictionary used for compression. We think they're talking about this pre-hashed dictionary.

I just thought I'd check with you guys: presumably that dictionary only depends on the already-present decompression dictionary (which they don't seem to have a problem with). Can it be generated at runtime instead of embedded in the binary?

(They may not be talking about that tbf since they mentioned 190kB binary size and that table is only 64kB; maybe the compression code itself is 130kB?)

Originally created by @Timmmm on GitHub (Sep 10, 2024). Hi, the Chrome guys have quite stringent binary size requirements and [are worried about the size of the dictionary used for compression](https://github.com/whatwg/compression/issues/34#issuecomment-2290651996). We think they're talking about [this pre-hashed dictionary](https://github.com/google/brotli/blob/master/c/enc/dictionary_hash.c). I just thought I'd check with you guys: presumably that dictionary only depends on the already-present decompression dictionary (which they don't seem to have a problem with). Can it be generated at runtime instead of embedded in the binary? (They may not be talking about that tbf since they mentioned 190kB binary size and that table is only 64kB; maybe the compression code itself is 130kB?)
Author
Owner

@eustas commented on GitHub (Jun 26, 2025):

This question have already raised several times. Yes, it is possible to make this data dynamically generated, but the fragile point is "singleton guarantees", and even worse "static resource freeing".

I will check, how much time it will take to implement, but the solution (API wise) will look like - BrotliEncoderGlobalInit / BrotliEncoderGlobalFree and it will be embedders sole responsibility to invoke those not too early and not too late. Likely for static linking only...

@eustas commented on GitHub (Jun 26, 2025): This question have already raised several times. Yes, it is possible to make this data dynamically generated, but the fragile point is "singleton guarantees", and even worse "static resource freeing". I will check, how much time it will take to implement, but the solution (API wise) will look like - `BrotliEncoderGlobalInit` / `BrotliEncoderGlobalFree` and it will be embedders sole responsibility to invoke those not too early and not too late. Likely for static linking only...
Author
Owner

@eustas commented on GitHub (Sep 4, 2025):

Added API for early/lazy static tables initialization

@eustas commented on GitHub (Sep 4, 2025): Added API for early/lazy static tables initialization
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/brotli#516