mirror of
https://github.com/google/brotli.git
synced 2026-07-08 17:56:58 +00:00
Decompression equivalent to BrotliEncoderPrepareDictionary? #539
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @pmeenan on GitHub (May 6, 2025).
For the shared brotli case, would there be a benefit to be able to prepare a dictionary in such a way that it can be reused in a decompression context across decodes like we can in compression with BrotliEncoderPrepareDictionary?
Currently we (Chrome) re-attach the raw dictionary to each new decode stream state which, I assume, involves some processing on the raw dictionary to build tables, etc.
For the document navigation case (and a few other cases like API calls), we reuse the same dictionary over and over and we could probably get a decent latency improvement by not having to rebuild the frequency tables/etc from a raw dictionary every time.
I'm assuming we could probably transform a raw dictionary into a serialized dictionary and maybe get some of the benefit, but I was hoping that we might also expose a shortcut to be able to reuse a fully-parsed raw dictionary.
For context, I'm adding an in-memory cache in Chrome right now for the dictionaries and if I could cache the processed version it might eek out a few more milliseconds of latency on slower devices.
@eustas commented on GitHub (May 27, 2025):
In decompression there is no processing of attached raw dictionaries, that it why there is no "prepare" API.
One case where decompression could be improved a wee is when multiple dictionaries are attached. Decoder would produce the same results, but work a bit faster if those dictionaries are concatenated into a single one.
@pmeenan commented on GitHub (May 27, 2025):
oh, good to know, thanks. Do you know if it scans the dictionary at some point to calculate the HighwayHash that shared brotli uses to match dictionaries or is that not used for raw dictionaries?
@eustas commented on GitHub (May 30, 2025):
In this library - no; it is done on embedder side. Could take a look on how it is done in Chromium, when I get spare cycles.
@pmeenan commented on GitHub (May 30, 2025):
Perfect. No need to check Chromium - Chromium uses the SHA-256 hash from the file header to validate the dictionaries independently. Thanks - sounds like there's basically zero overhead on the brotli side to start up the decompression context with a dictionary.