mirror of
https://github.com/google/brotli.git
synced 2026-07-08 17:56:58 +00:00
How do we create custom dictionaries? #247
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @JoeUX on GitHub (Jul 13, 2018).
As per Jyrki's request, I'm filing this issue: How do we create custom dictionaries? I haven't found any user docs on it.
For example, with Zstd we would do something like:
zstd --train FullPathToTrainingSet/* -o dictionaryNameThen we'd compress and decompress with that custom generated dictionary. I've trained dictionaries on JS libraries, jquery, etc. to good effect.
How do we do this with brotli? Also, is the custom dictionary included in the compressed file or is it freestanding? (Zstd's is freestanding – you can't decompress without having the dictionary file, kind of like protobuf message types.)
@eustas commented on GitHub (Jul 17, 2018):
Dictionary is freestanding.
There is
dictionary_generatortool inresearch/that generates dictionaries.Currently there is only
Bazelbuild file for the tool.Going to add CMake, make, etc. build files soon and move the tool to
c/tools/