Passing additional dictionary? #453

New Issue

claunia · 2026-01-29T20:44:07Z

claunia commented

2026-01-29 20:44:07 +00:00

Originally created by @kovalensky on GitHub (Apr 27, 2023).

This is kind of a question I had for a long time.
I have hundreds of thousands of files on my web server compressed by gzip and returned to clients, they are sharing the same extension format, the thing is every one of those files contains their own dictionary inside which is wasting additional space (calculated unnecessary dictionary data ~450GB), with gzip it's not possible to provide a proper dictionary; it's kinda possible but not a proper simple way I'm searching for (predefined index).

So my question is:

Is it possible to provide custom predefined built-in dictionary before compressing, so that only its byte address references would be saved inside compressed data, and Brotli would also analyze data for creating dynamic dictionary, and when sending our compressed data to user's browser we could join this custom built-in dictionary with it?

Tell me if you couldn't catch with my explanation.
Sorry if this an odd question.

Originally created by @kovalensky on GitHub (Apr 27, 2023). This is kind of a question I had for a long time. I have hundreds of thousands of files on my web server compressed by gzip and returned to clients, they are sharing the same extension format, the thing is every one of those files contains their own dictionary inside which is wasting additional space (calculated unnecessary dictionary data ~450GB), with gzip it's not possible to provide a proper dictionary; it's kinda possible but not a proper simple way I'm searching for (predefined index). So my question is: Is it possible to provide custom predefined built-in dictionary before compressing, so that only its byte address references would be saved inside compressed data, and Brotli would also analyze data for creating dynamic dictionary, and when sending our compressed data to user's browser we could join this custom built-in dictionary with it? Tell me if you couldn't catch with my explanation. Sorry if this an odd question.

claunia closed this issue

2026-01-29 20:44:08 +00:00

claunia commented

2026-01-29 20:44:08 +00:00

@eustas commented on GitHub (May 10, 2023):

Hello. Brotli encoder/decoder allows attaching dictionaries.
See -D parameter in brotli CLI. For HTTP currently br Content-Encoding does not allow using dictionaries. We have plans on introducing sbr Content-Encoding that would allow sharing dictionaries between web-server and browser. That would significantly improve compression density for the use-case you have described.

@eustas commented on GitHub (May 10, 2023): Hello. Brotli encoder/decoder allows attaching dictionaries. See `-D` parameter in brotli CLI. For HTTP currently `br` `Content-Encoding` does not allow using dictionaries. We have plans on introducing `sbr` `Content-Encoding` that would allow sharing dictionaries between web-server and browser. That would significantly improve compression density for the use-case you have described.

claunia referenced this issue

2026-01-29 20:52:24 +00:00

[PR #455] [MERGED] Fix OSX gcc-4.x compilation #1080

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/brotli#453