Question: Using Brotli in embedded devices #265

New Issue

claunia · 2026-01-29T20:41:07Z

claunia commented

2026-01-29 20:41:07 +00:00

Originally created by @slaff on GitHub (Dec 26, 2018).

We are trying to use Brotli in Sming Framework ( an embedded framework for a microprocessor with limited RAM and CPU).
We would like to use only the decompression functionality to decompress on the-fly incoming HTTP data. The brotli-encoded data comes in chunks of variable sizes from 512 to 1024. For the output we have allocated 3K buffer that is fetched and cleared after every chunk is decompressed.

The most important part of the code that we use can be found here: https://github.com/slaff/Sming/blob/feature/httpclient-content-decoders/Sming/SmingCore/Data/Coder/Decoder.cpp. In our tests under Linux all works fine.

BUT

The problem that we experience when running it on the microcontroller is that brotli tries to allocate 16K RAM ( to be precise 16K + 56 bytes), which is luxury for an embedded device and we rarely have so much RAM available. Are there any compiler directives that we can use to instruct it to use minimal RAM ?

Originally created by @slaff on GitHub (Dec 26, 2018). We are trying to use Brotli in [Sming Framework](https://github.com/SmingHub/Sming) ( an embedded framework for a microprocessor with limited RAM and CPU). We would like to use only the decompression functionality to decompress on the-fly incoming HTTP data. The brotli-encoded data comes in chunks of variable sizes from 512 to 1024. For the output we have allocated 3K buffer that is fetched and cleared after every chunk is decompressed. The most important part of the code that we use can be found here: https://github.com/slaff/Sming/blob/feature/httpclient-content-decoders/Sming/SmingCore/Data/Coder/Decoder.cpp. In our tests under Linux all works fine. BUT The problem that we experience when running it on the microcontroller is that `brotli` tries to allocate 16K RAM ( to be precise 16K + 56 bytes), which is luxury for an embedded device and we rarely have so much RAM available. Are there any compiler directives that we can use to instruct it to use minimal RAM ?

claunia commented

2026-01-29 20:41:08 +00:00

@eustas commented on GitHub (Feb 18, 2019):

Brotli have to do that. This piece of memory is allocated for "ring-buffer". It is possible to have smaller ring-buffer if it is not in use: just force files to be compressed with smaller "window". Minimal window is 1024 bytes, but it will still have some tail (+42 bytes).

Also, if possible, avoid allocating "output buffer"; TakeOutput API allows reading ready output directly from the ring-buffer.

Tip: using BROTLI_DECODER_PARAM_DISABLE_RING_BUFFER_REALLOCATION guarantees that ringbuffer is not reallocated -> might reduce peak memory usage.

PS: sorry for ultra-long answer; very busy with JPEG XL...

@eustas commented on GitHub (Feb 18, 2019): Brotli have to do that. This piece of memory is allocated for "ring-buffer". It is possible to have smaller ring-buffer if it is not in use: just force files to be compressed with smaller "window". Minimal window is 1024 bytes, but it will still have some tail (+42 bytes). Also, if possible, avoid allocating "output buffer"; `TakeOutput` API allows reading ready output directly from the ring-buffer. Tip: using `BROTLI_DECODER_PARAM_DISABLE_RING_BUFFER_REALLOCATION` guarantees that ringbuffer is not reallocated -> might reduce peak memory usage. PS: sorry for ultra-long answer; very busy with JPEG XL...

claunia referenced this issue

2026-01-29 20:51:17 +00:00

[PR #265] [CLOSED] Avoid integer overflow on x86_64-mingw, closes #264 #864

claunia referenced this issue

2026-01-29 20:51:19 +00:00

[PR #265] Avoid integer overflow on x86_64-mingw, closes #264 #869

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/brotli#265