There is no good way to estimate the size of out when decompressing #103

New Issue

claunia · 2026-01-29T20:34:46Z

claunia commented

2026-01-29 20:34:46 +00:00

Originally created by @mshneer on GitHub (Mar 25, 2016).

I am using BrotliDecompressBufferStreaming which does not provide a way to determine required size for output buffer. For large outputs, I am receiving BROTLI_RESULT_NEEDS_MORE_OUTPUT and the only solution I could work so far is to blindly increase the size of the output buffer and retry.

Possible ways to address these shortcomings.

when BROTLI_RESULT_NEEDS_MORE_OUTPUT is returned, available_in would contain the size of the required buffer.
Even better solution, that would avoid memory fragmentation, is to allow the caller placing output into chained buffers. I.e. available_in could return the size of the input buffer processed so far. In this case, one could create new buffer in a loop until available_in does not drop to 0.

Originally created by @mshneer on GitHub (Mar 25, 2016). I am using BrotliDecompressBufferStreaming which does not provide a way to determine required size for output buffer. For large outputs, I am receiving BROTLI_RESULT_NEEDS_MORE_OUTPUT and the only solution I could work so far is to blindly increase the size of the output buffer and retry. Possible ways to address these shortcomings. 1. when BROTLI_RESULT_NEEDS_MORE_OUTPUT is returned, available_in would contain the size of the required buffer. 2. Even better solution, that would avoid memory fragmentation, is to allow the caller placing output into chained buffers. I.e. available_in could return the size of the input buffer processed so far. In this case, one could create new buffer in a loop until available_in does not drop to 0.

claunia closed this issue

2026-01-29 20:34:47 +00:00

claunia commented

2026-01-29 20:34:48 +00:00

@danielrh commented on GitHub (Apr 18, 2016):

I generally encode the output size in a header on my branch here
https://github.com/danielrh/brotli/tree/header

it's also good to have a magic number and the commit ID of the binary that encoded it

@danielrh commented on GitHub (Apr 18, 2016): I generally encode the output size in a header on my branch here https://github.com/danielrh/brotli/tree/header it's also good to have a magic number and the commit ID of the binary that encoded it

claunia commented

2026-01-29 20:34:51 +00:00

@eustas commented on GitHub (Apr 29, 2016):

There is BrotliDecompressedSize method.
Unfortunately, it works only is stream contains only one block. But it is the most frequent case.
We plan to add framing format in future, but currently you need to attach your own header, as Daniel said.

@eustas commented on GitHub (Apr 29, 2016): There is [BrotliDecompressedSize](https://github.com/google/brotli/blob/master/dec/decode.h#L46) method. Unfortunately, it works only is stream contains only one block. But it is the most frequent case. We plan to add framing format in future, but currently you need to attach your own header, as Daniel said.

claunia referenced this issue

2026-01-29 20:50:28 +00:00

[PR #103] [MERGED] Handle multiple compressed files per original in the test. #708

claunia referenced this issue

2026-01-29 20:50:29 +00:00

[PR #103] Handle multiple compressed files per original in the test. #711

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/brotli#103