BrotliDecompressStream not decrementing available_in #126

New Issue

claunia · 2026-01-29T20:36:53Z

claunia commented

2026-01-29 20:36:53 +00:00

Originally created by @jeroen on GitHub (Jul 31, 2016).

For the new version of the R bindings I am switching to BrotliDecompressStream because this is what bro is using so I guess this part of the API should work. Two questions:

What is the difference with BrotliDecoderDecompressStream?
The docs state that:

After each call, available_in will be decremented by the amount of input bytes consumed, and the next_in pointer will be incremented by that amount.

However if I look at my debugging output it seems that the first few iterations available_in is not being decremented, even though total_out is growing and next_in is moving.

brotli_decompress(buf)
available_in:   5355157 - available_out:         0 - total_out:    131072
available_in:   5355157 - available_out:         0 - total_out:    262144
available_in:   5355157 - available_out:         0 - total_out:    524288
available_in:   5355157 - available_out:         0 - total_out:   1048576
available_in:   5355157 - available_out:         0 - total_out:   2097152
available_in:   4670826 - available_out:         0 - total_out:   4194304
available_in:   3603155 - available_out:         0 - total_out:   8388608
available_in:   2321579 - available_out:         0 - total_out:  16777216
available_in:    447663 - available_out:         0 - total_out:  33554432
available_in:         0 - available_out:  21996849 - total_out:  45112015

Originally created by @jeroen on GitHub (Jul 31, 2016). For the new version of the R bindings I am switching to `BrotliDecompressStream` because this is what `bro` is using so I guess this part of the API should work. Two questions: - What is the difference with `BrotliDecoderDecompressStream`? - The docs state that: > After each call, `available_in` will be decremented by the amount of input bytes consumed, and the `next_in` pointer will be incremented by that amount. However if I look at my debugging output it seems that the first few iterations `available_in` is not being decremented, even though `total_out` is growing and `next_in` is moving. ``` r brotli_decompress(buf) available_in: 5355157 - available_out: 0 - total_out: 131072 available_in: 5355157 - available_out: 0 - total_out: 262144 available_in: 5355157 - available_out: 0 - total_out: 524288 available_in: 5355157 - available_out: 0 - total_out: 1048576 available_in: 5355157 - available_out: 0 - total_out: 2097152 available_in: 4670826 - available_out: 0 - total_out: 4194304 available_in: 3603155 - available_out: 0 - total_out: 8388608 available_in: 2321579 - available_out: 0 - total_out: 16777216 available_in: 447663 - available_out: 0 - total_out: 33554432 available_in: 0 - available_out: 21996849 - total_out: 45112015 ```

claunia closed this issue

2026-01-29 20:36:54 +00:00

claunia commented

2026-01-29 20:36:55 +00:00

@eustas commented on GitHub (Aug 3, 2016):

Hello.

BrotliDecompressStream and BrotliDecoderDecompressStream are the same, except the position of "state" argument.

Decoder has internal storage - "ring buffer". In your example it seems to be 4MB, so in given example lines 2-5 do not consume more input, because there is some pending output in "ring buffer". I suppose that the whole compressed file is bigger than 5355157 bytes...

@eustas commented on GitHub (Aug 3, 2016): Hello. BrotliDecompressStream and BrotliDecoderDecompressStream are the same, except the position of "state" argument. Decoder has internal storage - "ring buffer". In your example it seems to be 4MB, so in given example lines 2-5 do not consume more input, because there is some pending output in "ring buffer". I suppose that the whole compressed file is bigger than 5355157 bytes...

claunia commented

2026-01-29 20:36:57 +00:00

@eustas commented on GitHub (Aug 22, 2016):

No feedback for 19 days; feel free to reopen if some concerns still remain.

@eustas commented on GitHub (Aug 22, 2016): _No feedback for 19 days; feel free to reopen if some concerns still remain._

claunia referenced this issue

2026-01-29 20:50:37 +00:00

[PR #126] [CLOSED] update the brotli encoder #737

Sign in to join this conversation.