Streamed decoding only provides unneccessarily huge chunks #77

Open
opened 2026-01-29 20:32:00 +00:00 by claunia · 0 comments
Owner

Originally created by @mbevin on GitHub (Dec 18, 2015).

When streaming encoding, it's working fine using small buffer-sizes (indeed only does so), thus presumably it should be able to decode in similarly small chunks, however the streaming decoder seems to only return 4.19mb chunks (i.e. i'm streaming data to BrotliDecompressBufferStreaming in 16kb chunks, but it only first returns anything whenever it has accumulated 4.19mb available).

Given that this seems unneccessary (since the data was streamed on write in far smaller chunks), it would be very preferable to be able to have it return more frequent + smaller chunk sizes, as this kind of delay+chunkiness makes it unusable for some streaming scenarios, and less ideal for others.

Originally created by @mbevin on GitHub (Dec 18, 2015). When streaming encoding, it's working fine using small buffer-sizes (indeed only does so), thus presumably it should be able to decode in similarly small chunks, however the streaming decoder seems to only return 4.19mb chunks (i.e. i'm streaming data to BrotliDecompressBufferStreaming in 16kb chunks, but it only first returns anything whenever it has accumulated 4.19mb available). Given that this seems unneccessary (since the data was streamed on write in far smaller chunks), it would be very preferable to be able to have it return more frequent + smaller chunk sizes, as this kind of delay+chunkiness makes it unusable for some streaming scenarios, and less ideal for others.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/brotli#77