mirror of
https://github.com/google/brotli.git
synced 2026-07-08 17:56:58 +00:00
Streamed decoding only provides unneccessarily huge chunks #80
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @mbevin on GitHub (Dec 18, 2015).
When streaming encoding, it's working fine using small buffer-sizes (indeed only does so), thus presumably it should be able to decode in similarly small chunks, however the streaming decoder seems to only return 4.19mb chunks (i.e. i'm streaming data to BrotliDecompressBufferStreaming in 16kb chunks, but it only first returns anything whenever it has accumulated 4.19mb available).
Given that this seems unneccessary (since the data was streamed on write in far smaller chunks), it would be very preferable to be able to have it return more frequent + smaller chunk sizes, as this kind of delay+chunkiness makes it unusable for some streaming scenarios, and less ideal for others.
@eustas commented on GitHub (Dec 18, 2015):
Decoder dumps all the output is has after it consumed the next chunk of input: https://github.com/google/brotli/blob/master/dec/decode.c#L1954
Please, could you provide sample data and code, so we could investigate what is going wrong?
@mbevin commented on GitHub (Dec 21, 2015):
Ok, looks like you fixed it since I originally discovered the issue (Oct 21), as pulling the latest from master now fixed the issue (average chunk size dropped from that very consistent 4.19mb to ~31kb. I originally reported it in the forum, but then getting no response there discovered this place after a month or so - next time I'll post here I guess :)