decompressed chunks are incomplete #95

Closed
opened 2026-01-29 20:33:05 +00:00 by claunia · 4 comments
Owner

Originally created by @mshneer on GitHub (Mar 7, 2016).

I have written a simple test (https://gist.github.com/mshneer/a9b442539917842bc0ef) that breaks HTML into chunks, then compresses, then decompressed. Somehow, there is a difference between chunks that come in and chunks that come out. My understanding that calling WriteBrotliData(... , */* force_flush / true, ...) should not have this effect.

I was also testing against 0.3.0 release.

Any ideas how to cause the compressor to not buffer things like this?

Originally created by @mshneer on GitHub (Mar 7, 2016). I have written a simple test (https://gist.github.com/mshneer/a9b442539917842bc0ef) that breaks HTML into chunks, then compresses, then decompressed. Somehow, there is a difference between chunks that come in and chunks that [come out](https://phabricator.fb.com/P56188426). My understanding that calling _WriteBrotliData(... , **/\* force_flush */ true,** ...)_ should not have this effect. I was also testing against 0.3.0 release. Any ideas how to cause the compressor to not buffer things like this?
Author
Owner

@KenjiBaheux commented on GitHub (Mar 7, 2016):

Thanks for filing this issue.
cc/ @jyrkialakuijala @eustas

@KenjiBaheux commented on GitHub (Mar 7, 2016): Thanks for filing this issue. cc/ @jyrkialakuijala @eustas
Author
Owner

@eustas commented on GitHub (Mar 8, 2016):

Hello.

After you call "BrotliDecompressBufferStreaming" compressedBytes, most likely, isn't zero. This is because "flush" flag of WriteBrotliData doesn't mean "hard flush". To make a "hard flush" you need to Write empty metablock after flushed data block.

Sorry for the unclear API documentation. Will fix it soon.

@eustas commented on GitHub (Mar 8, 2016): Hello. After you call "BrotliDecompressBufferStreaming" compressedBytes, most likely, isn't zero. This is because "flush" flag of WriteBrotliData doesn't mean "hard flush". To make a "hard flush" you need to Write empty metablock after flushed data block. Sorry for the unclear API documentation. Will fix it soon.
Author
Owner

@eustas commented on GitHub (Mar 8, 2016):

s/empty metablock/empty metadata block/

@eustas commented on GitHub (Mar 8, 2016): s/empty metablock/empty metadata block/
Author
Owner

@eustas commented on GitHub (Jun 17, 2016):

Please consider using our new streaming API: BrotliEncoderCompressStream with op=BROTLI_OPERATION_FLUSH does exactly what is expected.
Sorry for the inconvenience and thank you for your interest to brotli.

@eustas commented on GitHub (Jun 17, 2016): Please consider using our new streaming API: `BrotliEncoderCompressStream` with op=`BROTLI_OPERATION_FLUSH` does exactly what is expected. Sorry for the inconvenience and thank you for your interest to brotli.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/brotli#95