mirror of
https://github.com/google/brotli.git
synced 2026-09-22 06:35:52 +00:00
BrotliDecoderDecompressStream returns inconsistent result code when decoding Youtube main page #322
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @sfionov on GitHub (May 19, 2020).
brotli CLI fails to decode this page (dump from https://youtube.com/)
youtube-page.brotli.zip
At some blocks, BrotliDecoderDecompressStream returns NEEDS_MORE_INPUT instead of NEEDS_MORE_OUTPUT.
As result, any code that use decode result for iterations can't decode this page.
But code that instead iterates until lack of progress or fatal error, can decode this page without problems.
So, Chrome doesn't show decode error on this page but brotli CLI does.
@eustas commented on GitHub (May 19, 2020):
Could you share raw (not compressed with brotli) file as well. Thanks.
@sfionov commented on GitHub (May 19, 2020):
Of course. Here it is:
youtube-page.html.zip
@eustas commented on GitHub (May 19, 2020):
Did you get compressed page via wget / curl?
Can not find options to compress page back to the same size.
Quality 5 gives 68199 bytes, quality 4 gives 75180.
Modified CLI to pull output anyways and decoding still fails.
Looking at decoder state at moment when it finally reports "needs more input" - it is just between blocks. Looks like compressor has flushed, but did not finalize file.
If so, the it is the problem in YouTube. Perhaps we could add some flags that could allow CLI to decode such unfinished files (e.g. keep output even in case of "corrupted" input).
To "finalize" the file try
echo -n -e '\x03' >> youtube-page.brotli. After this CLI will be able to decompress the file.@sfionov commented on GitHub (May 19, 2020):
This file is downloaded via curl without --compressed flag, so it is compressed by YouTube. Seems that they enabled new experimental UI on some accounts, and it has this issue. :(
Yes, "finalized" file is decompressed without problems.