Incorrect compressed output from Android build #287

New Issue

claunia · 2026-01-29T20:41:24Z

claunia commented

2026-01-29 20:41:24 +00:00

Originally created by @mhsmith on GitHub (Aug 25, 2019).

I'm trying to make a Python build of Brotli for Chaquopy on Android (https://github.com/chaquo/chaquopy/issues/137). The build process itself is no problem (I'm using the 1.0.7 sdist ZIP from PyPI), but the resulting package doesn't work correctly.

With the official Linux build from PyPI:

>>> plain = b"it was the best of times, it was the worst of times"
>>> compressed = brotli.compress(plain)
>>> compressed.hex()
'1b3200e015a1d23bed4dee0e4ce480bd2df5f0808f5a17442f98540f37f0a583fe06'
>>> brotli.decompress(compressed)
b'it was the best of times, it was the worst of times'

With my own Android build:

>>> compressed.hex()
'1b3200e015a1d29df626770726728f5a17442f98540f37f0a583fe06'

>>> brotli.decompress(compressed)
Traceback (most recent call last):
   File "<console>", line 1, in <module>
brotli.error: BrotliDecompress failed

Notice that the invalid output is the same at the beginning and end, but the middle section is both different and shorter. I get the same output on both x86 and arm64-v8a.

Can anyone suggest what might be going on here?

Originally created by @mhsmith on GitHub (Aug 25, 2019). I'm trying to make a Python build of Brotli for Chaquopy on Android (https://github.com/chaquo/chaquopy/issues/137). The build process itself is no problem (I'm using the 1.0.7 sdist ZIP from PyPI), but the resulting package doesn't work correctly. With the official Linux build from PyPI: ``` >>> plain = b"it was the best of times, it was the worst of times" >>> compressed = brotli.compress(plain) >>> compressed.hex() '1b3200e015a1d23bed4dee0e4ce480bd2df5f0808f5a17442f98540f37f0a583fe06' >>> brotli.decompress(compressed) b'it was the best of times, it was the worst of times' ``` With my own Android build: ``` >>> compressed.hex() '1b3200e015a1d29df626770726728f5a17442f98540f37f0a583fe06' >>> brotli.decompress(compressed) Traceback (most recent call last): File "<console>", line 1, in <module> brotli.error: BrotliDecompress failed ``` Notice that the invalid output is the same at the beginning and end, but the middle section is both different and shorter. I get the same output on both x86 and arm64-v8a. Can anyone suggest what might be going on here?

claunia closed this issue

2026-01-29 20:41:24 +00:00

claunia commented

2026-01-29 20:41:25 +00:00

@mhsmith commented on GitHub (Aug 25, 2019):

I should mention, I'm using the Crystax NDK with a relatively old compiler (GCC 4.9.3).

@mhsmith commented on GitHub (Aug 25, 2019): I should mention, I'm using the Crystax NDK with a relatively old compiler (GCC 4.9.3).

claunia commented

2026-01-29 20:41:25 +00:00

@mhsmith commented on GitHub (Aug 27, 2019):

I eventually tracked this down to BROTLI_RESTRICT. With that macro disabled, compression worked correctly on all ABIs.

However, decompression still gave the same error on arm64-v8a until I added BROTLI_BUILD_NO_RBIT. My test device was a Nexus 5X (Qualcomm Snapdragon 808, 2x Cortex-A57, 4x Cortex-A53), but I suspect the compiler is at fault here.

This compiler version is no longer supported on Android, and I'll be moving off it myself as soon as possible. So there's no need to take any further action on this, unless you want to.

@mhsmith commented on GitHub (Aug 27, 2019): I eventually tracked this down to `BROTLI_RESTRICT`. With that macro disabled, compression worked correctly on all ABIs. However, decompression still gave the same error on arm64-v8a until I added `BROTLI_BUILD_NO_RBIT`. My test device was a Nexus 5X (Qualcomm Snapdragon 808, 2x Cortex-A57, 4x Cortex-A53), but I suspect the compiler is at fault here. This compiler version is no longer supported on Android, and I'll be moving off it myself as soon as possible. So there's no need to take any further action on this, unless you want to.

claunia commented

2026-01-29 20:41:25 +00:00

@eustas commented on GitHub (Aug 27, 2019):

Thanks for the investigation.
Still makes sense to:

warn developers about possible problems
see what is wrong with restrict - this might mean that something is marked wrong, i.e. two areas could intersect
add unit test that checks that rbit works well if compiler thinks it should

@eustas commented on GitHub (Aug 27, 2019): Thanks for the investigation. Still makes sense to: * warn developers about possible problems * see what is wrong with `restrict` - this might mean that something is marked wrong, i.e. two areas could intersect * add unit test that checks that `rbit` works well if compiler thinks it should

claunia commented

2026-01-29 20:41:25 +00:00

@jinfeihan57 commented on GitHub (Nov 14, 2019):

GCC 4.8 is not working .
The compiler version is too low, try a newer version.
GCC 5.4 is OK.
or try a cross compiler。

@jinfeihan57 commented on GitHub (Nov 14, 2019): GCC 4.8 is not working . The compiler version is too low, try a newer version. GCC 5.4 is OK. or try a cross compiler。

claunia commented

2026-01-29 20:41:25 +00:00

@mhsmith commented on GitHub (Jan 31, 2020):

I tried building brotli 1.0.7 using the Google NDK r18 (Clang version 7.0.2), and it produces the correct output on all ABIs, without needing any of the above changes. So I think that confirms it was the Crystax NDK that was at fault.

@mhsmith commented on GitHub (Jan 31, 2020): I tried building brotli 1.0.7 using the Google NDK r18 (Clang version 7.0.2), and it produces the correct output on all ABIs, without needing any of the above changes. So I think that confirms it was the Crystax NDK that was at fault.

claunia commented

2026-01-29 20:41:26 +00:00

@eustas commented on GitHub (Jul 2, 2020):

@mhsmith thanks for investigating.

@eustas commented on GitHub (Jul 2, 2020): @mhsmith thanks for investigating.

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/brotli#287