mirror of
https://github.com/google/brotli.git
synced 2026-07-08 17:56:58 +00:00
Brotli v1.1.0 tests fail with pypy3 #478
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @anthrotype on GitHub (Sep 5, 2023).
(moved from https://github.com/google/brotli-wheels/issues/19)
I tried to update the submodule to brotli v1.1.0 in
c7db3350aa(https://github.com/google/brotli-wheels/issues/18), but encountered an issue with pypy. The wheels get built but then when the tests are run, we get these sort of errors:e.g. see CI log https://github.com/google/brotli-wheels/actions/runs/6085235897/job/16508840821#step:5:944
I don't know how to fix that, so for the time being I will simply disable building wheels for pypy...
If anybody would like to help fix this, they'll be welcome to send a PR.
Also note that pypy users may benefit from using the alternative CFFI-based bindings from https://github.com/python-hyper/brotlicffi, whose API is mostly a drop-in replacement for our official brotli bindings.
@mgorny commented on GitHub (Sep 12, 2023):
git bisect points to
c8df4b3049.I'm going to report this to PyPy upstream as well, as that RPython assertion looks suspicious.
Curious enough, with tip of
masterI'm getting a segfault rather than the assertion. I'm going to run a second bisect for that.@mgorny commented on GitHub (Sep 12, 2023):
The segfault is introduced in
acc265655d.@mgorny commented on GitHub (Sep 12, 2023):
CC @animalize, @fischbacher
@mgorny commented on GitHub (Sep 12, 2023):
Here's what I managed to gdb out of the segfault:
I'm going to report the assertion prior to that commit first, and see what comes out of that.
@mgorny commented on GitHub (Sep 12, 2023):
I've filed to PyPy upstream at https://foss.heptapod.net/pypy/pypy/-/issues/4000
@wRAR commented on GitHub (Sep 12, 2023):
We get a suspicious crash in a Scrapy test (which passes garbage into the brotli decompressor among other things), the Python traceback is unhelpful as it's truncated, but the gdb one points to brotli:
It doesn't look like this bug needs more info but if does please let me know!
@ghost commented on GitHub (Sep 13, 2023):
If change these invokings, does it fix the problem?
PyBytes_AS_STRINGtoPyBytes_AsStringPy_SIZEtoPyBytes_Size@mgorny commented on GitHub (Sep 14, 2023):
Unfortunately, the segfault seems to be a heisenbug and I can't reproduce it right now. As for the assertion from pypy3, these changes don't help, though it seems to happen elsewhere. I've pasted the log for that to the PyPy bug; it's failing on:
@mattip commented on GitHub (Jan 24, 2024):
I can reliably reproduce this failure when running the command line from the segfaulting test
The cause of the segfault is in this stanza
buffer->allocated - avail_outis zero, but the list of length 65 has data to be copied. I am not sure where the calculation ofbuffer->allocatedis going wrong@muxator commented on GitHub (Jan 25, 2024):
Here's an example that reliably crashes on brotli v1.1.0 and does not crash on brotli v1.0.9.
I am on Fedora 39. Pypy 3.10 is taken from the system's packages:
Get brotli v1.1.0 from PyPI and compile it. If in this command you replace the version with
1.0.9, there will be no crashes at the subsequent pass:Here's a minimal program that causes the crash:
Crashes with:
View the core dump (I am assuming there where no other crashes in the system after this one):
Which seems to show that the crash is in
BlocksOutputBuffer_Finish()(maybe caused elsewhere):Something similar happens when calling
brotli.decompress().