brotli extension not linked with libstdc++ on pypy3 for Linux #13

New Issue

claunia · 2026-01-29T20:28:55Z

claunia commented

2026-01-29 20:28:55 +00:00

Originally created by @anthrotype on GitHub (Aug 21, 2015).

There is an issue with the Python 3-compatible pypy (aka pypy3) whereby distutils fails to use the proper C++ compiler/linker while compiling C++ extension modules (such as Brotli).

https://bitbucket.org/pypy/pypy/issues/1763/not-using-proper-c-compilers-linker-while

In the Python2.7-compatible pypy, they have fixed this by patching distutils/unixccompiler.py so that it uses c++ as default C++ compiler instead of cc:
c6e45dfbda

However, they haven't ported this patch to pypy3 yet.

At behdad/fonttools, we use Brotli for WOFF2 and we test it on Travis under different python versions, including pypy3.

Because the C++ runtime library isn't being linked when compiling Brotli for pypy3, the module fails to be loaded with undefined symbol error.
For details, see: https://github.com/behdad/fonttools/issues/339

As far as I could test, this problem only occurs on pypy3 for Linux (the Travis python setup runs on Ubuntu 12.04). On OS X, where distutils.unixccompiler is also used, I verified that brotli is compiled and imported without problems when using the same pypy3 version (2.4.0) as the one used on Travis -- but I guess it's because on OSX the name cc is just a symbolic link to clang, and the latter does the right thing.

As a workaround, we currently do something like this:

python setup.py build_ext --libraries=stdc++

While pypy developers fixes the issue, Brotli's setup.py could be modified to link with libstdc++ by default whenever platform.python_implementation() == "PyPy" and sys.version_info[0] == 3. (I wonder if there would be portability issues if libstdc++ were linked all the time, on any python platform/version/implementation?)

A second alternative would be to change the value of UnixCCompiler.compiler_cxx variable from the current ['cc'] to ['c++'] if running on pypy3 (exactly like in the above mentioned pypy2 patch). This is what they do in spaCy for example (I found this by chance).

Now, both approaches seem to work, though I'm not sure which one is preferable.

A third approach, of course, is simply take note and wait.

Anyway, I have mentioned this on pypy's bug tracker, where I found at least two duplicate issues (#1099 and #1763) referencing the same problem.

Please let me know if anyone else has encountered the same problem.
Thanks,

Cosimo

Originally created by @anthrotype on GitHub (Aug 21, 2015). There is an issue with the Python 3-compatible pypy (aka pypy3) whereby distutils fails to use the proper C++ compiler/linker while compiling C++ extension modules (such as Brotli). https://bitbucket.org/pypy/pypy/issues/1763/not-using-proper-c-compilers-linker-while In the Python2.7-compatible pypy, they have fixed this by patching `distutils/unixccompiler.py` so that it uses `c++` as default C++ compiler instead of `cc`: https://bitbucket.org/pypy/pypy/commits/c6e45dfbda905fa9e626782c8d2dd313ff3f54cf However, they haven't ported this patch to pypy3 yet. At [behdad/fonttools](https://github.com/behdad/fonttools), we use Brotli for WOFF2 and we test it on Travis under different python versions, including pypy3. Because the C++ runtime library isn't being linked when compiling Brotli for pypy3, the module fails to be loaded with `undefined symbol` error. For details, see: https://github.com/behdad/fonttools/issues/339 As far as I could test, this problem only occurs on **pypy3 for Linux** (the Travis python setup runs on Ubuntu 12.04). On **OS X**, where `distutils.unixccompiler` is also used, I verified that brotli is compiled and imported without problems when using the same pypy3 version (2.4.0) as the one used on Travis -- but I guess it's because on OSX the name `cc` is just a symbolic link to `clang`, and the latter does the right thing. As a workaround, we currently do something like this: ``` python setup.py build_ext --libraries=stdc++ ``` While pypy developers fixes the issue, Brotli's setup.py could be modified to link with libstdc++ by default whenever `platform.python_implementation() == "PyPy" and sys.version_info[0] == 3`. (I wonder if there would be portability issues if libstdc++ were linked all the time, on any python platform/version/implementation?) A second alternative would be to change the value of `UnixCCompiler.compiler_cxx` variable from the current `['cc']` to `['c++']` if running on pypy3 (exactly like in the above mentioned pypy2 patch). This is what they do in [spaCy](https://github.com/honnibal/spaCy/blob/master/setup.py#L21) for example (I found this by chance). Now, both approaches seem to work, though I'm not sure which one is preferable. A third approach, of course, is simply take note and wait. Anyway, I have mentioned this on pypy's bug tracker, where I found at least two duplicate issues ([#1099](https://bitbucket.org/pypy/pypy/issues/1099/distutils-not-correctly-using-g-cplusplus) and [#1763](https://bitbucket.org/pypy/pypy/issues/1763/not-using-proper-c-compilers-linker-while)) referencing the same problem. Please let me know if anyone else has encountered the same problem. Thanks, Cosimo

claunia closed this issue

2026-01-29 20:28:55 +00:00

claunia commented

2026-01-29 20:28:55 +00:00

@anthrotype commented on GitHub (Aug 22, 2015):

The issue with C++ and pypy3 has already been fixed here.
It should be available in the next pypy3 release.
I shall close this for now.

@anthrotype commented on GitHub (Aug 22, 2015): The issue with C++ and pypy3 has already been fixed [here](https://bitbucket.org/pypy/pypy/issues/1763/not-using-proper-c-compilers-linker-while#comment-21101778). It should be available in the next pypy3 release. I shall close this for now.

claunia referenced this issue

2026-01-29 20:46:17 +00:00

[PR #13] [MERGED] Fix potential output buffer overflow in encoder. #589

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/brotli#13