Support for free-threaded Python #530

New Issue

claunia · 2026-01-29T20:45:14Z

claunia commented

2026-01-29 20:45:14 +00:00

Originally created by @lysnikolaou on GitHub (Jan 10, 2025).

Hi everyone! 👋

I'm opening an issue first of all to ask whether there's somebody that's already working on free-threading support. If not, I would like to help as much as I can. The steps are roughly the following:

Set up CI for free-threading (Not sure if this is relevant here, since AFAIU brotli does not exhaustively test different Python versions)
Audit C extension module for thread-safety issues
Mark C extension module as thread-safe with PyUnstable_Module_SetGIL
Build & upload nightly wheels for the free-threaded build (This includes some minor changes over at google/brotli-wheels, but those shouldn't be complicated at all)

Originally created by @lysnikolaou on GitHub (Jan 10, 2025). Hi everyone! 👋 I'm opening an issue first of all to ask whether there's somebody that's already working on free-threading support. If not, I would like to help as much as I can. The steps are roughly the following: - [ ] Set up CI for free-threading (Not sure if this is relevant here, since AFAIU brotli does not exhaustively test different Python versions) - [ ] Audit C extension module for thread-safety issues - [ ] Mark C extension module as thread-safe with `PyUnstable_Module_SetGIL` - [ ] Build & upload nightly wheels for the free-threaded build (This includes some minor changes over at google/brotli-wheels, but those shouldn't be complicated at all)

claunia commented

2026-01-29 20:45:15 +00:00

@lysnikolaou commented on GitHub (Jan 13, 2025):

I've looked a bit deeper into this and it looks like there's no global state n the Python side that we need to lock around. However, it should be mentioned that Compressor, Decompressor instances cannot be shared across threads because of #501. There's two options we could go with here:

Explicitly state that instances of these two classes cannot be shared between threads and be done with it. In that case, marking the extension as free-threading compatible would be the only remaining thing.
Lock around calls to the C layer. We could use the PyObject-specific PyMutex for that and it should work okay. I can look into that if people think that's the best option.

@lysnikolaou commented on GitHub (Jan 13, 2025): I've looked a bit deeper into this and it looks like there's no global state n the Python side that we need to lock around. However, it should be mentioned that `Compressor`, `Decompressor` instances cannot be shared across threads because of #501. There's two options we could go with here: - Explicitly state that instances of these two classes cannot be shared between threads and be done with it. In that case, marking the extension as free-threading compatible would be the only remaining thing. - Lock around calls to the C layer. We could use the `PyObject`-specific `PyMutex` for that and it should work okay. I can look into that if people think that's the best option.

claunia commented

2026-01-29 20:45:15 +00:00

@lysnikolaou commented on GitHub (Jan 16, 2025):

A third option that was pointed out to me by @ngoldbaum, which is what python-zstandard does as well, is to add an atomic flag to the Python-level object that signifies that the compressor/decompressor instance is in use by a thread, and then raise an error if another thread tries to use it as well.

@lysnikolaou commented on GitHub (Jan 16, 2025): A third option that was pointed out to me by @ngoldbaum, which is what python-zstandard does as well, is to add an atomic flag to the Python-level object that signifies that the compressor/decompressor instance is in use by a thread, and then raise an error if another thread tries to use it as well.

claunia commented

2026-01-29 20:45:15 +00:00

@ngoldbaum commented on GitHub (Jan 16, 2025):

which is what python-zstandard does as well

Not quite, there's only an open PR. We're still waiting to hear back from the python-zstandard maintainer. Which reminds me, I should give them a ping...

@ngoldbaum commented on GitHub (Jan 16, 2025): > which is what python-zstandard does as well Not quite, there's only an open PR. We're still waiting to hear back from the python-zstandard maintainer. Which reminds me, I should give them a ping...

claunia commented

2026-01-29 20:45:16 +00:00

@lysnikolaou commented on GitHub (Jan 16, 2025):

Not quite, there's only an open PR.

Right! That's what I meant to say. Thanks!

@lysnikolaou commented on GitHub (Jan 16, 2025): > Not quite, there's only an open PR. Right! That's what I meant to say. Thanks!

claunia commented

2026-01-29 20:45:16 +00:00

@laggron42 commented on GitHub (Sep 30, 2025):

Hi! With 3.14 around the corner and full support for free-threaded, is there news on this?

@laggron42 commented on GitHub (Sep 30, 2025): Hi! With 3.14 around the corner and full support for free-threaded, is there news on this?

claunia commented

2026-01-29 20:45:16 +00:00

@eustas commented on GitHub (Sep 30, 2025):

Latest revision should be thread-safe. Soon we will land multi-phase initialization.

@eustas commented on GitHub (Sep 30, 2025): Latest revision should be thread-safe. Soon we will land multi-phase initialization.

claunia commented

2026-01-29 20:45:16 +00:00

@eustas commented on GitHub (Oct 1, 2025):

Let's postpone this untill 1.2.0 released. Right after release I'm going to drop Py support below 3.6, thus we could start growing ifdefs anew.

@eustas commented on GitHub (Oct 1, 2025): Let's postpone this untill 1.2.0 released. Right after release I'm going to drop Py support below 3.6, thus we could start growing ifdefs anew.

claunia commented

2026-01-29 20:45:17 +00:00

@Dhairya3391 commented on GitHub (Oct 31, 2025):

Hey, since 1.2 is out now, any update on free-threaded Python support? I was experimenting with it locally on 3.14 and made a few small changes to get it working, but it broke... probably because I’m still figuring out how it behaves with the new free-threaded build. Just curious if this is planned soon.

@Dhairya3391 commented on GitHub (Oct 31, 2025): Hey, since 1.2 is out now, any update on free-threaded Python support? I was experimenting with it locally on 3.14 and made a few small changes to get it working, but it broke... probably because I’m still figuring out how it behaves with the new free-threaded build. Just curious if this is planned soon.

claunia commented

2026-01-29 20:45:17 +00:00

@cclauss commented on GitHub (Nov 4, 2025):

A rapid local demonstration that import brotli reenables the GIL on free-threaded Python 3.14t.

% uvx --with=brotli python3.14t -c "import brotli"

Built brotli==1.1.0
    Installed 1 package in 2ms
<frozen importlib._bootstrap>:491: RuntimeWarning: The global interpreter lock (GIL) has been enabled to 
    load module '_brotli', which has not declared that it can run safely without the GIL.
    To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0.

@cclauss commented on GitHub (Nov 4, 2025): A rapid local demonstration that `import brotli` reenables the GIL on free-threaded Python 3.14t. % `uvx --with=brotli python3.14t -c "import brotli"` ``` Built brotli==1.1.0 Installed 1 package in 2ms <frozen importlib._bootstrap>:491: RuntimeWarning: The global interpreter lock (GIL) has been enabled to load module '_brotli', which has not declared that it can run safely without the GIL. To override this behavior and keep the GIL disabled (at your own risk), run with PYTHON_GIL=0 or -Xgil=0. ```

claunia commented

2026-01-29 20:45:17 +00:00

@ngoldbaum commented on GitHub (Nov 12, 2025):

x-ref https://github.com/google/brotli/pull/1386, which will hopefully close this

@ngoldbaum commented on GitHub (Nov 12, 2025): x-ref https://github.com/google/brotli/pull/1386, which will hopefully close this

claunia referenced this issue

2026-01-29 20:52:51 +00:00

[PR #530] [MERGED] Actually publich artifacts to Bintray #1177

claunia referenced this issue

2026-01-29 20:52:51 +00:00

[PR #530] Actually publich artifacts to Bintray #1181

Sign in to join this conversation.