pre-compiled wheels for linux? #157

New Issue

claunia · 2026-01-29T20:38:58Z

claunia commented

2026-01-29 20:38:58 +00:00

Originally created by @anthrotype on GitHub (Feb 9, 2017).

The python bindings are currently provided as wheel packages on Github and PyPI, but only for macOS and Windows, and for python2.7, 3.4, and 3.5. For other platforms like Linux, a source distribution is provided, but this of course requires users to have a C compiler.

I would like to help setting up Travis CI to also build manylinux1 wheels which are portable across most Linux distributions.

I've done this for several other projects using multibuild, a set of shell scripts originally developed for the Python scientific stack (numpy, scipy, etc.), which makes super easy to set up Travis/Appveyor as wheel builders for Linux, macOS and Windows.

Before I proceed working on a pull request, I first wanted to gauge interest and ask for comments.

We have two options:

add multibuild to the google/brotli repo and configure Travis with additional jobs to also build the Linux wheels.
create a separate google/brotli-wheels repository with its own Travis/Appveyor CI configuration, which would clone the upstream repo as a git submodule, and would only trigger new builds when the submodule is manually updated, e.g. before a new release.

The latter is also the approach recommended in multibuild's README, and used by projects like https://github.com/MacPython/numpy-wheels, https://github.com/MacPython/matplotlib-wheels
and others.

I'd tend to opt for the second approach, because I see that brotli's Travis matrix is already quite crowded, and adding support for manylinux1 would add at least 8 new jobs (4 jobs cover all the python2.7, 32 vs 64 bit, "narrow" vs "wide" unicode build; plus 4 more to cover 3.5 and 3.6, 32 and 64 bit), thus further slowing it down.

What do you think?

Originally created by @anthrotype on GitHub (Feb 9, 2017). The python bindings are currently provided as wheel packages on [Github](https://github.com/google/brotli/releases/tag/v0.5.2) and [PyPI](https://pypi.python.org/pypi/Brotli/0.5.2), but only for macOS and Windows, and for python2.7, 3.4, and 3.5. For other platforms like Linux, a source distribution is provided, but this of course requires users to have a C compiler. I would like to help setting up Travis CI to also build [`manylinux1`](https://www.python.org/dev/peps/pep-0513/) wheels which are portable across most Linux distributions. I've done this for several other projects using [`multibuild`](https://github.com/matthew-brett/multibuild), a set of shell scripts originally developed for the Python scientific stack (numpy, scipy, etc.), which makes super easy to set up Travis/Appveyor as wheel builders for Linux, macOS and Windows. Before I proceed working on a pull request, I first wanted to gauge interest and ask for comments. We have two options: 1) add `multibuild` to the `google/brotli` repo and configure Travis with additional jobs to also build the Linux wheels. 2) create a separate `google/brotli-wheels` repository with its own Travis/Appveyor CI configuration, which would clone the upstream repo as a git submodule, and would only trigger new builds when the submodule is manually updated, e.g. before a new release. The latter is also the approach recommended in multibuild's README, and used by projects like https://github.com/MacPython/numpy-wheels, https://github.com/MacPython/matplotlib-wheels and others. I'd tend to opt for the second approach, because I see that brotli's Travis matrix is already quite crowded, and adding support for manylinux1 would add at least 8 new jobs (4 jobs cover all the python2.7, 32 vs 64 bit, "narrow" vs "wide" unicode build; plus 4 more to cover 3.5 and 3.6, 32 and 64 bit), thus further slowing it down. What do you think?

claunia closed this issue

2026-01-29 20:38:58 +00:00

claunia commented

2026-01-29 20:38:59 +00:00

@anthrotype commented on GitHub (Mar 1, 2017):

ping @szabadka

I can volunteer to maintain this brotli-wheels repository, and set it up to build wheels when the upstream brotli is released, and have it upload them automatically to PyPI.

I could create the new repository under my Github username, or if you wish, you can create it under the google Github organization and add me as collaborator for that repo.

I would also require access to the brotli PyPI account.

@anthrotype commented on GitHub (Mar 1, 2017): ping @szabadka I can volunteer to maintain this `brotli-wheels` repository, and set it up to build wheels when the upstream `brotli` is released, and have it upload them automatically to PyPI. I could create the new repository under my Github username, or if you wish, you can create it under the `google` Github organization and add me as collaborator for that repo. I would also require access to the `brotli` PyPI account.

claunia commented

2026-01-29 20:38:59 +00:00

@szabadka commented on GitHub (Mar 1, 2017):

Having a separate brotli-wheels looks like a good solution for this. How
can I give access to the brotli PyPI account?

On Wed, Mar 1, 2017 at 1:55 PM, Cosimo Lupo notifications@github.com
wrote:

ping @szabadka https://github.com/szabadka

I can volunteer to maintain this brotli-wheels repository, and set it up
to build wheels when the upstream brotli is released, and have it upload
them automatically to PyPI.

I could create the new repository under my Github username, or if you
wish, you can create it under the google Github organization and add me
as collaborator for that repo.

I would also require access to the brotli PyPI account.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/google/brotli/issues/507#issuecomment-283332384, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AIp1d6C0bMkv0uqXCT8Hb1Cc8YhU3qxiks5rhWqmgaJpZM4L7_ED
.

@szabadka commented on GitHub (Mar 1, 2017): Having a separate brotli-wheels looks like a good solution for this. How can I give access to the brotli PyPI account? On Wed, Mar 1, 2017 at 1:55 PM, Cosimo Lupo <notifications@github.com> wrote: > ping @szabadka <https://github.com/szabadka> > > I can volunteer to maintain this brotli-wheels repository, and set it up > to build wheels when the upstream brotli is released, and have it upload > them automatically to PyPI. > > I could create the new repository under my Github username, or if you > wish, you can create it under the google Github organization and add me > as collaborator for that repo. > > I would also require access to the brotli PyPI account. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/google/brotli/issues/507#issuecomment-283332384>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AIp1d6C0bMkv0uqXCT8Hb1Cc8YhU3qxiks5rhWqmgaJpZM4L7_ED> > . >

claunia commented

2026-01-29 20:38:59 +00:00

@eustas commented on GitHub (Mar 1, 2017):

Hello. Sorry, this request has fallen out of my view.
I'll give access to PyPI account soon.

@eustas commented on GitHub (Mar 1, 2017): Hello. Sorry, this request has fallen out of my view. I'll give access to PyPI account soon.

claunia commented

2026-01-29 20:39:00 +00:00

@anthrotype commented on GitHub (Mar 1, 2017):

you need to add me as collaborator in the "roles" section on the brotli PyPI page:

https://pypi.python.org/pypi?:action=role_form&package_name=brotli

My pypi username is also anthrotype.

@anthrotype commented on GitHub (Mar 1, 2017): you need to add me as collaborator in the "roles" section on the brotli PyPI page: https://pypi.python.org/pypi?:action=role_form&package_name=brotli My pypi username is also `anthrotype`.

claunia commented

2026-01-29 20:39:00 +00:00

@eustas commented on GitHub (Mar 1, 2017):

Going to investigate what bureaucracy needs to be done to create google/brotli-wheels repo.

@eustas commented on GitHub (Mar 1, 2017): Going to investigate what bureaucracy needs to be done to create google/brotli-wheels repo.

claunia commented

2026-01-29 20:39:00 +00:00

@anthrotype commented on GitHub (Mar 1, 2017):

Thank you!

@anthrotype commented on GitHub (Mar 1, 2017): Thank you!

claunia commented

2026-01-29 20:39:01 +00:00