Fix native dependency for XZ compression #3

Open
opened 2026-01-29 16:26:51 +00:00 by claunia · 6 comments
Owner

Originally created by @qmfrederik on GitHub (Sep 26, 2017).

RPM packages use xz compression for compressing the package payload.

Currently XZ compression is implemented via the XZOutputStream which uses liblzma under the hoods to perform the compression.

This P/Invokes into liblzma. Different Linux distributions may ship different versions of liblizma, so loading the correct version of liblzma may be a bit of a challenge. For example, on RedHad, the so name is liblzma.so.5.

We should either amend the code that loads liblzma to search for different versions of liblzma (possibly depending on the host OS); or replace liblzma by a managed implementation.

Originally created by @qmfrederik on GitHub (Sep 26, 2017). RPM packages use xz compression for compressing the package payload. Currently XZ compression is implemented via the [XZOutputStream](https://github.com/qmfrederik/dotnet-packaging/blob/358567e85fa0a9372327e1f0ca9569d45cb726d3/Packaging.Targets/IO/XZOutputStream.cs) which uses liblzma under the hoods to perform the compression. This P/Invokes into liblzma. Different Linux distributions may ship different versions of liblizma, so loading the correct version of liblzma may be a bit of a challenge. For example, on RedHad, the so name is `liblzma.so.5`. We should either amend [the code that loads liblzma](https://github.com/qmfrederik/dotnet-packaging/blob/133ffd8883621cb83de2887364ee67937ca8824c/Packaging.Targets/IO/NativeMethods.cs#L43:L55) to search for different versions of liblzma (possibly depending on the host OS); or replace liblzma by a managed implementation.
claunia added the up-for-grabs label 2026-01-29 16:26:51 +00:00
Author
Owner

@qmfrederik commented on GitHub (Sep 26, 2017):

A cursory search on GitHub and NuGet didn't reveal any fully-managed xz compression implementations (there are some decompressors, though). So we may be stuck with the native dependency for now.

The better option the may be to amend NativeMethods so that it searchers for different versions of liblzma, and possible also set up Travis support for running some tests in a CentOS container to excercise this code.

133ffd8883/Packaging.Targets/IO/NativeMethods.cs (L43-L55)

@qmfrederik commented on GitHub (Sep 26, 2017): A cursory search on GitHub and NuGet didn't reveal any fully-managed xz compression implementations (there are some decompressors, though). So we may be stuck with the native dependency for now. The better option the may be to amend NativeMethods so that it searchers for different versions of liblzma, and possible also set up Travis support for running some tests in a CentOS container to excercise this code. https://github.com/qmfrederik/dotnet-packaging/blob/133ffd8883621cb83de2887364ee67937ca8824c/Packaging.Targets/IO/NativeMethods.cs#L43-L55
Author
Owner

@qmfrederik commented on GitHub (Sep 26, 2017):

/cc @tmds split off from #6

@qmfrederik commented on GitHub (Sep 26, 2017): /cc @tmds split off from #6
Author
Owner

@kekekeks commented on GitHub (Oct 2, 2017):

Have you considered building liblzma with some ancient version of glibc (so library will work across all glibc-based distros) and shipping it inside NuGet package? I can do that, if you want.

@kekekeks commented on GitHub (Oct 2, 2017): Have you considered building liblzma with some ancient version of glibc (so library will work across all glibc-based distros) and shipping it inside NuGet package? I can do that, if you want.
Author
Owner

@qmfrederik commented on GitHub (Oct 2, 2017):

@kekekeks This would create a "portable" version of liblzma, pretty much like .NET Core is now portable accross glibc-based distributions, right?

That sounds like a good plan, it helps us work around the requirement to have the user install liblzma.

Only requirement I'd have is that the build process is open & reproducible (ideally via travis or a similar service). I can configure Travis to build as part of this repository, if that helps

@qmfrederik commented on GitHub (Oct 2, 2017): @kekekeks This would create a "portable" version of `liblzma`, pretty much like .NET Core is now portable accross glibc-based distributions, right? That sounds like a good plan, it helps us work around the requirement to have the user install liblzma. Only requirement I'd have is that the build process is open & reproducible (ideally via travis or a similar service). I can configure Travis to build as part of this repository, if that helps
Author
Owner

@kekekeks commented on GitHub (Oct 2, 2017):

is that the build process is open & reproducible

Is a Dockerfile for creating a builder image OK?

@kekekeks commented on GitHub (Oct 2, 2017): >is that the build process is open & reproducible Is a `Dockerfile` for creating a builder image OK?
Author
Owner

@qmfrederik commented on GitHub (Oct 2, 2017):

Is a Dockerfile for creating a builder image OK?

Absolutely!

@qmfrederik commented on GitHub (Oct 2, 2017): > Is a Dockerfile for creating a builder image OK? Absolutely!
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/dotnet-packaging#3