Standardize some magic bytes to identify a brotli stream #87

Closed
opened 2026-01-29 20:32:19 +00:00 by claunia · 10 comments
Owner

Originally created by @jrmuizel on GitHub (Jan 15, 2016).

This would be very useful for dealing with something like .tar.br fies.

Originally created by @jrmuizel on GitHub (Jan 15, 2016). This would be very useful for dealing with something like .tar.br fies.
Author
Owner

@eustas commented on GitHub (Jan 18, 2016):

Unfortunately, there is no magic bytes for brotli.
In future brotli streams will be wrapped in some framing format which will have magic bytes that could be used to identify it.

@eustas commented on GitHub (Jan 18, 2016): Unfortunately, there is no magic bytes for brotli. In future brotli streams will be wrapped in some framing format which will have magic bytes that could be used to identify it.
Author
Owner

@eustas commented on GitHub (Jan 18, 2016):

Mark Adler helped us much with proof-testing brotli and refining specification.
Here is his framing format proposal:
https://github.com/madler/brotli/blob/master/br-format-v3.txt

Signature in hexadecimal (four bytes):

ce b2 cf 81

@eustas commented on GitHub (Jan 18, 2016): Mark Adler helped us much with proof-testing brotli and refining specification. Here is his framing format proposal: https://github.com/madler/brotli/blob/master/br-format-v3.txt > Signature in hexadecimal (four bytes): > > ce b2 cf 81
Author
Owner

@catlee commented on GitHub (Jan 19, 2016):

That's very helpful, thank you!

Do you have any idea when the new framing could be introduced? And how to transition content and clients over to the new format?

@catlee commented on GitHub (Jan 19, 2016): That's very helpful, thank you! Do you have any idea when the new framing could be introduced? And how to transition content and clients over to the new format?
Author
Owner

@dsnet commented on GitHub (Jan 20, 2016):

A framing or file format of some type sounds great.

Is there a process through which people are peer reviewing how the new framing format is being designed?

Has any thought been given to possibly re-using a "framing" format like the XZ file format where brotli is just one of the filter chains that can be used?

@dsnet commented on GitHub (Jan 20, 2016): A framing or file format of some type sounds great. Is there a process through which people are peer reviewing how the new framing format is being designed? Has any thought been given to possibly re-using a "framing" format like the [XZ file format](http://tukaani.org/xz/xz-file-format-1.0.4.txt) where brotli is just one of the filter chains that can be used?
Author
Owner

@stokito commented on GitHub (Oct 27, 2018):

Hi, could you add the signature to the file tool?
It looks like you need to add it here:
c7d910ee88/src/compress.c
and here:
c7d910ee88/magic/Magdir/archive

I created a separate ticket for this #727

@stokito commented on GitHub (Oct 27, 2018): Hi, could you add the signature to the `file` tool? It looks like you need to add it here: https://github.com/file/file/blob/c7d910ee8879dfbe80fe14bfc13c8fa6c6fe1d32/src/compress.c and here: https://github.com/file/file/blob/c7d910ee8879dfbe80fe14bfc13c8fa6c6fe1d32/magic/Magdir/archive I created a separate ticket for this #727
Author
Owner

@jason-sachs commented on GitHub (Nov 4, 2020):

This issue was closed in Oct 2016. What was the resolution?

@jason-sachs commented on GitHub (Nov 4, 2020): This issue was closed in Oct 2016. What was the resolution?
Author
Owner

@pavelloz commented on GitHub (Nov 24, 2021):

Im also interested in detecting brotli archives in a convenient way.

@pavelloz commented on GitHub (Nov 24, 2021): Im also interested in detecting brotli archives in a convenient way.
Author
Owner

@matijaskala commented on GitHub (Jan 16, 2022):

I've written a wrapper program around brotli and have been using it locally on my system for quite some time

@matijaskala commented on GitHub (Jan 16, 2022): I've written a [wrapper program](https://github.com/matijaskala/brzip.parallel) around brotli and have been using it locally on my system for quite some time
Author
Owner

@simonmittag commented on GitHub (Jan 23, 2022):

has this ever been implemented? identifying binary streams without relying on external headers (HTTP) is commonplace functionality

@simonmittag commented on GitHub (Jan 23, 2022): has this ever been implemented? identifying binary streams without relying on external headers (HTTP) is commonplace functionality
Author
Owner

@ross-spencer commented on GitHub (Aug 20, 2025):

Looking around (and testing this locally) there doesn't seem to be any resolution, yet multiple workflows rely on basic magic capability, my own use case, digital preservation and archiving where we need to identify file formats and understand their intellectual payload.

Can this issue be reopened?

@ross-spencer commented on GitHub (Aug 20, 2025): Looking around (and testing this locally) there doesn't seem to be any resolution, yet multiple workflows rely on basic magic capability, my own use case, digital preservation and archiving where we need to identify file formats and understand their intellectual payload. Can this issue be reopened?
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/brotli#87