brotli CLI: how to stop continuous compression and write valid file? #531

New Issue

claunia · 2026-01-29T20:45:17Z

claunia commented

2026-01-29 20:45:17 +00:00

Originally created by @derhuerst on GitHub (Dec 3, 2024).

I'm trying to compress a the data from continuous stream of logs for a certain period of time. At some point, I want the brotli CLI to stop compressing and write a clean/valid brotli-compressed file.

bash -c 'while true; do echo "some log line"; sleep 1; done' | brotli >logs.txt.br
# After some time, I send SIGINT via ctrl + c.
# ^C

# Once the `brotli` process has exited, I would expect to be able to read the compressed file. However, this is not possible:
brotli -d foo.txt.br | wc -l
# corrupt input [foo.txt.br]
#        0

This also doesn't work with SIGQUIT, SIGTERM and SIGHUP.

Originally created by @derhuerst on GitHub (Dec 3, 2024). I'm trying to compress a the data from continuous stream of logs for a certain period of time. At some point, I want the `brotli` CLI to stop compressing and write a clean/valid brotli-compressed file. ```shell bash -c 'while true; do echo "some log line"; sleep 1; done' | brotli >logs.txt.br # After some time, I send SIGINT via ctrl + c. # ^C # Once the `brotli` process has exited, I would expect to be able to read the compressed file. However, this is not possible: brotli -d foo.txt.br | wc -l # corrupt input [foo.txt.br] # 0 ``` This also doesn't work with `SIGQUIT`, `SIGTERM` and `SIGHUP`.

claunia closed this issue

2026-01-29 20:45:17 +00:00

claunia commented

2026-01-29 20:45:17 +00:00

@eustas commented on GitHub (May 27, 2025):

brotli waits for end-of-stream (Ctrl-D).
We currently do not set interruption handlers in CLI. You could turn this issue to feature-request if there is no workaround to emulate end-of-stream.

@eustas commented on GitHub (May 27, 2025): `brotli` waits for end-of-stream (`Ctrl-D`). We currently do not set interruption handlers in CLI. You could turn this issue to feature-request if there is no workaround to emulate end-of-stream.

claunia commented

2026-01-29 20:45:17 +00:00

@derhuerst commented on GitHub (Oct 23, 2025):

if there is no workaround to emulate end-of-stream.

Actually, I found one: AFAICT, if you run a pipeline, typing Ctrl+C will send SIGINT to all of its processes. A workaround would be to "shield" the brotli process from the SIGINT, letting it work until it hits EOF.

As documented in a StackOverflow answer, this can be achieved with a subshell and trap '' INT:

$ for i in $(seq 1 10); do 1>&2 echo "$i"; for j in $(seq 1 1000); do echo "$j"; done; sleep 1; done | (trap '' INT; brotli >/tmp/foo.txt.br)
# 1
# 2
# 3
^C
$ ls -lh /tmp/foo.txt.br 
# -rw-r--r--  1 j  wheel   1.3K Oct 23 15:15 /tmp/foo.txt.br
$ for i in $(seq 1 10); do 1>&2 echo "$i"; for j in $(seq 1 1000); do echo "$j"; done; sleep 1; done | brotli >/tmp/foo.txt.br
# 1
# 2
# 3
^C
$ ls -lh /tmp/foo.txt.br 
# -rw-r--r--  1 j  wheel     0B Oct 23 15:15 /tmp/foo.txt.br

However, I'd argue that

this is not trivial to find out: With non-expert knowledge of shell/POSIX APIs, it seems unlikely that someone will come up with that idea by themselves. Search for a solution is not exactly because there are lots of related-sounding-but-different questions/answers about pipelines in shells.
even outside the interactive shell use case, I'd argue that having the option to compress & flush the current input upon Ctrl+C would still be useful.

What do you think, does this warrant a feature request?

@derhuerst commented on GitHub (Oct 23, 2025): > if there is no workaround to emulate end-of-stream. Actually, I found one: AFAICT, if you run a pipeline, typing Ctrl+C will send `SIGINT` to *all* of its processes. A workaround would be to "shield" the `brotli` process from the SIGINT, letting it work until it hits EOF. As [documented in a StackOverflow answer](https://stackoverflow.com/a/47693330/1072129), this can be achieved with a subshell and `trap '' INT`: ```shell $ for i in $(seq 1 10); do 1>&2 echo "$i"; for j in $(seq 1 1000); do echo "$j"; done; sleep 1; done | (trap '' INT; brotli >/tmp/foo.txt.br) # 1 # 2 # 3 ^C $ ls -lh /tmp/foo.txt.br # -rw-r--r-- 1 j wheel 1.3K Oct 23 15:15 /tmp/foo.txt.br $ for i in $(seq 1 10); do 1>&2 echo "$i"; for j in $(seq 1 1000); do echo "$j"; done; sleep 1; done | brotli >/tmp/foo.txt.br # 1 # 2 # 3 ^C $ ls -lh /tmp/foo.txt.br # -rw-r--r-- 1 j wheel 0B Oct 23 15:15 /tmp/foo.txt.br ``` However, I'd argue that - this is not trivial to find out: With non-expert knowledge of shell/POSIX APIs, it seems unlikely that someone will come up with that idea by themselves. Search for a solution is not exactly because there are *lots* of related-sounding-but-different questions/answers about pipelines in shells. - even outside the interactive shell use case, I'd argue that **having the option to compress & flush the current input upon Ctrl+C would still be useful**. What do you think, does this warrant a feature request?

claunia referenced this issue

2026-01-29 20:52:51 +00:00

[PR #531] [MERGED] Fix bintray release structure #1174

claunia referenced this issue

2026-01-29 20:52:51 +00:00

[PR #531] Fix bintray release structure #1178

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/brotli#531