mirror of
https://github.com/google/brotli.git
synced 2026-07-08 17:56:58 +00:00
Encoder reset (for c2go translated code) #311
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @CAFxX on GitHub (Feb 4, 2020).
In #679 it was noted that native Brotli is currently missing a way to reset an encoder. We are currently using a c2go translation of native brotli (to avoid cgo, native dependencies and all the pain that entails), and that means that all memory used by brotli is allocated on the Go heap, and is therefore subject to GC.
Unfortunately, because native brotli is missing the encoder reset, the GC overhead becomes significant as there is no effective way to reuse instantiated encoders.
Our use-case (a custom HTTP reverse-proxy) does not change encoding parameters between uses, so it would not be required to efficiently handle the case in which encoding parameters change.
@eustas commented on GitHub (Aug 3, 2023):
IIRC there are over 80 places where encoder allocates. Resetting only top-level things will not help much.
But there is another way to solve the problem. Use custom memory manager feature. Biggest allocations will always have the same size... So via caching those areas between runs and reusing them will do the trick.
There are some variable size allocations... But we can address that later to improve efficiency.