Fixes#1550 - Docker builds were broken after PR #1535 switched from
vendored GPAC to system GPAC.
Changes:
- Switch from Alpine to Debian Bookworm (Alpine's musl libc has issues
with Rust bindgen's libclang dynamic loading)
- Support three build variants via BUILD_TYPE argument:
- minimal: No OCR support
- ocr (default): Tesseract OCR for bitmap subtitles
- hardsubx: OCR + FFmpeg for burned-in subtitle extraction
- Support dual source modes via USE_LOCAL_SOURCE argument:
- 0 (default): Clone from GitHub (standalone Dockerfile)
- 1: Use local source (faster for developers)
- Add .dockerignore to exclude build artifacts (~2.7GB -> ~900KB context)
- Update README.md with comprehensive build instructions
Tested all three variants successfully:
- minimal: ~130MB image
- ocr: ~215MB image
- hardsubx: ~610MB image
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The AVC parser would fail with "Leading bytes are non-zero" error when
processing HLS/Twitch stream segments that start mid-stream without
proper NAL unit headers at the beginning.
Root cause: When process_avc encountered non-zero leading bytes, it
returned an error with 0 bytes processed. The C code would not remove
any bytes from the buffer, causing subsequent data to accumulate with
the corrupt beginning, leading to infinite errors.
Fix:
- Add find_nal_start_code() to search for valid NAL start codes
- If buffer doesn't start with 0x00 0x00, search for first NAL start
- Skip garbage data before first valid NAL unit
- Return full buffer length when no NAL found (clears the buffer)
- Change forbidden_zero_bit error from fatal to skip-and-continue
Tested with 6 Twitch HLS sample files - all now process correctly.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Old versions of ccextractor accepted single-dash long options like
-quiet, -stdout, -autoprogram. The new Rust-based argument parser
(clap) only accepts double-dash options (--quiet, --stdout, etc.).
When users ran scripts with -quiet, clap parsed it as individual
short options -q -u -i -e -t and failed with exit code 7. Users
with stderr redirected never saw the error, causing silent failures
with zero-length output files.
This adds a normalize_legacy_option() function that pre-processes
arguments before passing them to clap:
- Single-dash long options (e.g., -quiet) convert to --quiet
- Double-dash options remain unchanged
- Short options like -o remain unchanged
- Numeric options like -1, -12 remain unchanged
Includes 6 unit tests for the new function.
Fixes#1576🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix raw caption file processing that would stop at exactly 9:43:00 (2MB).
Root causes and fixes:
1. Premature EOF: After processing first chunk (BUFSIZE ~2MB), data->len
was never reset. On next iteration, general_get_more_data() calculated
want = BUFSIZE - len = 0 and returned EOF immediately.
Fix: Reset data->len = 0 after each chunk and change loop condition.
2. 32-bit integer overflow: The calculation cb_field1 * 1001 / 30 * 90
overflowed for large cb_field1 values (>1M). For example,
34,989,487 * 90 = 3,149,053,830 exceeds 32-bit signed max.
Fix: Cast cb_field1 to LLONG before multiplication.
3. Timing initialization: Raw mode needs min_pts=0, sync_pts=0, and
pts_set=MinPtsSet for correct fts_now calculation.
Tested with sample files from issue #1565:
- DTV3.raw: Now processes to 17:59:56 (was stopping at 9:43)
- DTV4.raw: Now processes to 14:00:00 (was stopping at 9:43)
- DTV5.raw: Now processes to 13:19:59 (was stopping at 9:43)
Closes#1565🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a new --list-tracks (-L) option that lists all tracks found in
media files without processing them. This is useful for exploring
media files before caption extraction.
Supports:
- Matroska (MKV/WebM) files
- MP4/MOV files
- MPEG Transport Stream files
The feature is implemented entirely in Rust with native parsers for
each format, avoiding dependency on external libraries.
Closes#1669🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The previous WTV timing fix (commit 300f8ca6) set min_pts and pts_set=2
(MinPtsSet) but didn't set sync_pts. This caused the Rust timing code
to detect a massive PTS jump when processing WTV files with large
initial timestamps (e.g., files recorded at 18:38:23).
The PTS jump detection computes (current_pts - sync_pts), and with
sync_pts=0 but current_pts=6039323550 (18:38:23 in PTS units), the
difference exceeded MAX_DIF and triggered the jump handling, resulting
in empty output.
This fix sets sync_pts to the same value as min_pts when first
initializing timing, preventing the false PTS jump detection.
Test results:
- Before: WTV files with large initial PTS produced empty output
- After: Timestamps match expected ground truth exactly
(e.g., 00:00:00,601 --> 00:00:02,801 for first caption)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When using --udp or --tcp options, ccxr_demuxer_open() was called with
a NULL file pointer, causing a crash in CStr::from_ptr().
The fix checks if the file pointer is NULL before dereferencing it,
and uses an empty string for network input modes.
Fixes#1846🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
WTV timing fix:
- Set min_pts on first valid timestamp to enable fts_now calculation
- Set pts_set = 2 (MinPtsSet) instead of 1 (Received)
- This fixes WTV files where all timestamps were clustered around 1 second
instead of being spread across the actual video duration
Latin-1 encoding fix:
- Change music note substitution from pilcrow (0xB6) to '#' (0x23)
- Pilcrow caused grep to treat output files as binary
- '#' is a more recognizable substitute for the musical note character
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The rcwt_loop() function set min_pts = 0 for RCWT files but did not
set pts_set = 2 (MinPtsSet). This caused the Rust timing code to skip
the fts_now calculation (which checks pts_set == MinPtsSet), resulting
in all captions having timestamps compressed near 0 instead of their
correct times spread across the file duration.
The fix adds pts_set = 2 after setting min_pts, which tells the timing
system that min_pts is valid and fts_now can be calculated properly.
Fixes Test 217 timing issue where:
- Before: 00:00:00,001 --> 00:00:00,091 (wrong)
- After: 00:00:02,402 --> 00:00:04,536 (correct)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This PR triggers a fresh CI run to verify the combined effect of:
- PR #1847: Hardsubx crash fix, memory leak fixes, rcwt exit code fix
- PR #1848: XDS empty content entries fix
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When outputting US TV Parental Guidelines ContentAdvisory XDS data,
the code was always calling xdsprint() for both the age rating and
the content flags (violence, language, etc). However, if there are
no content flags (e.g., for TV-G which has no additional advisories),
the content string is empty.
This caused duplicate XDS entries in the output - one with the age
rating and one with an empty string. The fix only outputs the content
string if it is not empty.
Fixes regression test 113 output mismatch.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The rcwt_loop function was returning exit code 10 (no captions) even
when CEA-608 captions were successfully extracted from RCWT/BIN format
files. This happened because CEA-608 decoding writes directly to the
encoder via printdata() without setting dec_sub->got_output.
Add a check after the main loop (similar to general_loop) that also
considers enc_ctx->srt_counter, enc_ctx->cea_708_counter, and
dec_ctx->saw_caption_block to properly detect when captions were found.
Fixes regression test 217 which was failing with exit code 10.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The hardsubx code was using C's free() on strings allocated by Rust's
CString::into_raw(). Since Rust and C use different memory allocators,
this caused heap corruption that manifested as garbage OCR output after
processing ~27 subtitle frames.
Changes:
- Export free_rust_c_string() from Rust as extern "C" function
- Declare free_rust_c_string() in hardsubx.h for C code
- Replace free(subtitle_text) with free_rust_c_string(subtitle_text)
in hardsubx_decoder.c for Rust-allocated strings
- Fix memory leaks in process_hardsubx_linear_frames_and_normal_subs()
where subtitle_text_hard and prev_subtitle_text_hard were not freed
- Remove dummy CI trigger file (no longer needed)
Testing:
- AddressSanitizer: No memory errors detected
- Valgrind: 0 bytes definitely lost, 0 bytes indirectly lost
- Manual testing: OCR output now correct for entire video duration
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Free basefilename in _dinit_hardsubx (allocated by get_basename)
- Free subtitle_text after each frame processing iteration
- Free prev_subtitle_text when replaced and at end of function
- Free sws_ctx with sws_freeContext (was never freed)
Reduces memory leaks from 63,926 bytes to 0 bytes.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1. Remove invalid free(tessdata_path) - probe_tessdata_location() returns
a pointer to static strings or getenv() result, not heap memory.
2. Fix alloc-dealloc mismatch in OCR text handling:
- TessBaseAPIGetUTF8Text() allocates with C++ operator new[]
- The code was freeing with C free() causing allocator mismatch
- Now properly copy string and use TessDeleteText() before returning
- Unified all OCR text return paths to use Rust-allocated strings
3. Previous fix: freep(&lctx->dec_sub) instead of freep(lctx->dec_sub)
These fixes resolve Test 241 (Hardsubx) crash on Sample Platform.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The freep() function expects a pointer-to-pointer (void**) so it can
dereference, free, and NULL-out the pointer. The code was passing
lctx->dec_sub directly instead of &lctx->dec_sub.
This caused freep to interpret the first 8 bytes of the cc_subtitle
struct as a pointer and attempt to free() it, resulting in a crash
(SIGABRT/exit code 134) in the memory allocator.
Fixes Test 241 (Hardsubx) crash on Sample Platform.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This PR triggers a fresh CI run to analyze all failing regression tests
and determine whether each needs a ground truth update or a code fix.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add changelog entries for recent merged PRs:
- Fix: Garbled captions from HDHomeRun and I/P-only H.264 streams (#1109)
- Fix: Enable stdout output for CEA-708 captions on Windows (#1693)
- Fix: McPoodle DVD raw format read/write (#1524)
- Fix: Variable shadowing in general_loop
- Fix: Double-free crash in teletext cleanup
- Fix: Uninitialized memory and memory leaks (Valgrind)
- Fix: Dangling pointers in Rust FFI
- New: Teletext subtitle pages in -out=report (#1034)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix comment spacing (single space before //)
- Mark is_two_byte_loop_marker as #[cfg(test)] since it's only used in tests
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Reading:
- Migrate DVD raw parser from C to Rust (src/rust/src/demuxer/dvdraw.rs)
- Add FFI exports: ccxr_process_dvdraw(), ccxr_is_dvdraw_header()
- Handle both McPoodle's single-byte and legacy 2-byte loop markers
- Add 15 unit tests covering all edge cases
Writing:
- Fix LC3/LC4 constants from 2-byte to 1-byte to match McPoodle's format
- Output files now have identical size to McPoodle's original
Fixes#1524🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix clippy warning: variable does not need to be mutable.
The current_index variable is only assigned once during initialization
and never modified afterward.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
For I/P-only streams (like HDHomeRun recordings), the caption buffer was
being flushed on every reference frame (I and P). Since ALL frames in these
streams are reference frames, this defeated the caption reordering mechanism,
causing garbled output.
The fix:
- Only flush the buffer and reset reference PTS on IDR frames (NAL type 5),
not on P-frames
- Initialize currefpts on first frame to avoid huge indices at stream start
- Properly flush buffer and reset reference when large PTS gaps are detected
This allows P-frames to accumulate in the buffer and be sorted by their
PTS-based indices before output.
Fixes#1109🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit fixes two issues uncovered during Sample Platform testing:
1. Variable shadowing in general_loop() (general_loop.c):
- The inner `int ret = process_non_multiprogram_general_loop(...)`
was shadowing the outer `ret` variable
- This caused the return value to always be 0, making ccextractor
report "No captions found" even when captions were extracted
- Also added `ret = 1` when captions are detected via counters,
needed for CEA-708 which writes directly via Rust
2. Missing private_data refresh in update_decoder_list_cinfo (lib_ccx.c):
- After PAT changes, dinit_cap() frees the teletext context and
NULLs dec_ctx->private_data
- But update_decoder_list_cinfo() returned existing decoder without
refreshing private_data from the new cap_info
- This caused all subsequent teletext processing to be skipped
- Fixed by updating dec_ctx->private_data when returning existing decoder
These fixes resolve Sample Platform test failures in CEA-708 and Teletext
categories where tests returned exit code 10 (no captions) unexpectedly.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- demux.rs: Update dummy_demuxer() to explicitly initialize all fields
instead of using ..Default::default(), which is not allowed when the
struct implements Drop
- common.rs, demuxer.rs: Apply cargo fmt formatting fixes
This fixes the Rust test compilation error:
"cannot move out of type CcxDemuxer which implements the Drop trait"
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add proper cleanup of xds_ctx in rcwt_loop() for --in=bin and --in=raw
formats. The general_loop() path already frees xds_ctx, but rcwt_loop()
was missing this cleanup, causing an 880-byte leak.
This fixes Valgrind tests 217 (--in=bin) and 218 (--in=raw).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- XDS encoder leak: Free xds_str when skipping subtitles with invalid timestamps
- XDS decoder cleanup: Add proper cleanup for leftover XDS strings in dinit_cc_decode()
- Remove incorrect free(p) after write_xds_string() - the pointer is stored
for later use by the encoder and must not be freed immediately
- Remove xds_ctx free from dinit_cc_decode() to avoid double-free
These fixes address the 100-byte XDS leak found in Valgrind test 114.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The embedded dec_sub struct in lib_cc_decode had its data field
allocated by write_cc_buffer() but never freed during cleanup.
Added cleanup in dinit_cc_decode() to:
- Free DVB bitmap data (data0/data1) if present
- Free the dec_sub.data field itself
This fixes ~1.7MB to ~2.6MB leaks seen in tests 89, 93, and 96.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit fixes several Valgrind-detected memory issues:
1. Use-after-free in Teletext during PAT changes:
- When parse_PAT() calls dinit_cap() to reinitialize stream info,
it freed the Teletext context but dec_ctx->private_data still
pointed to the freed memory
- Fixed by NULLing out dec_ctx->private_data in dinit_cap() when
freeing shared codec private data
- Also added NULL check in process_data() before calling teletext
functions to gracefully handle freed contexts
2. Uninitialized variables in general_loop():
- stream_mode, get_more_data, ret, and program_iter were declared
without initialization
- While logically set before use, Valgrind tracked them as
potentially uninitialized through complex control flow
- Fixed by initializing all variables at declaration
These fixes eliminate millions of Valgrind errors in teletext tests
(tests 78, 80) and uninitialized value warnings (tests 67, 84, 86).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit fixes several significant memory leaks found by Valgrind testing:
1. Dtvcc::new encoder leak (decoder/mod.rs):
- Previously always allocated a new encoder_ctx even when ctx.encoder
was not null, then threw away the allocation
- Fix: Only allocate when ctx.encoder is null
- Impact: Eliminated 55MB-331MB leaks per video processing run
2. ccxr_demuxer_isopen optimization (demuxer.rs):
- Previously copied entire demuxer structure just to check infd
- Fix: Directly check (*ctx).infd != -1
- Impact: Eliminated repeated allocations during file processing
3. ccxr_demuxer_close optimization (demuxer.rs):
- Previously did full copy roundtrip (C->Rust->C) to close a file
- Fix: Work directly on C struct, call close() and activity callback
- Impact: Eliminated copy-related allocations and leaks
4. CcxDemuxer Drop implementation (common_types.rs):
- pid_buffers and pids_programs contain raw pointers from Box::into_raw
- These were never freed when CcxDemuxer was dropped
- Fix: Implement Drop to free all non-null Box pointers
- Impact: Eliminates remaining FFI-related leaks
Test results show dramatic improvement:
- Test 24: 55MB leak -> 0 bytes (PERFECT)
- Test 26: 9.75MB leak -> 0 bytes (PERFECT)
- Test 27: 237MB leak -> 0 bytes (PERFECT)
- Test 28: 331MB leak -> 0 bytes (PERFECT)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
