When converting CEA-708 decoder settings from C to Rust via from_ctype(),
a null timing pointer would cause the entire conversion to fail and return
None. This triggered the unwrap_or(default()) fallback, resetting critical
settings like `enabled` and `services_enabled` to false/0.
This caused CEA-708 captions to not be extracted (exit code 10) even when
--service was specified, because the decoder's is_active flag was reset
to 0 during demuxer initialization.
The fix handles null timing pointer gracefully by using a default
CommonTimingCtx instead of propagating None, preserving the other
decoder settings.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Redirect stderr to /dev/null for the GPAC source file search to avoid
showing "No such file or directory" error when GPAC is not installed.
The build continues to work correctly in both cases.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Homebrew installs leptonica as 'libleptonica.dylib', not 'liblept.dylib'.
Changed AC_CHECK_LIB from [lept] to [leptonica] to match the actual
library name on macOS.
The AC_CHECK_LIB checks in configure.ac need LDFLAGS and CPPFLAGS
to find libraries installed via Homebrew (in /opt/homebrew on Apple
Silicon or /usr/local on Intel Macs).
Fixes#1173 - Error in ./configure enabling hardsubx on Mac
Fixes#1306 - Add HARDSUBX compilation docs for macOS
The configure.ac script failed on macOS with "binary operator expected"
because pkg-config output was unquoted. When pkg-config returns multiple
libraries (e.g., "-ltesseract -lcurl"), the unquoted expansion caused
`test ! -z` to receive multiple arguments instead of a single string.
Changes:
- Quote pkg-config output in TESSERACT_PRESENT conditional (mac & linux)
- Add macOS section to docs/HARDSUBX.txt with all build methods
- Add GitHub Actions jobs to test HARDSUBX builds on macOS:
- build_shell_hardsubx: Tests ./build.command -hardsubx
- build_autoconf_hardsubx: Tests ./configure --enable-hardsubx
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The include "../lib_hash/sha2.h" in params.c requires an include path
that makes "../lib_hash" resolve to "thirdparty/lib_hash".
Changed -I../src/lib_hash (which doesn't exist) to
-I../src/thirdparty/lib_hash. With this path, the compiler searches
for "../lib_hash/sha2.h" as:
../src/thirdparty/lib_hash/../lib_hash/sha2.h
= ../src/thirdparty/lib_hash/sha2.h ✓
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Tests all three Dockerfile build types in parallel:
- minimal: Basic CCExtractor without OCR
- ocr: CCExtractor with Tesseract OCR support
- hardsubx: CCExtractor with burned-in subtitle extraction
Each job builds from local source and verifies the image works
by running --version. Uses GitHub Actions cache for faster rebuilds.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a new `-system-libs` flag to mac/build.command that uses
system-installed libraries via pkg-config instead of bundled ones.
This enables Homebrew formula compatibility while preserving the
default standalone build behavior.
When `-system-libs` is passed:
- Uses pkg-config for: freetype2, gpac, libpng, libprotobuf-c,
libutf8proc, zlib
- Does not compile bundled thirdparty sources
- Links against system libraries
Default behavior (no flag):
- Compiles bundled libraries as before
- No change to existing builds
Also adds a CI job `build_shell_system_libs` to test the new flag.
Refs #1580, #1534🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously, configure would succeed even without GPAC installed,
leading to a confusing compile-time error:
"gpac/isomedia.h: No such file or directory"
Now configure checks for GPAC via pkg-config and fails early with
a helpful error message listing the package names for common distros:
- gpac-devel (Fedora/RHEL)
- libgpac-dev (Debian/Ubuntu)
- gpac (Arch)
Fixes#1584🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes#1550 - Docker builds were broken after PR #1535 switched from
vendored GPAC to system GPAC.
Changes:
- Switch from Alpine to Debian Bookworm (Alpine's musl libc has issues
with Rust bindgen's libclang dynamic loading)
- Support three build variants via BUILD_TYPE argument:
- minimal: No OCR support
- ocr (default): Tesseract OCR for bitmap subtitles
- hardsubx: OCR + FFmpeg for burned-in subtitle extraction
- Support dual source modes via USE_LOCAL_SOURCE argument:
- 0 (default): Clone from GitHub (standalone Dockerfile)
- 1: Use local source (faster for developers)
- Add .dockerignore to exclude build artifacts (~2.7GB -> ~900KB context)
- Update README.md with comprehensive build instructions
Tested all three variants successfully:
- minimal: ~130MB image
- ocr: ~215MB image
- hardsubx: ~610MB image
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Previously, when using -out=mcc with raw input files (-in=raw),
CCExtractor would print "Output format not supported" and produce
no output. This was because the raw file processing path decoded
CEA-608 data to text, but MCC format requires raw cc_data bytes.
The fix adds a new code path that bypasses the 608 decoder when
MCC output is requested:
- Added process_raw_for_mcc() helper function that:
- Converts 2-byte raw pairs to 3-byte cc_data format
- Wraps each CC pair in CDP format via mcc_encode_cc_data()
- Maintains proper timing at 29.97fps
- Modified raw_loop() to detect MCC output and use the new path
Test results with McPoodle raw files:
- Before: "Output format not supported" (exit code 10)
- After: Valid MCC file with proper timing and CDP-wrapped data
Fixes#1542🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes#1455
When read_video_pes_header() encounters a malformed or truncated PES
packet (returns -1), copy_capbuf_demux_data() previously returned
CCX_EOF which terminated the entire file processing. This was overly
aggressive - a single broken PES packet should be skipped, not
terminate the entire file.
UK Freeview DVB recordings from September 2022 onwards contain some
malformed PES packets in the DVB subtitle stream that triggered this
condition, causing ccextractor to stop at 0% with "Processing ended
prematurely" error even though VLC could display the subtitles.
The fix changes the error handling to skip the broken packet and
continue processing:
- Before: return CCX_EOF (terminates file)
- After: return CCX_OK (skips packet, continues)
Test results with UK Freeview sample:
- Before: 0% processed, 0 subtitles extracted
- After: 100% processed, 10 subtitles extracted correctly
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The HEVC NAL type constants are defined for completeness and reference,
but not all are currently used in the codebase.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
HEVC uses B-frames extensively, causing CC data to arrive in decode
order instead of presentation order. This was causing character pairs
to be scrambled (e.g., "MEDIOCRE" became "MIOEDCRE").
Changes:
- Implement PTS-based sequence numbering for HEVC CC data (similar to H.264)
- Change flush logic to only trigger on IDR frames (not every VCL NAL)
- Add HEVC fallback detection for streams without PAT/PMT
Fixes#1639 (ATSC 3.0 HEVC caption extraction)
Tested with issue_1639_sample.ts and caption_test_1690.ts
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fixes#1690 - Captions fail to extract on HEVC video stream
HEVC video streams with embedded EIA-608/708 captions weren't being
extracted, even though VLC/MPV could display them.
Root causes fixed:
1. HEVC stream type (0x24) wasn't recognized for CC extraction
2. HEVC NAL parsing used H.264 format (1-byte) instead of HEVC (2-byte)
3. HEVC SEI types (39/40) weren't handled (only H.264 SEI type 6)
4. CC data accumulation across SEIs caused u8 overflow/garbled output
Changes:
- C code: Add HEVC stream detection, CCX_HEVC buffer type, is_hevc flag
- Rust code: HEVC NAL header parsing (2-byte, type=(byte[0]>>1)&0x3F),
HEVC SEI handling (PREFIX_SEI=39, SUFFIX_SEI=40), immediate CC flush
Thanks to @trufio465-bot for the initial research in PR #1735.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The AVC parser would fail with "Leading bytes are non-zero" error when
processing HLS/Twitch stream segments that start mid-stream without
proper NAL unit headers at the beginning.
Root cause: When process_avc encountered non-zero leading bytes, it
returned an error with 0 bytes processed. The C code would not remove
any bytes from the buffer, causing subsequent data to accumulate with
the corrupt beginning, leading to infinite errors.
Fix:
- Add find_nal_start_code() to search for valid NAL start codes
- If buffer doesn't start with 0x00 0x00, search for first NAL start
- Skip garbage data before first valid NAL unit
- Return full buffer length when no NAL found (clears the buffer)
- Change forbidden_zero_bit error from fatal to skip-and-continue
Tested with 6 Twitch HLS sample files - all now process correctly.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Old versions of ccextractor accepted single-dash long options like
-quiet, -stdout, -autoprogram. The new Rust-based argument parser
(clap) only accepts double-dash options (--quiet, --stdout, etc.).
When users ran scripts with -quiet, clap parsed it as individual
short options -q -u -i -e -t and failed with exit code 7. Users
with stderr redirected never saw the error, causing silent failures
with zero-length output files.
This adds a normalize_legacy_option() function that pre-processes
arguments before passing them to clap:
- Single-dash long options (e.g., -quiet) convert to --quiet
- Double-dash options remain unchanged
- Short options like -o remain unchanged
- Numeric options like -1, -12 remain unchanged
Includes 6 unit tests for the new function.
Fixes#1576🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Fix raw caption file processing that would stop at exactly 9:43:00 (2MB).
Root causes and fixes:
1. Premature EOF: After processing first chunk (BUFSIZE ~2MB), data->len
was never reset. On next iteration, general_get_more_data() calculated
want = BUFSIZE - len = 0 and returned EOF immediately.
Fix: Reset data->len = 0 after each chunk and change loop condition.
2. 32-bit integer overflow: The calculation cb_field1 * 1001 / 30 * 90
overflowed for large cb_field1 values (>1M). For example,
34,989,487 * 90 = 3,149,053,830 exceeds 32-bit signed max.
Fix: Cast cb_field1 to LLONG before multiplication.
3. Timing initialization: Raw mode needs min_pts=0, sync_pts=0, and
pts_set=MinPtsSet for correct fts_now calculation.
Tested with sample files from issue #1565:
- DTV3.raw: Now processes to 17:59:56 (was stopping at 9:43)
- DTV4.raw: Now processes to 14:00:00 (was stopping at 9:43)
- DTV5.raw: Now processes to 13:19:59 (was stopping at 9:43)
Closes#1565🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a new --list-tracks (-L) option that lists all tracks found in
media files without processing them. This is useful for exploring
media files before caption extraction.
Supports:
- Matroska (MKV/WebM) files
- MP4/MOV files
- MPEG Transport Stream files
The feature is implemented entirely in Rust with native parsers for
each format, avoiding dependency on external libraries.
Closes#1669🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The previous WTV timing fix (commit 300f8ca6) set min_pts and pts_set=2
(MinPtsSet) but didn't set sync_pts. This caused the Rust timing code
to detect a massive PTS jump when processing WTV files with large
initial timestamps (e.g., files recorded at 18:38:23).
The PTS jump detection computes (current_pts - sync_pts), and with
sync_pts=0 but current_pts=6039323550 (18:38:23 in PTS units), the
difference exceeded MAX_DIF and triggered the jump handling, resulting
in empty output.
This fix sets sync_pts to the same value as min_pts when first
initializing timing, preventing the false PTS jump detection.
Test results:
- Before: WTV files with large initial PTS produced empty output
- After: Timestamps match expected ground truth exactly
(e.g., 00:00:00,601 --> 00:00:02,801 for first caption)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When using --udp or --tcp options, ccxr_demuxer_open() was called with
a NULL file pointer, causing a crash in CStr::from_ptr().
The fix checks if the file pointer is NULL before dereferencing it,
and uses an empty string for network input modes.
Fixes#1846🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
WTV timing fix:
- Set min_pts on first valid timestamp to enable fts_now calculation
- Set pts_set = 2 (MinPtsSet) instead of 1 (Received)
- This fixes WTV files where all timestamps were clustered around 1 second
instead of being spread across the actual video duration
Latin-1 encoding fix:
- Change music note substitution from pilcrow (0xB6) to '#' (0x23)
- Pilcrow caused grep to treat output files as binary
- '#' is a more recognizable substitute for the musical note character
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The rcwt_loop() function set min_pts = 0 for RCWT files but did not
set pts_set = 2 (MinPtsSet). This caused the Rust timing code to skip
the fts_now calculation (which checks pts_set == MinPtsSet), resulting
in all captions having timestamps compressed near 0 instead of their
correct times spread across the file duration.
The fix adds pts_set = 2 after setting min_pts, which tells the timing
system that min_pts is valid and fts_now can be calculated properly.
Fixes Test 217 timing issue where:
- Before: 00:00:00,001 --> 00:00:00,091 (wrong)
- After: 00:00:02,402 --> 00:00:04,536 (correct)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This PR triggers a fresh CI run to verify the combined effect of:
- PR #1847: Hardsubx crash fix, memory leak fixes, rcwt exit code fix
- PR #1848: XDS empty content entries fix
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When outputting US TV Parental Guidelines ContentAdvisory XDS data,
the code was always calling xdsprint() for both the age rating and
the content flags (violence, language, etc). However, if there are
no content flags (e.g., for TV-G which has no additional advisories),
the content string is empty.
This caused duplicate XDS entries in the output - one with the age
rating and one with an empty string. The fix only outputs the content
string if it is not empty.
Fixes regression test 113 output mismatch.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The rcwt_loop function was returning exit code 10 (no captions) even
when CEA-608 captions were successfully extracted from RCWT/BIN format
files. This happened because CEA-608 decoding writes directly to the
encoder via printdata() without setting dec_sub->got_output.
Add a check after the main loop (similar to general_loop) that also
considers enc_ctx->srt_counter, enc_ctx->cea_708_counter, and
dec_ctx->saw_caption_block to properly detect when captions were found.
Fixes regression test 217 which was failing with exit code 10.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The hardsubx code was using C's free() on strings allocated by Rust's
CString::into_raw(). Since Rust and C use different memory allocators,
this caused heap corruption that manifested as garbage OCR output after
processing ~27 subtitle frames.
Changes:
- Export free_rust_c_string() from Rust as extern "C" function
- Declare free_rust_c_string() in hardsubx.h for C code
- Replace free(subtitle_text) with free_rust_c_string(subtitle_text)
in hardsubx_decoder.c for Rust-allocated strings
- Fix memory leaks in process_hardsubx_linear_frames_and_normal_subs()
where subtitle_text_hard and prev_subtitle_text_hard were not freed
- Remove dummy CI trigger file (no longer needed)
Testing:
- AddressSanitizer: No memory errors detected
- Valgrind: 0 bytes definitely lost, 0 bytes indirectly lost
- Manual testing: OCR output now correct for entire video duration
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>