* ci: add Linux and macOS workflows for FFmpeg MP4 build
Adds a cmake_ffmpeg_mp4 job to both build_linux.yml and build_mac.yml
that configures CCExtractor with -DWITH_FFMPEG=ON -DWITH_OCR=ON
-DWITH_HARDSUBX=ON and builds it, so the FFmpeg-based MP4 demuxer
path is exercised on every PR. Linux job pulls the full set of
ffmpeg/tesseract/leptonica dev packages (including libavdevice-dev);
macOS job installs the corresponding Homebrew bottles.
* build: wire FFmpeg libs into ccx_rust link order
Add the compile-time option to build CCExtractor's MP4 demuxer on top
of libavformat: set -DENABLE_FFMPEG_MP4 and pull libswresample as a
required dependency alongside libavformat/libavutil/libavcodec/
libavfilter/libswscale when -DWITH_FFMPEG=ON.
Link-order handling. Corrosion places ccx_rust at the end of the
ccextractor link line. On Linux (GNU ld), ccx_rust contains rsmpeg
which references FFmpeg symbols like swr_get_out_samples, so the
FFmpeg shared libs must appear after ccx_rust. Collect them into a
separate EXTRA_FFMPEG_LIBS variable and attach them as
INTERFACE_LINK_LIBRARIES on the ccx_rust target so CMake emits them
right after ccx_rust. Make ccx's own link dependencies PRIVATE so
the same libs don't propagate earlier and get deduplicated against
the INTERFACE copy.
Force bridge symbols to be pulled from libccx. GNU ld only pulls
object files from a static archive when they resolve currently-needed
symbols. Bridge functions in libccx.a (ccx_mp4_process_nal_sample,
ccx_mp4_process_cc_packet, etc.) aren't needed until libccx_rust.a
is processed, but libccx.a precedes it on the command line. Use
-Wl,--undefined=<symbol> on ccextractor for each bridge entry
point so the linker pulls them early — same pattern already used
for decode_vbi/do_cb/store_hdcc.
Rust crate wiring. Add the optional rsmpeg dependency guarded behind
the enable_mp4_ffmpeg feature, with platform-specific feature flags
(link_system_ffmpeg on Linux + macOS, link_vcpkg_ffmpeg on Windows,
all using the ffmpeg7 bindings). Extend bindgen's wrapper.h to
expose the bridge headers so Rust can call back into ccx from
mp4_rust_bridge.c.
* feat(mp4): FFmpeg MP4 demuxer with GPAC-parity caption extraction
Alternative MP4 demuxing backend built on rsmpeg (Rust FFmpeg bindings)
that matches GPAC's caption output on every sample reviewed. Activated
via -DWITH_FFMPEG=ON at compile time; default build keeps the GPAC
path untouched.
Architecture
- src/rust/src/demuxer/mp4.rs: opens the MP4 with rsmpeg, classifies
tracks (AVC, HEVC, c608, c708, tx3g), and drives packet dispatch.
- src/rust/src/mp4_ffmpeg_exports.rs: #[no_mangle] entry points
(ccxr_processmp4, ccxr_dumpchapters) called from ccextractor.c.
- src/lib_ccx/mp4_rust_bridge.c/.h: thin C shim around do_NAL,
process608, process_cc_data, ccdp_find_data, store_hdcc, and
encode_sub so the Rust side can feed decoded payloads into
CCExtractor's existing CEA-608/708 pipeline.
Caption parity
GPAC-equivalent output on all six samples from the Apr 18 review:
- 132d7df7e993.mov 108 290 B byte-identical
- 1974a299f050.mov 127 828 B byte-identical
- 99e5eaafdc55.mov 164 099 B byte-identical
- 8849331ddae9.mp4 48 485 B identical size, content, caption
count; uniform ~2 ms timing shift
- b2771c84c2a3.mp4 2 607 B byte-identical
- 5df914ce773d.mp4 1 164 B byte-identical
SEI-embedded CEA-608 in H.264 video
The last caption finishing on the final sample was never encoded
because the interleaved av_read_frame loop exited without an
equivalent of GPAC's per-track encode_sub. Drain sub.got_output at
EOF. Intentionally avoid calling process_hdcc there: the last IDR's
slice_header already flushed the HD-CC buffer, and re-running
process_hdcc re-emits partial post-IDR caption state as trailing
garbage.
c608 payload handling
libavformat delivers c608/c708 samples in two shapes:
1) atom-wrapped raw 608 pairs — [u32 length][4cc cdat|cdt2|ccdp]
[payload]. Strip the 8-byte header to match GPAC's process_clcp.
2) bare cc_data triplets — [cc_info][b1][b2]. Detect via len % 3 == 0
and (payload[0] & 0xF8) == 0xF8. For c608 tracks, extract each
field-1/field-2 pair from the triplet, set dec_ctx->current_field
so process608 picks the right decoder context, and call
process608 directly. Routing through do_cb would hit its
CCX_H264 guard (set by interleaved H.264 packets) and suppress
the cb_field increments process608 relies on for
caption-boundary timing, which merged short captions into their
successors. For c708, keep ccdp_find_data + process_cc_data.
Bridge design
Single unified entry point ccx_mp4_process_nal_sample(..., int
is_hevc, ...) replaces the separate AVC/HEVC helpers — their NAL
iteration was ~90% identical. Uses utility.h's RB16/RB32 macros
instead of hand-rolled byte-swap helpers. Handles HEVC's
end-of-sample cc_data flush inline.
Supported and unsupported tracks are documented in the mp4.rs
module header. Known limitation: dvdsub/bitmap subtitles in MP4
are not decoded (neither GPAC nor this backend handles them).
* chore(rust): fix clippy 1.95 warnings blocking CI
The format_rust CI job runs `cargo clippy --lib -- -D warnings` and
rust-1.95.0 promoted a handful of lints that hadn't been tripped on
master before this branch went through CI. Address all six:
collapsible_match (4 sites):
- src/demuxer/demux.rs: fold the nested `if matches!(...)` inside
the `Some(false) =>` arm into a match guard on the arm itself.
- src/encoder/common.rs: three header-write arms (Ccd, Scc, Raw)
each had a nested `if write_raw(...) == -1 { return -1; }`.
Collapse each into a match guard; the body now just logs and
returns.
unnecessary_cast (2 sites):
- src/libccxr_exports/demuxer.rs: drop `as i64` from
demux_ctx.get_filesize() — it already returns i64.
- src/parser.rs: drop `as u64` from `t as u64` when writing to
UTC_REFVALUE — t is already u64.
Also two lints in the new MP4 demuxer file from this branch:
- if_same_then_else at src/demuxer/mp4.rs:198 — the FOURCC_TX3G
and AV_CODEC_ID_MOV_TEXT arms both produced TrackType::Tx3g;
same for FOURCC_C608 and AV_CODEC_ID_EIA_608 → TrackType::Cea608.
Fold each pair into a single `||` branch.
- manual_is_multiple_of at src/demuxer/mp4.rs:399 —
`packet_count % 100 == 0` → `packet_count.is_multiple_of(100)`.
No behavior change. Caption parity against GPAC on all six mentor
samples verified post-fix.
---------
Co-authored-by: GAURAV KARMAKAR <gaurav02081@users.noreply.github.com>
Build workflows were not triggering on CMakeLists.txt changes.
Added **CMakeLists.txt and **.cmake patterns to path filters for:
- build_linux.yml
- build_mac.yml
- build_windows.yml
- build_docker.yml
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The AC_CHECK_LIB checks in configure.ac need LDFLAGS and CPPFLAGS
to find libraries installed via Homebrew (in /opt/homebrew on Apple
Silicon or /usr/local on Intel Macs).
Fixes#1173 - Error in ./configure enabling hardsubx on Mac
Fixes#1306 - Add HARDSUBX compilation docs for macOS
The configure.ac script failed on macOS with "binary operator expected"
because pkg-config output was unquoted. When pkg-config returns multiple
libraries (e.g., "-ltesseract -lcurl"), the unquoted expansion caused
`test ! -z` to receive multiple arguments instead of a single string.
Changes:
- Quote pkg-config output in TESSERACT_PRESENT conditional (mac & linux)
- Add macOS section to docs/HARDSUBX.txt with all build methods
- Add GitHub Actions jobs to test HARDSUBX builds on macOS:
- build_shell_hardsubx: Tests ./build.command -hardsubx
- build_autoconf_hardsubx: Tests ./configure --enable-hardsubx
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a new `-system-libs` flag to mac/build.command that uses
system-installed libraries via pkg-config instead of bundled ones.
This enables Homebrew formula compatibility while preserving the
default standalone build behavior.
When `-system-libs` is passed:
- Uses pkg-config for: freetype2, gpac, libpng, libprotobuf-c,
libutf8proc, zlib
- Does not compile bundled thirdparty sources
- Links against system libraries
Default behavior (no flag):
- Compiles bundled libraries as before
- No change to existing builds
Also adds a CI job `build_shell_system_libs` to test the new flag.
Refs #1580, #1534🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* feat: unpack gpac
* fix: linux ci
* fix: mac build
* fix: remove unused [no ci]
* fix: ignore config.h [no ci]
* temp commit, will drop this soon
* fix: install gpac
* fix: gpac
* fix: formatting
* fix: preproccessor directive
* fix: comment display version for now
* fix: display dlls code
* fix: bundle vcruntime in hardsubx windows
* fix: again
* fix: erros in ci
* fix: ci
* fix: add vcruntime in additional dependencies
* fix: try to copy vcruntime after build
* fix: space in runtime library
* fix: remove for now [no ci]
* fix: things in vcxproj
* fix: ci for leptonica sys
* fix: docs
* fix: copy dlls on post build event
* fix: copy vcruntime after build
* feat: add arguments through clap
* fix: type of some arguments
* fix: "-" and "--" in comments
* fix: format files
* fix: add argument parsing till mkvlang
* fix: one todo item
* chore: lint fixes
* fix: nocodec value
* fix: for nocodec
* fix: add cfg feature for hardsubx
* feat: complete till startcreditstext
* fix: add more notes, args: option affect processed
* feat: port all till network stuff
* fix: complete almost all argument parsing
* fix: error free code
* fix: complete params port
* fix: hardsubx erros
* feat: clean up main function
* fix: pr reviews
* fix: make input,output function better
* fix: variant not used warning
* fix: warnings
* fix: all clippy warnings
* feat: add tests
* feat: add tests
* chore: lint fixes
* fix: move unit tests to correct folder
* fix: remove unncessary files
* fix: make function for parse_args
* fix: review changes
* fix: Impl CcxOptions whenever I could
* fix: try to convert rust to c
* chore: push c code
* fix: add more rust to c conversions
* fix: use set methods for bitfield
* fix: errors
* fix: arguments parsing
* fix: all issues
* fix: many errors
* chore: lint fix
* fix: err
* fix: unsafe function error
* fix: unsafe warning
* fix: safety lint
* chore: add docs
* fix: windows build
* fix: function
* fix: dependencies
* fix: set_binary_mode
* chore: lint fix
* fix: set_binary_mode for windows
* fix: error
* fix: undefined reference error
* chore: remove comment
* fix: output field
* chore: fix lint
* fix: ru1, ru2, ru3
* fix: undef before
* fix: parameter and update deps
* chore: update vcpkg
* feat: add release-with-debug profile
* fix; uncomment code
* fix: update visual studio to 2022
* chore: update docs
* fix: use default vcpkg
* fix: caching logic on release ci
* fix: vcpkg caching
* fix: add setup vcpkg
* chore: remove unneccesary formatting
* fix: Always write 2 bytes for UTF-16BE
* fix: formatting
* feat: add rest of the notes to bring continuity
* fix: remove extra line
* fix: add hardsubx note
* fix: source code format error
* chore: lint fixes acc to rustfmt
* feat: add unit test ci
* fix: conversion of strings, add file queue handling
* fix: decoder cfg
* fix: update dependencies
* chore: lint fix
* chore: add safety doc
* fix: default value for CcxOptions
* fix(rust): default value for teletext
* fix: leptonica version for windows
* fix: format errors
* fix: workflow
* Revert "fix: leptonica version for windows"
This reverts commit 461ef55e7b.
* fix: pin ffmpeg to 6 for mac
* fix(parser): default values and unwrap's
* fix(parser): hardsubx fixes
* chore(parse): lint fixes
* fix(windows): switch back to sdk 2019
* fix(workflow): windows workflow revert
* fix(windows): revert to old files which were working before
* fix(workflow): pin vcpkg packages
* chore(rust): downgrade leptonica
* fix(windows): move vcpkg.json to correct place
* fix(windows): improve vcxproj
* fix(windows): workflow
* fix(windows): workflow
* fix(windows): workflow clone from vcpkg everytime
* fix(workflow): error
* fix(workflow): don't skip building vcpkg
* fix: remove depth from vcpkg
* temporary commit
* fix(windows): pin gpac and use local vcpkg manifest properly
* fix(windows): install vcpkg dependencies manually
* fix(windows): update dll names
* fix(windows); dependencies copy
* fix(windows): don't continue on error for release
* fix(macos): build ffmpeg for mac workflow
* fix: move ffmpeg to current workspace
* fix: re-add profile for windows
* fix: pkg config for mac
* fix(mac): use ffmpeg@6 from brew
* fix(macos): there is no ffmpeg_prebuilt
* fix(macos): specify ffmpeg pkg config
* fix(macos): globally define pkg config
* fix(macos): add ffmpeg include and libs dir
* fix(macos): include ffmpeg headers in makefile
* fix: include ffmpeg libraries and include directories
* fix: try to manually specify ffmpeg header in rust
* fix: also include leptonica headres
* fix: leptonica name
* fix: test
* fix: string null when output_filename is empty
* fix: error
* fix: remove cflgas
* fix(mac): disable cmake ocr hardsubx
* chore: update gitignore
* fix: null if string is empty
* fix: allow --in
* chore: bump version to 1.0 in rust
* chore: add space to trigger sp
* fix: don't panic with rust
* fix: add double dashes to indicate parameters
* chore: update CHANGES.txt
* fix: test
* fix(workflow): update workflow name
* fix(rust): linux output_filename in sampleplatform
* fix(rust): parser default values
* fix(rust): exit with MalformedParameter instead of panic
* fix(decoder): revert always write 2 bytes
* chore(rust): format
* chore: update lock file
* fix(test): test lib_ccxr and rename to test
* fix(mac): remove failing cmake_ocr test
* fix: ci errors
* fix: feature related changes
* fix: trim down default features
* fix: don't check clippy for all features
* feat: unpack gpac
* fix: linux ci
* fix: mac build
* fix: remove unused [no ci]
* fix: ignore config.h [no ci]
* temp commit, will drop this soon
* fix: install gpac
* fix: gpac
* fix: formatting
* fix: preproccessor directive
* fix: comment display version for now
* fix: display dlls code
* fix: bundle vcruntime in hardsubx windows
* fix: again
* fix: erros in ci
* fix: ci
* fix: add vcruntime in additional dependencies
* fix: try to copy vcruntime after build
* fix: space in runtime library
* fix: remove for now [no ci]
* fix: things in vcxproj
* fix: ci for leptonica sys
* fix: docs
* fix: copy dlls on post build event
* fix: copy vcruntime after build
* feat: mac ci
* fix: ci dependencies
* fix: more depdendencies
* fix: libavcodec not found
* fix: include directories in mac
* fix: error in endif()
