* fix(dvb): Multiple fixes for DVB subtitle extraction from Chinese broadcasts (#224)
This commit addresses multiple issues with DVB subtitle extraction reported in #224:
1. **PMT parsing crash fix** (ts_tables.c):
- Added minimum length check (16 bytes) to prevent out-of-bounds access
- Added bounds check before memcpy to prevent buffer overflow when section > 1021 bytes
2. **Negative subtitle timing fix** (general_loop.c):
- For DVB subtitle streams, properly initialize min_pts from audio/subtitle PTS
- This fixes the issue where all timestamps were negative (~95000 seconds off)
3. **OCR improvements** (ocr.c):
- Fixed ignore_alpha_at_edge() which could create invalid crop windows
- Added image inversion for DVB subtitles (light text on dark background)
to improve Tesseract OCR accuracy
- Added contrast normalization to further improve character recognition
- Fixed nofontcolor check to respect --no-fontcolor parameter
- Added iteration safety limit in color detection loop
4. **--ocrlang parameter fix** (Rust files):
- Changed ocrlang from Language enum to String to accept Tesseract language
names directly (e.g., "chi_tra", "chi_sim", "eng")
- Added case-insensitive matching for --dvblang parameter
- Added better error messages for invalid language codes
Tested with 12GB Chinese DVB broadcast file:
- Timing: All timestamps now positive (0.235s, 2.594s, etc.)
- OCR: ~80-90% accuracy with chi_tra traineddata (improved from ~70%)
- No crashes during full file processing
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(ocr): Fix crashes in DVB subtitle color detection
Two issues fixed in the OCR color detection code:
1. Tesseract crash during iteration:
- The color detection pass used raw color images without preprocessing
- Tesseract expects dark text on light background, but DVB subtitles
have light text on dark background
- Added grayscale conversion, inversion, and contrast enhancement
(same preprocessing as the main OCR pass)
2. Heap corruption in histogram calculation:
- The histogram loop had no bounds checking on array accesses
- Tesseract could return invalid bounding boxes causing buffer overflows
- Added validation of bounding box coordinates before processing
- Added safe index checking for copy->data and histogram arrays
Also added skip_color_detection label for clean error handling and
proper cleanup of the preprocessed image.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix(dvb): Fix zero-duration subtitles and overlaps during PTS jumps
Add start_pts field to cc_subtitle struct to track raw PTS values
independent of FTS timeline resets. Modify end_time calculation in
dvbsub_handle_display_segment() to cap duration at 4 seconds when
PTS jumps cause timeline discontinuities, preventing zero-duration
and overlapping subtitles.
Also update .gitignore to exclude plans/ directory and temp files.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Apply clang-format to all C/H files in src/
- Apply cargo fmt to Rust code
- Update Cargo.lock with latest compatible dependency versions
- Add 24 new entries to CHANGES.TXT for recent fixes and features
Changes in CHANGES.TXT cover:
- CEA-708 bounds checks and UTF-16BE encoding fixes
- New --ttxtforcelatin option for Teletext
- TS files without PAT/PMT fallback support
- Timing accuracy improvements across MP4/MPEG/TS
- Memory safety improvements (null checks, buffer overruns)
- Multi-file processing fixes
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* feat: added demuxer module
* Cargo Lock Update
* Completed file_functions and demuxer
* Completed file_functions and demuxer
* written extern functions for demuxer
* Removed libc completely, added tests for gxf and ported gxf to C
* Hardsubx error fixed
* Fixing format issues
* clippy errors fixed
* fixing format issues
* fixing format issues
* Windows failing tests
* Windows failing tests
* demuxer: added demuxer data transfer functions and removed some structs
* made Demuxer and File Functions
* Minor formatting changes
* Minor Rebasing changes
* demuxer: format rust and unit test rust checks
* C formatting
* Windows Failing test
* Windows Failing test
* Update CHANGES.TXT
* Update CHANGES.TXT
* Windows Failing Tests
* Windows Failing Tests
* Problem in Copy to Rust and some typos that copilot review suggested
* Minor Formatting Error
* Windows Failing Regressions
* Windows Failing Regressions
* Minor Comment Change
* Data transfer module for DemuxerData added and more rustlike syntax to ctorust.rs
* Minor Formatting Changes
* demuxer: Rebase and a few tweaks to file_functions
* demuxer: Minor Formatting Error
* [FIX] 134 Codes in XDS and General Tests (#1708)
* Made pointers valid in Unit Tests of Decoder
* fix: test_do_cb
* Copilot Suggestions
* Suggestions about Redundancy
* Suggestions about Redundancy
* [FEAT] Add `bitstream` module in `lib_ccxr` (#1649)
* feat: Add bitstream module
* run code formatters
* Run cargo clippy --fix
* Run cargo fmt --all
* refactor: remove rust pointer from C struct
* feat: Add bitstream module
* run code formatters
* Run cargo clippy --fix
* Run cargo fmt --all
* refactor: remove rust pointer from C struct
* Added Bitstream to libccxr_exports
* Minor Formatting Issue
* Bitstream: Removed redundant CType
* bitstream: recommended changes for is_byte_aligned
* bitstream: recommended changes for long comments
* bitstream: comment fix
* bitstream: removed redundant comparism comments
---------
Co-authored-by: Deepnarayan Sett <depnra1@gmail.com>
Co-authored-by: Deepnarayan Sett <71217129+steel-bucket@users.noreply.github.com>
* demuxer: minor formatting changes
* Demuxer: Changes to mistakes in CHANGES.txt
* Demuxer: Removed extra newline in ccextractor.c
* Demuxer: Changes to Encoding resolved
* Demuxer: Moved CCX_NOPTS to common structs and some changes to Demuxer Data regd. MPEG_CLOCK_FREQ
* some refactoring to CCX_NOPTS
* Demuxer: Minor Mistake regarding CHANGES.txt
* Demuxer: Unit test rust failing because of CCX_NOPTS
* Demuxer: changed common_structs to common_types
* Demuxer: Removed redundant libraries from Cargo.toml and moved tempfile to dev-dependencies
* Demuxer: Removed to_vec function and renamed PSIBuffer/PMTEntry from_ctype functions
* Demuxer: Renamed Stream_Type, improved Time complexity of the default() function and removed redundant comments
* Demuxer: Removed two repeated code blocks and removed redundant comments
* Demuxer: Removed two code blocks
* Demuxer: Review Changes
* Demuxer: Removed redundant tests
* Update src/rust/src/demuxer/demux.rs
Co-authored-by: Prateek Sunal <prtksunal@gmail.com>
* Demuxer: Errors due to Rebase
* Demuxer: Removed get_stream_mode
* Demuxer: Errors due to rebasing and removing redundant CType Functions
* Demuxer: Failing ES regressions
* Demuxer: MythTV failing regression
* Demuxer: Removed redundant comments
* Demuxer: Unplugged ES for now
* Demuxer: Replugged in ES
* Demuxer: Formatting error
* Demuxer: Windows failing CI
* Demuxer: Windows failing CI
* Demuxer: Windows failing Regressions
* Demuxer: Formatting
* Demuxer: Minor Cargo Clippy change
* Demuxer: running regressions again
* Demuxer: Cargo Lockfile Change
* Demuxer: running regressions again
* Demuxer: running regressions again
---------
Co-authored-by: Swastik Patel <swastikpatel29@gmail.com>
Co-authored-by: Prateek Sunal <prtksunal@gmail.com>
* Removal: Removed redundant C code already ported to Rust
* Removal: C formatting
* Removal: More Removal and CI issues in Mac
* Removal: CI issues in Mac
* Removal: Changes due to Rebase
* Removal: Failing CI on mac
* Removal: Failing regression test on dvdraw
* Fix hardsubx_decoder.c compilation with ENABLE_FFMPEG
Fix unresolved function reference when compiling with ENABLE_FFMPEG
* Fix regression compilation ffmpeg_intgr.c to support ffmpeg 5
Fix regression bug for compiling with ENABLE_FFMPEG and ffmpeg 5, introduced in https://github.com/CCExtractor/ccextractor/issues/1418
* Update CHANGES.TXT
* Update ffmpeg_intgr.c
Update for changes to FFMPEG 5 API
* [FIX] Corrected bitness check for 64-bit systems
* Improve Dockerfile: cleanup, parallel build, and remove redundancies
- Replaced cd with WORKDIR for clarity and Docker best practices.
- Removed unused LIB_CLANG_PATH export, as it only affected a single build layer; the library is automatically detected during build.
- Parallelized the GPAC build using make -j$(nproc).
- Removed redundant CMD instruction, as ENTRYPOINT already defines the container's execution command.
* [DOCS] Update CHANGES.TXT for Dockerfile improvements
---------
Co-authored-by: AhmedYasserrr <ahmdyasrj@gamil.com>
* Fix implicit declaration error on some systems.
This commit fixes a compile-time error regarding an implicit declaration
of mapclut_paletee() on some compilers and compiler versions. Notably,
Arch Linux and Ubuntu 24.10 seem to be affected.
The error resolved is:
```
../src/lib_ccx/ocr.c: In function 'ocr_rect':
../src/lib_ccx/ocr.c:922:9: error: implicit declaration of function 'mapclut_paletee' [-Wimplicit-function-declaration]
922 | mapclut_paletee(palette, alpha, (uint32_t *)rect->data1, rect->nb_colors);
| ^~~~~~~~~~~~~~~
```
This was resolved by `#include`-ing "ccx_encoders_spupng.h" in the file
src/lib_ccx/ocr.c. Thanks to GitHub user @steel-bucket for sharing the
fix in this issue's comments.
Fixes: #1646
* Update CHANGES.TXT.
Mention the fix for #1646.
Fixes: #1646
* Add flag for Page Segmentation Modes control
I added an flag --psm for controlling PSM (Page Segmentation Modes) in Tesseract. The default option (3) gives me quite bad results. When I use 6, 11, or 12 for Bulgarian, it gives me much better OCR results. I haven't tested other languages yet, but I expect improvements as well if other mode is used.
* feat: add psm for rust parser
* fix: add psm to options
* fix: add default value of psm to 3
* fix: correct type of ocr oem
* fix(rust): use fatal! instead of exit
---------
Co-authored-by: Prateek Sunal <prtksunal@gmail.com>
* feat: Add new function to allocate any object to heap with zero allocated
* feat: Add unit tests for `decoder/commands.rs`
* docs: Mention about PR in changelogs
* feat: Add unit tests for `decoder/windows.rs`
Refactor the code and use Default where needed
Implement `PartialEq` also
* fix: Intialise tmp extern C values for easy mocking
* feat: Add unit tests for `decoder/timing.rs`
* feat: Add unit tests for `decoder/output.rs`
* feat: Add unit tests for `decoder/mod.rs`
* feat: Add unit tests for `decoder/tv_screen.rs`
* feat: Add unit tests for `lib.rs`
* fix: Failing test
* feat: [WIP] Add unit tests for `decoder/service_decoder.rs`
* feat: Add unit tests for `decoder/service_decoder.rs`
* feat: Add unit tests for `hardsubx/imgops.rs`
* feat: Add unit tests for `hardsubx/utility.rs`
* fix: cargo clippy
* fix: doctest for `lib_ccxr` module
* feat: Add test `lib_ccxr/util/mod.rs`
* feat: Add test `lib_ccxr/util/levenshtein.rs`
* feat: Add test `lib_ccxr/util/bits.rs`
* feat: Add test `lib_ccxr/time/units.rs`
* chore: Change function name
* fix: Failing of missing values `tlt_config`
* ci: Run unit test cases in `lib_ccxr` module also
* ci: Run clippy & fmt in `lib_ccxr` module also
* chore(clippy): Fix clippy warnings
* feat: unpack gpac
* fix: linux ci
* fix: mac build
* fix: remove unused [no ci]
* fix: ignore config.h [no ci]
* temp commit, will drop this soon
* fix: install gpac
* fix: gpac
* fix: formatting
* fix: preproccessor directive
* fix: comment display version for now
* fix: display dlls code
* fix: bundle vcruntime in hardsubx windows
* fix: again
* fix: erros in ci
* fix: ci
* fix: add vcruntime in additional dependencies
* fix: try to copy vcruntime after build
* fix: space in runtime library
* fix: remove for now [no ci]
* fix: things in vcxproj
* fix: ci for leptonica sys
* fix: docs
* fix: copy dlls on post build event
* fix: copy vcruntime after build
* feat: add arguments through clap
* fix: type of some arguments
* fix: "-" and "--" in comments
* fix: format files
* fix: add argument parsing till mkvlang
* fix: one todo item
* chore: lint fixes
* fix: nocodec value
* fix: for nocodec
* fix: add cfg feature for hardsubx
* feat: complete till startcreditstext
* fix: add more notes, args: option affect processed
* feat: port all till network stuff
* fix: complete almost all argument parsing
* fix: error free code
* fix: complete params port
* fix: hardsubx erros
* feat: clean up main function
* fix: pr reviews
* fix: make input,output function better
* fix: variant not used warning
* fix: warnings
* fix: all clippy warnings
* feat: add tests
* feat: add tests
* chore: lint fixes
* fix: move unit tests to correct folder
* fix: remove unncessary files
* fix: make function for parse_args
* fix: review changes
* fix: Impl CcxOptions whenever I could
* fix: try to convert rust to c
* chore: push c code
* fix: add more rust to c conversions
* fix: use set methods for bitfield
* fix: errors
* fix: arguments parsing
* fix: all issues
* fix: many errors
* chore: lint fix
* fix: err
* fix: unsafe function error
* fix: unsafe warning
* fix: safety lint
* chore: add docs
* fix: windows build
* fix: function
* fix: dependencies
* fix: set_binary_mode
* chore: lint fix
* fix: set_binary_mode for windows
* fix: error
* fix: undefined reference error
* chore: remove comment
* fix: output field
* chore: fix lint
* fix: ru1, ru2, ru3
* fix: undef before
* fix: parameter and update deps
* chore: update vcpkg
* feat: add release-with-debug profile
* fix; uncomment code
* fix: update visual studio to 2022
* chore: update docs
* fix: use default vcpkg
* fix: caching logic on release ci
* fix: vcpkg caching
* fix: add setup vcpkg
* chore: remove unneccesary formatting
* fix: Always write 2 bytes for UTF-16BE
* fix: formatting
* feat: add rest of the notes to bring continuity
* fix: remove extra line
* fix: add hardsubx note
* fix: source code format error
* chore: lint fixes acc to rustfmt
* feat: add unit test ci
* fix: conversion of strings, add file queue handling
* fix: decoder cfg
* fix: update dependencies
* chore: lint fix
* chore: add safety doc
* fix: default value for CcxOptions
* fix(rust): default value for teletext
* fix: leptonica version for windows
* fix: format errors
* fix: workflow
* Revert "fix: leptonica version for windows"
This reverts commit 461ef55e7b.
* fix: pin ffmpeg to 6 for mac
* fix(parser): default values and unwrap's
* fix(parser): hardsubx fixes
* chore(parse): lint fixes
* fix(windows): switch back to sdk 2019
* fix(workflow): windows workflow revert
* fix(windows): revert to old files which were working before
* fix(workflow): pin vcpkg packages
* chore(rust): downgrade leptonica
* fix(windows): move vcpkg.json to correct place
* fix(windows): improve vcxproj
* fix(windows): workflow
* fix(windows): workflow
* fix(windows): workflow clone from vcpkg everytime
* fix(workflow): error
* fix(workflow): don't skip building vcpkg
* fix: remove depth from vcpkg
* temporary commit
* fix(windows): pin gpac and use local vcpkg manifest properly
* fix(windows): install vcpkg dependencies manually
* fix(windows): update dll names
* fix(windows); dependencies copy
* fix(windows): don't continue on error for release
* fix(macos): build ffmpeg for mac workflow
* fix: move ffmpeg to current workspace
* fix: re-add profile for windows
* fix: pkg config for mac
* fix(mac): use ffmpeg@6 from brew
* fix(macos): there is no ffmpeg_prebuilt
* fix(macos): specify ffmpeg pkg config
* fix(macos): globally define pkg config
* fix(macos): add ffmpeg include and libs dir
* fix(macos): include ffmpeg headers in makefile
* fix: include ffmpeg libraries and include directories
* fix: try to manually specify ffmpeg header in rust
* fix: also include leptonica headres
* fix: leptonica name
* fix: test
* fix: string null when output_filename is empty
* fix: error
* fix: remove cflgas
* fix(mac): disable cmake ocr hardsubx
* chore: update gitignore
* fix: null if string is empty
* fix: allow --in
* chore: bump version to 1.0 in rust
* chore: add space to trigger sp
* fix: don't panic with rust
* fix: add double dashes to indicate parameters
* chore: update CHANGES.txt
* fix: test
* fix(workflow): update workflow name
* fix(rust): linux output_filename in sampleplatform
* fix(rust): parser default values
* fix(rust): exit with MalformedParameter instead of panic
* fix(decoder): revert always write 2 bytes
* chore(rust): format
* chore: update lock file
* fix(test): test lib_ccxr and rename to test
* fix(mac): remove failing cmake_ocr test
* fix: ci errors
* fix: feature related changes
* fix: trim down default features
* fix: don't check clippy for all features
* chore: Add cargo dependencies
* feat: Make time module in `lib_ccxr`
* feat: Add conversion guide in `time/mod.rs` module & Create `units` module
* feat: Add time units code
* feat: Make time module in `lib_ccxr/util` & Add helper function
* feat: Add utils time related functions
* feat: Add extern functions in `libccxr_exports`
* feat: Add extern functions in C and use in proper place
* docs: Mention in Changelogs
* feat: Add common module
common module is made for all `ccx_common_*` files
* feat: Add constants module within common module
Used to have all constants enums listed in ccx_common_constants C file
* feat: Add all constants, enums in rust equivaleent to `ccx_common_constansts` C file
* docs: Mention in Changelogs
* docs: Add more conversion data
* chore: Add bitflags crate as dependency
* feat: Add function to initialize Rust logger using options in C
* feat: Add new module `log`
* refactor: Add ccx_s_option into list of bindgen struct
* feat: Add Initialize logger function
* feat: All logging functions & macros
* chore: Fix clippy
* docs: Mention in Changelogs
* chore: format issue fix
* fix: Remove activity_header from rust & use initially to print in C
* refactor: Remove debugging statements
* fix: Add `\n` in info!
* create lib_ccxr and libccxr_exports
* chore: Fix bindgen crate version
* chore: Fix rsmpeg crate version
* docs: Add PR info in Changelogs
---------
Co-authored-by: Elbert Ronnie <elbert.ronniep@gmail.com>
* feat: Add timing functions for SCC format in C & Rust
* feat: Add SCC support to Rust 708 decoder
* feat: Add SCC support to C 708 decoder
* docs: fix symbol in scc_time format
* chore: clippy fixes
* docs: Add new feature in Changelog
* fix: update SCC timing functions according to need
* feat: Add new member(old caption end time) for overlapping situations
* fix: update SCC timing functions according to need
* feat: Add support for overlapping captions situations
* fix: frame formula for timings
* feat: Add support for orientation of subtitles in C
by adding necessary labels needed for it
* feat: Add support for orientation of subtitles in Rust
by adding necessary labels needed for it
* docs: Add info for scc labels
* chore: clippy fixes
* docs: Add what `add_needed_scc_labels` do and correct parameters name
* Update stream_functions.c: fix MP4 file type detector
On bad inputs containing e.g. the following sequence of bytes within the first 1MiB "ff ff ff ff 6d 65 74 61" `detect_stream_type` was executing an infinite loop because "ff ff ff ff" was interpreted as a length of the candidate "meta" MP4 box, caused the size_t overflow inside `isValidMP4Box` which pointed `nextBoxLocation` to the previous byte and the execution flow processed the same "meta" again.
* Update CHANGES.TXT
* Treat a candidate MP4 box as invalid instead of bailing out
* Fix stuck mp4 processing in `process_avc_sample`
On corrupted inputs it could read data past the sample end and also get stuck in an infinite loop.
* Fix the stats code to not count zero-sized NALs and avoid dereferencing memory past the NAL end
* Add comment.
* Format changes
* [FIX] Added a note for Ubuntu 23.10
libgpac-dev isn't available on Ubuntu 23.10 (Mantic) added a note instructing to build it from source instead.
* [FIX] Added build instructions for Ubuntu 23.10 and later
libgpac-dev isn't available in Ubuntu 23.10 and later, hence causing the build to fail. added the instructions to build it from source.
* feat: unpack gpac
* fix: linux ci
* fix: mac build
* fix: remove unused [no ci]
* fix: ignore config.h [no ci]
* temp commit, will drop this soon
* fix: install gpac
* fix: gpac
* fix: formatting
* fix: preproccessor directive
* fix: comment display version for now
* fix: display dlls code
* fix: bundle vcruntime in hardsubx windows
* fix: again
* fix: erros in ci
* fix: ci
* fix: add vcruntime in additional dependencies
* fix: try to copy vcruntime after build
* fix: space in runtime library
* fix: remove for now [no ci]
* fix: things in vcxproj
* fix: ci for leptonica sys
* fix: docs
* fix: copy dlls on post build event
* fix: copy vcruntime after build
* Delete (probably) wrongly committed vs config file
* Remove Nuklear GUI
* Clean up SLN configs (Reduce to 64 bit full debug & release)
* Sync bat scripts, prepare to move
* Build rust in release when release
* Update changelog
* Delete rustx86.bat
