starred/ccextractor
1
0
Fork 0
mirror of https://github.com/CCExtractor/ccextractor.git synced 2026-04-05 21:51:23 +00:00
Code Issues 1.5k Packages Projects Releases 21 Wiki Activity

Revert "Merge pull request #1912 from Rahul-2k4/final"

Browse Source 
This reverts commit 2a6d27f9ff, reversing
changes made to 74e64c0421.
This commit is contained in:
Carlos Fernandez
2026-01-18 13:28:15 -08:00
parent 2a6d27f9ff
commit 3d18b38c32
54 changed files with 452 additions and 2614 deletions
Download Patch File Download Diff File Expand all files Collapse all files

63
.gitignore vendored
View File

@@ -166,66 +166,3 @@ plans/
tess.log
**/tess.log
ut=srt*
# User custom ignores
.env
test_split_dvb/output/
test_split_dvb/logs/
# Debug/Logs
debug_*.log
debug_output/
diagnosis_output/
diag_*.log
logs/
# Test artifacts
test_output/
test_out/
test_split_verification/
test_verification_output/
linux/alltests_*
linux/multiprogram_spain_*
linux/test_*
test_04e4*
test_basename*
*_test
manual_test_*
multiprogram_spain_*
test_output_*
# Tool state (agent/debugging)
.agent/
.claude/
.gemini/
.ralph/
.ralph_session
.ralph-tui/
.call_count
.circuit_breaker_*
.exit_signals
.last_reset
.mcp.json
# Root-level scripts and temp files
build_script.sh
rebuild.sh
safe_build.sh
run_mass_test.sh
verify_dvb_split.py
write_prompt.py
update_prompt_final.py
test_basename.c
help
# Working docs and temp notes
DVB_SPLIT_*.md
test_summary.md
progress.json
progress.txt
prd.json
status.json
specs/
# Project-specific working notes
CLAUDE.md

8
docs/CHANGES.TXT
View File

@@ -4,10 +4,6 @@
- Fix: Prevent infinite loop on truncated MKV files
- Fix: Various memory safety and stability fixes in demuxers (MP4, PS, MKV, DVB)
- Fix: Delete empty output files instead of leaving 0-byte files (#1282)
- Fix: Resolve double-free crash in DVB split pipeline cleanup
- New: DVB subtitle deduplication to prevent repeated subtitles in multi-stream extraction
- Automatically detects and skips duplicate subtitles based on PTS, composition ID, and ancillary ID
- Use --no-dvb-dedup to disable deduplication if needed
0.96.5 (2026-01-05)
-------------------
@@ -33,10 +29,6 @@
0.96.3 (2025-12-29)
-------------------
- New: DVB multi-stream subtitle extraction with --split-dvb-subs option (#447)
- Extract multiple DVB subtitle streams to separate output files simultaneously
- Output files named with language and PID (e.g., output_dan_0x091F.srt)
- Handles PAT changes in transport streams without crashing
- New: VOBSUB subtitle extraction with OCR support for MP4 files
- New: VOBSUB subtitle extraction support for MKV/Matroska files
- New: Native SCC (Scenarist Closed Caption) input file support - CCExtractor can now read SCC files

115
docs/DVB_DEDUPLICATION_TECHNICAL_REPORT.md
View File

@@ -1,115 +0,0 @@
# Technical Report: DVB Subtitle Deduplication & Stability Fixes
**Author**: The Scribe (Gemini CLI)
**Date**: January 13, 2026
**Target**: Senior Engineering / Maintainers
**Scope**: `src/lib_ccx/dvb_subtitle_decoder.c`
---
## 1. Executive Summary
We have resolved a critical noise issue in DVB subtitle extraction where "carousel" streams (common in European DVB broadcasts) caused massive repetition of identical subtitle entries. For example, a 60-second clip of `multiprogram_spain.ts` generated ~2,600 duplicate entries.
**The Solution**: We implemented a lightweight, hash-based deduplication layer within the DVB decoder pipeline. By computing a checksum of the rendered bitmap and color palette, we identify and discard frames that are visually identical to the previous frame *before* they enter the encoder pipeline.
**Outcome**:
- **Repetition**: Reduced from ~2,600 entries to 1 unique entry for the test case.
- **Stability**: Fixed potential double-free and segmentation faults associated with the skipped frames.
- **Performance**: Negligible overhead (FNV-1a hash on small bitmaps).
---
## 2. Architecture Changes
We injected a validation step into the `write_dvb_sub` function. The decoder now maintains state about the *last rendered frame*.
```mermaid
graph TD
A[DVB Packet] --> B[Parse Regions/Objects];
B --> C{Render Bitmap?};
C -- No --> D[Return];
C -- Yes --> E[Compute FNV-1a Hash];
E --> F{Hash == Previous?};
F -- Yes (Duplicate) --> G[Free Memory & Abort];
F -- No (New Content) --> H[Update Prev Hash];
H --> I[Assign to Subtitle Struct];
I --> J[OCR & Output];
```
---
## 3. Implementation Deep Dive
### 3.1 Context Expansion
We extended `struct DVBSubContext` to persist the hash of the previously processed frame.
```c
typedef struct DVBSubContext {
// ... existing fields ...
uint32_t prev_bitmap_hash; // New State Tracking
// ...
} DVBSubContext;
```
This ensures that even across different calls to the decoder (as packets arrive), we remember the visual state of the subtitle stream.
### 3.2 Hashing Algorithm
We selected **FNV-1a (32-bit)** for its simplicity and speed on small data blocks (subtitle bitmaps are typically small, sparse arrays).
```c
static uint32_t fnv1a_32(const uint8_t *data, size_t len) {
uint32_t hash = 2166136261u;
for (size_t i = 0; i < len; i++) {
hash ^= data[i];
hash *= 16777619u;
}
return hash;
}
```
### 3.3 The Logic in `write_dvb_sub`
This function is responsible for assembling regions into a final `cc_bitmap`. We inserted the check immediately after rendering but *before* committing the bitmap to the `cc_subtitle` object.
**Key Logic:**
1. Render `rect->data0` (pixels) and `rect->data1` (palette).
2. Calculate `hash(pixels) ^ hash(palette)`.
3. **If Match**:
* **CRITICAL**: `free(rect->data0)`, `free(rect->data1)`, `free(rect)`.
* **CRITICAL**: Set `sub->nb_data = 0`. This signals to the downstream encoder (`encode_sub`) that no data exists, preventing null pointer dereferences.
* Return `0`.
4. **If Mismatch**:
* Update `ctx->prev_bitmap_hash`.
* Assign `sub->data = rect`.
* Set `sub->nb_data = 1`.
### 3.4 Safety & Crash Prevention
During implementation, we encountered two classes of bugs which we fixed:
1. **Double Free**: Initially, `sub->data` was assigned *before* the hash check. When we freed `rect`, the `sub` struct still held a pointer to it. Later, `free_subtitle(sub)` tried to free it again.
* **Fix**: We moved the assignment `sub->data = rect` to *after* the deduplication check.
2. **Segmentation Fault**: Returning `0` from `write_dvb_sub` without resetting `sub->nb_data` left the subtitle object in an inconsistent state (positive `nb_data` but `data == NULL`). The `encode_sub` function would then try to iterate over NULL data.
* **Fix**: Explicitly set `sub->nb_data = 0` in the skip block.
---
## 4. Verification
### 4.1 Carousel Stream (`multiprogram_spain.ts`)
* **Before**: ~2,659 entries (infinite repetitions of the same static screen).
* **After**: 1 entry.
* **Log**:
```text
Done, processing time = 3 seconds
Issues? Open a ticket here...
```
### 4.2 Standard Stream (`mp_spain...C49.ts`)
* **Check**: Regression testing.
* **Result**: Subtitles generated normally. No missing entries. File size matches expected baseline logic.
---
## 5. Conclusion
The DVB subsystem is now robust against carousel behavior. This fix is purely logical and does not rely on OCR stability, making it effective even for image-only extraction modes (like `.spupng`).

88
docs/DVB_SPLIT_FIX_TECHNICAL_REPORT.md
View File

@@ -1,88 +0,0 @@
# Post-Mortem: DVB Split Logic & Buffer Safety Fixes
**Author**: The Scribe (Gemini CLI)
**Date**: January 13, 2026
**Target**: Senior Engineering
**Scope**: `dvb_subtitle_decoder.c` logic lifecycle, `ts_functions.c` memory safety.
---
## 1. Executive Summary
We resolved two critical stability issues in the DVB Split subsystem of CCExtractor:
1. **Logic Flaw**: A "heuristic" in the DVB decoder forced the `dirty` flag to `1` without ever clearing it, causing the decoder to re-render the same region data for every incoming Transport Stream packet.
2. **Security/Crash**: An unchecked read in `ts_readstream` caused a global buffer overflow (ASan abort) when parsing malformed or short PES packets.
---
## 2. Deep Dive: The Logic Fix (Bug 1)
### The Problem
The `--split-dvb-subs` mode separates DVB streams. To handle streams that lack explicit `DISPLAY_SEGMENT` commands (like some Arte streams), a heuristic was added:
```c
if (!ctx->display_list) {
// ... build list from regions ...
r->dirty = 1; // Forced ON
}
```
This heuristic forces the decoder to render. However, because it modified the region state (which persists) and never reset it, **every subsequent call** to `write_dvb_sub` (triggered by PCR updates or padding packets) saw `dirty=1` and re-rendered the subtitle.
### The Fix
We modified `src/lib_ccx/dvb_subtitle_decoder.c` to:
1. **Disable the forced dirty flag** (`// r->dirty = 1;`). We now rely on the natural state: `region->dirty` is set to 1 only when new pixel data is actually parsed.
2. **Enforce cleanup**: Added `region->dirty = 0;` inside the rendering loop to ensure that once a region is rendered, it is marked clean until new data arrives.
**Code Change**:
```c
// src/lib_ccx/dvb_subtitle_decoder.c
// 1. Disable forced dirty
// r->dirty = 1;
// 2. Clear dirty after render
for (...) {
if (region->dirty) {
// ... render ...
}
if (ccx_options.split_dvb_subs) region->dirty = 0; // Fix
}
```
**Outcome**:
- **Regression Test**: `mp_spain...C49.ts` produces clean, non-duplicated subtitles.
- **Stress Test**: `multiprogram_spain.ts` (a problematic file) no longer causes infinite internal loops, though stream idiosyncrasies may still trigger frequent updates.
---
## 3. Deep Dive: Buffer Overflow (Bug 2)
### The Problem
In `src/lib_ccx/ts_functions.c`, the parser checked `if (payload.pesstart)` and immediately read 3 bytes from `payload.start` to check the PES prefix.
If `payload.start` pointed to the very end of the buffer (e.g., a truncated packet), this read violated memory safety.
### The Fix
We added a bounds check:
```c
if (payload.pesstart && payload.length >= 6)
```
This ensures we have enough bytes for the standard PES header before reading.
**Outcome**:
- **Before**: Immediate crash with AddressSanitizer.
- **After**: Stable execution (Exit Code 0 or 10).
---
## 4. System Diagram
```mermaid
graph TD
A[TS Packet] -->|ts_functions.c| B{Length >= 6?};
B -- No --> C[Skip (Safe)];
B -- Yes --> D[Parse DVB];
D -->|Pixel Data| E[Set Dirty=1];
E --> F[write_dvb_sub];
F --> G{Dirty?};
G -- Yes --> H[Render];
H --> I[Set Dirty=0];
I --> J[Output];
```

6
linux/Makefile.am
View File

@@ -128,10 +128,8 @@ ccextractor_SOURCES = \
../src/lib_ccx/configuration.c \
../src/lib_ccx/configuration.h \
../src/lib_ccx/disable_warnings.h \
../src/lib_ccx/dvb_subtitle_decoder.c \
../src/lib_ccx/dvb_subtitle_decoder.h \
../src/lib_ccx/dvb_dedup.c \
../src/lib_ccx/dvb_dedup.h \
../src/lib_ccx/dvb_subtitle_decoder.c \
../src/lib_ccx/dvb_subtitle_decoder.h \
../src/lib_ccx/dvd_subtitle_decoder.c \
../src/lib_ccx/dvd_subtitle_decoder.h \
../src/lib_ccx/es_functions.c \

6
mac/Makefile.am
View File

@@ -100,10 +100,8 @@ ccextractor_SOURCES = \
../src/lib_ccx/configuration.c \
../src/lib_ccx/configuration.h \
../src/lib_ccx/disable_warnings.h \
../src/lib_ccx/dvb_subtitle_decoder.c \
../src/lib_ccx/dvb_subtitle_decoder.h \
../src/lib_ccx/dvb_dedup.c \
../src/lib_ccx/dvb_dedup.h \
../src/lib_ccx/dvb_subtitle_decoder.c \
../src/lib_ccx/dvb_subtitle_decoder.h \
../src/lib_ccx/dvd_subtitle_decoder.c \
../src/lib_ccx/dvd_subtitle_decoder.h \
../src/lib_ccx/es_functions.c \

2
src/ccextractor.c
View File

@@ -221,7 +221,6 @@ int start_ccx()
if (!ret)
ret = tmp;
break;
#ifdef GPAC_AVAILABLE
case CCX_SM_MP4:
mprint("\rAnalyzing data with GPAC (MP4 library)\n");
close_input_file(ctx); // No need to have it open. GPAC will do it for us
@@ -242,7 +241,6 @@ int start_ccx()
if (!ret)
ret = tmp;
break;
#endif
case CCX_SM_MKV:
mprint("\rAnalyzing data in Matroska mode\n");
tmp = matroska_loop(ctx);

15
src/lib_ccx/CMakeLists.txt
View File

@@ -11,13 +11,10 @@ if(WIN32)
endif(WIN32)
find_package(PkgConfig)
pkg_check_modules (GPAC gpac)
pkg_check_modules (GPAC REQUIRED gpac)
if (GPAC_FOUND)
set (EXTRA_INCLUDES ${EXTRA_INCLUDES} ${GPAC_INCLUDE_DIRS})
set (EXTRA_LIBS ${EXTRA_LIBS} ${GPAC_LIBRARIES})
add_definitions(-DGPAC_AVAILABLE)
endif()
set (EXTRA_INCLUDES ${EXTRA_INCLUDES} ${GPAC_INCLUDE_DIRS})
set (EXTRA_LIBS ${EXTRA_LIBS} ${GPAC_LIBRARIES})
if (WITH_FFMPEG)
find_package(PkgConfig)
@@ -62,10 +59,6 @@ endif (WITH_OCR)
aux_source_directory ("${PROJECT_SOURCE_DIR}/lib_ccx/" SOURCEFILE)
if (NOT GPAC_FOUND)
list(FILTER SOURCEFILE EXCLUDE REGEX "mp4.c$")
endif()
add_library (ccx ${SOURCEFILE} ccx_dtvcc.h ccx_dtvcc.c ccx_encoders_mcc.c ccx_encoders_mcc.h)
target_link_libraries (ccx ${EXTRA_LIBS})
target_include_directories (ccx PUBLIC ${EXTRA_INCLUDES})
@@ -108,4 +101,4 @@ file (WRITE ccx.pc "prefix=${CMAKE_INSTALL_PREFIX}\n"
install (TARGETS ccx DESTINATION lib)
install (FILES ${HeaderFiles} DESTINATION include)
install (FILES ccx.pc DESTINATION lib/pkgconfig)
install (FILES ccx.pc DESTINATION lib/pkgconfig)

32
src/lib_ccx/cc_bitstream.h
View File

@@ -26,25 +26,25 @@ struct bitstream
int _i_bpos;
};
#define read_u8(bstream) (uint8_t) bitstream_get_num(bstream, 1, 1)
#define read_u16(bstream) (uint16_t) bitstream_get_num(bstream, 2, 1)
#define read_u32(bstream) (uint32_t) bitstream_get_num(bstream, 4, 1)
#define read_u64(bstream) (uint64_t) bitstream_get_num(bstream, 8, 1)
#define read_i8(bstream) (int8_t) bitstream_get_num(bstream, 1, 1)
#define read_i16(bstream) (int16_t) bitstream_get_num(bstream, 2, 1)
#define read_i32(bstream) (int32_t) bitstream_get_num(bstream, 4, 1)
#define read_i64(bstream) (int64_t) bitstream_get_num(bstream, 8, 1)
#define read_u8(bstream) (uint8_t)bitstream_get_num(bstream, 1, 1)
#define read_u16(bstream) (uint16_t)bitstream_get_num(bstream, 2, 1)
#define read_u32(bstream) (uint32_t)bitstream_get_num(bstream, 4, 1)
#define read_u64(bstream) (uint64_t)bitstream_get_num(bstream, 8, 1)
#define read_i8(bstream) (int8_t)bitstream_get_num(bstream, 1, 1)
#define read_i16(bstream) (int16_t)bitstream_get_num(bstream, 2, 1)
#define read_i32(bstream) (int32_t)bitstream_get_num(bstream, 4, 1)
#define read_i64(bstream) (int64_t)bitstream_get_num(bstream, 8, 1)
#define skip_u32(bstream) (void)bitstream_get_num(bstream, 4, 1)
#define next_u8(bstream) (uint8_t) bitstream_get_num(bstream, 1, 0)
#define next_u16(bstream) (uint16_t) bitstream_get_num(bstream, 2, 0)
#define next_u32(bstream) (uint32_t) bitstream_get_num(bstream, 4, 0)
#define next_u64(bstream) (uint64_t) bitstream_get_num(bstream, 8, 0)
#define next_i8(bstream) (int8_t) bitstream_get_num(bstream, 1, 0)
#define next_i16(bstream) (int16_t) bitstream_get_num(bstream, 2, 0)
#define next_i32(bstream) (int32_t) bitstream_get_num(bstream, 4, 0)
#define next_i64(bstream) (int64_t) bitstream_get_num(bstream, 8, 0)
#define next_u8(bstream) (uint8_t)bitstream_get_num(bstream, 1, 0)
#define next_u16(bstream) (uint16_t)bitstream_get_num(bstream, 2, 0)
#define next_u32(bstream) (uint32_t)bitstream_get_num(bstream, 4, 0)
#define next_u64(bstream) (uint64_t)bitstream_get_num(bstream, 8, 0)
#define next_i8(bstream) (int8_t)bitstream_get_num(bstream, 1, 0)
#define next_i16(bstream) (int16_t)bitstream_get_num(bstream, 2, 0)
#define next_i32(bstream) (int32_t)bitstream_get_num(bstream, 4, 0)
#define next_i64(bstream) (int64_t)bitstream_get_num(bstream, 8, 0)
int init_bitstream(struct bitstream *bstr, unsigned char *start, unsigned char *end);
uint64_t next_bits(struct bitstream *bstr, unsigned bnum);

3
src/lib_ccx/ccx_common_option.c
View File

@@ -181,7 +181,4 @@ void init_options(struct ccx_s_options *options)
stringztoms(DEF_VAL_STARTCREDITSFORATMOST, &options->enc_cfg.startcreditsforatmost);
stringztoms(DEF_VAL_ENDCREDITSFORATLEAST, &options->enc_cfg.endcreditsforatleast);
stringztoms(DEF_VAL_ENDCREDITSFORATMOST, &options->enc_cfg.endcreditsforatmost);
options->split_dvb_subs = 0; // Default: legacy single-stream behavior
options->no_dvb_dedup = 0; // Default: deduplication enabled
}

4
src/lib_ccx/ccx_common_option.h
View File

@@ -201,9 +201,7 @@ struct ccx_s_options // Options from user parameters
int multiprogram;
int out_interval;
int segment_on_key_frames_only;
int split_dvb_subs; // Enable per-stream DVB subtitle extraction (0=disabled, 1=enabled)
int no_dvb_dedup; // Disable DVB subtitle deduplication (0=dedup enabled, 1=dedup disabled)
int scc_framerate; // SCC input framerate: 0=29.97 (default), 1=24, 2=25, 3=30
int scc_framerate; // SCC input framerate: 0=29.97 (default), 1=24, 2=25, 3=30
#ifdef WITH_LIBCURL
char *curlposturl;
#endif

244
src/lib_ccx/ccx_common_platform.h
View File

@@ -1,122 +1,122 @@
#ifndef CCX_PLATFORM_H
#define CCX_PLATFORM_H
// Default includes (cross-platform)
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include <time.h>
#include <fcntl.h>
#include <stdarg.h>
#include <errno.h>
#define __STDC_FORMAT_MACROS
#ifdef _WIN32
#define inline _inline
#define typeof decltype
#include <io.h>
#include <ws2tcpip.h>
#include <windows.h>
#define STDIN_FILENO 0
#define STDOUT_FILENO 1
#define STDERR_FILENO 2
#include "inttypes.h"
#undef UINT64_MAX
#define UINT64_MAX _UI64_MAX
typedef int socklen_t;
#if !defined(__MINGW64__) && !defined(__MINGW32__)
typedef int ssize_t;
#endif
typedef uint32_t in_addr_t;
#ifndef IN_CLASSD
#define IN_CLASSD(i) (((INT32)(i)&0xf0000000) == 0xe0000000)
#define IN_MULTICAST(i) IN_CLASSD(i)
#endif
#include <direct.h>
#define mkdir(path, mode) _mkdir(path)
#ifndef snprintf
// Added ifndef because VS2013 warns for macro redefinition.
#define snprintf(buf, len, fmt, ...) _snprintf(buf, len, fmt, __VA_ARGS__)
#endif
#define sleep(sec) Sleep((sec)*1000)
#include <fcntl.h>
#else // _WIN32
#include <unistd.h>
#define __STDC_LIMIT_MACROS
#include <inttypes.h>
#include <stdint.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <netdb.h>
#include <sys/stat.h>
#include <sys/types.h>
#endif // _WIN32
// #include "disable_warnings.h"
#if defined(_MSC_VER) && !defined(__clang__)
#include "stdintmsc.h"
// Don't bug me with strcpy() deprecation warnings
#pragma warning(disable : 4996)
#else
#include <stdint.h>
#endif
#ifdef __OpenBSD__
#define FOPEN64 fopen
#define OPEN open
#define FSEEK fseek
#define FTELL ftell
#define LSEEK lseek
#define FSTAT fstat
#else
#ifdef _WIN32
#define OPEN _open
// 64 bit file functions
#if defined(_MSC_VER)
#define FSEEK _fseeki64
#define FTELL _ftelli64
#else
// For MinGW
#define FSEEK fseeko64
#define FTELL ftello64
#endif
#define TELL _telli64
#define LSEEK _lseeki64
typedef struct _stati64 FSTATSTRUCT;
#else
// Linux internally maps these functions to 64bit usage,
// if _FILE_OFFSET_BITS macro is set to 64
#define FOPEN64 fopen
#define OPEN open
#define LSEEK lseek
#define FSEEK fseek
#define FTELL ftell
#define FSTAT fstat
#define TELL tell
#include <stdint.h>
#endif
#endif
#ifndef int64_t_C
#define int64_t_C(c) (c##LL)
#define uint64_t_C(c) (c##ULL)
#endif
#ifndef O_BINARY
#define O_BINARY 0 // Not present in Linux because it's always binary
#endif
#ifndef max
#define max(a, b) (((a) > (b)) ? (a) : (b))
#endif
typedef int64_t LLONG;
typedef uint64_t ULLONG;
typedef uint8_t UBYTE;
#endif // CCX_PLATFORM_H
#ifndef CCX_PLATFORM_H
#define CCX_PLATFORM_H
// Default includes (cross-platform)
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include <time.h>
#include <fcntl.h>
#include <stdarg.h>
#include <errno.h>
#define __STDC_FORMAT_MACROS
#ifdef _WIN32
#define inline _inline
#define typeof decltype
#include <io.h>
#include <ws2tcpip.h>
#include <windows.h>
#define STDIN_FILENO 0
#define STDOUT_FILENO 1
#define STDERR_FILENO 2
#include "inttypes.h"
#undef UINT64_MAX
#define UINT64_MAX _UI64_MAX
typedef int socklen_t;
#if !defined(__MINGW64__) && !defined(__MINGW32__)
typedef int ssize_t;
#endif
typedef uint32_t in_addr_t;
#ifndef IN_CLASSD
#define IN_CLASSD(i) (((INT32)(i) & 0xf0000000) == 0xe0000000)
#define IN_MULTICAST(i) IN_CLASSD(i)
#endif
#include <direct.h>
#define mkdir(path, mode) _mkdir(path)
#ifndef snprintf
// Added ifndef because VS2013 warns for macro redefinition.
#define snprintf(buf, len, fmt, ...) _snprintf(buf, len, fmt, __VA_ARGS__)
#endif
#define sleep(sec) Sleep((sec) * 1000)
#include <fcntl.h>
#else // _WIN32
#include <unistd.h>
#define __STDC_LIMIT_MACROS
#include <inttypes.h>
#include <stdint.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <netdb.h>
#include <sys/stat.h>
#include <sys/types.h>
#endif // _WIN32
// #include "disable_warnings.h"
#if defined(_MSC_VER) && !defined(__clang__)
#include "stdintmsc.h"
// Don't bug me with strcpy() deprecation warnings
#pragma warning(disable : 4996)
#else
#include <stdint.h>
#endif
#ifdef __OpenBSD__
#define FOPEN64 fopen
#define OPEN open
#define FSEEK fseek
#define FTELL ftell
#define LSEEK lseek
#define FSTAT fstat
#else
#ifdef _WIN32
#define OPEN _open
// 64 bit file functions
#if defined(_MSC_VER)
#define FSEEK _fseeki64
#define FTELL _ftelli64
#else
// For MinGW
#define FSEEK fseeko64
#define FTELL ftello64
#endif
#define TELL _telli64
#define LSEEK _lseeki64
typedef struct _stati64 FSTATSTRUCT;
#else
// Linux internally maps these functions to 64bit usage,
// if _FILE_OFFSET_BITS macro is set to 64
#define FOPEN64 fopen
#define OPEN open
#define LSEEK lseek
#define FSEEK fseek
#define FTELL ftell
#define FSTAT fstat
#define TELL tell
#include <stdint.h>
#endif
#endif
#ifndef int64_t_C
#define int64_t_C(c) (c##LL)
#define uint64_t_C(c) (c##ULL)
#endif
#ifndef O_BINARY
#define O_BINARY 0 // Not present in Linux because it's always binary
#endif
#ifndef max
#define max(a, b) (((a) > (b)) ? (a) : (b))
#endif
typedef int64_t LLONG;
typedef uint64_t ULLONG;
typedef uint8_t UBYTE;
#endif // CCX_PLATFORM_H

58
src/lib_ccx/ccx_decoders_common.c
View File

@@ -513,13 +513,11 @@ void flush_cc_decode(struct lib_cc_decode *ctx, struct cc_subtitle *sub)
}
struct encoder_ctx *copy_encoder_context(struct encoder_ctx *ctx)
{
// mprint("DEBUG-TRACE: copy_encoder_context entered\n"); fflush(stdout);
struct encoder_ctx *ctx_copy = NULL;
ctx_copy = malloc(sizeof(struct encoder_ctx));
if (!ctx_copy)
fatal(EXIT_NOT_ENOUGH_MEMORY, "In copy_encoder_context: Out of memory allocating ctx_copy.");
memcpy(ctx_copy, ctx, sizeof(struct encoder_ctx));
// mprint("DEBUG-TRACE: copy_encoder_context struct copied\n"); fflush(stdout);
// Initialize copied pointers to NULL before re-allocating
ctx_copy->buffer = NULL;
@@ -531,7 +529,7 @@ struct encoder_ctx *copy_encoder_context(struct encoder_ctx *ctx)
ctx_copy->start_credits_text = NULL;
ctx_copy->end_credits_text = NULL;
ctx_copy->prev = NULL;
ctx_copy->last_str = NULL;
ctx_copy->last_string = NULL;
if (ctx->buffer)
{
@@ -557,7 +555,6 @@ struct encoder_ctx *copy_encoder_context(struct encoder_ctx *ctx)
}
if (ctx->timing)
{
// mprint("DEBUG-TRACE: copy_encoder_context copying timing\n"); fflush(stdout);
ctx_copy->timing = malloc(sizeof(struct ccx_common_timing_ctx));
if (!ctx_copy->timing)
fatal(EXIT_NOT_ENOUGH_MEMORY, "In copy_encoder_context: Out of memory allocating timing.");
@@ -673,53 +670,13 @@ struct cc_subtitle *copy_subtitle(struct cc_subtitle *sub)
memcpy(sub_copy, sub, sizeof(struct cc_subtitle));
sub_copy->datatype = sub->datatype;
sub_copy->data = NULL;
sub_copy->prev = NULL; // Don't copy prev chain to avoid double-free
if (sub->data)
{
if (sub->datatype == CC_DATATYPE_DVB)
{
// Deep copy for DVB bitmap - must copy pixel data to avoid aliasing
struct cc_bitmap *src_bmp = (struct cc_bitmap *)sub->data;
struct cc_bitmap *dst_bmp = malloc(sizeof(struct cc_bitmap));
if (!dst_bmp)
fatal(EXIT_NOT_ENOUGH_MEMORY, "In copy_subtitle: Out of memory allocating bitmap.");
memcpy(dst_bmp, src_bmp, sizeof(struct cc_bitmap));
// Deep copy pixel data buffers (these are the actual bitmap pixels)
dst_bmp->data0 = NULL;
dst_bmp->data1 = NULL;
if (src_bmp->data0 && src_bmp->h > 0 && src_bmp->linesize0 > 0)
{
size_t size0 = (size_t)src_bmp->h * (size_t)src_bmp->linesize0;
dst_bmp->data0 = malloc(size0);
if (dst_bmp->data0)
memcpy(dst_bmp->data0, src_bmp->data0, size0);
}
if (src_bmp->data1 && src_bmp->h > 0 && src_bmp->linesize1 > 0)
{
size_t size1 = (size_t)src_bmp->h * (size_t)src_bmp->linesize1;
dst_bmp->data1 = malloc(size1);
if (dst_bmp->data1)
memcpy(dst_bmp->data1, src_bmp->data1, size1);
}
#ifdef ENABLE_OCR
// Deep copy OCR text if present
if (src_bmp->ocr_text)
dst_bmp->ocr_text = strdup(src_bmp->ocr_text);
else
dst_bmp->ocr_text = NULL;
#endif
sub_copy->data = dst_bmp;
}
else
{
// Original behavior for eia608_screen (608 captions)
sub_copy->data = malloc(sub->nb_data * sizeof(struct eia608_screen));
if (!sub_copy->data)
fatal(EXIT_NOT_ENOUGH_MEMORY, "In copy_subtitle: Out of memory allocating data.");
memcpy(sub_copy->data, sub->data, sub->nb_data * sizeof(struct eia608_screen));
}
sub_copy->data = malloc(sub->nb_data * sizeof(struct eia608_screen));
if (!sub_copy->data)
fatal(EXIT_NOT_ENOUGH_MEMORY, "In copy_subtitle: Out of memory allocating data.");
memcpy(sub_copy->data, sub->data, sub->nb_data * sizeof(struct eia608_screen));
}
return sub_copy;
}
@@ -736,11 +693,8 @@ void free_encoder_context(struct encoder_ctx *ctx)
freep(&ctx->subline);
freep(&ctx->start_credits_text);
freep(&ctx->end_credits_text);
// NOTE: Do NOT recurse into ctx->prev here. Ownership of prev is managed by
// dinit_encoder() which explicitly calls free_encoder_context(ctx->prev).
// Recursing here causes double-free when dinit_encoder cleans up.
freep(&ctx->prev);
freep(&ctx->last_str);
freep(&ctx->last_string);
freep(&ctx);
}
void free_decoder_context(struct lib_cc_decode *ctx)

80
src/lib_ccx/ccx_decoders_isdb.c
View File

@@ -141,77 +141,29 @@ typedef uint32_t rgba;
static rgba Default_clut[128] =
{
// 0-7
RGBA(0, 0, 0, 255),
RGBA(255, 0, 0, 255),
RGBA(0, 255, 0, 255),
RGBA(255, 255, 0, 255),
RGBA(0, 0, 255, 255),
RGBA(255, 0, 255, 255),
RGBA(0, 255, 255, 255),
RGBA(255, 255, 255, 255),
RGBA(0, 0, 0, 255), RGBA(255, 0, 0, 255), RGBA(0, 255, 0, 255), RGBA(255, 255, 0, 255),
RGBA(0, 0, 255, 255), RGBA(255, 0, 255, 255), RGBA(0, 255, 255, 255), RGBA(255, 255, 255, 255),
// 8-15
RGBA(0, 0, 0, 0),
RGBA(170, 0, 0, 255),
RGBA(0, 170, 0, 255),
RGBA(170, 170, 0, 255),
RGBA(0, 0, 170, 255),
RGBA(170, 0, 170, 255),
RGBA(0, 170, 170, 255),
RGBA(170, 170, 170, 255),
RGBA(0, 0, 0, 0), RGBA(170, 0, 0, 255), RGBA(0, 170, 0, 255), RGBA(170, 170, 0, 255),
RGBA(0, 0, 170, 255), RGBA(170, 0, 170, 255), RGBA(0, 170, 170, 255), RGBA(170, 170, 170, 255),
// 16-23
RGBA(0, 0, 85, 255),
RGBA(0, 85, 0, 255),
RGBA(0, 85, 85, 255),
RGBA(0, 85, 170, 255),
RGBA(0, 85, 255, 255),
RGBA(0, 170, 85, 255),
RGBA(0, 170, 255, 255),
RGBA(0, 255, 85, 255),
RGBA(0, 0, 85, 255), RGBA(0, 85, 0, 255), RGBA(0, 85, 85, 255), RGBA(0, 85, 170, 255),
RGBA(0, 85, 255, 255), RGBA(0, 170, 85, 255), RGBA(0, 170, 255, 255), RGBA(0, 255, 85, 255),
// 24-31
RGBA(0, 255, 170, 255),
RGBA(85, 0, 0, 255),
RGBA(85, 0, 85, 255),
RGBA(85, 0, 170, 255),
RGBA(85, 0, 255, 255),
RGBA(85, 85, 0, 255),
RGBA(85, 85, 85, 255),
RGBA(85, 85, 170, 255),
RGBA(0, 255, 170, 255), RGBA(85, 0, 0, 255), RGBA(85, 0, 85, 255), RGBA(85, 0, 170, 255),
RGBA(85, 0, 255, 255), RGBA(85, 85, 0, 255), RGBA(85, 85, 85, 255), RGBA(85, 85, 170, 255),
// 32-39
RGBA(85, 85, 255, 255),
RGBA(85, 170, 0, 255),
RGBA(85, 170, 85, 255),
RGBA(85, 170, 170, 255),
RGBA(85, 170, 255, 255),
RGBA(85, 255, 0, 255),
RGBA(85, 255, 85, 255),
RGBA(85, 255, 170, 255),
RGBA(85, 85, 255, 255), RGBA(85, 170, 0, 255), RGBA(85, 170, 85, 255), RGBA(85, 170, 170, 255),
RGBA(85, 170, 255, 255), RGBA(85, 255, 0, 255), RGBA(85, 255, 85, 255), RGBA(85, 255, 170, 255),
// 40-47
RGBA(85, 255, 255, 255),
RGBA(170, 0, 85, 255),
RGBA(170, 0, 255, 255),
RGBA(170, 85, 0, 255),
RGBA(170, 85, 85, 255),
RGBA(170, 85, 170, 255),
RGBA(170, 85, 255, 255),
RGBA(170, 170, 85, 255),
RGBA(85, 255, 255, 255), RGBA(170, 0, 85, 255), RGBA(170, 0, 255, 255), RGBA(170, 85, 0, 255),
RGBA(170, 85, 85, 255), RGBA(170, 85, 170, 255), RGBA(170, 85, 255, 255), RGBA(170, 170, 85, 255),
// 48-55
RGBA(170, 170, 255, 255),
RGBA(170, 255, 0, 255),
RGBA(170, 255, 85, 255),
RGBA(170, 255, 170, 255),
RGBA(170, 255, 255, 255),
RGBA(255, 0, 85, 255),
RGBA(255, 0, 170, 255),
RGBA(255, 85, 0, 255),
RGBA(170, 170, 255, 255), RGBA(170, 255, 0, 255), RGBA(170, 255, 85, 255), RGBA(170, 255, 170, 255),
RGBA(170, 255, 255, 255), RGBA(255, 0, 85, 255), RGBA(255, 0, 170, 255), RGBA(255, 85, 0, 255),
// 56-63
RGBA(255, 85, 85, 255),
RGBA(255, 85, 170, 255),
RGBA(255, 85, 255, 255),
RGBA(255, 170, 0, 255),
RGBA(255, 170, 85, 255),
RGBA(255, 170, 170, 255),
RGBA(255, 170, 255, 255),
RGBA(255, 255, 85, 255),
RGBA(255, 85, 85, 255), RGBA(255, 85, 170, 255), RGBA(255, 85, 255, 255), RGBA(255, 170, 0, 255),
RGBA(255, 170, 85, 255), RGBA(255, 170, 170, 255), RGBA(255, 170, 255, 255), RGBA(255, 255, 85, 255),
// 64
RGBA(255, 255, 170, 255),
// 65-127 are calculated later.

8
src/lib_ccx/ccx_demuxer.c
View File

@@ -323,11 +323,6 @@ void ccx_demuxer_delete(struct ccx_demuxer **ctx)
}
freep(&lctx->filebuffer);
// Reset potential stream discovery data
lctx->potential_stream_count = 0;
memset(lctx->potential_streams, 0, sizeof(lctx->potential_streams));
freep(ctx);
}
@@ -411,9 +406,6 @@ struct ccx_demuxer *init_demuxer(void *parent, struct demuxer_cfg *cfg)
init_ts(ctx);
ctx->filebuffer = NULL;
// Initialize stream discovery for multi-stream DVB subtitle extraction
ctx->potential_stream_count = 0;
return ctx;
}

30
src/lib_ccx/ccx_demuxer.h
View File

@@ -31,26 +31,6 @@ struct ccx_demux_report
unsigned mp4_cc_track_cnt;
};
#define MAX_POTENTIAL_STREAMS 64
/** Stream type identifiers for internal classification */
#define CCX_STREAM_TYPE_UNKNOWN 0
#define CCX_STREAM_TYPE_DVB_SUB 1
#define CCX_STREAM_TYPE_TELETEXT 2
/**
* ccx_stream_metadata - Metadata for a discovered subtitle stream
*/
struct ccx_stream_metadata
{
int pid; // Transport Stream Packet ID (0-8191)
int stream_type; // Logical type (CCX_STREAM_TYPE_*)
int mpeg_type; // Raw MPEG stream type from PMT (e.g., 0x06)
char lang[4]; // ISO 639-2/B three-letter language code
int composition_id;
int ancillary_id;
};
struct program_info
{
int pid;
@@ -67,7 +47,7 @@ struct program_info
int16_t pcr_pid;
uint64_t got_important_streams_min_pts[COUNT];
int has_all_min_pts;
char virtual_channel[16]; // Stores ATSC virtual channel like "2.1"
char virtual_channel[16]; // Stores ATSC virtual channel like "2.1"
};
struct cap_info
@@ -83,7 +63,6 @@ struct cap_info
int prev_counter;
void *codec_private_data;
int ignore;
char lang[4]; // ISO 639-2 language code for DVB split mode
/**
List joining all stream in TS
@@ -183,12 +162,7 @@ struct ccx_demuxer
int (*open)(struct ccx_demuxer *ctx, const char *file_name);
int (*is_open)(struct ccx_demuxer *ctx);
int (*get_stream_mode)(struct ccx_demuxer *ctx);
LLONG(*get_filesize)
(struct ccx_demuxer *ctx);
// Stream discovery for multi-stream DVB subtitle extraction
struct ccx_stream_metadata potential_streams[MAX_POTENTIAL_STREAMS];
int potential_stream_count;
LLONG (*get_filesize)(struct ccx_demuxer *ctx);
};
struct demuxer_data

21
src/lib_ccx/ccx_encoders_common.c
View File

@@ -723,8 +723,6 @@ void dinit_encoder(struct encoder_ctx **arg, LLONG current_fts)
dinit_teletext_outputs(ctx);
free_encoder_context(ctx->prev);
ctx->prev = NULL; // Ensure it's nulled after freeing
freep(&ctx->last_str);
dinit_output_ctx(ctx);
freep(&ctx->subline);
freep(&ctx->buffer);
@@ -742,8 +740,7 @@ struct encoder_ctx *init_encoder(struct encoder_cfg *opt)
{
int ret;
int i;
// Use calloc to initialize all fields to 0/NULL (Safety fix for copy_encoder_context)
struct encoder_ctx *ctx = calloc(1, sizeof(struct encoder_ctx));
struct encoder_ctx *ctx = malloc(sizeof(struct encoder_ctx));
if (!ctx)
return NULL;
@@ -791,20 +788,10 @@ struct encoder_ctx *init_encoder(struct encoder_cfg *opt)
ctx->encoding = opt->encoding;
ctx->write_format = opt->write_format;
ctx->last_str = NULL;
ctx->is_mkv = 0;
ctx->last_string = NULL;
// Deep copy transcript settings because opt is often stack-allocated and temporary
// Storing &opt->transcript_settings leads to Use-After-Free in copy_encoder_context
ctx->transcript_settings = malloc(sizeof(struct ccx_encoders_transcript_format));
if (ctx->transcript_settings)
memcpy(ctx->transcript_settings, &opt->transcript_settings, sizeof(struct ccx_encoders_transcript_format));
else
{
freep(&ctx->buffer);
dinit_output_ctx(ctx);
free(ctx);
return NULL;
}
ctx->transcript_settings = &opt->transcript_settings;
ctx->no_bom = opt->no_bom;
ctx->sentence_cap = opt->sentence_cap;
ctx->filter_profanity = opt->filter_profanity;

14
src/lib_ccx/ccx_encoders_common.h
View File

@@ -173,8 +173,8 @@ struct encoder_ctx
struct encoder_ctx *prev;
int write_previous;
// for dvb in .mkv
int is_mkv; // are we working with .mkv file
char *last_str; // last recognized DVB sub
int is_mkv; // are we working with .mkv file
char *last_string; // last recognized DVB sub
// Segmenting
int segment_pending;
@@ -245,11 +245,11 @@ int write_cc_subtitle_as_smptett(struct cc_subtitle *sub, struct encoder_ctx *co
int write_cc_subtitle_as_spupng(struct cc_subtitle *sub, struct encoder_ctx *context);
int write_cc_subtitle_as_transcript(struct cc_subtitle *sub, struct encoder_ctx *context);
int write_stringz_as_srt(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);
int write_stringz_as_ssa(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);
int write_stringz_as_webvtt(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);
int write_stringz_as_sami(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);
void write_stringz_as_smptett(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);
int write_stringz_as_srt(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);
int write_stringz_as_ssa(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);
int write_stringz_as_webvtt(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);
int write_stringz_as_sami(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);
void write_stringz_as_smptett(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);
int write_cc_bitmap_as_srt(struct cc_subtitle *sub, struct encoder_ctx *context);
int write_cc_bitmap_as_ssa(struct cc_subtitle *sub, struct encoder_ctx *context);

8
src/lib_ccx/ccx_encoders_sami.c
View File

@@ -6,7 +6,7 @@
#include "utility.h"
#include "ccx_encoders_helpers.h"
int write_stringz_as_sami(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end)
int write_stringz_as_sami(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end)
{
int used;
int len = 0;
@@ -26,7 +26,7 @@ int write_stringz_as_sami(char *str_arg, struct encoder_ctx *context, LLONG ms_s
if (ret != used)
return ret;
len = strlen(str_arg);
len = strlen(string);
unescaped = (unsigned char *)malloc(len + 1);
if (unescaped == NULL)
{
@@ -47,14 +47,14 @@ int write_stringz_as_sami(char *str_arg, struct encoder_ctx *context, LLONG ms_s
// Scan for \n in the string and replace it with a 0
while (pos_r < len)
{
if (str_arg[pos_r] == '\\' && str_arg[pos_r + 1] == 'n')
if (string[pos_r] == '\\' && string[pos_r + 1] == 'n')
{
unescaped[pos_w] = 0;
pos_r += 2;
}
else
{
unescaped[pos_w] = str_arg[pos_r];
unescaped[pos_w] = string[pos_r];
pos_r++;
}
pos_w++;

8
src/lib_ccx/ccx_encoders_smptett.c
View File

@@ -29,12 +29,12 @@
#include "utility.h"
#include "ccx_encoders_helpers.h"
void write_stringz_as_smptett(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end)
void write_stringz_as_smptett(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end)
{
int used;
unsigned h1, m1, s1, ms1;
unsigned h2, m2, s2, ms2;
int len = strlen(str_arg);
int len = strlen(string);
unsigned char *unescaped = (unsigned char *)malloc(len + 1);
if (!unescaped)
fatal(EXIT_NOT_ENOUGH_MEMORY, "In write_stringz_as_smptett() - not enough memory for unescaped buffer.\n");
@@ -61,14 +61,14 @@ void write_stringz_as_smptett(char *str_arg, struct encoder_ctx *context, LLONG
// Scan for \n in the string and replace it with a 0
while (pos_r < len)
{
if (str_arg[pos_r] == '\\' && str_arg[pos_r + 1] == 'n')
if (string[pos_r] == '\\' && string[pos_r + 1] == 'n')
{
unescaped[pos_w] = 0;
pos_r += 2;
}
else
{
unescaped[pos_w] = str_arg[pos_r];
unescaped[pos_w] = string[pos_r];
pos_r++;
}
pos_w++;

16
src/lib_ccx/ccx_encoders_srt.c
View File

@@ -8,7 +8,7 @@
/* Helper function to write SRT to a specific output file (issue #665 - teletext multi-page)
Takes output file descriptor and counter pointer as parameters */
static int write_stringz_as_srt_to_output(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end,
static int write_stringz_as_srt_to_output(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end,
int out_fh, unsigned int *srt_counter)
{
int used;
@@ -16,7 +16,7 @@ static int write_stringz_as_srt_to_output(char *str_arg, struct encoder_ctx *con
unsigned h2, m2, s2, ms2;
char timeline[128];
if (!str_arg || !str_arg[0])
if (!string || !string[0])
return 0;
millis_to_time(ms_start, &h1, &m1, &s1, &ms1);
@@ -32,7 +32,7 @@ static int write_stringz_as_srt_to_output(char *str_arg, struct encoder_ctx *con
dbg_print(CCX_DMT_DECODER_608, "%s", timeline);
write_wrapped(out_fh, context->buffer, used);
int len = strlen(str_arg);
int len = strlen(string);
unsigned char *unescaped = (unsigned char *)malloc(len + 1);
if (!unescaped)
fatal(EXIT_NOT_ENOUGH_MEMORY, "In write_stringz_as_srt() - not enough memory for unescaped buffer.\n");
@@ -47,14 +47,14 @@ static int write_stringz_as_srt_to_output(char *str_arg, struct encoder_ctx *con
// Scan for \n in the string and replace it with a 0
while (pos_r < len)
{
if (str_arg[pos_r] == '\\' && str_arg[pos_r + 1] == 'n')
if (string[pos_r] == '\\' && string[pos_r + 1] == 'n')
{
unescaped[pos_w] = 0;
pos_r += 2;
}
else
{
unescaped[pos_w] = str_arg[pos_r];
unescaped[pos_w] = string[pos_r];
pos_r++;
}
pos_w++;
@@ -86,9 +86,9 @@ static int write_stringz_as_srt_to_output(char *str_arg, struct encoder_ctx *con
/* The timing here is not PTS based, but output based, i.e. user delay must be accounted for
if there is any */
int write_stringz_as_srt(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end)
int write_stringz_as_srt(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end)
{
return write_stringz_as_srt_to_output(str_arg, context, ms_start, ms_end,
return write_stringz_as_srt_to_output(string, context, ms_start, ms_end,
context->out->fh, &context->srt_counter);
}
@@ -117,7 +117,7 @@ int write_cc_bitmap_as_srt(struct cc_subtitle *sub, struct encoder_ctx *context)
if (context->is_mkv == 1)
{
// Save recognized string for later use in matroska.c
context->last_str = str;
context->last_string = str;
}
else
{

10
src/lib_ccx/ccx_encoders_ssa.c
View File

@@ -5,14 +5,14 @@
#include "ccx_encoders_helpers.h"
#include "ocr.h"
int write_stringz_as_ssa(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end)
int write_stringz_as_ssa(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end)
{
int used;
unsigned h1, m1, s1, ms1;
unsigned h2, m2, s2, ms2;
char timeline[128];
if (!str_arg || !str_arg[0])
if (!string || !string[0])
return 0;
millis_to_time(ms_start, &h1, &m1, &s1, &ms1);
@@ -25,7 +25,7 @@ int write_stringz_as_ssa(char *str_arg, struct encoder_ctx *context, LLONG ms_st
dbg_print(CCX_DMT_DECODER_608, "%s", timeline);
write_wrapped(context->out->fh, context->buffer, used);
int len = strlen(str_arg);
int len = strlen(string);
unsigned char *unescaped = (unsigned char *)malloc(len + 1);
if (!unescaped)
fatal(EXIT_NOT_ENOUGH_MEMORY, "In write_stringz_as_ssa() - not enough memory for unescaped buffer.\n");
@@ -40,14 +40,14 @@ int write_stringz_as_ssa(char *str_arg, struct encoder_ctx *context, LLONG ms_st
// Scan for \n in the string and replace it with a 0
while (pos_r < len)
{
if (str_arg[pos_r] == '\\' && str_arg[pos_r + 1] == 'n')
if (string[pos_r] == '\\' && string[pos_r + 1] == 'n')
{
unescaped[pos_w] = 0;
pos_r += 2;
}
else
{
unescaped[pos_w] = str_arg[pos_r];
unescaped[pos_w] = string[pos_r];
pos_r++;
}
pos_w++;

8
src/lib_ccx/ccx_encoders_webvtt.c
View File

@@ -120,7 +120,7 @@ static const char *webvtt_pac_row_percent[] = {"10", "15.33", "20.66", "26", "31
/* The timing here is not PTS based, but output based, i.e. user delay must be accounted for
if there is any */
int write_stringz_as_webvtt(char *str_arg, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end)
int write_stringz_as_webvtt(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end)
{
int used;
unsigned h1, m1, s1, ms1;
@@ -140,7 +140,7 @@ int write_stringz_as_webvtt(char *str_arg, struct encoder_ctx *context, LLONG ms
written = write(context->out->fh, context->buffer, used);
if (written != used)
return -1;
int len = strlen(str_arg);
int len = strlen(string);
unsigned char *unescaped = (unsigned char *)malloc(len + 1);
if (!unescaped)
fatal(EXIT_NOT_ENOUGH_MEMORY, "In write_stringz_as_webvtt() - not enough memory for unescaped buffer.\n");
@@ -155,14 +155,14 @@ int write_stringz_as_webvtt(char *str_arg, struct encoder_ctx *context, LLONG ms
// Scan for \n in the string and replace it with a 0
while (pos_r < len)
{
if (str_arg[pos_r] == '\\' && str_arg[pos_r + 1] == 'n')
if (string[pos_r] == '\\' && string[pos_r + 1] == 'n')
{
unescaped[pos_w] = 0;
pos_r += 2;
}
else
{
unescaped[pos_w] = str_arg[pos_r];
unescaped[pos_w] = string[pos_r];
pos_r++;
}
pos_w++;

48
src/lib_ccx/dvb_dedup.c
View File

@@ -1,48 +0,0 @@
#include "dvb_dedup.h"
#include <string.h>
void dvb_dedup_init(struct dvb_dedup_ring *ring)
{
if (!ring)
return;
memset(ring, 0, sizeof(*ring));
ring->head = 0;
}
int dvb_dedup_is_duplicate(struct dvb_dedup_ring *ring,
uint64_t pts, uint32_t pid,
uint16_t composition_id, uint16_t ancillary_id)
{
if (!ring)
return 0;
for (int i = 0; i < DVB_DEDUP_RING_SIZE; i++)
{
const struct dvb_dedup_entry *e = &ring->entries[i];
if (e->pts == pts &&
e->pid == pid &&
e->composition_id == composition_id &&
e->ancillary_id == ancillary_id)
{
return 1;
}
}
return 0;
}
void dvb_dedup_add(struct dvb_dedup_ring *ring,
uint64_t pts, uint32_t pid,
uint16_t composition_id, uint16_t ancillary_id)
{
if (!ring)
return;
ring->entries[ring->head].pts = pts;
ring->entries[ring->head].pid = pid;
ring->entries[ring->head].composition_id = composition_id;
ring->entries[ring->head].ancillary_id = ancillary_id;
ring->head = (ring->head + 1) % DVB_DEDUP_RING_SIZE;
}

34
src/lib_ccx/dvb_dedup.h
View File

@@ -1,34 +0,0 @@
/*
* DVB subtitle deduplication ring buffer
* Prevents repeated subtitles from propagating to output files
*/
#ifndef DVBSUB_DEDUP_H
#define DVBSUB_DEDUP_H
#include <stdint.h>
#include <stdlib.h>
#define DVB_DEDUP_RING_SIZE 8
struct dvb_dedup_entry
{
uint64_t pts; /* Presentation timestamp (ms) */
uint32_t pid; /* Stream PID */
uint16_t composition_id; /* DVB composition ID */
uint16_t ancillary_id; /* DVB ancillary ID */
uint8_t valid; /* Entry is valid */
};
struct dvb_dedup_ring
{
struct dvb_dedup_entry entries[DVB_DEDUP_RING_SIZE];
uint8_t head; /* Next position to write */
uint8_t count; /* Number of valid entries */
};
void dvb_dedup_init(struct dvb_dedup_ring *ring);
int dvb_dedup_is_duplicate(struct dvb_dedup_ring *ring, uint64_t pts, uint32_t pid, uint16_t composition_id, uint16_t ancillary_id);
void dvb_dedup_add(struct dvb_dedup_ring *ring, uint64_t pts, uint32_t pid, uint16_t composition_id, uint16_t ancillary_id);
#endif /* DVBSUB_DEDUP_H */

606
src/lib_ccx/dvb_subtitle_decoder.c
View File

@@ -25,26 +25,6 @@
#include "utility.h"
#include "ccx_decoders_common.h"
#include "ocr.h"
#include "dvb_dedup.h"
#include "ccx_common_option.h"
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>
#include <time.h>
// Debug stub for dump_rect_and_log - used in OCR debugging but not critical
static void dump_rect_and_log(const char *label, const uint8_t *data, int w, int h, int linesize, int mode, int pid, int idx)
{
// Intentionally empty - enable for debugging bitmap data if needed
(void)label;
(void)data;
(void)w;
(void)h;
(void)linesize;
(void)mode;
(void)pid;
(void)idx;
}
#define DVBSUB_PAGE_SEGMENT 0x10
#define DVBSUB_REGION_SEGMENT 0x11
@@ -53,11 +33,6 @@ static void dump_rect_and_log(const char *label, const uint8_t *data, int w, int
#define DVBSUB_DISPLAYDEFINITION_SEGMENT 0x14
#define DVBSUB_DISPLAY_SEGMENT 0x80
// Maximum reasonable subtitle duration in milliseconds.
// DVB page timeouts can be very long (e.g., 65 seconds), but actual subtitles
// rarely exceed 10 seconds. This cap prevents corrupt timestamps from page timeouts.
#define DVB_MAX_SUBTITLE_DURATION_MS 10000
#define SCALEBITS 10
#define ONE_HALF (1 << (SCALEBITS - 1))
#define FIX(x) ((int)((x) * (1 << SCALEBITS) + 0.5))
@@ -205,7 +180,6 @@ typedef struct DVBSubContext
int version;
/* Store time in ms */
LLONG time_out;
uint32_t prev_bitmap_hash;
#ifdef ENABLE_OCR
void *ocr_ctx;
int ocr_initialized; // Flag to track if OCR has been lazily initialized
@@ -216,45 +190,8 @@ typedef struct DVBSubContext
DVBSubRegionDisplay *display_list;
DVBSubDisplayDefinition *display_definition;
struct dvb_dedup_ring dedup_ring; // Deduplication ring buffer
} DVBSubContext;
size_t dvbsub_get_context_size(void)
{
return sizeof(DVBSubContext);
}
void dvbsub_copy_context(void *dst, void *src)
{
if (dst && src)
{
DVBSubContext *d = (DVBSubContext *)dst;
DVBSubContext *s = (DVBSubContext *)src;
// Copy scalar values only - DO NOT copy pointers to avoid aliasing
// The linked lists (region_list, clut_list, object_list, display_list)
// are owned by the source context and must not be shared
d->composition_id = s->composition_id;
d->ancillary_id = s->ancillary_id;
d->lang_index = s->lang_index;
d->version = s->version;
d->time_out = s->time_out;
// Initialize pointers to NULL to avoid use-after-free
d->region_list = NULL;
d->clut_list = NULL;
d->object_list = NULL;
d->display_list = NULL;
d->display_definition = NULL;
#ifdef ENABLE_OCR
// OCR context is shared, just copy the pointer (it's managed externally)
d->ocr_ctx = s->ocr_ctx;
#endif
}
}
static __inline unsigned int bytestream_get_byte(const uint8_t **b)
{
(*b) += 1;
@@ -492,9 +429,6 @@ void *dvbsub_init_decoder(struct dvb_config *cfg)
}
memset(ctx, 0, sizeof(DVBSubContext));
// Initialize deduplication ring buffer
dvb_dedup_init(&ctx->dedup_ring);
if (cfg)
{
ctx->composition_id = cfg->composition_id[0];
@@ -516,7 +450,6 @@ void *dvbsub_init_decoder(struct dvb_config *cfg)
ctx->ocr_initialized = 0;
#endif
ctx->version = -1;
ctx->prev_bitmap_hash = 0;
default_clut.id = -1;
default_clut.next = NULL;
@@ -1139,23 +1072,18 @@ static int dvbsub_parse_object_segment(void *dvb_ctx, const uint8_t *buf,
object = get_object(ctx, object_id);
if (!object)
return 0;
return 0; // Unsure if we should return error
int version = ((*buf) >> 4) & 15;
coding_method = ((*buf) >> 2) & 3;
non_modifying_color = ((*buf++) >> 1) & 1;
if (object->version == version)
return 0;
object->version = version;
if (coding_method == 0)
{
top_field_len = RB16(buf);
buf += 2;
bottom_field_len = RB16(buf);
buf += 2;
if (buf + top_field_len + bottom_field_len > buf_end)
{
mprint("dvbsub_parse_object_segment(): Field data size too large\n");
@@ -1185,6 +1113,7 @@ static int dvbsub_parse_object_segment(void *dvb_ctx, const uint8_t *buf,
if (dvbsub_parse_pixel_data_block(dvb_ctx, display, block, bfl, 1,
non_modifying_color))
{
// Problems. Hope for the best.
mprint("dvbsub_parse_object_segment(): Something went wrong. Giving up on block (2).\n");
return -1;
}
@@ -1414,7 +1343,6 @@ static void dvbsub_parse_region_segment(void *dvb_ctx, const uint8_t *buf,
fatal(EXIT_NOT_ENOUGH_MEMORY, "In dvbsub_parse_region_segment: Out of memory allocating object.");
}
memset(object, 0, sizeof(struct DVBSubObject));
object->version = -1;
object->id = object_id;
object->next = ctx->object_list;
@@ -1468,7 +1396,6 @@ static void dvbsub_parse_page_segment(void *dvb_ctx, const uint8_t *buf,
int page_state;
int timeout;
int version;
int has_region_definitions; // Declaration moved here for C89 compliance
if (buf_size < 2)
return;
@@ -1477,92 +1404,74 @@ static void dvbsub_parse_page_segment(void *dvb_ctx, const uint8_t *buf,
version = ((*buf) >> 4) & 15;
page_state = ((*buf++) >> 2) & 3;
// Version check removed to always allow page state check (Fix for Arte stream)
// if version same mean we are already updated
if (ctx->version == version)
{
return;
}
/* Convert time from second to ms */
ctx->time_out = timeout * 1000;
ctx->version = version;
//
// Issue 5: Spec-compliant Page Segment handling
// KEY FIX: Only rebuild display_list if new regions are defined
has_region_definitions = (buf + 6 <= buf_end); // Need at least 6 bytes for one region
if (page_state == 1 || page_state == 2)
{
// Mode change (1) or Acquisition point (2): Always clear display list
dbg_print(CCX_DMT_DVB, ", PAGE STATE %d", page_state);
delete_regions(ctx);
delete_objects(ctx);
delete_cluts(ctx);
tmp_display_list = ctx->display_list;
ctx->display_list = NULL;
// If regions are provided, parsing loop below will rebuild ctx->display_list
// If no regions provided, ctx->display_list remains NULL (empty)
}
else if (has_region_definitions)
{
// Normal case (0) WITH new region definitions: clear and rebuild
tmp_display_list = ctx->display_list;
ctx->display_list = NULL;
}
else
{
// Normal case (0) WITHOUT region definitions: keep existing display_list (Arte fix)
// Do not clear ctx->display_list
tmp_display_list = NULL; // Nothing to free from old list
}
// Rebuild display list if we have regions to parse
if (has_region_definitions)
tmp_display_list = ctx->display_list;
ctx->display_list = NULL;
while (buf + 5 < buf_end)
{
while (buf + 6 <= buf_end)
region_id = *buf++;
buf += 1;
dbg_print(CCX_DMT_DVB, ", REGION %d ADDED", region_id);
display = tmp_display_list;
tmp_ptr = &tmp_display_list;
while (display && display->region_id != region_id)
{
region_id = *buf++;
buf += 1;
display = tmp_display_list;
tmp_ptr = &tmp_display_list;
while (display && display->region_id != region_id)
{
tmp_ptr = &display->next;
display = display->next;
}
tmp_ptr = &display->next;
display = display->next;
}
if (!display)
{
display = (struct DVBSubRegionDisplay *)malloc(
sizeof(struct DVBSubRegionDisplay));
if (!display)
{
display = (struct DVBSubRegionDisplay *)malloc(
sizeof(struct DVBSubRegionDisplay));
if (!display)
{
fatal(EXIT_NOT_ENOUGH_MEMORY, "In dvbsub_parse_page_segment: Out of memory allocating display.");
}
memset(display, 0, sizeof(struct DVBSubRegionDisplay));
fatal(EXIT_NOT_ENOUGH_MEMORY, "In dvbsub_parse_page_segment: Out of memory allocating display.");
}
display->region_id = region_id;
display->x_pos = RB16(buf);
buf += 2;
display->y_pos = RB16(buf);
buf += 2;
*tmp_ptr = display->next;
display->next = ctx->display_list;
ctx->display_list = display;
memset(display, 0, sizeof(struct DVBSubRegionDisplay));
}
display->region_id = region_id;
display->x_pos = RB16(buf);
buf += 2;
display->y_pos = RB16(buf);
buf += 2;
*tmp_ptr = display->next;
display->next = ctx->display_list;
ctx->display_list = display;
}
// Free any leftover regions that weren't reused
while (tmp_display_list)
{
display = tmp_display_list;
tmp_display_list = display->next;
free(display);
}
assert(buf <= buf_end);
}
@@ -1609,17 +1518,6 @@ static void dvbsub_parse_display_definition_segment(void *dvb_ctx,
}
}
static uint32_t fnv1a_32(const uint8_t *data, size_t len)
{
uint32_t hash = 2166136261u;
for (size_t i = 0; i < len; i++)
{
hash ^= data[i];
hash *= 16777619u;
}
return hash;
}
/**
* Write Subtitle in cc_subtitle structure in CC_BITMAP format
* when OCR subsystem is present then it also write recognised text in
@@ -1639,61 +1537,6 @@ static int write_dvb_sub(struct lib_cc_decode *dec_ctx, struct cc_subtitle *sub)
ctx = (DVBSubContext *)dec_ctx->private_data;
// Safety check: Context may be NULL after PAT change
if (!ctx)
return -1;
// Validate we have something to display
if (!ctx->display_list)
{
if (ctx->region_list)
{
// Heuristic Fix for Arte stream: Valid regions exist but Page Segment was empty.
// We auto-populate the display list assuming (0,0) coordinates.
DVBSubRegion *r = ctx->region_list;
while (r)
{
// Only add regions with actual pixel data (non-empty pbuf)
int has_content = 0;
if (r->pbuf && r->buf_size > 0)
{
for (int i = 0; i < r->buf_size; i++)
{
if (r->pbuf[i] != 0)
{
has_content = 1;
break;
}
}
}
if (has_content)
{
DVBSubRegionDisplay *d = (DVBSubRegionDisplay *)malloc(sizeof(struct DVBSubRegionDisplay));
if (d)
{
memset(d, 0, sizeof(*d));
d->region_id = r->id;
d->x_pos = 0;
d->y_pos = 0;
d->next = ctx->display_list;
ctx->display_list = d;
// Force dirty so this region gets rendered
// r->dirty = 1;
}
}
else
{
}
r = r->next;
}
}
else
{
return 0;
}
}
display_def = ctx->display_definition;
sub->type = CC_BITMAP;
sub->lang_index = ctx->lang_index;
@@ -1704,17 +1547,11 @@ static int write_dvb_sub(struct lib_cc_decode *dec_ctx, struct cc_subtitle *sub)
offset_y = display_def->y;
}
// Initialize nb_data before counting dirty regions to avoid stale values
// from copy_subtitle() carrying over between calls
sub->nb_data = 0;
for (display = ctx->display_list; display; display = display->next)
{
region = get_region(ctx, display->region_id);
if (region && region->dirty)
{
sub->nb_data++;
}
}
if (sub->nb_data <= 0)
{
@@ -1729,6 +1566,14 @@ static int write_dvb_sub(struct lib_cc_decode *dec_ctx, struct cc_subtitle *sub)
rect->data0 = NULL;
rect->data1 = NULL;
sub->flags |= SUB_EOD_MARKER;
sub->got_output = 1;
sub->data = rect;
sub->datatype = CC_DATATYPE_DVB;
// TODO: if different regions have different cluts, only the last one will be saved.
// Don't know if it will affect anything.
// The first loop, to determine the size of the whole subtitle (made up of different display/regions)
for (display = ctx->display_list; display; display = display->next)
@@ -1805,9 +1650,6 @@ static int write_dvb_sub(struct lib_cc_decode *dec_ctx, struct cc_subtitle *sub)
}
memset(rect->data1, 0, 1024);
memcpy(rect->data1, clut_table, (1 << region->depth) * sizeof(uint32_t));
rect->nb_colors = (1 << region->depth); // CRITICAL FIX: OCR needs this to know palette size
// User Quick Test
assert(((1 << region->depth) * sizeof(uint32_t)) <= 1024);
}
@@ -1837,8 +1679,6 @@ static int write_dvb_sub(struct lib_cc_decode *dec_ctx, struct cc_subtitle *sub)
int x_off = display->x_pos - x_pos;
int y_off = display->y_pos - y_pos;
static int oob_warning_printed = 0;
int oob_count = 0;
for (int y = 0; y < region->height; y++)
{
for (int x = 0; x < region->width; x++)
@@ -1846,80 +1686,21 @@ static int write_dvb_sub(struct lib_cc_decode *dec_ctx, struct cc_subtitle *sub)
int offset = ((y + y_off) * width) + x_off + x;
if (offset >= (width * height) || offset < 0)
{
oob_count++;
if (!oob_warning_printed)
{
mprint("write_dvb_sub(): Out of bounds pixels detected (showing first occurrence only)\n");
mprint(" Formula: offset=((y + y_off) * width) + x_off + x\n");
mprint(" y=%d, y_off=%d, width=%d, x_off=%d, x=%d, offset=%d\n",
y, y_off, width, x_off, x, offset);
oob_warning_printed = 1;
}
mprint("write_dvb_sub(): Offset %d (out of bounds!) ignored.\n",
offset);
mprint(" Formula: offset=((y + y_off) * width) + x_off + x\n");
mprint(" y=%d, y_off=%d, width=%d, x_off=%d, x=%d\n",
y, y_off, width, x_off, x);
}
else
{
uint8_t c = (uint8_t)region->pbuf[y * region->width + x];
if (c != 0)
{
// DEBUG: Found a non-zero pixel!
// mprint("DEBUG-PBUF: Found valid pixel %d at %d,%d\n", c, x, y);
// Only print once per frame to avoid spam, or rely on the final dump
}
rect->data0[offset] = c;
}
}
}
region->dirty = 0;
}
if (rect->data0 && rect->data1)
{
uint32_t current_hash = 2166136261u;
// Hash the geometry
current_hash ^= (uint32_t)rect->x;
current_hash *= 16777619u;
current_hash ^= (uint32_t)rect->y;
current_hash *= 16777619u;
current_hash ^= (uint32_t)rect->w;
current_hash *= 16777619u;
current_hash ^= (uint32_t)rect->h;
current_hash *= 16777619u;
// Hash the pixels
current_hash ^= fnv1a_32(rect->data0, width * height);
current_hash *= 16777619u;
// Hash the palette (color table)
// We use the last region's depth/nb_colors as that's what determined the palette size
current_hash ^= fnv1a_32(rect->data1, (1 << region->depth) * sizeof(uint32_t));
current_hash *= 16777619u;
if (ctx->prev_bitmap_hash == current_hash)
{
dbg_print(CCX_DMT_DVB, "Duplicate DVB subtitle frame detected (Hash: %08X). Skipping.\n", current_hash);
// Free the rect we just allocated since we aren't using it
free(rect->data0);
free(rect->data1);
free(rect);
sub->nb_data = 0; // Ensure nb_data is 0 so the encoder doesn't try to access null data
sub->got_output = 0; // CRITICAL: Mark as no output to prevent duplicate encoding
return 0; // Return 0 to indicate no new subtitle produced
}
ctx->prev_bitmap_hash = current_hash;
}
sub->flags |= SUB_EOD_MARKER;
sub->got_output = 1;
sub->data = rect;
sub->datatype = CC_DATATYPE_DVB;
// DEBUG: Verify nonzero count manually
int nz_count = 0;
for (int k = 0; k < width * height; k++)
if (rect->data0[k])
nz_count++;
sub->nb_data = 1; // Set nb_data to 1 since we have merged the images into one image.
// Perform OCR
@@ -1933,28 +1714,18 @@ static int write_dvb_sub(struct lib_cc_decode *dec_ctx, struct cc_subtitle *sub)
}
if (ctx->ocr_ctx && region)
{
// DEBUG: Dump before OCR
// dump_rect_and_log("before_ocr", rect->data0, rect->w, rect->h, rect->linesize0, 1, 0, 0);
int ret = ocr_rect(ctx->ocr_ctx, rect, &ocr_str, region->bgcolor, dec_ctx->ocr_quantmode);
if (ret >= 0 && ocr_str)
{
if (ret >= 0)
rect->ocr_text = ocr_str;
}
else
{
rect->ocr_text = NULL;
}
// DEBUG: Dump after OCR (if modified)
// dump_rect_and_log("after_ocr", rect->data0, rect->w, rect->h, rect->linesize0, ctx->display_definition ? 3 : 1, 0, 0);
dbg_print(CCX_DMT_DVB, "\nOCR Result: %s\n", rect->ocr_text ? rect->ocr_text : "NULL");
}
else
{
rect->ocr_text = NULL;
}
#endif
return 0;
}
@@ -1967,120 +1738,79 @@ void dvbsub_handle_display_segment(struct encoder_ctx *enc_ctx,
LLONG current_pts = dec_ctx->timing->current_pts;
if (!enc_ctx)
return;
// Deduplication check: Skip if this subtitle is a duplicate
// We use composition_id + ancillary_id + PTS to uniquely identify a subtitle
// PTS is converted from microseconds to milliseconds for consistency
// Skip dedup if --no-dvb-dedup flag is set
if (!ccx_options.no_dvb_dedup)
{
uint64_t pts_ms = (uint64_t)(current_pts / 1000);
uint32_t pid = (uint32_t)dec_ctx->program_number; // Use program number as PID proxy
if (dvb_dedup_is_duplicate(&ctx->dedup_ring, pts_ms, pid,
(uint16_t)ctx->composition_id,
(uint16_t)ctx->ancillary_id))
{
dbg_print(CCX_DMT_DVB, "DVB: Skipping duplicate subtitle (PTS=%lld, comp_id=%d, anc_id=%d)\n",
current_pts, ctx->composition_id, ctx->ancillary_id);
return;
}
// Add to dedup ring buffer
dvb_dedup_add(&ctx->dedup_ring, pts_ms, pid,
(uint16_t)ctx->composition_id,
(uint16_t)ctx->ancillary_id);
}
if (enc_ctx->write_previous) // this condition is used for the first subtitle - write_previous will be 0 first so we don't encode a non-existing previous sub
{
enc_ctx->prev->last_str = NULL; // Reset last recognized sub text
// Validate current_field before calling get_fts (valid: 1=field1, 2=field2, 3=CEA-708)
int caption_field = dec_ctx->current_field;
if (caption_field < 1 || caption_field > 3)
{
dbg_print(CCX_DMT_DVB, "DVB: invalid current_field %d, using default 1\n", caption_field);
caption_field = 1;
}
enc_ctx->prev->last_string = NULL; // Reset last recognized sub text
// Get the current FTS, which will be the start_time of the new subtitle
LLONG next_start_time = get_fts(dec_ctx->timing, caption_field);
if (!sub->prev)
LLONG next_start_time = get_fts(dec_ctx->timing, dec_ctx->current_field);
// For DVB subtitles, a subtitle is displayed until the next one appears.
// Use next_start_time as the end_time to ensure subtitle N ends when N+1 starts.
// This prevents any overlap between consecutive subtitles.
if (next_start_time > sub->prev->start_time)
{
// Previous subtitle is missing or invalid, skipping write_previous
enc_ctx->write_previous = 0;
if (enc_ctx->prev)
{
free_encoder_context(enc_ctx->prev);
enc_ctx->prev = NULL;
enc_ctx->prev = copy_encoder_context(enc_ctx);
}
sub->prev->end_time = next_start_time;
}
else
{
// For DVB subtitles, a subtitle is displayed until the next one appears.
// Use next_start_time as the end_time to ensure subtitle N ends when N+1 starts.
// This prevents any overlap between consecutive subtitles.
// Issue 7: Use FTS-based duration calculation always.
// If pre_fts_max is available (captured before potential PTS jumps), use it
// to accurately end the previous sub at the end of the previous timeline.
LLONG fts_end_time = (pre_fts_max > 0) ? pre_fts_max : next_start_time;
LLONG duration = fts_end_time - sub->prev->start_time;
LLONG capped_timeout = (sub->prev->time_out > 0 && sub->prev->time_out < DVB_MAX_SUBTITLE_DURATION_MS)
? sub->prev->time_out
: DVB_MAX_SUBTITLE_DURATION_MS;
if (duration <= 0 || duration > capped_timeout)
// PTS jump or timeline reset - next_start is at or before our start.
// Calculate duration from raw PTS, but cap to reasonable maximum (5 seconds)
// to avoid creating subtitles that overlap excessively with subsequent ones.
LLONG duration_ms = 0;
if (sub->prev->start_pts > 0 && current_pts > sub->prev->start_pts)
{
fts_end_time = sub->prev->start_time + capped_timeout;
duration_ms = (current_pts - sub->prev->start_pts) / (MPEG_CLOCK_FREQ / 1000);
}
sub->prev->end_time = fts_end_time;
// Sanity check: if end_time still <= start_time (e.g. due to resets), force 1ms
if (sub->prev->end_time <= sub->prev->start_time)
// Cap duration to 4 seconds or timeout if smaller
LLONG max_duration = 4000; // 4 seconds
if (sub->prev->time_out > 0 && sub->prev->time_out < max_duration)
{
dbg_print(CCX_DMT_DVB, "DVB timing: end <= start, using start+1\n");
sub->prev->end_time = sub->prev->start_time + 1;
max_duration = sub->prev->time_out;
}
int timeok = 1;
if (dec_ctx->extraction_start.set &&
sub->prev->start_time < dec_ctx->extraction_start.time_in_ms)
timeok = 0;
if (dec_ctx->extraction_end.set &&
sub->prev->end_time > dec_ctx->extraction_end.time_in_ms)
if (duration_ms > max_duration)
{
timeok = 0;
dec_ctx->processed_enough = 1;
duration_ms = max_duration;
}
if (timeok)
{
encode_sub(enc_ctx->prev, sub->prev); // we encode it
sub->prev->end_time = sub->prev->start_time + duration_ms;
}
// Sanity check: if end_time still <= start_time, use minimal duration
if (sub->prev->end_time <= sub->prev->start_time)
{
dbg_print(CCX_DMT_DVB, "DVB timing: end <= start, using start+1\n");
sub->prev->end_time = sub->prev->start_time + 1;
}
// Apply timeout limit if specified
if (sub->prev->time_out > 0 && sub->prev->time_out < sub->prev->end_time - sub->prev->start_time)
{
sub->prev->end_time = sub->prev->start_time + sub->prev->time_out;
}
int timeok = 1;
if (dec_ctx->extraction_start.set &&
sub->prev->start_time < dec_ctx->extraction_start.time_in_ms)
timeok = 0;
if (dec_ctx->extraction_end.set &&
sub->prev->end_time > dec_ctx->extraction_end.time_in_ms)
{
timeok = 0;
dec_ctx->processed_enough = 1;
}
if (timeok)
{
encode_sub(enc_ctx->prev, sub->prev); // we encode it
// Update last recognized string (used in Matroska)
// Move ownership from prev to main context
if (enc_ctx->prev)
{
enc_ctx->last_str = enc_ctx->prev->last_str;
enc_ctx->prev->last_str = NULL;
}
enc_ctx->last_string = enc_ctx->prev->last_string; // Update last recognized string (used in Matroska)
enc_ctx->prev->last_string = NULL;
enc_ctx->srt_counter = enc_ctx->prev->srt_counter; // for dvb subs we need to update the current srt counter because we always encode the previous subtitle (and the counter is increased for the previous context)
enc_ctx->prev_start = enc_ctx->prev->prev_start;
sub->prev->got_output = 0;
if (enc_ctx->write_format == CCX_OF_WEBVTT)
{ // we already wrote header, but since we encoded last sub, we must prevent multiple headers in future
enc_ctx->wrote_webvtt_header = 1;
}
enc_ctx->srt_counter = enc_ctx->prev->srt_counter; // for dvb subs we need to update the current srt counter because we always encode the previous subtitle (and the counter is increased for the previous context)
enc_ctx->prev_start = enc_ctx->prev->prev_start;
sub->prev->got_output = 0;
if (enc_ctx->write_format == CCX_OF_WEBVTT)
{ // we already wrote header, but since we encoded last sub, we must prevent multiple headers in future
enc_ctx->wrote_webvtt_header = 1;
}
}
}
/* copy previous encoder context*/
free_encoder_context(enc_ctx->prev);
enc_ctx->prev = NULL;
enc_ctx->prev = copy_encoder_context(enc_ctx);
@@ -2095,31 +1825,19 @@ void dvbsub_handle_display_segment(struct encoder_ctx *enc_ctx,
{
fatal(EXIT_NOT_ENOUGH_MEMORY, "In dvbsub_handle_display_segment: Out of memory allocating private_data.");
}
// Use safe copy that doesn't alias linked list pointers
dvbsub_copy_context(dec_ctx->prev->private_data, dec_ctx->private_data);
// Issue 6: Removed workaround. Version management should be handled by logic, not forcing -1.
// Reference: ((DVBSubContext *)dec_ctx->prev->private_data)->version = -1;
memcpy(dec_ctx->prev->private_data, dec_ctx->private_data, sizeof(struct DVBSubContext));
/* copy previous subtitle */
free_subtitle(sub->prev);
sub->time_out = ctx->time_out;
sub->prev = NULL;
sub->prev = copy_subtitle(sub);
// Use get_fts() which properly handles PTS jumps and maintains monotonic timing
// Validate current_field (valid: 1=field1, 2=field2, 3=CEA-708)
int sub_caption_field = dec_ctx->current_field;
if (sub_caption_field < 1 || sub_caption_field > 3)
sub_caption_field = 1;
sub->prev->start_time = get_fts(dec_ctx->timing, sub_caption_field);
sub->prev->start_time = get_fts(dec_ctx->timing, dec_ctx->current_field);
// Store the raw PTS for accurate duration calculation (not affected by PTS jump handling)
sub->prev->start_pts = current_pts;
// Use current dec_ctx (not prev) because we need valid region/object data for rendering
// dec_ctx->prev has NULL pointers to avoid memory corruption from aliased linked lists
write_dvb_sub(dec_ctx, sub->prev); // we write the current dvb sub to update decoder context
enc_ctx->write_previous = 1; // we update our boolean value so next time the program reaches this block of code, it encodes the previous sub
write_dvb_sub(dec_ctx->prev, sub->prev); // we write the current dvb sub to update decoder context
enc_ctx->write_previous = 1; // we update our boolean value so next time the program reaches this block of code, it encodes the previous sub
#ifdef ENABLE_OCR
if (sub->prev)
{
@@ -2155,25 +1873,10 @@ int dvbsub_decode(struct encoder_ctx *enc_ctx, struct lib_cc_decode *dec_ctx, co
int ret = 0;
int got_segment = 0;
// Safety check: Context may be NULL after PAT change
if (!ctx)
return -1;
// Clear last recognized string to avoid leakage between calls
if (enc_ctx)
freep(&enc_ctx->last_str);
// Sync loop: Advance buffer until we find the 0x0F sync byte
while (buf_size > 0 && *buf != 0x0f)
if (buf_size <= 6 || *buf != 0x0f)
{
buf++;
buf_size--;
}
if (buf_size <= 6)
{
// Reduced verbosity: only print if we actually failed to find anything useful
// mprint("dvbsub_decode: incomplete, broken or empty packet (size = %d)\n", buf_size);
mprint("dvbsub_decode: incomplete, broken or empty packet (size = %d, first byte=%02X)\n",
buf_size, *buf);
return -1;
}
@@ -2206,11 +1909,10 @@ int dvbsub_decode(struct encoder_ctx *enc_ctx, struct lib_cc_decode *dec_ctx, co
if (page_id == ctx->composition_id || page_id == ctx->ancillary_id || ctx->composition_id == -1 || ctx->ancillary_id == -1)
{
// debug traces
// Unconditional trace for debugging
dbg_print(CCX_DMT_DVB, "DVBSUB - PTS: %" PRId64 ", ", dec_ctx->timing->current_pts);
dbg_print(CCX_DMT_DVB, "FTS: %d, ", dec_ctx->timing->fts_now);
dbg_print(CCX_DMT_DVB, "SEGMENT TYPE: %2X, ", segment_type);
switch (segment_type)
{
case DVBSUB_PAGE_SEGMENT:
@@ -2243,18 +1945,11 @@ int dvbsub_decode(struct encoder_ctx *enc_ctx, struct lib_cc_decode *dec_ctx, co
segment_length);
break;
case DVBSUB_DISPLAY_SEGMENT: // when we get a display segment, we save the current page
// dbg_print(CCX_DMT_DVB, "(DVBSUB_DISPLAY_SEGMENT), SEGMENT LENGTH: %d", segment_length);
dbg_print(CCX_DMT_DVB, "(DVBSUB_DISPLAY_SEGMENT), SEGMENT LENGTH: %d", segment_length);
dvbsub_handle_display_segment(enc_ctx, dec_ctx, sub, pre_fts_max);
got_segment |= 16;
break;
default:
if (segment_type == 0) // Padding
{
p += segment_length;
continue;
}
dbg_print(CCX_DMT_DVB, "Subtitling segment type 0x%x, page id %d, length %d\n",
segment_type, page_id, segment_length);
break;
@@ -2304,45 +1999,54 @@ int parse_dvb_description(struct dvb_config *cfg, unsigned char *data,
return -1;
}
if (cfg->n_language > MAX_LANGUAGE_PER_DESC)
{
mprint("Warning: more than %d languages in DVB descriptor, only parsing first %d\n", MAX_LANGUAGE_PER_DESC, MAX_LANGUAGE_PER_DESC);
cfg->n_language = MAX_LANGUAGE_PER_DESC;
}
if (cfg->n_language > 1)
{
mprint("DVB subtitles with multiple languages\n");
mprint("DVB subtitles with multiple languages");
}
for (int i = 0; i < cfg->n_language; i++)
if (cfg->n_language > MAX_LANGUAGE_PER_DESC)
{
unsigned char *ptr = data + (i * 8);
char lang_name[4];
lang_name[0] = ptr[0];
lang_name[1] = ptr[1];
lang_name[2] = ptr[2];
lang_name[3] = '\0';
mprint("not supported more then %d language", MAX_LANGUAGE_PER_DESC);
}
for (int j = 0; language[j] != NULL; j++)
unsigned char *data_ptr = data;
for (int i = 0; i < cfg->n_language; i++, data_ptr += i * 8)
{
/* setting language to undefined if not found in language lkup table */
char lang_name[4];
dbg_print(CCX_DMT_DVB, "DVBSUB - LANGUAGE \"");
for (int char_index = 0; char_index < 3; char_index++)
{
lang_name[char_index] = cctolower(data_ptr[char_index]);
dbg_print(CCX_DMT_DVB, "%c", lang_name[char_index]);
}
dbg_print(CCX_DMT_DVB, "\" FOUND\n");
int j = 0;
for (j = 0, cfg->lang_index[i] = 0; language[j] != NULL; j++)
{
if (!strncmp(lang_name, language[j], 3))
{
cfg->lang_index[i] = j;
break;
}
}
cfg->sub_type[i] = ptr[3];
cfg->composition_id[i] = RB16(ptr + 4);
cfg->ancillary_id[i] = RB16(ptr + 6);
cfg->sub_type[i] = data_ptr[3];
cfg->composition_id[i] = RB16(data_ptr + 4);
cfg->ancillary_id[i] = RB16(data_ptr + 6);
}
/*
Abhinav95: The way this function is called right now, only cfg->lang_index[0]
gets populated. E.g. for 3 stream languages, it will be called 3 times, and
set the language index in only the first element each time. This works with the
current state of the DVB code.
*/
if (ccx_options.dvblang)
{
if (strcmp(ccx_options.dvblang, language[cfg->lang_index[0]]) && strncmp(ccx_options.dvblang, (const char *)data, 3))
if (strcmp(ccx_options.dvblang, language[cfg->lang_index[0]]) && strncmp(ccx_options.dvblang, data, 3))
{
mprint("Ignoring stream language index %d not equal to dvblang '%s'\n",
cfg->lang_index[0], ccx_options.dvblang);
mprint("Ignoring stream language '%s' not equal to dvblang '%s'\n",
data, ccx_options.dvblang);
return -1;
}
}

17
src/lib_ccx/dvb_subtitle_decoder.h
View File

@@ -70,23 +70,6 @@ extern "C"
int parse_dvb_description(struct dvb_config *cfg, unsigned char *data,
unsigned int len);
/*
* @func dvbsub_get_context_size
* Get the size of DVBSubContext for external allocation
*
* @return size in bytes of DVBSubContext structure
*/
size_t dvbsub_get_context_size(void);
/*
* @func dvbsub_copy_context
* Copy DVBSubContext from src to dst
*
* @param dst destination pointer (must be allocated with at least dvbsub_get_context_size() bytes)
* @param src source DVBSubContext pointer
*/
void dvbsub_copy_context(void *dst, void *src);
/*
* @func dvbsub_set_write the output structure in dvb
* set ccx_s_write structure in dvb_ctx

241
src/lib_ccx/general_loop.c
View File

@@ -202,8 +202,6 @@ int ps_get_more_data(struct lib_ccx_ctx *ctx, struct demuxer_data **ppdata)
}
// FIXME: Temporary bypass
data->bufferdatatype = CCX_DVD_SUBTITLE;
// Use substream ID as stream_pid for PS files to differentiate DVB subtitle streams
data->stream_pid = nextheader[7];
data->len = result;
enough = 1;
@@ -873,14 +871,10 @@ int process_data(struct encoder_ctx *enc_ctx, struct lib_cc_decode *dec_ctx, str
}
else if (data_node->bufferdatatype == CCX_DVB_SUBTITLE)
{
// Safety check: Skip if decoder was freed due to PAT change
if (dec_ctx->private_data)
{
ret = dvbsub_decode(enc_ctx, dec_ctx, data_node->buffer + 2, data_node->len - 2, dec_sub);
if (ret < 0)
mprint("Return from dvbsub_decode: %d\n", ret);
set_fts(dec_ctx->timing);
}
ret = dvbsub_decode(enc_ctx, dec_ctx, data_node->buffer + 2, data_node->len - 2, dec_sub);
if (ret < 0)
mprint("Return from dvbsub_decode: %d\n", ret);
set_fts(dec_ctx->timing);
got = data_node->len;
}
else if (data_node->bufferdatatype == CCX_PES)
@@ -1139,19 +1133,10 @@ int process_non_multiprogram_general_loop(struct lib_ccx_ctx *ctx,
// struct encoder_ctx *enc_ctx = NULL;
// Find most promising stream: teletex, DVB, ISDB
int pid = get_best_stream(ctx->demux_ctx);
// NOTE: For DVB split mode, we do NOT mutate pid here.
// Mutating pid to -1 causes demuxer PES buffer errors because it changes
// the stream selection semantics unexpectedly. Instead, we keep primary
// stream processing unchanged and handle DVB streams in a separate
// read-only secondary pass after the primary stream is processed.
if (pid < 0)
{
// Let get_best_data pick the primary stream (usually Teletext)
*data_node = get_best_data(*datalist);
}
else
{
ignore_other_stream(ctx->demux_ctx, pid);
@@ -1296,140 +1281,23 @@ int process_non_multiprogram_general_loop(struct lib_ccx_ctx *ctx,
}
if ((*data_node)->bufferdatatype == CCX_TELETEXT && (*dec_ctx)->private_data) // if we have teletext subs, we set the min_pts here
set_tlt_delta(*dec_ctx, (*dec_ctx)->timing->current_pts);
// Primary stream processing
// In split DVB mode, DVB streams are handled in the secondary pass below.
// We skip primary processing for DVB here to avoid double-processing (or processing as 'default' output).
// Teletext/other streams still go through here.
if (*data_node && !(ccx_options.split_dvb_subs && (*dec_ctx)->codec == CCX_CODEC_DVB))
{
ret = process_data(*enc_ctx, *dec_ctx, *data_node);
}
ret = process_data(*enc_ctx, *dec_ctx, *data_node);
if (*enc_ctx != NULL)
{
if ((*enc_ctx)->srt_counter || (*enc_ctx)->cea_708_counter || (*dec_ctx)->saw_caption_block || ret == 1)
{
*caps = 1;
/* Also update ret to indicate captions were found.
This is needed for CEA-708 which writes directly via Rust
and doesn't set got_output like CEA-608/DVB do. */
ret = 1;
}
}
// SECONDARY PASS: Process DVB streams in split mode
// DVB streams are parallel consumers, processed AFTER the primary stream
// This ensures Teletext controls timing/loop lifetime while DVB is extracted separately
if (ccx_options.split_dvb_subs)
{
// Guard: Only process DVB if timing has been initialized by Teletext
// if ((*dec_ctx)->timing == NULL || (*dec_ctx)->timing->pts_set == 0)
// {
// goto skip_dvb_secondary_pass;
struct demuxer_data *dvb_ptr = *datalist;
while (dvb_ptr)
{
// Process DVB nodes (in split mode, even if they were the "best" stream,
// we route them here to ensure they get a proper named pipeline)
if (dvb_ptr->codec == CCX_CODEC_DVB &&
dvb_ptr->len > 0)
{
int stream_pid = dvb_ptr->stream_pid;
char *lang = "unk";
// Find language for this PID - check potential_streams first (PMT discovery)
for (int k = 0; k < ctx->demux_ctx->potential_stream_count; k++)
{
if (ctx->demux_ctx->potential_streams[k].pid == stream_pid &&
ctx->demux_ctx->potential_streams[k].lang[0])
{
lang = ctx->demux_ctx->potential_streams[k].lang;
break;
}
}
// Fallback to cinfo if potential_streams didn't have it
if (strcmp(lang, "unk") == 0)
{
struct cap_info *cinfo = get_cinfo(ctx->demux_ctx, stream_pid);
if (cinfo && cinfo->lang[0])
lang = cinfo->lang;
}
// Get or create pipeline for this DVB stream
struct ccx_subtitle_pipeline *pipe = get_or_create_pipeline(ctx, stream_pid, CCX_STREAM_TYPE_DVB_SUB, lang);
// Reinitialize decoder if it was NULLed out by PAT change
if (pipe && pipe->dec_ctx && !pipe->decoder)
{
struct dvb_config dvb_cfg = {0};
dvb_cfg.n_language = 1;
// Lookup composition/ancillary IDs from potential_streams
if (ctx->demux_ctx)
{
for (int j = 0; j < ctx->demux_ctx->potential_stream_count; j++)
{
if (ctx->demux_ctx->potential_streams[j].pid == stream_pid)
{
dvb_cfg.composition_id[0] = ctx->demux_ctx->potential_streams[j].composition_id;
dvb_cfg.ancillary_id[0] = ctx->demux_ctx->potential_streams[j].ancillary_id;
break;
}
}
}
pipe->decoder = dvbsub_init_decoder(&dvb_cfg);
if (pipe->decoder)
pipe->dec_ctx->private_data = pipe->decoder;
}
if (pipe && pipe->encoder && pipe->decoder && pipe->dec_ctx)
{
// Use pipeline's own independent timing context
// This avoids synchronization issues if primary stream (Teletext) has different timing characteristics
pipe->dec_ctx->timing = pipe->timing;
pipe->encoder->timing = pipe->timing;
// Set the PTS for this DVB packet before decoding
// Without this, the DVB decoder will use stale timing
set_pipeline_pts(pipe, dvb_ptr->pts);
// Create subtitle structure if needed
if (!pipe->dec_ctx->dec_sub.prev)
{
// This should be handled by get_or_create_pipeline but safety check
struct cc_subtitle *sub = malloc(sizeof(struct cc_subtitle));
memset(sub, 0, sizeof(struct cc_subtitle));
pipe->dec_ctx->dec_sub.prev = sub;
}
// Decode DVB using the per-pipeline decoder context
// This ensures each stream has its own prev pointers
// Skip first 2 bytes (PES header) as done in process_data for DVB
// Safety check: Skip if decoder was freed due to PAT change
if (pipe->decoder && pipe->dec_ctx->private_data)
{
int offset = 2;
// Auto-detect offset based on sync byte 0x0F
if (dvb_ptr->len > 2 && dvb_ptr->buffer[2] == 0x0f)
offset = 2;
else if (dvb_ptr->len > 0 && dvb_ptr->buffer[0] == 0x0f)
offset = 0;
dvbsub_decode(pipe->encoder, pipe->dec_ctx, dvb_ptr->buffer + offset, dvb_ptr->len - offset, &pipe->dec_ctx->dec_sub);
}
// CRITICAL: Reset length so buffer can be reused/filled again
dvb_ptr->len = 0;
}
}
dvb_ptr = dvb_ptr->next_stream;
}
}
skip_dvb_secondary_pass:
// Process the last subtitle for DVB
if (!(!terminate_asap && !end_of_file && is_decoder_processed_enough(ctx) == CCX_FALSE))
{
if ((*data_node)->bufferdatatype == CCX_DVB_SUBTITLE &&
(*dec_ctx)->dec_sub.prev && (*dec_ctx)->dec_sub.prev->end_time == 0)
if ((*data_node)->bufferdatatype == CCX_DVB_SUBTITLE && (*dec_ctx)->dec_sub.prev->end_time == 0)
{
// Use get_fts() which properly handles PTS jumps and maintains monotonic timing
(*dec_ctx)->dec_sub.prev->end_time = get_fts((*dec_ctx)->timing, (*dec_ctx)->current_field);
@@ -1615,12 +1483,7 @@ int general_loop(struct lib_ccx_ctx *ctx)
}
}
// In split DVB mode, skip primary processing for DVB streams
// They will be handled in the secondary DVB pass below (Bug 3 fix)
if (!(ccx_options.split_dvb_subs && dec_ctx && dec_ctx->codec == CCX_CODEC_DVB))
{
ret = process_data(enc_ctx, dec_ctx, data_node);
}
ret = process_data(enc_ctx, dec_ctx, data_node);
if (enc_ctx != NULL)
{
if (
@@ -1634,9 +1497,7 @@ int general_loop(struct lib_ccx_ctx *ctx)
if (data_node->bufferdatatype == CCX_DVB_SUBTITLE && dec_ctx && dec_ctx->dec_sub.prev && dec_ctx->dec_sub.prev->end_time == 0)
{
dec_ctx->dec_sub.prev->end_time = (dec_ctx->timing->current_pts - dec_ctx->timing->min_pts) / (MPEG_CLOCK_FREQ / 1000);
// In split DVB mode, skip flushing to main encoder (Bug 3 fix)
// Pipeline encoders get flushed via lib_ccx.c:285-313
if (enc_ctx != NULL && !(ccx_options.split_dvb_subs && dec_ctx && dec_ctx->codec == CCX_CODEC_DVB))
if (enc_ctx != NULL)
encode_sub(enc_ctx->prev, dec_ctx->dec_sub.prev);
dec_ctx->dec_sub.prev->got_output = 0;
}
@@ -1645,86 +1506,6 @@ int general_loop(struct lib_ccx_ctx *ctx)
if (!data_node)
continue;
}
// MULTIPROGRAM DVB SECONDARY PASS: Route DVB streams to per-language pipelines
// This mirrors the single-program DVB secondary pass at lines 1321-1417 (Bug 3 fix)
if (ccx_options.split_dvb_subs)
{
struct demuxer_data *dvb_ptr = datalist;
while (dvb_ptr)
{
if (dvb_ptr->codec == CCX_CODEC_DVB && dvb_ptr->len > 0)
{
int stream_pid = dvb_ptr->stream_pid;
char *lang = "unk";
// Find language for this PID from potential_streams (PMT discovery)
for (int k = 0; k < ctx->demux_ctx->potential_stream_count; k++)
{
if (ctx->demux_ctx->potential_streams[k].pid == stream_pid &&
ctx->demux_ctx->potential_streams[k].lang[0])
{
lang = ctx->demux_ctx->potential_streams[k].lang;
break;
}
}
// Fallback to cinfo
if (strcmp(lang, "unk") == 0)
{
struct cap_info *cinfo = get_cinfo(ctx->demux_ctx, stream_pid);
if (cinfo && cinfo->lang[0])
lang = cinfo->lang;
}
// Get or create pipeline for this DVB stream
struct ccx_subtitle_pipeline *pipe = get_or_create_pipeline(ctx, stream_pid, CCX_STREAM_TYPE_DVB_SUB, lang);
// Reinitialize decoder if NULL (e.g., after PAT change)
if (pipe && pipe->dec_ctx && !pipe->decoder)
{
struct dvb_config dvb_cfg = {0};
dvb_cfg.n_language = 1;
for (int j = 0; j < ctx->demux_ctx->potential_stream_count; j++)
{
if (ctx->demux_ctx->potential_streams[j].pid == stream_pid)
{
dvb_cfg.composition_id[0] = ctx->demux_ctx->potential_streams[j].composition_id;
dvb_cfg.ancillary_id[0] = ctx->demux_ctx->potential_streams[j].ancillary_id;
break;
}
}
pipe->decoder = dvbsub_init_decoder(&dvb_cfg);
if (pipe->decoder)
pipe->dec_ctx->private_data = pipe->decoder;
}
if (pipe && pipe->encoder && pipe->decoder && pipe->dec_ctx)
{
// Use pipeline's independent timing context
pipe->dec_ctx->timing = pipe->timing;
pipe->encoder->timing = pipe->timing;
set_pipeline_pts(pipe, dvb_ptr->pts);
// Create subtitle structure if needed
if (!pipe->dec_ctx->dec_sub.prev)
{
struct cc_subtitle *sub = malloc(sizeof(struct cc_subtitle));
memset(sub, 0, sizeof(struct cc_subtitle));
pipe->dec_ctx->dec_sub.prev = sub;
}
// Decode DVB (skip first 2 bytes - PES header)
if (pipe->decoder && pipe->dec_ctx->private_data)
{
dvbsub_decode(pipe->encoder, pipe->dec_ctx, dvb_ptr->buffer + 2, dvb_ptr->len - 2, &pipe->dec_ctx->dec_sub);
}
}
}
dvb_ptr = dvb_ptr->next_stream;
}
}
if (ctx->live_stream)
{
LLONG t = get_fts(dec_ctx->timing, dec_ctx->current_field);

376
src/lib_ccx/lib_ccx.c
View File

@@ -5,10 +5,6 @@
#include "dvb_subtitle_decoder.h"
#include "ccx_decoders_708.h"
#include "ccx_decoders_isdb.h"
#include "ccx_decoders_common.h"
#ifdef ENABLE_OCR
#include "ocr.h"
#endif
struct ccx_common_logging_t ccx_common_logging;
static struct ccx_decoders_common_settings_t *init_decoder_setting(
@@ -53,33 +49,6 @@ static int init_ctx_outbase(struct ccx_s_options *opt, struct lib_ccx_ctx *ctx)
if (opt->output_filename)
{
ctx->basefilename = get_basename(opt->output_filename);
if (ctx->basefilename)
{
size_t len = strlen(ctx->basefilename);
if (len > 0 && (ctx->basefilename[len - 1] == '/' || ctx->basefilename[len - 1] == '\\'))
{
char *input_base = NULL;
if (opt->input_source == CCX_DS_FILE && ctx->inputfile && ctx->inputfile[0])
input_base = get_basename(ctx->inputfile[0]);
else if (opt->input_source == CCX_DS_STDIN)
input_base = get_basename("stdin");
else if (opt->input_source == CCX_DS_NETWORK || opt->input_source == CCX_DS_TCP)
input_base = get_basename("network");
if (input_base)
{
size_t new_len = len + strlen(input_base) + 1;
char *new_base = malloc(new_len);
if (new_base)
{
snprintf(new_base, new_len, "%s%s", ctx->basefilename, input_base);
free(ctx->basefilename);
ctx->basefilename = new_base;
}
free(input_base);
}
}
}
}
else
{
@@ -233,18 +202,6 @@ struct lib_ccx_ctx *init_libraries(struct ccx_s_options *opt)
ctx->segment_counter = 0;
ctx->system_start_time = -1;
// Initialize pipeline infrastructure
ctx->pipeline_count = 0;
ctx->dec_dvb_default = NULL;
#ifdef _WIN32
InitializeCriticalSection(&ctx->pipeline_mutex);
#else
pthread_mutex_init(&ctx->pipeline_mutex, NULL);
#endif
ctx->pipeline_mutex_initialized = 1;
ctx->shared_ocr_ctx = NULL;
memset(ctx->pipelines, 0, sizeof(ctx->pipelines));
end:
if (ret != EXIT_OK)
{
@@ -306,89 +263,6 @@ void dinit_libraries(struct lib_ccx_ctx **ctx)
}
}
// Cleanup subtitle pipelines (split DVB mode)
for (i = 0; i < lctx->pipeline_count; i++)
{
struct ccx_subtitle_pipeline *p = lctx->pipelines[i];
if (!p)
continue;
// Correct closing order to ensure last subtitle is written
if (p->dec_ctx && p->encoder)
{
// Check if there's a pending subtitle in the prev buffer
if (p->dec_ctx->dec_sub.prev && p->dec_ctx->dec_sub.prev->data && p->encoder->prev)
{
// Calculate end time for the last subtitle
LLONG current_fts = 0;
if (p->dec_ctx->timing)
{
current_fts = get_fts(p->dec_ctx->timing, p->dec_ctx->current_field);
}
// Force end time if missing
if (p->dec_ctx->dec_sub.prev->end_time == 0)
{
if (current_fts > p->dec_ctx->dec_sub.prev->start_time)
p->dec_ctx->dec_sub.prev->end_time = current_fts;
else
p->dec_ctx->dec_sub.prev->end_time = p->dec_ctx->dec_sub.prev->start_time + 2000; // 2s fallback
}
encode_sub(p->encoder->prev, p->dec_ctx->dec_sub.prev);
}
}
if (p->decoder)
dvbsub_close_decoder(&p->decoder);
#ifdef ENABLE_OCR
if (p->ocr_ctx)
delete_ocr(&p->ocr_ctx);
#endif
if (p->encoder)
dinit_encoder(&p->encoder, 0);
if (p->timing)
dinit_timing_ctx(&p->timing);
if (p->dec_ctx)
{
// private_data points to p->decoder which is freed above
if (p->dec_ctx->prev)
{
// prev->private_data is allocated by copy_decoder_context
freep(&p->dec_ctx->prev->private_data);
free(p->dec_ctx->prev);
}
// Free dec_sub.prev which was allocated in init_cc_decode or general_loop
// Note: free_subtitle already frees the struct via freep(&sub), so we don't call free() again
if (p->dec_ctx->dec_sub.prev)
{
free_subtitle(p->dec_ctx->dec_sub.prev);
p->dec_ctx->dec_sub.prev = NULL;
}
p->dec_ctx->private_data = NULL;
free(p->dec_ctx);
}
free_subtitle(p->sub.prev);
free(p);
lctx->pipelines[i] = NULL;
}
lctx->pipeline_count = 0;
// Cleanup mutex
if (lctx->pipeline_mutex_initialized)
{
#ifdef _WIN32
DeleteCriticalSection(&lctx->pipeline_mutex);
#else
pthread_mutex_destroy(&lctx->pipeline_mutex);
#endif
lctx->pipeline_mutex_initialized = 0;
}
// free EPG memory
EPG_free(lctx);
freep(&lctx->freport.data_from_608);
@@ -401,11 +275,6 @@ void dinit_libraries(struct lib_ccx_ctx **ctx)
for (i = 0; i < lctx->num_input_files; i++)
freep(&lctx->inputfile[i]);
freep(&lctx->inputfile);
freep(&lctx->inputfile);
#ifdef ENABLE_OCR
if (lctx->shared_ocr_ctx)
delete_ocr(&lctx->shared_ocr_ctx);
#endif
freep(ctx);
}
@@ -618,248 +487,3 @@ struct encoder_ctx *update_encoder_list(struct lib_ccx_ctx *ctx)
{
return update_encoder_list_cinfo(ctx, NULL);
}
/**
* Get or create a subtitle pipeline for a specific PID/language.
* Used when --split-dvb-subs is enabled to route each DVB stream to its own output file.
*/
struct ccx_subtitle_pipeline *get_or_create_pipeline(struct lib_ccx_ctx *ctx, int pid, int stream_type, const char *lang)
{
int i;
// Lock mutex for thread safety
#ifdef _WIN32
EnterCriticalSection(&ctx->pipeline_mutex);
#else
pthread_mutex_lock(&ctx->pipeline_mutex);
#endif
// Search for existing pipeline
for (i = 0; i < ctx->pipeline_count; i++)
{
struct ccx_subtitle_pipeline *p = ctx->pipelines[i];
if (p && p->pid == pid && p->stream_type == stream_type &&
strcmp(p->lang, lang) == 0)
{
#ifdef _WIN32
LeaveCriticalSection(&ctx->pipeline_mutex);
#else
pthread_mutex_unlock(&ctx->pipeline_mutex);
#endif
return p;
}
}
// Check capacity
if (ctx->pipeline_count >= MAX_SUBTITLE_PIPELINES)
{
mprint("Warning: Maximum subtitle pipelines (%d) reached, cannot create new pipeline for PID 0x%X\n",
MAX_SUBTITLE_PIPELINES, pid);
#ifdef _WIN32
LeaveCriticalSection(&ctx->pipeline_mutex);
#else
pthread_mutex_unlock(&ctx->pipeline_mutex);
#endif
return NULL;
}
// Allocate new pipeline
struct ccx_subtitle_pipeline *pipe = calloc(1, sizeof(struct ccx_subtitle_pipeline));
if (!pipe)
{
mprint("Error: Failed to allocate memory for subtitle pipeline\n");
#ifdef _WIN32
LeaveCriticalSection(&ctx->pipeline_mutex);
#else
pthread_mutex_unlock(&ctx->pipeline_mutex);
#endif
return NULL;
}
pipe->pid = pid;
pipe->stream_type = stream_type;
snprintf(pipe->lang, sizeof(pipe->lang), "%.3s", lang ? lang : "und");
// Generate output filename: {basefilename}_{lang}_{PID}.ext
// Always include PID to handle multiple streams with same language
const char *ext = ctx->extension ? ctx->extension : ".srt";
if (strcmp(pipe->lang, "und") == 0 || strcmp(pipe->lang, "unk") == 0 || pipe->lang[0] == '\0')
{
snprintf(pipe->filename, sizeof(pipe->filename), "%s_0x%04X%s",
ctx->basefilename, pid, ext);
}
else
{
snprintf(pipe->filename, sizeof(pipe->filename), "%s_%s_0x%04X%s",
ctx->basefilename, pipe->lang, pid, ext);
}
// Initialize encoder for this pipeline
struct encoder_cfg cfg = ccx_options.enc_cfg;
cfg.output_filename = pipe->filename;
pipe->encoder = init_encoder(&cfg);
// DVB subtitles require a 'prev' encoder context for buffering (write_previous logic).
// Without this, the first call to dvbsub_handle_display_segment may fail or result in no output.
if (pipe->encoder)
{
pipe->encoder->prev = copy_encoder_context(pipe->encoder);
if (!pipe->encoder->prev)
{
mprint("Error: Failed to allocate prev context for PID 0x%X\n", pid);
dinit_encoder(&pipe->encoder, 0);
free(pipe);
return NULL;
}
// FIX Bug 3: Set write_previous=1 so the FIRST subtitle gets written
// With write_previous=0, first subtitle is only buffered and never encoded
// unless a second subtitle arrives. Setting to 1 enables immediate encoding.
pipe->encoder->write_previous = 1;
// Issue 4: Ensure prev context exists and is initialized
// This forces the "previous" subtitle (which is effectively the first one we see)
// to be eligible for writing when the NEXT segment arrives.
// pipe->sub.prev will be allocated when we set up dec_ctx below.
}
if (!pipe->encoder)
{
mprint("Error: Failed to create encoder for pipeline PID 0x%X\n", pid);
free(pipe);
#ifdef _WIN32
LeaveCriticalSection(&ctx->pipeline_mutex);
#else
pthread_mutex_unlock(&ctx->pipeline_mutex);
#endif
return NULL;
}
// Timing context: Create independent timing context for pipeline
// This ensures DVB streams track their own PTS/FTS without race conditions
// with the primary Teletext stream.
pipe->timing = init_timing_ctx(&ccx_common_timing_settings);
if (!pipe->timing)
{
mprint("Error: Failed to initialize timing for pipeline PID 0x%X\n", pid);
dinit_encoder(&pipe->encoder, 0);
free(pipe);
#ifdef _WIN32
LeaveCriticalSection(&ctx->pipeline_mutex);
#else
pthread_mutex_unlock(&ctx->pipeline_mutex);
#endif
return NULL;
}
// Initialize DVB decoder
struct dvb_config dvb_cfg = {0};
dvb_cfg.n_language = 1;
// Lookup metadata to use correct Composition and Ancillary Page IDs
// This ensures we respect the configuration advertised in the PMT
if (ctx->demux_ctx)
{
for (i = 0; i < ctx->demux_ctx->potential_stream_count; i++)
{
if (ctx->demux_ctx->potential_streams[i].pid == pid)
{
dvb_cfg.composition_id[0] = ctx->demux_ctx->potential_streams[i].composition_id;
dvb_cfg.ancillary_id[0] = ctx->demux_ctx->potential_streams[i].ancillary_id;
break;
}
}
}
pipe->decoder = dvbsub_init_decoder(&dvb_cfg);
if (!pipe->decoder)
{
mprint("Error: Failed to create DVB decoder for pipeline PID 0x%X\n", pid);
dinit_encoder(&pipe->encoder, 0);
dinit_timing_ctx(&pipe->timing);
free(pipe);
#ifdef _WIN32
LeaveCriticalSection(&ctx->pipeline_mutex);
#else
pthread_mutex_unlock(&ctx->pipeline_mutex);
#endif
return NULL;
}
// Initialize per-pipeline decoder context for DVB state management
pipe->dec_ctx = calloc(1, sizeof(struct lib_cc_decode));
if (!pipe->dec_ctx)
{
mprint("Error: Failed to create decoder context for pipeline PID 0x%X\n", pid);
dvbsub_close_decoder(&pipe->decoder);
dinit_encoder(&pipe->encoder, 0);
free(pipe);
#ifdef _WIN32
LeaveCriticalSection(&ctx->pipeline_mutex);
#else
pthread_mutex_unlock(&ctx->pipeline_mutex);
#endif
return NULL;
}
pipe->dec_ctx->private_data = pipe->decoder;
pipe->dec_ctx->codec = CCX_CODEC_DVB;
pipe->dec_ctx->prev = calloc(1, sizeof(struct lib_cc_decode));
if (pipe->dec_ctx->prev)
{
pipe->dec_ctx->prev->private_data = malloc(dvbsub_get_context_size());
if (pipe->dec_ctx->prev->private_data)
{
dvbsub_copy_context(pipe->dec_ctx->prev->private_data, pipe->decoder);
}
pipe->dec_ctx->prev->codec = CCX_CODEC_DVB;
}
// Initialize persistent cc_subtitle for DVB prev tracking
memset(&pipe->sub, 0, sizeof(struct cc_subtitle));
pipe->sub.prev = calloc(1, sizeof(struct cc_subtitle));
if (pipe->sub.prev)
{
pipe->sub.prev->start_time = -1;
pipe->sub.prev->end_time = 0;
}
// Register pipeline
ctx->pipelines[ctx->pipeline_count++] = pipe;
mprint("Created subtitle pipeline for PID 0x%X lang=%s -> %s\n", pid, pipe->lang, pipe->filename);
#ifdef _WIN32
LeaveCriticalSection(&ctx->pipeline_mutex);
#else
pthread_mutex_unlock(&ctx->pipeline_mutex);
#endif
return pipe;
}
/**
* Set PTS for a subtitle pipeline's timing context.
* This ensures the DVB decoder uses the correct timestamp for each packet.
*/
void set_pipeline_pts(struct ccx_subtitle_pipeline *pipe, LLONG pts)
{
if (!pipe || !pipe->timing || pts == CCX_NOPTS)
return;
set_current_pts(pipe->timing, pts);
// Initialize min_pts if not set
if (pipe->timing->min_pts == 0x01FFFFFFFFLL)
{
pipe->timing->min_pts = pts;
pipe->timing->pts_set = 2; // MinPtsSet
pipe->timing->sync_pts = pts;
}
// For DVB subtitle pipelines, directly calculate fts_now
// The standard set_fts() relies on video frame type detection which doesn't
// work for DVB-only streams. Simple calculation: (current_pts - min_pts) in ms
// MPEG_CLOCK_FREQ = 90000, so divide by 90 to get milliseconds
pipe->timing->fts_now = (pts - pipe->timing->min_pts) / 90;
// Also update fts_max if this is the highest timestamp seen
if (pipe->timing->fts_now > pipe->timing->fts_max)
pipe->timing->fts_max = pipe->timing->fts_now;
}

43
src/lib_ccx/lib_ccx.h
View File

@@ -25,12 +25,6 @@
#include <curl/curl.h>
#endif
#ifdef _WIN32
#include <windows.h>
#else
#include <pthread.h>
#endif
// #include "ccx_decoders_708.h"
/* Report information */
@@ -87,28 +81,6 @@ struct ccx_s_mp4Cfg
unsigned int mp4vidtrack : 1;
};
#define MAX_SUBTITLE_PIPELINES 64
/**
* ccx_subtitle_pipeline - Encapsulates all components for a single subtitle output stream
*/
struct ccx_subtitle_pipeline
{
int pid;
int stream_type;
char lang[4];
char filename[1024]; // Using fixed size instead of PATH_MAX to avoid header issues
struct ccx_s_write *writer;
struct encoder_ctx *encoder;
struct ccx_common_timing_ctx *timing;
void *decoder; // Pointer to decoder context (e.g., ccx_decoders_dvb_context)
struct lib_cc_decode *dec_ctx; // Full decoder context for DVB state management
struct cc_subtitle sub; // Persistent cc_subtitle for DVB prev tracking
#ifdef ENABLE_OCR
void *ocr_ctx; // Per-pipeline OCR context for thread safety
#endif
};
struct lib_ccx_ctx
{
// Stuff common to both loops
@@ -183,23 +155,8 @@ struct lib_ccx_ctx
int segment_on_key_frames_only;
int segment_counter;
LLONG system_start_time;
// Registration for multi-stream subtitle extraction
struct ccx_subtitle_pipeline *pipelines[MAX_SUBTITLE_PIPELINES];
int pipeline_count;
#ifdef _WIN32
CRITICAL_SECTION pipeline_mutex;
#else
pthread_mutex_t pipeline_mutex;
#endif
int pipeline_mutex_initialized;
void *dec_dvb_default; // Default decoder used in non-split mode
void *shared_ocr_ctx; // Shared OCR context to reduce memory usage
};
struct ccx_subtitle_pipeline *get_or_create_pipeline(struct lib_ccx_ctx *ctx, int pid, int stream_type, const char *lang);
void set_pipeline_pts(struct ccx_subtitle_pipeline *pipe, LLONG pts);
struct lib_ccx_ctx *init_libraries(struct ccx_s_options *opt);
void dinit_libraries(struct lib_ccx_ctx **ctx);

12
src/lib_ccx/list.h
View File

@@ -18,7 +18,7 @@
*/
#ifndef ccx_offsetof
#define ccx_offsetof(TYPE, MEMBER) ((size_t) & ((TYPE *)0)->MEMBER)
#define ccx_offsetof(TYPE, MEMBER) ((size_t)&((TYPE *)0)->MEMBER)
#endif
/**
@@ -53,10 +53,7 @@ struct list_head
struct list_head *next, *prev;
};
#define LIST_HEAD_INIT(name) \
{ \
&(name), &(name) \
}
#define LIST_HEAD_INIT(name) {&(name), &(name)}
#define LIST_HEAD(name) \
struct list_head name = LIST_HEAD_INIT(name)
@@ -374,10 +371,7 @@ struct hlist_node
struct hlist_node *next, **pprev;
};
#define HLIST_HEAD_INIT \
{ \
.first = NULL \
}
#define HLIST_HEAD_INIT {.first = NULL}
#define HLIST_HEAD(name) struct hlist_head name = {.first = NULL}
#define INIT_HLIST_HEAD(ptr) ((ptr)->first = NULL)
#define INIT_HLIST_NODE(ptr) ((ptr)->next = NULL, (ptr)->pprev = NULL)

4
src/lib_ccx/matroska.c
View File

@@ -1,4 +1,4 @@
#include "lib_ccx.h"
#include "lib_ccx.h"
#include "utility.h"
#include "matroska.h"
#include "ccx_encoders_helpers.h"
@@ -352,7 +352,7 @@ struct matroska_sub_sentence *parse_segment_cluster_block_group_block(struct mat
There can be 2 types of DVB in .mkv. One is when each display block is followed by empty block in order to
allow gaps in time between display blocks. Another one is when display block is followed by another display block.
This code handles both cases but we don't save and use empty blocks as sentences, only time_starts of them. */
char *dvb_message = enc_ctx->last_str;
char *dvb_message = enc_ctx->last_string;
if (ret < 0 || dvb_message == NULL)
{
// No text - no sentence is returned. Free the memory

11
src/lib_ccx/params.c
View File

@@ -363,15 +363,6 @@ void print_usage(void)
mprint(" stream will be processed. e.g. 'eng' for English.\n");
mprint(" If there are multiple languages, only this specified\n");
mprint(" language stream will be processed (default).\n");
mprint(" --split-dvb-subs: Extract each DVB subtitle stream to a separate file.\n");
mprint(" Each file will be named with the base filename plus a\n");
mprint(" language suffix (e.g., output_deu.srt, output_fra.srt).\n");
mprint(" For streams without language tags, uses PID as suffix.\n");
mprint(" Incompatible with: stdout output, manual PID selection,\n");
mprint(" multiprogram mode. Only works with SRT, SAMI, WebVTT.\n");
mprint(" --no-dvb-dedup: Disable DVB subtitle deduplication. By default, CCExtractor\n");
mprint(" filters out duplicate DVB subtitles to prevent repetition.\n");
mprint(" Use this flag to output all subtitles as-is.\n");
mprint(" --ocrlang: Manually select the name of the Tesseract .traineddata\n");
mprint(" file. Helpful if you want to OCR a caption stream of\n");
mprint(" one language with the data of another language.\n");
@@ -784,9 +775,7 @@ void version(char *location)
mprint(" Leptonica Version: %s\n", leptversion);
lept_free(leptversion);
#endif // ENABLE_OCR
#ifdef GPAC_AVAILABLE
mprint(" libGPAC Version: %s\n", GPAC_VERSION);
#endif
mprint(" zlib: %s\n", ZLIB_VERSION);
mprint(" utf8proc Version: %s\n", (const char *)utf8proc_version());
mprint(" libpng Version: %s\n", PNG_LIBPNG_VER_STRING);

24
src/lib_ccx/teletext.h
View File

@@ -8,7 +8,7 @@
// #include <inttypes.h>
#define MAX_TLT_PAGES 1000
#define MAX_TLT_PAGES_EXTRACT 8 // Maximum pages to extract simultaneously (must match lib_ccx.h)
#define MAX_TLT_PAGES_EXTRACT 8 // Maximum pages to extract simultaneously (must match lib_ccx.h)
typedef struct
{
@@ -22,23 +22,23 @@ typedef struct
// Per-page state for multi-page extraction (issue #665)
typedef struct
{
uint16_t page_number; // BCD-encoded page number (0 = unused slot)
teletext_page_t page_buffer; // Current page content being received
char *page_buffer_prev; // Previous formatted output
char *page_buffer_cur; // Current formatted output
uint16_t page_number; // BCD-encoded page number (0 = unused slot)
teletext_page_t page_buffer; // Current page content being received
char *page_buffer_prev; // Previous formatted output
char *page_buffer_cur; // Current formatted output
unsigned page_buffer_cur_size;
unsigned page_buffer_cur_used;
unsigned page_buffer_prev_size;
unsigned page_buffer_prev_used;
uint64_t *ucs2_buffer_prev; // Previous comparison string
uint64_t *ucs2_buffer_cur; // Current comparison string
uint64_t *ucs2_buffer_prev; // Previous comparison string
uint64_t *ucs2_buffer_cur; // Current comparison string
unsigned ucs2_buffer_cur_size;
unsigned ucs2_buffer_cur_used;
unsigned ucs2_buffer_prev_size;
unsigned ucs2_buffer_prev_used;
uint64_t prev_hide_timestamp;
uint64_t prev_show_timestamp;
uint8_t receiving_data; // Currently receiving data for this page
uint8_t receiving_data; // Currently receiving data for this page
} teletext_page_state_t;
// application states -- flags for notices that should be printed only once
@@ -86,10 +86,10 @@ struct TeletextCtx
uint32_t global_timestamp;
// Multi-page extraction state (issue #665)
teletext_page_state_t page_states[MAX_TLT_PAGES_EXTRACT]; // Per-page state
int num_active_pages; // Number of pages being extracted
int current_page_idx; // Index of page currently receiving data (-1 = none)
int multi_page_mode; // 1 = multi-page mode active
teletext_page_state_t page_states[MAX_TLT_PAGES_EXTRACT]; // Per-page state
int num_active_pages; // Number of pages being extracted
int current_page_idx; // Index of page currently receiving data (-1 = none)
int multi_page_mode; // 1 = multi-page mode active
// Current and previous page buffers (legacy single-page mode)
// These are still used when multi_page_mode == 0 for backward compatibility

4
src/lib_ccx/telxcc.c
View File

@@ -445,9 +445,9 @@ void remap_g0_charset(uint8_t c)
fprintf(stderr, "- G0 Latin National Subset ID 0x%1x.%1x is not implemented\n", (c >> 3), (c & 0x7));
return;
}
else if (m >= array_length(G0_LATIN_NATIONAL_SUBSETS))
else if (m >= 14)
{
fprintf(stderr, "- G0 Latin National Subset index %d is out of bounds (max %zu)\n", m, array_length(G0_LATIN_NATIONAL_SUBSETS) - 1);
fprintf(stderr, "- G0 Latin National Subset index %d is out of bounds\n", m);
return;
}
else

135
src/lib_ccx/ts_functions.c
View File

@@ -525,7 +525,7 @@ struct demuxer_data *get_best_data(struct demuxer_data *data)
{
if (ptr->codec == CCX_CODEC_DVB)
{
ret = ptr;
ret = data;
goto end;
}
}
@@ -581,14 +581,6 @@ int copy_capbuf_demux_data(struct ccx_demuxer *ctx, struct demuxer_data **data,
return CCX_OK;
}
vpesdatalen = read_video_pes_header(ctx, ptr, cinfo->capbuf, &pesheaderlen, cinfo->capbuflen);
if (vpesdatalen < 0 && ccx_options.split_dvb_subs && cinfo->codec == CCX_CODEC_DVB)
{
// Fallback: Treat as raw payload if PES header is missing/invalid in split mode
vpesdatalen = cinfo->capbuflen;
pesheaderlen = 0;
dbg_print(CCX_DMT_VERBOSE, "Fallback: Treating broken PES packet as raw DVB payload.\n");
}
if (ccx_options.pes_header_to_stdout && cinfo->codec == CCX_CODEC_DVB) // for teletext we have its own header dump
{
pes_header_dump(cinfo->capbuf, pesheaderlen);
@@ -604,7 +596,6 @@ int copy_capbuf_demux_data(struct ccx_demuxer *ctx, struct demuxer_data **data,
if (ccx_options.hauppauge_mode)
{
// if (cinfo->pid == 0x104) mprint("DEBUG-HAUP: 0x104 detected\n");
if (haup_capbuflen % 12 != 0)
mprint("Warning: Inconsistent Hauppage's buffer length\n");
if (!haup_capbuflen)
@@ -654,9 +645,11 @@ int copy_capbuf_demux_data(struct ccx_demuxer *ctx, struct demuxer_data **data,
{
if (ptr->len + databuflen >= BUFSIZE)
{
mprint("Warning: PES data packet (%ld) larger than remaining buffer (%lld), skipping packet.\n",
databuflen, BUFSIZE - ptr->len);
return CCX_OK;
fatal(CCX_COMMON_EXIT_BUG_BUG,
"PES data packet (%ld) larger than remaining buffer (%lld).\n"
"Please send bug report!",
databuflen, BUFSIZE - ptr->len);
return CCX_EAGAIN;
}
memcpy(ptr->buffer + ptr->len, databuf, databuflen);
ptr->len += databuflen;
@@ -684,11 +677,7 @@ int copy_payload_to_capbuf(struct cap_info *cinfo, struct ts_payload *payload)
cinfo->stream != CCX_STREAM_TYPE_VIDEO_HEVC) ||
!ccx_options.analyze_video_stream))
{
// In split DVB mode, allow DVB subtitle packets even if ignored
if (!(ccx_options.split_dvb_subs && cinfo->codec == CCX_CODEC_DVB))
{
return CCX_OK;
}
return CCX_OK;
}
// Verify PES before copy to capbuf
@@ -697,18 +686,11 @@ int copy_payload_to_capbuf(struct cap_info *cinfo, struct ts_payload *payload)
if (payload->start[0] != 0x00 || payload->start[1] != 0x00 ||
payload->start[2] != 0x01)
{
if (ccx_options.split_dvb_subs && cinfo->codec == CCX_CODEC_DVB)
{
dbg_print(CCX_DMT_VERBOSE, "Notice: Missing PES header in DVB packet (allowing due to split mode)\n");
}
else
{
mprint("Notice: Missing PES header\n");
dump(CCX_DMT_DUMPDEF, payload->start, payload->length, 0, 0);
cinfo->saw_pesstart = 0;
errno = EINVAL;
return -1;
}
mprint("Notice: Missing PES header\n");
dump(CCX_DMT_DUMPDEF, payload->start, payload->length, 0, 0);
cinfo->saw_pesstart = 0;
errno = EINVAL;
return -1;
}
}
@@ -798,11 +780,6 @@ int64_t ts_readstream(struct ccx_demuxer *ctx, struct demuxer_data **data)
if (ret != CCX_OK)
break;
if (payload.pid == 0x104)
{
// mprint("DEBUG-RAW: pid=0x104 err=%d len=%d\n", payload.transport_error, payload.length);
}
// Skip damaged packets, they could do more harm than good
if (payload.transport_error)
{
@@ -887,9 +864,8 @@ int64_t ts_readstream(struct ccx_demuxer *ctx, struct demuxer_data **data)
case 0: // First time we see this PID
if (ctx->PIDs_programs[payload.pid])
{
unsigned int st = ctx->PIDs_programs[payload.pid]->printable_stream_type;
dbg_print(CCX_DMT_PARSE, "\nNew PID found: %u (%s), belongs to program: %u\n", payload.pid,
(st < 256) ? desc[st] : "Unknown",
desc[ctx->PIDs_programs[payload.pid]->printable_stream_type],
ctx->PIDs_programs[payload.pid]->program_number);
ctx->PIDs_seen[payload.pid] = 2;
}
@@ -904,10 +880,9 @@ int64_t ts_readstream(struct ccx_demuxer *ctx, struct demuxer_data **data)
case 1: // Saw it before but we didn't know what program it belonged to. Luckier now?
if (ctx->PIDs_programs[payload.pid])
{
unsigned int st = ctx->PIDs_programs[payload.pid]->printable_stream_type;
dbg_print(CCX_DMT_PARSE, "\nProgram for PID: %u (previously unknown) is: %u (%s)\n", payload.pid,
ctx->PIDs_programs[payload.pid]->program_number,
(st < 256) ? desc[st] : "Unknown");
desc[ctx->PIDs_programs[payload.pid]->printable_stream_type]);
ctx->PIDs_seen[payload.pid] = 2;
}
break;
@@ -918,7 +893,7 @@ int64_t ts_readstream(struct ccx_demuxer *ctx, struct demuxer_data **data)
}
// PTS calculation
if (payload.pesstart && payload.length >= 14) // if there is PES Header data in the payload and we didn't get the first pts of that stream
if (payload.pesstart) // if there is PES Header data in the payload and we didn't get the first pts of that stream
{
// Packetized Elementary Stream (PES) 32-bit start code
uint64_t pes_prefix = (payload.start[0] << 16) | (payload.start[1] << 8) | payload.start[2];
@@ -932,16 +907,13 @@ int64_t ts_readstream(struct ccx_demuxer *ctx, struct demuxer_data **data)
pts = get_pts(payload.start);
// keep in mind we already checked if we have this stream id
// we find the index of the packet PID in the have_PIDs array
int pid_index = -1;
int pid_index;
for (int i = 0; i < ctx->num_of_PIDs; i++)
if (payload.pid == ctx->have_PIDs[i])
pid_index = i;
if (pid_index != -1)
{
ctx->stream_id_of_each_pid[pid_index] = pes_stream_id;
if (pts < ctx->min_pts[pid_index])
ctx->min_pts[pid_index] = pts; // and add its packet pts
}
ctx->stream_id_of_each_pid[pid_index] = pes_stream_id;
if (pts < ctx->min_pts[pid_index])
ctx->min_pts[pid_index] = pts; // and add its packet pts
}
}
@@ -985,15 +957,6 @@ int64_t ts_readstream(struct ccx_demuxer *ctx, struct demuxer_data **data)
}
cinfo = get_cinfo(ctx, payload.pid);
cinfo = get_cinfo(ctx, payload.pid);
if (payload.pid == 0x104 || payload.pid == 0x106)
{
// mprint("DEBUG-PID: pid=0x%X cinfo=%p len=%d\n", payload.pid, cinfo, payload.length);
if (cinfo)
{
// mprint("DEBUG-INFO: ignore=%d codec=%d pesstart=%d\n", cinfo->ignore, cinfo->codec, payload.pesstart);
}
}
if (cinfo == NULL)
{
if (!packet_analysis_mode)
@@ -1009,58 +972,44 @@ int64_t ts_readstream(struct ccx_demuxer *ctx, struct demuxer_data **data)
cinfo->stream != CCX_STREAM_TYPE_VIDEO_HEVC) ||
!ccx_options.analyze_video_stream))
{
// In split DVB mode, do NOT skip/cleanup DVB streams
if (ccx_options.split_dvb_subs && cinfo->codec == CCX_CODEC_DVB)
if (cinfo->codec_private_data)
{
// Fall through - process this DVB packet
}
else
{
if (cinfo->codec_private_data)
switch (cinfo->codec)
{
switch (cinfo->codec)
{
case CCX_CODEC_TELETEXT:
telxcc_close(&cinfo->codec_private_data, NULL);
break;
case CCX_CODEC_DVB:
dvbsub_close_decoder(&cinfo->codec_private_data);
break;
case CCX_CODEC_ISDB_CC:
delete_isdb_decoder(&cinfo->codec_private_data);
default:
break;
}
cinfo->codec_private_data = NULL;
case CCX_CODEC_TELETEXT:
telxcc_close(&cinfo->codec_private_data, NULL);
break;
case CCX_CODEC_DVB:
dvbsub_close_decoder(&cinfo->codec_private_data);
break;
case CCX_CODEC_ISDB_CC:
delete_isdb_decoder(&cinfo->codec_private_data);
default:
break;
}
cinfo->codec_private_data = NULL;
}
if (cinfo->capbuflen > 0)
{
freep(&cinfo->capbuf);
cinfo->capbufsize = 0;
cinfo->capbuflen = 0;
delete_demuxer_data_node_by_pid(data, cinfo->pid);
}
continue;
if (cinfo->capbuflen > 0)
{
freep(&cinfo->capbuf);
cinfo->capbufsize = 0;
cinfo->capbuflen = 0;
delete_demuxer_data_node_by_pid(data, cinfo->pid);
}
continue;
}
// Video PES start
if (payload.pesstart && payload.length >= 6)
if (payload.pesstart)
{
cinfo->saw_pesstart = 1;
cinfo->prev_counter = payload.counter - 1;
}
// Discard packets when no pesstart was found.
// Exception: DVB in split mode - allow packets to accumulate
if (!cinfo->saw_pesstart)
{
if (!(ccx_options.split_dvb_subs && cinfo->codec == CCX_CODEC_DVB))
{
continue;
}
}
continue;
if ((cinfo->prev_counter == 15 ? 0 : cinfo->prev_counter + 1) != payload.counter)
{

27
src/lib_ccx/ts_info.c
View File

@@ -43,18 +43,7 @@ void ignore_other_stream(struct ccx_demuxer *ctx, int pid)
list_for_each_entry(iter, &ctx->cinfo_tree.all_stream, all_stream, struct cap_info)
{
if (iter->pid != pid)
{
// In split DVB mode, do NOT ignore DVB subtitle streams
// They need to remain in the datalist for secondary pass processing
if (ccx_options.split_dvb_subs && iter->codec == CCX_CODEC_DVB)
{
iter->ignore = 0; // Keep DVB streams active
}
else
{
iter->ignore = 1;
}
}
iter->ignore = 1;
}
}
@@ -308,20 +297,6 @@ void dinit_cap(struct ccx_demuxer *ctx)
if (dec_ctx->private_data == saved_private_data)
dec_ctx->private_data = NULL;
}
// Also check subtitle pipelines for shared decoder references
// Pipelines have their own decoder and dec_ctx->private_data
for (int i = 0; i < lctx->pipeline_count; i++)
{
struct ccx_subtitle_pipeline *p = lctx->pipelines[i];
if (p)
{
if (p->decoder == saved_private_data)
p->decoder = NULL;
if (p->dec_ctx && p->dec_ctx->private_data == saved_private_data)
p->dec_ctx->private_data = NULL;
}
}
}
}
free(iter);

122
src/lib_ccx/ts_tables.c
View File

@@ -281,7 +281,7 @@ int parse_PMT(struct ccx_demuxer *ctx, unsigned char *buf, int len, struct progr
if (i + 5 + ES_info_length > len)
{
dbg_print(CCX_DMT_GENERIC_NOTICES, "Warning: ES_info_length exceeds buffer, skipping.\n");
continue;
break;
}
unsigned char *es_info = buf + i + 5;
@@ -346,7 +346,7 @@ int parse_PMT(struct ccx_demuxer *ctx, unsigned char *buf, int len, struct progr
if (i + 5 + ES_info_length > len)
{
dbg_print(CCX_DMT_GENERIC_NOTICES, "Warning: ES_info_length exceeds buffer, skipping.\n");
continue;
break;
}
unsigned char *es_info = buf + i + 5;
@@ -385,60 +385,6 @@ int parse_PMT(struct ccx_demuxer *ctx, unsigned char *buf, int len, struct progr
if (CCX_MPEG_DSC_DVB_SUBTITLE == descriptor_tag)
{
struct dvb_config cnf;
char detected_lang[4] = "und";
if (desc_len >= 3)
{
// ISO 639-2 compliant: accept only [a-zA-Z]{3}, convert to lowercase
int is_valid_lang = 1;
for (int li = 0; li < 3; li++)
{
char c = (char)es_info[li];
if (c >= 'A' && c <= 'Z')
detected_lang[li] = c + 32; // to lowercase
else if (c >= 'a' && c <= 'z')
detected_lang[li] = c;
else
{
is_valid_lang = 0;
break;
}
}
detected_lang[3] = '\0';
if (!is_valid_lang)
{
strcpy(detected_lang, "und");
dbg_print(CCX_DMT_PMT, "Warning: Invalid language code, using 'und'\n");
}
}
// If split mode enabled, track for pipeline creation
if (ccx_options.split_dvb_subs && ctx->potential_stream_count < MAX_POTENTIAL_STREAMS)
{
int found = 0;
for (int k = 0; k < ctx->potential_stream_count; k++)
{
if (ctx->potential_streams[k].pid == (int)elementary_PID)
{
found = 1;
break;
}
}
if (!found)
{
int p_idx = ctx->potential_stream_count;
ctx->potential_streams[p_idx].pid = (int)elementary_PID;
ctx->potential_streams[p_idx].stream_type = CCX_STREAM_TYPE_DVB_SUB;
ctx->potential_streams[p_idx].mpeg_type = stream_type;
memcpy(ctx->potential_streams[p_idx].lang, detected_lang, 4);
// Issue 2: Race Condition fix - populate metadata BEFORE incrementing count
// We can't fully populate yet (composition_id is parsed below), so we defer increment
// OR we just use the index p_idx and increment later.
}
}
#ifndef ENABLE_OCR
if (ccx_options.write_format != CCX_OF_SPUPNG)
{
@@ -446,67 +392,17 @@ int parse_PMT(struct ccx_demuxer *ctx, unsigned char *buf, int len, struct progr
continue;
}
#endif
if (!IS_FEASIBLE(ctx->codec, ctx->nocodec, CCX_CODEC_DVB) &&
!(ccx_options.split_dvb_subs && ctx->codec != CCX_CODEC_DVB))
if (!IS_FEASIBLE(ctx->codec, ctx->nocodec, CCX_CODEC_DVB))
continue;
memset((void *)&cnf, 0, sizeof(struct dvb_config));
ret = parse_dvb_description(&cnf, es_info, desc_len);
if (ret < 0)
break;
// Update metadata with specific IDs
if (ccx_options.split_dvb_subs)
{
int k_idx = -1;
int found = 0;
// Find if we already added it (or find the spot we are about to add)
for (int k = 0; k < ctx->potential_stream_count; k++)
{
if (ctx->potential_streams[k].pid == (int)elementary_PID)
{
k_idx = k;
found = 1;
break;
}
}
if (!found && ctx->potential_stream_count < MAX_POTENTIAL_STREAMS)
{
// It's the new one we are building
k_idx = ctx->potential_stream_count;
ctx->potential_streams[k_idx].pid = (int)elementary_PID;
ctx->potential_streams[k_idx].stream_type = CCX_STREAM_TYPE_DVB_SUB;
ctx->potential_streams[k_idx].mpeg_type = stream_type;
memcpy(ctx->potential_streams[k_idx].lang, detected_lang, 4);
}
if (k_idx != -1)
{
ctx->potential_streams[k_idx].composition_id = cnf.composition_id[0];
ctx->potential_streams[k_idx].ancillary_id = cnf.ancillary_id[0];
dbg_print(CCX_DMT_GENERIC_NOTICES, "Discovered DVB stream PID 0x%X lang=%s composition_id=%d ancillary_id=%d\n",
elementary_PID, detected_lang, cnf.composition_id[0], cnf.ancillary_id[0]);
// Only increment if it was a new one
if (!found && ctx->potential_stream_count < MAX_POTENTIAL_STREAMS)
{
ctx->potential_stream_count++;
}
}
}
ptr = dvbsub_init_decoder(&cnf);
if (ptr == NULL)
break;
update_capinfo(ctx, elementary_PID, stream_type, CCX_CODEC_DVB, program_number, ptr);
// Populate cap_info.lang with discovered language for fallback lookup
struct cap_info *cinfo = get_cinfo(ctx, elementary_PID);
if (cinfo && detected_lang[0] && detected_lang[0] != 'u') // Skip "und"
{
memset(cinfo->lang, 0, sizeof(cinfo->lang));
strncpy(cinfo->lang, detected_lang, sizeof(cinfo->lang) - 1);
}
max_dif = 30;
}
}
@@ -765,16 +661,8 @@ int parse_PAT(struct ccx_demuxer *ctx)
dinit_cap(ctx);
clear_PMT_array(ctx);
memset(ctx->PIDs_seen, 0, sizeof(int) * 65536); // Forget all we saw
ctx->num_of_PIDs = 0;
memset(ctx->have_PIDs, -1, (MAX_PSI_PID + 1) * sizeof(int));
for (int i = 0; i < (MAX_PSI_PID + 1); i++)
{
ctx->min_pts[i] = UINT64_MAX;
}
memset(ctx->stream_id_of_each_pid, 0, (MAX_PSI_PID + 1) * sizeof(uint8_t));
if (!tlt_config.user_page) // If the user didn't select a page...
tlt_config.page = 0; // ..forget whatever we detected.
if (!tlt_config.user_page) // If the user didn't select a page...
tlt_config.page = 0; // ..forget whatever we detected.
gotpes = 1;
}

32
src/lib_ccx/utility.c
View File

@@ -402,8 +402,6 @@ struct encoder_ctx *change_filename(struct encoder_ctx *enc_ctx)
char *get_basename(char *filename)
{
char *c;
char *last_dot = NULL;
char *last_slash = NULL;
int len;
char *basefilename;
@@ -419,32 +417,12 @@ char *get_basename(char *filename)
memcpy(basefilename, filename, len + 1);
for (c = basefilename; *c; c++)
for (c = basefilename + len; c > basefilename && *c != '.'; c--)
{
if (*c == '.')
last_dot = c;
else if (*c == '/' || *c == '\\')
last_slash = c;
}
if (last_dot)
{
// If there is a slash, the dot must be AFTER the slash to be an extension
if (last_slash)
{
if (last_dot > last_slash)
*last_dot = 0;
}
else
{
// No slash, so dot is extension.
// However, if the file starts with ., it's usually not considered an extension (e.g. .gitignore)
// But for CCExtractor context, we usually strip extension.
// If filename is just ".", we shouldn't strip it to empty?
if (last_dot != basefilename)
*last_dot = 0;
}
}
;
} // Get last .
if (*c == '.')
*c = 0;
return basefilename;
}

2
src/lib_ccx/zvbi/misc.h
View File

@@ -1,7 +1,7 @@
/*
* libzvbi -- Miscellaneous cows and chickens
*
* Copyright (C) 2000-2003 Iñaki García Etxebarria
* Copyright (C) 2000-2003 Iñaki García Etxebarria
* Copyright (C) 2002-2007 Michael H. Schimek
*
* This library is free software; you can redistribute it and/or

6
src/rust/lib_ccxr/src/common/options.rs
View File

@@ -521,10 +521,6 @@ pub struct Options {
pub multiprogram: bool,
pub out_interval: i32,
pub segment_on_key_frames_only: bool,
/// Enable per-stream DVB subtitle extraction
pub split_dvb_subs: bool,
/// Disable DVB subtitle deduplication
pub no_dvb_dedup: bool,
/// SCC input framerate: 0=29.97 (default), 1=24, 2=25, 3=30
pub scc_framerate: i32,
/// SCC accurate timing (issue #1120): if true, use bandwidth-aware timing for broadcast compliance
@@ -632,8 +628,6 @@ impl Default for Options {
multiprogram: Default::default(),
out_interval: -1,
segment_on_key_frames_only: Default::default(),
split_dvb_subs: Default::default(),
no_dvb_dedup: Default::default(),
scc_framerate: 0, // 0 = 29.97fps (default)
scc_accurate_timing: false, // Off by default for backwards compatibility (issue #1120)
debug_mask: DebugMessageMask::new(

14
src/rust/src/args.rs
View File

@@ -591,20 +591,6 @@ pub struct Args {
/// language stream will be processed (default).
#[arg(long, verbatim_doc_comment, help_heading=OUTPUT_AFFECTING_OUTPUT_FILES)]
pub dvblang: Option<String>,
/// Extract each DVB subtitle stream to a separate file.
/// Each file will be named with the base filename plus a
/// language suffix (e.g., output_deu.srt, output_fra.srt).
/// For streams without language tags, uses PID as suffix.
/// Incompatible with: stdout output, manual PID selection,
/// multiprogram mode. Only works with SRT, SAMI, WebVTT.
#[arg(long, verbatim_doc_comment, help_heading=OUTPUT_AFFECTING_OUTPUT_FILES)]
pub split_dvb_subs: bool,
/// Disable DVB subtitle deduplication when using --split-dvb-subs.
/// By default, CCExtractor filters out duplicate DVB subtitles
/// to prevent repetition in split output files. Use this flag
/// to disable deduplication and output all subtitles as-is.
#[arg(long, verbatim_doc_comment, help_heading=OUTPUT_AFFECTING_OUTPUT_FILES)]
pub no_dvb_dedup: bool,
/// Manually select the name of the Tesseract .traineddata
/// file. Helpful if you want to OCR a caption stream of
/// one language with the data of another language.

5
src/rust/src/common.rs
View File

@@ -277,8 +277,6 @@ pub unsafe fn copy_from_rust(ccx_s_options: *mut ccx_s_options, options: Options
(*ccx_s_options).multiprogram = options.multiprogram as _;
(*ccx_s_options).out_interval = options.out_interval;
(*ccx_s_options).segment_on_key_frames_only = options.segment_on_key_frames_only as _;
(*ccx_s_options).split_dvb_subs = options.split_dvb_subs as _;
(*ccx_s_options).no_dvb_dedup = options.no_dvb_dedup as _;
(*ccx_s_options).scc_framerate = options.scc_framerate;
// Also copy to enc_cfg so the encoder uses the same frame rate for SCC output
(*ccx_s_options).enc_cfg.scc_framerate = options.scc_framerate;
@@ -541,8 +539,6 @@ pub unsafe fn copy_to_rust(ccx_s_options: *const ccx_s_options) -> Options {
options.multiprogram = (*ccx_s_options).multiprogram != 0;
options.out_interval = (*ccx_s_options).out_interval;
options.segment_on_key_frames_only = (*ccx_s_options).segment_on_key_frames_only != 0;
options.split_dvb_subs = (*ccx_s_options).split_dvb_subs != 0;
options.no_dvb_dedup = (*ccx_s_options).no_dvb_dedup != 0;
options.scc_framerate = (*ccx_s_options).scc_framerate;
options.scc_accurate_timing = (*ccx_s_options).enc_cfg.scc_accurate_timing != 0;
@@ -1029,7 +1025,6 @@ impl CType<cap_info> for CapInfo {
prev_counter: self.prev_counter,
codec_private_data: self.codec_private_data,
ignore: self.ignore,
lang: [0; 4],
all_stream: self.all_stream,
sib_head: self.sib_head,
sib_stream: self.sib_stream,

6
src/rust/src/libccxr_exports/time.rs
View File

@@ -477,11 +477,7 @@ pub unsafe extern "C" fn ccxr_get_fts(
1 => CaptionField::Field1,
2 => CaptionField::Field2,
3 => CaptionField::Cea708,
_ => {
// DVB subtitles may pass 0 or other values when decoder context
// current_field is uninitialized. Default to Field1 to avoid crash.
CaptionField::Field1
}
_ => panic!("incorrect value for caption field"),
};
let ans = c::get_fts(&mut context, caption_field);

55
src/rust/src/parser.rs
View File

@@ -726,16 +726,6 @@ impl OptionsExt for Options {
self.ocrlang = Some(ocrlang.clone());
}
// Handle --split-dvb-subs flag
if args.split_dvb_subs {
self.split_dvb_subs = true;
}
// Handle --no-dvb-dedup flag
if args.no_dvb_dedup {
self.no_dvb_dedup = true;
}
if let Some(ref quant) = args.quant {
if !(0..=2).contains(quant) {
fatal!(
@@ -1677,51 +1667,6 @@ impl OptionsExt for Options {
{
self.enc_cfg.curlposturl = self.curlposturl.clone();
}
// Validate --split-dvb-subs conflicts
if self.split_dvb_subs {
if self.cc_to_stdout {
fatal!(
cause = ExitCause::IncompatibleParameters;
"--split-dvb-subs cannot be used with stdout output.\n\
Multiple output files cannot be written to stdout."
);
}
if self.demux_cfg.ts_forced_cappid {
fatal!(
cause = ExitCause::IncompatibleParameters;
"--split-dvb-subs cannot be used with manual PID selection (-pn).\n\
Automatic stream detection is required for multi-stream extraction."
);
}
if self.multiprogram {
fatal!(
cause = ExitCause::IncompatibleParameters;
"--split-dvb-subs cannot be used with -multiprogram.\n\
These modes have conflicting output file naming schemes."
);
}
// Validate supported output formats
match self.write_format {
OutputFormat::Srt
| OutputFormat::Sami
| OutputFormat::WebVtt
| OutputFormat::Null
| OutputFormat::Transcript
| OutputFormat::Ssa
| OutputFormat::SimpleXml
| OutputFormat::SmpteTt => {}
_ => {
fatal!(
cause = ExitCause::IncompatibleParameters;
"Unsupported OutputFormat: {:?}. --split-dvb-subs requires SRT, SAMI, WebVTT, Transcript, SSA, SimpleXML, SMPTE-TT or NULL output format.", self.write_format
);
}
}
}
}
}
#[cfg(test)]

BIN
tessdata/chi_sim.traineddata
View File

Binary file not shown.

BIN
tessdata/eng.traineddata
View File

Binary file not shown.

BIN
tessdata/spa.traineddata
View File

Binary file not shown.

127
tests/dvb_dedup_test.sh
View File

@@ -1,127 +0,0 @@
#!/bin/bash
# DVB Split Deduplication Test Suite
# Tests DVB-001 through DVB-008 requirements
set -e
echo "CCExtractor DVB Split Deduplication Test Suite"
echo "==============================================="
echo ""
# Configuration
CCEXTRACTOR="./ccextractor"
TEST_DIR="/home/rahul/Desktop/all_tests"
OUTPUT_DIR="/tmp/dvb_split_test_$$"
# Test files
MULTIPROGRAM_SPAIN="$TEST_DIR/multiprogram_spain.ts"
MP_SPAIN_C49="$TEST_DIR/mp_spain_20170112_C49.ts"
BBC1="$TEST_DIR/BBC1.mp4"
mkdir -p "$OUTPUT_DIR"
echo "Output directory: $OUTPUT_DIR"
echo ""
# DVB-004: Test multilang split (5 files)
echo "DVB-004: Testing multilingual DVB split..."
if [ -f "$MULTIPROGRAM_SPAIN" ]; then
$CCEXTRACTOR "$MULTIPROGRAM_SPAIN" --split-dvb-subs -o "$OUTPUT_DIR/mp_spain" 2>&1 | grep -i "dvb\|split" | head -10
FILE_COUNT=$(ls -1 "$OUTPUT_DIR"/mp_spain*.srt 2>/dev/null | wc -l)
echo " Created $FILE_COUNT output files"
if [ $FILE_COUNT -ge 4 ]; then
echo " ✓ DVB-004: PASS (created $FILE_COUNT files, expected 4-5)"
else
echo " ✗ DVB-004: FAIL (created $FILE_COUNT files, expected 4-5)"
fi
# Check file sizes (should not be 0 bytes if OCR enabled)
echo " File sizes:"
ls -lh "$OUTPUT_DIR"/mp_spain*.srt 2>/dev/null | awk '{print " " $9 ": " $5}'
else
echo " ⚠ SKIP: Test file not found: $MULTIPROGRAM_SPAIN"
fi
echo ""
# DVB-005: Test single program DVB extraction (using -pn to select one program)
echo "DVB-005: Testing single program DVB extraction..."
if [ -f "$MULTIPROGRAM_SPAIN" ]; then
# Use program number 530 which has DVB subtitles on PID 0xD3
$CCEXTRACTOR --split-dvb-subs --program-number 530 -o "$OUTPUT_DIR/single_prog" "$MULTIPROGRAM_SPAIN" 2>&1 | grep -i "dvb\|Created\|split" | head -5
FILE_COUNT=$(ls -1 "$OUTPUT_DIR"/single_prog*.srt 2>/dev/null | wc -l)
echo " Created $FILE_COUNT output files"
if [ $FILE_COUNT -ge 1 ]; then
echo " ✓ DVB-005: PASS"
else
echo " ✗ DVB-005: FAIL (Note: mp_spain_20170112_C49.ts has Teletext, not DVB subs)"
fi
else
echo " ⚠ SKIP: Test file not found: $MULTIPROGRAM_SPAIN"
fi
echo ""
# DVB-006: Test non-DVB file
echo "DVB-006: Testing non-DVB file (should work without errors)..."
if [ -f "$BBC1" ]; then
$CCEXTRACTOR "$BBC1" --split-dvb-subs -o "$OUTPUT_DIR/bbc1" 2>&1 | tail -5
echo " ✓ DVB-006: PASS (no crash)"
else
echo " ⚠ SKIP: Test file not found: $BBC1"
fi
echo ""
# DVB-007: Test repetition bug fix (check for excessive duplicates)
echo "DVB-007: Testing deduplication effectiveness..."
SRT_FILES=$(ls "$OUTPUT_DIR"/mp_spain_*.srt 2>/dev/null)
if [ -f "$MULTIPROGRAM_SPAIN" ] && [ -n "$SRT_FILES" ]; then
for file in "$OUTPUT_DIR"/mp_spain_*.srt; do
if [ -f "$file" ] && [ -s "$file" ]; then
# Count consecutive duplicate subtitle blocks
DUPES=$(awk '/^[0-9]+$/{n=$0} /^$/{if(prev==n)dup++; prev=n} END{print dup+0}' "$file")
TOTAL=$(grep -c "^[0-9]*$" "$file" || echo "0")
TOTAL=$(echo "$TOTAL" | tr -d '\n')
if [ "$TOTAL" -gt 0 ]; then
RATIO=$(awk "BEGIN {printf \"%.1f\", ($DUPES/$TOTAL)*100}")
echo " File: $(basename $file)"
echo " Total subtitles: $TOTAL"
echo " Potential duplicates: $DUPES"
echo " Ratio: ${RATIO}%"
if [ "$DUPES" -lt 3 ]; then
echo " ✓ Good deduplication"
fi
fi
fi
done
echo " ✓ DVB-007: Check complete (review ratios above)"
else
echo " ⚠ SKIP: No output files to check"
fi
echo ""
# DVB-008: Test --no-dvb-dedup flag
echo "DVB-008: Testing --no-dvb-dedup flag..."
if [ -f "$MULTIPROGRAM_SPAIN" ]; then
$CCEXTRACTOR "$MULTIPROGRAM_SPAIN" --split-dvb-subs --no-dvb-dedup -o "$OUTPUT_DIR/mp_spain_nodedup" 2>&1 | tail -3
FILE_COUNT=$(ls -1 "$OUTPUT_DIR"/mp_spain_nodedup*.srt 2>/dev/null | wc -l)
if [ $FILE_COUNT -ge 1 ]; then
echo " ✓ DVB-008: PASS (flag accepted, created $FILE_COUNT files)"
else
echo " ✗ DVB-008: FAIL"
fi
else
echo " ⚠ SKIP: Test file not found"
fi
echo ""
echo "Test suite complete!"
echo "Output files in: $OUTPUT_DIR"
echo ""
echo "NOTE: Without OCR (Tesseract), DVB subtitle files may be empty."
echo " The deduplication logic is still applied correctly."

0
tests/dvb_split_suite.c
View File

250
tests/integration/dvb_split_tests.sh
View File

@@ -1,250 +0,0 @@
#!/bin/bash
# Integration tests for --split-dvb-subs feature
# This script tests the DVB subtitle splitting functionality
set -e
# Configuration
CCX_BIN="./linux/ccextractor"
TEST_DIR="/home/rahul/Desktop/all_tests"
OUTPUT_DIR="/tmp/dvb_split_test_output"
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
NC='\033[0m' # No Color
# Test counters
TESTS_RUN=0
TESTS_PASSED=0
TESTS_FAILED=0
# Helper functions
print_header() {
echo "========================================"
echo "$1"
echo "========================================"
}
print_test() {
echo -n "TEST: $1 ... "
((TESTS_RUN++))
}
print_pass() {
echo -e "${GREEN}PASS${NC}"
((TESTS_PASSED++))
}
print_fail() {
echo -e "${RED}FAIL${NC}"
echo " Reason: $1"
((TESTS_FAILED++))
}
print_skip() {
echo -e "${YELLOW}SKIP${NC}"
echo " Reason: $1"
}
cleanup() {
rm -rf "$OUTPUT_DIR"
}
# Setup
mkdir -p "$OUTPUT_DIR"
trap cleanup EXIT
print_header "DVB Split Subtitle Integration Tests"
# Test 1: Happy Path - Single DVB Stream Split
print_test "Happy Path - Single DVB Stream Split"
TEST_FILE="$TEST_DIR/multiprogram_spain.ts"
OUTPUT_BASE="$OUTPUT_DIR/test1_output"
if [ ! -f "$TEST_FILE" ]; then
print_fail "Test file not found: $TEST_FILE"
else
# Run ccextractor with split-dvb-subs
"$CCX_BIN" "$TEST_FILE" --split-dvb-subs -o "$OUTPUT_BASE" > "$OUTPUT_DIR/test1.log" 2>&1
# Check if split files were created
SPLIT_FILES=$(ls "$OUTPUT_BASE"_*.srt 2>/dev/null | wc -l)
if [ "$SPLIT_FILES" -eq 0 ]; then
print_fail "No split files created"
else
# Check if split files have content
EMPTY_COUNT=0
for file in "$OUTPUT_BASE"_*.srt; do
if [ ! -s "$file" ] || [ $(stat -c%s "$file") -eq 0 ]; then
((EMPTY_COUNT++))
fi
done
if [ "$EMPTY_COUNT" -eq "$SPLIT_FILES" ]; then
print_fail "All split files are empty (0 bytes)"
else
# Check if files have valid SRT format
VALID_SRT=0
for file in "$OUTPUT_BASE"_*.srt; do
if [ -s "$file" ] && [ $(stat -c%s "$file") -gt 0 ]; then
# Check for SRT format markers (timestamps)
if grep -q "^-->" "$file" && grep -q "^[0-9]" "$file"; then
((VALID_SRT++))
fi
fi
done
if [ "$VALID_SRT" -gt 0 ]; then
print_pass "Split files created with valid SRT content"
else
print_fail "Split files created but invalid SRT format"
fi
fi
fi
fi
# Test 2: Edge Case - Multiple DVB Streams with Different Languages
print_test "Edge Case - Multiple DVB Streams with Different Languages"
TEST_FILE="$TEST_DIR/04e47919de5908edfa1fddc522a811d56bc67a1d4020f8b3972709e25b15966c.ts"
OUTPUT_BASE="$OUTPUT_DIR/test2_output"
if [ ! -f "$TEST_FILE" ]; then
print_skip "Test file not found (large file may not be available)"
else
# Run ccextractor with split-dvb-subs
timeout 300 "$CCX_BIN" "$TEST_FILE" --split-dvb-subs -o "$OUTPUT_BASE" > "$OUTPUT_DIR/test2.log" 2>&1
EXIT_CODE=$?
if [ $EXIT_CODE -eq 124 ]; then
print_skip "Test timed out (large file)"
elif [ $EXIT_CODE -ne 0 ]; then
print_fail "ccextractor exited with code $EXIT_CODE"
else
# Check for language-specific split files
LANG_FILES=$(ls "$OUTPUT_BASE"_*.srt 2>/dev/null | wc -l)
if [ "$LANG_FILES" -ge 3 ]; then
print_pass "Multiple language split files created"
else
print_fail "Expected at least 3 language files, got $LANG_FILES"
fi
fi
fi
# Test 3: Edge Case - File with No DVB Subtitles
print_test "Edge Case - File with No DVB Subtitles"
TEST_FILE="$TEST_DIR/BBC1.mp4"
OUTPUT_BASE="$OUTPUT_DIR/test3_output"
if [ ! -f "$TEST_FILE" ]; then
print_skip "Test file not found"
else
# Run ccextractor with split-dvb-subs
"$CCX_BIN" "$TEST_FILE" --split-dvb-subs -o "$OUTPUT_BASE" > "$OUTPUT_DIR/test3.log" 2>&1
# Check that no split files were created
SPLIT_FILES=$(ls "$OUTPUT_BASE"_*.srt 2>/dev/null | wc -l)
if [ "$SPLIT_FILES" -eq 0 ]; then
# Check log for "No captions found" message
if grep -q "No captions were found" "$OUTPUT_DIR/test3.log"; then
print_pass "No split files created (no DVB subtitles found)"
else
print_fail "No split files but no 'No captions' message in log"
fi
else
print_fail "Split files created for file with no DVB subtitles"
fi
fi
# Test 4: Edge Case - Single DVB Stream (C49)
print_test "Edge Case - Single DVB Stream (C49)"
TEST_FILE="$TEST_DIR/mp_spain_20170112_C49.ts"
OUTPUT_BASE="$OUTPUT_DIR/test4_output"
if [ ! -f "$TEST_FILE" ]; then
print_skip "Test file not found"
else
"$CCX_BIN" "$TEST_FILE" --split-dvb-subs -o "$OUTPUT_BASE" > "$OUTPUT_DIR/test4.log" 2>&1
SPLIT_FILES=$(ls "$OUTPUT_BASE"_*.srt 2>/dev/null | wc -l)
if [ "$SPLIT_FILES" -eq 1 ]; then
FILE_SIZE=$(stat -c%s "$OUTPUT_BASE"_*.srt)
if [ "$FILE_SIZE" -gt 0 ]; then
print_pass "Single DVB stream split successfully"
else
print_fail "Split file is empty (0 bytes)"
fi
else
print_fail "Expected 1 split file, got $SPLIT_FILES"
fi
fi
# Test 5: Edge Case - Single DVB Stream (C55)
print_test "Edge Case - Single DVB Stream (C55)"
TEST_FILE="$TEST_DIR/mp_spain_20170112_C55.ts"
OUTPUT_BASE="$OUTPUT_DIR/test5_output"
if [ ! -f "$TEST_FILE" ]; then
print_skip "Test file not found"
else
"$CCX_BIN" "$TEST_FILE" --split-dvb-subs -o "$OUTPUT_BASE" > "$OUTPUT_DIR/test5.log" 2>&1
SPLIT_FILES=$(ls "$OUTPUT_BASE"_*.srt 2>/dev/null | wc -l)
if [ "$SPLIT_FILES" -eq 1 ]; then
FILE_SIZE=$(stat -c%s "$OUTPUT_BASE"_*.srt)
if [ "$FILE_SIZE" -gt 0 ]; then
print_pass "Single DVB stream split successfully"
else
print_fail "Split file is empty (0 bytes)"
fi
else
print_fail "Expected 1 split file, got $SPLIT_FILES"
fi
fi
# Test 6: Regression - Verify Bug Fix (Empty Split Files)
print_test "Regression - Verify Bug Fix (Empty Split Files)"
TEST_FILE="$TEST_DIR/multiprogram_spain.ts"
OUTPUT_BASE="$OUTPUT_DIR/test6_output"
if [ ! -f "$TEST_FILE" ]; then
print_skip "Test file not found"
else
"$CCX_BIN" "$TEST_FILE" --split-dvb-subs -o "$OUTPUT_BASE" > "$OUTPUT_DIR/test6.log" 2>&1
# Check for the specific bug: 0-byte split files
EMPTY_COUNT=0
for file in "$OUTPUT_BASE"_*.srt; do
if [ -f "$file" ] && [ $(stat -c%s "$file") -eq 0 ]; then
((EMPTY_COUNT++))
fi
done
if [ "$EMPTY_COUNT" -eq 0 ]; then
print_pass "Bug fixed - no empty split files"
else
print_fail "Bug still present - $EMPTY_COUNT empty split files found"
fi
fi
# Summary
print_header "Test Summary"
echo "Tests Run: $TESTS_RUN"
echo -e "Tests Passed: ${GREEN}$TESTS_PASSED${NC}"
echo -e "Tests Failed: ${RED}$TESTS_FAILED${NC}"
echo "Tests Skipped: $((TESTS_RUN - TESTS_PASSED - TESTS_FAILED))"
echo ""
echo "Test logs available in: $OUTPUT_DIR"
# Exit with appropriate code
if [ $TESTS_FAILED -eq 0 ]; then
exit 0
else
exit 1

6
tests/regression/dvb_split.txt
View File

@@ -1,6 +0,0 @@
TEST: split_dvb_multilang
INPUT: arte_multiaudio.ts
ARGS: --split-dvb-subs -o output_split
EXPECT: output_split_fra.srt exists
EXPECT: output_split_deu.srt exists

0
tests/samples/sbs_append_string_00.srt
View File

12
windows/ccextractor.vcxproj
View File

@@ -1,4 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ItemGroup Label="ProjectConfigurations">
<ProjectConfiguration Include="Debug-Full|x64">
@@ -34,9 +34,8 @@
<ClInclude Include="..\src\lib_ccx\ccx_encoders_helpers.h" />
<ClInclude Include="..\src\lib_ccx\ccx_encoders_mcc.h" />
<ClInclude Include="..\src\lib_ccx\disable_warnings.h" />
<ClInclude Include="..\src\lib_ccx\dvb_subtitle_decoder.h" />
<ClInclude Include="..\src\lib_ccx\dvb_dedup.h" />
<ClInclude Include="..\src\lib_ccx\lib_ccx.h" />
<ClInclude Include="..\src\lib_ccx\dvb_subtitle_decoder.h" />
<ClInclude Include="..\src\lib_ccx\lib_ccx.h" />
<ClInclude Include="..\src\lib_ccx\teletext.h" />
<ClInclude Include="..\src\lib_ccx\utility.h" />
<ClInclude Include="..\src\lib_ccx\vobsub_decoder.h" />
@@ -104,9 +103,8 @@
<ClCompile Include=" ..\src\lib_ccx\ccx_gxf.c" />
<ClCompile Include=" ..\src\lib_ccx\cc_bitstream.c" />
<ClCompile Include=" ..\src\lib_ccx\configuration.c" />
<ClCompile Include=" ..\src\lib_ccx\dvb_subtitle_decoder.c" />
<ClCompile Include=" ..\src\lib_ccx\dvb_dedup.c" />
<ClCompile Include=" ..\src\lib_ccx\dvd_subtitle_decoder.c" />
<ClCompile Include=" ..\src\lib_ccx\dvb_subtitle_decoder.c" />
<ClCompile Include=" ..\src\lib_ccx\dvd_subtitle_decoder.c" />
<ClCompile Include=" ..\src\lib_ccx\es_functions.c" />
<ClCompile Include=" ..\src\lib_ccx\es_userdata.c" />
<ClCompile Include=" ..\src\lib_ccx\ffmpeg_intgr.c" />