[PR #537] [MERGED] Split to sentences implementation #1366

New Issue

claunia · 2026-01-29T17:16:00Z

claunia commented

2026-01-29 17:16:00 +00:00

📋 Pull Request Information

Original PR: https://github.com/CCExtractor/ccextractor/pull/537
Author: @maxkoryukov
Created: 12/15/2016
Status: ✅ Merged
Merged: 12/16/2016
Merged by: @cfsmp3

Base: master ← Head: feature/sbs

📝 Commits (3)

66393a8 Break incoming subs into sentences (through a buffer), and remove duplicates
4743eb4 Try to make changes compatible with upstream
54f628f remove debug-files, increase verbosity for tests

📊 Changes

9 files changed (+875 additions, -114 deletions)

View changed files

📝 .gitignore (+6 -0)
📝 src/lib_ccx/ccx_encoders_common.c (+10 -12)
📝 src/lib_ccx/ccx_encoders_common.h (+9 -11)
📝 src/lib_ccx/ccx_encoders_splitbysentence.c (+418 -91)
➕ tests/Makefile (+59 -0)
➕ tests/README.md (+43 -0)
➕ tests/ccx_encoders_splitbysentence_suite.c (+305 -0)
➕ tests/ccx_encoders_splitbysentence_suite.h (+4 -0)
➕ tests/runtest.c (+21 -0)

📄 Description

Split incoming subs to separated sentences, with duplicate-detection.

Replacement for PR #491

Usage:

./ccextractor -sbs ~/source.ts

Currently, it works only with sub->type == CC_BITMAP. Implementation details - in comments to the PR.

The PR contains tests, a concise description attached (test/README.md)

Long example

New output

1
00:00:00,001 --> 00:00:00,189
Oleon costs.

2
00:00:00,191 --> 00:00:00,783
buried in the annex, 95 Oleon costs.

3
00:00:00,785 --> 00:00:05,159
Didn't want to acknowledge the pressures on hospitals, schools and infrastructure.

Old output

1
00:00:00,001 --> 00:00:00,000
Oleon

2
00:00:00,001 --> 00:00:00,189
Oleon costs.

3
00:00:00,190 --> 00:00:00,889
buried in the annex, 95 Oleon costs.
Didn't

4
00:00:00,890 --> 00:00:01,129
buried in the annex, 95 Oleon costs.
Didn't want

5
00:00:01,130 --> 00:00:01,359
buried in the annex, 95 Oleon costs.
Didn't want to

6
00:00:01,360 --> 00:00:02,059
buried in the annex, 95 Oleon costs.
Didn't want to acknowledge

7
00:00:02,060 --> 00:00:02,299
buried in the annex, 95 Oleon costs.
Didn't want to acknowledge the

8
00:00:02,300 --> 00:00:03,419
Didn't want to acknowledge the
pressures

9
00:00:03,420 --> 00:00:03,609
Didn't want to acknowledge the
pressures on

10
00:00:03,610 --> 00:00:04,029
Didn't want to acknowledge the
pressures on hospitals,

11
00:00:04,030 --> 00:00:04,779
Didn't want to acknowledge the
pressures on hospitals, schools

12
00:00:04,780 --> 00:00:05,019
Didn't want to acknowledge the
pressures on hospitals, schools and

13
00:00:05,020 --> 00:00:05,159
pressures on hospitals, schools and
infrastructure.

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/CCExtractor/ccextractor/pull/537 **Author:** [@maxkoryukov](https://github.com/maxkoryukov) **Created:** 12/15/2016 **Status:** ✅ Merged **Merged:** 12/16/2016 **Merged by:** [@cfsmp3](https://github.com/cfsmp3) **Base:** `master` ← **Head:** `feature/sbs` --- ### 📝 Commits (3) - [`66393a8`](https://github.com/CCExtractor/ccextractor/commit/66393a80f2b06565080319231df381e033c93f7f) Break incoming subs into sentences (through a buffer), and remove duplicates - [`4743eb4`](https://github.com/CCExtractor/ccextractor/commit/4743eb46c1ca9e5bf741c5453b4e9a7697c9fcff) Try to make changes compatible with upstream - [`54f628f`](https://github.com/CCExtractor/ccextractor/commit/54f628f7a668019e1680b792fd3b16da55c5e954) remove debug-files, increase verbosity for tests ### 📊 Changes **9 files changed** (+875 additions, -114 deletions) <details> <summary>View changed files</summary> 📝 `.gitignore` (+6 -0) 📝 `src/lib_ccx/ccx_encoders_common.c` (+10 -12) 📝 `src/lib_ccx/ccx_encoders_common.h` (+9 -11) 📝 `src/lib_ccx/ccx_encoders_splitbysentence.c` (+418 -91) ➕ `tests/Makefile` (+59 -0) ➕ `tests/README.md` (+43 -0) ➕ `tests/ccx_encoders_splitbysentence_suite.c` (+305 -0) ➕ `tests/ccx_encoders_splitbysentence_suite.h` (+4 -0) ➕ `tests/runtest.c` (+21 -0) </details> ### 📄 Description Split incoming subs to separated sentences, with duplicate-detection. Replacement for PR #491 Usage: ```shell ./ccextractor -sbs ~/source.ts ``` Currently, it works only with `sub->type == CC_BITMAP`. Implementation details - in comments to the PR. The PR contains tests, a concise description attached (`test/README.md`) ### Long example #### New output ``` 1 00:00:00,001 --> 00:00:00,189 Oleon costs. 2 00:00:00,191 --> 00:00:00,783 buried in the annex, 95 Oleon costs. 3 00:00:00,785 --> 00:00:05,159 Didn't want to acknowledge the pressures on hospitals, schools and infrastructure. ``` #### Old output ``` 1 00:00:00,001 --> 00:00:00,000 Oleon 2 00:00:00,001 --> 00:00:00,189 Oleon costs. 3 00:00:00,190 --> 00:00:00,889 buried in the annex, 95 Oleon costs. Didn't 4 00:00:00,890 --> 00:00:01,129 buried in the annex, 95 Oleon costs. Didn't want 5 00:00:01,130 --> 00:00:01,359 buried in the annex, 95 Oleon costs. Didn't want to 6 00:00:01,360 --> 00:00:02,059 buried in the annex, 95 Oleon costs. Didn't want to acknowledge 7 00:00:02,060 --> 00:00:02,299 buried in the annex, 95 Oleon costs. Didn't want to acknowledge the 8 00:00:02,300 --> 00:00:03,419 Didn't want to acknowledge the pressures 9 00:00:03,420 --> 00:00:03,609 Didn't want to acknowledge the pressures on 10 00:00:03,610 --> 00:00:04,029 Didn't want to acknowledge the pressures on hospitals, 11 00:00:04,030 --> 00:00:04,779 Didn't want to acknowledge the pressures on hospitals, schools 12 00:00:04,780 --> 00:00:05,019 Didn't want to acknowledge the pressures on hospitals, schools and 13 00:00:05,020 --> 00:00:05,159 pressures on hospitals, schools and infrastructure. ``` --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>

claunia added the pull-request label 2026-01-29 17:16:00 +00:00

claunia closed this issue

2026-01-29 17:16:01 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: starred/ccextractor#1366