starred/ccextractor
1
0
Fork 0
mirror of https://github.com/CCExtractor/ccextractor.git synced 2026-02-03 21:23:48 +00:00
Code Issues 1.5k Packages Projects Releases 20 Wiki Activity

'live' raw data problem #275

New Issue
Closed
opened 2026-01-29 16:39:39 +00:00 by claunia · 4 comments
claunia commented 2026-01-29 16:39:39 +00:00
Owner
Copy Link

Originally created by @CTayl0r on GitHub (Feb 10, 2017).

I'm trying to send raw CC data (i.e. mcpoodle raw format) into ccextractor from a live source (extracted form a 2110.40 stream). I've found that ccExtractor buffers quite a lot of raw data before processing any, and then when it does process that, it stops once it sees no more data, despite the live (-s) option. This is the case whether I send the data in via UDP, pipe line it in, or use a file. I've tried the non-buffering options, but they seem to make no difference. Is there a way to get 'live' raw data processed?

Originally created by @CTayl0r on GitHub (Feb 10, 2017). I'm trying to send raw CC data (i.e. mcpoodle raw format) into ccextractor from a live source (extracted form a 2110.40 stream). I've found that ccExtractor buffers quite a lot of raw data before processing any, and then when it does process that, it stops once it sees no more data, despite the live (-s) option. This is the case whether I send the data in via UDP, pipe line it in, or use a file. I've tried the non-buffering options, but they seem to make no difference. Is there a way to get 'live' raw data processed?
claunia added the needs-confirmation-of-being-brokendifficulty: easy labels 2026-01-29 16:39:39 +00:00
claunia commented 2026-01-29 16:39:40 +00:00
Author
Owner
Copy Link

@CTayl0r commented on GitHub (Feb 16, 2017):

Some more info, and a way to see this.

I made a tcl script to kick out raw CC data slowly to show the issue.
send_raw_cc_slow.txt

Rename it to send_raw_cc_slow.tcl and make it executable.

If one then pipes the output into ccextractor, nothing happens. This is ccextractor buffering up the data before it does anything. You can ctrl-c to give up.

[ctaylor@czlnx7 ~]$ ./send_raw_cc_slow.tcl | ccextractor/linux/ccextractor -stdout -stdin -in=raw -s -nobi -ff -out=txt
CCExtractor 0.85, Carlos Fernandez Sanz, Volker Quetschke.
Teletext portions taken from Petr Kutalek's telxcc
--------------------------------------------------------------------------
Input: stdin
[Extract: 1] [Stream mode: McPoodle's raw]
[Program : Auto ] [Hauppage mode: No] [Use MythTV code: Auto]
[Timing mode: Auto] [Debug: No] [Buffer input: No]
[Use pic_order_cnt_lsb for H.264: No] [Print CC decoder traces: No]
[Target format: .txt] [Encoding: UTF-8] [Delay: 0] [Trim lines: No]
[Add font color data: Yes] [Add font typesetting: Yes]
[Convert case: No] [Video-edit join: No]
[Extraction start time: not set (from start)]
[Extraction end time: not set (to end)]
[Live stream: Yes, no timeout] [Clock frequency: 90000]
[Teletext page: Autodetect]
[Start credits text: None]

-----------------------------------------------------------------
Reading from standard input
Analyzing data in McPoodle raw mode
Sending captions to stdout.
^C

If you add the '-buff' option to my script, it sends lots of null CC characters after each line. After 5 seconds or so, when lots of data is buffered in, ccextractor wakes up and processes the data; but then it stops; despite the '-s' live feature enabled.

[ctaylor@czlnx7 ~]$ ./send_raw_cc_slow.tcl -buff | ccextractor/linux/ccextractor -stdout -stdin -in=raw -s -nobi -ff -out=txt
CCExtractor 0.85, Carlos Fernandez Sanz, Volker Quetschke.
Teletext portions taken from Petr Kutalek's telxcc
--------------------------------------------------------------------------
Input: stdin
[Extract: 1] [Stream mode: McPoodle's raw]
[Program : Auto ] [Hauppage mode: No] [Use MythTV code: Auto]
[Timing mode: Auto] [Debug: No] [Buffer input: No]
[Use pic_order_cnt_lsb for H.264: No] [Print CC decoder traces: No]
[Target format: .txt] [Encoding: UTF-8] [Delay: 0] [Trim lines: No]
[Add font color data: Yes] [Add font typesetting: Yes]
[Convert case: No] [Video-edit join: No]
[Extraction start time: not set (from start)]
[Extraction end time: not set (to end)]
[Live stream: Yes, no timeout] [Clock frequency: 90000]
[Teletext page: Autodetect]
[Start credits text: None]

-----------------------------------------------------------------
Reading from standard input
Analyzing data in McPoodle raw mode
Sending captions to stdout.
abcd
efgh
ijkl
mnop
qrst


Total frames time:	  00:00:00:000  (0 frames at 29.97fps)

Min PTS:				-03:32:12:370
Max PTS:				-03:32:12:370
Length:				 00:00:00:000
Done, processing time = 5 seconds
Issues? Open a ticket here
https://github.com/CCExtractor/ccextractor/issues
error writing "stdout": broken pipe
    while executing
"puts -nonewline [binary format cccc 128 128 128 128]"
    ("while" body line 6)
    invoked from within
"while 1 {
  puts -nonewline [binary format cccccc [expr 97+4*$c%24] [expr 98+128+4*$c%24] [expr 99+4*$c%24] [expr 100+128+4*$c%24] [expr 0x14+128] [ex..."
    (file "./send_raw_cc_slow.tcl" line 9)

Just for reference, if you run this script into a file (ctrl-c after a while), and then 'cat' that into ccextractor (i.e. a non-live scenario) , ccextractor processes the data nicely.

[ctaylor@czlnx7 ~]$ ./send_raw_cc_slow.tcl > tst.raw
^C
[ctaylor@czlnx7 ~]$ cat tst.raw | ccextractor/linux/ccextractor -stdout -stdin -in=raw -s -nobi -ff -out=txt
CCExtractor 0.85, Carlos Fernandez Sanz, Volker Quetschke.
Teletext portions taken from Petr Kutalek's telxcc
--------------------------------------------------------------------------
Input: stdin
[Extract: 1] [Stream mode: McPoodle's raw]
[Program : Auto ] [Hauppage mode: No] [Use MythTV code: Auto]
[Timing mode: Auto] [Debug: No] [Buffer input: No]
[Use pic_order_cnt_lsb for H.264: No] [Print CC decoder traces: No]
[Target format: .txt] [Encoding: UTF-8] [Delay: 0] [Trim lines: No]
[Add font color data: Yes] [Add font typesetting: Yes]
[Convert case: No] [Video-edit join: No]
[Extraction start time: not set (from start)]
[Extraction end time: not set (to end)]
[Live stream: Yes, no timeout] [Clock frequency: 90000]
[Teletext page: Autodetect]
[Start credits text: None]

-----------------------------------------------------------------
Reading from standard input
Analyzing data in McPoodle raw mode
Sending captions to stdout.
abcd
efgh
ijkl
mnop
qrst
uvwx
abcd
efgh
ijkl
mnop
qrst
uvwx
abcd
efgh
ijkl
mnop
qrst


Total frames time:	  00:00:00:000  (0 frames at 29.97fps)

Min PTS:				00:00:00:001
Max PTS:				00:00:01:802
Length:				 00:00:01:801
Done, processing time = 1 seconds
Issues? Open a ticket here
https://github.com/CCExtractor/ccextractor/issues
uvwx
[ctaylor@czlnx7 ~]$
@CTayl0r commented on GitHub (Feb 16, 2017): Some more info, and a way to see this. I made a tcl script to kick out raw CC data slowly to show the issue. [send_raw_cc_slow.txt](https://github.com/CCExtractor/ccextractor/files/781063/send_raw_cc_slow.txt) Rename it to send_raw_cc_slow.tcl and make it executable. If one then pipes the output into ccextractor, nothing happens. This is ccextractor buffering up the data before it does anything. You can ctrl-c to give up. ``` [ctaylor@czlnx7 ~]$ ./send_raw_cc_slow.tcl | ccextractor/linux/ccextractor -stdout -stdin -in=raw -s -nobi -ff -out=txt CCExtractor 0.85, Carlos Fernandez Sanz, Volker Quetschke. Teletext portions taken from Petr Kutalek's telxcc -------------------------------------------------------------------------- Input: stdin [Extract: 1] [Stream mode: McPoodle's raw] [Program : Auto ] [Hauppage mode: No] [Use MythTV code: Auto] [Timing mode: Auto] [Debug: No] [Buffer input: No] [Use pic_order_cnt_lsb for H.264: No] [Print CC decoder traces: No] [Target format: .txt] [Encoding: UTF-8] [Delay: 0] [Trim lines: No] [Add font color data: Yes] [Add font typesetting: Yes] [Convert case: No] [Video-edit join: No] [Extraction start time: not set (from start)] [Extraction end time: not set (to end)] [Live stream: Yes, no timeout] [Clock frequency: 90000] [Teletext page: Autodetect] [Start credits text: None] ----------------------------------------------------------------- Reading from standard input Analyzing data in McPoodle raw mode Sending captions to stdout. ^C ``` If you add the '-buff' option to my script, it sends lots of null CC characters after each line. After 5 seconds or so, when lots of data is buffered in, ccextractor wakes up and processes the data; but then it stops; despite the '-s' live feature enabled. ``` [ctaylor@czlnx7 ~]$ ./send_raw_cc_slow.tcl -buff | ccextractor/linux/ccextractor -stdout -stdin -in=raw -s -nobi -ff -out=txt CCExtractor 0.85, Carlos Fernandez Sanz, Volker Quetschke. Teletext portions taken from Petr Kutalek's telxcc -------------------------------------------------------------------------- Input: stdin [Extract: 1] [Stream mode: McPoodle's raw] [Program : Auto ] [Hauppage mode: No] [Use MythTV code: Auto] [Timing mode: Auto] [Debug: No] [Buffer input: No] [Use pic_order_cnt_lsb for H.264: No] [Print CC decoder traces: No] [Target format: .txt] [Encoding: UTF-8] [Delay: 0] [Trim lines: No] [Add font color data: Yes] [Add font typesetting: Yes] [Convert case: No] [Video-edit join: No] [Extraction start time: not set (from start)] [Extraction end time: not set (to end)] [Live stream: Yes, no timeout] [Clock frequency: 90000] [Teletext page: Autodetect] [Start credits text: None] ----------------------------------------------------------------- Reading from standard input Analyzing data in McPoodle raw mode Sending captions to stdout. abcd efgh ijkl mnop qrst Total frames time: 00:00:00:000 (0 frames at 29.97fps) Min PTS: -03:32:12:370 Max PTS: -03:32:12:370 Length: 00:00:00:000 Done, processing time = 5 seconds Issues? Open a ticket here https://github.com/CCExtractor/ccextractor/issues error writing "stdout": broken pipe while executing "puts -nonewline [binary format cccc 128 128 128 128]" ("while" body line 6) invoked from within "while 1 { puts -nonewline [binary format cccccc [expr 97+4*$c%24] [expr 98+128+4*$c%24] [expr 99+4*$c%24] [expr 100+128+4*$c%24] [expr 0x14+128] [ex..." (file "./send_raw_cc_slow.tcl" line 9) ``` Just for reference, if you run this script into a file (ctrl-c after a while), and then 'cat' that into ccextractor (i.e. a non-live scenario) , ccextractor processes the data nicely. ``` [ctaylor@czlnx7 ~]$ ./send_raw_cc_slow.tcl > tst.raw ^C [ctaylor@czlnx7 ~]$ cat tst.raw | ccextractor/linux/ccextractor -stdout -stdin -in=raw -s -nobi -ff -out=txt CCExtractor 0.85, Carlos Fernandez Sanz, Volker Quetschke. Teletext portions taken from Petr Kutalek's telxcc -------------------------------------------------------------------------- Input: stdin [Extract: 1] [Stream mode: McPoodle's raw] [Program : Auto ] [Hauppage mode: No] [Use MythTV code: Auto] [Timing mode: Auto] [Debug: No] [Buffer input: No] [Use pic_order_cnt_lsb for H.264: No] [Print CC decoder traces: No] [Target format: .txt] [Encoding: UTF-8] [Delay: 0] [Trim lines: No] [Add font color data: Yes] [Add font typesetting: Yes] [Convert case: No] [Video-edit join: No] [Extraction start time: not set (from start)] [Extraction end time: not set (to end)] [Live stream: Yes, no timeout] [Clock frequency: 90000] [Teletext page: Autodetect] [Start credits text: None] ----------------------------------------------------------------- Reading from standard input Analyzing data in McPoodle raw mode Sending captions to stdout. abcd efgh ijkl mnop qrst uvwx abcd efgh ijkl mnop qrst uvwx abcd efgh ijkl mnop qrst Total frames time: 00:00:00:000 (0 frames at 29.97fps) Min PTS: 00:00:00:001 Max PTS: 00:00:01:802 Length: 00:00:01:801 Done, processing time = 1 seconds Issues? Open a ticket here https://github.com/CCExtractor/ccextractor/issues uvwx [ctaylor@czlnx7 ~]$ ```
claunia commented 2026-01-29 16:39:40 +00:00
Author
Owner
Copy Link

@cfsmp3 commented on GitHub (Feb 22, 2017):

GSoC qualification: 2 points

@cfsmp3 commented on GitHub (Feb 22, 2017): GSoC qualification: 2 points
claunia commented 2026-01-29 16:39:41 +00:00
Author
Owner
Copy Link

@cfsmp3 commented on GitHub (Jan 25, 2020):

@CTayl0r Is this still a problem in the current CCExtractor?

@cfsmp3 commented on GitHub (Jan 25, 2020): @CTayl0r Is this still a problem in the current CCExtractor?
claunia commented 2026-01-29 16:39:41 +00:00
Author
Owner
Copy Link

@cfsmp3 commented on GitHub (Mar 28, 2021):

Closing as reporting user doesn't reply.

@cfsmp3 commented on GitHub (Mar 28, 2021): Closing as reporting user doesn't reply.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
bug
CEA-708
difficulty: easy
difficulty: hard
difficulty: medium
DTMB
DVB
enhancement
enhancement
Flutter GUI
GCI19
good-first-task
GSoC 2021
GSOC-2023
GSOC-2023
GSOC-2023
GSoC-related
Hacktoberfest
HardsubX
help wanted
invalid
JokerTV
Mac / OSX
more-info-needed
needs-commercial-sponsor
needs-commercial-sponsor
needs-commercial-sponsor
needs-confirmation-of-being-broken
OCR
pull-request
Mirrored from GitHub Pull Request
 Python
regression
Rust
teletext
No Label difficulty: easy needs-confirmation-of-being-broken
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claunia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/ccextractor#275
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?