Merge branch 'pr/n411_Abhinav95'

Skip another to the SCTE 57 2003 sections in PMT

.gitignore

@@ -9,8 +9,9 @@ CVS
 *.o
 linux/ccextractor
 linux/depend
-windows/debug/**
-windows/release/**
+windows/Debug/**
+windows/Release/**
+build/
 
 ####
 # Visual Studio project Ignored files
@@ -19,3 +20,14 @@ windows/release/**
 *.sdf
 *.opensdf
 *.user
+*.opendb
+
+####
+# Ignore the header file that is updated upon build
+src/lib_ccx/compile_info_real.h
+
+#### Ignore windows OCR libraries and folders
+windows/libs/leptonica/**
+windows/libs/tesseract/**
+windows/Release-OCR/**
+windows/Debug-OCR/**

README.md

@@ -4,9 +4,9 @@ ccextractor
 Carlos' version (mainstream) is the most stable branch.
 
 Extracting subtitles has never been so easy. Just type the following command:
-ccextrator "name of input"
+ccextractor "name of input"
 
-Gui lovers should download the Sorceforge version of CCExtractor, the Git Version is not your cup of tea.
+GUI lovers should download the Sourceforge version of CCExtractor, the Git Version is not your cup of tea.
 http://ccextractor.sourceforge.net/download-ccextractor.html
 
 For News about release, please find CHANGES.TXT

docs/CHANGES.TXT

@@ -1,3 +1,62 @@
+0.82(2016-08-15)
+-----------------
+- New: HardsubX - Burned in subtitle extraction subsystem.
+- New: Color Detection in DVB Subtitles
+- Fix: Corrected sentence capitalization
+- Fix: Skipping redundant bytes at the end of tx3g atom in MP4
+- Fix: Illegal SRT files being created from DVB subtitles
+- Fix: Incorrect Progress Display
+
+0.81 (2016-06-13)
+-----------------
+- New: --version parameter for extensive version information (version number, compile date, executable hash, git commit (if appropriate))
+- New: Add -sem (semaphore) to create a .sem file when an output file is open and delete it when it's closed.
+- New: Add --append parameter. This will prevent overwriting of existing files.
+- New: File Rotation support added. The user has to send a USR1 signal to rotate.
+- Fix: Issues with files <1 Mb
+- Fix: Preview of generated transcript.
+- Fix: Statistics were not generated anymore.
+- Fix: Correcting display of sub mode and info in transcripts.
+- Fix: Teletext page number displayed in -UCLA.
+- Fix: Removal of excessive XDS notices about aspect ratio info.
+- Fix: Force Flushing of file buffers works for all files now.
+- Fix: mp4 void atoms that was causing some .mp4 files to fail.
+- Fix: Memory usage caused by EPG processing was high due to many non-dynamic buffers.
+- Fix: Project files for Visual Studio now include OCR support in Windows.
+
+0.80 (2016-04-24)
+-----------------
+- Fix: "Premature end of file" (one of the scenarios)
+- Fix: XDS data is always parsed again (needed to extract information such as program name)
+- Fix: Teletext parsing: @ was incorrectly exported as * - X/26 packet specifications in ETS 300 706 v1.2.1 now better followed
+- Fix: Teletext parsing: Latin G2 subsets and accented characters not displaying properly
+- Fix: Timing in -ucla
+- Fix: Timing in ISDB (some instances)
+- Fix: "mfra" mp4 box weight changed to 1 (this helps with correct file format detection)
+- Fix: Fix for TARGET File is null. 
+- Fix: Fixed SegFaults while parsing parameters (if mandatory parameter is not present in -outinterval, -codec or -nocodec)
+- Fix: Crash when input small is too small
+- Fix: Update some URLs in code (references to docs)
+- Fix: -delay now updates final timestamp in ISDB, too
+- Fix: Removed minor compiler warnings
+- Fix: Visual Studio solution files working again
+- Fix: ffmpeg integration working again
+- New: Added --forceflush (-ff). If used, output file descriptors will be flushed immediately after being written to
+- New: Hexdump XDS packets that we cannot parse (shouldn't be many of those anyway)
+- New: If input file cannot be open, provide a decent human readable explanation
+- New: GXF support
+
+0.79 (2016-01-09)
+-----------------
+- Support for Grid Format (g608)
+- Show Correct number of teletext packet processed
+- Removed Segfault on incorrect mp4 detection
+- Remove xml header from transcript format
+- Help message updated for Teletext
+- Added --help and -h for help message
+- Added --nohtmlescape option
+- Added --noscte20 option
+
 0.78 (2015-12-12)
 -----------------
 - Support to extract Closed Caption from MultiProgram at once.

docs/G608.TXT

@@ -0,0 +1,58 @@
+G608
+====
+G608 (for grid 608) is generated by CCExtractor by using -out=g608.
+
+This is a verbose format that exports the contents of the 608 grid verbatim
+so there's no loss of positioning or colors due the limitations or complexy
+or other output formats.
+
+G608 is a text file with a structure based on .srt and looks like this:
+
+1
+00:00:02,019 --> 00:00:03,585
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+                                99999999999999999999999999999999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+  - Previously on The Tudors... 90000000000000000000000000000009RIIIIIIIIIIIIIIIIRRRRRRRRRRRRRRR
+  - Your Holy Father offs you   99000000000000000000000000000999RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
+
+For each subtitle frame there's exactly 15 rows (which represent the 15 rows of the 608 screen)
+of 96 characters each.
+
+Each row is divided in 3 blocks: 32 characters for the text, 32 characters for the color, and 
+32 characters for the font. 
+
+The possible color values are:
+	COL_WHITE = 0,
+	COL_GREEN = 1,
+	COL_BLUE = 2,
+	COL_CYAN = 3,
+	COL_RED = 4,
+	COL_YELLOW = 5,
+	COL_MAGENTA = 6,
+	COL_USERDEFINED = 7,
+	COL_BLACK = 8,
+	COL_TRANSPARENT = 9
+	
+And the possible font values are:
+    R => Regular
+    I => Italic
+    U => Underlined
+    B => Underlined + italic
+
+If a 'E' is found in ether color or font that means a bug in CCExtractor. Should you ever get
+an E please send us a .bin file that causes it.
+	
+This format is intended for post processing tools that need to represent the output of a 608
+decoder accurately but that don't want to deal with the madness of other more generic subtitle
+formats.

docs/HARDSUBX.txt

@@ -0,0 +1,45 @@
+
+Overview
+========
+Subtitles which are burned into the video (or hard subbed) can be extracted using the -hardsubx flag.
+The system works by processing video frames and extracting only the subtitles from them, followed
+by an OCR recognition using Tesseract.
+
+Dependencies
+============
+Tesseract (OCR library by Google)
+Leptonica (C Image processing library)
+FFMpeg (Video Processing Library)
+
+Compilation
+===========
+
+Linux
+-----
+
+Make sure Tesseract, Leptonica and FFMPeg are installed, and that their libraries can be found using pkg-config.
+Refer to OCR.txt for installation details.
+
+To install FFmpeg (libav), follow the steps at:-
+https://trac.ffmpeg.org/wiki/CompilationGuide/Ubuntu - For Ubuntu, Debian and Linux Mint
+https://trac.ffmpeg.org/wiki/CompilationGuide/Generic - For generic Linux compilation
+
+To validate your FFMpeg installation, make sure you can run the following commands on your terminal:-
+pkg-config --cflags libavcodec
+pkg-config --cflags libavformat
+pkg-config --cflags libavutil
+pkg-config --cflags libswscale
+pkg-config --libs libavcodec
+pkg-config --libs libavformat
+pkg-config --libs libavutil
+pkg-config --libs libswscale
+
+On success, you should see the correct include directory path and the linker flags.
+
+To build the program with hardsubx support, from the linux directory run:-
+make ENABLE_HARDSUBX=yes
+
+Windows
+-------
+
+Coming Soon

docs/README.TXT

@@ -1,4 +1,4 @@
-ccextractor, 0.78
+ccextractor, 0.81
 -----------------
 Authors: Carlos Fernández (cfsmp3), Volker Quetschke.
 Maintainer: cfsmp3
@@ -14,12 +14,12 @@ You can subscribe to new releases notifications at freshmeat:
 http://freshmeat.net/projects/ccextractor
 
 Google Summer of Code 2014 students
-- Willem van iseghem
+- Willem Van Iseghem
 - Ruslan KuchumoV
 - Anshul Maheshwari
 
 Google Summer of Code 2015 students
-- Willem van iseghem
+- Willem Van Iseghem
 - Ruslan Kuchumov
 - Anshul Maheshwari
 - Nurendra Choudhary

docs/gpl-2.0.txt

@@ -0,0 +1,339 @@
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The licenses for most software are designed to take away your
+freedom to share and change it.  By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users.  This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it.  (Some other Free Software Foundation software is covered by
+the GNU Lesser General Public License instead.)  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+  To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have.  You must make sure that they, too, receive or can get the
+source code.  And you must show them these terms so they know their
+rights.
+
+  We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+  Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software.  If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+  Finally, any free program is threatened constantly by software
+patents.  We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary.  To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                    GNU GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License.  The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language.  (Hereinafter, translation is included without limitation in
+the term "modification".)  Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+  1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+  2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) You must cause the modified files to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    b) You must cause any work that you distribute or publish, that in
+    whole or in part contains or is derived from the Program or any
+    part thereof, to be licensed as a whole at no charge to all third
+    parties under the terms of this License.
+
+    c) If the modified program normally reads commands interactively
+    when run, you must cause it, when started running for such
+    interactive use in the most ordinary way, to print or display an
+    announcement including an appropriate copyright notice and a
+    notice that there is no warranty (or else, saying that you provide
+    a warranty) and that users may redistribute the program under
+    these conditions, and telling the user how to view a copy of this
+    License.  (Exception: if the Program itself is interactive but
+    does not normally print such an announcement, your work based on
+    the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+    a) Accompany it with the complete corresponding machine-readable
+    source code, which must be distributed under the terms of Sections
+    1 and 2 above on a medium customarily used for software interchange; or,
+
+    b) Accompany it with a written offer, valid for at least three
+    years, to give any third party, for a charge no more than your
+    cost of physically performing source distribution, a complete
+    machine-readable copy of the corresponding source code, to be
+    distributed under the terms of Sections 1 and 2 above on a medium
+    customarily used for software interchange; or,
+
+    c) Accompany it with the information you received as to the offer
+    to distribute corresponding source code.  (This alternative is
+    allowed only for noncommercial distribution and only if you
+    received the program in object code or executable form with such
+    an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it.  For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable.  However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License.  Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+  5. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Program or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+  6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+  7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded.  In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+  9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation.  If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+  10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission.  For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this.  Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+                            NO WARRANTY
+
+  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 2 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License along
+    with this program; if not, write to the Free Software Foundation, Inc.,
+    51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+    Gnomovision version 69, Copyright (C) year name of author
+    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary.  Here is a sample; alter the names:
+
+  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+  `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+  <signature of Ty Coon>, 1 April 1989
+  Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs.  If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.

linux/Makefile

@@ -2,9 +2,10 @@ SHELL = /bin/sh
 
 CC = gcc
 SYS := $(shell gcc -dumpmachine)
-CFLAGS = -O3 -std=gnu99
-INCLUDE = -I../src/gpacmp4/ -I../src/libpng -I../src/zlib -I../src/lib_ccx -I../src/.
-ALL_FLAGS = -Wno-write-strings -D_FILE_OFFSET_BITS=64
+CFLAGS = -O3 -std=gnu99 -s
+INCLUDE = -I../src/gpacmp4/ -I../src/libpng -I../src/lib_hash -I../src/zlib -I../src/lib_ccx -I../src/.
+INCLUDE += -I../src/zvbi
+ALL_FLAGS = -Wno-write-strings -D_FILE_OFFSET_BITS=64 -DVERSION_FILE_PRESENT
 LDFLAGS = -lm
 
 ifneq (, $(findstring linux, $(SYS)))
@@ -13,7 +14,7 @@ endif
 TARGET = ccextractor
 
 OBJS_DIR = objs
-VPATH = ../src:../src/gpacmp4:../src/libpng:../src/zlib:../src/lib_ccx
+VPATH = ../src:../src/gpacmp4:../src/libpng:../src/zlib:../src/lib_ccx:../src/zvbi:../src/lib_hash
 
 SRCS_DIR = ../src
 SRCS_C = $(wildcard $(SRCS_DIR)/*.c)
@@ -27,6 +28,10 @@ SRCS_PNG_DIR = $(SRCS_DIR)/libpng
 SRCS_PNG = $(wildcard $(SRCS_PNG_DIR)/*.c)
 OBJS_PNG = $(SRCS_PNG:$(SRCS_PNG_DIR)/%.c=$(OBJS_DIR)/%.o)
 
+SRCS_ZVBI_DIR = $(SRCS_DIR)/zvbi
+SRCS_ZVBI = $(wildcard $(SRCS_ZVBI_DIR)/*.c)
+OBJS_ZVBI = $(SRCS_ZVBI:$(SRCS_ZVBI_DIR)/%.c=$(OBJS_DIR)/%.o)
+
 SRCS_GPACMP4_DIR = $(SRCS_DIR)/gpacmp4
 SRCS_GPACMP4_C = $(wildcard $(SRCS_GPACMP4_DIR)/*.c)
 SRCS_GPACMP4_CPP = $(wildcard $(SRCS_GPACMP4_DIR)/*.cpp)
@@ -37,10 +42,33 @@ SRCS_ZLIB_DIR = $(SRCS_DIR)/zlib
 SRCS_ZLIB = $(wildcard $(SRCS_ZLIB_DIR)/*.c)
 OBJS_ZLIB = $(SRCS_ZLIB:$(SRCS_ZLIB_DIR)/%.c=$(OBJS_DIR)/%.o)
 
+SRCS_HASH_DIR = $(SRCS_DIR)/lib_hash
+SRCS_HASH = $(wildcard $(SRCS_HASH_DIR)/*.c)
+OBJS_HASH = $(SRCS_HASH:$(SRCS_HASH_DIR)/%.c=$(OBJS_DIR)/%.o)
+
 INSTLALL = cp -f -p
 INSTLALL_PROGRAM = $(INSTLALL)
 DESTDIR = /usr/bin
 
+ifeq ($(ENABLE_HARDSUBX),yes)
+ENABLE_OCR=yes
+CFLAGS+=-DENABLE_HARDSUBX
+CFLAGS+= $(shell pkg-config --cflags libavcodec)
+CFLAGS+= $(shell pkg-config --cflags libavformat)
+CFLAGS+= $(shell pkg-config --cflags libavutil)
+CFLAGS+= $(shell pkg-config --cflags libswscale)
+AV_LDFLAGS+= $(shell pkg-config --libs libavcodec )
+AV_LDFLAGS+= $(shell pkg-config --libs libavformat )
+AV_LDFLAGS+= $(shell pkg-config --libs libavutil )
+AV_LDFLAGS+= $(shell pkg-config --libs libswscale )
+ifeq ($(AV_LDFLAGS),$(EMPTY))
+$(error **ERROR** "libav not found")
+else
+$(info "libav found")
+endif
+LDFLAGS+= $(AV_LDFLAGS)
+endif
+
 ifeq ($(ENABLE_OCR),yes)
 CFLAGS+=-DENABLE_OCR -DPNG_NO_CONFIG_H
 TESS_LDFLAGS+= $(shell pkg-config --libs tesseract)
@@ -78,14 +106,14 @@ LDFLAGS+= $(shell pkg-config --libs libavutil )
 endif
 
 .PHONY: all
-all: objs_dir $(TARGET) 
+all: pre-build objs_dir $(TARGET) 
 
 .PHONY: objs_dir
 objs_dir: 
 	mkdir -p $(OBJS_DIR)
 
-$(TARGET): $(OBJS) $(OBJS_PNG) $(OBJS_GPACMP4) $(OBJS_ZLIB) $(OBJS_CCX)
-	$(CC) $(ALL_FLAGS) $(CFLAGS) $(OBJS) $(OBJS_CCX) $(OBJS_PNG) $(OBJS_GPACMP4) $(OBJS_ZLIB) $(LDFLAGS) -o $@
+$(TARGET): $(OBJS) $(OBJS_PNG) $(OBJS_GPACMP4) $(OBJS_ZVBI) $(OBJS_ZLIB) $(OBJS_HASH) $(OBJS_CCX)
+	$(CC) $(ALL_FLAGS) $(CFLAGS) $(OBJS) $(OBJS_CCX) $(OBJS_PNG) $(OBJS_ZVBI) $(OBJS_GPACMP4) $(OBJS_ZLIB) $(OBJS_HASH) $(LDFLAGS) -o $@
 
 $(OBJS_DIR)/%.o: %.c
 	$(CC) -c $(ALL_FLAGS) $(INCLUDE) $(CFLAGS) $< -o $@  
@@ -99,7 +127,7 @@ $(OBJS_DIR)/ccextractor.o: ccextractor.c
 .PHONY: clean
 clean:
 	rm -rf $(TARGET) 2>/dev/null || true
-	rm -rf $(OBJS_CCX) $(OBJS_PNG) $(OBJS_ZLIB) $(OBJS_GPACMP4) $(OBJS) 2>/dev/null || true
+	rm -rf $(OBJS_CCX) $(OBJS_PNG) $(OBJS_ZLIB) $(OBJS_GPACMP4) $(OBJS_HASH) $(OBJS) 2>/dev/null || true
 	rm -rdf $(OBJS_DIR) 2>/dev/null || true
 	rm -rf .depend 2>/dev/null || true
 
@@ -113,8 +141,12 @@ uninstall:
 
 .PHONY: depend dep
 depend dep:
-	$(CC) $(CFLAGS) $(INCLUDE) -E -MM $(SRCS_C) $(SRCS_PNG) $(SRCS_ZLIB) $(SRCS_CCX) \
+	$(CC) $(CFLAGS) $(INCLUDE) -E -MM $(SRCS_C) $(SRCS_PNG) $(SRCS_ZVBI) $(SRCS_ZLIB) $(SRCS_HASH) $(SRCS_CCX) \
 		$(SRCS_GPACMP4_C) $(SRCS_GPACMP4_CPP) |\
 		sed 's/^[a-zA-Z_0-9]*.o/$(OBJS_DIR)\/&/' > .depend
 
+.PHONY: pre-build
+pre-build:
+	./pre-build.sh
+
 -include .depend

linux/build

@@ -1,11 +1,14 @@
 #!/bin/bash
-BLD_FLAGS="-std=gnu99 -Wno-write-strings -DGPAC_CONFIG_LINUX -D_FILE_OFFSET_BITS=64"
-BLD_INCLUDE="-I../src/lib_ccx/ -I../src/gpacmp4/ -I../src/libpng/ -I../src/zlib/"
+BLD_FLAGS="-g -std=gnu99 -Wno-write-strings -DGPAC_CONFIG_LINUX -D_FILE_OFFSET_BITS=64 -DVERSION_FILE_PRESENT -DENABLE_OCR"
+BLD_INCLUDE="-I /usr/include/leptonica/ -I /usr/include/tesseract/ -I../src/lib_ccx/ -I../src/gpacmp4/ -I../src/libpng/ -I../src/zlib/ -I../src/zvbi -I../src/lib_hash"
 SRC_LIBPNG="$(find ../src/libpng/ -name '*.c')"
 SRC_ZLIB="$(find ../src/zlib/ -name '*.c')"
+SRC_ZVBI="$(find ../src/zvbi/ -name '*.c')"
 SRC_CCX="$(find ../src/lib_ccx/ -name '*.c')"
 SRC_GPAC="$(find ../src/gpacmp4/ -name '*.c')"
-BLD_SOURCES="../src/ccextractor.c $SRC_CCX $SRC_GPAC $SRC_ZLIB $SRC_LIBPNG"
-BLD_LINKER="-lm -zmuldefs"
+SRC_HASH="$(find ../src/lib_hash/ -name '*.c')"
+BLD_SOURCES="../src/ccextractor.c $SRC_CCX $SRC_GPAC $SRC_ZLIB $SRC_ZVBI $SRC_LIBPNG $SRC_HASH"
+BLD_LINKER="-lm -zmuldefs -l tesseract -l lept"
 
-gcc $BLD_FLAGS $BLD_INCLUDE -o ccextractor $BLD_SOURCES $BLD_LINKER
+./pre-build.sh
+gcc $BLD_FLAGS $BLD_INCLUDE -o ccextractor $BLD_SOURCES $BLD_LINKER

linux/builddebug

@@ -1,11 +1,14 @@
 #!/bin/bash
-BLD_FLAGS="-g -std=gnu99 -Wno-write-strings -DGPAC_CONFIG_LINUX -D_FILE_OFFSET_BITS=64"
-BLD_INCLUDE="-I../src/lib_ccx/ -I../src/gpacmp4/ -I../src/libpng/ -I../src/zlib/"
+BLD_FLAGS="-g -std=gnu99 -Wno-write-strings -DGPAC_CONFIG_LINUX -D_FILE_OFFSET_BITS=64 -DVERSION_FILE_PRESENT"
+BLD_INCLUDE="-I../src/lib_ccx/ -I../src/gpacmp4/ -I../src/libpng/ -I../src/zlib/ -I../src/zvbi -I../src/lib_hash"
 SRC_LIBPNG="$(find ../src/libpng/ -name '*.c')"
 SRC_ZLIB="$(find ../src/zlib/ -name '*.c')"
+SRC_ZVBI="$(find ../src/zvbi/ -name '*.c')"
 SRC_CCX="$(find ../src/lib_ccx/ -name '*.c')"
 SRC_GPAC="$(find ../src/gpacmp4/ -name '*.c')"
-BLD_SOURCES="../src/ccextractor.c $SRC_CCX $SRC_GPAC $SRC_ZLIB $SRC_LIBPNG"
+SRC_HASH="$(find ../src/lib_hash/ -name '*.c')"
+BLD_SOURCES="../src/ccextractor.c $SRC_CCX $SRC_GPAC $SRC_ZLIB  $SRC_ZVBI $SRC_LIBPNG $SRC_HASH"
 BLD_LINKER="-lm -zmuldefs"
 
-gcc $BLD_FLAGS $BLD_INCLUDE -o ccextractor $BLD_SOURCES $BLD_LINKER
+./pre-build.sh
+gcc $BLD_FLAGS $BLD_INCLUDE -o ccextractor $BLD_SOURCES $BLD_LINKER

linux/pre-build.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+echo "Obtaining Git commit"
+commit=(`git rev-parse HEAD 2>/dev/null`)
+if [ -z "$commit" ]; then
+	echo "Git command not present, trying folder approach"
+	if [ -d "../.git" ]; then 
+		echo "Git folder found, using HEAD file"
+		head="`cat ../.git/HEAD`"
+		ref_pos=(`expr match "$head" 'ref: '`)
+		if [ $ref_pos -eq 5 ]; then	
+			echo "HEAD file contains a ref, following"
+			commit=(`cat "../.git/${head:5}"`)
+			echo "Extracted commit: $commit"
+		else
+			echo "HEAD contains a commit, using it"
+			commit="$head"
+			echo "Extracted commit: $commit"	
+		fi 
+	fi
+fi
+if [ -z "$commit" ]; then
+	commit="Unknown"
+fi
+builddate=`date +%Y-%m-%d`
+echo "Storing variables in file"
+echo "Commit: $commit"
+echo "Date: $builddate"
+echo "#ifndef CCX_CCEXTRACTOR_COMPILE_REAL_H" > ../src/lib_ccx/compile_info_real.h
+echo "#define CCX_CCEXTRACTOR_COMPILE_REAL_H" >> ../src/lib_ccx/compile_info_real.h
+echo "#define GIT_COMMIT \"$commit\"" >> ../src/lib_ccx/compile_info_real.h
+echo "#define COMPILE_DATE \"$builddate\"" >> ../src/lib_ccx/compile_info_real.h
+echo "#endif" >> ../src/lib_ccx/compile_info_real.h
+echo "Stored all in compile.h"
+echo "Done."

mac/build.command

@@ -1,2 +1,17 @@
-gcc -std=gnu99 -Wno-write-strings -DGPAC_CONFIG_DARWIN -D_FILE_OFFSET_BITS=64 -Dfopen64=fopen -Dopen64=open -Dlseek64=lseek -I ../src/gpacmp4 -I ../src/lib_ccx -I ../src/libpng -I ../src/zlib -o ccextractor $(find ../src/ -name '*.cpp' \! -name 'win_*') $(find ../src/ -name '*.c' \! -name 'win_*') -liconv
+#!/bin/bash
+cd `dirname $0`
+BLD_FLAGS="-std=gnu99 -Wno-write-strings -DGPAC_CONFIG_DARWIN -D_FILE_OFFSET_BITS=64 -DVERSION_FILE_PRESENT -Dfopen64=fopen -Dopen64=open -Dlseek64=lseek"
+BLD_INCLUDE="-I../src/lib_ccx -I../src/gpacmp4 -I../src/libpng -I../src/zlib -I../src/zvbi -I../src/lib_hash"
+SRC_LIBPNG="$(find ../src/libpng -name '*.c')"
+SRC_ZLIB="$(find ../src/zlib -name '*.c')"
+SRC_ZVBI="$(find ../src/zvbi -name '*.c')"
+SRC_CCX="$(find ../src/lib_ccx -name '*.c')"
+SRC_GPAC="$(find ../src/gpacmp4 -name '*.c')"
+SRC_LIB_HASH="$(find ../src/lib_hash -name '*.c')"
+BLD_SOURCES="../src/ccextractor.c $SRC_CCX $SRC_GPAC $SRC_ZVBI $SRC_ZLIB $SRC_LIBPNG $SRC_LIB_HASH"
+BLD_LINKER="-lm -liconv"
+
+./pre-build.sh
+gcc $BLD_FLAGS $BLD_INCLUDE -o ccextractor $BLD_SOURCES $BLD_LINKER
+
 

mac/pre-build.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+echo "Obtaining Git commit"
+commit=(`git rev-parse HEAD 2>/dev/null`)
+if [ -z "$commit" ]; then
+	echo "Git command not present, trying folder approach"
+	if [ -d "../.git" ]; then 
+		echo "Git folder found, using HEAD file"
+		head="`cat ../.git/HEAD`"
+		ref_pos=(`expr match "$head" 'ref: '`)
+		if [ $ref_pos -eq 5 ]; then	
+			echo "HEAD file contains a ref, following"
+			commit=(`cat "../.git/${head:5}"`)
+			echo "Extracted commit: $commit"
+		else
+			echo "HEAD contains a commit, using it"
+			commit="$head"
+			echo "Extracted commit: $commit"	
+		fi 
+	fi
+fi
+if [ -z "$commit" ]; then
+	commit="Unknown"
+fi
+builddate=`date +%Y-%m-%d`
+echo "Storing variables in file"
+echo "Commit: $commit"
+echo "Date: $builddate"
+echo "#ifndef CCX_CCEXTRACTOR_COMPILE_REAL_H" > ../src/lib_ccx/compile_info_real.h
+echo "#define CCX_CCEXTRACTOR_COMPILE_REAL_H" >> ../src/lib_ccx/compile_info_real.h
+echo "#define GIT_COMMIT \"$commit\"" >> ../src/lib_ccx/compile_info_real.h
+echo "#define COMPILE_DATE \"$builddate\"" >> ../src/lib_ccx/compile_info_real.h
+echo "#endif" >> ../src/lib_ccx/compile_info_real.h
+echo "Stored all in compile.h"
+echo "Done."

src/CMakeLists.txt

@@ -20,6 +20,8 @@ configure_file (
 include_directories ("${PROJECT_SOURCE_DIR}")
 include_directories ("${PROJECT_SOURCE_DIR}/lib_ccx")
 include_directories ("${PROJECT_SOURCE_DIR}/gpacmp4/")
+include_directories ("${PROJECT_SOURCE_DIR}/zvbi")
+aux_source_directory ("${PROJECT_SOURCE_DIR}/zvbi" SOURCEFILE)
 
 #Adding some platform specific library path
 link_directories (/opt/local/lib)
@@ -44,9 +46,15 @@ if(PKG_CONFIG_FOUND)
     set (EXTRA_LIBS ${EXTRA_LIBS} png)
   else (PNG_FOUND)
     include_directories ("${PROJECT_SOURCE_DIR}/libpng/")
+    include_directories ("${PROJEXT_SOURCE_DIR}/zlib/")
     aux_source_directory ("${PROJECT_SOURCE_DIR}/libpng/" SOURCEFILE)
     aux_source_directory ("${PROJECT_SOURCE_DIR}/zlib/" SOURCEFILE)
   endif(PNG_FOUND)
+else(PKG_CONFIG_FOUND)
+    include_directories ("${PROJECT_SOURCE_DIR}/libpng/")
+    include_directories ("${PROJEXT_SOURCE_DIR}/zlib/")
+    aux_source_directory ("${PROJECT_SOURCE_DIR}/libpng/" SOURCEFILE)
+    aux_source_directory ("${PROJECT_SOURCE_DIR}/zlib/" SOURCEFILE)
 endif (PKG_CONFIG_FOUND)
 
 ########################################################

src/ccextractor.c

@@ -8,9 +8,9 @@ License: GPL 2.0
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <signal.h>
-#include "ffmpeg_intgr.h"
 #include "ccx_common_option.h"
 #include "ccx_mp4.h"
+#include "hardsubx.h"
 
 struct lib_ccx_ctx *signal_ctx;
 void sigint_handler()
@@ -24,9 +24,6 @@ void sigint_handler()
 struct ccx_s_options ccx_options;
 int main(int argc, char *argv[])
 {
-#ifdef ENABLE_FFMPEG
-	void *ffmpeg_ctx = NULL;
-#endif
 	struct lib_ccx_ctx *ctx;
 	struct lib_cc_decode *dec_ctx = NULL;
 	int ret = 0;
@@ -41,10 +38,22 @@ int main(int argc, char *argv[])
 		usage ();
 		fatal (EXIT_NO_INPUT_FILES, "(This help screen was shown because there were no input files)\n");
 	}
+	else if (ret == EXIT_WITH_HELP)
+	{
+		return EXIT_OK;
+	}
 	else if (ret != EXIT_OK)
 	{
 		exit(ret);
 	}
+#ifdef ENABLE_HARDSUBX
+	if(ccx_options.hardsubx)
+	{
+		// Perform burned in subtitle extraction
+		hardsubx(&ccx_options);
+		return 0;
+	}
+#endif
 	// Initialize libraries
 	ctx = init_libraries(&ccx_options);
 	if (!ctx && errno == ENOMEM)
@@ -84,59 +93,32 @@ int main(int argc, char *argv[])
 	if (ccx_options.binary_concat)
 	{
 		ctx->total_inputsize=gettotalfilessize(ctx);
-		if (ctx->total_inputsize==-1)
-			fatal (EXIT_UNABLE_TO_DETERMINE_FILE_SIZE, "Failed to determine total file size.\n");
+		if (ctx->total_inputsize < 0)
+		{
+			switch (ctx->total_inputsize)
+			{
+			case -1*ENOENT:
+				fatal(EXIT_NO_INPUT_FILES, "Failed to open file: File not Exist");
+			case -1*EACCES:
+				fatal(EXIT_READ_ERROR, "Failed to open file: Unable to access");
+			case -1*EINVAL:
+				fatal(EXIT_READ_ERROR, "Failed to open file: Invalid opening flag.");
+			case -1*EMFILE:
+				fatal(EXIT_TOO_MANY_INPUT_FILES, "Failed to open file: Too many files are open.");
+			default:
+				fatal(EXIT_READ_ERROR, "Failed to open file: Reason unknown");
+			}
+		}
 	}
 
 #ifndef _WIN32
 	signal_ctx = ctx;
 	m_signal(SIGINT, sigint_handler);
+	create_signal();
 #endif
-
 	while (switch_to_next_file(ctx, 0))
 	{
 		prepare_for_new_file(ctx);
-#ifdef ENABLE_FFMPEG
-		close_input_file(ctx);
-		ffmpeg_ctx =  init_ffmpeg(ctx->inputfile[0]);
-		if(ffmpeg_ctx)
-		{
-			do
-			{
-				int ret = 0;
-				unsigned char *bptr = ctx->buffer;
-				int len = ff_get_ccframe(ffmpeg_ctx, bptr, 1024);
-                                int cc_count = 0;
-				if(len == AVERROR(EAGAIN))
-				{
-					continue;
-				}
-				else if(len == AVERROR_EOF)
-					break;
-				else if(len == 0)
-					continue;
-				else if(len < 0 )
-				{
-					mprint("Error extracting Frame\n");
-					break;
-
-				}
-                                else
-                                    cc_count = len/3;
-				ret = process_cc_data(dec_ctx, bptr, cc_count, &dec_ctx->dec_sub);
-				if(ret >= 0 && dec_ctx->dec_sub.got_output)
-				{
-					encode_sub(enc_ctx, &dec_ctx->dec_sub);
-					dec_ctx->dec_sub.got_output = 0;
-				}
-			}while(1);
-			continue;
-		}
-		else
-		{
-			mprint ("\rFailed to initialized ffmpeg falling back to legacy\n");
-		}
-#endif
 		stream_mode = ctx->demux_ctx->get_stream_mode(ctx->demux_ctx);
 		// Disable sync check for raw formats - they have the right timeline.
 		// Also true for bin formats, but -nosync might have created a
@@ -169,6 +151,10 @@ int main(int argc, char *argv[])
 			case CCX_SM_PROGRAM:
 			case CCX_SM_ASF:
 			case CCX_SM_WTV:
+			case CCX_SM_GXF:
+#ifdef ENABLE_FFMPEG
+			case CCX_SM_FFMPEG:
+#endif
 				if (!ccx_options.use_gop_as_pts) // If !0 then the user selected something
 					ccx_options.use_gop_as_pts = 0; 
 				mprint ("\rAnalyzing data in general mode\n");
@@ -190,7 +176,7 @@ int main(int argc, char *argv[])
 			case CCX_SM_MP4:
 				mprint ("\rAnalyzing data with GPAC (MP4 library)\n");
 				close_input_file(ctx); // No need to have it open. GPAC will do it for us
-				processmp4 (ctx, &ctx->mp4_cfg, ctx->inputfile[0]);
+				processmp4(ctx, &ctx->mp4_cfg, ctx->inputfile[ctx->current_file]);
 				if (ccx_options.print_file_reports)
 					print_file_report(ctx);
 				break;
@@ -205,68 +191,6 @@ int main(int argc, char *argv[])
 				break;
 		}
 
-#if 0
-		if (ctx->total_pulldownframes)
-			mprint ("incl. pulldown frames:  %s  (%u frames at %.2ffps)\n",
-					print_mstime( (LLONG)(ctx->total_pulldownframes*1000/current_fps) ),
-					ctx->total_pulldownframes, current_fps);
-		if (pts_set >= 1 && min_pts != 0x01FFFFFFFFLL)
-		{
-			LLONG postsyncms = (LLONG) (ctx->frames_since_last_gop*1000/current_fps);
-			mprint ("\nMin PTS:				%s\n",
-					print_mstime( min_pts/(MPEG_CLOCK_FREQ/1000) - fts_offset));
-			if (pts_big_change)
-				mprint ("(Reference clock was reset at some point, Min PTS is approximated)\n");
-			mprint ("Max PTS:				%s\n",
-					print_mstime( sync_pts/(MPEG_CLOCK_FREQ/1000) + postsyncms));
-
-			mprint ("Length:				 %s\n",
-					print_mstime( sync_pts/(MPEG_CLOCK_FREQ/1000) + postsyncms
-								  - min_pts/(MPEG_CLOCK_FREQ/1000) + fts_offset ));
-		}
-		// dvr-ms files have invalid GOPs
-		if (gop_time.inited && first_gop_time.inited && stream_mode != CCX_SM_ASF)
-		{
-			mprint ("\nInitial GOP time:	   %s\n",
-				print_mstime(first_gop_time.ms));
-			mprint ("Final GOP time:		 %s%+3dF\n",
-				print_mstime(gop_time.ms),
-				ctx->frames_since_last_gop);
-			mprint ("Diff. GOP length:	   %s%+3dF",
-				print_mstime(gop_time.ms - first_gop_time.ms),
-				ctx->frames_since_last_gop);
-			mprint ("	(%s)\n",
-				print_mstime(gop_time.ms - first_gop_time.ms
-				+(LLONG) ((ctx->frames_since_last_gop)*1000/29.97)) );
-		}
-
-		if (ctx->false_pict_header)
-			mprint ("\nNumber of likely false picture headers (discarded): %d\n",ctx->false_pict_header);
-
-		if (ctx->stat_numuserheaders)
-			mprint("\nTotal user data fields: %d\n", ctx->stat_numuserheaders);
-		if (ctx->stat_dvdccheaders)
-			mprint("DVD-type user data fields: %d\n", ctx->stat_dvdccheaders);
-		if (ctx->stat_scte20ccheaders)
-			mprint("SCTE-20 type user data fields: %d\n", ctx->stat_scte20ccheaders);
-		if (ctx->stat_replay4000headers)
-			mprint("ReplayTV 4000 user data fields: %d\n", ctx->stat_replay4000headers);
-		if (ctx->stat_replay5000headers)
-			mprint("ReplayTV 5000 user data fields: %d\n", ctx->stat_replay5000headers);
-		if (ctx->stat_hdtv)
-			mprint("HDTV type user data fields: %d\n", ctx->stat_hdtv);
-		if (ctx->stat_dishheaders)
-			mprint("Dish Network user data fields: %d\n", ctx->stat_dishheaders);
-		if (ctx->stat_divicom)
-		{
-			mprint("CEA608/Divicom user data fields: %d\n", ctx->stat_divicom);
-
-			mprint("\n\nNOTE! The CEA 608 / Divicom standard encoding for closed\n");
-			mprint("caption is not well understood!\n\n");
-			mprint("Please submit samples to the developers.\n\n\n");
-		}
-#endif
-
 		list_for_each_entry(dec_ctx, &ctx->dec_ctx_head, list, struct lib_cc_decode)
 		{
 			mprint("\n");
@@ -283,9 +207,12 @@ int main(int argc, char *argv[])
 			dbg_print(CCX_DMT_DECODER_608, "Max. FTS:	   %s  (without caption blocks since then)\n",
 				print_mstime(get_fts_max(dec_ctx->timing)));
 
-			mprint ("\nTotal frames time:	  %s  (%u frames at %.2ffps)\n",
-			print_mstime( (LLONG)(total_frames_count*1000/current_fps) ),
-			total_frames_count, current_fps);
+			if (dec_ctx->codec == CCX_CODEC_ATSC_CC)
+			{
+				mprint ("\nTotal frames time:	  %s  (%u frames at %.2ffps)\n",
+				print_mstime( (LLONG)(total_frames_count*1000/current_fps) ),
+				total_frames_count, current_fps);
+			}
 
 			if (ctx->stat_hdtv)
 			{
@@ -312,8 +239,76 @@ int main(int argc, char *argv[])
 			cb_field1 = 0; cb_field2 = 0; cb_708 = 0;
 			dec_ctx->timing->fts_now = 0;
 			dec_ctx->timing->fts_max = 0;
+		
+
+
+
+		if (dec_ctx->total_pulldownframes)
+			mprint ("incl. pulldown frames:  %s  (%u frames at %.2ffps)\n",
+					print_mstime( (LLONG)(dec_ctx->total_pulldownframes*1000/current_fps) ),
+					dec_ctx->total_pulldownframes, current_fps);
+		if (dec_ctx->timing->pts_set >= 1 && dec_ctx->timing->min_pts != 0x01FFFFFFFFLL)
+		{
+			LLONG postsyncms = (LLONG) (dec_ctx->frames_since_last_gop*1000/current_fps);
+			mprint ("\nMin PTS:				%s\n",
+					print_mstime( dec_ctx->timing->min_pts/(MPEG_CLOCK_FREQ/1000) - dec_ctx->timing->fts_offset));
+			if (pts_big_change)
+				mprint ("(Reference clock was reset at some point, Min PTS is approximated)\n");
+			mprint ("Max PTS:				%s\n",
+					print_mstime( dec_ctx->timing->sync_pts/(MPEG_CLOCK_FREQ/1000) + postsyncms));
+
+			mprint ("Length:				 %s\n",
+					print_mstime( dec_ctx->timing->sync_pts/(MPEG_CLOCK_FREQ/1000) + postsyncms
+								  - dec_ctx->timing->min_pts/(MPEG_CLOCK_FREQ/1000) + dec_ctx->timing->fts_offset ));
 		}
 
+
+		// dvr-ms files have invalid GOPs
+		if (gop_time.inited && first_gop_time.inited && stream_mode != CCX_SM_ASF)
+		{
+			mprint ("\nInitial GOP time:	   %s\n",
+				print_mstime(first_gop_time.ms));
+			mprint ("Final GOP time:		 %s%+3dF\n",
+				print_mstime(gop_time.ms),
+				dec_ctx->frames_since_last_gop);
+			mprint ("Diff. GOP length:	   %s%+3dF",
+				print_mstime(gop_time.ms - first_gop_time.ms),
+				dec_ctx->frames_since_last_gop);
+			mprint ("	(%s)\n\n",
+				print_mstime(gop_time.ms - first_gop_time.ms
+				+(LLONG) ((dec_ctx->frames_since_last_gop)*1000/29.97)) );
+		}
+
+		if (dec_ctx->false_pict_header)
+			mprint ("\nNumber of likely false picture headers (discarded): %d\n",dec_ctx->false_pict_header);
+
+		if (dec_ctx->stat_numuserheaders)
+			mprint("\nTotal user data fields: %d\n", dec_ctx->stat_numuserheaders);
+		if (dec_ctx->stat_dvdccheaders)
+			mprint("DVD-type user data fields: %d\n", dec_ctx->stat_dvdccheaders);
+		if (dec_ctx->stat_scte20ccheaders)
+			mprint("SCTE-20 type user data fields: %d\n", dec_ctx->stat_scte20ccheaders);
+		if (dec_ctx->stat_replay4000headers)
+			mprint("ReplayTV 4000 user data fields: %d\n", dec_ctx->stat_replay4000headers);
+		if (dec_ctx->stat_replay5000headers)
+			mprint("ReplayTV 5000 user data fields: %d\n", dec_ctx->stat_replay5000headers);
+		if (dec_ctx->stat_hdtv)
+			mprint("HDTV type user data fields: %d\n", dec_ctx->stat_hdtv);
+		if (dec_ctx->stat_dishheaders)
+			mprint("Dish Network user data fields: %d\n", dec_ctx->stat_dishheaders);
+		if (dec_ctx->stat_divicom)
+		{
+			mprint("CEA608/Divicom user data fields: %d\n", dec_ctx->stat_divicom);
+
+			mprint("\n\nNOTE! The CEA 608 / Divicom standard encoding for closed\n");
+			mprint("caption is not well understood!\n\n");
+			mprint("Please submit samples to the developers.\n\n\n");
+		}
+
+		}
+
+		
+
 		if(is_decoder_processed_enough(ctx) == CCX_TRUE)
 			break;
 	} // file loop
@@ -336,11 +331,7 @@ int main(int argc, char *argv[])
 			s1, s2);
 	}
 #endif
-	dbg_print(CCX_DMT_708, "[CEA-708] The 708 decoder was reset [%d] times.\n", ctx->freport.data_from_708->reset_count);
-/*
-	if (ccx_options.teletext_mode == CCX_TXT_IN_USE)
-		mprint ( "Teletext decoder: %"PRIu32" packets processed, %"PRIu32" SRT frames written.\n", tlt_packet_counter, tlt_frames_produced);
-*/
+
 	if (is_decoder_processed_enough(ctx) == CCX_TRUE)
 	{
 		mprint ("\rNote: Processing was cancelled before all data was processed because\n");

src/gpacmp4/box_code_3gpp.c

@@ -417,9 +417,18 @@ GF_Err tx3g_Read(GF_Box *s, GF_BitStream *bs)
 	ptr->size -= 18 + GPP_BOX_SIZE + GPP_STYLE_SIZE;
 
 	while (ptr->size) {
+		if (ptr->size < 8) {
+			/* Sometimes there are void atoms at the end of an atom and they are not parsed correctecly */
+			/* So just skip them if we have less than 8 bytes to parse */
+			/* "The actual size of an atom cannot be less than 8 bytes" from QTFF specs */
+			ptr->size = 0;
+			return GF_OK;
+		}
+
 		e = gf_isom_parse_box(&a, bs);
 		if (e) return e;
 		if (ptr->size<a->size) return GF_ISOM_INVALID_FILE;
+
 		ptr->size -= a->size;
 		if (a->type==GF_ISOM_BOX_TYPE_FTAB) {
 			if (ptr->font_table) gf_isom_box_del((GF_Box *) ptr->font_table);

src/gpacmp4/box_funcs.c

@@ -102,7 +102,7 @@ proceed_box:
 	}
 	
 	//handle large box
-	if (size == 1) {
+	if (size == 1 && type != GF_ISOM_BOX_TYPE_VOID) {
 		size = gf_bs_read_u64(bs);
 		hdr_size += 8;
 	}

src/lib_ccx/avc_functions.c

@@ -78,6 +78,12 @@ struct avc_ctx *init_avc(void)
 
 	ctx->last_pic_order_cnt_lsb = -1;
 	ctx->last_slice_pts = -1;
+
+	ctx->num_nal_unit_type_7 = 0;
+	ctx->num_vcl_hrd = 0;
+	ctx->num_nal_hrd = 0;
+	ctx->num_jump_in_frames = 0;
+	ctx->num_unexpected_sei_length = 0;	
 	return ctx;
 }
 
@@ -1023,18 +1029,14 @@ void slice_header (struct lib_cc_decode *ctx, unsigned char *heabuf, unsigned ch
 	}
 
 
-#if 0
-Leaving it for record,  after testing garbled_dishHD.mpg, and
-2014 SugarHouse Casino Mummers Parade Fancy Brigades_new.ts might need to come back
 	// Sometimes two P-slices follow each other, see garbled_dishHD.mpg,
 	// in this case we only treat the first as a reference pic
 	if (isref && ctx->frames_since_last_gop <= 3) // Used to be == 1, but the sample file
 	{ // 2014 SugarHouse Casino Mummers Parade Fancy Brigades_new.ts was garbled
 		// Probably doing a proper PTS sort would be a better solution.
-		//isref = 0;
+		isref = 0;
 		dbg_print(CCX_DMT_TIME, "Ignoring this reference pic.\n");
 	}
-#endif
 
 	// if slices are buffered - flush
 	if (isref)

src/lib_ccx/ccx_common_common.h

@@ -17,6 +17,7 @@
 #define EXIT_UNABLE_TO_DETERMINE_FILE_SIZE      6
 #define EXIT_MALFORMED_PARAMETER                7
 #define EXIT_READ_ERROR                         8
+#define EXIT_WITH_HELP                          9
 #define EXIT_NOT_CLASSIFIED                     300
 #define EXIT_ERROR_IN_CAPITALIZATION_FILE       501
 #define EXIT_BUFFER_FULL                        502

src/lib_ccx/ccx_common_constants.c

@@ -130,9 +130,104 @@ enum
  */
 const char *language[NB_LANGUAGE] =
 {
-	"und",
-	"eng",
-	"fin",
-	"spa",
+	/*0*/"und", //Undefined
+	/*1*/"eng",
+	/*2*/"afr",
+	/*3*/"amh",
+	/*4*/"ara",
+	/*5*/"asm",
+	/*6*/"aze",
+	/*7*/"bel",
+	/*8*/"ben",
+	/*9*/"bod",
+	/*10*/"bos",
+	/*11*/"bul",
+	/*12*/"cat",
+	/*13*/"ceb",
+	/*14*/"ces",
+	/*15*/"chs",
+	/*16*/"chi",
+	/*17*/"chr",
+	/*18*/"cym",
+	/*19*/"dan",
+	/*20*/"deu",
+	/*21*/"dzo",
+	/*22*/"ell",
+	/*23*/"enm",
+	/*24*/"epo",
+	/*25*/"equ",
+	/*26*/"est",
+	/*27*/"eus",
+	/*28*/"fas",
+	/*29*/"fin",
+	/*30*/"fra",
+	/*31*/"frk",
+	/*32*/"frm",
+	/*33*/"gle",
+	/*34*/"glg",
+	/*35*/"grc",
+	/*36*/"guj",
+	/*37*/"hat",
+	/*38*/"heb",
+	/*39*/"hin",
+	/*40*/"hrv",
+	/*41*/"hun",
+	/*42*/"iku",
+	/*43*/"ind",
+	/*44*/"isl",
+	/*45*/"ita",
+	/*46*/"jav",
+	/*47*/"jpn",
+	/*48*/"kan",
+	/*49*/"kat",
+	/*50*/"kaz",
+	/*51*/"khm",
+	/*52*/"kir",
+	/*53*/"kor",
+	/*54*/"kur",
+	/*55*/"lao",
+	/*56*/"lat",
+	/*57*/"lav",
+	/*58*/"lit",
+	/*59*/"mal",
+	/*60*/"mar",
+	/*61*/"mkd",
+	/*62*/"mlt",
+	/*63*/"msa",
+	/*64*/"mya",
+	/*65*/"nep",
+	/*66*/"nld",
+	/*67*/"nor",
+	/*68*/"ori",
+	/*69*/"osd",
+	/*70*/"pan",
+	/*71*/"pol",
+	/*72*/"por",
+	/*73*/"pus",
+	/*74*/"ron",
+	/*75*/"rus",
+	/*76*/"san",
+	/*77*/"sin",
+	/*78*/"slk",
+	/*79*/"slv",
+	/*80*/"spa",
+	/*81*/"sqi",
+	/*82*/"srp",
+	/*83*/"swa",
+	/*84*/"swe",
+	/*85*/"syr",
+	/*86*/"tam",
+	/*87*/"tel",
+	/*88*/"tgk",
+	/*89*/"tgl",
+	/*90*/"tha",
+	/*91*/"tir",
+	/*92*/"tur",
+	/*93*/"uig",
+	/*94*/"ukr",
+	/*95*/"urd",
+	/*96*/"uzb",
+	/*97*/"vie",
+	/*98*/"yid",
 	NULL
 };

src/lib_ccx/ccx_common_constants.h

@@ -162,9 +162,10 @@ enum ccx_output_format
 	CCX_OF_NULL = 5,
 	CCX_OF_SMPTETT = 6,
 	CCX_OF_SPUPNG = 7,
-	CCX_OF_DVDRAW = 8, // See -d at http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/SCC_TOOLS.HTML#CCExtract
+	CCX_OF_DVDRAW = 8, // See -d at http://www.theneitherworld.com/mcpoodle/SCC_TOOLS/DOCS/SCC_TOOLS.HTML#CCExtract
 	CCX_OF_WEBVTT = 9,
 	CCX_OF_SIMPLE_XML = 10,
+	CCX_OF_G608 = 11,
 };
 
 enum ccx_output_date_format
@@ -190,6 +191,10 @@ enum ccx_stream_mode_enum
 	CCX_SM_HEX_DUMP = 8, // Hexadecimal dump generated by wtvccdump
 #endif
 	CCX_SM_WTV = 9,
+#ifdef ENABLE_FFMPEG
+	CCX_SM_FFMPEG = 10,
+#endif
+	CCX_SM_GXF = 11,
 	CCX_SM_AUTODETECT = 16
 };
 
@@ -212,6 +217,8 @@ enum ccx_bufferdata_type
 	CCX_PRIVATE_MPEG2_CC = 6,
 	CCX_DVB_SUBTITLE = 7,
 	CCX_ISDB_SUBTITLE = 8,
+	/* BUffer where cc data contain 3 byte cc_valid ccdata 1 ccdata 2 */
+	CCX_RAW_TYPE = 9,
 };
 
 enum ccx_frame_type
@@ -282,7 +289,7 @@ enum cdp_section_type
 #define CCX_TXT_AUTO_NOT_YET_FOUND	1
 #define CCX_TXT_IN_USE			2 // Positive autodetected, or forced, etc
 
-#define NB_LANGUAGE 5
+#define NB_LANGUAGE 100
 extern const char *language[NB_LANGUAGE];
 
 #define DEF_VAL_STARTCREDITSNOTBEFORE 	"0"

src/lib_ccx/ccx_common_option.c

@@ -14,6 +14,8 @@ void init_options (struct ccx_s_options *options)
 #endif
 	options->nofontcolor=0; // 1 = don't put <font color> tags
 	options->notypesetting=0; // 1 = Don't put <i>, <u>, etc typesetting tags
+	options->no_rollup = 0;
+	options->noscte20 = 0;
 
 
 	options->settings_608.direct_rollup = 0;
@@ -53,6 +55,23 @@ void init_options (struct ccx_s_options *options)
 	options->xmltvliveinterval=10; // interval in seconds between writting xmltv output files in live mode
 	options->xmltvoutputinterval=0; // interval in seconds between writting xmltv full file output
 	options->xmltvonlycurrent=0; // 0 off 1 on
+	options->keep_output_closed = 0; // By default just keep the file open.
+	options->force_flush = 0; // Don't flush whenever content is written.
+	options->append_mode = 0; //By default, files are overwritten.
+	options->ucla = 0; // By default, -UCLA not used
+	options->hardsubx = 0; // By default, don't try to extract hard subtitles
+	options->dvbcolor = 0; // By default, only text detected in DVB
+	options->dvblang = NULL; // By default, autodetect DVB language
+	options->ocrlang = NULL; // By default, autodetect .traineddata file
+
+	/*HardsubX related stuff*/
+	options->hardsubx_ocr_mode = 0;
+	options->hardsubx_subcolor = 0;
+	options->hardsubx_min_sub_duration = 0.5;
+	options->hardsubx_detect_italics = 0;
+	options->hardsubx_conf_thresh = 0.0;
+	options->hardsubx_hue = 0.0;
+	options->hardsubx_lum_thresh = 95.0;
 
 	options->transcript_settings = ccx_encoders_default_transcript_settings;
 	options->millis_separator=',';
@@ -103,6 +122,7 @@ void init_options (struct ccx_s_options *options)
 	options->enc_cfg.no_bom = 0; // Use BOM by default.
 	options->enc_cfg.services_charsets = NULL;
 	options->enc_cfg.all_services_charset = NULL;
+	options->enc_cfg.with_semaphore = 0; 
 
 	options->settings_dtvcc.enabled = 0;
 	options->settings_dtvcc.active_services_count = 0;

src/lib_ccx/ccx_common_option.h

@@ -34,6 +34,10 @@ struct encoder_cfg
 	int gui_mode_reports; // If 1, output in stderr progress updates so the GUI can grab them
 	char *output_filename;
 	enum ccx_output_format write_format; // 0=Raw, 1=srt, 2=SMI
+	int keep_output_closed;
+	int force_flush; // Force flush on content write
+	int append_mode; //Append mode for output files
+	int ucla; // 1 if -UCLA used, 0 if not
 	
 	enum ccx_encoding_type encoding;
 	enum ccx_output_date_format date_format;
@@ -41,6 +45,7 @@ struct encoder_cfg
 	int autodash; // Add dashes (-) before each speaker automatically?
 	int trim_subs; // "    Remove spaces at sides?    "
 	int sentence_cap ; // FIX CASE? = Fix case?
+	int with_semaphore; // Write a .sem file on file open and delete it on close?
 	/* Credit stuff */
 	char *start_credits_text;
 	char *end_credits_text;
@@ -70,9 +75,11 @@ struct ccx_s_options // Options from user parameters
 {
 	int extract; // Extract 1st, 2nd or both fields
 	int no_rollup;
+	int noscte20;
 	int cc_channel; // Channel we want to dump in srt mode
 	int buffer_input;
 	int nofontcolor;
+	int nohtmlescape;
 	int notypesetting;
 	struct ccx_boundary_time extraction_start, extraction_end; // Segment we actually process
 	int print_file_reports;
@@ -109,6 +116,23 @@ struct ccx_s_options // Options from user parameters
 	int xmltvliveinterval; // interval in seconds between writting xmltv output files in live mode
 	int xmltvoutputinterval; // interval in seconds between writting xmltv full file output
 	int xmltvonlycurrent; // 0 off 1 on
+	int keep_output_closed;
+	int force_flush; // Force flush on content write
+	int append_mode; //Append mode for output files
+	int ucla; // 1 if UCLA used, 0 if not
+	int hardsubx; // 1 if burned-in subtitles to be extracted
+	int dvbcolor; // 1 if Color to be detected for DVB
+	char *dvblang; // The name of the language stream for DVB
+	char *ocrlang; // The name of the .traineddata file to be loaded with tesseract
+
+	/*HardsubX related stuff*/
+	int hardsubx_ocr_mode;
+	int hardsubx_subcolor;
+	float hardsubx_min_sub_duration;
+	int hardsubx_detect_italics;
+	float hardsubx_conf_thresh;
+	float hardsubx_hue;
+	float hardsubx_lum_thresh;
 
 	ccx_encoders_transcript_format transcript_settings; // Keeps the settings for generating transcript output files.
 	enum ccx_output_date_format date_format;

src/lib_ccx/ccx_common_timing.c

@@ -55,6 +55,9 @@ struct ccx_common_timing_ctx *init_timing_ctx(struct ccx_common_timing_settings_
 	ctx->max_pts = 0;
 	ctx->sync_pts = 0; 
 	ctx->minimum_fts = 0;
+	ctx->sync_pts2fts_set = 0;
+	ctx->sync_pts2fts_fts = 0;
+	ctx->sync_pts2fts_pts = 0;
 
 	ctx->fts_now = 0; // Time stamp of current file (w/ fts_offset, w/o fts_global)
 	ctx->fts_offset = 0; // Time before first sync_pts
@@ -175,6 +178,7 @@ int set_fts(struct ccx_common_timing_ctx *ctx)
 
 			// Start counting again from here
 			ctx->pts_set = 1; // Force min to be set again
+			ctx->sync_pts2fts_set = 0; // Make note of the new conversion values
 
 			// Avoid next async test - the gap might have occured on
 			// current_tref != 0.
@@ -208,6 +212,12 @@ int set_fts(struct ccx_common_timing_ctx *ctx)
 			// If pts_set is TRUE we have min_pts
 			ctx->fts_now = (LLONG)((ctx->current_pts - ctx->min_pts)/(MPEG_CLOCK_FREQ/1000)
 					+ ctx->fts_offset);
+			if (!ctx->sync_pts2fts_set)
+			{
+				ctx->sync_pts2fts_pts = ctx->current_pts;
+				ctx->sync_pts2fts_fts = ctx->fts_now;
+				ctx->sync_pts2fts_set = 1;
+			}
 		}
 		else
 		{

src/lib_ccx/ccx_common_timing.h

@@ -46,6 +46,9 @@ struct ccx_common_timing_ctx
 	LLONG fts_fc_offset; // Time before first GOP
 	LLONG fts_max; // Remember the maximum fts that we saw in current file
 	LLONG fts_global; // Duration of previous files (-ve mode)
+	int sync_pts2fts_set; //0 = No, 1 = Yes
+	LLONG sync_pts2fts_fts; 
+	LLONG sync_pts2fts_pts;
 };
 // Count 608 (per field) and 708 blocks since last set_fts() call
 extern int cb_field1, cb_field2, cb_708;

src/lib_ccx/ccx_decoders_608.c

@@ -1141,8 +1141,8 @@ int process608(const unsigned char *data, int length, void *private_data, struct
 			continue;
 		}
 		if (hi>=0x10 && hi<=0x1F) // Non-character code or special/extended char
-			// http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/CC_CODES.HTML
-			// http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/CC_CHARS.HTML
+			// http://www.theneitherworld.com/mcpoodle/SCC_TOOLS/DOCS/CC_CODES.HTML
+			// http://www.theneitherworld.com/mcpoodle/SCC_TOOLS/DOCS/CC_CHARS.HTML
 		{
 			if (!context || context->my_field == 2)
 				in_xds_mode=0; // Back to normal (CEA 608-8.6.2)

src/lib_ccx/ccx_decoders_708_output.c

@@ -149,7 +149,7 @@ void ccx_dtvcc_write_srt(ccx_dtvcc_writer_ctx *writer, dtvcc_tv_screen *tv, stru
 	sprintf(buf + strlen(buf), " --> ");
 	mstime_sprintf(tv->time_ms_hide + encoder->subs_delay,
 				   "%02u:%02u:%02u,%03u", buf + strlen(buf));
-	sprintf(buf + strlen(buf), (char *) encoder->encoded_crlf);
+	sprintf(buf + strlen(buf),"%s", (char *) encoder->encoded_crlf);
 
 	write(encoder->dtvcc_writers[tv->service_number - 1].fd, buf, strlen(buf));
 
@@ -422,4 +422,4 @@ void ccx_dtvcc_writer_output(ccx_dtvcc_writer_ctx *writer, dtvcc_tv_screen *tv,
 	}
 
 	_ccx_dtvcc_write(writer, tv, encoder);
-}
+}

src/lib_ccx/ccx_decoders_common.c

@@ -240,6 +240,7 @@ struct lib_cc_decode* init_cc_decode (struct ccx_decoders_common_settings_t *set
 	setting->settings_608->no_rollup = setting->no_rollup;
 	setting->settings_dtvcc->no_rollup = setting->no_rollup;
 	ctx->no_rollup = setting->no_rollup;
+	ctx->noscte20 = setting->noscte20;
 
 	ctx->dtvcc = ccx_dtvcc_init(setting->settings_dtvcc);
 	ctx->dtvcc->is_active = setting->settings_dtvcc->enabled;
@@ -284,7 +285,19 @@ struct lib_cc_decode* init_cc_decode (struct ccx_decoders_common_settings_t *set
 	ctx->max_gop_length = 0;
 	ctx->has_ccdata_buffered = 0;
 	ctx->in_bufferdatatype = CCX_UNKNOWN;
-	ctx->frames_since_last_gop = 0;
+	ctx->frames_since_last_gop  = 0;
+	ctx->total_pulldownfields   = 0;
+	ctx->total_pulldownframes   = 0;
+	ctx->stat_numuserheaders    = 0;
+	ctx->stat_dvdccheaders      = 0;
+	ctx->stat_scte20ccheaders   = 0;
+	ctx->stat_replay5000headers = 0;
+	ctx->stat_replay4000headers = 0;
+	ctx->stat_dishheaders       = 0;
+	ctx->stat_hdtv              = 0;
+	ctx->stat_divicom           = 0;
+	ctx->false_pict_header = 0;
+
 	memcpy(&ctx->extraction_start, &setting->extraction_start,sizeof(struct ccx_boundary_time));
 	memcpy(&ctx->extraction_end, &setting->extraction_end,sizeof(struct ccx_boundary_time));
 
@@ -301,6 +314,7 @@ struct lib_cc_decode* init_cc_decode (struct ccx_decoders_common_settings_t *set
 		setting->output_format==CCX_OF_TRANSCRIPT ||
 		setting->output_format==CCX_OF_SPUPNG ||
 		setting->output_format==CCX_OF_SIMPLE_XML ||
+		setting->output_format==CCX_OF_G608 ||
 		setting->output_format==CCX_OF_NULL)
 		ctx->writedata = process608;
 	else
@@ -338,12 +352,13 @@ struct lib_cc_decode* init_cc_decode (struct ccx_decoders_common_settings_t *set
 
 	ctx->anchor_seq_number = -1;
 	// Init XDS buffers
-	if(setting->ignore_xds == CCX_TRUE)
-		ctx->xds_ctx = NULL;
-	else
-		ctx->xds_ctx = ccx_decoders_xds_init_library(ctx->timing);
-	//xds_cea608_test(ctx->xds_ctx);
+	if(setting->output_format!=CCX_OF_TRANSCRIPT)
+	{
+		setting->xds_write_to_file = 0;
+	}
+	ctx->xds_ctx = ccx_decoders_xds_init_library(ctx->timing, setting->xds_write_to_file);
 
+	ctx->vbi_decoder = NULL;
 	return ctx;
 }
 

src/lib_ccx/ccx_decoders_isdb.c

@@ -373,6 +373,7 @@ static int reserve_buf(struct ISDBText *text, size_t len)
 	text->buf = ptr;
 	text->len = blen;
 	isdb_log ("expanded ctx->text(%lu)\n", blen);
+	return CCX_OK;
 }
 
 static int append_char(ISDBSubContext *ctx, const char ch)

src/lib_ccx/ccx_decoders_structs.h

@@ -90,6 +90,7 @@ struct ccx_decoders_common_settings_t
 	int extract; // Extract 1st, 2nd or both fields
 	int fullbin; // Disable pruning of padding cc blocks
 	int no_rollup;
+	int noscte20;
 	struct ccx_decoder_608_settings *settings_608; //  Contains the settings for the 608 decoder.
 	ccx_decoder_dtvcc_settings *settings_dtvcc; //Same for cea 708 captions decoder (dtvcc)
 	int cc_channel; // Channel we want to dump in srt mode
@@ -97,7 +98,7 @@ struct ccx_decoders_common_settings_t
 	unsigned int hauppauge_mode; // If 1, use PID=1003, process specially and so on
 	int program_number;
 	enum ccx_code_type codec;
-	int ignore_xds;
+	int xds_write_to_file;
 	void *private_data;
 };
 
@@ -113,6 +114,7 @@ struct lib_cc_decode
 	void *context_cc608_field_2;
 
 	int no_rollup; // If 1, write one line at a time
+	int noscte20;
 	int fix_padding; // Replace 0000 with 8080 in HDTV (needed for some cards)
 	enum ccx_output_format write_format; // 0=Raw, 1=srt, 2=SMI
 	struct ccx_boundary_time extraction_start, extraction_end; // Segment we actually process
@@ -164,6 +166,17 @@ struct lib_cc_decode
 	/* Reguired in es_function.c and es_userdata.c */
 	unsigned top_field_first; // Needs to be global
 
+    /* Stats. Modified in es_userdata.c*/
+    int stat_numuserheaders;
+    int stat_dvdccheaders;
+    int stat_scte20ccheaders;
+    int stat_replay5000headers;
+    int stat_replay4000headers;
+    int stat_dishheaders;
+    int stat_hdtv;
+    int stat_divicom;
+    int false_pict_header;
+
 	ccx_dtvcc_ctx *dtvcc;
 	int current_field;
 	// Analyse/use the picture information
@@ -179,6 +192,7 @@ struct lib_cc_decode
 	// B-Frames belong to this I- or P-frame.
 	int anchor_seq_number;
 	struct ccx_decoders_xds_context *xds_ctx;
+	struct ccx_decoder_vbi_ctx *vbi_decoder;
 
 	int (*writedata)(const unsigned char *data, int length, void *private_data, struct cc_subtitle *sub);
 };

src/lib_ccx/ccx_decoders_vbi.c

@@ -0,0 +1,100 @@
+#include "ccx_decoders_vbi.h"
+#include "ccx_decoders_common.h"
+#include "ccx_common_constants.h"
+#include "ccx_common_common.h"
+#include "utility.h"
+#include "stdlib.h"
+
+void delete_decoder_vbi(struct ccx_decoder_vbi_ctx** arg)
+{
+	struct ccx_decoder_vbi_ctx* ctx = *arg;
+	vbi_raw_decoder_destroy (&ctx->zvbi_decoder);
+
+	freep(arg);
+}
+struct ccx_decoder_vbi_ctx* init_decoder_vbi(struct ccx_decoder_vbi_cfg *cfg)
+{
+	struct ccx_decoder_vbi_ctx *vbi;
+
+	vbi = malloc(sizeof(*vbi));
+	if(!vbi)
+		return NULL;
+
+#ifdef VBI_DEBUG
+	vbi->vbi_debug_dump = fopen("dump_720.vbi","w");
+#endif
+	vbi_raw_decoder_init (&vbi->zvbi_decoder);
+
+	if(cfg == NULL)
+	{
+		/* Specify the image format. */
+		vbi->zvbi_decoder.scanning             = 525;
+
+		/* The decoder ignores chroma data. */
+		vbi->zvbi_decoder.sampling_format      = VBI_PIXFMT_YUV420;
+
+		/* You may have to adjust this. */
+		vbi->zvbi_decoder.sampling_rate        = 13.5e6; /* Hz */
+		vbi->zvbi_decoder.bytes_per_line       = 720;
+
+		/* Sampling starts 9.7 µs from the front edge of the
+		hor. sync pulse. You may have to adjust this. */
+		vbi->zvbi_decoder.offset               = 9.7e-6 * 13.5e6;
+
+		/* Which lines were captured from the first field.
+			You may have to adjust this. */
+		vbi->zvbi_decoder.start[0]             = 21;
+		vbi->zvbi_decoder.count[0]             = 1;
+
+		/* Second field. */
+		vbi->zvbi_decoder.start[1]             = 284;
+		vbi->zvbi_decoder.count[1]             = 1;
+
+		/* The image contains all lines of the first field,
+			followed by all lines of the second field. */
+		vbi->zvbi_decoder.interlaced           = CCX_TRUE;
+
+		/* The first field is always first in memory. */
+		vbi->zvbi_decoder.synchronous          = CCX_TRUE;
+
+		/* Specify the services you want. */
+		vbi_raw_decoder_add_services (&vbi->zvbi_decoder, VBI_SLICED_CAPTION_525, /* strict */ 0);
+
+	}
+	return vbi;
+}
+
+int decode_vbi(struct lib_cc_decode *dec_ctx, uint8_t field, unsigned char *buffer, size_t len,  struct cc_subtitle *sub)
+{
+	int i = 0;
+	unsigned int n_lines;
+	vbi_sliced sliced[52];
+	if(dec_ctx->vbi_decoder == NULL)
+	{
+		dec_ctx->vbi_decoder = init_decoder_vbi(NULL);
+	}
+
+	len -= 720;
+
+	n_lines = vbi_raw_decode(&dec_ctx->vbi_decoder->zvbi_decoder, buffer, sliced);
+	//n_lines = vbi3_raw_decoder_decode (&dec_ctx->vbi_decoder->zvbi_decoder, sliced, 2, buffer);
+	if (n_lines > 0)
+	{
+		for (i = 0; i < n_lines; ++i)
+		{
+			int index = 0;
+			//for(index = 0; index < 56; index += 2)
+			{
+				unsigned char data[3];
+				if (field == 1)
+					data[0] = 0x04;
+				else
+					data[0] = 0x05;
+				data[1] = sliced[i].data[index];
+				data[2] = sliced[i].data[index+1];
+				do_cb(dec_ctx, data, sub);
+			}
+		}
+	}
+	return CCX_OK;
+}

src/lib_ccx/ccx_decoders_vbi.h

@@ -0,0 +1,28 @@
+#ifndef CCX_DECODER_VBI
+#define CCX_DECODER_VBI
+
+#include <zvbi_decoder.h>
+#define VBI_DEBUG
+
+#include "ccx_decoders_structs.h"
+
+struct ccx_decoder_vbi_cfg
+{
+#ifdef VBI_DEBUG
+	char *debug_file_name;
+#endif
+};
+
+struct ccx_decoder_vbi_ctx
+{
+	int vbi_decoder_inited;
+	vbi_raw_decoder zvbi_decoder;
+	//vbi3_raw_decoder zvbi_decoder;
+#ifdef VBI_DEBUG
+	FILE *vbi_debug_dump;
+#endif
+};
+
+
+int decode_vbi(struct lib_cc_decode *dec_ctx, uint8_t field, unsigned char *buffer, size_t len,  struct cc_subtitle *sub);
+#endif

src/lib_ccx/ccx_decoders_xds.c

@@ -2,6 +2,7 @@
 #include "ccx_common_constants.h"
 #include "ccx_common_timing.h"
 #include "ccx_common_common.h"
+#include "utility.h" 
 
 LLONG ts_start_of_xds = -1; // Time at which we switched to XDS mode, =-1 hasn't happened yet
 
@@ -115,9 +116,14 @@ typedef struct ccx_decoders_xds_context
 	int cur_xds_packet_type;
 	struct ccx_common_timing_ctx *timing;
 
+	unsigned current_ar_start;
+	unsigned current_ar_end;
+
+	int xds_write_to_file; // Set to 1 if XDS data is to be written to file
+
 } ccx_decoders_xds_context_t;
 
-struct ccx_decoders_xds_context *ccx_decoders_xds_init_library(struct ccx_common_timing_ctx *timing)
+struct ccx_decoders_xds_context *ccx_decoders_xds_init_library(struct ccx_common_timing_ctx *timing, int xds_write_to_file)
 {
 	int i;
 	struct ccx_decoders_xds_context *ctx = NULL;
@@ -157,6 +163,8 @@ struct ccx_decoders_xds_context *ccx_decoders_xds_init_library(struct ccx_common
 	ctx->cur_xds_packet_type	= 0;
 	ctx->timing = timing;
 
+	ctx->xds_write_to_file = xds_write_to_file;
+
 	return ctx;
 }
 
@@ -192,6 +200,8 @@ int write_xds_string(struct cc_subtitle *sub, struct ccx_decoders_xds_context *c
 
 void xdsprint (struct cc_subtitle *sub, struct ccx_decoders_xds_context *ctx, const char *fmt,...)
 {
+	if(!ctx->xds_write_to_file)
+		return;
 	/* Guess we need no more than 100 bytes. */
 	int n, size = 100;
 	char *p, *np;
@@ -697,6 +707,7 @@ int xds_do_current_and_future (struct cc_subtitle *sub, struct ccx_decoders_xds_
 		case XDS_TYPE_ASPECT_RATIO_INFO:
 			{
 				unsigned ar_start, ar_end;
+				int changed = 0;
 				was_proc = 1;
 				if (ctx->cur_xds_payload_length < 5) // We need 2 data bytes
 					break;
@@ -709,8 +720,20 @@ int xds_do_current_and_future (struct cc_subtitle *sub, struct ccx_decoders_xds_
 				ar_end = 262 - (ctx->cur_xds_payload[3] & 0x1F);
 				unsigned active_picture_height = ar_end - ar_start;
 				float aspect_ratio = (float) 320 / active_picture_height;
-				ccx_common_logging.log_ftn("\rXDS Notice: Aspect ratio info, start line=%u, end line=%u\n", ar_start,ar_end);
-				ccx_common_logging.log_ftn("\rXDS Notice: Aspect ratio info, active picture height=%u, ratio=%f\n", active_picture_height, aspect_ratio);
+
+				if (ar_start != ctx->current_ar_start)
+				{	
+					ctx->current_ar_start = ar_start;
+					ctx->current_ar_end = ar_end;
+					changed = 1;
+					ccx_common_logging.log_ftn("\rXDS Notice: Aspect ratio info, start line=%u, end line=%u\n", ar_start, ar_end);
+					ccx_common_logging.log_ftn("\rXDS Notice: Aspect ratio info, active picture height=%u, ratio=%f\n", active_picture_height, aspect_ratio);
+				}
+				else
+				{
+					ccx_common_logging.debug_ftn(CCX_DMT_DECODER_XDS, "\rXDS Notice: Aspect ratio info, start line=%u, end line=%u\n", ar_start, ar_end);
+					ccx_common_logging.debug_ftn(CCX_DMT_DECODER_XDS, "\rXDS Notice: Aspect ratio info, active picture height=%u, ratio=%f\n", active_picture_height, aspect_ratio);
+				}
 
 			}
 		case XDS_TYPE_PROGRAM_DESC_1:
@@ -959,7 +982,8 @@ void do_end_of_xds (struct cc_subtitle *sub, struct ccx_decoders_xds_context *ct
 
 	if (!was_proc)
 	{
-		ccx_common_logging.log_ftn ("Note: We found an currently unsupported XDS packet.\n");
+		ccx_common_logging.log_ftn ("Note: We found a currently unsupported XDS packet.\n");
+		dump (CCX_DMT_DECODER_XDS,ctx->cur_xds_payload,ctx->cur_xds_payload_length,0,0);
 	}
 	clear_xds_buffer (ctx, ctx->cur_xds_buffer_idx);
 

src/lib_ccx/ccx_decoders_xds.h

@@ -7,7 +7,7 @@ struct ccx_decoders_xds_context;
 void process_xds_bytes (struct ccx_decoders_xds_context *ctx, const unsigned char hi, int lo);
 void do_end_of_xds (struct cc_subtitle *sub, struct ccx_decoders_xds_context *ctx, unsigned char expected_checksum);
 
-struct ccx_decoders_xds_context *ccx_decoders_xds_init_library(struct ccx_common_timing_ctx *timing);
+struct ccx_decoders_xds_context *ccx_decoders_xds_init_library(struct ccx_common_timing_ctx *timing, int xds_write_to_file);
 
 void xds_cea608_test();
 #endif

src/lib_ccx/ccx_demuxer.c

@@ -2,6 +2,7 @@
 #include "activity.h"
 #include "lib_ccx.h"
 #include "utility.h"
+#include "ffmpeg_intgr.h"
 
 static void ccx_demuxer_reset(struct ccx_demuxer *ctx)
 {
@@ -28,6 +29,25 @@ static int ccx_demuxer_isopen(struct ccx_demuxer *ctx)
 }
 static int ccx_demuxer_open(struct ccx_demuxer *ctx, const char *file)
 {
+	ctx->past = 0;
+	ctx->min_global_timestamp = 0;
+	ctx->global_timestamp_inited = 0;
+	ctx->last_global_timestamp = 0;
+	ctx->offset_global_timestamp = 0;
+
+#ifdef ENABLE_FFMPEG
+	ctx->ffmpeg_ctx =  init_ffmpeg(file);
+	if(ctx->ffmpeg_ctx)
+	{
+		ctx->stream_mode = CCX_SM_FFMPEG;
+		ctx->auto_stream = CCX_SM_FFMPEG;
+		return 0;
+	}
+	else
+	{
+		mprint ("\rFailed to initialized ffmpeg falling back to legacy\n");
+	}
+#endif
 	init_file_buffer(ctx);
 	if (ccx_options.input_source==CCX_DS_STDIN)
 	{
@@ -112,6 +132,9 @@ static int ccx_demuxer_open(struct ccx_demuxer *ctx, const char *file)
 			case CCX_SM_MP4:
 				mprint ("\rFile seems to be a MP4\n");
 				break;
+			case CCX_SM_GXF:
+				mprint ("\rFile seems to be a GXF\n");
+				break;
 #ifdef WTV_DEBUG
 			case CCX_SM_HEX_DUMP:
 				mprint ("\rFile seems to be an hexadecimal dump\n");					
@@ -145,7 +168,7 @@ static int ccx_demuxer_open(struct ccx_demuxer *ctx, const char *file)
 			{
 				case CCX_SM_ELEMENTARY_OR_NOT_FOUND:
 				case CCX_SM_PROGRAM:
-					if ( detect_myth(ctx->parent) )
+					if ( detect_myth(ctx) )
 					{
 						ctx->stream_mode=CCX_SM_MYTH;
 					}
@@ -156,12 +179,6 @@ static int ccx_demuxer_open(struct ccx_demuxer *ctx, const char *file)
 			}
 			break;					
 	}
-	ctx->past = 0;
-	ctx->min_global_timestamp = 0;
-	ctx->global_timestamp_inited = 0;
-	ctx->last_global_timestamp = 0;
-	ctx->offset_global_timestamp = 0;
-
 
 	return 0;
 }
@@ -374,6 +391,8 @@ struct demuxer_data* alloc_demuxer_data(void)
 	data->codec = CCX_CODEC_NONE;
 	data->len = 0;
 	data->pts = CCX_NOPTS;
+	data->tb.num = 1;
+	data->tb.den = 90000;
 	data->next_stream = 0;
 	data->next_program = 0;
 	return data;

src/lib_ccx/ccx_demuxer.h

@@ -5,6 +5,7 @@
 #include "ts_functions.h"
 #include "list.h"
 #include "activity.h"
+#include "utility.h"
 
 /* Report information */
 #define SUB_STREAMS_CNT 10
@@ -129,8 +130,14 @@ struct ccx_demuxer
 	// Remember if the last header was valid. Used to suppress too much output
 	// and the expected unrecognized first header for TiVo files.
 	int strangeheader;
+#ifdef ENABLE_FFMPEG
+	void *ffmpeg_ctx;
+#endif
 
 	void *parent;
+
+	//Will contain actual Demuxer Context
+	void *private_data;
 	void (*print_cfg)(struct ccx_demuxer *ctx);
 	void (*reset)(struct ccx_demuxer *ctx);
 	void (*close)(struct ccx_demuxer *ctx);
@@ -150,6 +157,7 @@ struct demuxer_data
 	unsigned char *buffer;
 	size_t len;
 	LLONG pts;
+	struct ccx_rational tb;
 	struct demuxer_data *next_stream;
 	struct demuxer_data *next_program;
 };

src/lib_ccx/ccx_encoders_common.c

@@ -10,6 +10,13 @@
 #include "ccx_encoders_xds.h"
 #include "ccx_encoders_helpers.h"
 
+#ifdef WIN32
+int fsync(int fd)
+{
+	return FlushFileBuffers((HANDLE)_get_osfhandle(fd)) ? 0 : -1;
+}
+#endif
+
 // These are the default settings for plain transcripts. No times, no CC or caption mode, and no XDS.
 ccx_encoders_transcript_format ccx_encoders_default_transcript_settings =
 {
@@ -50,7 +57,7 @@ static const char *smptett_header = "<?xml version=\"1.0\" encoding=\"UTF-8\" ?>
 "  <body>\n"
 "    <div>\n";
 
-static const char *webvtt_header = "WEBVTT\r\n\r\n";
+static const char *webvtt_header = "WEBVTT\r\n";
 
 static const char *simple_xml_header = "<?xml version=\"1.0\" encoding=\"utf-8\"?>\r\n<captions>\r\n";
 
@@ -99,7 +106,6 @@ int change_utf8_encoding(unsigned char* dest, unsigned char* src, int len, enum
 {
 	unsigned char *orig = dest; // Keep for calculating length
 	unsigned char *orig_src = src; // Keep for calculating length
-	int bytes = 0;
 	for (int i = 0; src < orig_src + len;)
 	{
 		unsigned char c = src[i];
@@ -382,6 +388,7 @@ static int write_subtitle_file_header(struct encoder_ctx *ctx, struct ccx_s_writ
 	switch (ctx->write_format)
 	{
 		case CCX_OF_SRT: // Subrip subtitles have no header
+		case CCX_OF_G608:
 			ret = write_bom(ctx, out);
 			if(ret < 0)
 				return -1;
@@ -460,6 +467,7 @@ static int write_subtitle_file_header(struct encoder_ctx *ctx, struct ccx_s_writ
 			ret = write_bom(ctx, out);
 			if(ret < 0)
 				return -1;
+			break;
 		case CCX_OF_SIMPLE_XML: // No header. Fall thru
 			ret = write_bom(ctx, out);
 			if(ret < 0)
@@ -472,6 +480,7 @@ static int write_subtitle_file_header(struct encoder_ctx *ctx, struct ccx_s_writ
 				mprint("WARNING: Unable to write complete Buffer \n");
 				return -1;
 			}
+			break;
 		default:
 			break;
 	}
@@ -495,7 +504,7 @@ int write_cc_subtitle_as_simplexml(struct cc_subtitle *sub, struct encoder_ctx *
 		str = strtok_r(str, "\r\n", &save_str);
 		do
 		{
-			length = get_str_basic(context->subline, str, context->trim_subs, sub->enc_type, context->encoding, strlen(str));
+			length = get_str_basic(context->subline, (unsigned char*)str, context->trim_subs, sub->enc_type, context->encoding, strlen(str));
 			if (length <= 0)
 			{
 				continue;
@@ -506,7 +515,7 @@ int write_cc_subtitle_as_simplexml(struct cc_subtitle *sub, struct encoder_ctx *
 				mprint("Warning:Loss of data\n");
 			}
 
-		} while (str = strtok_r(NULL, "\r\n", &save_str) );
+		} while ( (str = strtok_r(NULL, "\r\n", &save_str) ));
 
 		freep(&sub->data);
 		lsub = sub;
@@ -519,6 +528,7 @@ int write_cc_subtitle_as_simplexml(struct cc_subtitle *sub, struct encoder_ctx *
 		lsub = sub;
 	}
 
+	return ret;
 }
 
 int write_cc_subtitle_as_transcript(struct cc_subtitle *sub, struct encoder_ctx *context)
@@ -542,7 +552,7 @@ int write_cc_subtitle_as_transcript(struct cc_subtitle *sub, struct encoder_ctx
 		if (context->sentence_cap)
 		{
 			//TODO capitalize (context, line_number,data);
-			//TODO correct_case(line_number, data);
+			//TODO correct_case_with_dictionary(line_number, data);
 		}
 
 		if (start_time == -1)
@@ -559,7 +569,7 @@ int write_cc_subtitle_as_transcript(struct cc_subtitle *sub, struct encoder_ctx
 		str = strtok_r(str, "\r\n", &save_str);
 		do
 		{
-			length = get_str_basic(context->subline, str, context->trim_subs, sub->enc_type, context->encoding, strlen(str));
+			length = get_str_basic(context->subline, (unsigned char*)str, context->trim_subs, sub->enc_type, context->encoding, strlen(str));
 			if (length <= 0)
 			{
 				continue;
@@ -588,7 +598,7 @@ int write_cc_subtitle_as_transcript(struct cc_subtitle *sub, struct encoder_ctx
 				char buf[80];
 				if (context->transcript_settings->relativeTimestamp)
 				{
-					millis_to_date(end_time, buf, context->date_format, context->millis_separator);
+					millis_to_date(end_time + context->subs_delay, buf, context->date_format, context->millis_separator);
 					fdprintf(context->out->fh, "%s|", buf);
 				}
 				else
@@ -606,12 +616,15 @@ int write_cc_subtitle_as_transcript(struct cc_subtitle *sub, struct encoder_ctx
 				if(context->in_fileformat == 1)
 					//TODO, data->my_field == 1 ? data->channel : data->channel + 2); // Data from field 2 is CC3 or 4
 					fdprintf(context->out->fh, "CC?|");
-				else
+				else if (!context->ucla || !strcmp(sub->mode,"TLT"))
 					fdprintf(context->out->fh, sub->info);
 			}
 			if (context->transcript_settings->showMode)
 			{
-				fdprintf(context->out->fh, "%s|", sub->mode);
+				if(context->ucla && strcmp(sub->mode,"TLT") == 0)
+					fdprintf(context->out->fh, "|");
+				else
+					fdprintf(context->out->fh, "%s|", sub->mode);
 			}
 			ret = write(context->out->fh, context->subline, length);
 			if(ret < length)
@@ -625,7 +638,7 @@ int write_cc_subtitle_as_transcript(struct cc_subtitle *sub, struct encoder_ctx
 				mprint("Warning:Loss of data\n");
 			}
 
-		} while (str = strtok_r(NULL, "\r\n", &save_str) );
+		} while ( (str = strtok_r(NULL, "\r\n", &save_str)) );
 
 		freep(&sub->data);
 		lsub = sub;
@@ -650,8 +663,8 @@ void write_cc_line_as_simplexml(struct eia608_screen *data, struct encoder_ctx *
 	char *cap1 = "</caption>";
 	if (context->sentence_cap)
 	{
-		capitalize (context, line_number, data);
-		correct_case(line_number, data);
+		if (clever_capitalize (context, line_number, data))
+			correct_case_with_dictionary(line_number, data);
 	}
 	length = get_str_basic (context->subline, data->characters[line_number],
 			context->trim_subs, CCX_ENC_ASCII, context->encoding, CCX_DECODER_608_SCREEN_WIDTH);
@@ -674,8 +687,8 @@ void write_cc_line_as_transcript2(struct eia608_screen *data, struct encoder_ctx
 	LLONG end_time = data->end_time;
 	if (context->sentence_cap)
 	{
-		capitalize (context, line_number, data);
-		correct_case(line_number, data);
+		if (clever_capitalize (context, line_number, data))
+			correct_case_with_dictionary(line_number, data);
 	}
 	int length = get_str_basic (context->subline, data->characters[line_number],
 			context->trim_subs, CCX_ENC_ASCII, context->encoding, CCX_DECODER_608_SCREEN_WIDTH);
@@ -718,8 +731,8 @@ void write_cc_line_as_transcript2(struct eia608_screen *data, struct encoder_ctx
 				fdprintf(context->out->fh, "%s|", buf2);
 			}
 			else {
-				time_t end_time_int = (end_time + context->subs_delay) / 1000;
-				int end_time_dec = (end_time + context->subs_delay) % 1000;
+				time_t end_time_int = end_time / 1000;
+				int end_time_dec = end_time % 1000;
 				struct tm *end_time_struct = gmtime(&end_time_int);
 				strftime(buf2, sizeof(buf2), "%Y%m%d%H%M%S", end_time_struct);
 				fdprintf(context->out->fh, "%s%c%03d|", buf2,context->millis_separator,end_time_dec);
@@ -899,6 +912,7 @@ int write_cc_bitmap_as_transcript(struct cc_subtitle *sub, struct encoder_ctx *c
 					fdprintf(context->out->fh,"DVB|");
 				}
 				fdprintf(context->out->fh,"%s",token);
+				write(context->out->fh, context->encoded_crlf, context->encoded_crlf_length);
 				token = strtok(NULL,"\r\n");
 
 			}
@@ -1055,6 +1069,9 @@ static int init_output_ctx(struct encoder_ctx *ctx, struct encoder_cfg *cfg)
 	if(!ctx->out)
 		return -1;
 	ctx->nb_out = nb_lang;
+	ctx->keep_output_closed = cfg->keep_output_closed;
+	ctx->force_flush = cfg->force_flush;
+	ctx->ucla = cfg->ucla;
 
 	if(cfg->cc_to_stdout == CCX_FALSE && cfg->send_to_srv == CCX_FALSE)
 	{
@@ -1068,19 +1085,14 @@ static int init_output_ctx(struct encoder_ctx *ctx, struct encoder_cfg *cfg)
 				basefilename = get_basename(cfg->output_filename);
 				extension = get_file_extension(cfg->write_format);
 
-				ret = init_write(&ctx->out[0], strdup(cfg->output_filename));
+				ret = init_write(&ctx->out[0], strdup(cfg->output_filename), cfg->with_semaphore);
 				check_ret(cfg->output_filename);
-				ret = init_write(&ctx->out[1], create_outfilename(basefilename, "_2", extension));
+				ret = init_write(&ctx->out[1], create_outfilename(basefilename, "_2", extension), cfg->with_semaphore);
 				check_ret(ctx->out[1].filename);
 			}
-			else if (cfg->extract == 1)
-			{
-				ret = init_write(ctx->out, strdup(cfg->output_filename));
-				check_ret(cfg->output_filename);
-			}
 			else
 			{
-				ret = init_write(ctx->out, strdup(cfg->output_filename));
+				ret = init_write(ctx->out, strdup(cfg->output_filename), cfg->with_semaphore );
 				check_ret(cfg->output_filename);
 			}
 		}
@@ -1091,19 +1103,14 @@ static int init_output_ctx(struct encoder_ctx *ctx, struct encoder_cfg *cfg)
 
 			if (cfg->extract == 12)
 			{
-				ret = init_write(&ctx->out[0], create_outfilename(basefilename, "_1", extension));
+				ret = init_write(&ctx->out[0], create_outfilename(basefilename, "_1", extension), cfg->with_semaphore);
 				check_ret(ctx->out[0].filename);
-				ret = init_write(&ctx->out[1], create_outfilename(basefilename, "_2", extension));
+				ret = init_write(&ctx->out[1], create_outfilename(basefilename, "_2", extension), cfg->with_semaphore);
 				check_ret(ctx->out[1].filename);
 			}
-			else if (cfg->extract == 1)
-			{
-				ret = init_write(ctx->out, create_outfilename(basefilename, NULL, extension));
-				check_ret(ctx->out->filename);
-			}
 			else
 			{
-				ret = init_write(ctx->out, create_outfilename(basefilename, NULL, extension));
+				ret = init_write(ctx->out, create_outfilename(basefilename, NULL, extension), cfg->with_semaphore);
 				check_ret(ctx->out->filename);
 			}
 		}
@@ -1116,6 +1123,8 @@ static int init_output_ctx(struct encoder_ctx *ctx, struct encoder_cfg *cfg)
 	{
 		ctx->out[0].fh = STDOUT_FILENO;
 		ctx->out[0].filename = NULL;
+		ctx->out[0].with_semaphore = 0;
+		ctx->out[0].semaphore_filename = NULL;
 		mprint ("Sending captions to stdout.\n");
 	}
 
@@ -1209,6 +1218,7 @@ struct encoder_ctx *init_encoder(struct encoder_cfg *opt)
 
 	ctx->capacity=INITIAL_ENC_BUFFER_CAPACITY;
 	ctx->srt_counter = 0;
+	ctx->wrote_webvtt_sync_header = 0;
 
 	ctx->program_number = opt->program_number;
 	ctx->send_to_srv = opt->send_to_srv;
@@ -1242,6 +1252,9 @@ struct encoder_ctx *init_encoder(struct encoder_cfg *opt)
 	ctx->no_type_setting = opt->no_type_setting;
 	ctx->gui_mode_reports = opt->gui_mode_reports;
 	ctx->extract = opt->extract;
+	ctx->keep_output_closed = opt->keep_output_closed;
+	ctx->force_flush = opt->force_flush;
+	ctx->ucla = opt->ucla;
 
 	ctx->subline = (unsigned char *) malloc (SUBLINESIZE);
 	if(!ctx->subline)
@@ -1310,6 +1323,8 @@ int encode_sub(struct encoder_ctx *context, struct cc_subtitle *sub)
 	if(!context)
 		return CCX_OK;
 
+	context = change_filename(context);
+
 	if (sub->type == CC_608)
 	{
 		struct eia608_screen *data = NULL;
@@ -1319,7 +1334,7 @@ int encode_sub(struct encoder_ctx *context, struct cc_subtitle *sub)
 			// Determine context based on channel. This replaces the code that was above, as this was incomplete (for cases where -12 was used for example)
 			out = get_output_ctx(context, data->my_field);
 
-			context->new_sentence = 1;
+			// context->new_sentence = 1; CFS: WHY??? Sentences may appear in different frames
 
 			if(data->format == SFORMAT_XDS)
 			{
@@ -1346,6 +1361,9 @@ int encode_sub(struct encoder_ctx *context, struct cc_subtitle *sub)
 					try_to_add_start_credits(context, data->start_time);
 				wrote_something = write_cc_buffer_as_srt(data, context);
 				break;
+			case CCX_OF_G608:
+				wrote_something = write_cc_buffer_as_g608(data, context);
+				break;
 			case CCX_OF_WEBVTT:
 				if (!context->startcredits_displayed && context->start_credits_text != NULL)
 					try_to_add_start_credits(context, data->start_time);
@@ -1368,7 +1386,17 @@ int encode_sub(struct encoder_ctx *context, struct cc_subtitle *sub)
 				wrote_something = write_cc_buffer_as_spupng(data, context);
 				break;
 			case CCX_OF_SIMPLE_XML:
+				if (ccx_options.keep_output_closed && context->out->temporarily_closed)
+				{
+					temporarily_open_output(context->out);
+					write_subtitle_file_header(context, context->out);
+				}
 				wrote_something = write_cc_buffer_as_simplexml(data, context);
+				if (ccx_options.keep_output_closed)
+				{
+					write_subtitle_file_footer(context, context->out);
+					temporarily_close_output(context->out);
+				}
 				break;
 			default:
 				break;
@@ -1473,6 +1501,8 @@ int encode_sub(struct encoder_ctx *context, struct cc_subtitle *sub)
 	}
 	if (!sub->nb_data)
 		freep(&sub->data);
+	if (wrote_something && context->force_flush)
+		fsync(context->out->fh); // Don't buffer
 	return wrote_something;
 }
 

src/lib_ccx/ccx_encoders_common.h

@@ -37,6 +37,9 @@ struct encoder_ctx
 	/* keep count of srt subtitle*/
 	unsigned int srt_counter;
 
+	/* Did we write the WebVTT sync header already? */
+	unsigned int wrote_webvtt_sync_header;
+
 	/* Input outputs */
 	/* Flag giving hint that output is send to server through network */
 	unsigned int send_to_srv;
@@ -50,6 +53,14 @@ struct encoder_ctx
 	int nb_out;
 	/* Input file format used in Teletext for exceptional output */
 	unsigned int in_fileformat; //1 =Normal, 2=Teletext
+	/* Keep output file closed when not actually writing to it and start over each time (add headers, etc) */
+	unsigned int keep_output_closed; 
+	/* Force a flush on the file buffer whenever content is written */
+	int force_flush;
+	/* Keep track of whether -UCLA used */
+	int ucla;
+
+	struct ccx_common_timing_ctx *timing; /* Some encoders need access to PTS, such as WebVTT */
 
 	/* Flag saying BOM to be written in each output file */
 	enum ccx_encoding_type encoding;
@@ -128,6 +139,8 @@ void dinit_encoder(struct encoder_ctx **arg, LLONG current_fts);
  */
 int encode_sub(struct encoder_ctx *ctx,struct cc_subtitle *sub);
 
+int write_cc_buffer_as_g608(struct eia608_screen *data, struct encoder_ctx *context);
+
 int write_cc_buffer_as_srt(struct eia608_screen *data, struct encoder_ctx *context);
 int write_cc_subtitle_as_srt(struct cc_subtitle *sub, struct encoder_ctx *context);
 int write_stringz_as_srt(char *string, struct encoder_ctx *context, LLONG ms_start, LLONG ms_end);

src/lib_ccx/ccx_encoders_g608.c

@@ -0,0 +1,109 @@
+#include "lib_ccx.h"
+#include "ccx_encoders_common.h"
+#include "ccx_encoders_helpers.h"
+
+static unsigned int get_line_encoded(struct encoder_ctx *ctx, unsigned char *buffer, int line_num, struct eia608_screen *data)
+{
+	unsigned char *orig = buffer; // Keep for debugging
+	unsigned char *line = data->characters[line_num];
+	for (int i = 0; i < 32; i++)
+	{
+		int bytes = 0;
+		switch (ctx->encoding)
+		{
+		case CCX_ENC_UTF_8:
+			bytes = get_char_in_utf_8(buffer, line[i]);
+			break;
+		case CCX_ENC_LATIN_1:
+			get_char_in_latin_1(buffer, line[i]);
+			bytes = 1;
+			break;
+		case CCX_ENC_UNICODE:
+			get_char_in_unicode(buffer, line[i]);
+			bytes = 2;
+		case CCX_ENC_ASCII:
+		    *buffer = line[i];
+			bytes = 1;
+			break;
+		}
+		buffer += bytes;
+	}
+	return (unsigned int)(buffer - orig); // Return length
+}
+static unsigned int get_color_encoded(struct encoder_ctx *ctx, unsigned char *buffer, int line_num, struct eia608_screen *data)
+{
+	unsigned char *orig = buffer; // Keep for debugging
+	for (int i = 0; i < 32; i++)
+	{
+		if (data->colors[line_num][i] < 10)
+			*buffer++ = data->colors[line_num][i] + '0';
+		else
+			*buffer++ = 'E';
+	}
+	*buffer = 0;
+	return (unsigned)(buffer - orig); // Return length
+}
+static unsigned int get_font_encoded(struct encoder_ctx *ctx, unsigned char *buffer, int line_num, struct eia608_screen *data)
+{
+	unsigned char *orig = buffer; // Keep for debugging
+	for (int i = 0; i < 32; i++)
+	{
+		if(data->fonts[line_num][i] == FONT_REGULAR)
+			*buffer++ = 'R';
+		else if(data->fonts[line_num][i] == FONT_UNDERLINED_ITALICS)
+			*buffer++ = 'B';
+		else if(data->fonts[line_num][i] == FONT_UNDERLINED)
+			*buffer++ = 'U';
+		else if(data->fonts[line_num][i] == FONT_ITALICS)
+			*buffer++ = 'I';
+		else
+			*buffer++ = 'E';
+	}
+	return (unsigned)(buffer - orig); // Return length
+}
+int write_cc_buffer_as_g608(struct eia608_screen *data, struct encoder_ctx *context)
+{
+	int used;
+	unsigned h1,m1,s1,ms1;
+	unsigned h2,m2,s2,ms2;
+	LLONG ms_start, ms_end;
+	int wrote_something = 0;
+	ms_start = data->start_time;
+
+	ms_start+=context->subs_delay;
+	if (ms_start<0) // Drop screens that because of subs_delay start too early
+		return 0;
+
+	ms_end = data->end_time;
+
+	mstotime (ms_start,&h1,&m1,&s1,&ms1);
+	mstotime (ms_end-1,&h2,&m2,&s2,&ms2); // -1 To prevent overlapping with next line.
+	char timeline[128];
+	context->srt_counter++;
+	sprintf(timeline, "%u%s", context->srt_counter, context->encoded_crlf);
+	used = encode_line(context, context->buffer,(unsigned char *) timeline);
+	write(context->out->fh, context->buffer, used);
+	sprintf (timeline, "%02u:%02u:%02u,%03u --> %02u:%02u:%02u,%03u%s",
+		h1, m1, s1, ms1, h2, m2, s2, ms2, context->encoded_crlf);
+	used = encode_line(context, context->buffer,(unsigned char *) timeline);
+
+
+	write (context->out->fh, context->buffer, used);
+
+	for (int i=0;i<15;i++)
+	{
+		int length = get_line_encoded (context, context->subline, i, data);
+		write(context->out->fh, context->subline, length);
+
+		length = get_color_encoded (context, context->subline, i, data);
+		write(context->out->fh, context->subline, length);
+
+		length = get_font_encoded (context, context->subline, i, data);
+		write(context->out->fh, context->subline, length);
+		write(context->out->fh, context->encoded_crlf, context->encoded_crlf_length);
+		wrote_something=1;
+	}
+
+	write (context->out->fh, context->encoded_crlf, context->encoded_crlf_length);
+	return wrote_something;
+}

src/lib_ccx/ccx_encoders_helpers.c

@@ -40,7 +40,7 @@ int string_cmp(const void *p1, const void *p2)
 	return string_cmp2(p1, p2, NULL);
 }
 
-void correct_case(int line_num, struct eia608_screen *data)
+void correct_case_with_dictionary(int line_num, struct eia608_screen *data)
 {
 	char delim[64] = {
 		' ', '\n', '\r', 0x89, 0x99,
@@ -54,6 +54,11 @@ void correct_case(int line_num, struct eia608_screen *data)
 	char *line = strdup(((char*)data->characters[line_num]));
 	char *oline = (char*)data->characters[line_num];
 	char *c = strtok(line, delim);
+	if (c == NULL)
+	{
+		free(line);
+		return;
+	}
 	do
 	{
 		char **index = bsearch(&c, spell_lower, spell_words, sizeof(*spell_lower), string_cmp);
@@ -68,8 +73,61 @@ void correct_case(int line_num, struct eia608_screen *data)
 	free(line);
 }
 
-void capitalize(struct encoder_ctx *context, int line_num, struct eia608_screen *data)
+void telx_correct_case(char *sub_line)
 {
+	char delim[64] = {
+		' ', '\n', '\r', 0x89, 0x99,
+		'!', '"', '#', '%', '&',
+		'\'', '(', ')', ';', '<',
+		'=', '>', '?', '[', '\\',
+		']', '*', '+', ',', '-',
+		'.', '/', ':', '^', '_',
+		'{', '|', '}', '~', '\0' };
+
+	char *line = strdup(((char*)sub_line));
+	char *oline = (char*)sub_line;
+	char *c = strtok(line, delim);
+	if (c == NULL)
+	{
+		free(line);
+		return;
+	}
+	do
+	{
+		char **index = bsearch(&c, spell_lower, spell_words, sizeof(*spell_lower), string_cmp);
+
+		if (index)
+		{
+			char *correct_c = *(spell_correct + (index - spell_lower));
+			size_t len = strlen(correct_c);
+			memcpy(oline + (c - line), correct_c, len);
+		}
+	} while ((c = strtok(NULL, delim)) != NULL);
+	free(line);
+}
+
+int is_all_caps(struct encoder_ctx *context, int line_num, struct eia608_screen *data)
+{
+	int saw_upper = 0, saw_lower = 0;
+
+	for (int i = 0; i < CCX_DECODER_608_SCREEN_WIDTH; i++)
+	{
+		if (islower(data->characters[line_num][i]))
+			saw_lower = 1;
+		else if (isupper(data->characters[line_num][i]))
+			saw_upper = 1;
+	}
+	return (saw_upper && !saw_lower); // 1 if we've seen upper and not lower, 0 otherwise
+}
+
+int clever_capitalize(struct encoder_ctx *context, int line_num, struct eia608_screen *data)
+{
+	// CFS: Tried doing to clever (see below) but some channels do all uppercase except for
+	// notes for deaf people (such as "(narrator)" which messes things up.
+		// First find out if we actually need to do it, don't mess with lines that come OK
+		//int doit = is_all_caps(context, line_num, data);
+	int doit = 1;
+
 	for (int i = 0; i < CCX_DECODER_608_SCREEN_WIDTH; i++)
 	{
 		switch (data->characters[line_num][i])
@@ -85,14 +143,18 @@ void capitalize(struct encoder_ctx *context, int line_num, struct eia608_screen
 			context->new_sentence = 1;
 			break;
 		default:
-			if (context->new_sentence)
-				data->characters[line_num][i] = cctoupper(data->characters[line_num][i]);
-			else
-				data->characters[line_num][i] = cctolower(data->characters[line_num][i]);
+			if (doit)
+			{
+				if (context->new_sentence)
+					data->characters[line_num][i] = cctoupper(data->characters[line_num][i]);
+				else
+					data->characters[line_num][i] = cctolower(data->characters[line_num][i]);
+			}
 			context->new_sentence = 0;
 			break;
 		}
 	}
+	return doit;
 }
 
 // Encodes a generic string. Note that since we use the encoders for closed caption

src/lib_ccx/ccx_encoders_helpers.h

@@ -22,8 +22,10 @@ struct ccx_encoders_helpers_settings_t {
 };
 
 // Helper functions
-void correct_case(int line_num, struct eia608_screen *data);
-void capitalize(struct encoder_ctx *context, int line_num, struct eia608_screen *data);
+void correct_case_with_dictionary(int line_num, struct eia608_screen *data);
+int is_all_caps(struct encoder_ctx *context, int line_num, struct eia608_screen *data);
+int clever_capitalize(struct encoder_ctx *context, int line_num, struct eia608_screen *data);
+void telx_correct_case(char *sub_line);
 unsigned get_decoder_line_encoded_for_gui(unsigned char *buffer, int line_num, struct eia608_screen *data);
 unsigned get_decoder_line_encoded(struct encoder_ctx *ctx, unsigned char *buffer, int line_num, struct eia608_screen *data);
 

src/lib_ccx/ccx_encoders_spupng.c

@@ -318,7 +318,7 @@ int spupng_write_ccbuffer(struct spupng_t *sp, struct eia608_screen* data,
 				sp->pngfile, strerror(errno));
 	}
 	fclose(sp->fppng);
-	write_sputag(sp,ms_start,ms_end);
+	write_sputag_open(sp,ms_start,ms_end);
 	for (row = 0; row < ROWS; row++)
 	{
 		if (data->row_used[row])
@@ -355,6 +355,7 @@ int spupng_write_ccbuffer(struct spupng_t *sp, struct eia608_screen* data,
 	}
 
 	write_spucomment(sp,str);
+	write_sputag_close(sp);
 	return 1;
 }
 
@@ -385,8 +386,9 @@ int spupng_write_string(struct spupng_t *sp, char *string, LLONG start_time, LLO
 				sp->pngfile, strerror(errno));
 	}
 	fclose(sp->fppng);
-	write_sputag(sp,ms_start,ms_end);
+	write_sputag_open(sp,ms_start,ms_end);
 	write_spucomment(sp,str);
+	write_sputag_close(sp);
 	return 1;
 }
 int write_cc_subtitle_as_spupng(struct cc_subtitle *sub, struct encoder_ctx *context)

src/lib_ccx/ccx_encoders_srt.c

@@ -113,7 +113,7 @@ int write_cc_bitmap_as_srt(struct cc_subtitle *sub, struct encoder_ctx *context)
 	if(sub->flags & SUB_EOD_MARKER)
 		context->prev_start =  sub->start_time;
 
-	str = paraof_ocrtext(sub);
+	str = paraof_ocrtext(sub, context->encoded_crlf, context->encoded_crlf_length);
 	if (str)
 	{
 		if (context->prev_start != -1 || !(sub->flags & SUB_EOD_MARKER))
@@ -159,6 +159,7 @@ int write_cc_subtitle_as_srt(struct cc_subtitle *sub,struct encoder_ctx *context
 			ret = write_stringz_as_srt(sub->data, context, sub->start_time, sub->end_time);
 			freep(&sub->data);
 			sub->nb_data = 0;
+			ret = 1;
 		}
 		lsub = sub;
 		sub = sub->next;
@@ -222,8 +223,8 @@ int write_cc_buffer_as_srt(struct eia608_screen *data, struct encoder_ctx *conte
 		{
 			if (context->sentence_cap)
 			{
-				capitalize (context, i, data);
-				correct_case(i,data);
+				if (clever_capitalize (context, i, data))
+					correct_case_with_dictionary(i, data);
 			}
 			if (context->autodash && context->trim_subs)
 			{

src/lib_ccx/ccx_encoders_structs.h

@@ -16,8 +16,16 @@ typedef struct ccx_encoders_transcript_format {
 struct ccx_s_write
 {
 	int fh;
+	int temporarily_closed; // 1 means the file was created OK before but we released the handle
 	char *filename;
 	void* spupng_data;
+	int with_semaphore; // 1 means create a .sem file when the file is open and delete it when it's closed
+	char *semaphore_filename;
+	int with_playlist; // For m3u8 /webvtt: If 1, we'll generate a playlist and segments
+	char *playlist_filename; 
+	int renaming_extension; //Used for file rotations
+	int append_mode; /* Append the file. Prevent overwriting of files */
+
 };
 
 #endif

src/lib_ccx/ccx_encoders_webvtt.c

@@ -3,7 +3,7 @@
 #include "ccx_encoders_common.h"
 #include "ccx_encoders_helpers.h"
 #include "utility.h"
-
+#include "ocr.h"
 
 
 /* The timing here is not PTS based, but output based, i.e. user delay must be accounted for
@@ -94,13 +94,95 @@ int write_stringz_as_webvtt(char *string, struct encoder_ctx *context, LLONG ms_
 	return 0;
 }
 
+int write_xtimestamp_header(struct encoder_ctx *context)
+{
+	if (context->wrote_webvtt_sync_header) // Already done
+		return 1;
+	if (context->timing->sync_pts2fts_set)
+	{
+		char header_string[200];
+		int used;
+		unsigned h1, m1, s1, ms1;
+		mstotime(context->timing->sync_pts2fts_fts, &h1, &m1, &s1, &ms1);
+		sprintf(header_string, "X-TIMESTAMP-MAP=MPEGTS:%ld, LOCAL %02u:%02u:%02u.%03u\r\n",
+			context->timing->sync_pts2fts_pts, 
+			h1, m1, s1, ms1);
+		used = encode_line(context, context->buffer, (unsigned char *)header_string);
+		write(context->out->fh, context->buffer, used);
+
+	}
+	// Add the additional CRLF to finish the header
+	write(context->out->fh, context->encoded_crlf, context->encoded_crlf_length);
+	context->wrote_webvtt_sync_header = 1; // Do it even if couldn't write the header, because it won't be possible anyway
+}
+
+
 int write_cc_bitmap_as_webvtt(struct cc_subtitle *sub, struct encoder_ctx *context)
 {
-	//TODO
 	int ret = 0;
+#ifdef ENABLE_OCR
+	struct cc_bitmap* rect;
+	LLONG ms_start, ms_end;
+	unsigned h1, m1, s1, ms1;
+	unsigned h2, m2, s2, ms2;
+	char timeline[128];
+	int len = 0;
+	int used;
+	int i = 0;
+	char *str;
+
+	if (context->prev_start != -1 && (sub->flags & SUB_EOD_MARKER))
+	{
+		ms_start = context->prev_start;
+		ms_end = sub->start_time;
+	}
+	else if (!(sub->flags & SUB_EOD_MARKER))
+	{
+		ms_start = sub->start_time;
+		ms_end = sub->end_time;
+	}
+	else if (context->prev_start == -1 && (sub->flags & SUB_EOD_MARKER))
+	{
+		ms_start = 1;
+		ms_end = sub->start_time;
+	}
+
+	if (sub->nb_data == 0)
+		return 0;
+
+	write_xtimestamp_header(context);
+
+	if (sub->flags & SUB_EOD_MARKER)
+		context->prev_start = sub->start_time;
+
+	str = paraof_ocrtext(sub, context->encoded_crlf, context->encoded_crlf_length);
+	if (str)
+	{
+		if (context->prev_start != -1 || !(sub->flags & SUB_EOD_MARKER))
+		{
+			mstotime(ms_start, &h1, &m1, &s1, &ms1);
+			mstotime(ms_end - 1, &h2, &m2, &s2, &ms2); // -1 To prevent overlapping with next line.
+			context->srt_counter++; // Not needed for WebVTT but let's keep it around for now
+			sprintf(timeline, "%02u:%02u:%02u.%03u --> %02u:%02u:%02u.%03u%s",
+				h1, m1, s1, ms1, h2, m2, s2, ms2, context->encoded_crlf);
+			used = encode_line(context, context->buffer, (unsigned char *)timeline);
+			write(context->out->fh, context->buffer, used);
+			len = strlen(str);
+			write(context->out->fh, str, len);
+			write(context->out->fh, context->encoded_crlf, context->encoded_crlf_length);
+		}
+		freep(&str);
+	}
+	for (i = 0, rect = sub->data; i < sub->nb_data; i++, rect++)
+	{
+		freep(rect->data);
+		freep(rect->data + 1);
+	}
+#endif
 	sub->nb_data = 0;
 	freep(&sub->data);
 	return ret;
+
 }
 
 int write_cc_subtitle_as_webvtt(struct cc_subtitle *sub, struct encoder_ctx *context)
@@ -158,6 +240,8 @@ int write_cc_buffer_as_webvtt(struct eia608_screen *data, struct encoder_ctx *co
 	if (ms_start<0) // Drop screens that because of subs_delay start too early
 		return 0;
 
+	write_xtimestamp_header(context);
+
 	ms_end = data->end_time;
 
 	mstotime(ms_start, &h1, &m1, &s1, &ms1);
@@ -180,8 +264,8 @@ int write_cc_buffer_as_webvtt(struct eia608_screen *data, struct encoder_ctx *co
 		{
 			if (context->sentence_cap)
 			{
-				capitalize(context, i, data);
-				correct_case(i, data);
+				if (clever_capitalize(context, i, data))
+					correct_case_with_dictionary(i, data);
 			}
 			if (context->autodash && context->trim_subs)
 			{

src/lib_ccx/ccx_gxf.h

@@ -0,0 +1,9 @@
+#ifndef CCX_GXF
+#define CCX_GXF
+
+#include "ccx_demuxer.h"
+
+int ccx_gxf_probe(unsigned char *buf, int len);
+int ccx_gxf_getmoredata(struct ccx_demuxer *ctx, struct demuxer_data **data);
+struct ccx_gxf *ccx_gxf_init(struct ccx_demuxer *arg);
+#endif

src/lib_ccx/compile_info.h

@@ -0,0 +1,9 @@
+#ifndef CCX_CCEXTRACTOR_COMPILE_H
+#define CCX_CCEXTRACTOR_COMPILE_H
+#ifndef VERSION_FILE_PRESENT
+#define GIT_COMMIT "Unknown"
+#define COMPILE_DATE "Unknown"
+#else
+#include "compile_info_real.h"
+#endif
+#endif

src/lib_ccx/dvb_subtitle_decoder.c

@@ -444,11 +444,6 @@ void* dvbsub_init_decoder(struct dvb_config* cfg)
 
 #ifdef ENABLE_OCR
 	ctx->ocr_ctx = init_ocr(ctx->lang_index);
-        if(!ctx->ocr_ctx)
-	{
-		freep(&ctx);
-		return NULL;
-	}
 #endif
 	ctx->version = -1;
 
@@ -552,7 +547,8 @@ int dvbsub_close_decoder(void **dvb_ctx)
 	}
 
 #ifdef ENABLE_OCR
-	delete_ocr(&ctx->ocr_ctx);
+	if (ctx->ocr_ctx)
+		delete_ocr(&ctx->ocr_ctx);
 #endif
 	freep(dvb_ctx);
 	return 0;
@@ -1537,9 +1533,18 @@ static int write_dvb_sub(struct lib_cc_decode *dec_ctx, struct cc_subtitle *sub)
 		rect->data[0] = malloc(region->buf_size);
 		memcpy(rect->data[0], region->pbuf, region->buf_size);
 #ifdef ENABLE_OCR
-		ret = ocr_rect(ctx->ocr_ctx, rect, &ocr_str);
-		if(ret >= 0)
-			rect->ocr_text = ocr_str;
+		if (ctx->ocr_ctx)
+		{
+			ret = ocr_rect(ctx->ocr_ctx, rect, &ocr_str,region->bgcolor);
+			if(ret >= 0)
+				rect->ocr_text = ocr_str;
+			else
+				rect->ocr_text = NULL;
+		}
+		else
+		{
+			rect->ocr_text = NULL;
+		}
 #endif
 		rect++;
 
@@ -1690,9 +1695,15 @@ int parse_dvb_description(struct dvb_config* cfg, unsigned char*data,
 	for (i = 0; i < cfg->n_language; i++, data += i * 8)
 	{
 		/* setting language to undefined if not found in language lkup table */
+		char lang_name[4];
+		for(int char_index = 0; char_index < 3; char_index++)
+		{
+			lang_name[char_index] = cctolower(data[char_index]);
+		}
+
 		for (j = 0, cfg->lang_index[i] = 0; language[j] != NULL; j++)
 		{
-			if (!strncmp((const char*) (data), language[j], 3))
+			if (!strncmp(lang_name, language[j], 3))
 				cfg->lang_index[i] = j;
 		}
 		cfg->sub_type[i] = data[3];
@@ -1701,5 +1712,21 @@ int parse_dvb_description(struct dvb_config* cfg, unsigned char*data,
 
 	}
 
+	/*
+		Abhinav95: The way this function is called right now, only cfg->lang_index[0]
+		gets populated. E.g. for 3 stream languages, it will be called 3 times, and
+		set the language index in only the first element each time. This works with the
+		current state of the DVB code.
+	*/
+	if(ccx_options.dvblang)
+	{
+		if(strcmp(ccx_options.dvblang, language[cfg->lang_index[0]])!=0)
+		{
+			mprint("Ignoring stream language '%s' not equal to dvblang '%s'\n",
+				language[cfg->lang_index[0]], ccx_options.dvblang);
+			return -1;
+		}
+	}
+
 	return 0;
 }

src/lib_ccx/es_userdata.c

@@ -1,4 +1,5 @@
 #include "lib_ccx.h"
+#include "ccx_decoders_vbi.h"
 
 
 // Parse the user data for captions. The udtype variable denotes
@@ -23,7 +24,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 	}
 
 	// Do something
-	//ctx->stat_numuserheaders++;
+	ctx->stat_numuserheaders++;
 	//header+=4;
 
 	unsigned char *ud_header = next_bytes(ustream, 4);
@@ -35,10 +36,10 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 	}
 
 	// DVD CC header, see
-	// <http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/SCC_FORMAT.HTML>
+	// <http://www.theneitherworld.com/mcpoodle/SCC_TOOLS/DOCS/SCC_FORMAT.HTML>
 	if ( !memcmp(ud_header,"\x43\x43", 2 ) )
 	{
-//		ctx->stat_dvdccheaders++;
+		ctx->stat_dvdccheaders++;
 
 		// Probably unneeded, but keep looking for extra caption blocks
 		int maybeextracb = 1;
@@ -138,13 +139,13 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 		dbg_print(CCX_DMT_VERBOSE, "Read %d/%d DVD CC blocks\n", rcbcount, ecbcount);
 	}
 	// SCTE 20 user data
-	else if (ud_header[0] == 0x03)
+	else if (!ctx->noscte20 && ud_header[0] == 0x03)
 	{
 		if ((ud_header[1]&0x7F) == 0x01)
 		{
 			unsigned char cc_data[3*31+1]; // Maximum cc_count is 31
 
-//			ctx->stat_scte20ccheaders++;
+			ctx->stat_scte20ccheaders++;
 			read_bytes(ustream, 2); // "03 01"
 
 			unsigned cc_count = (unsigned int) read_bits(ustream,5);
@@ -153,7 +154,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 			unsigned field_number;
 			unsigned cc_data1;
 			unsigned cc_data2;
-			/* unsigned marker; */
+            
 			for (unsigned j=0;j<cc_count;j++)
 			{
 				skip_bits(ustream,2); // priority - unused
@@ -161,7 +162,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 				skip_bits(ustream,5); // line_offset - unused
 				cc_data1 = (unsigned int) read_bits(ustream,8);
 				cc_data2 = (unsigned int) read_bits(ustream,8);
-				/* marker = (unsigned int)read_bits(ustream,1); // TODO: Add syntax check */
+				read_bits(ustream,1); // TODO: Add syntax check */
 
 				if (ustream->bitsleft < 0)
 					fatal(CCX_COMMON_EXIT_BUG_BUG, "Oops!");
@@ -205,12 +206,12 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 			&& ud_header[1] == 0x02 )
 	{
 		unsigned char data[3];
-#if 0
+
 		if (ud_header[0]==0xbb)
 			ctx->stat_replay4000headers++;
 		else
 			ctx->stat_replay5000headers++;
-#endif
+
 
 		read_bytes(ustream, 2); // "BB 02" or "99 02"
 		data[0]=0x05; // Field 2
@@ -226,7 +227,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 	// HDTV - see A/53 Part 4 (Video)
 	else if ( !memcmp(ud_header,"\x47\x41\x39\x34", 4 ) )
 	{
-//		ctx->stat_hdtv++;
+		ctx->stat_hdtv++;
 
 		read_bytes(ustream, 4); // "47 41 39 34"
 
@@ -287,7 +288,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 
 		dbg_print(CCX_DMT_VERBOSE, "Reading Dish Network user data\n");
 
-//		ctx->stat_dishheaders++;
+		ctx->stat_dishheaders++;
 
 		read_bytes(ustream, 2); // "05 02"
 
@@ -452,7 +453,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 	else if ( !memcmp(ud_header,"\x02\x09", 2 ) )
 	{
 		// Either a documentation or more examples are needed.
-//		ctx->stat_divicom++;
+		ctx->stat_divicom++;
 
 		unsigned char data[3];
 
@@ -465,6 +466,31 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 		do_cb(ctx, data, sub);
 		// This is probably incomplete!
 	}
+	// GXF vbi OEM code
+	else if ( !memcmp(ud_header,"\x73\x52\x21\x06", 4 ) )
+	{
+		int udatalen = ustream->end - ustream->pos;
+		uint16_t line_nb;
+		uint8_t line_type;
+		uint8_t field = 1;
+		read_bytes(ustream, 4); //skip header code
+		read_bytes(ustream, 2); //skip data length 
+		line_nb = read_bits(ustream, 16);
+		line_type = read_u8(ustream);
+		field = (line_type & 0x03);
+		if(field == 0)
+			mprint("MPEG:VBI: Invalid field\n");
+
+		line_type = line_type >> 2;
+		if(line_type != 1)
+			mprint("MPEG:VBI: only support Luma line\n");
+
+		if (udatalen < 720)
+			mprint("MPEG:VBI: Minimum 720 bytes in luma line required\n");
+
+		decode_vbi(ctx, field, ustream->pos, 720, sub); 
+		dbg_print(CCX_DMT_VERBOSE, "GXF (vbi line %d) user data:\n", line_nb);
+	}
 	else
 	{
 		// Some other user data

src/lib_ccx/ffmpeg_intgr.c

@@ -21,6 +21,7 @@ struct ffmpeg_ctx
 	AVFrame *frame;
 	int stream_index;
 };
+
 /**
  * call back function to be registered for avlog
  */
@@ -52,13 +53,14 @@ static void log_cb(void* ptr, int level, const char* fmt, va_list vl)
 
 	vfprintf(flog, fmt, vl);
 }
+
 /**
  * @path this path could be relative or absolute path of static file
  * 	 this path could be path of device
  *
  * @return ctx Context of ffmpeg
  */
-void *init_ffmpeg(char *path)
+void *init_ffmpeg(const char *path)
 {
 	int ret = 0;
 	int stream_index = 0;
@@ -120,6 +122,7 @@ void *init_ffmpeg(char *path)
 fail:
 	return ctx;
 }
+
 /**
  * @param ctx context of ffmpeg
  * @param data preallocated buffer where data will be recieved
@@ -158,10 +161,10 @@ int ff_get_ccframe(void *arg, unsigned char*data, int maxlen)
 	{
 		return AVERROR(EAGAIN);
 	}
-	current_pts = av_frame_get_best_effort_timestamp(ctx->frame);
-	if(!pts_set)
-		pts_set = 1;
-	set_fts();
+//	current_pts = av_frame_get_best_effort_timestamp(ctx->frame);
+//	if(!pts_set)
+//		pts_set = 1;
+//	set_fts();
 	for(int i = 0;i< ctx->frame->nb_side_data;i++)
 	{
 		if(ctx->frame->side_data[i]->type == AV_FRAME_DATA_A53_CC)
@@ -184,4 +187,56 @@ int ff_get_ccframe(void *arg, unsigned char*data, int maxlen)
 
 	return len;
 }
+
+int ffmpeg_getmoredata(struct ccx_demuxer *ctx, struct demuxer_data **ppdata)
+{
+	struct demuxer_data *data;
+	int ret = 0;
+    if(!*ppdata)
+    {
+        *ppdata = alloc_demuxer_data();
+        if(!*ppdata)
+            return -1;
+        data = *ppdata;
+        //TODO Set to dummy, find and set actual value
+        data->program_number = 1;
+        data->stream_pid = 1;
+        data->codec  = CCX_CODEC_ATSC_CC;
+		data->bufferdatatype = CCX_RAW_TYPE;
+    }
+    else
+    {
+        data = *ppdata;
+    }
+
+	do
+	{
+		int len = ff_get_ccframe(ctx->ffmpeg_ctx, data->buffer, BUFSIZE);
+		if(len == AVERROR(EAGAIN))
+		{
+			continue;
+		}
+		else if(len == AVERROR_EOF)
+		{
+			ret = CCX_EOF;
+			break;
+		}
+		else if(len == 0)
+			continue;
+		else if(len < 0 )
+		{
+			mprint("Error extracting Frame\n");
+			break;
+
+		}
+		else
+		{
+			data->len = len;
+			break;
+		}
+
+	}while(1);
+
+	return ret;
+}
 #endif

src/lib_ccx/ffmpeg_intgr.h

@@ -5,19 +5,14 @@
 #include "libavutil/common.h"
 #include "libavutil/error.h"
 #endif
+#include "ccx_demuxer.h"
 /**
  * @path this path could be relative or absolute path of static file
  * 	 this path could be path of device
  *
  * @return ctx Context of ffmpeg
  */
-void *init_ffmpeg(char *path);
+void *init_ffmpeg(const char *path);
 
-/**
- * @param ctx context of ffmpeg
- * @param data preallocated buffer where data will be recieved
- * @param maxlen length of buffer, where data will be copied
- * @return number of bytes recieved as data
- */
-int ff_get_ccframe(void *arg, unsigned char*data, int maxlen);
+int ffmpeg_getmoredata(struct ccx_demuxer *ctx, struct demuxer_data **ppdata);
 #endif

src/lib_ccx/file_buffer.h

@@ -89,4 +89,6 @@ static size_t inline buffered_read_byte(struct ccx_demuxer *ctx, unsigned char *
 unsigned short buffered_get_be16(struct ccx_demuxer *ctx);
 unsigned char buffered_get_byte (struct ccx_demuxer *ctx);
 unsigned int buffered_get_be32(struct ccx_demuxer *ctx);
+unsigned short buffered_get_le16(struct ccx_demuxer *ctx);
+unsigned int buffered_get_le32(struct ccx_demuxer *ctx);
 #endif

src/lib_ccx/file_functions.c

@@ -37,11 +37,23 @@ LLONG gettotalfilessize (struct lib_ccx_ctx *ctx) // -1 if one or more files fai
 #else
 		h = OPEN (ctx->inputfile[i], O_RDONLY);
 #endif
-		if (h == -1)
-		{
-			mprint ("\rUnable to open %s\r\n", ctx->inputfile[i]);
-			return -1;
+
+		if (h == -1) {
+			switch (errno)
+			{
+			case ENOENT:
+				return -1 * ENOENT;
+			case EACCES:
+				return -1 * EACCES;
+			case EINVAL:
+				return -1 * EINVAL;
+			case EMFILE:
+				return -1 * EMFILE;
+			default:
+				return -1;
+			}
 		}
+
 		if (!ccx_options.live_stream)
 			ts += getfilesize (h);
 		close (h);
@@ -71,18 +83,21 @@ void prepare_for_new_file (struct lib_ccx_ctx *ctx)
 	pts_big_change              = 0;
 	firstcall                   = 1;
 
-	for(int x = 0; x < 0xfff; x++)
+	if(ctx->epg_inited)
 	{
-		ctx->epg_buffers[x].buffer   = NULL;
-		ctx->epg_buffers[x].ccounter = 0;
+		for(int x = 0; x < 0xfff; x++)
+		{
+			ctx->epg_buffers[x].buffer   = NULL;
+			ctx->epg_buffers[x].ccounter = 0;
+		}
+		for (int i = 0; i < TS_PMT_MAP_SIZE; i++)
+		{
+			ctx->eit_programs[i].array_len = 0;
+			ctx->eit_current_events[i] = -1;
+		}
+		ctx->epg_last_output      = -1;
+		ctx->epg_last_live_output = -1;
 	}
-	for (int i = 0; i < TS_PMT_MAP_SIZE; i++)
-	{
-		ctx->eit_programs[i].array_len = 0;
-		ctx->eit_current_events[i] = -1;
-	}
-	ctx->epg_last_output      = -1;
-	ctx->epg_last_live_output = -1;
 }
 
 /* Close input file if there is one and let the GUI know */
@@ -123,6 +138,7 @@ int switch_to_next_file (struct lib_ccx_ctx *ctx, LLONG bytesinbuffer)
 	/* Close current and make sure things are still sane */
 	if (ctx->demux_ctx->is_open(ctx->demux_ctx))
 	{
+		dbg_print(CCX_DMT_708, "[CEA-708] The 708 decoder was reset [%d] times.\n", ctx->freport.data_from_708->reset_count);
 		if (ccx_options.print_file_reports)
 			print_file_report(ctx);
 
@@ -286,6 +302,7 @@ void return_to_buffer (struct ccx_demuxer *ctx, unsigned char *buffer, unsigned
  */
 size_t buffered_read_opt (struct ccx_demuxer *ctx, unsigned char *buffer, size_t bytes)
 {
+	size_t origin_buffer_size = bytes;
 	size_t copied   = 0;
 	time_t seconds = 0;
 
@@ -378,7 +395,7 @@ size_t buffered_read_opt (struct ccx_demuxer *ctx, unsigned char *buffer, size_t
 				{
 					/* If live stream, don't try to switch - acknowledge eof here as it won't
 					   cause a loop end */
-					if (ccx_options.live_stream || !(ccx_options.binary_concat && switch_to_next_file(ctx->parent, copied)))
+					if (ccx_options.live_stream || ((struct lib_ccx_ctx *)ctx->parent)->inputsize <= origin_buffer_size || !(ccx_options.binary_concat && switch_to_next_file(ctx->parent, copied)))
 						eof = 1;
 				}
 				ctx->filebuffer_pos = keep;
@@ -484,3 +501,23 @@ unsigned int buffered_get_be32(struct ccx_demuxer *ctx)
 	val |= buffered_get_be16(ctx);
 	return val;
 }
+
+unsigned short buffered_get_le16(struct ccx_demuxer *ctx)
+{
+	unsigned char a,b;
+	unsigned char *a_p = &a; // Just to suppress warnings
+	unsigned char *b_p = &b;
+	buffered_read_byte(ctx, a_p);
+	ctx->past++;
+	buffered_read_byte(ctx, b_p);
+	ctx->past++;
+	return ( (unsigned short) (b<<8) )| ( (unsigned short) a);
+}
+
+unsigned int buffered_get_le32(struct ccx_demuxer *ctx)
+{
+	unsigned int val;
+	val = buffered_get_le16(ctx);
+	val |= buffered_get_le16(ctx) << 16;
+	return val;
+}

src/lib_ccx/general_loop.c

@@ -13,6 +13,8 @@
 #include "ccx_demuxer.h"
 #include "file_buffer.h"
 #include "ccx_decoders_isdb.h"
+#include "ffmpeg_intgr.h"
+#include "ccx_gxf.h"
 
 unsigned int rollover_bits = 0; // The PTS rolls over every 26 hours and that can happen in the middle of a stream.
 int end_of_file=0; // End of file?
@@ -580,7 +582,7 @@ int process_data(struct encoder_ctx *enc_ctx, struct lib_cc_decode *dec_ctx, str
 	else if (data_node->bufferdatatype == CCX_TELETEXT)
 	{
 		//telxcc_update_gt(dec_ctx->private_data, ctx->demux_ctx->global_timestamp);
-		ret = tlt_process_pes_packet (dec_ctx, data_node->buffer, data_node->len, dec_sub);
+		ret = tlt_process_pes_packet (dec_ctx, data_node->buffer, data_node->len, dec_sub, enc_ctx->sentence_cap);
 		if(ret == CCX_EINVAL)
 			return ret;
 		got = data_node->len;
@@ -632,6 +634,10 @@ int process_data(struct encoder_ctx *enc_ctx, struct lib_cc_decode *dec_ctx, str
 		dec_ctx->in_bufferdatatype = CCX_H264;
 		got = process_avc(dec_ctx, data_node->buffer, data_node->len, dec_sub);
 	}
+	else if (data_node->bufferdatatype == CCX_RAW_TYPE)
+	{
+		got = process_raw_with_field(dec_ctx, dec_sub, data_node->buffer, data_node->len);
+	}
 	else if (data_node->bufferdatatype == CCX_ISDB_SUBTITLE)
 	{
 		isdbsub_decode(dec_ctx, data_node->buffer, data_node->len, dec_sub);
@@ -739,6 +745,14 @@ void general_loop(struct lib_ccx_ctx *ctx)
 			case CCX_SM_WTV:
 				ret = wtv_getmoredata(ctx, &datalist);
 				break;
+			case CCX_SM_GXF:
+				ret = ccx_gxf_getmoredata(ctx->demux_ctx, &datalist);
+				break;
+#ifdef ENABLE_FFMPEG
+			case CCX_SM_FFMPEG:
+				ret = ffmpeg_getmoredata(ctx->demux_ctx, &datalist);
+				break;
+#endif
 			default:
 				fatal(CCX_COMMON_EXIT_BUG_BUG, "Impossible stream_mode");
 		}
@@ -773,15 +787,37 @@ void general_loop(struct lib_ccx_ctx *ctx)
 			enc_ctx = update_encoder_list_cinfo(ctx, cinfo);
 			dec_ctx = update_decoder_list_cinfo(ctx, cinfo);
 			dec_ctx->dtvcc->encoder = (void *)enc_ctx; //WARN: otherwise cea-708 will not work
+			enc_ctx->timing = dec_ctx->timing;
+			
 			if(data_node->pts != CCX_NOPTS)
-				set_current_pts(dec_ctx->timing, data_node->pts);
-
-			if( data_node->bufferdatatype == CCX_ISDB_SUBTITLE && ctx->demux_ctx->global_timestamp_inited)
 			{
-				uint64_t tstamp = ( ctx->demux_ctx->global_timestamp + ctx->demux_ctx->offset_global_timestamp )
-							- ctx->demux_ctx->min_global_timestamp;
+				struct ccx_rational tb = {1,MPEG_CLOCK_FREQ};
+				LLONG pts;
+				if(data_node->tb.num != 1 || data_node->tb.den != MPEG_CLOCK_FREQ)
+				{
+					pts = change_timebase(data_node->pts, data_node->tb, tb);
+				}
+				else
+					pts = data_node->pts;
+				set_current_pts(dec_ctx->timing, pts);
+				set_fts(dec_ctx->timing);
+			}
+
+			if( data_node->bufferdatatype == CCX_ISDB_SUBTITLE)
+			{
+				uint64_t tstamp;
+				if(ctx->demux_ctx->global_timestamp_inited)
+				{
+					tstamp = ( ctx->demux_ctx->global_timestamp + ctx->demux_ctx->offset_global_timestamp )
+								- ctx->demux_ctx->min_global_timestamp;
+				}
+				else
+				{
+					// Fix if global timestamp not inited
+					tstamp = get_fts(dec_ctx->timing, dec_ctx->current_field);
+				}
 				isdb_set_global_time(dec_ctx, tstamp);
-                        }
+			}
 
 			ret = process_data(enc_ctx, dec_ctx, data_node);
 			if( ret != CCX_OK)
@@ -810,6 +846,7 @@ void general_loop(struct lib_ccx_ctx *ctx)
 				enc_ctx = update_encoder_list_cinfo(ctx, cinfo);
 				dec_ctx = update_decoder_list_cinfo(ctx, cinfo);
 				dec_ctx->dtvcc->encoder = (void *)enc_ctx; //WARN: otherwise cea-708 will not work
+				dec_ctx->timing = enc_ctx->timing;
 				if(data_node->pts != CCX_NOPTS)
 					set_current_pts(dec_ctx->timing, data_node->pts);
 				process_data(enc_ctx, dec_ctx, data_node);
@@ -853,7 +890,13 @@ void general_loop(struct lib_ccx_ctx *ctx)
 			telxcc_close(&dec_ctx->private_data, &dec_ctx->dec_sub);
 		// Flush remaining HD captions
 		if (dec_ctx->has_ccdata_buffered)
-                	process_hdcc(dec_ctx, &dec_ctx->dec_sub);
+					process_hdcc(dec_ctx, &dec_ctx->dec_sub);
+		
+	mprint ("\nNumber of NAL_type_7: %ld\n",dec_ctx->avc_ctx->num_nal_unit_type_7);
+	mprint ("Number of VCL_HRD: %ld\n",dec_ctx->avc_ctx->num_vcl_hrd);
+	mprint ("Number of NAL HRD: %ld\n",dec_ctx->avc_ctx->num_nal_hrd);
+	mprint ("Number of jump-in-frames: %ld\n",dec_ctx->avc_ctx->num_jump_in_frames);
+	mprint ("Number of num_unexpected_sei_length: %ld", dec_ctx->avc_ctx->num_unexpected_sei_length);
 	}
 
 	delete_datalist(datalist);
@@ -915,6 +958,7 @@ void rcwt_loop(struct lib_ccx_ctx *ctx)
 	dec_ctx = update_decoder_list(ctx);
 	if (parsebuf[6] == 0 && parsebuf[7] == 2)
 	{
+		dec_ctx->codec = CCX_CODEC_TELETEXT;
 		dec_ctx->private_data = telxcc_init();
 	}
 	dec_sub = &dec_ctx->dec_sub;

src/lib_ccx/hardsubx.c

@@ -0,0 +1,291 @@
+#ifdef ENABLE_HARDSUBX
+#include "hardsubx.h"
+#include "capi.h"
+#include "allheaders.h"
+#include "ocr.h"
+#include "utility.h"
+
+//TODO: Correct FFMpeg integration
+#include <libavcodec/avcodec.h>
+#include <libavformat/avformat.h>
+#include <libavutil/imgutils.h>
+#include <libswscale/swscale.h>
+
+int hardsubx_process_data(struct lib_hardsubx_ctx *ctx)
+{
+	// Get the required media attributes and initialize structures
+	av_register_all();
+	
+	if(avformat_open_input(&ctx->format_ctx, ctx->inputfile[0], NULL, NULL)!=0)
+	{
+		fatal (EXIT_READ_ERROR, "Error reading input file!\n");
+	}
+
+	if(avformat_find_stream_info(ctx->format_ctx, NULL)<0)
+	{
+		fatal (EXIT_READ_ERROR, "Error reading input stream!\n");
+	}
+
+	// Important call in order to determine media information using ffmpeg
+	// TODO: Handle multiple inputs
+	av_dump_format(ctx->format_ctx, 0, ctx->inputfile[0], 0);
+	
+
+	ctx->video_stream_id = -1;
+	for(int i = 0; i < ctx->format_ctx->nb_streams; i++)
+	{
+		if(ctx->format_ctx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
+		{
+			ctx->video_stream_id = i;
+			break;
+		}
+	}
+	if(ctx->video_stream_id == -1)
+	{
+		fatal (EXIT_READ_ERROR, "Video Stream not found!\n");
+	}
+
+	ctx->codec_ctx = ctx->format_ctx->streams[ctx->video_stream_id]->codec;
+	ctx->codec = avcodec_find_decoder(ctx->codec_ctx->codec_id);
+	if(ctx->codec == NULL)
+	{
+		fatal (EXIT_READ_ERROR, "Input codec is not supported!\n");
+	}
+
+	if(avcodec_open2(ctx->codec_ctx, ctx->codec, &ctx->options_dict) < 0)
+	{
+		fatal (EXIT_READ_ERROR, "Error opening input codec!\n");
+	}
+
+	ctx->frame = av_frame_alloc();
+	ctx->rgb_frame = av_frame_alloc();
+	if(!ctx->frame || !ctx->rgb_frame)
+	{
+		fatal(EXIT_NOT_ENOUGH_MEMORY, "Not enough memory to initialize frame!");
+	}
+
+	int frame_bytes = av_image_get_buffer_size(AV_PIX_FMT_RGB24, ctx->codec_ctx->width, ctx->codec_ctx->height, 16);
+	ctx->rgb_buffer = (uint8_t *)av_malloc(frame_bytes*sizeof(uint8_t));
+	
+	ctx->sws_ctx = sws_getContext(
+			ctx->codec_ctx->width,
+			ctx->codec_ctx->height,
+			ctx->codec_ctx->pix_fmt,
+			ctx->codec_ctx->width,
+			ctx->codec_ctx->height,
+			AV_PIX_FMT_RGB24,
+			SWS_BILINEAR,
+			NULL,NULL,NULL
+		);
+
+	av_image_fill_arrays(ctx->rgb_frame->data, ctx->rgb_frame->linesize, ctx->rgb_buffer, AV_PIX_FMT_RGB24, ctx->codec_ctx->width, ctx->codec_ctx->height, 1);
+
+	// int frame_bytes = av_image_get_buffer_size(AV_PIX_FMT_RGB24, 1280, 720, 16);
+	// ctx->rgb_buffer = (uint8_t *)av_malloc(frame_bytes*sizeof(uint8_t));
+	
+	// ctx->sws_ctx = sws_getContext(
+	// 		ctx->codec_ctx->width,
+	// 		ctx->codec_ctx->height,
+	// 		ctx->codec_ctx->pix_fmt,
+	// 		1280,
+	// 		720,
+	// 		AV_PIX_FMT_RGB24,
+	// 		SWS_BILINEAR,
+	// 		NULL,NULL,NULL
+	// 	);
+	// avpicture_fill((AVPicture*)ctx->rgb_frame, ctx->rgb_buffer, AV_PIX_FMT_RGB24, 1280, 720);
+	// av_image_fill_arrays(ctx->rgb_frame->data, ctx->rgb_frame->linesize, ctx->rgb_buffer, AV_PIX_FMT_RGB24, 1280, 720, 1);
+
+	// Pass on the processing context to the appropriate functions
+	struct encoder_ctx *enc_ctx;
+	enc_ctx = init_encoder(&ccx_options.enc_cfg);
+	
+	mprint("Beginning burned-in subtitle detection...\n");
+	hardsubx_process_frames_linear(ctx, enc_ctx);
+
+	dinit_encoder(&enc_ctx, 0); //TODO: Replace 0 with end timestamp
+
+	// Free the allocated memory for frame processing
+	av_free(ctx->rgb_buffer);
+	av_free(ctx->rgb_frame);
+	av_free(ctx->frame);
+	avcodec_close(ctx->codec_ctx);
+	avformat_close_input(&ctx->format_ctx);
+}
+
+void _hardsubx_params_dump(struct ccx_s_options *options, struct lib_hardsubx_ctx *ctx)
+{
+	// Print the relevant passed parameters onto the screen
+	mprint("Input : %s\n", ctx->inputfile[0]);
+
+	switch(ctx->subcolor)
+	{
+		case HARDSUBX_COLOR_WHITE:
+			mprint("Subtitle Color : White\n");
+			break;
+		case HARDSUBX_COLOR_YELLOW:
+			mprint("Subtitle Color : Yellow\n");
+			break;
+		case HARDSUBX_COLOR_GREEN:
+			mprint("Subtitle Color : Green\n");
+			break;
+		case HARDSUBX_COLOR_CYAN:
+			mprint("Subtitle Color : Cyan\n");
+			break;
+		case HARDSUBX_COLOR_BLUE:
+			mprint("Subtitle Color : Blue\n");
+			break;
+		case HARDSUBX_COLOR_MAGENTA:
+			mprint("Subtitle Color : Magenta\n");
+			break;
+		case HARDSUBX_COLOR_RED:
+			mprint("Subtitle Color : Red\n");
+			break;
+		case HARDSUBX_COLOR_CUSTOM:
+			mprint("Subtitle Color : Custom Hue - %0.2f\n",ctx->hue);
+			break;
+		default:
+			fatal(EXIT_MALFORMED_PARAMETER,"Invalid Subtitle Color");
+	}
+
+	switch(ctx->ocr_mode)
+	{
+		case HARDSUBX_OCRMODE_WORD:
+			mprint("OCR Mode : Word-wise\n");
+			break;
+		case HARDSUBX_OCRMODE_LETTER:
+			mprint("OCR Mode : Letter-wise\n");
+			break;
+		case HARDSUBX_OCRMODE_FRAME:
+			mprint("OCR Mode : Frame-wise (simple)\n");
+			break;
+		default:
+			fatal(EXIT_MALFORMED_PARAMETER,"Invalid OCR Mode");
+	}
+
+	if(ctx->conf_thresh > 0)
+	{
+		mprint("OCR Confidence Threshold : %.2f\n",ctx->conf_thresh);
+	}
+	else
+	{
+		mprint("OCR Confidence Threshold : %.2f (Default)\n",ctx->conf_thresh);
+	}
+
+	if(ctx->subcolor == HARDSUBX_COLOR_WHITE)
+	{
+		if(ctx->lum_thresh != 95.0)
+		{
+			mprint("OCR Luminance Threshold : %.2f\n",ctx->lum_thresh);
+		}
+		else
+		{
+			mprint("OCR Luminance Threshold : %.2f (Default)\n",ctx->lum_thresh);
+		}
+	}
+
+	if(ctx->detect_italics == 1)
+	{
+		mprint("OCR Italic Detection : On\n");
+	}
+	else
+	{
+		mprint("OCR Italic Detection : Off\n");
+	}
+
+	if(ctx->min_sub_duration == 0.5)
+	{
+		mprint("Minimum subtitle duration : 0.5 seconds (Default)\n");
+	}
+	else
+	{
+		mprint("Minimum subtitle duration : %0.2f seconds (Default)\n",ctx->min_sub_duration);
+	}
+
+	mprint("FFMpeg Media Information:-\n");
+
+}
+
+struct lib_hardsubx_ctx* _init_hardsubx(struct ccx_s_options *options)
+{
+	// Initialize HardsubX data structures
+	struct lib_hardsubx_ctx *ctx = (struct lib_hardsubx_ctx *)malloc(sizeof(struct lib_hardsubx_ctx));
+	if(!ctx)
+		fatal(EXIT_NOT_ENOUGH_MEMORY, "lib_hardsubx_ctx");
+	memset(ctx, 0, sizeof(struct lib_hardsubx_ctx));
+	
+	ctx->tess_handle = TessBaseAPICreate();
+	if(TessBaseAPIInit3(ctx->tess_handle, NULL, "eng") != 0)
+	{
+		fatal(EXIT_NOT_ENOUGH_MEMORY, "Not enough memory to initialize Tesseract!");
+	}
+
+	//Initialize attributes common to lib_ccx context
+	ctx->basefilename = get_basename(options->output_filename);//TODO: Check validity, add stdin, network
+	ctx->current_file = -1;
+	ctx->inputfile = options->inputfile;
+	ctx->num_input_files = options->num_input_files;
+	ctx->extension = get_file_extension(options->write_format);
+	ctx->write_format = options->write_format;
+	ctx->subs_delay = options->subs_delay;
+	ctx->cc_to_stdout = options->cc_to_stdout;
+
+	//Initialize subtitle text parameters
+	ctx->ocr_mode = options->hardsubx_ocr_mode;
+	ctx->subcolor = options->hardsubx_subcolor;
+	ctx->min_sub_duration = options->hardsubx_min_sub_duration;
+	ctx->detect_italics = options->hardsubx_detect_italics;
+	ctx->conf_thresh = options->hardsubx_conf_thresh;
+	ctx->hue = options->hardsubx_hue;
+	ctx->lum_thresh = options->hardsubx_lum_thresh;
+
+	//Initialize subtitle structure memory
+	ctx->dec_sub = (struct cc_subtitle *)malloc(sizeof(struct cc_subtitle));
+	memset (ctx->dec_sub, 0,sizeof(struct cc_subtitle));
+
+	return ctx;
+}
+
+void _dinit_hardsubx(struct lib_hardsubx_ctx **ctx)
+{
+	struct lib_hardsubx_ctx *lctx = *ctx;
+	// Free all memory allocated to everything in the context
+
+	// Free OCR
+	TessBaseAPIEnd(lctx->tess_handle);
+	TessBaseAPIDelete(lctx->tess_handle);
+
+	//Free subtitle
+	freep(lctx->dec_sub);
+	freep(ctx);
+}
+
+void hardsubx(struct ccx_s_options *options)
+{
+	// This is similar to the 'main' function in ccextractor.c, but for hard subs
+	mprint("HardsubX (Hard Subtitle Extractor) - Burned-in subtitle extraction subsystem\n");
+
+	// Initialize HardsubX data structures
+	struct lib_hardsubx_ctx *ctx;
+	ctx = _init_hardsubx(options);
+
+	// Dump parameters (Not using params_dump since completely different parameters)
+	_hardsubx_params_dump(options, ctx);
+
+	// Configure output settings
+
+	// Data processing loop
+	time_t start, end;
+	time(&start);
+	hardsubx_process_data(ctx);
+
+	// Show statistics (time taken, frames processed, mode etc)
+	time(&end);
+	long processing_time=(long) (end-start);
+	mprint ("\rDone, processing time = %ld seconds\n", processing_time);
+
+	// Free all allocated memory for the data structures
+	_dinit_hardsubx(&ctx);
+}
+
+#endif

src/lib_ccx/hardsubx.h

@@ -0,0 +1,107 @@
+#ifndef HARDSUBX_H
+#define HARDSUBX_H
+
+#include "lib_ccx.h"
+#include "utility.h"
+
+#ifdef ENABLE_HARDSUBX
+//TODO: Correct FFMpeg integration
+#include <libavcodec/avcodec.h>
+#include <libavformat/avformat.h>
+#include <libswscale/swscale.h>
+#include "allheaders.h"
+#include "capi.h"
+
+enum hardsubx_color_type
+{
+	HARDSUBX_COLOR_WHITE = 0,
+	HARDSUBX_COLOR_YELLOW = 1,
+	HARDSUBX_COLOR_GREEN = 2,
+	HARDSUBX_COLOR_CYAN = 3,
+	HARDSUBX_COLOR_BLUE = 4,
+	HARDSUBX_COLOR_MAGENTA = 5,
+	HARDSUBX_COLOR_RED = 6,
+	HARDSUBX_COLOR_CUSTOM = 7,
+};
+
+enum hardsubx_ocr_mode
+{
+	HARDSUBX_OCRMODE_FRAME = 0,
+	HARDSUBX_OCRMODE_WORD = 1,
+	HARDSUBX_OCRMODE_LETTER = 2,
+};
+
+struct lib_hardsubx_ctx
+{
+	// The main context for hard subtitle extraction
+
+	// Attributes common to the lib_ccx context
+	int cc_to_stdout;
+	LLONG subs_delay;
+	LLONG last_displayed_subs_ms;
+	char *basefilename;
+	const char *extension;
+	int current_file;
+	char **inputfile;
+	int num_input_files; 
+	LLONG system_start_time;
+	enum ccx_output_format write_format;
+
+	// Media file context
+	AVFormatContext *format_ctx;
+	AVCodecContext *codec_ctx;
+	AVCodec *codec;
+	AVFrame *frame;
+	AVFrame *rgb_frame;
+	AVPacket packet;
+	AVDictionary *options_dict;
+	struct SwsContext *sws_ctx;
+	uint8_t *rgb_buffer;
+	int video_stream_id;
+
+	// Leptonica Image and Tesseract Context
+	PIX *im;
+	TessBaseAPI *tess_handle;
+
+	// Classifier parameters
+
+	// Subtitle text parameters
+	struct cc_subtitle *dec_sub;
+	int ocr_mode;
+	int subcolor;
+	float min_sub_duration;
+	int detect_italics;
+	float conf_thresh;
+	float hue;
+	float lum_thresh;
+};
+
+struct lib_hardsubx_ctx* _init_hardsubx(struct ccx_s_options *options);
+void _hardsubx_params_dump(struct ccx_s_options *options, struct lib_hardsubx_ctx *ctx);
+void hardsubx(struct ccx_s_options *options);
+
+//hardsubx_decoder.c
+int hardsubx_process_frames_linear(struct lib_hardsubx_ctx *ctx, struct encoder_ctx *enc_ctx);
+int hardsubx_process_frames_binary(struct lib_hardsubx_ctx *ctx);
+
+//hardsubx_imgops.c
+void rgb2hsv(float R, float G, float B,float *H, float *S, float *V);
+void rgb2lab(float R, float G, float B,float *L, float *a, float *b);
+
+//hardsubx_classifier.c
+char *get_ocr_text_simple(struct lib_hardsubx_ctx *ctx, PIX *image);
+char *get_ocr_text_wordwise(struct lib_hardsubx_ctx *ctx, PIX *image);
+char *get_ocr_text_letterwise(struct lib_hardsubx_ctx *ctx, PIX *image);
+char *get_ocr_text_simple_threshold(struct lib_hardsubx_ctx *ctx, PIX *image, float threshold);
+char *get_ocr_text_wordwise_threshold(struct lib_hardsubx_ctx *ctx, PIX *image, float threshold);
+char *get_ocr_text_letterwise_threshold(struct lib_hardsubx_ctx *ctx, PIX *image, float threshold);
+
+//hardsubx_utility.c
+int edit_distance(char * word1, char * word2, int len1, int len2);
+int64_t convert_pts_to_ms(int64_t pts, AVRational time_base);
+int64_t convert_pts_to_ns(int64_t pts, AVRational time_base);
+int64_t convert_pts_to_s(int64_t pts, AVRational time_base);
+
+#endif
+
+#endif

src/lib_ccx/hardsubx_classifier.c

@@ -0,0 +1,317 @@
+#include "lib_ccx.h"
+#include "utility.h"
+
+#ifdef ENABLE_HARDSUBX
+//TODO: Correct FFMpeg integration
+#include <libavcodec/avcodec.h>
+#include <libavformat/avformat.h>
+#include <libavutil/imgutils.h>
+#include <libswscale/swscale.h>
+#include "allheaders.h"
+#include "hardsubx.h"
+
+char *get_ocr_text_simple(struct lib_hardsubx_ctx *ctx, PIX *image)
+{
+	char *text_out=NULL;
+
+	TessBaseAPISetImage2(ctx->tess_handle, image);
+	if(TessBaseAPIRecognize(ctx->tess_handle, NULL) != 0)
+	{	
+		//TODO: Display error message
+		printf("Error in Tesseract recognition\n");
+		return NULL;
+	}
+
+	if((text_out = TessBaseAPIGetUTF8Text(ctx->tess_handle)) == NULL)
+	{
+		//TODO: Display error message
+		printf("Error getting text\n");
+	}
+	return text_out;
+}
+
+char *get_ocr_text_wordwise(struct lib_hardsubx_ctx *ctx, PIX *image)
+{
+	char *text_out=NULL;
+
+	TessBaseAPISetImage2(ctx->tess_handle, image);
+	if(TessBaseAPIRecognize(ctx->tess_handle, NULL) != 0)
+	{	
+		//TODO: Display error message
+		printf("Error in Tesseract recognition\n");
+		return NULL;
+	}
+
+	TessResultIterator *it = TessBaseAPIGetIterator(ctx->tess_handle);
+	TessPageIteratorLevel level = RIL_WORD;
+
+	int prev_ital = 0;
+
+	if(it!=0)
+	{
+		do
+		{
+			char* word = TessResultIteratorGetUTF8Text(it, level);
+			if(word==NULL || strlen(word)==0)
+				continue;
+			if(text_out == NULL)
+			{
+				if(ctx->detect_italics)
+				{
+					int italic=0;
+					int dummy=0;
+					TessResultIteratorWordFontAttributes(it, &dummy, &italic,&dummy, &dummy, &dummy,&dummy, &dummy, &dummy);
+					if(italic==1 && prev_ital==0)
+					{
+						char *word_copy = strdup(word);
+						word = realloc(word, strlen(word)+strlen("<i>")+2);
+						strcpy(word,"<i>");
+						strcat(word, word_copy);
+						free(word_copy);
+						prev_ital = 1;
+					}
+					else if(italic == 0 && prev_ital == 1)
+					{
+						word = realloc(word, strlen(word)+strlen("</i>")+2);
+						strcat(word, "</i>");
+						prev_ital = 0;
+					}	
+				}
+				text_out = strdup(word);
+				text_out = realloc(text_out, strlen(text_out)+2);
+				strcat(text_out, " ");
+				continue;
+			}
+			if(ctx->detect_italics)
+			{
+				int italic=0;
+				int dummy=0;
+				TessResultIteratorWordFontAttributes(it, &dummy, &italic,&dummy, &dummy, &dummy,&dummy, &dummy, &dummy);
+				if(italic==1 && prev_ital==0)
+				{
+					char *word_copy = strdup(word);
+					word = realloc(word, strlen(word)+strlen("<i>")+2);
+					strcpy(word,"<i>");
+					strcat(word, word_copy);
+					free(word_copy);
+					prev_ital = 1;
+				}
+				else if(italic == 0 && prev_ital == 1)
+				{
+					word = realloc(word, strlen(word)+strlen("</i>")+2);
+					strcat(word, "</i>");
+					prev_ital = 0;
+				}
+			}
+			text_out = realloc(text_out, strlen(text_out)+strlen(word)+2);
+			strcat(text_out, word);
+			strcat(text_out, " ");
+			free(word);
+		} while(TessPageIteratorNext((TessPageIterator *)it, level));
+	}
+
+	if(ctx->detect_italics && prev_ital == 1)
+	{
+		text_out = realloc(text_out, strlen(text_out)+strlen("</i>")+2);
+		strcat(text_out, "</i>");
+	}
+
+	TessResultIteratorDelete(it);
+
+	return text_out;
+}
+
+char *get_ocr_text_letterwise(struct lib_hardsubx_ctx *ctx, PIX *image)
+{
+	char *text_out=NULL;
+
+	TessBaseAPISetImage2(ctx->tess_handle, image);
+	if(TessBaseAPIRecognize(ctx->tess_handle, NULL) != 0)
+	{	
+		//TODO: Display error message
+		printf("Error in Tesseract recognition\n");
+		return NULL;
+	}
+
+	TessResultIterator *it = TessBaseAPIGetIterator(ctx->tess_handle);
+	TessPageIteratorLevel level = RIL_SYMBOL;
+
+	if(it!=0)
+	{
+		do
+		{
+			char* letter = TessResultIteratorGetUTF8Text(it, level);
+			if(letter==NULL || strlen(letter)==0)
+				continue;
+			if(text_out==NULL)
+			{
+				text_out = strdup(letter);
+				continue;
+			}
+			text_out = realloc(text_out, strlen(text_out) + strlen(letter) + 1);
+			strcat(text_out, letter);
+			free(letter);
+		} while(TessPageIteratorNext((TessPageIterator *)it, level));
+	}
+
+	TessResultIteratorDelete(it);
+
+	return text_out;
+}
+
+char *get_ocr_text_simple_threshold(struct lib_hardsubx_ctx *ctx, PIX *image, float threshold)
+{
+	char *text_out=NULL;
+
+	TessBaseAPISetImage2(ctx->tess_handle, image);
+	if(TessBaseAPIRecognize(ctx->tess_handle, NULL) != 0)
+	{	
+		//TODO: Display error message
+		printf("Error in Tesseract recognition\n");
+		return NULL;
+	}
+
+	int conf = TessBaseAPIMeanTextConf(ctx->tess_handle);
+
+	if(conf < threshold)
+		return NULL;
+
+	return text_out;
+}
+
+char *get_ocr_text_wordwise_threshold(struct lib_hardsubx_ctx *ctx, PIX *image, float threshold)
+{
+	char *text_out=NULL;
+
+	TessBaseAPISetImage2(ctx->tess_handle, image);
+	if(TessBaseAPIRecognize(ctx->tess_handle, NULL) != 0)
+	{	
+		//TODO: Display error message
+		printf("Error in Tesseract recognition\n");
+		return NULL;
+	}
+
+	TessResultIterator *it = TessBaseAPIGetIterator(ctx->tess_handle);
+	TessPageIteratorLevel level = RIL_WORD;
+
+	int prev_ital = 0;
+
+	if(it!=0)
+	{
+		do
+		{
+			char* word = TessResultIteratorGetUTF8Text(it, level);
+			if(word==NULL || strlen(word)==0)
+				continue;
+			float conf = TessResultIteratorConfidence(it,level);
+			if(conf < threshold)
+				continue;
+			if(text_out == NULL)
+			{
+				if(ctx->detect_italics)
+				{
+					int italic=0;
+					int dummy=0;
+					TessResultIteratorWordFontAttributes(it, &dummy, &italic,&dummy, &dummy, &dummy,&dummy, &dummy, &dummy);
+					if(italic==1 && prev_ital==0)
+					{
+						char *word_copy = strdup(word);
+						word = realloc(word, strlen(word)+strlen("<i>")+2);
+						strcpy(word,"<i>");
+						strcat(word, word_copy);
+						free(word_copy);
+						prev_ital = 1;
+					}
+					else if(italic == 0 && prev_ital == 1)
+					{
+						word = realloc(word, strlen(word)+strlen("</i>")+2);
+						strcat(word, "</i>");
+						prev_ital = 0;
+					}	
+				}
+				text_out = strdup(word);
+				text_out = realloc(text_out, strlen(text_out)+2);
+				strcat(text_out, " ");
+				continue;
+			}
+			if(ctx->detect_italics)
+			{
+				int italic=0;
+				int dummy=0;
+				TessResultIteratorWordFontAttributes(it, &dummy, &italic,&dummy, &dummy, &dummy,&dummy, &dummy, &dummy);
+				if(italic==1 && prev_ital==0)
+				{
+					char *word_copy = strdup(word);
+					word = realloc(word, strlen(word)+strlen("<i>")+2);
+					strcpy(word,"<i>");
+					strcat(word, word_copy);
+					free(word_copy);
+					prev_ital = 1;
+				}
+				else if(italic == 0 && prev_ital == 1)
+				{
+					word = realloc(word, strlen(word)+strlen("</i>")+2);
+					strcat(word, "</i>");
+					prev_ital = 0;
+				}
+			}
+			text_out = realloc(text_out, strlen(text_out)+strlen(word)+2);
+			strcat(text_out, word);
+			strcat(text_out, " ");
+			free(word);
+		} while(TessPageIteratorNext((TessPageIterator *)it, level));
+	}
+
+	if(ctx->detect_italics && prev_ital == 1)
+	{
+		text_out = realloc(text_out, strlen(text_out)+strlen("</i>")+2);
+		strcat(text_out, "</i>");
+	}
+
+	TessResultIteratorDelete(it);
+
+	return text_out;
+}
+
+char *get_ocr_text_letterwise_threshold(struct lib_hardsubx_ctx *ctx, PIX *image, float threshold)
+{
+	char *text_out=NULL;
+
+	TessBaseAPISetImage2(ctx->tess_handle, image);
+	if(TessBaseAPIRecognize(ctx->tess_handle, NULL) != 0)
+	{	
+		//TODO: Display error message
+		printf("Error in Tesseract recognition\n");
+		return NULL;
+	}
+
+	TessResultIterator *it = TessBaseAPIGetIterator(ctx->tess_handle);
+	TessPageIteratorLevel level = RIL_SYMBOL;
+
+	if(it!=0)
+	{
+		do
+		{
+			char* letter = TessResultIteratorGetUTF8Text(it, level);
+			if(letter==NULL || strlen(letter)==0)
+				continue;
+			float conf = TessResultIteratorConfidence(it,level);
+			if(conf < threshold)
+				continue;
+			if(text_out==NULL)
+			{
+				text_out = strdup(letter);
+				continue;
+			}
+			text_out = realloc(text_out, strlen(text_out) + strlen(letter) + 1);
+			strcat(text_out, letter);
+			free(letter);
+		} while(TessPageIteratorNext((TessPageIterator *)it, level));
+	}
+
+	TessResultIteratorDelete(it);
+
+	return text_out;
+}
+
+#endif

src/lib_ccx/hardsubx_decoder.c

@@ -0,0 +1,370 @@
+#include "lib_ccx.h"
+#include "utility.h"
+
+#ifdef ENABLE_HARDSUBX
+//TODO: Correct FFMpeg integration
+#include <libavcodec/avcodec.h>
+#include <libavformat/avformat.h>
+#include <libavutil/imgutils.h>
+#include <libswscale/swscale.h>
+#include "allheaders.h"
+#include "hardsubx.h"
+#include "capi.h"
+
+char* _process_frame_white_basic(struct lib_hardsubx_ctx *ctx, AVFrame *frame, int width, int height, int index)
+{
+	//printf("frame : %04d\n", index);
+	PIX *im;
+	// PIX *edge_im;
+	PIX *lum_im;
+	char *subtitle_text=NULL;
+	im = pixCreate(width,height,32);
+	lum_im = pixCreate(width,height,32);
+	// edge_im = pixCreate(width,height,8);
+	int i,j;
+	for(i=(3*height)/4;i<height;i++)
+	{
+		for(j=0;j<width;j++)
+		{
+			int p=j*3+i*frame->linesize[0];
+			int r=frame->data[0][p];
+			int g=frame->data[0][p+1];
+			int b=frame->data[0][p+2];
+			pixSetRGBPixel(im,j,i,r,g,b);
+			float L,A,B;
+			rgb2lab((float)r,(float)g,(float)b,&L,&A,&B);
+			if(L > ctx->lum_thresh)
+				pixSetRGBPixel(lum_im,j,i,255,255,255);
+			else
+				pixSetRGBPixel(lum_im,j,i,0,0,0);
+		}
+	}
+
+	if(ctx->detect_italics)
+	{
+		ctx->ocr_mode = HARDSUBX_OCRMODE_WORD;
+	}
+
+	// TESSERACT OCR FOR THE FRAME HERE
+	switch(ctx->ocr_mode)
+	{
+		case HARDSUBX_OCRMODE_WORD:
+			if(ctx->conf_thresh > 0)
+				subtitle_text = get_ocr_text_wordwise_threshold(ctx, lum_im, ctx->conf_thresh);
+			else
+				subtitle_text = get_ocr_text_wordwise(ctx, lum_im);
+			break;
+		case HARDSUBX_OCRMODE_LETTER:
+			if(ctx->conf_thresh > 0)
+				subtitle_text = get_ocr_text_letterwise_threshold(ctx, lum_im, ctx->conf_thresh);
+			else
+				subtitle_text = get_ocr_text_letterwise(ctx, lum_im);
+			break;
+		case HARDSUBX_OCRMODE_FRAME:
+			if(ctx->conf_thresh > 0)
+				subtitle_text = get_ocr_text_simple_threshold(ctx, lum_im, ctx->conf_thresh);
+			else
+				subtitle_text = get_ocr_text_simple(ctx, lum_im);
+			break;
+		default:
+			fatal(EXIT_MALFORMED_PARAMETER,"Invalid OCR Mode");
+	}
+
+	pixDestroy(&lum_im);
+	pixDestroy(&im);
+
+	return subtitle_text;
+}
+
+char *_process_frame_color_basic(struct lib_hardsubx_ctx *ctx, AVFrame *frame, int width, int height, int index)
+{
+	PIX *im;
+	PIX *edge_im;
+	PIX *hue_im;
+	PIX *feat_im;
+	char *subtitle_text=NULL;
+	im = pixCreate(width,height,32);
+	hue_im = pixCreate(width,height,32);
+	feat_im = pixCreate(width,height,32);
+	edge_im = pixCreate(width,height,8);
+	int i,j;
+	for(i=(3*height)/4;i<height;i++)
+	{
+		for(j=0;j<width;j++)
+		{
+			int p=j*3+i*frame->linesize[0];
+			int r=frame->data[0][p];
+			int g=frame->data[0][p+1];
+			int b=frame->data[0][p+2];
+			pixSetRGBPixel(im,j,i,r,g,b);
+			float H,S,V;
+			rgb2lab((float)r,(float)g,(float)b,&H,&S,&V);
+			if(abs(H - ctx->hue)>20)
+				pixSetRGBPixel(hue_im,j,i,255,255,255);
+			else
+				pixSetRGBPixel(hue_im,j,i,0,0,0);
+		}
+	}
+
+	// Based on hue image and edge image, create feature image
+	for(i=3*(height/4);i<height;i++)
+	{
+		for(j=0;j<width;j++)
+		{
+			unsigned int p1,p2,p3;
+			// pixGetPixel(edge_im,j,i,&p1);
+			// pixGetPixel(pixd,j,i,&p2);
+			pixGetPixel(hue_im,j,i,&p3);
+			if(p3>0)//if(p2==0&&p1==0&&p3>0)
+			{
+				pixSetRGBPixel(feat_im,j,i,255,255,255);
+			}
+		}
+	}
+
+	if(ctx->detect_italics)
+	{
+		ctx->ocr_mode = HARDSUBX_OCRMODE_WORD;
+	}
+
+	// TESSERACT OCR FOR THE FRAME HERE
+	switch(ctx->ocr_mode)
+	{
+		case HARDSUBX_OCRMODE_WORD:
+			if(ctx->conf_thresh > 0)
+				subtitle_text = get_ocr_text_wordwise_threshold(ctx, feat_im, ctx->conf_thresh);
+			else
+				subtitle_text = get_ocr_text_wordwise(ctx, feat_im);
+			break;
+		case HARDSUBX_OCRMODE_LETTER:
+			if(ctx->conf_thresh > 0)
+				subtitle_text = get_ocr_text_letterwise_threshold(ctx, feat_im, ctx->conf_thresh);
+			else
+				subtitle_text = get_ocr_text_letterwise(ctx, feat_im);
+			break;
+		case HARDSUBX_OCRMODE_FRAME:
+			if(ctx->conf_thresh > 0)
+				subtitle_text = get_ocr_text_simple_threshold(ctx, feat_im, ctx->conf_thresh);
+			else
+				subtitle_text = get_ocr_text_simple(ctx, feat_im);
+			break;
+		default:
+			fatal(EXIT_MALFORMED_PARAMETER,"Invalid OCR Mode");
+	}
+
+	pixDestroy(&feat_im);
+	pixDestroy(&im);
+	pixDestroy(&edge_im);
+	pixDestroy(&hue_im);
+
+	return subtitle_text;
+}
+
+void _display_frame(struct lib_hardsubx_ctx *ctx, AVFrame *frame, int width, int height, int timestamp)
+{
+	// Debug: Display the frame after processing
+	PIX *im;
+	im = pixCreate(width,height,32);
+	PIX *hue_im = pixCreate(width,height,32);
+
+	int i,j;
+	for(i=0;i<height;i++)
+	{
+		for(j=0;j<width;j++)
+		{
+			int p=j*3+i*frame->linesize[0];
+			int r=frame->data[0][p];
+			int g=frame->data[0][p+1];
+			int b=frame->data[0][p+2];
+			pixSetRGBPixel(im,j,i,r,g,b);
+			float H,S,V;
+			rgb2hsv((float)r,(float)g,(float)b,&H,&S,&V);
+			if(abs(H-60)<20)
+			{
+				pixSetRGBPixel(hue_im,j,i,255,255,255);
+			}
+		}
+	}
+
+	PIX *edge_im = pixCreate(width,height,8),*edge_im_2 = pixCreate(width,height,8);
+	edge_im = pixConvertRGBToGray(im,0.0,0.0,0.0);
+	pixCopy(edge_im_2,edge_im);
+	edge_im = pixSobelEdgeFilter(edge_im, L_VERTICAL_EDGES);
+	edge_im = pixDilateGray(edge_im, 21, 11);
+	edge_im = pixThresholdToBinary(edge_im,50);
+	PIX *pixd = pixCreate(width,height,1);
+	pixSauvolaBinarize(edge_im_2, 11, 0.3, 1, NULL, NULL, NULL, &pixd);
+
+	PIX *feat_im = pixCreate(width,height,32);
+	for(i=3*(height/4);i<height;i++)
+	{
+		for(j=0;j<width;j++)
+		{
+			unsigned int p1,p2,p3;
+			pixGetPixel(edge_im,j,i,&p1);
+			pixGetPixel(pixd,j,i,&p2);
+			pixGetPixel(hue_im,j,i,&p3);
+			if(p1==0)//if(p2==0&&p1==0&&p3>0)
+			{
+				pixSetRGBPixel(feat_im,j,i,255,255,255);
+			}
+		}
+	}
+
+	char *txt=NULL;
+	// txt = get_ocr_text_simple(ctx, feat_im);
+	// txt=get_ocr_text_wordwise_threshold(ctx, feat_im, ctx->conf_thresh);
+	// if(txt != NULL)printf("%s\n", txt);
+
+	char write_path[100];
+	sprintf(write_path,"./ffmpeg-examples/frames/temp%04d.jpg",timestamp);
+
+	pixWrite(write_path,feat_im,IFF_JFIF_JPEG);
+	pixDestroy(&im);
+	pixDestroy(&edge_im);
+	pixDestroy(&feat_im);
+	pixDestroy(&edge_im_2);
+	pixDestroy(&pixd);
+}
+
+int hardsubx_process_frames_linear(struct lib_hardsubx_ctx *ctx, struct encoder_ctx *enc_ctx)
+{
+	// Do an exhaustive linear search over the video
+	int got_frame;
+	int dist;
+	int cur_sec,total_sec,progress;
+	int frame_number = 0;
+	int64_t begin_time = 0,end_time = 0,prev_packet_pts = 0;
+	char *subtitle_text=NULL;
+	char *prev_subtitle_text=NULL;
+
+	while(av_read_frame(ctx->format_ctx, &ctx->packet)>=0)
+	{
+		if(ctx->packet.stream_index == ctx->video_stream_id)
+		{
+			frame_number++;
+
+			//Decode the video stream packet
+			avcodec_decode_video2(ctx->codec_ctx, ctx->frame, &got_frame, &ctx->packet);
+
+			if(got_frame && frame_number % 25 == 0)
+			{
+				float diff = (float)convert_pts_to_ms(ctx->packet.pts - prev_packet_pts, ctx->format_ctx->streams[ctx->video_stream_id]->time_base);
+				if(abs(diff) < 1000*ctx->min_sub_duration) //If the minimum duration of a subtitle line is exceeded, process packet
+					continue;
+
+				// sws_scale is used to convert the pixel format to RGB24 from all other cases
+				sws_scale(
+						ctx->sws_ctx,
+						(uint8_t const * const *)ctx->frame->data,
+						ctx->frame->linesize,
+						0,
+						ctx->codec_ctx->height,
+						ctx->rgb_frame->data,
+						ctx->rgb_frame->linesize
+					);
+				
+
+				// Send the frame to other functions for processing
+				if(ctx->subcolor==HARDSUBX_COLOR_WHITE)
+				{
+					subtitle_text = _process_frame_white_basic(ctx,ctx->rgb_frame,ctx->codec_ctx->width,ctx->codec_ctx->height,frame_number);
+				}
+				else
+				{
+					subtitle_text = _process_frame_color_basic(ctx, ctx->rgb_frame, ctx->codec_ctx->width,ctx->codec_ctx->height,frame_number);
+				}
+				_display_frame(ctx, ctx->rgb_frame,ctx->codec_ctx->width,ctx->codec_ctx->height,frame_number);
+
+				cur_sec = (int)convert_pts_to_s(ctx->packet.pts, ctx->format_ctx->streams[ctx->video_stream_id]->time_base);
+				total_sec = (int)convert_pts_to_s(ctx->format_ctx->duration, AV_TIME_BASE_Q);
+				progress = (cur_sec*100)/total_sec;
+				activity_progress(progress,cur_sec/60,cur_sec%60);
+
+				if(subtitle_text==NULL)
+					continue;
+				if(!strlen(subtitle_text))
+					continue;
+				subtitle_text = strtok(subtitle_text,"\n");
+				end_time = convert_pts_to_ms(ctx->packet.pts, ctx->format_ctx->streams[ctx->video_stream_id]->time_base);
+				if(prev_subtitle_text)
+				{
+					//TODO: Encode text with highest confidence
+					dist = edit_distance(subtitle_text, prev_subtitle_text, strlen(subtitle_text), strlen(prev_subtitle_text));
+
+					if(dist > (0.2 * fmin(strlen(subtitle_text), strlen(prev_subtitle_text))))
+					{
+						add_cc_sub_text(ctx->dec_sub, prev_subtitle_text, begin_time, end_time, "", "BURN", CCX_ENC_UTF_8);
+						encode_sub(enc_ctx, ctx->dec_sub);
+						begin_time = end_time + 1;
+					}
+				}
+
+				prev_subtitle_text = strdup(subtitle_text);
+				prev_packet_pts = ctx->packet.pts;
+			}
+		}
+		av_packet_unref(&ctx->packet);
+	}
+
+	add_cc_sub_text(ctx->dec_sub, prev_subtitle_text, begin_time, end_time, "", "BURN", CCX_ENC_UTF_8);
+	encode_sub(enc_ctx, ctx->dec_sub);
+	activity_progress(100,cur_sec/60,cur_sec%60);
+
+}
+
+int hardsubx_process_frames_binary(struct lib_hardsubx_ctx *ctx)
+{
+	// Do a binary search over the input video for faster processing
+	// printf("Duration: %d\n", (int)ctx->format_ctx->duration);
+	int got_frame;
+	int seconds_time = 0;
+	for(seconds_time=0;seconds_time<20;seconds_time++){
+	int64_t seek_time = (int64_t)(seconds_time*AV_TIME_BASE);
+	seek_time = av_rescale_q(seek_time, AV_TIME_BASE_Q, ctx->format_ctx->streams[ctx->video_stream_id]->time_base);
+
+	int ret = av_seek_frame(ctx->format_ctx, ctx->video_stream_id, seek_time, AVSEEK_FLAG_BACKWARD);
+	// printf("%d\n", ret);
+	// if(ret < 0)
+	// {
+	// 	printf("seeking back\n");
+	// 	ret = av_seek_frame(ctx->format_ctx, -1, seek_time, AVSEEK_FLAG_BACKWARD);
+	// }
+	if(ret >= 0)
+	{
+		while(av_read_frame(ctx->format_ctx, &ctx->packet)>=0)
+		{
+			if(ctx->packet.stream_index == ctx->video_stream_id)
+			{
+				avcodec_decode_video2(ctx->codec_ctx, ctx->frame, &got_frame, &ctx->packet);
+				if(got_frame)
+				{
+					// printf("%d\n", seek_time);
+					if(ctx->packet.pts < seek_time)
+						continue;
+					// printf("GOT FRAME: %d\n",ctx->packet.pts);
+					// sws_scale is used to convert the pixel format to RGB24 from all other cases
+					sws_scale(
+							ctx->sws_ctx,
+							(uint8_t const * const *)ctx->frame->data,
+							ctx->frame->linesize,
+							0,
+							ctx->codec_ctx->height,
+							ctx->rgb_frame->data,
+							ctx->rgb_frame->linesize
+						);
+					// Send the frame to other functions for processing
+					_display_frame(ctx, ctx->rgb_frame,ctx->codec_ctx->width,ctx->codec_ctx->height,seconds_time);
+					break;
+				}
+			}
+		}
+	}
+	else
+	{
+		printf("Seeking to timestamp failed\n");
+	}
+	}
+}
+
+#endif

src/lib_ccx/hardsubx_imgops.c

@@ -0,0 +1,108 @@
+#include "lib_ccx.h"
+#include "utility.h"
+
+#ifdef ENABLE_HARDSUBX
+//TODO: Correct FFMpeg integration
+#include <libavcodec/avcodec.h>
+#include <libavformat/avformat.h>
+#include <libavutil/imgutils.h>
+#include <libswscale/swscale.h>
+#include "allheaders.h"
+#include "hardsubx.h"
+
+#define BLACK 20.0
+#define YELLOW 70.0
+
+#define min_f(a, b, c)  (fminf(a, fminf(b, c)))
+#define max_f(a, b, c) (fmaxf(a, fmaxf(b, c)))
+
+void rgb2hsv(float R, float G, float B,float *H, float *S, float *V)
+{
+	//Conversion into HSV color space to get Hue
+	float r = R / 255.0f;
+	float g = G / 255.0f;
+	float b = B / 255.0f;
+
+	float h, s, v; // h:0-360.0, s:0.0-1.0, v:0.0-1.0
+
+	float max = max_f(r, g, b);
+	float min = min_f(r, g, b);
+
+	v = max;
+
+	if (max == 0.0f) {
+		s = 0;
+		h = 0;
+	}
+	else if (max - min == 0.0f) {
+		s = 0;
+		h = 0;
+	}
+	else {
+		s = (max - min) / max;
+
+		if (max == r) {
+			h = 60 * ((g - b) / (max - min)) + 0;
+		}
+		else if (max == g) {
+			h = 60 * ((b - r) / (max - min)) + 120;
+		}
+		else {
+			h = 60 * ((r - g) / (max - min)) + 240;
+		}
+	}
+
+	if (h < 0) h += 360.0f;
+
+	*H = (unsigned char)(h);       // dst_h : 0-360
+	*S = (unsigned char)(s * 255); // dst_s : 0-255
+	*V = (unsigned char)(v * 255); // dst_v : 0-255
+}
+
+void rgb2lab(float R, float G, float B,float *L, float *a, float *b)
+{
+	//Conversion to the CIE-LAB color space to get the Luminance
+	float X, Y, Z, fX, fY, fZ;
+
+	X = 0.412453*R + 0.357580*G + 0.180423*B;
+	Y = 0.212671*R + 0.715160*G + 0.072169*B;
+	Z = 0.019334*R + 0.119193*G + 0.950227*B;
+
+	X /= (255 * 0.950456);
+	Y /=  255;
+	Z /= (255 * 1.088754);
+
+	if (Y > 0.008856)
+	{
+		fY = pow(Y, 1.0/3.0);
+		*L = 116.0*fY - 16.0;
+	}
+	else
+	{
+		fY = 7.787*Y + 16.0/116.0;
+		*L = 903.3*Y;
+	}
+
+	if (X > 0.008856)
+		fX = pow(X, 1.0/3.0);
+	else
+		fX = 7.787*X + 16.0/116.0;
+
+	if (Z > 0.008856)
+		fZ = pow(Z, 1.0/3.0);
+	else
+		fZ = 7.787*Z + 16.0/116.0;
+
+	*a = 500.0*(fX - fY);
+	*b = 200.0*(fY - fZ);
+
+	if (*L < BLACK) {
+	*a *= exp((*L - BLACK) / (BLACK / 4));
+	*b *= exp((*L - BLACK) / (BLACK / 4));
+	*L = BLACK;
+	}
+	if (*b > YELLOW)
+	*b = YELLOW;
+}
+
+#endif

src/lib_ccx/hardsubx_utility.c

@@ -0,0 +1,68 @@
+#include "lib_ccx.h"
+#include "utility.h"
+
+#ifdef ENABLE_HARDSUBX
+//TODO: Correct FFMpeg integration
+#include <libavcodec/avcodec.h>
+#include <libavformat/avformat.h>
+#include <libavutil/imgutils.h>
+#include <libswscale/swscale.h>
+#include "allheaders.h"
+#include "hardsubx.h"
+
+int64_t convert_pts_to_ms(int64_t pts, AVRational time_base)
+{
+	return av_rescale_q(pts, time_base, AV_TIME_BASE_Q)/1000;
+}
+
+int64_t convert_pts_to_ns(int64_t pts, AVRational time_base)
+{
+	return av_rescale_q(pts, time_base, AV_TIME_BASE_Q);
+}
+
+int64_t convert_pts_to_s(int64_t pts, AVRational time_base)
+{
+	return av_rescale_q(pts, time_base, AV_TIME_BASE_Q)/1000000;
+}
+
+int edit_distance(char * word1, char * word2, int len1, int len2)
+{
+	int matrix[len1 + 1][len2 + 1];
+	int i,delete,insert,substitute,minimum;
+	for (i = 0; i <= len1; i++)matrix[i][0] = i;
+	for (i = 0; i <= len2; i++)matrix[0][i] = i;
+	for (i = 1; i <= len1; i++)
+	{
+		int j;
+		char c1;
+		c1 = word1[i-1];
+		for (j = 1; j <= len2; j++)
+		{
+			char c2;
+			c2 = word2[j-1];
+			if (c1 == c2)
+			{
+				matrix[i][j] = matrix[i-1][j-1];
+			}
+			else 
+			{
+				delete = matrix[i-1][j] + 1;
+				insert = matrix[i][j-1] + 1;
+				substitute = matrix[i-1][j-1] + 1;
+				minimum = delete;
+				if (insert < minimum)
+				{
+					minimum = insert;
+				}
+				if (substitute < minimum)
+				{
+					minimum = substitute;
+				}
+				matrix[i][j] = minimum;
+			}
+		}
+	}
+	return matrix[len1][len2];
+}
+
+#endif

src/lib_ccx/lib_ccx.c

@@ -22,6 +22,7 @@ static struct ccx_decoders_common_settings_t *init_decoder_setting(
 	setting->extract = opt->extract;
 	setting->fullbin = opt->fullbin;
 	setting->no_rollup = opt->no_rollup;
+	setting->noscte20 = opt->noscte20;
 	memcpy(&setting->extraction_start,&opt->extraction_start,sizeof(struct ccx_boundary_time));
 	memcpy(&setting->extraction_end,&opt->extraction_end,sizeof(struct ccx_boundary_time));
 	setting->cc_to_stdout = opt->cc_to_stdout;
@@ -30,8 +31,8 @@ static struct ccx_decoders_common_settings_t *init_decoder_setting(
 	setting->cc_channel = opt->cc_channel;
 	setting->send_to_srv = opt->send_to_srv;
 	setting->hauppauge_mode = opt->hauppauge_mode;
-	/* if in transcript setting xds is not selected then set ignore xds flag */
-	setting->ignore_xds = !opt->transcript_settings.xds;
+	setting->xds_write_to_file = opt->transcript_settings.xds;
+	
 	return setting;
 }
 static void dinit_decoder_setting (struct ccx_decoders_common_settings_t **setting)
@@ -97,6 +98,29 @@ struct lib_ccx_ctx* init_libraries(struct ccx_s_options *opt)
 		ccx_common_logging.fatal_ftn(EXIT_NOT_ENOUGH_MEMORY, "lib_ccx_ctx");
 	memset(ctx, 0, sizeof(struct lib_ccx_ctx));
 
+	if(opt->xmltv)
+	{
+		ctx->epg_inited = 1;
+		ctx->epg_buffers = (struct PSI_buffer *)malloc(sizeof(struct PSI_buffer)*(0xfff+1));
+		ctx->eit_programs = (struct EIT_program *)malloc(sizeof(struct EIT_program)*(TS_PMT_MAP_SIZE+1));
+		ctx->eit_current_events = (int32_t *)malloc(sizeof(int32_t)*(TS_PMT_MAP_SIZE+1));
+		ctx->ATSC_source_pg_map = (int16_t *)malloc(sizeof(int16_t)*(0xffff));
+		memset(ctx->epg_buffers, 0, sizeof(struct PSI_buffer)*(0xfff+1));
+		memset(ctx->eit_programs, 0, sizeof(struct EIT_program)*(TS_PMT_MAP_SIZE+1));
+		memset(ctx->eit_current_events, 0, sizeof(int32_t)*(TS_PMT_MAP_SIZE+1));
+		memset(ctx->ATSC_source_pg_map, 0, sizeof(int16_t)*(0xffff));
+		if(!ctx->epg_buffers || !ctx->eit_programs || !ctx->eit_current_events || !ctx->ATSC_source_pg_map)
+			ccx_common_logging.fatal_ftn(EXIT_NOT_ENOUGH_MEMORY, "lib_ccx_ctx");
+	}
+	else
+	{
+		ctx->epg_inited = 0;
+		ctx->epg_buffers = NULL;
+		ctx->eit_programs = NULL;
+		ctx->eit_current_events = NULL;
+		ctx->ATSC_source_pg_map = NULL;
+	}
+
 	struct ccx_decoder_608_report *report_608 = malloc(sizeof(struct ccx_decoder_608_report));
 	if (!report_608)
 		ccx_common_logging.fatal_ftn(EXIT_NOT_ENOUGH_MEMORY, "report_608");
@@ -138,6 +162,7 @@ struct lib_ccx_ctx* init_libraries(struct ccx_s_options *opt)
 	if (ret < 0) {
 		goto end;
 	}
+	ctx->extension = get_file_extension(opt->write_format);
 
 	ctx->subs_delay = opt->subs_delay;
 
@@ -184,8 +209,8 @@ void dinit_libraries( struct lib_ccx_ctx **ctx)
 		if (dec_ctx->codec == CCX_CODEC_DVB)
 			dvbsub_close_decoder(&dec_ctx->private_data);
 		//Test memory for teletext
-		//else if (dec_ctx->codec == CCX_CODEC_TELETEXT)
-		//	telxcc_close(&dec_ctx->private_data, NULL);
+		else if (dec_ctx->codec == CCX_CODEC_TELETEXT)
+			telxcc_close(&dec_ctx->private_data, NULL);
 		else if (dec_ctx->codec == CCX_CODEC_ISDB_CC)
 			delete_isdb_decoder(&dec_ctx->private_data);
 

src/lib_ccx/lib_ccx.h

@@ -1,7 +1,7 @@
 #ifndef CCX_CCEXTRACTOR_H
 #define CCX_CCEXTRACTOR_H
 
-#define VERSION "0.78"
+#define VERSION "0.82"
 
 // Load common includes and constants for library usage
 #include "ccx_common_platform.h"
@@ -57,6 +57,7 @@ struct ccx_s_teletext_config
 	unsigned send_to_srv;
 	enum ccx_encoding_type encoding;
 	int nofontcolor;
+	int nohtmlescape;
 	char millis_separator;
 };
 
@@ -114,10 +115,11 @@ struct lib_ccx_ctx
 
 	unsigned teletext_warning_shown; // Did we detect a possible PAL (with teletext subs) and told the user already?
 
-	struct PSI_buffer epg_buffers[0xfff+1];
-	struct EIT_program eit_programs[TS_PMT_MAP_SIZE+1];
-	int32_t eit_current_events[TS_PMT_MAP_SIZE+1];
-	int16_t ATSC_source_pg_map[0xffff];
+	int epg_inited;
+	struct PSI_buffer *epg_buffers;
+	struct EIT_program *eit_programs;
+	int32_t *eit_current_events;
+	int16_t *ATSC_source_pg_map;
 	int epg_last_output; 
 	int epg_last_live_output; 
 	struct file_report freport;
@@ -198,13 +200,15 @@ void print_file_report(struct lib_ccx_ctx *ctx);
 
 // output.c
 void dinit_write(struct ccx_s_write *wb);
-int init_write (struct ccx_s_write *wb,char *filename);
+int temporarily_open_output(struct ccx_s_write *wb);
+int temporarily_close_output(struct ccx_s_write *wb);
+int init_write(struct ccx_s_write *wb, char *filename, int with_semaphore);
 int writeraw (const unsigned char *data, int length, void *private_data, struct cc_subtitle *sub);
 void flushbuffer (struct lib_ccx_ctx *ctx, struct ccx_s_write *wb, int closefile);
 void writercwtdata (struct lib_cc_decode *ctx, const unsigned char *data, struct cc_subtitle *sub);
 
 // stream_functions.c
-int isValidMP4Box(unsigned char *buffer, long position, long *nextBoxLocation, int *boxScore);
+int isValidMP4Box(unsigned char *buffer, size_t position, size_t *nextBoxLocation, int *boxScore);
 void detect_stream_type (struct ccx_demuxer *ctx);
 int detect_myth( struct ccx_demuxer *ctx );
 int read_video_pes_header (struct ccx_demuxer *ctx, struct demuxer_data *data, unsigned char *nextheader, int *headerlength, int sbuflen);
@@ -237,6 +241,9 @@ void timestamp_to_srttime(uint64_t timestamp, char *buffer);
 void timestamp_to_smptetttime(uint64_t timestamp, char *buffer);
 int levenshtein_dist (const uint64_t *s1, const uint64_t *s2, unsigned s1len, unsigned s2len);
 void millis_to_date (uint64_t timestamp, char *buffer, enum ccx_output_date_format date_format, char millis_separator);
+void create_signal(void);
+void signal_handler(int sig_type);
+struct encoder_ctx* change_filename(struct encoder_ctx*);
 #ifndef _WIN32
 void m_signal(int sig, void (*func)(int));
 #endif
@@ -245,7 +252,7 @@ void m_signal(int sig, void (*func)(int));
 void buffered_seek (struct ccx_demuxer *ctx, int offset);
 extern void build_parity_table(void);
 
-int tlt_process_pes_packet(struct lib_cc_decode *dec_ctx, uint8_t *buffer, uint16_t size, struct cc_subtitle *sub);
+int tlt_process_pes_packet(struct lib_cc_decode *dec_ctx, uint8_t *buffer, uint16_t size, struct cc_subtitle *sub, int sentence_cap);
 void* telxcc_init(void);
 void telxcc_close(void **ctx, struct cc_subtitle *sub);
 void tlt_read_rcwt(void *codec, unsigned char *buf, struct cc_subtitle *sub);

src/lib_ccx/ocr.c

@@ -7,6 +7,7 @@
 #include <dirent.h>
 #include "spupng_encoder.h"
 #include "ccx_encoders_helpers.h"
+#include "ocr.h"
 #undef OCR_DEBUG
 struct ocrCtx
 {
@@ -18,6 +19,7 @@ struct transIntensity
 	uint8_t *t;
 	png_color *palette;
 };
+
 static int check_trans_tn_intensity(const void *p1, const void *p2, void *arg)
 {
 	struct transIntensity *ti = arg;
@@ -27,8 +29,8 @@ static int check_trans_tn_intensity(const void *p1, const void *p2, void *arg)
 	unsigned char act_i;
 	/** TODO verify that RGB follow ITU-R BT.709
 	 *  Below fomula is valid only for 709 standurd
-         *  Y = 0.2126 R + 0.7152 G + 0.0722 B
-         */
+		 *  Y = 0.2126 R + 0.7152 G + 0.0722 B
+		 */
 	tmp_i = (0.2126 * ti->palette[*tmp].red) + (0.7152 * ti->palette[*tmp].green) + (0.0722 * ti->palette[*tmp].blue);
 	act_i = (0.2126 * ti->palette[*act].red) + (0.7152 * ti->palette[*act].green) + (0.0722 * ti->palette[*act].blue);;
 
@@ -41,8 +43,14 @@ static int check_trans_tn_intensity(const void *p1, const void *p2, void *arg)
 	return 1;
 }
 
-static int search_language_pack(const char *dirname,const char *lang)
+static int search_language_pack(const char *dir_name,const char *lang_name)
 {
+	//Search for a tessdata folder in the specified directory
+	char *lang = strdup(lang_name);
+	char *dirname = strdup(dir_name);
+	dirname = realloc(dirname,strlen(dirname)+strlen("/tessdata/")+1);
+	strcat(dirname,"/tessdata/");
+
 	DIR *dp;
 	struct dirent *dirp;
 	char filename[256];
@@ -66,13 +74,13 @@ static int search_language_pack(const char *dirname,const char *lang)
 void delete_ocr (void** arg)
 {
 	struct ocrCtx* ctx = *arg;
-        TessBaseAPIEnd(ctx->api);
-        TessBaseAPIDelete(ctx->api);
+	TessBaseAPIEnd(ctx->api);
+	TessBaseAPIDelete(ctx->api);
 	freep(arg);
 }
 void* init_ocr(int lang_index)
 {
-	int ret;
+	int ret = -1;
 	struct ocrCtx* ctx;
 
 	ctx = (struct ocrCtx*)malloc(sizeof(struct ocrCtx));
@@ -87,15 +95,44 @@ void* init_ocr(int lang_index)
 		lang_index = 1;
 	}
 
-	/* if langauge pack not found use english */
-	ret = search_language_pack("tessdata",language[lang_index]);
-	if(ret < 0 )
+	if(ccx_options.dvblang)
 	{
+		if(strcmp(language[lang_index],ccx_options.dvblang)!=0)
+			goto fail;
+	}
+
+	/*Priority of Tesseract traineddata file search paths:-
+		1. tessdata in TESSDATA_PREFIX, if it is specified. Overrides others
+		2. tessdata in current working directory
+	*/
+	int data_location = 0;
+	char *tessdata_dir_path=".";
+	if(!getenv("TESSDATA_PREFIX"))
+	{
+		ret = search_language_pack(tessdata_dir_path,language[lang_index]);
+	}
+	if(ret < 0)
+	{
+		data_location = 1;
+		if(getenv("TESSDATA_PREFIX"))
+			ret = search_language_pack(getenv("TESSDATA_PREFIX"), language[lang_index]);
+		else
+			ret = -1;
+	}
+	if(ret < 0 && lang_index != 1 && ccx_options.ocrlang==NULL)
+	{
+		mprint("%s.traineddata not found! Switching to English\n",language[lang_index]);
 		/* select english */
 		lang_index = 1;
 	}
 
-	ret = TessBaseAPIInit3(ctx->api, NULL, language[lang_index]);
+	if(ccx_options.ocrlang)
+		ret = TessBaseAPIInit3(ctx->api, NULL, ccx_options.ocrlang);
+	else if(data_location == 1)
+		ret = TessBaseAPIInit3(ctx->api, NULL, language[lang_index]);
+	else
+		ret = TessBaseAPIInit3(ctx->api, tessdata_dir_path, language[lang_index]);
+
 	if(ret < 0)
 	{
 		goto fail;
@@ -107,7 +144,7 @@ fail:
 
 }
 
-int ignore_alpha_at_edge(png_byte *alpha, unsigned char* indata, int w, int h, PIX *in, PIX **out)
+BOX* ignore_alpha_at_edge(png_byte *alpha, unsigned char* indata, int w, int h, PIX *in, PIX **out)
 {
 	int i, j, index, start_y, end_y;
 	int find_end_x = CCX_FALSE;
@@ -133,14 +170,16 @@ int ignore_alpha_at_edge(png_byte *alpha, unsigned char* indata, int w, int h, P
 	}
 	cropWindow = boxCreate(start_y, 0, (w - (start_y + ( w - end_y) )), h - 1);
 	*out = pixClipRectangle(in, cropWindow, NULL);
-	boxDestroy(&cropWindow);
+	//boxDestroy(&cropWindow);
 
-	return 0;
+	return cropWindow;
 }
-char* ocr_bitmap(void* arg, png_color *palette,png_byte *alpha, unsigned char* indata,int w, int h)
+char* ocr_bitmap(void* arg, png_color *palette,png_byte *alpha, unsigned char* indata,int w, int h, struct image_copy *copy)
 {
 	PIX	*pix = NULL;
 	PIX	*cpix = NULL;
+	PIX *color_pix = NULL;
+	PIX *color_pix_out = NULL;
 	char*text_out= NULL;
 	int i,j,index;
 	unsigned int wpl;
@@ -148,7 +187,8 @@ char* ocr_bitmap(void* arg, png_color *palette,png_byte *alpha, unsigned char* i
 	BOOL tess_ret = FALSE;
 	struct ocrCtx* ctx = arg;
 	pix = pixCreate(w, h, 32);
-	if(pix == NULL)
+	color_pix = pixCreate(w, h, 32);
+	if(pix == NULL||color_pix == NULL)
 	{
 		return NULL;
 	}
@@ -156,6 +196,7 @@ char* ocr_bitmap(void* arg, png_color *palette,png_byte *alpha, unsigned char* i
 	data = pixGetData(pix);
 #if LEPTONICA_VERSION > 69
 	pixSetSpp(pix, 4);
+	pixSetSpp(color_pix, 4);
 #endif
 	for (i = 0; i < h; i++)
 	{
@@ -169,12 +210,27 @@ char* ocr_bitmap(void* arg, png_color *palette,png_byte *alpha, unsigned char* i
 		}
 	}
 	ignore_alpha_at_edge(alpha, indata, w, h, pix, &cpix);
+	// For the unquantized bitmap
+	wpl = pixGetWpl(color_pix);
+	data = pixGetData(color_pix);
+	for (i = 0; i < h; i++)
+	{
+		ppixel = data + i * wpl;
+		for (j = 0; j < w; j++)
+		{
+			index = copy->data[i * w + (j)];
+			composeRGBPixel(copy->palette[index].red, copy->palette[index].green,copy->palette[index].blue, ppixel);
+			SET_DATA_BYTE(ppixel, L_ALPHA_CHANNEL,copy->alpha[index]);
+			ppixel++;
+		}
+	}
+	BOX *crop_points = ignore_alpha_at_edge(copy->alpha, copy->data, w, h, color_pix, &color_pix_out);
 #ifdef OCR_DEBUG
 	{
 	char str[128] = "";
 	static int i = 0;
-	sprintf(str,"temp/file_c_%d.png",i);
-	pixWrite(str, cpix, IFF_PNG);
+	sprintf(str,"temp/file_c_%d.jpg",i);
+	pixWrite(str, color_pix_out, IFF_JFIF_JPEG);
 	i++;
 	}
 #endif
@@ -184,8 +240,242 @@ char* ocr_bitmap(void* arg, png_color *palette,png_byte *alpha, unsigned char* i
 		printf("\nsomething messy\n");
 
 	text_out = TessBaseAPIGetUTF8Text(ctx->api);
+
+	// Begin color detection
+	if(ccx_options.dvbcolor && strlen(text_out)>0)
+	{
+		float h0 = -100;
+		int written_tag = 0;
+		TessBaseAPISetImage2(ctx->api, color_pix_out);
+		tess_ret = TessBaseAPIRecognize(ctx->api, NULL);
+		if( tess_ret != 0)
+			printf("\nsomething messy\n");
+		TessResultIterator* ri = TessBaseAPIGetIterator(ctx->api);
+		TessPageIteratorLevel level = RIL_WORD;
+
+		if(ri!=0)
+		{
+			do
+			{
+				char* word = TessResultIteratorGetUTF8Text(ri,level);
+				float conf = TessResultIteratorConfidence(ri,level);
+				int x1, y1, x2, y2;
+				TessPageIteratorBoundingBox((TessPageIterator *)ri,level, &x1, &y1, &x2, &y2);
+				// printf("word: '%s';  \tconf: %.2f; BoundingBox: %d,%d,%d,%d;",word, conf, x1, y1, x2, y2);
+				// printf("word: '%s';", word);
+				// {
+				// char str[128] = "";
+				// static int i = 0;
+				// sprintf(str,"temp/file_c_%d.jpg",i);
+				// pixWrite(str, pixClipRectangle(color_pix_out, boxCreate(x1,y1,x2-x1,y2-y1) ,NULL), IFF_JFIF_JPEG);
+				// i++;
+				// }
+
+				uint32_t *histogram = NULL;
+				uint8_t *iot = NULL;
+				uint32_t *mcit = NULL;
+				int ret = 0;
+				int max_color=2;
+
+				histogram = (uint32_t*) malloc(copy->nb_colors * sizeof(uint32_t));
+				iot = (uint8_t*) malloc(copy->nb_colors * sizeof(uint8_t));
+				mcit = (uint32_t*) malloc(copy->nb_colors * sizeof(uint32_t));
+				struct transIntensity ti = {copy->alpha,copy->palette};
+				memset(histogram, 0, copy->nb_colors * sizeof(uint32_t));
+
+				/* initializing intensity  ordered table with serial order of unsorted color table */
+				for (int i = 0; i < copy->nb_colors; i++)
+				{
+					iot[i] = i;
+				}
+				memset(mcit, 0, copy->nb_colors * sizeof(uint32_t));
+
+				/* calculate histogram of image */
+				int firstpixel = copy->data[0];  //TODO: Verify this border pixel assumption holds
+				for(int i=y1;i<=y2;i++)
+				{
+					for(int j=x1;j<=x2;j++)
+					{
+						if(copy->data[(crop_points->y+i)*w + (crop_points->x+j)]!=firstpixel)
+							histogram[copy->data[(crop_points->y+i)*w + (crop_points->x+j)]]++;
+					}
+				}
+				/* sorted in increasing order of intensity */
+				shell_sort((void*)iot, copy->nb_colors, sizeof(*iot), check_trans_tn_intensity, (void*)&ti);
+				// ccx_common_logging.log_ftn("Intensity ordered table\n");
+				// for (int i = 0; i < copy->nb_colors; i++)
+				// {
+				// 	ccx_common_logging.log_ftn("%02d) map %02d hist %02d\n",
+				// 		i, iot[i], histogram[iot[i]]);
+				// }
+				/**
+				 * using selection  sort since need to find only max_color
+				 * Histogram becomes invalid in this loop
+				 */
+				for (int i = 0; i < max_color; i++)
+				{
+					uint32_t max_val = 0;
+					uint32_t max_ind = 0;
+					int j;
+					for (j = 0; j < copy->nb_colors; j++)
+					{
+						if (max_val < histogram[iot[j]])
+						{
+							max_val = histogram[iot[j]];
+							max_ind = j;
+						}
+					}
+					for (j = i; j > 0 && max_ind < mcit[j - 1]; j--)
+					{
+						mcit[j] = mcit[j - 1];
+					}
+					mcit[j] = max_ind;
+					histogram[iot[max_ind]] = 0;
+				}
+				for (int i = 0; i < copy->nb_colors; i++)
+				{
+					palette[i].red = copy->palette[i].red;
+					palette[i].green = copy->palette[i].green;
+					palette[i].blue = copy->palette[i].blue;
+					alpha[i]=copy->alpha[i];
+				}
+				
+				for (int i = 0, mxi = 0; i < copy->nb_colors; i++)
+				{
+					int step, inc;
+					if (i == mcit[mxi])
+					{
+						mxi = (mxi < max_color) ? mxi + 1 : mxi;
+						continue;
+					}
+					inc = (mxi) ? -1 : 0;
+					step = mcit[mxi + inc] + ((mcit[mxi] - mcit[mxi + inc]) / 2);
+					if (i <= step)
+					{
+						int index = iot[mcit[mxi + inc]];
+						alpha[iot[i]] = alpha[index];
+						palette[iot[i]].red = palette[index].red;
+						palette[iot[i]].blue = palette[index].blue;
+						palette[iot[i]].green = palette[index].green;
+					}
+					else
+					{
+						int index = iot[mcit[mxi]];
+						alpha[iot[i]] = alpha[index];
+						palette[iot[i]].red = palette[index].red;
+						palette[iot[i]].blue = palette[index].blue;
+						palette[iot[i]].green = palette[index].green;
+					}
+
+				}
+				
+				// Detecting the color present in quantized word image			
+				int r_avg=0,g_avg=0,b_avg=0,denom=0;
+				for (int i = 0; i < copy->nb_colors; i++)
+				{
+					if(palette[i].red == ((copy->bgcolor >> 16) & 0xff) &&
+					   palette[i].green == ((copy->bgcolor >> 8) & 0xff) && 
+					   palette[i].blue == ((copy->bgcolor >> 0) & 0xff))
+						continue;
+					denom++;
+					r_avg+=palette[i].red;
+					g_avg+=palette[i].green;
+					b_avg+=palette[i].blue;
+				}
+				if(denom!=0)
+				{
+					r_avg/=denom;
+					g_avg/=denom;
+					b_avg/=denom;
+				}
+
+				// Getting the hue value
+				float h;
+				float max = (((r_avg > g_avg) && (r_avg > b_avg)) ? r_avg : (g_avg > b_avg) ? g_avg : b_avg);
+				float min = (((r_avg < g_avg) && (r_avg < b_avg)) ? r_avg : (g_avg < b_avg) ? g_avg : b_avg);
+				if(max==0.0f||max-min==0.0f) h = 0;
+				else if(max==r_avg) h = 60 * ((g_avg - b_avg)/(max - min)) + 0;
+				else if(max==g_avg) h = 60 * ((b_avg - r_avg)/(max - min)) + 120;
+				else h = 60 * ((r_avg - g_avg)/(max - min)) + 240;
+
+				if(abs(h-h0)>50) // Color has changed
+				{
+					// Write <font> tags for SRT and WebVTT
+					if(ccx_options.write_format==CCX_OF_SRT ||
+					   ccx_options.write_format==CCX_OF_WEBVTT)
+					{
+						char *substr;
+						if(written_tag)
+						{
+							substr = (char*)malloc(sizeof("</font><font color=\"#000000\">"));
+							sprintf(substr,"</font><font color=\"#%02x%02x%02x\">",r_avg,g_avg,b_avg);
+						}
+						else
+						{
+							substr = (char*)malloc(sizeof("<font color=\"#000000\">"));
+							sprintf(substr,"<font color=\"#%02x%02x%02x\">",r_avg,g_avg,b_avg);
+						}
+						if(strstr(text_out,word))
+						{
+							char *text_out_copy = strdup(text_out);
+							free(text_out);
+							text_out = malloc(strlen(text_out_copy)+strlen(substr)+1);
+							memset(text_out,0,strlen(text_out_copy)+strlen(substr)+1);
+							int pos = (int)(strstr(text_out_copy,word)-text_out_copy);
+							strncpy(text_out,text_out_copy,pos);
+							int len = strlen(text_out);
+							strcpy(text_out+len,substr);
+							strcpy(text_out+len+strlen(substr),text_out_copy+len);
+							free(text_out_copy);
+							written_tag = 1;
+						}
+						else if(!written_tag)
+						{
+							char *text_out_copy = strdup(text_out);
+							free(text_out);
+							text_out = malloc(strlen(text_out_copy)+strlen(substr)+1);
+							memset(text_out,0,strlen(text_out_copy)+strlen(substr)+1);
+							strcpy(text_out,substr);
+							strcpy(text_out+strlen(substr),text_out_copy);
+							written_tag = 1;
+						}
+						free(substr);
+					}
+				}
+
+				h0=h;
+
+				freep(&histogram);
+				freep(&mcit);
+				freep(&iot);
+				
+			} while (TessPageIteratorNext((TessPageIterator *)ri,level));
+
+			//Write closing </font> at the end of the line
+			if(ccx_options.write_format==CCX_OF_SRT ||
+			   ccx_options.write_format==CCX_OF_WEBVTT)
+			{
+				char *substr = "</font>";
+				char *text_out_copy = strdup(text_out);
+				free(text_out);
+				text_out = malloc(strlen(text_out_copy)+strlen(substr)+1);
+				memset(text_out,0,strlen(text_out_copy)+strlen(substr)+1);
+				char *str = strtok(text_out_copy,"\n");
+				strcpy(text_out,str);
+				strcpy(text_out+strlen(str),substr);
+			// printf("%s\n", text_out);
+			}
+		}
+
+		TessResultIteratorDelete(ri);
+
+	}
+	// End Color Detection
+
 	pixDestroy(&pix);
 	pixDestroy(&cpix);
+	pixDestroy(&color_pix);
+	pixDestroy(&color_pix_out);
 
 	return text_out;
 }
@@ -336,33 +626,53 @@ static int quantize_map(png_byte *alpha, png_color *palette,
 	return ret;
 }
 
-int ocr_rect(void* arg, struct cc_bitmap *rect, char **str)
+int ocr_rect(void* arg, struct cc_bitmap *rect, char **str, int bgcolor)
 {
 	int ret = 0;
 	png_color *palette = NULL;
 	png_byte *alpha = NULL;
+	
+	struct image_copy *copy;
+	copy = (struct image_copy *)malloc(sizeof(struct image_copy));
+	copy->nb_colors = rect->nb_colors;
+	copy->palette = (png_color*) malloc(rect->nb_colors * sizeof(png_color));
+	copy->alpha = (png_byte*) malloc(rect->nb_colors * sizeof(png_byte));
+	copy->bgcolor = bgcolor;
 
 	palette = (png_color*) malloc(rect->nb_colors * sizeof(png_color));
-	if(!palette)
+	if(!palette||!copy->palette)
 	{
 		ret = -1;
 		goto end;
 	}
-        alpha = (png_byte*) malloc(rect->nb_colors * sizeof(png_byte));
-        if(!alpha)
-        {
-                ret = -1;
-                goto end;
-        }
+		alpha = (png_byte*) malloc(rect->nb_colors * sizeof(png_byte));
+		if(!alpha||!copy->alpha)
+		{
+				ret = -1;
+				goto end;
+		}
 
-        mapclut_paletee(palette, alpha, (uint32_t *)rect->data[1],rect->nb_colors);
+		mapclut_paletee(palette, alpha, (uint32_t *)rect->data[1],rect->nb_colors);
+		mapclut_paletee(copy->palette, copy->alpha, (uint32_t *)rect->data[1],rect->nb_colors);
 
-        quantize_map(alpha, palette, rect->data[0], rect->w * rect->h, 3, rect->nb_colors);
-        *str = ocr_bitmap(arg, palette, alpha, rect->data[0], rect->w, rect->h);
+		int size = rect->w * rect->h;
+		copy->data = (unsigned char *)malloc(sizeof(unsigned char)*size);
+		for(int i = 0; i < size; i++)
+		{
+			copy->data[i] = rect->data[0][i];
+		}
+
+
+		quantize_map(alpha, palette, rect->data[0], size, 3, rect->nb_colors);
+		*str = ocr_bitmap(arg, palette, alpha, rect->data[0], rect->w, rect->h, copy);
 
 end:
 	freep(&palette);
 	freep(&alpha);
+	freep(&copy->palette);
+	freep(&copy->alpha);
+	freep(&copy->data);
+	freep(&copy);
 	return ret;
 
 }
@@ -390,11 +700,29 @@ int compare_rect_by_ypos(const void*p1, const void *p2, void*arg)
 	}
 }
 
+void add_ocrtext2str(char *dest, char *src, const char *crlf, unsigned crlf_length)
+{
+	while (*dest != '\0')
+		dest++;
+	while(*src != '\0' && *src != '\n')
+	{
+		*dest = *src;
+		src++;
+		dest++;
+	}
+	memcpy(dest, crlf, crlf_length);
+	dest[crlf_length] = 0;	
+	/*
+	*dest++ = '\n';
+	*dest = '\0'; */
+}
+
 /**
  * Check multiple rectangles and combine them to give one paragraph
  * for all text detected from rectangles
  */
-char *paraof_ocrtext(struct cc_subtitle *sub)
+
+char *paraof_ocrtext(struct cc_subtitle *sub, const char *crlf, unsigned crlf_length)
 {
 	int i;
 	int len = 0;
@@ -411,7 +739,7 @@ char *paraof_ocrtext(struct cc_subtitle *sub)
 		return NULL;
 	else
 	{
-		str = malloc(len+1);
+		str = malloc(len+1+10); //Extra space for possible trailing '/n's at the end of tesseract UTF8 text
 		if(!str)
 			return NULL;
 		*str = '\0';
@@ -419,13 +747,13 @@ char *paraof_ocrtext(struct cc_subtitle *sub)
 
 	for(i = 0, rect = sub->data; i < sub->nb_data; i++, rect++)
 	{
-		strcat(str, rect->ocr_text);
-		freep(&rect->ocr_text);
+		add_ocrtext2str(str, rect->ocr_text, crlf, crlf_length);
+		TessDeleteText(rect->ocr_text);
 	}
 	return str;
 }
 #else
-char* ocr_bitmap(png_color *palette,png_byte *alpha, unsigned char* indata,unsigned char d,int w, int h)
+char* ocr_bitmap(png_color *palette,png_byte *alpha, unsigned char* indata,unsigned char d,int w, int h, struct image_copy *copy)
 {
 	mprint("ocr not supported without tesseract\n");
 	return NULL;

src/lib_ccx/ocr.h

@@ -2,10 +2,19 @@
 #define OCR_H
 #include <png.h>
 
+struct image_copy //A copy of the original OCR image, used for color detection
+{
+	int nb_colors;
+	png_color *palette;
+	png_byte *alpha;
+	unsigned char *data;
+	int bgcolor;
+};
+
 void delete_ocr (void** arg);
 void* init_ocr(int lang_index);
-char* ocr_bitmap(void* arg, png_color *palette,png_byte *alpha, unsigned char* indata,int w, int h);
-int ocr_rect(void* arg, struct cc_bitmap *rect, char **str);
-char *paraof_ocrtext(struct cc_subtitle *sub);
+char* ocr_bitmap(void* arg, png_color *palette,png_byte *alpha, unsigned char* indata,int w, int h, struct image_copy *copy);
+int ocr_rect(void* arg, struct cc_bitmap *rect, char **str, int bgcolor);
+char *paraof_ocrtext(struct cc_subtitle *sub, const char *crlf, unsigned crlf_length);
 
 #endif

src/lib_ccx/output.c

@@ -11,18 +11,72 @@ void dinit_write(struct ccx_s_write *wb)
 	if (wb->fh > 0)
 		close(wb->fh);
 	freep(&wb->filename);	
+	if (wb->with_semaphore && wb->semaphore_filename)
+		unlink(wb->semaphore_filename);
+	freep(&wb->semaphore_filename);
 }
-int init_write (struct ccx_s_write *wb, char *filename)
+
+int temporarily_close_output(struct ccx_s_write *wb)
 {
-	memset(wb, 0, sizeof(struct ccx_s_write));
-	wb->fh=-1;
-	wb->filename = filename;
-	mprint ("Creating %s\n", filename);
-	wb->fh = open (filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, S_IREAD | S_IWRITE);
+	close(wb->fh);
+	wb->fh = -1;
+	wb->temporarily_closed = 1;
+	return 0;
+}
+
+int temporarily_open_output(struct ccx_s_write *wb)
+{
+	int t = 0;
+	// Try a few times before giving up. This is because this close/open stuff exists for processes
+	// that demand exclusive access to the file, so often we'll find out that we cannot reopen the
+	// file immediately.
+	while (t < 5 && wb->fh == -1)
+	{
+		wb->fh = open(wb->filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, S_IREAD | S_IWRITE);
+		sleep(1);
+	}
 	if (wb->fh == -1)
 	{
 		return CCX_COMMON_EXIT_FILE_CREATION_FAILED;
 	}
+	wb->temporarily_closed = 0;
+	return EXIT_OK;
+}
+
+
+
+int init_write (struct ccx_s_write *wb, char *filename, int with_semaphore)
+{
+	memset(wb, 0, sizeof(struct ccx_s_write));
+	wb->fh=-1;
+	wb->temporarily_closed = 0; 
+	wb->filename = filename;
+	wb->with_semaphore = with_semaphore;
+	wb->append_mode = ccx_options.enc_cfg.append_mode;
+	mprint ("Creating %s\n", filename);
+	if(!(wb->append_mode))
+		wb->fh = open (filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, S_IREAD | S_IWRITE);
+	else
+		wb->fh = open (filename, O_RDWR | O_CREAT | O_APPEND | O_BINARY, S_IREAD | S_IWRITE);
+	wb->renaming_extension = 0;
+	if (wb->fh == -1)
+	{
+		return CCX_COMMON_EXIT_FILE_CREATION_FAILED;
+	}
+	if (with_semaphore)
+	{
+		wb->semaphore_filename = (char *)malloc(strlen(filename) + 6);
+		if (!wb->semaphore_filename)
+			return EXIT_NOT_ENOUGH_MEMORY;
+		sprintf(wb->semaphore_filename, "%s.sem", filename);
+		int t = open(wb->semaphore_filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, S_IREAD | S_IWRITE);
+		if (t == -1)
+		{
+			close(wb->fh);
+			return CCX_COMMON_EXIT_FILE_CREATION_FAILED;
+		}
+		close(t);
+	}
 	return EXIT_OK;
 }
 

src/lib_ccx/params.c

@@ -5,6 +5,11 @@
 #include "ccx_encoders_helpers.h"
 #include "ccx_common_common.h"
 #include "ccx_decoders_708.h"
+#include "compile_info.h"
+#include "../lib_hash/sha2.h"
+#ifdef ENABLE_HARDSUBX
+#include "hardsubx.h"
+#endif
 
 static int inputfile_capacity=0;
 
@@ -214,6 +219,8 @@ void set_output_format (struct ccx_s_options *opt, const char *format)
 		opt->write_format=CCX_OF_SPUPNG;
 	else if (strcmp (format,"simplexml")==0)
 		opt->write_format=CCX_OF_SIMPLE_XML;
+	else if (strcmp (format,"g608")==0)
+		opt->write_format=CCX_OF_G608;
 	else
 		fatal (EXIT_MALFORMED_PARAMETER, "Unknown output file format: %s\n", format);
 }
@@ -272,7 +279,8 @@ void usage (void)
 	mprint ("     .mp4, HDHomeRun are known to work).\n\n");
 	mprint ("  Syntax:\n");
 	mprint ("  ccextractor [options] inputfile1 [inputfile2...] [-o outputfilename]\n");
-	mprint ("               [-o1 outputfilename1] [-o2 outputfilename2]\n\n");
+	mprint ("\n");
+	mprint ("To see This Help Message: -h or --help\n\n");
 	mprint ("File name related options:\n");
 	mprint ("            inputfile: file(s) to process\n");
 	mprint ("    -o outputfilename: Use -o parameters to define output filename if you don't\n");
@@ -304,17 +312,21 @@ void usage (void)
 	mprint ("                              port) instead of reading a file. Host can be a\n");
 	mprint ("                              hostname or IPv4 address. If host is not specified\n");
 	mprint ("                              then listens on the local host.\n\n");
-	mprint ("            -sendto host[:port]: Sends data in BIN format to the server according\n");
-	mprint ("                                 to the CCExtractor's protocol over TCP. For IPv6\n");
-	mprint ("                                 use [addres]:port\n");
-	mprint ("            -tcp port: Reads the input data in BIN format according to CCExtractor's\n");
-	mprint ("                       protocol, listening specified port on the local host\n");
-	mprint ("            -tcppassword password: Sets server password for new connections to tcp server\n");
-	mprint ("            -tcpdesc description: Sends to the server short description about captions e.g.\n");
-	mprint ("                                  channel name or file name\n");
+	mprint ("            -sendto host[:port]: Sends data in BIN format to the server\n");
+	mprint ("                                 according to the CCExtractor's protocol over\n");
+	mprint ("                                 TCP. For IPv6 use [address]:port\n");
+	mprint ("            -tcp port: Reads the input data in BIN format according to\n");
+	mprint ("                        CCExtractor's protocol, listening specified port on the\n");
+	mprint ("                        local host\n");
+	mprint ("            -tcppassword password: Sets server password for new connections to\n");
+	mprint ("                                   tcp server\n");
+	mprint ("            -tcpdesc description: Sends to the server short description about\n");
+	mprint ("                                  captions e.g. channel name or file name\n");
 	mprint ("Options that affect what will be processed:\n");
 	mprint ("          -1, -2, -12: Output Field 1 data, Field 2 data, or both\n");
 	mprint ("                       (DEFAULT is -1)\n");
+	mprint ("Use --append to prevent overwriting of existing files. The output will be\n");
+	mprint ("      appended instead.\n");
 	mprint ("                 -cc2: When in srt/sami mode, process captions in channel 2\n");
 	mprint ("                       instead of channel 1.\n");
 	mprint ("-svc --service N1[cs1],N2[cs2]...:\n");
@@ -324,11 +336,12 @@ void usage (void)
 	mprint ("                       primary and secondary language services.\n");
 	mprint ("                       Pass \"all\" to process all services found.\n");
 	mprint ("\n");
-	mprint ("                       If captions in a service are stored in 16-bit encoding, you can\n");
-	mprint ("                       specify what charset or encoding was used. Pass its name after\n");
-	mprint ("                       service number (e.g. \"1[EUC-KR],3\" or \"all[EUC-KR]\") and it will\n");
-	mprint ("                       encode specified charset to UTF-8 using iconv. See iconv documentation\n");
-	mprint ("                       to check if required encoding/charset is supported.\n");
+	mprint ("                       If captions in a service are stored in 16-bit encoding,\n");
+	mprint ("                       you can specify what charset or encoding was used. Pass\n");
+	mprint ("                       its name after service number (e.g. \"1[EUC-KR],3\" or\n");
+	mprint ("                       \"all[EUC-KR]\") and it will encode specified charset to\n");
+	mprint ("                       UTF-8 using iconv. See iconv documentation to check if\n");
+	mprint ("                       required encoding/charset is supported.\n");
 	mprint ("\n");
 	mprint ("In general, if you want English subtitles you don't need to use these options\n");
 	mprint ("as they are broadcast in field 1, channel 1. If you want the second language\n");
@@ -372,7 +385,6 @@ void usage (void)
 	mprint ("                      report  -> Prints to stdout information about captions\n");
 	mprint ("                                 in specified input. Don't produce any file\n");
 	mprint ("                                 output\n\n");
-	mprint ("       Note: Teletext output can only be srt, webvtt, txt or ttxt for now.\n\n");
 
 	mprint ("Options that affect how input files will be processed.\n");
 
@@ -453,20 +465,22 @@ void usage (void)
 	mprint ("                       reference to the received data. Use this parameter if\n");
 	mprint ("                       you prefer your own reference. Note: Current this only\n");
 	mprint ("                       affects Teletext in timed transcript with -datets.\n");
+	mprint ("           --noscte20: Ignore SCTE-20 data if present.\n");
 	mprint ("\n");
 	mprint ("Options that affect what kind of output will be produced:\n");
-	mprint("				  -bom: Append a BOM (Byte Order Mark) to output files.\n");
-	mprint("						Note that most text processing tools in linux will not\n");
-	mprint("						like BOM.\n");
-	mprint("						This is the default in Windows builds.\n");
-	mprint("				-nobom: Do not append a BOM (Byte Order Mark) to output files.\n");
-	mprint("						Note that this may break files when using Windows.\n");
-	mprint("						This is the default in non-Windows builds.\n");
+	mprint ("                 -bom: Append a BOM (Byte Order Mark) to output files.\n");
+	mprint ("                       Note that most text processing tools in linux will not\n");
+	mprint ("                       like BOM.\n");
+	mprint ("                       This is the default in Windows builds.\n");
+	mprint ("                       -nobom: Do not append a BOM (Byte Order Mark) to output\n");
+	mprint ("                       files. Note that this may break files when using\n");
+	mprint ("                       Windows. This is the default in non-Windows builds.\n");
 	mprint ("             -unicode: Encode subtitles in Unicode instead of Latin-1.\n");
 	mprint ("                -utf8: Encode subtitles in UTF-8 (no longer needed.\n");
 	mprint ("                       because UTF-8 is now the default).\n");
 	mprint ("              -latin1: Encode subtitles in Latin-1\n");
 	mprint ("  -nofc --nofontcolor: For .srt/.sami/.vtt, don't add font color tags.\n");
+	mprint ("  --nohtmlescape: For .srt/.sami/.vtt, don't covert html unsafe character\n");
 	mprint ("-nots --notypesetting: For .srt/.sami/.vtt, don't add typesetting tags.\n");
 	mprint ("                -trim: Trim lines.\n");
 	mprint ("   -dc --defaultcolor: Select a different default color (instead of\n");
@@ -503,16 +517,22 @@ void usage (void)
 	mprint ("            -autodash: Based on position on screen, attempt to determine\n");
 	mprint ("                       the different speakers and a dash (-) when each\n");
 	mprint ("                       of them talks (.srt/.vtt only, -trim required).\n");
-	mprint ("         -xmltv mode:  produce an XMLTV file containing the EPG data from\n");
+	mprint ("          -xmltv mode: produce an XMLTV file containing the EPG data from\n");
 	mprint ("                       the source TS file. Mode: 1 = full output\n");
-	mprint ("                       2 = live output. 3 = both\n\n");
-
+	mprint ("                       2 = live output. 3 = both\n");
+	mprint ("                 -sem: Create a .sem file for each output file that is open\n");
+	mprint ("                       and delete it on file close.\n");
+	mprint ("\n");
 	mprint ("Options that affect how ccextractor reads and writes (buffering):\n");
 
 	mprint ("    -bi --bufferinput: Forces input buffering.\n");
 	mprint (" -nobi -nobufferinput: Disables input buffering.\n");
 	mprint (" -bs --buffersize val: Specify a size for reading, in bytes (suffix with K or\n");
 	mprint ("                       or M for kilobytes and megabytes). Default is 16M.\n");
+	mprint ("                 -koc: keep-output-close. If used then CCExtractor will close\n"); 
+	mprint ("                       the output file after writing each subtitle frame and\n");
+	mprint ("                       attempt to create it again when needed.\n");
+	mprint ("     -ff --forceflush: Flush the file buffer whenever content is written.\n");
 	mprint ("\n");
 
 	mprint ("Options that affect the built-in closed caption decoder:\n");
@@ -648,27 +668,29 @@ void usage (void)
 	mprint ("                       then CEA-608/708 processing is disabled).\n");
 	mprint ("\n");
 
-	mprint("Transcript customizing options:\n");
+	mprint ("Transcript customizing options:\n");
 
-	mprint("    -customtxt format: Use the passed format to customize the (Timed) Transcript\n");
-	mprint("                       output. The format must be like this: 1100100 (7 digits).\n");
-	mprint("                       These indicate whether the next things should be displayed\n");
-	mprint("                       or not in the (timed) transcript. They represent (in order):\n");
-	mprint("                           - Display start time\n");
-	mprint("                           - Display end time\n");
-	mprint("                           - Display caption mode\n");
-	mprint("                           - Display caption channel\n");
-	mprint("                           - Use a relative timestamp ( relative to the sample)\n");
-	mprint("                           - Display XDS info\n");
-	mprint("                           - Use colors\n");
-	mprint("                       Examples:\n");
-	mprint("                       0000101 is the default setting for transcripts\n");
-	mprint("                       1110101 is the default for timed transcripts\n");
-	mprint("                       1111001 is the default setting for -ucla\n");
-	mprint("                       Make sure you use this parameter after others that might\n");
-	mprint("                       affect these settings (-out, -ucla, -xds, -txt, -ttxt, ...)\n");
+	mprint ("    -customtxt format: Use the passed format to customize the (Timed) Transcript\n");
+	mprint ("                       output. The format must be like this: 1100100 (7 digits).\n");
+	mprint ("                       These indicate whether the next things should be\n");
+	mprint ("                       displayed or not in the (timed) transcript. They\n");
+	mprint ("                       represent (in order): \n");
+	mprint ("                           - Display start time\n");
+	mprint ("                           - Display end time\n");
+	mprint ("                           - Display caption mode\n");
+	mprint ("                           - Display caption channel\n");
+	mprint ("                           - Use a relative timestamp ( relative to the sample)\n");
+	mprint ("                           - Display XDS info\n");
+	mprint ("                           - Use colors\n");
+	mprint ("                       Examples:\n");
+	mprint ("                       0000101 is the default setting for transcripts\n");
+	mprint ("                       1110101 is the default for timed transcripts\n");
+	mprint ("                       1111001 is the default setting for -ucla\n");
+	mprint ("                       Make sure you use this parameter after others that might\n");
+	mprint ("                       affect these settings (-out, -ucla, -xds, -txt, \n");
+	mprint ("                       -ttxt ...)\n");
 
-	mprint("\n");
+	mprint ("\n");
 
 	mprint ("Communication with other programs and console output:\n");
 
@@ -708,6 +730,92 @@ void usage (void)
 	mprint("    /tmp/output_2.d/sub0000.png\n");
 	mprint("    /tmp/output_2.d/sub0001.png\n");
 	mprint("    ...\n");
+	mprint("\n");
+	mprint("Burned-in subtitle extraction:\n");
+	mprint("         -hardsubx : Enable the burned-in subtitle extraction subsystem.\n");
+	mprint("\n");
+	mprint("         NOTE: The following options will work only if -hardsubx is \n");
+	mprint("                specified before them:-\n");
+	mprint("\n");
+	mprint("         -ocr_mode : Set the OCR mode to either frame-wise, word-wise\n");
+	mprint("                     or letter wise.\n");
+	mprint("                     e.g. -ocr_mode frame (default), -ocr_mode word, \n");
+	mprint("                     -ocr_mode letter\n");
+	mprint("\n");
+	mprint("         -subcolor : Specify the color of the subtitles\n");
+	mprint("                     Possible values are in the set \n");
+	mprint("                     {white,yellow,green,cyan,blue,magenta,red}.\n");
+	mprint("                     Alternatively, a custom hue value between 1 and 360 \n");
+	mprint("                     may also be specified.\n");
+	mprint("                     e.g. -subcolor white or -subcolor 270 (for violet).\n");
+	mprint("                     Refer to an HSV color chart for values.\n");
+	mprint("\n");
+	mprint(" -min_sub_duration : Specify the minimum duration that a subtitle line \n");
+	mprint("                     must exist on the screen.\n");
+	mprint("                     The value is specified in seconds.\n");
+	mprint("                     A lower value gives better results, but takes more \n");
+	mprint("                     processing time.\n");
+	mprint("                     The recommended value is 0.5 (default).\n");
+	mprint("                     e.g. -min_sub_duration 1.0 (for a duration of 1 second)\n");
+	mprint("\n");
+	mprint("   -detect_italics : Specify whether italics are to be detected from the \n");
+	mprint("                     OCR text.\n");
+	mprint("                     Italic detection automatically enforces the OCR mode \n");
+	mprint("                     to be word-wise");
+	mprint("\n");
+	mprint("      -conf_thresh : Specify the classifier confidence threshold between\n");
+	mprint("                      1 and 100.\n");
+	mprint("                     Try and use a threshold which works for you if you get \n");
+	mprint("                     a lot of garbage text.\n");
+	mprint("                     e.g. -conf_thresh 50\n");
+	mprint("\n");
+	mprint(" -whiteness_thresh : For white subtitles only, specify the luminance \n");
+	mprint("                     threshold between 1 and 100\n");
+	mprint("                     This threshold is content dependent, and adjusting\n");
+	mprint("                     values may give you better results\n");
+	mprint("                     Recommended values are in the range 80 to 100.\n");
+	mprint("                     The default value is 95\n");
+	mprint("\n");
+	mprint("An example command is as follows:-\n");
+	mprint("ccextractor video.mp4 -hardsubx -subcolor white -detect_italics \n");
+	mprint("-whiteness_thresh 90 -conf_thresh 60\n");
+	mprint("\n");
+}
+
+unsigned char sha256_buf[16384];
+
+char *calculateSHA256(char *location) {
+	int	size_read, bytes_read, fh = 0;
+	SHA256_CTX	ctx256;
+
+	SHA256_Init(&ctx256);
+
+	#ifdef _WIN32
+		fh = OPEN(location, O_RDONLY | O_BINARY);
+	#else
+		fh = OPEN(location, O_RDONLY);
+	#endif
+
+	if (fh < 0) {
+		return "Could not open file";
+	}
+	size_read = 0;
+	while ((bytes_read = read(fh, sha256_buf, 16384)) > 0) {
+		size_read += bytes_read;
+		SHA256_Update(&ctx256, (unsigned char*)sha256_buf, bytes_read);
+	}
+	close(fh);
+	SHA256_End(&ctx256, sha256_buf);
+	return sha256_buf;
+}
+
+void version(char *location) {
+	char *hash = calculateSHA256(location);
+	mprint("CCExtractor detailed version info\n");
+	mprint("	Version: %s\n", VERSION);
+	mprint("	Git commit: %s\n", GIT_COMMIT);
+	mprint("	Compilation date: %s\n", COMPILE_DATE);
+	mprint("	File SHA256: %s\n", hash);
 }
 
 void parse_708_services (struct ccx_s_options *opts, char *s)
@@ -829,6 +937,16 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 	// Parse parameters
 	for (int i=1; i<argc; i++)
 	{
+		if (!strcmp (argv[i],"--help") || !strcmp(argv[i], "-h"))
+		{
+			usage();
+			return EXIT_WITH_HELP;
+		}
+		if (!strcmp(argv[i], "--version"))
+		{
+			version(argv[0]);
+			return EXIT_WITH_HELP;
+		}
 		if (strcmp (argv[i], "-")==0 || strcmp(argv[i], "-stdin") == 0)
 		{
 
@@ -854,6 +972,168 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			}
 			continue;
 		}
+		
+#ifdef ENABLE_HARDSUBX
+		// Parse -hardsubx and related parameters
+		if (strcmp(argv[i], "-hardsubx")==0)
+		{
+			opt->hardsubx = 1;
+			continue;
+		}
+		if (opt->hardsubx == 1)
+		{
+			if (strcmp(argv[i], "-ocr_mode")==0)
+			{
+				if(i < argc - 1)
+				{
+					if(strcmp(argv[i+1], "simple")==0 || strcmp(argv[i+1], "frame")==0)
+					{
+						opt->hardsubx_ocr_mode = HARDSUBX_OCRMODE_FRAME;
+					}
+					else if(strcmp(argv[i+1], "word")==0)
+					{
+						opt->hardsubx_ocr_mode = HARDSUBX_OCRMODE_WORD;
+					}
+					else if(strcmp(argv[i+1], "letter")==0 || strcmp(argv[i+1], "symbol")==0)
+					{
+						opt->hardsubx_ocr_mode = HARDSUBX_OCRMODE_LETTER;
+					}
+					else
+					{
+						fatal (EXIT_MALFORMED_PARAMETER, "-ocr_mode has an invalid value.\nValid values are {frame,word,letter}");
+					}
+				}
+				else
+				{
+					fatal (EXIT_MALFORMED_PARAMETER, "-ocr_mode has no argument.\nValid values are {frame,word,letter}");
+				}
+				i++;
+				continue;
+			}
+			if (strcmp(argv[i], "-subcolor")==0 || strcmp(argv[i], "-sub_color")==0)
+			{
+				if(i < argc - 1)
+				{
+					if(strcmp(argv[i+1], "white")==0)
+					{
+						opt->hardsubx_subcolor = HARDSUBX_COLOR_WHITE;
+						opt->hardsubx_hue = 0.0;
+					}
+					else if(strcmp(argv[i+1], "yellow")==0)
+					{
+						opt->hardsubx_subcolor = HARDSUBX_COLOR_YELLOW;
+						opt->hardsubx_hue = 60.0;
+					}
+					else if(strcmp(argv[i+1], "green")==0)
+					{
+						opt->hardsubx_subcolor = HARDSUBX_COLOR_GREEN;
+						opt->hardsubx_hue = 120.0;
+					}
+					else if(strcmp(argv[i+1], "cyan")==0)
+					{
+						opt->hardsubx_subcolor = HARDSUBX_COLOR_CYAN;
+						opt->hardsubx_hue = 180.0;
+					}
+					else if(strcmp(argv[i+1], "blue")==0)
+					{
+						opt->hardsubx_subcolor = HARDSUBX_COLOR_BLUE;
+						opt->hardsubx_hue = 240.0;
+					}
+					else if(strcmp(argv[i+1], "magenta")==0)
+					{
+						opt->hardsubx_subcolor = HARDSUBX_COLOR_MAGENTA;
+						opt->hardsubx_hue = 300.0;
+					}
+					else if(strcmp(argv[i+1], "red")==0)
+					{
+						opt->hardsubx_subcolor = HARDSUBX_COLOR_RED;
+						opt->hardsubx_hue = 0.0;
+					}
+					else
+					{
+						// Take a custom hue from the user
+						opt->hardsubx_subcolor = HARDSUBX_COLOR_CUSTOM;
+						char *str=(char*)malloc(sizeof(argv[i+1]));
+						sprintf(str,"%s", argv[i+1]); // Done this way to avoid error with getting (i+1)th env variable
+						opt->hardsubx_hue = atof(str);
+						if(opt->hardsubx_hue <= 0.0 || opt->hardsubx_hue > 360.0)
+						{
+							fatal (EXIT_MALFORMED_PARAMETER, "-subcolor has either 0 or an invalid hue value supplied.\nIf you want to detect red subtitles, pass '-subcolor red' or a slightly higher hue value (e.g. 0.1)\n");
+						}
+					}
+				}
+				else
+				{
+					fatal (EXIT_MALFORMED_PARAMETER, "-subcolor has no argument.\nValid values are {white,yellow,green,cyan,blue,magenta,red} or a custom hue value between 0 and 360\n");
+				}
+				i++;
+				continue;
+			}
+			if (strcmp(argv[i], "-min_sub_duration")==0)
+			{
+				if(i < argc - 1)
+				{
+					char *str=(char*)malloc(sizeof(argv[i+1]));
+					sprintf(str,"%s", argv[i+1]); // Done this way to avoid error with getting (i+1)th env variable
+					opt->hardsubx_min_sub_duration = atof(str);
+					if(opt->hardsubx_min_sub_duration == 0.0)
+					{
+						fatal (EXIT_MALFORMED_PARAMETER, "-min_sub_duration has either 0 or an invalid value supplied\n");
+					}
+				}
+				else
+				{
+					fatal (EXIT_MALFORMED_PARAMETER, "-min_sub_duration has no argument.");
+				}
+				i++;
+				continue;
+			}
+			if (strcmp(argv[i], "-detect_italics")==0)
+			{
+				opt->hardsubx_detect_italics = 1;
+				continue;
+			}
+			if (strcmp(argv[i], "-conf_thresh")==0)
+			{
+				if(i < argc - 1)
+				{
+					char *str=(char*)malloc(sizeof(argv[i+1]));
+					sprintf(str,"%s", argv[i+1]); // Done this way to avoid error with getting (i+1)th env variable
+					opt->hardsubx_conf_thresh = atof(str);
+					if(opt->hardsubx_conf_thresh <= 0.0 || opt->hardsubx_conf_thresh > 100.0)
+					{
+						fatal (EXIT_MALFORMED_PARAMETER, "-conf_thresh has either 0 or an invalid value supplied\nValid values are in (0.0,100.0)");
+					}
+				}
+				else
+				{
+					fatal (EXIT_MALFORMED_PARAMETER, "-conf_thresh has no argument.");
+				}
+				i++;
+				continue;
+			}
+			if (strcmp(argv[i], "-whiteness_thresh")==0 || strcmp(argv[i], "-lum_thresh")==0)
+			{
+				if(i < argc - 1)
+				{
+					char *str=(char*)malloc(sizeof(argv[i+1]));
+					sprintf(str,"%s", argv[i+1]); // Done this way to avoid error with getting (i+1)th env variable
+					opt->hardsubx_lum_thresh = atof(str);
+					if(opt->hardsubx_lum_thresh <= 0.0 || opt->hardsubx_conf_thresh > 100.0)
+					{
+						fatal (EXIT_MALFORMED_PARAMETER, "-whiteness_thresh has either 0 or an invalid value supplied\nValid values are in (0.0,100.0)");
+					}
+				}
+				else
+				{
+					fatal (EXIT_MALFORMED_PARAMETER, "-whiteness_thresh has no argument.");
+				}
+				i++;
+				continue;
+			}
+		}
+#endif
+
 		if (strcmp (argv[i],"-bi")==0 ||
 				strcmp (argv[i],"--bufferinput")==0)
 		{
@@ -866,6 +1146,21 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			opt->buffer_input = 0;
 			continue;
 		}
+		if (strcmp(argv[i], "-koc") == 0)
+		{
+			opt->keep_output_closed = 1;
+			continue;
+		}
+		if (strcmp(argv[i], "-ff") == 0 || strcmp(argv[i], "--forceflush") == 0)
+		{
+			opt->force_flush = 1;
+			continue;
+		}
+		if (strcmp(argv[i], "--append") == 0)
+		{
+			opt->append_mode = 1;
+			continue;
+		}
 		if ((strcmp (argv[i],"-bs")==0 || strcmp (argv[i],"--buffersize")==0) && i<argc-1)
 		{
 			FILEBUFFERSIZE = atol_size(argv[i+1]);
@@ -885,6 +1180,11 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			opt->nofontcolor=1;
 			continue;
 		}
+		if (strcmp (argv[i],"--nohtmlescape")==0)
+		{
+			opt->nohtmlescape=1;
+			continue;
+		}
 		if (strcmp(argv[i], "-bom") == 0){
 			opt->enc_cfg.no_bom = 0;
 			continue;
@@ -893,6 +1193,10 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			opt->enc_cfg.no_bom = 1;
 			continue;
 		}
+		if (strcmp(argv[i], "-sem") == 0){
+			opt->enc_cfg.with_semaphore = 1;
+			continue;		
+		}	
 		if (strcmp (argv[i],"-nots")==0 ||
 				strcmp (argv[i],"--notypesetting")==0)
 		{
@@ -921,7 +1225,7 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 
 		/*user specified subtitle to be selected */
 
-		if(!strcmp (argv[i],"-codec"))
+		if(!strcmp (argv[i],"-codec") && i < argc - 1)
 		{
 			i++;
 			if(!strcmp (argv[i],"teletext"))
@@ -940,7 +1244,7 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 		}
 		/*user specified subtitle to be selected */
 
-		if(!strcmp (argv[i],"-nocodec"))
+		if(!strcmp (argv[i],"-nocodec") && i < argc - 1)
 		{
 			i++;
 			if(!strcmp (argv[i],"teletext"))
@@ -957,6 +1261,33 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			}
 			continue;
 		}
+
+		if(strcmp(argv[i],"-dvbcolor")==0)
+		{
+			opt->dvbcolor = 1;
+			continue;
+		}
+
+		if(strcmp(argv[i],"-dvblang")==0 && i < argc-1)
+		{
+			i++;
+			opt->dvblang = (char *)malloc(sizeof(argv[i]));
+			sprintf(opt->dvblang,"%s",argv[i]);
+			for(int char_index=0; char_index < strlen(opt->dvblang);char_index++)
+				opt->dvblang[char_index] = cctolower(opt->dvblang[char_index]);
+			continue;
+		}
+
+		if(strcmp(argv[i],"-ocrlang")==0 && i < argc-1)
+		{
+			i++;
+			opt->ocrlang = (char *)malloc(sizeof(argv[i]));
+			sprintf(opt->ocrlang,"%s",argv[i]);
+			for(int char_index=0; char_index < strlen(opt->ocrlang);char_index++)
+				opt->ocrlang[char_index] = cctolower(opt->ocrlang[char_index]);
+			continue;
+		}
+
 		/* Output file formats */
 		if (strcmp (argv[i],"-srt")==0 ||
 				strcmp (argv[i],"-dvdraw")==0 ||
@@ -1086,6 +1417,11 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			MPEG_CLOCK_FREQ=90090;
 			continue;
 		}
+		if (strcmp (argv[i],"--noscte20")==0)
+		{
+			opt->noscte20 = 1;
+			continue;
+		}
 		if (strcmp (argv[i],"-noru")==0 ||
 				strcmp (argv[i],"--norollup")==0)
 		{
@@ -1114,7 +1450,7 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			opt->enc_cfg.trim_subs=1;
 			continue;
 		}
-		if (strcmp (argv[i],"-outinterval")==0)
+		if (strcmp (argv[i],"-outinterval")==0 && i<argc-1)
 		{
 			opt->out_interval = atoi(argv[i+1]);
 			i++;
@@ -1311,6 +1647,7 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 		}
 		if (strcmp (argv[i],"-xdsdebug")==0)
 		{
+			opt->transcript_settings.xds = 1;
 			opt->debug_mask |= CCX_DMT_DECODER_XDS;
 			continue;
 		}
@@ -1458,7 +1795,8 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 		}
 		if (strcmp (argv[i],"-UCLA")==0 || strcmp (argv[i],"-ucla")==0)
 		{
-			opt->millis_separator='.';
+			opt->ucla = 1;
+			opt->millis_separator = '.';
 			opt->enc_cfg.no_bom = 1;
 			if (!opt->transcript_settings.isFinal){
 				opt->transcript_settings.showStartTime = 1;
@@ -1485,6 +1823,11 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			opt->enc_cfg.autodash = 1;
 			continue;
 		}
+		if (strcmp(argv[i], "-sem") == 0)
+		{
+			opt->enc_cfg.autodash = 1;
+			continue;
+		}
 		if (strcmp (argv[i],"-xmltv")==0)
 		{
 			if (i==argc-1 // Means no following argument
@@ -1713,6 +2056,7 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 	tlt_config.noautotimeref = opt->noautotimeref;
 	tlt_config.send_to_srv = opt->send_to_srv;
 	tlt_config.nofontcolor = opt->nofontcolor;
+	tlt_config.nohtmlescape = opt->nohtmlescape;
 	tlt_config.millis_separator = opt->millis_separator;
 
 	// teletext page number out of range
@@ -1787,13 +2131,15 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 	opt->enc_cfg.transcript_settings = opt->transcript_settings;
 	opt->enc_cfg.millis_separator = opt->millis_separator;
 	opt->enc_cfg.no_font_color = opt->nofontcolor;
+	opt->enc_cfg.force_flush = opt->force_flush;
+	opt->enc_cfg.append_mode = opt->append_mode;
+	opt->enc_cfg.ucla = opt->ucla;
 	opt->enc_cfg.no_type_setting = opt->notypesetting;
 	opt->enc_cfg.subs_delay = opt->subs_delay;
+	opt->enc_cfg.gui_mode_reports = opt->gui_mode_reports;
 	if(opt->output_filename && opt->multiprogram == CCX_FALSE)
 		opt->enc_cfg.output_filename = strdup(opt->output_filename);
 	else
 		opt->enc_cfg.output_filename = NULL;
 	return EXIT_OK;
-
 }
-

src/lib_ccx/params_dump.c

@@ -225,8 +225,7 @@ void print_cc_report(struct lib_ccx_ctx *ctx, struct cap_info* info)
 }
 void print_file_report(struct lib_ccx_ctx *ctx)
 {
-	struct lib_cc_decode *dec_ctx = NULL;
-	enum ccx_stream_mode_enum stream_mode;
+	struct lib_cc_decode *dec_ctx = NULL;	
 	struct ccx_demuxer *demux_ctx = ctx->demux_ctx;
 
 	printf("File: ");

src/lib_ccx/sequencing.c

@@ -22,8 +22,6 @@ void init_hdcc (struct lib_cc_decode *ctx)
 // Buffer caption blocks for later sorting/flushing.
 void store_hdcc(struct lib_cc_decode *ctx, unsigned char *cc_data, int cc_count, int sequence_number, LLONG current_fts_now, struct cc_subtitle *sub)
 {
-	enum ccx_stream_mode_enum stream_mode;
-
 	//stream_mode = ctx->demux_ctx->get_stream_mode(ctx->demux_ctx);
 	// Uninitialized?
 	if (ctx->anchor_seq_number < 0)

src/lib_ccx/spupng_encoder.c

@@ -298,7 +298,7 @@ void draw_char_indexed(uint8_t * canvas, int rowstride,  uint8_t * pen,
 			underline * (3 << 24) /* cell row 24, 25 */);
 }
 
-void write_sputag(struct spupng_t *sp,LLONG ms_start,LLONG ms_end)
+void write_sputag_open (struct spupng_t *sp,LLONG ms_start,LLONG ms_end)
 {
         fprintf(sp->fpxml, "<spu start=\"%.3f\"", ((double)ms_start) / 1000);
         fprintf(sp->fpxml, " end=\"%.3f\"", ((double)ms_end) / 1000);
@@ -306,9 +306,13 @@ void write_sputag(struct spupng_t *sp,LLONG ms_start,LLONG ms_end)
         fprintf(sp->fpxml, " xoffset=\"%d\"", sp->xOffset);
         fprintf(sp->fpxml, " yoffset=\"%d\"", sp->yOffset);
         fprintf(sp->fpxml, ">\n");
-        fprintf(sp->fpxml, "</spu>\n");
-
 }
+
+void write_sputag_close(struct spupng_t *sp)
+{
+		fprintf(sp->fpxml, "</spu>\n");
+}
+
 void write_spucomment(struct spupng_t *sp,const char *str)
 {
         fprintf(sp->fpxml, "<!--\n");
@@ -467,20 +471,27 @@ int write_cc_bitmap_as_spupng(struct cc_subtitle *sub, struct encoder_ctx *conte
 	struct cc_bitmap* rect;
 	png_color *palette = NULL;
 	png_byte *alpha = NULL;
+	int wrote_opentag = 1;
 
         x_pos = -1;
         y_pos = -1;
         width = 0;
         height = 0;
 
+	
+
 	if (context->prev_start != -1 && (sub->flags & SUB_EOD_MARKER))
-		write_sputag(sp, context->prev_start, sub->start_time);
-	else if ( !(sub->flags & SUB_EOD_MARKER))
-		write_sputag(sp, sub->start_time, sub->end_time);
+		write_sputag_open(sp, context->prev_start, sub->start_time);
+	else if (!(sub->flags & SUB_EOD_MARKER))
+		write_sputag_open(sp, sub->start_time, sub->end_time);
+	else
+		wrote_opentag = 0;
 
 	if(sub->nb_data == 0 && (sub->flags & SUB_EOD_MARKER))
 	{
 		context->prev_start = -1;
+		if (wrote_opentag)
+			write_sputag_close(sp);
 		return 0;
 	}
 	rect = sub->data;
@@ -521,7 +532,7 @@ int write_cc_bitmap_as_spupng(struct cc_subtitle *sub, struct encoder_ctx *conte
 	}
 	inc_spupng_fileindex(sp);
 	filename = get_spupng_filename(sp);
-	set_spupng_offset(sp,y_pos,x_pos);
+	set_spupng_offset(sp, x_pos, y_pos);
 	if ( sub->flags & SUB_EOD_MARKER )
 		context->prev_start =  sub->start_time;
 	pbuf = (uint8_t*) malloc(width * height);
@@ -554,15 +565,29 @@ int write_cc_bitmap_as_spupng(struct cc_subtitle *sub, struct encoder_ctx *conte
 	/* TODO do rectangle wise, one color table should not be used for all rectangles */
         mapclut_paletee(palette, alpha, (uint32_t *)rect[0].data[1],rect[0].nb_colors);
 #ifdef ENABLE_OCR
-	if (rect[0].ocr_text && *(rect[0].ocr_text))
 	{
-		write_spucomment(sp, rect[0].ocr_text);
+		char *str;
+		str = paraof_ocrtext(sub, context->encoded_crlf, context->encoded_crlf_length);
+		if (str)
+		{
+			write_spucomment(sp, str);
+			freep(&str);
+		}
 	}
 #endif
 	save_spupng(filename,pbuf,width, height, palette, alpha,rect[0].nb_colors);
+	freep(&pbuf);
 
 
 end:
+	if (wrote_opentag)
+		write_sputag_close(sp);
+
+	for(i = 0, rect = sub->data; i < sub->nb_data; i++, rect++)
+	{
+		freep(rect->data);
+		freep(rect->data+1);
+	}
 	sub->nb_data = 0;
 	freep(&sub->data);
 	freep(&palette);

src/lib_ccx/spupng_encoder.h

@@ -28,7 +28,8 @@ void write_spumux_header(struct encoder_ctx *ctx, struct ccx_s_write *out);
 void write_spumux_footer(struct ccx_s_write *out);
 void draw_char_indexed(uint8_t * canvas, int rowstride,  uint8_t * pen,
 		     int unicode, int italic, int underline);
-void write_sputag(struct spupng_t *sp,LLONG ms_start,LLONG ms_end);
+void write_sputag_open(struct spupng_t *sp,LLONG ms_start,LLONG ms_end);
+void write_sputag_close(struct spupng_t *sp);
 void write_spucomment(struct spupng_t *sp,const char *str);
 char* get_spupng_filename(void *ctx);
 void inc_spupng_fileindex(void *ctx);

src/lib_ccx/stream_functions.c

@@ -9,6 +9,7 @@
 #include "utility.h"
 #include "ccx_common_timing.h"
 #include "file_buffer.h"
+#include "ccx_gxf.h"
 
 void detect_stream_type (struct ccx_demuxer *ctx)
 {
@@ -27,6 +28,14 @@ void detect_stream_type (struct ccx_demuxer *ctx)
 				ctx->startbytes[3]==0x75)
 			ctx->stream_mode=CCX_SM_ASF;
 	}
+	if (ctx->stream_mode == CCX_SM_ELEMENTARY_OR_NOT_FOUND)
+	{
+		if (ccx_gxf_probe(ctx->startbytes, ctx->startbytes_avail) == CCX_TRUE)
+		{
+			ctx->stream_mode = CCX_SM_GXF;
+			ctx->private_data = ccx_gxf_init(ctx);
+		}
+	}
 	if (ctx->stream_mode == CCX_SM_ELEMENTARY_OR_NOT_FOUND && ctx->startbytes_avail >= 4)
 	{
 		if(ctx->startbytes[0]==0xb7 &&
@@ -64,11 +73,10 @@ void detect_stream_type (struct ccx_demuxer *ctx)
 	if ((ctx->stream_mode == CCX_SM_ELEMENTARY_OR_NOT_FOUND || ccx_options.print_file_reports)
 			&& ctx->startbytes_avail >= 4) // Still not found
 	{
-		long idx = 0, nextBoxLocation = 0, lastBoxLocation = 0;
+		size_t idx = 0, nextBoxLocation = 0;
 		int boxScore = 0;
 		// Scan the buffer for valid succeeding MP4 boxes.
-		while (idx < ctx->startbytes_avail - 7){
-			lastBoxLocation = idx;
+		while (idx < ctx->startbytes_avail - 8){
 			// Check if we have a valid box
 			if (isValidMP4Box(ctx->startbytes, idx, &nextBoxLocation, &boxScore))
 			{
@@ -187,7 +195,7 @@ int detect_myth( struct ccx_demuxer *ctx )
 {
 	int vbi_blocks=0;
 	// VBI data? if yes, use myth loop
-	// STARTBTYTESLENGTH is 1MB, if the file is shorter we will never detect
+	// STARTBYTESLENGTH is 1MB, if the file is shorter we will never detect
 	// it as a mythTV file
 	if (ctx->startbytes_avail==STARTBYTESLENGTH)
 	{
@@ -410,18 +418,25 @@ typedef struct ccx_stream_mp4_box
 * An asterisk (*) marks a mandatory box for a regular file.
 * Box types that are on the second level or deeper are omitted.
 */
-ccx_stream_mp4_box ccx_stream_mp4_boxes[11] = {
+ccx_stream_mp4_box ccx_stream_mp4_boxes[16] = {
 		{ "ftyp", 6 }, // File type and compatibility*
 		{ "pdin", 1 }, // Progressive download information
 		{ "moov", 5 }, // Container for all metadata*
 		{ "moof", 4 }, // Movie fragment
-		{ "mfra", 3 }, // Movie fragment random access
+		{ "mfra", 1 }, // Movie fragment random access
 		{ "mdat", 2 }, // Media data container
 		{ "free", 1 }, // Free space
 		{ "skip", 1 }, // Free space
 		{ "meta", 1 }, // Metadata
 		{ "wide", 1 }, // For boxes that are > 2^32 bytes (https://developer.apple.com/library/mac/documentation/QuickTime/QTFF/QTFFChap1/qtff1.html)
-		{ "void", 1 }  // Unknown where this is from/for, assume free space.
+		{ "void", 1 },  // Unknown where this is from/for, assume free space.
+
+		// new ones in standard ISO/IEC 14496-12:2015
+		{ "meco", 1 }, // additional metadata container
+		{ "styp", 1 }, // segment type
+		{ "sidx", 1 }, // segment index
+		{ "ssix", 1 }, // subsegment index
+		{ "prft", 1 }  // producer reference time
 };
 
 /*
@@ -431,15 +446,15 @@ ccx_stream_mp4_box ccx_stream_mp4_boxes[11] = {
  *
  * Returns 1 when a box is found, 0 when none is found.
  */
-int isValidMP4Box(unsigned char *buffer, long position, long *nextBoxLocation, int *boxScore)
+int isValidMP4Box(unsigned char *buffer, size_t position, size_t *nextBoxLocation, int *boxScore)
 {
-	for (int idx = 0; idx < 11; idx++)
+	for (int idx = 0; idx < 16; idx++)
 	{
 		if (buffer[position + 4] == ccx_stream_mp4_boxes[idx].boxType[0] && buffer[position + 5] == ccx_stream_mp4_boxes[idx].boxType[1] &&
 				buffer[position + 6] == ccx_stream_mp4_boxes[idx].boxType[2] && buffer[position + 7] == ccx_stream_mp4_boxes[idx].boxType[3]){
 			mprint("Detected MP4 box with name: %s\n", ccx_stream_mp4_boxes[idx].boxType);
 			// Box name matches. Do crude validation of possible box size, and if valid, add points for "valid" box
-			long boxSize = buffer[position] << 24;
+			size_t boxSize = buffer[position] << 24;
 			boxSize |= buffer[position + 1] << 16;
 			boxSize |= buffer[position + 2] << 8;
 			boxSize |= buffer[position + 3];

src/lib_ccx/telxcc.c

@@ -34,6 +34,7 @@ Werner Brückner -- Teletext in digital television
 #include "teletext.h"
 #include <signal.h>
 #include "activity.h"
+#include "ccx_encoders_helpers.h"
 
 #ifdef __MINGW32__
 // switch stdin and all normal files into binary mode -- needed for Windows
@@ -52,6 +53,7 @@ typedef struct {
 	uint64_t show_timestamp; // show at timestamp (in ms)
 	uint64_t hide_timestamp; // hide at timestamp (in ms)
 	uint16_t text[25][40]; // 25 lines x 40 cols (1 screen/page) of wide chars
+	uint8_t g2_char_present[25][40]; // 0- Supplementary G2 character set NOT used at this position 1-Supplementary G2 character set used at this position
 	uint8_t tainted; // 1 = text variable contains any data
 } teletext_page_t;
 
@@ -86,7 +88,6 @@ struct TeletextCtx
 	enum ccx_output_date_format date_format;
 	int noautotimeref; // Do NOT set time automatically?
 	unsigned send_to_srv;
-	int nofontcolor;
 	char millis_separator;
 	uint32_t global_timestamp;
 
@@ -125,6 +126,9 @@ struct TeletextCtx
 	uint8_t using_pts;
 	int64_t delta;
 	uint32_t t0;
+
+	int sentence_cap;//Set to 1 if -sc is passed
+	int new_sentence;
 };
 typedef enum
 {
@@ -197,8 +201,9 @@ typedef enum
 	GREEK,
 	ARABIC,
 	HEBREW
-} g0_charsets_t;
+} g0_charsets_type;
 
+g0_charsets_type default_g0_charset; 
 
 // Note: All characters are encoded in UCS-2
 
@@ -345,7 +350,7 @@ const uint16_t G2[1][96] = {
 //	},
 //	{ // Greek G2 Supplementary Set
 //	},
-//	{ // Arabic G2 Supplementary Set
+//	{ // Arabic G2 Supplementary Set
 //	}
 };
 
@@ -504,6 +509,26 @@ uint32_t unham_24_18(uint32_t a)
 	return (a & 0x000004) >> 2 | (a & 0x000070) >> 3 | (a & 0x007f00) >> 4 | (a & 0x7f0000) >> 5;
 }
 
+//Default G0 Character Set 
+void set_g0_charset(uint32_t triplet)
+{
+	// ETS 300 706, Table 32
+	if((triplet & 0x3c00) == 0x1000)
+	{
+		if((triplet & 0x0380) == 0x0000)
+			default_g0_charset = CYRILLIC1;
+		else if((triplet & 0x0380) == 0x0200)
+			default_g0_charset = CYRILLIC2;
+		else if((triplet & 0x0380) == 0x0280)
+			default_g0_charset = CYRILLIC3;
+		else
+			default_g0_charset = LATIN;	
+	}
+	else
+		default_g0_charset = LATIN;
+}
+
+// Latin National Subset Selection
 void remap_g0_charset(uint8_t c)
 {
 	if (c != primary_charset.current)
@@ -524,7 +549,6 @@ void remap_g0_charset(uint8_t c)
 }
 
 
-
 // wide char (16 bits) to utf-8 conversion
 void ucs2_to_utf8(char *r, uint16_t ch)
 {
@@ -560,7 +584,7 @@ uint16_t telx_to_ucs2(uint8_t c)
 
 	uint16_t r = c & 0x7f;
 	if (r >= 0x20)
-		r = G0[LATIN][r - 0x20];
+		r = G0[default_g0_charset][r - 0x20];
 	return r;
 }
 
@@ -569,6 +593,36 @@ uint16_t bcd_page_to_int (uint16_t bcd)
 	return ((bcd&0xf00)>>8)*100 + ((bcd&0xf0)>>4)*10 + (bcd&0xf);
 }
 
+void telx_case_fix (struct TeletextCtx *context)
+{
+	//Capitalizing first letter of every sentence
+	int line_len = strlen(context->page_buffer_cur);
+	for(int i = 0; i < line_len; i++)
+	{
+		switch(context->page_buffer_cur[i])
+		{
+			case ' ':
+			//case 0x89: // This is a transparent space
+			case '-':
+				break;
+			case '.': // Fallthrough
+			case '?': // Fallthrough
+			case '!':
+			case ':':
+				context->new_sentence = 1;
+				break;
+			default:
+				if (context->new_sentence)
+					context->page_buffer_cur[i] = cctoupper(context->page_buffer_cur[i]);
+				else
+					context->page_buffer_cur[i] = cctolower(context->page_buffer_cur[i]);
+				context->new_sentence = 0;
+				break;
+		}
+	}
+	telx_correct_case(context->page_buffer_cur);
+}
+
 void telxcc_dump_prev_page (struct TeletextCtx *ctx, struct cc_subtitle *sub)
 {
 	char info[4]; 
@@ -756,7 +810,6 @@ void process_page(struct TeletextCtx *ctx, teletext_page_t *page, struct cc_subt
 				{
 					sprintf (c_tempb, "<font color=\"%s\">", TTXT_COLOURS[foreground_color]);
 					page_buffer_add_string (ctx, c_tempb);
-					// if (ctx->wbout1.fh!=-1) fdprintf(ctx->wbout1.fh, "<font color=\"%s\">", TTXT_COLOURS[foreground_color]);
 					font_tag_opened = YES;
 				}
 			}
@@ -772,7 +825,6 @@ void process_page(struct TeletextCtx *ctx, teletext_page_t *page, struct cc_subt
 						if (font_tag_opened == YES)
 						{
 							page_buffer_add_string (ctx, "</font>");
-							// if (ctx->wbout1.fh!=-1) fdprintf(ctx->wbout1.fh, "</font> ");
 							font_tag_opened = NO;
 						}
 
@@ -782,7 +834,6 @@ void process_page(struct TeletextCtx *ctx, teletext_page_t *page, struct cc_subt
 						{
 							sprintf (c_tempb, "<font color=\"%s\">", TTXT_COLOURS[v]);
 							page_buffer_add_string (ctx, c_tempb);
-							// if (ctx->wbout1.fh!=-1) fdprintf(ctx->wbout1.fh, "<font color=\"%s\">", TTXT_COLOURS[v]);
 							font_tag_opened = YES;
 						}
 					}
@@ -795,17 +846,13 @@ void process_page(struct TeletextCtx *ctx, teletext_page_t *page, struct cc_subt
 					ucs2_to_utf8(u, v);
 					uint64_t ucs2_char=(u[0]<<24) | (u[1]<<16) | (u[2]<<8) | u[3];
 					ucs2_buffer_add_char(ctx, ucs2_char);
-				}
 
-				if (v >= 0x20)
-				{
 					// translate some chars into entities, if in colour mode
-					if (!tlt_config.nofontcolor)
+					if (!tlt_config.nofontcolor && !tlt_config.nohtmlescape)
 					{
 						for (uint8_t i = 0; i < array_length(ENTITIES); i++)
 							if (v == ENTITIES[i].character)
 							{
-								//if (ctx->wbout1.fh!=-1) fdprintf(ctx->wbout1.fh, "%s", ENTITIES[i].entity);
 								page_buffer_add_string (ctx, ENTITIES[i].entity);
 								// v < 0x20 won't be printed in next block
 								v = 0;
@@ -813,11 +860,8 @@ void process_page(struct TeletextCtx *ctx, teletext_page_t *page, struct cc_subt
 							}
 					}
 				}
-
-
 				if (v >= 0x20)
 				{
-					//if (ctx->wbout1.fh!=-1) fdprintf(ctx->wbout1.fh, "%s", u);
 					page_buffer_add_string (ctx, u);
 					if (tlt_config.gui_mode_reports) // For now we just handle the easy stuff
 						fprintf (stderr,"%s",u);
@@ -828,7 +872,6 @@ void process_page(struct TeletextCtx *ctx, teletext_page_t *page, struct cc_subt
 		// no tag will left opened!
 		if ((!tlt_config.nofontcolor) && (font_tag_opened == YES))
 		{
-			//if (ctx->wbout1.fh!=-1) fdprintf(ctx->wbout1.fh, "</font>");
 			page_buffer_add_string (ctx, "</font>");
 			font_tag_opened = NO;
 		}
@@ -882,17 +925,9 @@ void process_page(struct TeletextCtx *ctx, teletext_page_t *page, struct cc_subt
 				ctx->prev_show_timestamp = page->show_timestamp;
 			}
 			break;
-		case CCX_OF_SMPTETT:
-#if 0
-			if (ctx->wbout1.fh!=-1)
-			{
-				timestamp_to_smptetttime(page->show_timestamp, timecode_show);
-				timestamp_to_smptetttime(page->hide_timestamp, timecode_hide);
-				fdprintf(ctx->wbout1.fh,"      <p region=\"speaker\" begin=\"%s\" end=\"%s\">%s</p>\n", timecode_show, timecode_hide, ctx->page_buffer_cur);
-			}
-#endif
-			break;
-		default: // Yes, this means everything else is .srt for now
+		default:
+			if (ctx->sentence_cap)
+				telx_case_fix(ctx);
 			add_cc_sub_text(sub, ctx->page_buffer_cur, page->show_timestamp,
 				page->hide_timestamp + 1, NULL, "TLT", CCX_ENC_UTF_8);
 	}
@@ -918,6 +953,7 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 
 	if (y == 0)
 	{
+
 		// CC map
 		uint8_t i = (unham_8_4(packet->data[1]) << 4) | unham_8_4(packet->data[0]);
 		uint8_t flag_subtitle = (unham_8_4(packet->data[5]) & 0x08) >> 3;
@@ -975,9 +1011,19 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 		if (page_number != tlt_config.page)
 			return;
 
+
 		// Now we have the begining of page transmission; if there is page_buffer pending, process it
 		if (ctx->page_buffer.tainted == YES)
 		{
+			// Convert telx to UCS-2 before processing
+			for(uint8_t yt = 1; yt <= 23; ++yt)
+			{
+				for(uint8_t it = 0; it < 40; it++)
+				{
+					if (ctx->page_buffer.text[yt][it] != 0x00 && ctx->page_buffer.g2_char_present[yt][it] == 0)
+						ctx->page_buffer.text[yt][it] = telx_to_ucs2(ctx->page_buffer.text[yt][it]);
+				}
+			}
 			// it would be nice, if subtitle hides on previous video frame, so we contract 40 ms (1 frame @25 fps)
 			ctx->page_buffer.hide_timestamp = timestamp - 40;
 			if (ctx->page_buffer.hide_timestamp > timestamp)
@@ -985,19 +1031,20 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 				ctx->page_buffer.hide_timestamp = 0;
 			}
 			process_page(ctx, &ctx->page_buffer, sub);
-
 		}
 
 		ctx->page_buffer.show_timestamp = timestamp;
 		ctx->page_buffer.hide_timestamp = 0;
 		memset(ctx->page_buffer.text, 0x00, sizeof(ctx->page_buffer.text));
+		memset(ctx->page_buffer.g2_char_present, 0x00, sizeof(ctx->page_buffer.g2_char_present));
 		ctx->page_buffer.tainted = NO;
 		ctx->receiving_data = YES;
-		primary_charset.g0_x28 = UNDEF;
-
-		c = (primary_charset.g0_m29 != UNDEF) ? primary_charset.g0_m29 : charset;
-		remap_g0_charset(c);
-
+		if(default_g0_charset == LATIN) // G0 Character National Option Sub-sets selection required only for Latin Character Sets
+		{
+			primary_charset.g0_x28 = UNDEF;
+			c = (primary_charset.g0_m29 != UNDEF) ? primary_charset.g0_m29 : charset;
+			remap_g0_charset(c);
+		}
 		/*
 		// I know -- not needed; in subtitles we will never need disturbing teletext page status bar
 		// displaying tv station name, current time etc.
@@ -1017,7 +1064,7 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 		for (uint8_t i = 0; i < 40; i++)
 		{
 			if (ctx->page_buffer.text[y][i] == 0x00)
-				ctx->page_buffer.text[y][i] = telx_to_ucs2(packet->data[i]);
+				ctx->page_buffer.text[y][i] = packet->data[i];
 		}
 		ctx->page_buffer.tainted = YES;
 	}
@@ -1063,7 +1110,23 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 			if ((mode == 0x0f) && (row_address_group == NO))
 			{
 				x26_col = address;
-				if (data > 31) ctx->page_buffer.text[x26_row][x26_col] = G2[0][data - 0x20];
+				if (data > 31) 
+				{
+					ctx->page_buffer.text[x26_row][x26_col] = G2[0][data - 0x20];
+					ctx->page_buffer.g2_char_present[x26_row][x26_col] = 1;
+				}
+			}
+
+			// ETS 300 706 v1.2.1, chapter 12.3.4, Table 29: G0 character without diacritical mark (display '@' instead of '*')
+			if ((mode == 0x10) && (row_address_group == NO))
+			{
+				x26_col = address;
+				if (data == 64) // check for @ symbol
+				{
+					remap_g0_charset(0);
+					ctx->page_buffer.text[x26_row][x26_col] = 0x40;
+				}
+
 			}
 
 			// ETS 300 706, chapter 12.3.1, table 27: G0 character with diacritical mark
@@ -1080,6 +1143,8 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 				// other
 				else
 					ctx->page_buffer.text[x26_row][x26_col] = telx_to_ucs2(data);
+				
+				ctx->page_buffer.g2_char_present[x26_row][x26_col] = 1;
 			}
 		}
 	}
@@ -1103,9 +1168,14 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 			{
 				// ETS 300 706, chapter 9.4.2: Packet X/28/0 Format 1 only
 				if ((triplet0 & 0x0f) == 0x00)
-				{
-					primary_charset.g0_x28 = (triplet0 & 0x3f80) >> 7;
-					remap_g0_charset(primary_charset.g0_x28);
+				{					
+					// ETS 300 706, Table 32
+					set_g0_charset(triplet0); // Deciding G0 Character Set
+					if(default_g0_charset == LATIN)
+					{
+						primary_charset.g0_x28 = (triplet0 & 0x3f80) >> 7;
+						remap_g0_charset(primary_charset.g0_x28);
+					}
 				}
 			}
 		}
@@ -1120,7 +1190,7 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 			// ETS 300 706, chapter 9.5.1: Packet M/29/0
 			// ETS 300 706, chapter 9.5.3: Packet M/29/4
 			uint32_t triplet0 = unham_24_18((packet->data[3] << 16) | (packet->data[2] << 8) | packet->data[1]);
-
+	
 			if (triplet0 == 0xffffffff)
 			{
 				// invalid data (HAM24/18 uncorrectable error detected), skip group
@@ -1132,11 +1202,15 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 				// ETS 300 706, table 13: Coding of Packet M/29/4
 				if ((triplet0 & 0xff) == 0x00)
 				{
-					primary_charset.g0_m29 = (triplet0 & 0x3f80) >> 7;
-					// X/28 takes precedence over M/29
-					if (primary_charset.g0_x28 == UNDEF)
+					set_g0_charset(triplet0);
+					if(default_g0_charset == LATIN)
 					{
-						remap_g0_charset(primary_charset.g0_m29);
+						primary_charset.g0_m29 = (triplet0 & 0x3f80) >> 7;
+						// X/28 takes precedence over M/29
+						if (primary_charset.g0_x28 == UNDEF)
+						{
+							remap_g0_charset(primary_charset.g0_m29);
+						}
 					}
 				}
 			}
@@ -1220,6 +1294,7 @@ void tlt_read_rcwt(void *codec, unsigned char *buf, struct cc_subtitle *sub)
 
 	ctx->last_timestamp = t;
 
+	ctx->tlt_packet_counter++;
 	process_telx_packet(ctx, id, pl, t, sub);
 }
 
@@ -1243,7 +1318,7 @@ int tlt_print_seen_pages(struct lib_cc_decode *dec_ctx)
 	}
 	return CCX_OK;
 }
-int tlt_process_pes_packet(struct lib_cc_decode *dec_ctx, uint8_t *buffer, uint16_t size, struct cc_subtitle *sub)
+int tlt_process_pes_packet(struct lib_cc_decode *dec_ctx, uint8_t *buffer, uint16_t size, struct cc_subtitle *sub, int sentence_cap)
 {
 	uint64_t pes_prefix;
 	uint8_t pes_stream_id;
@@ -1253,6 +1328,7 @@ int tlt_process_pes_packet(struct lib_cc_decode *dec_ctx, uint8_t *buffer, uint1
 	uint32_t t = 0;
 	uint16_t i;
 	struct TeletextCtx *ctx = dec_ctx->private_data;
+	ctx->sentence_cap = sentence_cap;
 
 	if(!ctx)
 	{
@@ -1427,33 +1503,12 @@ void* telxcc_init(void)
 	ctx->delta = 0;
 	ctx->t0 = 0;
 
+	ctx->sentence_cap = 0;
+	ctx->new_sentence = 0;
+
 	return ctx;
 }
 
-void telxcc_configure (void *codec, struct ccx_s_teletext_config *cfg)
-{
-	struct TeletextCtx *ctx = codec;
-	ctx->verbose = cfg->verbose;
-	ctx->page = cfg->page;
-	ctx->tid = cfg->tid;
-	ctx->offset = cfg->offset;
-	ctx->bom = cfg->bom;
-	ctx->nonempty = cfg->nonempty;
-	ctx->user_page = cfg->user_page;
-	ctx->levdistmincnt = cfg->levdistmincnt;
-	ctx->levdistmaxpct = cfg->levdistmaxpct;
-	ctx->extraction_start = cfg->extraction_start;
-	ctx->extraction_end = cfg->extraction_end;
-	ctx->write_format = cfg->write_format;
-	ctx->gui_mode_reports = cfg->gui_mode_reports;
-	ctx->date_format = cfg->date_format;
-	ctx->noautotimeref = cfg->noautotimeref;
-	ctx->send_to_srv = cfg->send_to_srv;
-	ctx->nofontcolor = cfg->nofontcolor;
-	ctx->millis_separator = cfg->millis_separator;
-
-}
-
 void telxcc_update_gt(void *codec, uint32_t global_timestamp)
 {
 	struct TeletextCtx *ctx = codec;
@@ -1468,11 +1523,21 @@ void telxcc_close(void **ctx, struct cc_subtitle *sub)
 	if(!ttext)
 		return;
 
+	mprint ( "\nTeletext decoder: %"PRIu32" packets processed \n", ttext->tlt_packet_counter);
 	if (tlt_config.write_format != CCX_OF_RCWT && sub)
 	{
 		// output any pending close caption
 		if (ttext->page_buffer.tainted == YES)
 		{
+			// Convert telx to UCS-2 before processing
+			for(uint8_t yt = 1; yt <= 23; ++yt)
+			{
+				for(uint8_t it = 0; it < 40; it++)
+				{
+					if (ttext->page_buffer.text[yt][it] != 0x00 && ttext->page_buffer.g2_char_present[yt][it] == 0)
+						ttext->page_buffer.text[yt][it] = telx_to_ucs2(ttext->page_buffer.text[yt][it]);
+				}
+			}
 			// this time we do not subtract any frames, there will be no more frames
 			ttext->page_buffer.hide_timestamp = ttext->last_timestamp;
 			process_page(ttext, &ttext->page_buffer, sub);
@@ -1484,4 +1549,4 @@ void telxcc_close(void **ctx, struct cc_subtitle *sub)
 	freep(&ttext->ucs2_buffer_cur);
 	freep(&ttext->page_buffer_cur);
 	freep(ctx);
-}
+}

src/lib_ccx/ts_functions.c

@@ -140,7 +140,7 @@ int ts_readpacket(struct ccx_demuxer* ctx, struct ts_payload *payload)
 		if (result != 4)
 		{
 			if (result>0)
-				mprint("Premature end of file!\n");
+				mprint("Premature end of file (incomplete TS packer header, expected 4 bytes, got %lld).\n",result);
 			return CCX_EOF;
 		}
 	}
@@ -150,7 +150,7 @@ int ts_readpacket(struct ccx_demuxer* ctx, struct ts_payload *payload)
 	if (result != 188)
 	{
 		if (result > 0)
-			mprint("Premature end of file!\n");
+			mprint("Premature end of file - Transport Stream packet is incomplete (expected 188 bytes, got %lld).\n", result);
 		return CCX_EOF;
 	}
 
@@ -221,8 +221,8 @@ int ts_readpacket(struct ccx_demuxer* ctx, struct ts_payload *payload)
 		// Take the PCR (Program Clock Reference) from here, in case PTS is not available (copied from telxcc).
 		adaptation_field_length = tspacket[4];
 
-		uint8_t af_pcr_exists = (tspacket[5] & 0x10) >> 4;
-		if (af_pcr_exists > 0 )
+		payload->have_pcr = (tspacket[5] & 0x10) >> 4;
+		if (payload->have_pcr)
 		{
 			payload->pcr = 0;
 			payload->pcr |= (tspacket[6] << 25);
@@ -518,8 +518,8 @@ int copy_payload_to_capbuf(struct cap_info *cinfo, struct ts_payload *payload)
 		if(payload->start[0] != 0x00 || payload->start[1] != 0x00 || 
 			payload->start[2] != 0x01)
 		{
-			mprint("Missing PES header!\n");
-			dump(CCX_DMT_GENERIC_NOTICES, payload->start, payload->length, 0, 0);
+			mprint("Notice: Missing PES header\n");
+			dump(CCX_DMT_DUMPDEF, payload->start, payload->length, 0, 0);
 			cinfo->saw_pesstart = 0;
 			errno = EINVAL;
 			return -1;
@@ -596,7 +596,9 @@ long ts_readstream(struct ccx_demuxer *ctx, struct demuxer_data **data)
 
 		for (j = 0; j < ctx->nb_program; j++)
 		{
-			if (ctx->pinfo[j].analysed_PMT_once == CCX_TRUE && ctx->pinfo[j].pcr_pid == payload.pid)
+			if (ctx->pinfo[j].analysed_PMT_once == CCX_TRUE &&
+				ctx->pinfo[j].pcr_pid == payload.pid &&
+				payload.have_pcr)
 			{
 				ctx->last_global_timestamp = ctx->global_timestamp;
 				ctx->global_timestamp = (uint32_t) payload.pcr / 90;
@@ -686,15 +688,15 @@ long ts_readstream(struct ccx_demuxer *ctx, struct demuxer_data **data)
 
 		}
 
-                // Skip packets with no payload.  This also fixes the problems
-                // with the continuity counter not being incremented in empty
-                // packets.
-                if ( !payload.length )
-                {   
-                        dbg_print(CCX_DMT_VERBOSE, "Packet (pid %u) skipped - no payload.\n",
-                                payload.pid);
-                        continue;
-                }   
+        // Skip packets with no payload.  This also fixes the problems
+        // with the continuity counter not being incremented in empty
+        // packets.
+        if ( !payload.length )
+        {   
+                dbg_print(CCX_DMT_VERBOSE, "Packet (pid %u) skipped - no payload.\n",
+                        payload.pid);
+                continue;
+        }   
 
 
 		cinfo = get_cinfo(ctx, payload.pid);

src/lib_ccx/ts_functions.h

@@ -9,6 +9,7 @@ struct ts_payload
 	unsigned pid;         // Stream PID
 	int counter;          // continuity counter
 	int transport_error;  // 0 = packet OK, non-zero damaged
+	int have_pcr;
 	int64_t pcr;
 	unsigned char section_buf[4098];
 	int section_index;

src/lib_ccx/ts_tables.c

@@ -108,6 +108,12 @@ int parse_PMT (struct ccx_demuxer *ctx, unsigned char *buf, int len,  struct pro
 	crc = (*(int32_t*)(sbuf+olen-4));
 	table_id = buf[0];
 
+	/* TO-DO: We're currently parsing the PMT making assumptions that there's only one section with table_id=2,
+	but that doesn't have to be the case. There's a sample (friends_tbs.ts) that shows a previous section with
+	table_id = 0xc0. I can't find any place that says that 0xc0 (Program Information Table) must come before
+	table_id = 2, so we should process sections in any order.
+	Check https://github.com/CCExtractor/ccextractor/issues/385 for more info
+	*/
 	if (table_id == 0xC0)
 	{
 		/*
@@ -116,13 +122,22 @@ int parse_PMT (struct ccx_demuxer *ctx, unsigned char *buf, int len,  struct pro
 		 * PROGRAM INFORMATION Table found in PMT
 		 */
 		dbg_print(CCX_DMT_PARSE, "PMT: PROGRAM INFORMATION Table need implementation");
-		return 0;
+		// For now, just parse its length and remove it from the buffer
+		unsigned c0length = (buf[1] << 8 | buf[2]) & 0xFFF; // 12 bytes
+		dbg_print(CCX_DMT_PARSE, "Program information table length: %u", c0length);
+		memmove(buf, buf + c0length + 3, len - c0length -3); // First 3 bytes are for the table_id and the length, don't count
+		table_id = buf[0];
+		// return 0;
 	}
 	else if (table_id == 0xC1)
 	{
                 //SCTE 57 2003
-		dbg_print(CCX_DMT_PARSE, "PMT: PROGRAM Name Table need implementation");
-		return 0;
+		dbg_print(CCX_DMT_PARSE, "PMT: PROGRAM NAME Table need implementation");
+		unsigned c0length = (buf[1] << 8 | buf[2]) & 0xFFF; // 12 bytes
+		dbg_print(CCX_DMT_PARSE, "Program name message length: %u", c0length);
+		memmove(buf, buf + c0length + 3, len - c0length - 3); // First 3 bytes are for the table_id and the length, don't count
+		table_id = buf[0];
+		//return 0;
 	}
 	else if(table_id != 0x2)
 	{

src/lib_ccx/ts_tables_epg.c

@@ -81,7 +81,7 @@ void EPG_DVB_calc_start_time(struct EPG_event *event, uint64_t time)
 		y = y + k + 1900;
 		m = m - 1 - k*12;
 
-		sprintf(event->start_time_string, "%02ld%02ld%02ld%06llx +0000",y,m,d,time&0xffffff);
+		sprintf(event->start_time_string, "%02ld%02ld%02ld%06"PRIu64 "+0000",y,m,d,time&0xffffff);
 	}
 }
 
@@ -300,6 +300,11 @@ void EPG_output(struct lib_ccx_ctx *ctx)
 	memcpy(filename, ctx->basefilename, strlen(ctx->basefilename)+1);
 	strcat(filename, "_epg.xml");
 	f = fopen(filename, "w");
+	if(!f)
+	{
+		dbg_print (CCX_DMT_GENERIC_NOTICES, "\rUnable to open %s\n", filename);
+		return;
+	}
 	freep(&filename);
 
 	fprintf(f, "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE tv SYSTEM \"xmltv.dtd\">\n\n<tv>\n");
@@ -1091,37 +1096,19 @@ void parse_EPG_packet(struct lib_ccx_ctx *ctx)
 
 // Free all memory used for EPG parsing
 void EPG_free(struct lib_ccx_ctx *ctx)
-{
-	int i = 0, j;
-	if(ccx_options.xmltv==2 || ccx_options.xmltv==3 || ccx_options.send_to_srv)
+{	
+	if(ctx->epg_inited)
 	{
-		if (ccx_options.send_to_srv)
-			EPG_output_net(ctx);
-		else
-			EPG_output_live(ctx);
-	}
-	for (i = 0; i < TS_PMT_MAP_SIZE; i++)
-	{
-		for(j = 0; j < ctx->eit_programs[i].array_len; j++)
+		if(ccx_options.xmltv==2 || ccx_options.xmltv==3 || ccx_options.send_to_srv)
 		{
-			if(ctx->eit_programs[i].epg_events[j].has_simple)
-			{
-				free(ctx->eit_programs[i].epg_events[j].event_name);
-				free(ctx->eit_programs[i].epg_events[j].text);
-			}
-			if(ctx->eit_programs[i].epg_events[j].extended_text!=NULL)
-				free(ctx->eit_programs[i].epg_events[j].extended_text);
-			if(ctx->eit_programs[i].epg_events[j].num_ratings>0)
-				free(ctx->eit_programs[i].epg_events[j].ratings);
-			if(ctx->eit_programs[i].epg_events[j].num_categories>0)
-				free(ctx->eit_programs[i].epg_events[j].categories);
+			if (ccx_options.send_to_srv)
+				EPG_output_net(ctx);
+			else
+				EPG_output_live(ctx);
 		}
-		ctx->eit_programs[i].array_len=0;
-	}
-	
-	for (i = 0; i < 0xfff; i++)
-	{
-		if(ctx->epg_buffers[i].buffer!=NULL)
-			free(ctx->epg_buffers[i].buffer);
 	}
+	free(ctx->epg_buffers);
+	free(ctx->eit_programs);
+	free(ctx->eit_current_events);
+	free(ctx->ATSC_source_pg_map);
 }

src/lib_ccx/utility.c

@@ -1,10 +1,13 @@
+#include <stdio.h>
 #include <signal.h>
+#include <string.h>
 #include "lib_ccx.h"
 #include "ccx_common_option.h"
 #include "activity.h"
+#include "utility.h"
 
 int temp_debug = 0; // This is a convenience variable used to enable/disable debug on variable conditions. Find references to understand.
-
+volatile sig_atomic_t change_filename_requested = 0;
 
 
 static uint32_t crc32_table [] = {
@@ -371,7 +374,91 @@ void m_signal(int sig, void (*func)(int))
 
 	return;
 }
+
+void create_signal(void)
+{
+        if (signal(SIGUSR1, signal_handler) == SIG_ERR)
+		mprint("Can't catch SIGUSR1.\n");
+}
+
+void signal_handler(int sig_type)
+{
+	int ret;
+
+        if (sig_type == SIGUSR1)
+        {
+        	mprint("Caught SIGUSR1. Filename Change Requested\n");
+        	change_filename_requested = 1;
+        }
+}
 #endif
+
+struct encoder_ctx *change_filename(struct encoder_ctx *enc_ctx)
+{
+	if(change_filename_requested == 0)
+	{
+		return enc_ctx;
+	}
+	struct encoder_ctx *temp_encoder = malloc(sizeof(struct encoder_ctx));
+	*temp_encoder = *enc_ctx;
+	if (enc_ctx->out->fh != -1)
+	{
+		if (enc_ctx->out->fh > 0)
+			close(enc_ctx->out->fh);
+		enc_ctx->out->fh=-1;
+		int iter;
+		char str_number[15];
+		char *current_name = malloc(sizeof(char)*(strlen(enc_ctx->out->filename)+10));
+		strcpy(current_name,enc_ctx->out->filename);
+		mprint ("Creating %s\n", enc_ctx->out->filename);
+		if(enc_ctx->out->renaming_extension)
+		{
+			strcat(current_name,".");
+			sprintf(str_number, "%d", enc_ctx->out->renaming_extension);
+			strcat(current_name,str_number);
+		}
+		enc_ctx->out->renaming_extension++;
+		for (iter = enc_ctx->out->renaming_extension; iter >= 1; iter--)
+		{
+			int ret;
+			char new_extension[6];
+			sprintf(new_extension, ".%d", iter); 
+			char *newname = malloc(sizeof(char)*(strlen(enc_ctx->out->filename)+10));
+			strcpy(newname,enc_ctx->out->filename);
+			strcat(newname,new_extension);
+			ret = rename(current_name, newname);
+			if(ret)
+			{
+				mprint("Failed to rename the file\n");
+
+			}
+			mprint ("Creating %s\n", newname);
+			strcpy(current_name,enc_ctx->out->filename);
+			
+			if(iter-2>0)
+			{
+				strcat(current_name,".");
+				sprintf(str_number, "%d", iter-2);
+				strcat(current_name,str_number);
+			}
+			free(newname);
+		
+		}
+
+		enc_ctx->out->fh = open(enc_ctx->out->filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, S_IREAD | S_IWRITE);
+		free(current_name);
+		if (enc_ctx->out->fh == -1)
+		{
+			mprint("Failed to create a new rotation file\n");
+			return temp_encoder;
+		}
+		free(temp_encoder);		
+		change_filename_requested = 0;
+		return enc_ctx;
+		
+	}
+	return temp_encoder;
+}
 char *get_basename(char *filename)
 {
 	char *c;
@@ -422,6 +509,8 @@ char *get_file_extension(enum ccx_output_format write_format)
 			return strdup(".dvdraw");
 		case CCX_OF_SIMPLE_XML:
 			return strdup(".xml");
+		case CCX_OF_G608:
+			return strdup(".g608");
 		case CCX_OF_NULL:
 			return NULL;
 		default:
@@ -494,6 +583,14 @@ size_t utf16_to_utf8(unsigned short utf16_char, unsigned char *out)
 	return 0;
 }
 
+LLONG change_timebase(LLONG val, struct ccx_rational cur_tb, struct ccx_rational dest_tb)
+{
+	/* val = (value * current timebase) / destination timebase */
+	val = val * cur_tb.num * dest_tb.den;
+	val = val / ( cur_tb.den * dest_tb.num);
+	return val;
+}
+
 #ifdef _WIN32
 char *strndup(const char *s, size_t n)
 {
@@ -508,6 +605,6 @@ char *strndup(const char *s, size_t n)
 }
 char *strtok_r(char *str, const char *delim, char **saveptr)
 {
-	strtok_s(str, delim, saveptr);
+	return strtok_s(str, delim, saveptr);
 }
 #endif //_WIN32

src/lib_ccx/utility.h

@@ -1,6 +1,6 @@
 #ifndef CC_UTILITY_H
 #define CC_UTILITY_H
-
+#include <signal.h>
 #ifndef _WIN32
 	#include <arpa/inet.h>
 #endif
@@ -14,7 +14,13 @@
 
 #define CCX_NOPTS	((int64_t)UINT64_C(0x8000000000000000))
 
+struct ccx_rational
+{
+	int num;
+	int den;
+};
 extern int temp_debug;
+volatile extern sig_atomic_t change_filename_requested;
 void init_boundary_time (struct ccx_boundary_time *bt);
 void print_error (int mode, const char *fmt, ...);
 int stringztoms (const char *s, struct ccx_boundary_time *bt);
@@ -23,7 +29,9 @@ char *get_file_extension(enum ccx_output_format write_format);
 char *create_outfilename(const char *basename, const char *suffix, const char *extension);
 int verify_crc32(uint8_t *buf, int len);
 size_t utf16_to_utf8(unsigned short utf16_char, unsigned char *out);
+LLONG change_timebase(LLONG val, struct ccx_rational cur_tb, struct ccx_rational dest_tb);
 
+void dump (LLONG mask, unsigned char *start, int l, unsigned long abs_start, unsigned clear_high_bit);
 #ifdef _WIN32
 char *strndup(const char *s, size_t n);
 char *strtok_r(char *str, const char *delim, char **saveptr);

src/lib_hash/README

@@ -0,0 +1,277 @@
+VERSION:
+
+This is version 1.0.1 RELEASE
+
+While this is my "release" version, due to lack of additional
+official test vectors against which to verify this implementation's
+correctness, beware that there may be implementation bugs.  Also,
+it has not yet been tested on very many other architectures,
+big-endian machines in particular.
+
+
+LICENSE:
+
+This implementation is released freely under an open-source BSD
+license which appears at the top of each source code file.
+
+
+WHAT IT IS:
+
+The files sha2.h and sha2.c implement the SHA-256, SHA-384, and SHA-512
+hash algorithms as described in the PDF document found at the following
+web address:
+
+  http://csrc.nist.gov/cryptval/shs/sha256-384-512.pdf
+
+The interface is similar to the interface to SHA-1 found in the OpenSSL
+library.
+
+The file sha2prog.c is a simple program that accepts input from either
+STDIN or reads one or more files specified on the command line, and then
+generates the specified hash (either SHA-256, SHA-384, SHA-512, or any
+combination thereof, including all three at once).
+
+
+LIMITATIONS:
+
+This implementation has several limitations:
+
+ * Input data is only accepted in octet-length increments.  No sub-byte
+   data is handled.  The NIST document describes how to handle sub-byte
+   input data, but for ease of implementation this version will only
+   accept message data in multiples of bytes.
+ * This implementation utilizes 64-bit integer data types.  If your
+   system and compiler does not have a 64-bit integer data type, this
+   implementation will not work.
+ * Because of the use of 64-bit operations, many 32-bit architectures
+   that do have 64-bit data types but do operations most efficiently
+   on 32-bit words, this implementation may be slower than an
+   implementation designed to use only 32-bit words (emulating the
+   64-bit operations).
+ * On platforms with 128-bit integer data types, the SHA-384 and SHA-512
+   bit counters used by this implementation might be better off using
+   the 128-bit type instead of simulating it with two 64-bit integers.
+ * This implementation was written in C in hopes of portability and for
+   the fun of it during my spare time.  It is probably not the most
+   efficient or fastest C implementation.  I welcome suggestions,
+   however, that suggest ways to speed things up without breaking
+   portability.  I also welcome suggestions to improve portability.
+ * As mentioned above, this code has NOT been thoroughly tested.
+   This is perhaps the most severe limitation.
+
+
+BEFORE YOU COMPILE (OPTIONS):
+
+Each of the options described below may either be defined in the sha2.h
+header file (or in the sha2.c file in some cases), or on the command
+line at compile time if your compiler supports such things.  For
+example:
+
+  #define SHA2_USE_INTTYPES_H
+  #define SHA2_UNROLL_TRANSFORM
+
+Or:
+
+  cc -c -DSHA2_UNROLL_TRANSFORM sha2.c
+  cc -c -DBYTE_ORDER=4321 -DBIG_ENDIAN=4321 sha2.c
+
+Here are the available options.  Read on below for a description of
+each one:
+
+  SHA2_USE_INTTYPES_H
+  SHA2_USE_MEMSET_MEMCPY/SHA2_USE_BZERO_BCOPY
+  SHA2_UNROLL_TRANSFORM
+  BYTE_ORDER (LITTLE_ENDIAN/BIG_ENDIAN)
+
+* SHA2_USE_INTTYPES_H option:
+By default, this code uses u_intXX_t data types for 8 bit, 32 bit, and
+64 bit unsigned integer type definitions.  Most BSD systems define these,
+as does Linux.  However, some (like Compaq's Tru64 Unix) may instead
+use uintXX_t data types as defined by recent ANSI C standards and as
+included in the inttypes.h header file.  Those wanting to use inttypes.h
+need to define this either in sha.h or at compile time.
+
+On those systems where NEITHER definitions are available, you will need
+to edit both sha2.h and sha2.c and define things by hand in the appropriate
+sections.
+
+* BYTE_ORDER definitions:
+This code assumes that BYTE_ORDER will be defined by the system during
+compile to either equal LITTLE_ENDIAN or BIG_ENDIAN.  If your system
+does not define these, you may need to define them by hand in the sha.c
+file according to the byte ordering conventions of your system.
+
+* SHA2_USE_MEMSET_MEMCPY or SHA2_USE_BZERO_BCOPY
+The code in sha2.c can use either memset()/memcpy() for memory block
+operations, or bzero()/mcopy().  If you define neither of these, the
+code will default to memset()/memcpy().  You can define either at the
+command line or in sha2.h or in sha2.c.
+
+* SHA2_UNROLL_TRANSFORM
+By defining this either on the command line or in sha2.h or sha2.c,
+the code will use macros to partially "unroll" the SHA transform
+function.  This usually generates bigger executables.  It CAN (but
+not necessarily WILL) generate faster code when you tell your compiler
+to optimize things.  For example, on the FreeBSD and Linux x86 systems
+I tested things on (using gcc), when I optimized with just -O2 and
+unrolled the transform, the hash transform was faster by 15-30%.  On
+these same systems, if I did NO optimization, the unrolled transform
+was SLOWER, much slower (I'm guessing because the code was breaking
+the cache, but I'm not sure).  Your mileage may vary.
+
+
+PORTABILITY:
+
+The code in sha2.c and sha2.h is intended to be portable.  It may
+require that you do a few #definitions in the .h file.  I've successfully
+compiled and tested the sha2.c and sha2.h code on Apple's OS X (on
+a PPC), FreeBSD 4.1.1 on Intel, Linux on Intel, FreeBSD on the Alpha,
+and even under Windows98SE using Metrowerks C.  The utility/example
+programs (sha2prog.c, sha2test.c, and sha2speed.c) will very likely
+have more trouble in portability since they do I/O.
+
+To get sha2.c/sha2.h working under Windows, I had to define
+SHA2_USE_INTTYPES_H, BYTE_ORDER, LITTLE_ENDIAN, and had to comment
+out the include of <sys/types.h> in sha2.h.  With a bit more work
+I got the test program to run and verified that all the test
+cases passed.
+
+
+SUGGESTIONS/BUG FIXES:
+
+If you make changes to get it working on other architectures, if you fix
+any bugs, or if you make changes that improve this implementation's
+efficiency that would be relatively portable and you're willing to release
+your changes under the same license, please send them to me for possible
+inclusion in future versions.
+
+If you know where I can find some additional test vectors, please let me
+know.
+
+
+CHANGE HISTORY:
+
+0.8 to 0.9 	- Fixed spelling errors, changed to u_intXX_t type usage,
+		  removed names from prototypes, added prototypes to sha2.c,
+		  and a few things I can't recall.
+
+0.9 to 0.9.5	- Add a new define in sha2.c that permits one to compile
+		  it to either use memcpy()/memset() or bcopy()/bzero()
+		  for memory block copying and zeroing.  Added support
+		  for unrolled SHA-256/384/512 transform loops.  Just
+		  compile with SHA2_UNROLL_TRANSFORM to enable.  It takes
+		  longer to compile, but I hope it is a bit faster.  I
+		  need to do some test to see whether or not it is. Oh,
+		  in sha2.c, you either need to define SHA2_USE_BZERO_BCOPY
+		  or SHA2_USE_MEMSET_MEMCPY to choose which way you want
+		  to compile.  *Whew*  It's amazing how quickly something
+		  simple starts to grow more complex even in the span of
+		  just a few hours.  I didn't really intend to do this much.
+0.9.5 to 0.9.6  - Added a test program (sha2test) which tests against several
+                  known test vectors.  WARNING: Some of the test output
+                  hashes are NOT from NIST's documentation and are the
+                  output of this implementation and so may be incorrect.
+0.9.6 to 0.9.7  - Fixed a bug that could cause invalid output in certain
+		  cases and added an assumed scenario where zero-length
+		  data is hashed.  Also changed the rotation macros to use
+		  a temporary variable as this reduces the number of operations.
+		  When data is fed in blocks of the right length, copying of
+		  data is reduced in this version.  Added SHAYXZ_Data()
+		  functions for ease of hashing a set of data.  Added another
+		  file sha2speed.c for doing speed testing.  Added another test
+		  vector with a larger data size (16KB).  Fixed u_intXX_t and
+		  uintXX_t handling by adding a define for SHA2_USE_INTTYPES_H
+		  as well as made a few other minor changes to get rid of
+		  warnings when compiling on Compaq's Tru64 Unix.
+0.9.7 to 0.9.8  - The bug fix in 0.9.7 was incomplete and in some cases made
+                  things worse.  I believe that 0.9.8 fixes the bug completely
+                  so that output is correct.  I cannot verify this, however,
+                  because of the lack of test vectors against which to do such
+                  verification.  All versions correctly matched the very few
+                  NIST-provided vectors, but unfortunately the bug only
+                  appeared in longer message data sets.
+0.9.8 to 0.9.9  - Fixed some really bad typos and mistakes on my part that
+                  only affected big-endian systems.  I didn't have direct
+                  access for testing before this version.  Thanks to
+                  Lucas Marshall for giving me access to his OS X system.
+0.9.9 to 1.0.0b1  Added a few more test samples and made a few changes to
+                  make things easier compiling on several other platforms.
+                  Also I experimented with alternate macro definitions
+                  in the SHA2_UNROLL_TRANSFORM version (see sha2.slower.c)
+                  and eliminated the T1 temporary variable (the compiler
+                  would of course still use internal temporary storage
+                  during expression evaluation, but I'd hoped the compiler
+                  would be more efficient), but unfortunately under FreeBSD
+                  4.1.1-STABLE on an x86 platform, the change slowed things
+                  down.
+1.0.0b1 to 1.0 RELEASE  Fixed an off-by-one implementation bug that affected
+                  SHA-256 when hashed data length L = 55 + 64 * X where X is
+                  either zero or a positive integer, and another (basically
+                  the same bug) bug in SHA-384 and SHA-512 that showed up when
+                  hashed data lengths L = 111 + 128 * X.  Thanks to Rogier
+		  van de Pol for sending me test data that revealed the bug.
+                  The fix was very simple (just two tiny changes).  Also,
+                  I finally put the files into RCS so future changes will be
+                  easier to manage.  The sha2prog.c file was rewritten to
+                  be more useful to me, and I got rid of the old C testing
+                  program and now use a perl script with a subdirectory full
+                  of test data.  It's a more flexible test system.
+
+1.0 to 1.0.1    - Specified the specific *_CTX structure in the MEMSET_BZERO
+                  macro doing clean-up after hashing.  This should eliminate
+                  some warnings using Clang in version 3.0 (trunk 135348).
+                  Thanks, Stephane Leon for reporting this.
+
+
+LATEST VERSION:
+
+The latest version and documentation (if any ;) should always be available
+on the web at:
+
+  http://www.aarongifford.com/computers/sha.html
+
+
+CONTACT ME:
+
+I can be reached via email at:
+
+  Aaron Gifford   <m e @ a a r o n g i f f o r d . c o m>
+
+Please don't send support questions.  I don't have the time to answer and
+they'll probably be ignored.  Bug fixes, or patches that add something useful
+will be gratefully accepted, however.
+
+If you use this implementation, I would enjoy getting a brief email message
+letting me know who you are and what use to which it is being put.  There
+is no requirement to do so.  I just think it would be fun.
+
+
+EXAMPLES:
+
+Here's an example of compiling and using the sha2 program (in this example
+I build it using the unrolled transform version with -O2 optimizations),
+and then running the perl testing script:
+
+  cc -O2 -DSHA2_UNROLL_TRANSFORM -Wall -o sha2 sha2prog.c sha2.c
+  % ./sha2test.pl
+
+  [most of the perl script output deleted for brevity]
+
+  ===== RESULTS (18 VECTOR DATA FILES HASHED) =====
+
+  HASH TYPE       NO. OF TESTS    PASSED  FAILED
+  ---------       ------------    ------  ------
+  SHA-256                   18        18       0
+  SHA-384                   18        18       0
+  SHA-512                   18        18       0
+  ----------------------------------------------
+  TOTAL:                    54        54       0
+
+  NO ERRORS!  ALL TESTS WERE SUCCESSFUL!
+
+  ALL TEST VECTORS PASSED!
+
+That's all folks!  Have fun!
+
+Aaron out.
+

src/lib_hash/sha2.h

@@ -0,0 +1,193 @@
+/*
+ * FILE:	sha2.h
+ * AUTHOR:	Aaron D. Gifford - http://www.aarongifford.com/
+ * 
+ * Copyright (c) 2000-2001, Aaron D. Gifford
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the copyright holder nor the names of contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ * 
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTOR(S) ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTOR(S) BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $Id: sha2.h,v 1.1 2001/11/08 00:02:01 adg Exp adg $
+ */
+
+#ifndef __SHA2_H__
+#define __SHA2_H__
+
+/*
+ * Import u_intXX_t size_t type definitions from system headers.  You
+ * may need to change this, or define these things yourself in this
+ * file.
+ */
+#include <sys/types.h>
+
+#ifdef _WIN32
+
+#include <inttypes.h>
+
+#define u_int8_t  uint8_t
+#define u_int32_t uint32_t
+#define u_int64_t uint64_t
+
+#define BYTE_ORDER LITTLE_ENDIAN
+
+#endif
+
+
+/*** SHA-256/384/512 Various Length Definitions ***********************/
+#define SHA256_BLOCK_LENGTH		64
+#define SHA256_DIGEST_LENGTH		32
+#define SHA256_DIGEST_STRING_LENGTH	(SHA256_DIGEST_LENGTH * 2 + 1)
+#define SHA384_BLOCK_LENGTH		128
+#define SHA384_DIGEST_LENGTH		48
+#define SHA384_DIGEST_STRING_LENGTH	(SHA384_DIGEST_LENGTH * 2 + 1)
+#define SHA512_BLOCK_LENGTH		128
+#define SHA512_DIGEST_LENGTH		64
+#define SHA512_DIGEST_STRING_LENGTH	(SHA512_DIGEST_LENGTH * 2 + 1)
+
+
+/*** SHA-256/384/512 Context Structures *******************************/
+/* NOTE: If your architecture does not define either u_intXX_t types or
+ * uintXX_t (from inttypes.h), you may need to define things by hand
+ * for your system:
+ */
+#if 0
+typedef unsigned char u_int8_t;		/* 1-byte  (8-bits)  */
+typedef unsigned int u_int32_t;		/* 4-bytes (32-bits) */
+typedef unsigned long long u_int64_t;	/* 8-bytes (64-bits) */
+#endif
+/*
+ * Most BSD systems already define u_intXX_t types, as does Linux.
+ * Some systems, however, like Compaq's Tru64 Unix instead can use
+ * uintXX_t types defined by very recent ANSI C standards and included
+ * in the file:
+ *
+ *   #include <inttypes.h>
+ *
+ * If you choose to use <inttypes.h> then please define: 
+ *
+ *   #define SHA2_USE_INTTYPES_H
+ *
+ * Or on the command line during compile:
+ *
+ *   cc -DSHA2_USE_INTTYPES_H ...
+ */
+#ifdef SHA2_USE_INTTYPES_H
+
+typedef struct _SHA256_CTX {
+	uint32_t	state[8];
+	uint64_t	bitcount;
+	uint8_t	buffer[SHA256_BLOCK_LENGTH];
+} SHA256_CTX;
+typedef struct _SHA512_CTX {
+	uint64_t	state[8];
+	uint64_t	bitcount[2];
+	uint8_t	buffer[SHA512_BLOCK_LENGTH];
+} SHA512_CTX;
+
+#else /* SHA2_USE_INTTYPES_H */
+
+typedef struct _SHA256_CTX {
+	u_int32_t	state[8];
+	u_int64_t	bitcount;
+	u_int8_t	buffer[SHA256_BLOCK_LENGTH];
+} SHA256_CTX;
+typedef struct _SHA512_CTX {
+	u_int64_t	state[8];
+	u_int64_t	bitcount[2];
+	u_int8_t	buffer[SHA512_BLOCK_LENGTH];
+} SHA512_CTX;
+
+#endif /* SHA2_USE_INTTYPES_H */
+
+typedef SHA512_CTX SHA384_CTX;
+
+
+/*** SHA-256/384/512 Function Prototypes ******************************/
+#ifndef NOPROTO
+#ifdef SHA2_USE_INTTYPES_H
+
+void SHA256_Init(SHA256_CTX *);
+void SHA256_Update(SHA256_CTX*, const uint8_t*, size_t);
+void SHA256_Final(uint8_t[SHA256_DIGEST_LENGTH], SHA256_CTX*);
+char* SHA256_End(SHA256_CTX*, char[SHA256_DIGEST_STRING_LENGTH]);
+char* SHA256_Data(const uint8_t*, size_t, char[SHA256_DIGEST_STRING_LENGTH]);
+
+void SHA384_Init(SHA384_CTX*);
+void SHA384_Update(SHA384_CTX*, const uint8_t*, size_t);
+void SHA384_Final(uint8_t[SHA384_DIGEST_LENGTH], SHA384_CTX*);
+char* SHA384_End(SHA384_CTX*, char[SHA384_DIGEST_STRING_LENGTH]);
+char* SHA384_Data(const uint8_t*, size_t, char[SHA384_DIGEST_STRING_LENGTH]);
+
+void SHA512_Init(SHA512_CTX*);
+void SHA512_Update(SHA512_CTX*, const uint8_t*, size_t);
+void SHA512_Final(uint8_t[SHA512_DIGEST_LENGTH], SHA512_CTX*);
+char* SHA512_End(SHA512_CTX*, char[SHA512_DIGEST_STRING_LENGTH]);
+char* SHA512_Data(const uint8_t*, size_t, char[SHA512_DIGEST_STRING_LENGTH]);
+
+#else /* SHA2_USE_INTTYPES_H */
+
+void SHA256_Init(SHA256_CTX *);
+void SHA256_Update(SHA256_CTX*, const u_int8_t*, size_t);
+void SHA256_Final(u_int8_t[SHA256_DIGEST_LENGTH], SHA256_CTX*);
+char* SHA256_End(SHA256_CTX*, char[SHA256_DIGEST_STRING_LENGTH]);
+char* SHA256_Data(const u_int8_t*, size_t, char[SHA256_DIGEST_STRING_LENGTH]);
+
+void SHA384_Init(SHA384_CTX*);
+void SHA384_Update(SHA384_CTX*, const u_int8_t*, size_t);
+void SHA384_Final(u_int8_t[SHA384_DIGEST_LENGTH], SHA384_CTX*);
+char* SHA384_End(SHA384_CTX*, char[SHA384_DIGEST_STRING_LENGTH]);
+char* SHA384_Data(const u_int8_t*, size_t, char[SHA384_DIGEST_STRING_LENGTH]);
+
+void SHA512_Init(SHA512_CTX*);
+void SHA512_Update(SHA512_CTX*, const u_int8_t*, size_t);
+void SHA512_Final(u_int8_t[SHA512_DIGEST_LENGTH], SHA512_CTX*);
+char* SHA512_End(SHA512_CTX*, char[SHA512_DIGEST_STRING_LENGTH]);
+char* SHA512_Data(const u_int8_t*, size_t, char[SHA512_DIGEST_STRING_LENGTH]);
+
+#endif /* SHA2_USE_INTTYPES_H */
+
+#else /* NOPROTO */
+
+void SHA256_Init();
+void SHA256_Update();
+void SHA256_Final();
+char* SHA256_End();
+char* SHA256_Data();
+
+void SHA384_Init();
+void SHA384_Update();
+void SHA384_Final();
+char* SHA384_End();
+char* SHA384_Data();
+
+void SHA512_Init();
+void SHA512_Update();
+void SHA512_Final();
+char* SHA512_End();
+char* SHA512_Data();
+
+#endif /* NOPROTO */
+
+#endif /* __SHA2_H__ */

src/zvbi/bcd.h

@@ -0,0 +1,216 @@
+/*
+ *  libzvbi -- BCD arithmetic for Teletext page numbers
+ *
+ *  Copyright (C) 2001, 2002 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: bcd.h,v 1.19 2008/02/21 07:18:52 mschimek Exp $ */
+
+#ifndef BCD_H
+#define BCD_H
+
+#include "ccx_common_platform.h"
+/**
+ * @addtogroup BCD BCD arithmetic for Teletext page numbers
+ * @ingroup HiDec
+ *
+ * Teletext page numbers are expressed as packed binary coded decimal
+ * numbers in range 0x100 to 0x8FF. The bcd format encodes one decimal
+ * digit in every hex nibble (four bits) of the number. Page numbers
+ * containing digits 0xA to 0xF are reserved for various system purposes
+ * and not intended for display.
+ */
+
+/* Public */
+
+/**
+ * @ingroup HiDec
+ * 
+ * Teletext or Closed Caption page number. For Teletext pages
+ * this is a packed bcd number in range 0x100 ... 0x8FF. Page
+ * numbers containing digits 0xA to 0xF are reserved for various
+ * system purposes, these pages are not intended for display.
+ * 
+ * Closed Caption page numbers between 1 ... 8 correspond
+ * to the four Caption and Text channels:
+ * <table>
+ * <tr><td>1</td><td>Caption 1</td><td>
+ *   "Primary synchronous caption service [English]"</td></tr>
+ * <tr><td>2</td><td>Caption 2</td><td>
+ *   "Special non-synchronous data that is intended to
+ *   augment information carried in the program"</td></tr>
+ * <tr><td>3</td><td>Caption 3</td><td>
+ *   "Secondary synchronous caption service, usually
+ *    second language [Spanish, French]"</td></tr>
+ * <tr><td>4</td><td>Caption 4</td><td>
+ *   "Special non-synchronous data similar to Caption 2"</td></tr>
+ * <tr><td>5</td><td>Text 1</td><td>
+ *   "First text service, data usually not program related"</td></tr>
+ * <tr><td>6</td><td>Text 2</td><td>
+ *   "Second text service, additional data usually not program related
+ *    [ITV data]"</td></tr>
+ * <tr><td>7</td><td>Text 3</td><td>
+ *   "Additional text channel"</td></tr>
+ * <tr><td>8</td><td>Text 4</td><td>
+ *   "Additional text channel"</td></tr>
+ * </table>
+ */
+/* XXX unsigned? */
+typedef int vbi_pgno;
+
+/**
+ * @ingroup HiDec
+ *
+ * This is the subpage number only applicable to Teletext pages,
+ * a packed bcd number in range 0x00 ... 0x99. On special 'clock' pages
+ * (for example listing the current time in different time zones)
+ * it can assume values between 0x0000 ... 0x2359 expressing
+ * local time. These are not actually subpages.
+ */
+typedef int vbi_subno;
+
+/**
+ * @ingroup HiDec
+ */
+#define VBI_ANY_SUBNO 0x3F7F
+/**
+ * @ingroup HiDec
+ */
+#define VBI_NO_SUBNO 0x3F7F
+
+/**
+ * @ingroup BCD
+ * @param dec Decimal number.
+ * 
+ * Converts a two's complement binary between 0 ... 999 to a
+ * packed bcd number in range  0x000 ... 0x999. Extra digits in
+ * the input will be discarded.
+ * 
+ * @return
+ * BCD number.
+ */
+inline unsigned int vbi_dec2bcd(unsigned int dec)
+{
+	return (dec % 10) + ((dec / 10) % 10) * 16 + ((dec / 100) % 10) * 256;
+}
+
+/**
+ * @ingroup BCD
+ * @since 0.2.28
+ */
+#define vbi_bin2bcd(n) vbi_dec2bcd(n)
+
+/**
+ * @ingroup BCD
+ * @param bcd BCD number.
+ * 
+ * Converts a packed bcd number between 0x000 ... 0xFFF to a two's
+ * complement binary in range 0 ... 999. Extra digits in the input
+ * will be discarded.
+ * 
+ * @return
+ * Decimal number. The result is undefined when the bcd number contains
+ * hex digits 0xA ... 0xF.
+ **/
+inline unsigned int vbi_bcd2dec(unsigned int bcd)
+{
+	return (bcd & 15) + ((bcd >> 4) & 15) * 10 + ((bcd >> 8) & 15) * 100;
+}
+
+/**
+ * @ingroup BCD
+ * @since 0.2.28
+ */
+#define vbi_bcd2bin(n) vbi_bcd2dec(n)
+
+/**
+ * @ingroup BCD
+ * @param a BCD number.
+ * @param b BCD number.
+ * 
+ * Adds two packed bcd numbers, returning a packed bcd sum. Arguments
+ * and result are in range 0xF000&nbsp;0000 ... 0x0999&nbsp;9999, that
+ * is -10**7 ... +10**7 - 1 in decimal notation. To subtract you can
+ * add the 10's complement, e. g. -1 = 0xF999&nbsp;9999.
+ * 
+ * @return
+ * Packed bcd number. The result is undefined when any of the arguments
+ * contain hex digits 0xA ... 0xF.
+ */
+inline unsigned int vbi_add_bcd(unsigned int a, unsigned int b)
+{
+	unsigned int t;
+
+	a += 0x06666666;
+	t  = a + b;
+	b ^= a ^ t;
+	b  = (~b & 0x11111110) >> 3;
+	b |= b * 2;
+
+	return t - b;
+}
+
+/**
+ * @ingroup BCD
+ * @param bcd BCD number.
+ * 
+ * Tests if @a bcd forms a valid BCD number. The argument must be
+ * in range 0x0000&nbsp;0000 ... 0x0999&nbsp;9999.
+ * 
+ * @return
+ * @c FALSE if @a bcd contains hex digits 0xA ... 0xF.
+ */
+inline int vbi_is_bcd(unsigned int bcd)
+{
+	static const unsigned int x = 0x06666666;
+
+	return (((bcd + x) ^ (bcd ^ x)) & 0x11111110) == 0;
+}
+
+/**
+ * @ingroup BCD
+ * @param bcd Unsigned BCD number.
+ * @param maximum Unsigned maximum value.
+ *
+ * Compares an unsigned packed bcd number digit-wise against a maximum
+ * value, for example 0x295959. @a maximum can contain digits 0x0
+ * ... 0xF.
+ *
+ * @return
+ * @c TRUE if any digit of @a bcd is greater than the
+ * corresponding digit of @a maximum.
+ *
+ * @since 0.2.28
+ */
+inline int vbi_bcd_digits_greater (unsigned int	bcd, unsigned int maximum)
+{
+	maximum ^= ~0;
+
+	return 0 != (((bcd + maximum) ^ bcd ^ maximum) & 0x11111110);
+}
+
+/* Private */
+
+#endif /* BCD_H */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/bit_slicer.h

@@ -0,0 +1,197 @@
+/*
+ *  libzvbi -- Bit slicer
+ *
+ *  Copyright (C) 2000-2007 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: bit_slicer.h,v 1.11 2008/02/24 14:18:13 mschimek Exp $ */
+
+#ifndef __ZVBI_BIT_SLICER_H__
+#define __ZVBI_BIT_SLICER_H__
+
+#include "sampling_par.h"
+
+VBI_BEGIN_DECLS
+
+/**
+ * @addtogroup BitSlicer
+ * @{
+ */
+
+/**
+ * @brief Modulation used for VBI data transmission.
+ */
+typedef enum {
+	/**
+	 * The data is 'non-return to zero' coded, logical '1' bits
+	 * are described by high sample values, logical '0' bits by
+	 * low values. The data is last significant bit first transmitted.
+	 */
+	VBI3_MODULATION_NRZ_LSB,
+
+	/**
+	 * 'Non-return to zero' coded, most significant bit first
+	 * transmitted.
+	 */
+	VBI3_MODULATION_NRZ_MSB,
+
+	/**
+	 * The data is 'bi-phase' coded. Each data bit is described
+	 * by two complementary signalling elements, a logical '1'
+	 * by a sequence of '10' elements, a logical '0' by a '01'
+	 * sequence. The data is last significant bit first transmitted.
+	 */
+	VBI3_MODULATION_BIPHASE_LSB,
+
+	/** 'Bi-phase' coded, most significant bit first transmitted. */
+	VBI3_MODULATION_BIPHASE_MSB
+} vbi3_modulation;
+
+/**
+ * @brief Bit slicer context.
+ *
+ * The contents of this structure are private.
+ * Call vbi3_bit_slicer_new() to allocate a bit slicer context.
+ */
+typedef struct _vbi3_bit_slicer vbi3_bit_slicer;
+
+typedef enum {
+	VBI3_CRI_BIT = 1,
+	VBI3_FRC_BIT,
+	VBI3_PAYLOAD_BIT
+} vbi3_bit_slicer_bit;
+
+/**
+ * @brief Bit slicer sampling point.
+ *
+ * This structure contains information about
+ * a bit sampled by the bit slicer.
+ */
+typedef struct {
+	/** Whether this struct refers to a CRI, FRC or payload bit. */
+	vbi3_bit_slicer_bit	kind;
+
+	/** Number of the sample times 256. */
+	unsigned int		index;
+
+	/** Signal amplitude at this sample, in range 0 to 65535. */
+	unsigned int		level;
+
+	/** 0/1 threshold at this sample, in range 0 to 65535. */
+	unsigned int		thresh;
+} vbi3_bit_slicer_point;
+
+extern vbi_bool
+vbi3_bit_slicer_slice_with_points
+				(vbi3_bit_slicer *	bs,
+				 uint8_t *		buffer,
+				 unsigned int		buffer_size,
+				 vbi3_bit_slicer_point *points,
+				 unsigned int *		n_points,
+				 unsigned int		max_points,
+				 const uint8_t *	raw)
+  _vbi_nonnull ((1, 2, 4, 5, 7));
+extern vbi_bool
+vbi3_bit_slicer_slice		(vbi3_bit_slicer *	bs,
+				 uint8_t *		buffer,
+				 unsigned int		buffer_size,
+				 const uint8_t *	raw)
+  _vbi_nonnull ((1, 2, 4));
+extern vbi_bool
+vbi3_bit_slicer_set_params	(vbi3_bit_slicer *	bs,
+				 vbi_pixfmt		sample_format,
+				 unsigned int		sampling_rate,
+				 unsigned int		sample_offset,
+				 unsigned int		samples_per_line,
+				 unsigned int		cri,
+				 unsigned int		cri_mask,
+				 unsigned int		cri_bits,
+				 unsigned int		cri_rate,
+				 unsigned int		cri_end,
+				 unsigned int		frc,
+				 unsigned int		frc_bits,
+				 unsigned int		payload_bits,
+				 unsigned int		payload_rate,
+				 vbi3_modulation	modulation)
+  _vbi_nonnull ((1));
+extern void
+vbi3_bit_slicer_set_log_fn	(vbi3_bit_slicer *	bs,
+				 vbi_log_mask		mask,
+				 vbi_log_fn *		log_fn,
+				 void *			user_data)
+  _vbi_nonnull ((1));
+extern void
+vbi3_bit_slicer_delete		(vbi3_bit_slicer *	bs);
+extern vbi3_bit_slicer *
+vbi3_bit_slicer_new		(void)
+  _vbi_alloc;
+
+/* Private */
+
+typedef vbi_bool
+_vbi3_bit_slicer_fn		(vbi3_bit_slicer *	bs,
+				 uint8_t *		buffer,
+				 vbi3_bit_slicer_point *points,
+				 unsigned int *		n_points,
+				 const uint8_t *	raw);
+
+/** @internal */
+struct _vbi3_bit_slicer {
+	_vbi3_bit_slicer_fn *	func;
+
+	vbi_pixfmt		sample_format;
+	unsigned int		cri;
+	unsigned int		cri_mask;
+	unsigned int		thresh;
+	unsigned int		thresh_frac;
+	unsigned int		cri_samples;
+	unsigned int		cri_rate;
+	unsigned int		oversampling_rate;
+	unsigned int		phase_shift;
+	unsigned int		step;
+	unsigned int		frc;
+	unsigned int		frc_bits;
+	unsigned int		total_bits;
+	unsigned int		payload;
+	unsigned int		endian;
+	unsigned int		bytes_per_sample;
+	unsigned int		skip;
+	unsigned int		green_mask;
+
+	_vbi_log_hook		log;
+};
+
+extern void
+_vbi3_bit_slicer_destroy	(vbi3_bit_slicer *	bs)
+  _vbi_nonnull ((1));
+extern vbi_bool
+_vbi3_bit_slicer_init		(vbi3_bit_slicer *	bs)
+  _vbi_nonnull ((1));
+
+/** @} */
+
+VBI_END_DECLS
+
+#endif /* __ZVBI_BIT_SLICER_H__ */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/decoder.c

@@ -0,0 +1,808 @@
+/*
+ *  libzvbi -- Old raw VBI decoder
+ *
+ *  Copyright (C) 2000, 2001, 2002 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: decoder.c,v 1.25 2008/02/19 00:35:15 mschimek Exp $ */
+
+/* Note this code is only retained for compatibility with older versions
+   of libzvbi. vbi_raw_decoder is now just a wrapper for the new raw
+   decoder (raw_decoder.c) and bit slicer (bit_slicer.c). We'll drop
+   the old API in libzvbi 0.3. Other modules (e.g. io-v4l2k.c) should
+   already use the new raw VBI decoder directly. */
+
+#include "misc.h"
+#include "zvbi_decoder.h"
+#include "raw_decoder.h"
+#include "ccx_common_common.h"
+
+/**
+ * @addtogroup Rawdec Raw VBI decoder
+ * @ingroup Raw
+ * @brief Converting raw VBI samples to bits and bytes.
+ *
+ * The libzvbi already offers hardware interfaces to obtain sliced
+ * VBI data for further processing. However if you want to write your own
+ * interface or decode data services not covered by libzvbi you can use
+ * these lower level functions.
+ */
+
+/*
+ *  Bit Slicer
+ */
+
+#define OVERSAMPLING 4		/* 1, 2, 4, 8 */
+#define THRESH_FRAC 9
+
+/*
+ * Note this is just a template. The code is inlined,
+ * with bpp and endian being const.
+ *
+ * This function translates from the image format to
+ * plain bytes, with linear interpolation of samples.
+ * Could be further improved with a lowpass filter.
+ */
+static inline unsigned int
+sample(uint8_t *raw, int offs, int bpp, int endian)
+{
+	unsigned char frac = offs;
+	int raw0, raw1;
+
+	switch (bpp) {
+	case 14: /* 1:5:5:5 LE/BE */
+		raw += (offs >> 8) * 2;
+		raw0 = (raw[0 + endian] + raw[1 - endian] * 256) & 0x07C0;
+		raw1 = (raw[2 + endian] + raw[3 - endian] * 256) & 0x07C0;
+		return (raw1 - raw0) * frac + (raw0 << 8);
+
+	case 15: /* 5:5:5:1 LE/BE */
+		raw += (offs >> 8) * 2;
+		raw0 = (raw[0 + endian] + raw[1 - endian] * 256) & 0x03E0;
+		raw1 = (raw[2 + endian] + raw[3 - endian] * 256) & 0x03E0;
+		return (raw1 - raw0) * frac + (raw0 << 8);
+
+	case 16: /* 5:6:5 LE/BE */
+		raw += (offs >> 8) * 2;
+		raw0 = (raw[0 + endian] + raw[1 - endian] * 256) & 0x07E0;
+		raw1 = (raw[2 + endian] + raw[3 - endian] * 256) & 0x07E0;
+		return (raw1 - raw0) * frac + (raw0 << 8);
+
+	default: /* 8 (intermediate bytes skipped by caller) */
+		raw += (offs >> 8) * bpp;
+		return (raw[bpp] - raw[0]) * frac + (raw[0] << 8);
+	}
+}
+
+/*
+ * Note this is just a template. The code is inlined,
+ * with bpp being const.
+ */
+static inline vbi_bool
+bit_slicer_tmpl(vbi_bit_slicer *d, uint8_t *raw,
+		uint8_t *buf, int bpp, int endian)
+{
+	unsigned int i, j, k;
+	unsigned int cl = 0, thresh0 = d->thresh, tr;
+	unsigned int c = 0, t;
+	unsigned char b, b1 = 0;
+	int raw0, raw1, mask;
+
+	raw += d->skip;
+
+	if (bpp == 14)
+		mask = 0x07C0;
+	else if (bpp == 15)
+		mask = 0x03E0;
+	else if (bpp == 16)
+		mask = 0x07E0;
+
+	for (i = d->cri_bytes; i > 0; raw += (bpp >= 14 && bpp <= 16) ? 2 : bpp, i--) {
+		if (bpp >= 14 && bpp <= 16) {
+			raw0 = (raw[0 + endian] + raw[1 - endian] * 256) & mask;
+			raw1 = (raw[2 + endian] + raw[3 - endian] * 256) & mask;
+			tr = d->thresh >> THRESH_FRAC;
+			d->thresh += ((raw0 - tr) * (int) ABS(raw1 - raw0)) >>
+				((bpp == 15) ? 2 : 3);
+			t = raw0 * OVERSAMPLING;
+		} else {
+			tr = d->thresh >> THRESH_FRAC;
+			d->thresh += ((int) raw[0] - tr) * (int) ABS(raw[bpp] - raw[0]);
+			t = raw[0] * OVERSAMPLING;
+		}
+
+		for (j = OVERSAMPLING; j > 0; j--) {
+			b = ((t + (OVERSAMPLING / 2)) / OVERSAMPLING >= tr);
+
+    			if (b ^ b1) {
+				cl = d->oversampling_rate >> 1;
+			} else {
+				cl += d->cri_rate;
+
+				if (cl >= (unsigned int) d->oversampling_rate) {
+					cl -= d->oversampling_rate;
+
+					c = c * 2 + b;
+
+					if ((c & d->cri_mask) == d->cri) {
+						i = d->phase_shift;
+						tr *= 256;
+						c = 0;
+
+						for (j = d->frc_bits; j > 0; j--) {
+							c = c * 2 + (sample(raw, i, bpp, endian) >= tr);
+    							i += d->step;
+						}
+
+						if (c ^= d->frc)
+							return CCX_FALSE;
+
+						/* CRI/FRC found, now get the
+						   payload and exit */
+						
+						switch (d->endian) {
+						case 3:
+							for (j = 0; j < (unsigned int) d->payload; j++) {
+					    			c >>= 1;
+								c += (sample(raw, i, bpp, endian) >= tr) << 7;
+			    					i += d->step;
+
+								if ((j & 7) == 7)
+									*buf++ = c;
+					    		}
+
+							*buf = c >> ((8 - d->payload) & 7);
+							break;
+
+						case 2:
+							for (j = 0; j < (unsigned int) d->payload; j++) {
+								c = c * 2 + (sample(raw, i, bpp, endian) >= tr);
+			    					i += d->step;
+
+								if ((j & 7) == 7)
+									*buf++ = c;
+					    		}
+
+							*buf = c & ((1 << (d->payload & 7)) - 1);
+							break;
+
+						case 1:
+							for (j = d->payload; j > 0; j--) {
+								for (k = 0; k < 8; k++) {
+						    			c >>= 1;
+									c += (sample(raw, i, bpp, endian) >= tr) << 7;
+			    						i += d->step;
+								}
+
+								*buf++ = c;
+					    		}
+
+							break;
+
+						case 0:
+							for (j = d->payload; j > 0; j--) {
+								for (k = 0; k < 8; k++) {
+									c = c * 2 + (sample(raw, i, bpp, endian) >= tr);
+			    						i += d->step;
+								}
+
+								*buf++ = c;
+					    		}
+
+							break;
+						}
+
+			    			return CCX_TRUE;
+					}
+				}
+			}
+
+			b1 = b;
+
+			if (OVERSAMPLING > 1) {
+				if (bpp >= 14 && bpp <= 16) {
+					t += raw1;
+					t -= raw0;
+				} else {
+					t += raw[bpp];
+					t -= raw[0];
+				}
+			}
+		}
+	}
+
+	d->thresh = thresh0;
+
+	return CCX_FALSE;
+}
+
+static vbi_bool
+bit_slicer_1(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 1, 0);
+}
+
+static vbi_bool
+bit_slicer_2(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 2, 0);
+}
+
+static vbi_bool
+bit_slicer_3(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 3, 0);
+}
+
+static vbi_bool
+bit_slicer_4(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 4, 0);
+}
+
+static vbi_bool
+bit_slicer_1555_le(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 14, 0);
+}
+
+static vbi_bool
+bit_slicer_5551_le(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 15, 0);
+}
+
+static vbi_bool
+bit_slicer_565_le(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 16, 0);
+}
+
+static vbi_bool
+bit_slicer_1555_be(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 14, 1);
+}
+
+static vbi_bool
+bit_slicer_5551_be(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 15, 1);
+}
+
+static vbi_bool
+bit_slicer_565_be(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 16, 1);
+}
+
+/**
+ * @param slicer Pointer to vbi_bit_slicer object to be initialized. 
+ * @param raw_samples Number of samples or pixels in one raw vbi line
+ *   later passed to vbi_bit_slice(). This limits the number of
+ *   bytes read from the sample buffer.
+ * @param sampling_rate Raw vbi sampling rate in Hz, that is the number of
+ *   samples or pixels sampled per second by the hardware. 
+ * @param cri_rate The Clock Run In is a NRZ modulated
+ *   sequence of '0' and '1' bits prepending most data transmissions to
+ *   synchronize data acquisition circuits. This parameter gives the CRI bit
+ *   rate in Hz, that is the number of CRI bits transmitted per second.
+ * @param bit_rate The transmission bit rate of all data bits following the CRI
+ *   in Hz.
+ * @param cri_frc The FRaming Code usually following the CRI is a bit sequence
+ *   identifying the data service, and per libzvbi definition modulated
+ *   and transmitted at the same bit rate as the payload (however nothing
+ *   stops you from counting all nominal CRI and FRC bits as CRI).
+ *   The bit slicer compares the bits in this word, lsb last transmitted,
+ *   against the transmitted CRI and FRC. Decoding of payload starts
+ *   with the next bit after a match.
+ * @param cri_mask Of the CRI bits in @c cri_frc, only these bits are
+ *   actually significant for a match. For instance it is wise
+ *   not to rely on the very first CRI bits transmitted. Note this
+ *   mask is not shifted left by @a frc_bits.
+ * @param cri_bits 
+ * @param frc_bits Number of CRI and FRC bits in @a cri_frc, respectively.
+ *   Their sum is limited to 32.
+ * @param payload Number of payload <em>bits</em>. Only this data
+ *   will be stored in the vbi_bit_slice() output. If this number
+ *   is no multiple of eight, the most significant bits of the
+ *   last byte are undefined.
+ * @param modulation Modulation of the vbi data, see vbi_modulation.
+ * @param fmt Format of the raw data, see vbi_pixfmt.
+ * 
+ * Initializes vbi_bit_slicer object. Usually you will not use this
+ * function but vbi_raw_decode(), the vbi image decoder which handles
+ * all these details.
+ */
+void
+vbi_bit_slicer_init(vbi_bit_slicer *slicer,
+		    int raw_samples, int sampling_rate,
+		    int cri_rate, int bit_rate,
+		    unsigned int cri_frc, unsigned int cri_mask,
+		    int cri_bits, int frc_bits, int payload,
+		    vbi_modulation modulation, vbi_pixfmt fmt)
+{
+	unsigned int c_mask = (unsigned int)(-(cri_bits > 0)) >> (32 - cri_bits);
+	unsigned int f_mask = (unsigned int)(-(frc_bits > 0)) >> (32 - frc_bits);
+	int gsh = 0;
+
+	slicer->func = bit_slicer_1;
+
+	switch (fmt) {
+	case VBI_PIXFMT_RGB24:
+	case VBI_PIXFMT_BGR24:
+	        slicer->func = bit_slicer_3;
+		slicer->skip = 1;
+		break;
+
+	case VBI_PIXFMT_RGBA32_LE:
+	case VBI_PIXFMT_BGRA32_LE:
+		slicer->func = bit_slicer_4;
+		slicer->skip = 1;
+		break;
+
+	case VBI_PIXFMT_RGBA32_BE:
+	case VBI_PIXFMT_BGRA32_BE:
+		slicer->func = bit_slicer_4;
+		slicer->skip = 2;
+		break;
+
+	case VBI_PIXFMT_RGB16_LE:
+	case VBI_PIXFMT_BGR16_LE:
+		slicer->func = bit_slicer_565_le;
+		gsh = 3; /* (green << 3) & 0x07E0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_RGBA15_LE:
+	case VBI_PIXFMT_BGRA15_LE:
+		slicer->func = bit_slicer_5551_le;
+		gsh = 2; /* (green << 2) & 0x03E0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_ARGB15_LE:
+	case VBI_PIXFMT_ABGR15_LE:
+		slicer->func = bit_slicer_1555_le;
+		gsh = 3; /* (green << 2) & 0x07C0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_RGB16_BE:
+	case VBI_PIXFMT_BGR16_BE:
+		slicer->func = bit_slicer_565_be;
+		gsh = 3; /* (green << 3) & 0x07E0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_RGBA15_BE:
+	case VBI_PIXFMT_BGRA15_BE:
+		slicer->func = bit_slicer_5551_be;
+		gsh = 2; /* (green << 2) & 0x03E0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_ARGB15_BE:
+	case VBI_PIXFMT_ABGR15_BE:
+		slicer->func = bit_slicer_1555_be;
+		gsh = 3; /* (green << 2) & 0x07C0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_YUV420:
+		slicer->func = bit_slicer_1;
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_YUYV:
+	case VBI_PIXFMT_YVYU:
+		slicer->func = bit_slicer_2;
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_UYVY:
+	case VBI_PIXFMT_VYUY:
+		slicer->func = bit_slicer_2;
+		slicer->skip = 1;
+		break;
+
+	default:
+		fprintf(stderr, "vbi_bit_slicer_init: unknown pixfmt %d\n", fmt);
+		exit(EXIT_FAILURE);
+	}
+
+	slicer->cri_mask		= cri_mask & c_mask;
+	slicer->cri		 	= (cri_frc >> frc_bits) & slicer->cri_mask;
+	/* We stop searching for CRI/FRC when the payload
+	   cannot possibly fit anymore. */
+	slicer->cri_bytes		= raw_samples
+		- ((long long) sampling_rate * (payload + frc_bits)) / bit_rate;
+	slicer->cri_rate		= cri_rate;
+	/* Raw vbi data is oversampled to account for low sampling rates. */
+	slicer->oversampling_rate	= sampling_rate * OVERSAMPLING;
+	/* 0/1 threshold */
+	slicer->thresh			= 105 << (THRESH_FRAC + gsh);
+	slicer->frc			= cri_frc & f_mask;
+	slicer->frc_bits		= frc_bits;
+	/* Payload bit distance in 1/256 raw samples. */
+	slicer->step			= (int)(sampling_rate * 256.0 / bit_rate);
+
+	if (payload & 7) {
+		slicer->payload	= payload;
+		slicer->endian	= 3;
+	} else {
+		slicer->payload	= payload >> 3;
+		slicer->endian	= 1;
+	}
+
+	switch (modulation) {
+	case VBI_MODULATION_NRZ_MSB:
+		slicer->endian--;
+	case VBI_MODULATION_NRZ_LSB:
+		slicer->phase_shift = (int)
+			(sampling_rate * 256.0 / cri_rate * .5
+			 + sampling_rate * 256.0 / bit_rate * .5 + 128);
+		break;
+
+	case VBI_MODULATION_BIPHASE_MSB:
+		slicer->endian--;
+	case VBI_MODULATION_BIPHASE_LSB:
+		/* Phase shift between the NRZ modulated CRI and the rest */
+		slicer->phase_shift = (int)
+			(sampling_rate * 256.0 / cri_rate * .5
+			 + sampling_rate * 256.0 / bit_rate * .25 + 128);
+		break;
+	}
+}
+
+/**
+ * @example examples/wss.c
+ * WSS capture example.
+ */
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param raw A raw vbi image as defined in the vbi_raw_decoder structure
+ *   (rd->sampling_format, rd->bytes_per_line, rd->count[0] + rd->count[1]
+ *    scan lines).
+ * @param out Buffer to store the decoded vbi_sliced data. Since every
+ *   vbi scan line may contain data, this must be an array of vbi_sliced
+ *   with the same number of entries as scan lines in the raw image
+ *   (rd->count[0] + rd->count[1]).
+ * 
+ * Decode a raw vbi image, consisting of several scan lines of raw vbi data,
+ * into sliced vbi data. The output is sorted by line number.
+ * 
+ * Note this function attempts to learn which lines carry which data
+ * service, or none, to speed up decoding. You should avoid using the same
+ * vbi_raw_decoder structure for different sources.
+ *
+ * @return
+ * The number of lines decoded, i. e. the number of vbi_sliced records
+ * written.
+ */
+int vbi_raw_decode (vbi_raw_decoder *	rd,
+				 uint8_t *		raw,
+				 vbi_sliced *		out)
+{
+	vbi3_raw_decoder *rd3;
+	unsigned int n_lines;
+
+	assert (NULL != rd);
+	assert (NULL != raw);
+	assert (NULL != out);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+	n_lines = rd->count[0] + rd->count[1];
+
+
+	n_lines = vbi3_raw_decoder_decode (rd3, out, n_lines, raw);
+
+	return n_lines;
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param start Array of start line indices for both fields
+ * @param count Array of line counts for both fields
+ * 
+ * Grows or shrinks the internal state arrays for VBI geometry changes
+ */
+void
+vbi_raw_decoder_resize		(vbi_raw_decoder *	rd,
+				 int *			start,
+				 unsigned int *		count)
+{
+	vbi_service_set service_set;
+	vbi3_raw_decoder *rd3;
+
+	assert (NULL != rd);
+	assert (NULL != start);
+	assert (NULL != count);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+
+
+	{
+		if ((rd->start[0] == start[0])
+		    && (rd->start[1] == start[1])
+		    && (rd->count[0] == (int) count[0])
+		    && (rd->count[1] == (int) count[1])) {
+			return;
+		}
+
+		rd->start[0] = start[0];
+		rd->start[1] = start[1];
+		rd->count[0] = count[0];
+		rd->count[1] = count[1];
+
+		service_set = vbi3_raw_decoder_set_sampling_par
+			(rd3, (vbi_sampling_par *) rd, /* strict */ 0);
+	}
+
+
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param services Set of @ref VBI_SLICED_ symbols.
+ * 
+ * Removes one or more data services to be decoded from the
+ * vbi_raw_decoder structure. This function can be called at any
+ * time and does not touch sampling parameters. 
+ * 
+ * @return 
+ * Set of @ref VBI_SLICED_ symbols describing the remaining data
+ * services that will be decoded.
+ */
+unsigned int
+vbi_raw_decoder_remove_services	(vbi_raw_decoder *	rd,
+				 unsigned int		services)
+{
+	vbi_service_set service_set;
+	vbi3_raw_decoder *rd3;
+
+	assert (NULL != rd);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+	service_set = services;
+
+	{
+		service_set = vbi3_raw_decoder_remove_services
+			(rd3, service_set);
+	}
+
+	return service_set;
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param services Set of @ref VBI_SLICED_ symbols.
+ * @param strict See description of vbi_raw_decoder_add_services()
+ *
+ * Check which of the given services can be decoded with current capture
+ * parameters at a given strictness level.
+ *
+ * @return
+ * Subset of services actually decodable.
+ */
+unsigned int
+vbi_raw_decoder_check_services	(vbi_raw_decoder *	rd,
+				 unsigned int		services,
+				 int			strict)
+{
+	vbi_service_set service_set;
+
+	assert (NULL != rd);
+
+	service_set = services;
+
+
+	{
+		service_set = vbi_sampling_par_check_services
+			((vbi_sampling_par *) rd, service_set, strict);
+	}
+
+
+	return (unsigned int) service_set;
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param services Set of @ref VBI_SLICED_ symbols.
+ * @param strict A value of 0, 1 or 2 requests loose, reliable or strict
+ *  matching of sampling parameters. For example if the data service
+ *  requires knowledge of line numbers while they are not known, @c 0
+ *  will accept the service (which may work if the scan lines are
+ *  populated in a non-confusing way) but @c 1 or @c 2 will not. If the
+ *  data service <i>may</i> use more lines than are sampled, @c 1 will
+ *  accept but @c 2 will not. If unsure, set to @c 1.
+ * 
+ * After you initialized the sampling parameters in @a rd (according to
+ * the abilities of your raw vbi source), this function adds one or more
+ * data services to be decoded. The libzvbi raw vbi decoder can decode up
+ * to eight data services in parallel. You can call this function while
+ * already decoding, it does not change sampling parameters and you must
+ * not change them either after calling this.
+ * 
+ * @return
+ * Set of @ref VBI_SLICED_ symbols describing the data services that actually
+ * will be decoded. This excludes those services not decodable given
+ * the sampling parameters in @a rd.
+ */
+unsigned int
+vbi_raw_decoder_add_services	(vbi_raw_decoder *	rd,
+				 unsigned int		services,
+				 int			strict)
+{
+	vbi_service_set service_set;
+	vbi3_raw_decoder *rd3;
+
+	assert (NULL != rd);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+	service_set = services;
+
+
+
+	{
+		vbi3_raw_decoder_set_sampling_par
+			(rd3, (vbi_sampling_par *) rd, strict);
+
+		service_set = vbi3_raw_decoder_add_services
+			(rd3, service_set, strict);
+	}
+
+	return service_set;
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param services Set of VBI_SLICED_ symbols. Here (and only here) you
+ *   can add @c VBI_SLICED_VBI_625 or @c VBI_SLICED_VBI_525 to include all
+ *   vbi scan lines in the calculated sampling parameters.
+ * @param scanning When 525 accept only NTSC services, when 625
+ *   only PAL/SECAM services. When scanning is 0, determine the scanning
+ *   from the requested services, an ambiguous set will pick
+ *   a 525 or 625 line system at random.
+ * @param max_rate If given, the highest data bit rate in Hz of all
+ *   services requested is stored here. (The sampling rate
+ *   should be at least twice as high; rd->sampling_rate will
+ *   be set to a more reasonable value of 27 MHz derived
+ *   from ITU-R Rec. 601.)
+ * 
+ * Calculate the sampling parameters in @a rd required to receive and
+ * decode the requested data @a services. rd->sampling_format will be
+ * @c VBI_PIXFMT_YUV420, rd->bytes_per_line set accordingly to a
+ * reasonable minimum. This function can be used to initialize hardware
+ * prior to calling vbi_raw_decoder_add_service().
+ * 
+ * @return
+ * Set of @ref VBI_SLICED_ symbols describing the data services covered
+ * by the calculated sampling parameters. This excludes services the libzvbi
+ * raw decoder cannot decode.
+ */
+unsigned int
+vbi_raw_decoder_parameters	(vbi_raw_decoder *	rd,
+				 unsigned int		services,
+				 int			scanning,
+				 int *			max_rate)
+{
+	vbi_videostd_set videostd_set;
+	vbi_service_set service_set;
+
+	switch (scanning) {
+	case 525:
+		videostd_set = VBI_VIDEOSTD_SET_525_60;
+		break;
+
+	case 625:
+		videostd_set = VBI_VIDEOSTD_SET_625_50;
+		break;
+
+	default:
+		videostd_set = 0;
+		break;
+	}
+
+	service_set = services;
+ 
+
+	{
+		service_set = vbi_sampling_par_from_services
+			((vbi_sampling_par *) rd,
+			 (unsigned int *) max_rate,
+			 videostd_set, service_set);
+	}
+
+	return (unsigned int) service_set;
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * 
+ * Reset a vbi_raw_decoder structure. This removes
+ * all previously added services to be decoded (if any)
+ * but does not touch the sampling parameters. You are
+ * free to change the sampling parameters after calling this.
+ */
+void
+vbi_raw_decoder_reset		(vbi_raw_decoder *	rd)
+{
+	vbi3_raw_decoder *rd3;
+
+	if (!rd)
+		return; /* compatibility */
+
+	assert (NULL != rd);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+
+
+	{
+		vbi3_raw_decoder_reset (rd3);
+	}
+}
+
+/**
+ * @param rd Pointer to initialized vbi_raw_decoder
+ *  structure, can be @c NULL.
+ * 
+ * Free all resources associated with @a rd.
+ */
+void
+vbi_raw_decoder_destroy		(vbi_raw_decoder *	rd)
+{
+	vbi3_raw_decoder *rd3;
+
+	assert (NULL != rd);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+
+	vbi3_raw_decoder_delete (rd3);
+
+	CLEAR (*rd);
+}
+
+/**
+ * @param rd Pointer to a vbi_raw_decoder structure.
+ * 
+ * Initializes a vbi_raw_decoder structure.
+ */
+void
+vbi_raw_decoder_init		(vbi_raw_decoder *	rd)
+{
+	vbi3_raw_decoder *rd3;
+
+	assert (NULL != rd);
+
+	CLEAR (*rd);
+
+
+	rd3 = vbi3_raw_decoder_new (/* sampling_par */ NULL);
+	assert (NULL != rd3);
+
+	rd->pattern = (int8_t *) rd3;
+}
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/macros.h

@@ -0,0 +1,156 @@
+/*
+ *  libzvbi -- Useful macros
+ *
+ *  Copyright (C) 2002, 2003, 2004, 2007 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: macros.h,v 1.12 2013/07/10 23:11:18 mschimek Exp $ */
+
+#ifndef __ZVBI_MACROS_H__
+#define __ZVBI_MACROS_H__
+
+#ifdef __cplusplus
+#  define VBI_BEGIN_DECLS extern "C" {
+#  define VBI_END_DECLS }
+#else
+#  define VBI_BEGIN_DECLS
+#  define VBI_END_DECLS
+#endif
+
+VBI_BEGIN_DECLS
+
+/* Public */
+
+#if __GNUC__ >= 4
+#  define _vbi_sentinel __attribute__ ((__sentinel__(0)))
+#  define _vbi_deprecated __attribute__ ((__deprecated__))
+#else
+#  define _vbi_sentinel
+#  define _vbi_deprecated
+#  define __restrict__
+#endif
+
+#if (__GNUC__ == 3 && __GNUC_MINOR__ >= 3) || __GNUC__ >= 4
+#  define _vbi_nonnull(params) __attribute__ ((__nonnull__ params))
+#  define _vbi_format(params) __attribute__ ((__format__ params))
+#else
+#  define _vbi_nonnull(params)
+#  define _vbi_format(params)
+#endif
+
+#if __GNUC__ >= 3
+#  define _vbi_pure __attribute__ ((__pure__))
+#  define _vbi_alloc __attribute__ ((__malloc__))
+#else
+#  define _vbi_pure
+#  define _vbi_alloc
+#endif
+
+#if __GNUC__ >= 2
+#  define _vbi_unused __attribute__ ((__unused__))
+#  define _vbi_const __attribute__ ((__const__))
+#  define _vbi_inline static __inline__
+#else
+#  define _vbi_unused
+#  define _vbi_const
+#  define _vbi_inline static
+#endif
+
+typedef int vbi_bool;
+/** @} */
+
+#ifndef NULL
+#  ifdef __cplusplus
+#    define NULL (0L)
+#  else
+#    define NULL ((void *) 0)
+#  endif
+#endif
+
+/* XXX Document me - for variadic funcs. */
+#define VBI_END ((void *) 0)
+
+#if 0
+typedef void
+vbi_lock_fn			(void *			user_data);
+typedef void
+vbi_unlock_fn			(void *			user_data);
+#endif
+
+/**
+ * @ingroup Basic
+ * @{
+ */
+typedef enum {
+	/** External error causes, for example lack of memory. */
+	VBI_LOG_ERROR		= 1 << 3,
+
+	/**
+	 * Invalid parameters and similar problems which suggest
+	 * a bug in the application using the library.
+	 */
+	VBI_LOG_WARNING		= 1 << 4,
+
+	/**
+	 * Causes of possibly undesired results, for example when a
+	 * data service cannot be decoded with the current video
+	 * standard setting.
+	 */
+	VBI_LOG_NOTICE		= 1 << 5,
+
+	/** Progress messages. */
+	VBI_LOG_INFO		= 1 << 6,
+
+	/** Information useful to debug the library. */
+	VBI_LOG_DEBUG		= 1 << 7,
+
+	/** Driver responses (strace). Not implemented yet. */
+	VBI_LOG_DRIVER		= 1 << 8,
+
+	/** More detailed debugging information. */
+	VBI_LOG_DEBUG2		= 1 << 9,
+	VBI_LOG_DEBUG3		= 1 << 10
+} vbi_log_mask;
+
+typedef void
+vbi_log_fn			(vbi_log_mask		level,
+				 const char *		context,
+				 const char *		message,
+				 void *			user_data);
+
+extern vbi_log_fn		vbi_log_on_stderr;
+/** @} */
+
+/* Private */
+
+typedef struct {
+	vbi_log_fn *		fn;
+	void *			user_data;
+	vbi_log_mask		mask;
+} _vbi_log_hook;
+
+VBI_END_DECLS
+
+#endif /* __ZVBI_MACROS_H__ */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/misc.h

@@ -0,0 +1,424 @@
+/*
+ *  libzvbi -- Miscellaneous cows and chickens
+ *
+ *  Copyright (C) 2000-2003 Iñaki García Etxebarria
+ *  Copyright (C) 2002-2007 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: misc.h,v 1.24 2013/07/02 02:32:31 mschimek Exp $ */
+
+#ifndef MISC_H
+#define MISC_H
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stddef.h>
+#include <stdarg.h>
+#include <string.h>
+#include <inttypes.h>		/* (u)intXX_t */
+#include <sys/types.h>		/* (s)size_t */
+#include <float.h>		/* DBL_MAX */
+#include <limits.h>		/* (S)SIZE_MAX */
+#include <assert.h>
+
+#include "macros.h"
+#include "ccx_common_platform.h"
+#include "ccx_common_constants.h"
+#include "ccx_common_structs.h"
+
+#define N_ELEMENTS(array) (sizeof (array) / sizeof (*(array)))
+
+#ifdef __GNUC__
+
+#if __GNUC__ < 3
+/* Expect expression usually true/false, schedule accordingly. */
+#  define likely(expr) (expr)
+#  define unlikely(expr) (expr)
+#else
+#  define likely(expr) __builtin_expect(expr, 1)
+#  define unlikely(expr) __builtin_expect(expr, 0)
+#endif
+
+#undef __i386__
+#undef __i686__
+/* FIXME #cpu is deprecated
+#if #cpu (i386)
+#  define __i386__ 1
+#endif
+#if #cpu (i686)
+#  define __i686__ 1
+#endif
+*/
+
+/* &x == PARENT (&x.tm_min, struct tm, tm_min),
+   safer than &x == (struct tm *) &x.tm_min. A NULL _ptr is safe and
+   will return NULL, not -offsetof(_member). */
+#undef PARENT
+#define PARENT(_ptr, _type, _member) ({					\
+	__typeof__ (&((_type *) 0)->_member) _p = (_ptr);		\
+	(_p != 0) ? (_type *)(((char *) _p) - offsetof (_type,		\
+	  _member)) : (_type *) 0;					\
+})
+
+/* Like PARENT(), to be used with const _ptr. */
+#define CONST_PARENT(_ptr, _type, _member) ({				\
+	__typeof__ (&((const _type *) 0)->_member) _p = (_ptr);		\
+	(_p != 0) ? (const _type *)(((const char *) _p) - offsetof	\
+	 (const _type, _member)) : (const _type *) 0;			\
+})
+
+/* Note the following macros have no side effects only when you
+   compile with GCC, so don't expect this. */
+
+/* Absolute value of int, long or long long without a branch.
+   Note ABS (INT_MIN) -> INT_MAX + 1. */
+#undef ABS
+#define ABS(n) ({							\
+	register __typeof__ (n) _n = (n), _t = _n;			\
+	if (-1 == (-1 >> 1)) { /* do we have signed shifts? */		\
+		_t >>= sizeof (_t) * 8 - 1;				\
+		_n ^= _t;						\
+		_n -= _t;						\
+	} else if (_n < 0) { /* also warns if n is unsigned type */	\
+		_n = -_n;						\
+	}								\
+	/* return */ _n;						\
+})
+
+#undef MIN
+#define MIN(x, y) ({							\
+	__typeof__ (x) _x = (x);					\
+	__typeof__ (y) _y = (y);					\
+	(void)(&_x == &_y); /* warn if types do not match */		\
+	/* return */ (_x < _y) ? _x : _y;				\
+})
+
+#undef MAX
+#define MAX(x, y) ({							\
+	__typeof__ (x) _x = (x);					\
+	__typeof__ (y) _y = (y);					\
+	(void)(&_x == &_y); /* warn if types do not match */		\
+	/* return */ (_x > _y) ? _x : _y;				\
+})
+
+/* Note other compilers may swap only int, long or pointer. */
+#undef SWAP
+#define SWAP(x, y)							\
+do {									\
+	__typeof__ (x) _x = x;						\
+	x = y;								\
+	y = _x;								\
+} while (0)
+
+#undef SATURATE
+#ifdef __i686__ /* has conditional move */
+#define SATURATE(n, min, max) ({					\
+	__typeof__ (n) _n = (n);					\
+	__typeof__ (n) _min = (min);					\
+	__typeof__ (n) _max = (max);					\
+	(void)(&_n == &_min); /* warn if types do not match */		\
+	(void)(&_n == &_max);						\
+	if (_n < _min)							\
+		_n = _min;						\
+	if (_n > _max)							\
+		_n = _max;						\
+	/* return */ _n;						\
+})
+#else
+#define SATURATE(n, min, max) ({					\
+	__typeof__ (n) _n = (n);					\
+	__typeof__ (n) _min = (min);					\
+	__typeof__ (n) _max = (max);					\
+	(void)(&_n == &_min); /* warn if types do not match */		\
+	(void)(&_n == &_max);						\
+	if (_n < _min)							\
+		_n = _min;						\
+	else if (_n > _max)						\
+		_n = _max;						\
+	/* return */ _n;						\
+})
+#endif
+
+#else /* !__GNUC__ */
+
+#define likely(expr) (expr)
+#define unlikely(expr) (expr)
+#undef __i386__
+#undef __i686__
+
+static char *
+PARENT_HELPER (char *p, unsigned int offset)
+{ return (0 == p) ? ((char *) 0) : p - offset; }
+
+static const char *
+CONST_PARENT_HELPER (const char *p, unsigned int offset)
+{ return (0 == p) ? ((char *) 0) : p - offset; }
+
+#define PARENT(_ptr, _type, _member)					\
+	((0 == offsetof (_type, _member)) ? (_type *)(_ptr)		\
+	 : (_type *) PARENT_HELPER ((char *)(_ptr), offsetof (_type, _member)))
+#define CONST_PARENT(_ptr, _type, _member)				\
+	((0 == offsetof (const _type, _member)) ? (const _type *)(_ptr)	\
+	 : (const _type *) CONST_PARENT_HELPER ((const char *)(_ptr),	\
+	  offsetof (const _type, _member)))
+
+#undef ABS
+#define ABS(n) (((n) < 0) ? -(n) : (n))
+
+#undef MIN
+#define MIN(x, y) (((x) < (y)) ? (x) : (y))
+
+#undef MAX
+#define MAX(x, y) (((x) > (y)) ? (x) : (y))
+
+#undef SWAP
+#define SWAP(x, y)							\
+do {									\
+	long _x = x;							\
+	x = y;								\
+	y = _x;								\
+} while (0)
+
+#undef SATURATE
+#define SATURATE(n, min, max) MIN (MAX (min, n), max)
+
+#endif /* !__GNUC__ */
+
+/* 32 bit constant byte reverse, e.g. 0xAABBCCDD -> 0xDDCCBBAA. */
+#define SWAB32(m)							\
+	(+ (((m) & 0xFF000000) >> 24)					\
+	 + (((m) & 0xFF0000) >> 8)					\
+	 + (((m) & 0xFF00) << 8)					\
+	 + (((m) & 0xFF) << 24))
+
+#ifdef HAVE_BUILTIN_POPCOUNT
+#  define popcnt(x) __builtin_popcount ((uint32_t)(x))
+#else
+#  define popcnt(x) _vbi_popcnt (x)
+#endif
+
+extern unsigned int
+_vbi_popcnt			(uint32_t		x);
+
+/* NB GCC inlines and optimizes these functions when size is const. */
+#define SET(var) memset (&(var), ~0, sizeof (var))
+
+#define CLEAR(var) memset (&(var), 0, sizeof (var))
+
+/* Useful to copy arrays, otherwise use assignment. */
+#define COPY(d, s)							\
+	(assert (sizeof (d) == sizeof (s)), memcpy (d, s, sizeof (d)))
+
+/* Copy string const into char array. */
+#define STRACPY(array, s)						\
+do {									\
+	/* Complain if s is no string const or won't fit. */		\
+	const char t_[sizeof (array) - 1] _vbi_unused = s;		\
+									\
+	memcpy (array, s, sizeof (s));					\
+} while (0)
+
+/* Copy bits through mask. */
+#define COPY_SET_MASK(dest, from, mask)					\
+	(dest ^= (from) ^ (dest & (mask)))
+
+/* Set bits if cond is TRUE, clear if FALSE. */
+#define COPY_SET_COND(dest, bits, cond)					\
+	 ((cond) ? (dest |= (bits)) : (dest &= ~(bits)))
+
+/* Set and clear bits. */
+#define COPY_SET_CLEAR(dest, set, clear)				\
+	(dest = (dest & ~(clear)) | (set))
+
+/* For applications, debugging and fault injection during unit tests. */
+
+#define vbi_malloc malloc
+#define vbi_realloc realloc
+#define vbi_strdup strdup
+#define vbi_free free
+
+#define vbi_cache_malloc vbi_malloc
+#define vbi_cache_free vbi_free
+
+/* Helper functions. */
+
+_vbi_inline int
+_vbi_to_ascii			(int			c)
+{
+	if (c < 0)
+		return '?';
+
+	c &= 0x7F;
+
+	if (c < 0x20 || c >= 0x7F)
+		return '.';
+
+	return c;
+}
+
+typedef struct {
+	const char *		key;
+	int			value;
+} _vbi_key_value_pair;
+
+extern vbi_bool
+_vbi_keyword_lookup		(int *			value,
+				 const char **		inout_s,
+				 const _vbi_key_value_pair * table,
+				 unsigned int		n_pairs)
+  _vbi_nonnull ((1, 2, 3));
+
+extern void
+_vbi_shrink_vector_capacity	(void **		vector,
+				 size_t *		capacity,
+				 size_t			min_capacity,
+				 size_t			element_size)
+  _vbi_nonnull ((1, 2));
+extern vbi_bool
+_vbi_grow_vector_capacity	(void **		vector,
+				 size_t *		capacity,
+				 size_t			min_capacity,
+				 size_t			element_size)
+  _vbi_nonnull ((1, 2));
+
+#define debug1 debug
+#define debug2 debug
+#define debug3 debug
+#define warning log
+#define error log
+#define info debug
+
+#define debug(a, fmt, ...)							\
+	ccx_common_logging.debug_ftn(CCX_DMT_PARSE, "VBI:%s:%d: "fmt , __FUNCTION__ ,__LINE__ , ##__VA_ARGS__)
+#define log(a, fmt, ...)							\
+	ccx_common_logging.log_ftn("VBI:%d: "fmt , __LINE__ , ##__VA_ARGS__)
+
+/* Portability stuff. */
+
+/* These should be defined in inttypes.h. */
+#ifndef PRId64
+#  define PRId64 "lld"
+#endif
+#ifndef PRIu64
+#  define PRIu64 "llu"
+#endif
+#ifndef PRIx64
+#  define PRIx64 "llx"
+#endif
+
+/* Should be defined in C99 limits.h? */
+#ifndef SIZE_MAX
+#  define SIZE_MAX ((size_t) -1)
+#endif
+
+#ifndef TIME_MIN
+#  define TIME_MIN (_vbi_time_min ())
+_vbi_inline time_t
+_vbi_time_min			(void)
+{
+	const time_t t = (time_t) -1.25;
+
+	if (t < -1) {
+		return (time_t)((sizeof (time_t) > 4) ? DBL_MIN : FLT_MIN);
+	} else if (t < 0) {
+		return ((uint64_t) 1) << (sizeof (time_t) * 8 - 1);
+	} else {
+		return 0;
+	}
+}
+#endif
+
+#ifndef TIME_MAX
+#  define TIME_MAX (_vbi_time_max ())
+_vbi_inline time_t
+_vbi_time_max			(void)
+{
+	const time_t t = (time_t) -1.25;
+
+	if (t < -1) {
+		return (time_t)((sizeof (time_t) > 4) ? DBL_MAX : FLT_MAX);
+	} else if (t < 0) {
+		/* Most likely signed 32 or 64 bit. */
+		return (((uint64_t) 1) << (sizeof (time_t) * 8 - 1)) - 1;
+	} else {
+		return -1;
+	}
+}
+#endif
+
+/* __va_copy is a GNU extension. */
+#ifndef __va_copy
+#  define __va_copy(ap1, ap2) do { ap1 = ap2; } while (0)
+#endif
+
+/* Use this instead of strncpy(). strlcpy() is a BSD extension. */
+#ifndef HAVE_STRLCPY
+#  define strlcpy _vbi_strlcpy
+#endif
+#undef strncpy
+#define strncpy use_strlcpy_instead
+
+extern size_t
+_vbi_strlcpy			(char *			dst,
+				 const char *		src,
+				 size_t			size)
+  _vbi_nonnull ((1, 2));
+
+/* strndup() is a BSD/GNU extension. */
+#ifndef HAVE_STRNDUP
+#  define strndup _vbi_strndup
+#endif
+
+extern char *
+_vbi_strndup			(const char *		s,
+				 size_t			len)
+  _vbi_nonnull ((1));
+
+/* vasprintf() is a GNU extension. */
+#ifndef HAVE_VASPRINTF
+#  define vasprintf _vbi_vasprintf
+#endif
+
+extern int
+_vbi_vasprintf			(char **		dstp,
+				 const char *		templ,
+				 va_list		ap)
+  _vbi_nonnull ((1, 2));
+
+/* asprintf() is a GNU extension. */
+#ifndef HAVE_ASPRINTF
+#  define asprintf _vbi_asprintf
+#endif
+
+extern int
+_vbi_asprintf			(char **		dstp,
+				 const char *		templ,
+				 ...)
+  _vbi_nonnull ((1, 2)) _vbi_format ((printf, 2, 3));
+
+#undef sprintf
+#define sprintf use_snprintf_or_asprintf_instead
+
+#endif /* MISC_H */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/raw_decoder.h

@@ -0,0 +1,212 @@
+/*
+ *  libzvbi -- Raw VBI decoder
+ *
+ *  Copyright (C) 2000-2004 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: raw_decoder.h,v 1.12 2008/02/19 00:35:21 mschimek Exp $ */
+
+#ifndef __ZVBI_RAW_DECODER_H__
+#define __ZVBI_RAW_DECODER_H__
+
+#include <stdio.h>
+
+#include "zvbi_decoder.h"
+#include "sampling_par.h"
+#include "bit_slicer.h"
+
+VBI_BEGIN_DECLS
+
+/*
+ * $ingroup RawDecoder
+ * $brief Raw VBI decoder.
+ *
+ * The contents of this structure are private.
+ * Call vbi3_raw_decoder_new() to allocate a raw VBI decoder.
+ */
+typedef struct _vbi3_raw_decoder vbi3_raw_decoder;
+
+/*
+ * $addtogroup RawDecoder
+ * ${
+ */
+extern vbi_bool
+vbi3_raw_decoder_sampling_point	(vbi3_raw_decoder *	rd,
+				 vbi3_bit_slicer_point *point,
+				 unsigned int		row,
+				 unsigned int		nth_bit);
+extern unsigned int
+vbi3_raw_decoder_decode		(vbi3_raw_decoder *	rd,
+				 vbi_sliced *		sliced,
+				 unsigned int		sliced_lines,
+				 const uint8_t *	raw);
+extern void
+vbi3_raw_decoder_reset		(vbi3_raw_decoder *	rd);
+extern vbi_service_set
+vbi3_raw_decoder_services	(vbi3_raw_decoder *	rd);
+extern vbi_service_set
+vbi3_raw_decoder_remove_services
+				(vbi3_raw_decoder *	rd,
+				 vbi_service_set	services);
+extern vbi_service_set
+vbi3_raw_decoder_add_services	(vbi3_raw_decoder *	rd,
+				 vbi_service_set	services,
+				 int			strict);
+extern vbi_bool
+vbi3_raw_decoder_debug		(vbi3_raw_decoder *	rd,
+				 vbi_bool		enable);
+extern vbi_service_set
+vbi3_raw_decoder_set_sampling_par
+				(vbi3_raw_decoder *	rd,
+				 const vbi_sampling_par *sp,
+				 int			strict);
+extern void
+vbi3_raw_decoder_get_sampling_par
+				(const vbi3_raw_decoder *rd,
+				 vbi_sampling_par *	sp);
+extern void
+vbi3_raw_decoder_set_log_fn	(vbi3_raw_decoder *	rd,
+				 vbi_log_fn *		log_fn,
+				 void *			user_data,
+				 vbi_log_mask		mask);
+extern void
+vbi3_raw_decoder_delete		(vbi3_raw_decoder *	rd);
+extern vbi3_raw_decoder *
+vbi3_raw_decoder_new		(const vbi_sampling_par *sp);
+
+/* $} */
+
+/* Private */
+
+/** @internal */
+#define _VBI3_RAW_DECODER_MAX_JOBS 8
+/** @internal */
+#define _VBI3_RAW_DECODER_MAX_WAYS 8
+
+/** @internal */
+typedef struct {
+	vbi_service_set		id;
+	vbi3_bit_slicer		slicer;
+} _vbi3_raw_decoder_job;
+
+/** @internal */
+typedef struct {
+	vbi3_bit_slicer_point   points[512];
+	unsigned int		n_points;
+} _vbi3_raw_decoder_sp_line;
+
+/**
+ * @internal
+ * Don't dereference pointers to this structure.
+ * I guarantee it will change.
+ */
+struct _vbi3_raw_decoder {
+	vbi_sampling_par	sampling;
+
+	vbi_service_set		services;
+
+	_vbi_log_hook		log;
+	vbi_bool		debug;
+
+	unsigned int		n_jobs;
+	unsigned int		n_sp_lines;
+	int			readjust;
+	int8_t *		pattern;	/* n scan lines * MAX_WAYS */
+	_vbi3_raw_decoder_job	jobs[_VBI3_RAW_DECODER_MAX_JOBS];
+	_vbi3_raw_decoder_sp_line *sp_lines;
+};
+
+/** @internal */
+typedef enum {
+	/** Requires field line numbers. */
+	_VBI_SP_LINE_NUM	= (1 << 0),
+	/** Requires field numbers. */
+	_VBI_SP_FIELD_NUM	= (1 << 1),
+} _vbi_service_par_flag;
+
+typedef struct _vbi_service_par _vbi_service_par;
+
+/** @internal */
+struct _vbi_service_par {
+	vbi_service_set		id;
+	const char *		label;
+
+	/**
+	 * Video standard
+	 * - 525 lines, FV = 59.94 Hz, FH = 15734 Hz
+	 * - 625 lines, FV = 50 Hz, FH = 15625 Hz
+	 */
+	vbi_videostd_set	videostd_set;
+
+	/**
+	 * Most scan lines used by the data service, first and last
+	 * line of first and second field. ITU-R numbering scheme.
+	 * Zero if no data from this field, requires field sync.
+	 */
+	unsigned int		first[2];
+        unsigned int		last[2];
+
+	/**
+	 * Leading edge hsync to leading edge first CRI one bit,
+	 * half amplitude points, in nanoseconds.
+	 */
+	unsigned int		offset;
+
+	unsigned int		cri_rate;	/**< Hz */
+	unsigned int		bit_rate;	/**< Hz */
+
+	/** Clock Run In and FRaming Code, LSB last txed bit of FRC. */
+	unsigned int		cri_frc;
+
+	/** CRI and FRC bits significant for identification. */
+	unsigned int		cri_frc_mask;
+
+	/**
+	 * Number of significat cri_bits (at cri_rate),
+	 * frc_bits (at bit_rate).
+	 */
+	unsigned int		cri_bits;
+	unsigned int		frc_bits;
+
+	unsigned int		payload;	/**< bits */
+	vbi_modulation		modulation;
+
+	_vbi_service_par_flag	flags;
+};
+
+extern const _vbi_service_par _vbi_service_table [];
+
+extern void
+_vbi3_raw_decoder_dump		(const vbi3_raw_decoder *rd,
+				 FILE *			fp);
+extern void
+_vbi3_raw_decoder_destroy	(vbi3_raw_decoder *	rd);
+extern vbi_bool
+_vbi3_raw_decoder_init		(vbi3_raw_decoder *	rd,
+				 const vbi_sampling_par *sp);
+
+VBI_END_DECLS
+
+#endif /* __ZVBI_RAW_DECODER_H__ */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/sampling_par.c

@@ -0,0 +1,589 @@
+/*
+ *  libzvbi -- Raw VBI sampling parameters
+ *
+ *  Copyright (C) 2000-2004 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: sampling_par.c,v 1.11 2009/02/18 15:37:11 mschimek Exp $ */
+
+#include <errno.h>
+
+#include "misc.h"
+#include "raw_decoder.h"
+#include "sampling_par.h"
+#include "sliced.h"
+#include "ccx_common_common.h" 
+
+#define vbi_pixfmt_bytes_per_pixel VBI_PIXFMT_BPP
+#define sp_sample_format sampling_format
+
+/**
+ * @addtogroup Sampling Raw VBI sampling
+ * @ingroup Raw
+ * @brief Raw VBI data sampling interface.
+ */
+
+/**
+ * @internal
+ * Compatibility.
+ */
+vbi_videostd_set
+_vbi_videostd_set_from_scanning
+				(int			scanning)
+{
+	switch (scanning) {
+	case 525:
+		return VBI_VIDEOSTD_SET_525_60;
+
+	case 625:
+		return VBI_VIDEOSTD_SET_625_50;
+
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+_vbi_inline vbi_bool
+range_check			(unsigned int		start,
+				 unsigned int		count,
+				 unsigned int		min,
+				 unsigned int		max)
+{
+	/* Check bounds and overflow. */
+	return (start >= min
+		&& (start + count) <= max
+		&& (start + count) >= start);
+}
+
+/**
+ * @internal
+ * @param sp Sampling parameters to verify.
+ * 
+ * @return
+ * CCX_TRUE if the sampling parameters are valid (as far as we can tell).
+ */
+vbi_bool
+_vbi_sampling_par_valid_log	(const vbi_sampling_par *sp,
+				 _vbi_log_hook *	log)
+{
+	vbi_videostd_set videostd_set;
+	unsigned int bpp;
+
+	assert (NULL != sp);
+
+	switch (sp->sp_sample_format) {
+	case VBI_PIXFMT_YUV420:
+		/* This conflicts with the ivtv driver, which returns an
+		   odd number of bytes per line.  The driver format is
+		   _GREY but libzvbi 0.2 has no VBI_PIXFMT_Y8. */
+		break;
+
+	default:
+		bpp = vbi_pixfmt_bytes_per_pixel (sp->sp_sample_format);
+		if (0 != (sp->bytes_per_line % bpp))
+			goto bad_samples;
+		break;
+	}
+
+	if (0 == sp->bytes_per_line)
+		goto no_samples;
+
+	if (0 == sp->count[0]
+	    && 0 == sp->count[1])
+		goto bad_range;
+
+	videostd_set = _vbi_videostd_set_from_scanning (sp->scanning);
+
+	if (VBI_VIDEOSTD_SET_525_60 & videostd_set) {
+		if (VBI_VIDEOSTD_SET_625_50 & videostd_set)
+			goto ambiguous;
+
+		if (0 != sp->start[0]
+		    && !range_check (sp->start[0], sp->count[0], 1, 262))
+			goto bad_range;
+
+		if (0 != sp->start[1]
+		    && !range_check (sp->start[1], sp->count[1], 263, 525))
+			goto bad_range;
+	} else if (VBI_VIDEOSTD_SET_625_50 & videostd_set) {
+		if (0 != sp->start[0]
+		    && !range_check (sp->start[0], sp->count[0], 1, 311))
+			goto bad_range;
+
+		if (0 != sp->start[1]
+		    && !range_check (sp->start[1], sp->count[1], 312, 625))
+			goto bad_range;
+	} else {
+	ambiguous:
+		info (log,
+		      "Ambiguous videostd_set 0x%lx.",
+		      (unsigned long) videostd_set);
+		return CCX_FALSE;
+	}
+
+	if (sp->interlaced
+	    && (sp->count[0] != sp->count[1]
+		|| 0 == sp->count[0])) {
+		info (log,
+			"Line counts %u, %u must be equal and "
+			"non-zero when raw VBI data is interlaced.",
+			sp->count[0], sp->count[1]);
+		return CCX_FALSE;
+	}
+
+	return CCX_TRUE;
+
+ no_samples:
+	info (log, "samples_per_line is zero.");
+	return CCX_FALSE;
+
+ bad_samples:
+	info (log,
+		"bytes_per_line value %u is no multiple of "
+		"the sample size %u.",
+		sp->bytes_per_line,
+		vbi_pixfmt_bytes_per_pixel (sp->sp_sample_format));
+	return CCX_FALSE;
+
+ bad_range:
+	info (log,
+		"Invalid VBI scan range %u-%u (%u lines), "
+		"%u-%u (%u lines).",
+		sp->start[0], sp->start[0] + sp->count[0] - 1,
+		sp->count[0],
+		sp->start[1], sp->start[1] + sp->count[1] - 1,
+		sp->count[1]);
+	return CCX_FALSE;
+}
+
+static vbi_bool
+_vbi_sampling_par_permit_service
+				(const vbi_sampling_par *sp,
+				 const _vbi_service_par *par,
+				 unsigned int		strict,
+				 _vbi_log_hook *	log)
+{
+	const unsigned int unknown = 0;
+	double signal;
+	unsigned int field;
+	unsigned int samples_per_line;
+	vbi_videostd_set videostd_set;
+
+	assert (NULL != sp);
+	assert (NULL != par);
+
+	videostd_set = _vbi_videostd_set_from_scanning (sp->scanning);
+	if (0 == (par->videostd_set & videostd_set)) {
+		info (log,
+		      "Service 0x%08x (%s) requires "
+		      "videostd_set 0x%lx, "
+		      "have 0x%lx.",
+		      par->id, par->label,
+		      (unsigned long) par->videostd_set,
+		      (unsigned long) videostd_set);
+		return CCX_FALSE;
+	}
+
+	if (par->flags & _VBI_SP_LINE_NUM) {
+                if ((par->first[0] > 0
+		     && unknown == (unsigned int) sp->start[0])
+                    || (par->first[1] > 0
+			&& unknown == (unsigned int) sp->start[1])) {
+			info (log,
+				"Service 0x%08x (%s) requires known "
+				"line numbers.",
+				par->id, par->label);
+			return CCX_FALSE;
+		}
+	}
+
+	{
+		unsigned int rate;
+
+		rate = MAX (par->cri_rate, par->bit_rate);
+
+		switch (par->id) {
+		case VBI_SLICED_WSS_625:
+			/* Effective bit rate is just 1/3 max_rate,
+			   so 1 * max_rate should suffice. */
+			break;
+
+		default:
+			rate = (rate * 3) >> 1;
+			break;
+		}
+
+		if (rate > (unsigned int) sp->sampling_rate) {
+			info (log,
+				"Sampling rate %f MHz too low "
+				"for service 0x%08x (%s).",
+				sp->sampling_rate / 1e6,
+				par->id, par->label);
+			return CCX_FALSE;
+		}
+	}
+
+	signal = par->cri_bits / (double) par->cri_rate
+		+ (par->frc_bits + par->payload) / (double) par->bit_rate;
+
+	samples_per_line = sp->bytes_per_line
+		/ VBI_PIXFMT_BPP (sp->sampling_format);
+
+	if (0 && sp->offset > 0 && strict > 0) {
+		double sampling_rate;
+		double offset;
+		double end;
+
+		sampling_rate = (double) sp->sampling_rate;
+
+		offset = sp->offset / sampling_rate;
+		end = (sp->offset + samples_per_line) / sampling_rate;
+
+		if (offset > (par->offset / 1e3 - 0.5e-6)) {
+			info (log,
+				"Sampling starts at 0H + %f us, too "
+				"late for service 0x%08x (%s) at "
+				"%f us.",
+				offset * 1e6,
+				par->id, par->label,
+				par->offset / 1e3);
+			return CCX_FALSE;
+		}
+
+		if (end < (par->offset / 1e9 + signal + 0.5e-6)) {
+			info (log,
+				"Sampling ends too early at 0H + "
+				"%f us for service 0x%08x (%s) "
+				"which ends at %f us",
+				end * 1e6,
+				par->id, par->label,
+				par->offset / 1e3
+				+ signal * 1e6 + 0.5);
+			return CCX_FALSE;
+		}
+	} else {
+		double samples;
+
+		samples = samples_per_line / (double) sp->sampling_rate;
+
+		if (strict > 0)
+			samples -= 1e-6; /* headroom */
+
+		if (samples < signal) {
+			info (log,
+				"Service 0x%08x (%s) signal length "
+				"%f us exceeds %f us sampling length.",
+				par->id, par->label,
+				signal * 1e6, samples * 1e6);
+			return CCX_FALSE;
+		}
+	}
+
+	if ((par->flags & _VBI_SP_FIELD_NUM)
+	    && !sp->synchronous) {
+		info (log,
+			"Service 0x%08x (%s) requires "
+			"synchronous field order.",
+			par->id, par->label);
+		return CCX_FALSE;
+	}
+
+	for (field = 0; field < 2; ++field) {
+		unsigned int start;
+		unsigned int end;
+
+		start = sp->start[field];
+		end = start + sp->count[field] - 1;
+
+		if (0 == par->first[field]
+		    || 0 == par->last[field]) {
+			/* No data on this field. */
+			continue;
+		}
+
+		if (0 == sp->count[field]) {
+			info (log,
+				"Service 0x%08x (%s) requires "
+				"data from field %u",
+				par->id, par->label, field + 1);
+			return CCX_FALSE;
+		}
+
+		/* (int) <= 0 for compatibility with libzvbi 0.2.x */
+		if ((int) strict <= 0 || 0 == sp->start[field])
+			continue;
+
+		if (1 == strict && par->first[field] > par->last[field]) {
+			/* May succeed if not all scanning lines
+			   available for the service are actually used. */
+			continue;
+		}
+		
+		if (start > par->first[field]
+		    || end < par->last[field]) {
+			info (log,
+				"Service 0x%08x (%s) requires "
+				"lines %u-%u, have %u-%u.",
+				par->id, par->label,
+				par->first[field],
+				par->last[field],
+				start, end);
+			return CCX_FALSE;
+		}
+	}
+
+	return CCX_TRUE;
+}
+
+/**
+ * @internal
+ */
+vbi_service_set
+_vbi_sampling_par_check_services_log
+				(const vbi_sampling_par *sp,
+				 vbi_service_set	services,
+				 unsigned int		strict,
+				 _vbi_log_hook *	log)
+{
+	const _vbi_service_par *par;
+	vbi_service_set rservices;
+
+	assert (NULL != sp);
+
+	rservices = 0;
+
+	for (par = _vbi_service_table; par->id; ++par) {
+		if (0 == (par->id & services))
+			continue;
+
+		if (_vbi_sampling_par_permit_service (sp, par, strict, log))
+			rservices |= par->id;
+	}
+
+	return rservices;
+}
+
+/**
+ * @internal
+ */
+vbi_service_set
+_vbi_sampling_par_from_services_log
+				(vbi_sampling_par *	sp,
+				 unsigned int *		max_rate,
+				 vbi_videostd_set	videostd_set_req,
+				 vbi_service_set	services,
+				 _vbi_log_hook *	log)
+{
+	const _vbi_service_par *par;
+	vbi_service_set rservices;
+	vbi_videostd_set videostd_set;
+	unsigned int rate;
+	unsigned int samples_per_line;
+
+	assert (NULL != sp);
+
+	videostd_set = 0;
+
+	if (0 != videostd_set_req) {
+		if (0 == (VBI_VIDEOSTD_SET_ALL & videostd_set_req)
+		    || ((VBI_VIDEOSTD_SET_525_60 & videostd_set_req)
+			&& (VBI_VIDEOSTD_SET_625_50 & videostd_set_req))) {
+			warning (log,
+				 "Ambiguous videostd_set 0x%lx.",
+				 (unsigned long) videostd_set_req);
+			CLEAR (*sp);
+			return 0;
+		}
+
+		videostd_set = videostd_set_req;
+	}
+
+	samples_per_line	= 0;
+	sp->sampling_rate	= 27000000;		/* ITU-R BT.601 */
+	sp->offset		= (int)(64e-6 * sp->sampling_rate);
+	sp->start[0]		= 30000;
+	sp->count[0]		= 0;
+	sp->start[1]		= 30000;
+	sp->count[1]		= 0;
+	sp->interlaced		= CCX_FALSE;
+	sp->synchronous		= CCX_TRUE;
+
+	rservices = 0;
+	rate = 0;
+
+	for (par = _vbi_service_table; par->id; ++par) {
+		double margin;
+		double signal;
+		int offset;
+		unsigned int samples;
+		unsigned int i;
+
+		if (0 == (par->id & services))
+			continue;
+
+		if (0 == videostd_set_req) {
+			vbi_videostd_set set;
+
+			set = par->videostd_set | videostd_set;
+
+			if (0 == (set & ~VBI_VIDEOSTD_SET_525_60)
+			    || 0 == (set & ~VBI_VIDEOSTD_SET_625_50))
+				videostd_set |= par->videostd_set;
+		}
+
+		if (VBI_VIDEOSTD_SET_525_60 & videostd_set)
+			margin = 1.0e-6;
+		else
+			margin = 2.0e-6;
+
+		if (0 == (par->videostd_set & videostd_set)) {
+			info (log,
+			      "Service 0x%08x (%s) requires "
+			      "videostd_set 0x%lx, "
+			      "have 0x%lx.",
+			      par->id, par->label,
+			      (unsigned long) par->videostd_set,
+			      (unsigned long) videostd_set);
+			continue;
+		}
+
+		rate = MAX (rate, par->cri_rate);
+		rate = MAX (rate, par->bit_rate);
+
+		signal = par->cri_bits / (double) par->cri_rate
+			+ ((par->frc_bits + par->payload) / (double) par->bit_rate);
+
+		offset = (int)((par->offset / 1e9) * sp->sampling_rate);
+		samples = (int)((signal + 1.0e-6) * sp->sampling_rate);
+
+		sp->offset = MIN (sp->offset, offset);
+
+		samples_per_line = MAX (samples_per_line + sp->offset,
+					samples + offset) - sp->offset;
+
+		for (i = 0; i < 2; ++i)
+			if (par->first[i] > 0
+			    && par->last[i] > 0) {
+				sp->start[i] = MIN
+					((unsigned int) sp->start[i],
+					 (unsigned int) par->first[i]);
+				sp->count[i] = MAX
+					((unsigned int) sp->start[i]
+					 + sp->count[i],
+					 (unsigned int) par->last[i] + 1)
+					- sp->start[i];
+			}
+
+		rservices |= par->id;
+	}
+
+	if (0 == rservices) {
+		CLEAR (*sp);
+		return 0;
+	}
+
+	if (0 == sp->count[1]) {
+		sp->start[1] = 0;
+
+		if (0 == sp->count[0]) {
+			sp->start[0] = 0;
+			sp->offset = 0;
+		}
+	} else if (0 == sp->count[0]) {
+		sp->start[0] = 0;
+	}
+
+	sp->scanning		= (videostd_set & VBI_VIDEOSTD_SET_525_60)
+		? 525 : 625;
+	sp->sp_sample_format	= VBI_PIXFMT_YUV420;
+
+	/* Note bpp is 1. */
+	sp->bytes_per_line = MAX (1440U, samples_per_line);
+
+	if (max_rate)
+		*max_rate = rate;
+
+	return rservices;
+}
+
+/**
+ * @param sp Sampling parameters to check against.
+ * @param services Set of data services.
+ * @param strict See description of vbi_raw_decoder_add_services().
+ *
+ * Check which of the given services can be decoded with the given
+ * sampling parameters at the given strictness level.
+ *
+ * @return
+ * Subset of @a services decodable with the given sampling parameters.
+ */
+vbi_service_set
+vbi_sampling_par_check_services
+				(const vbi_sampling_par *sp,
+				 vbi_service_set	services,
+				 unsigned int		strict)
+{
+	return _vbi_sampling_par_check_services_log (sp, services, strict,
+						      /* log_hook */ NULL);
+}
+
+/**
+ * @param sp Sampling parameters calculated by this function
+ *   will be stored here.
+ * @param max_rate If not NULL, the highest data bit rate in Hz of
+ *   all services requested will be stored here. The sampling rate
+ *   should be at least twice as high; @sp sampling_rate will
+ *   be set to a more reasonable value of 27 MHz, which is twice
+ *   the video sampling rate defined by ITU-R Rec. BT.601.
+ * @param videostd_set Create sampling parameters matching these
+ *   video standards. When 0 determine video standard from requested
+ *   services.
+ * @param services Set of VBI_SLICED_ symbols. Here (and only here) you
+ *   can add @c VBI_SLICED_VBI_625 or @c VBI_SLICED_VBI_525 to include all
+ *   vbi scan lines in the calculated sampling parameters.
+ *
+ * Calculate the sampling parameters required to receive and decode the
+ * requested data @a services. The @a sp sampling_format will be
+ * @c VBI_PIXFMT_Y8, offset and bytes_per_line will be set to
+ * reasonable minimums. This function can be used to initialize hardware
+ * prior to creating a vbi_raw_decoder object.
+ * 
+ * @return
+ * Subset of @a services covered by the calculated sampling parameters.
+ */
+vbi_service_set
+vbi_sampling_par_from_services	(vbi_sampling_par *	sp,
+				 unsigned int *		max_rate,
+				 vbi_videostd_set	videostd_set,
+				 vbi_service_set	services)
+{
+	return _vbi_sampling_par_from_services_log (sp, max_rate,
+						     videostd_set,
+						     services,
+						     /* log_hook */ NULL);
+}
+
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/sampling_par.h

@@ -0,0 +1,87 @@
+/*
+ *  libzvbi -- Raw VBI sampling parameters
+ *
+ *  Copyright (C) 2000-2004 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: sampling_par.h,v 1.9 2008/02/24 14:17:06 mschimek Exp $ */
+
+#ifndef __SAMPLING_PAR_H__
+#define __SAMPLING_PAR_H__
+
+#include "zvbi_decoder.h"
+
+VBI_BEGIN_DECLS
+
+/* Public */
+
+typedef vbi_raw_decoder vbi_sampling_par;
+
+#define VBI_VIDEOSTD_SET_EMPTY 0
+#define VBI_VIDEOSTD_SET_PAL_BG 1
+#define VBI_VIDEOSTD_SET_625_50 1
+#define VBI_VIDEOSTD_SET_525_60 2
+#define VBI_VIDEOSTD_SET_ALL 3
+typedef uint64_t vbi_videostd_set;
+
+/* Private */
+
+extern vbi_service_set
+vbi_sampling_par_from_services (vbi_sampling_par *    sp,
+				unsigned int *         max_rate,
+				vbi_videostd_set      videostd_set,
+				vbi_service_set       services);
+extern vbi_service_set
+vbi_sampling_par_check_services
+                                (const vbi_sampling_par *sp,
+                                 vbi_service_set       services,
+                                 unsigned int           strict)
+  _vbi_pure;
+
+extern vbi_videostd_set
+_vbi_videostd_set_from_scanning	(int			scanning);
+
+extern vbi_service_set
+_vbi_sampling_par_from_services_log
+                                (vbi_sampling_par *    sp,
+                                 unsigned int *         max_rate,
+                                 vbi_videostd_set      videostd_set,
+                                 vbi_service_set       services,
+                                 _vbi_log_hook *       log);
+extern vbi_service_set
+_vbi_sampling_par_check_services_log
+                                (const vbi_sampling_par *sp,
+                                 vbi_service_set       services,
+                                 unsigned int           strict,
+                                 _vbi_log_hook *       log)
+  _vbi_pure;
+extern vbi_bool
+_vbi_sampling_par_valid_log    (const vbi_sampling_par *sp,
+                                 _vbi_log_hook *       log)
+  _vbi_pure;
+
+VBI_END_DECLS
+
+#endif /* __SAMPLING_PAR_H__ */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/sliced.h

@@ -0,0 +1,363 @@
+/*
+ *  libzvbi -- Sliced VBI data
+ *
+ *  Copyright (C) 2000, 2001 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: sliced.h,v 1.11 2008/02/24 14:17:02 mschimek Exp $ */
+
+#ifndef SLICED_H
+#define SLICED_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Public */
+
+#include <inttypes.h>
+
+/**
+ * @addtogroup Sliced Sliced VBI data
+ * @ingroup Raw
+ * @brief Definition of sliced VBI data.
+ *
+ * The output of the libzvbi raw VBI decoder, and input to the data
+ * service decoder, is VBI data in binary format as defined in this
+ * section. It is similar to the output of hardware VBI decoders
+ * and VBI data transmitted in digital TV streams.
+ */
+
+/**
+ * @name Data service symbols
+ * @ingroup Sliced
+ * @{
+ */
+
+/**
+ * @anchor VBI_SLICED_
+ * No data service, blank vbi_sliced structure.
+ */
+#define VBI_SLICED_NONE			0
+
+/**
+ * Unknown data service (vbi_dvb_demux).
+ * @since 0.2.10
+ */
+#define VBI_SLICED_UNKNOWN              0
+
+/**
+ * Antiope a.k.a. Teletext System A
+ *
+ * Reference: <a href="http://www.itu.ch">ITU-R BT.653
+ * "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 37 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_ANTIOPE              0x00002000
+/**
+ * Synonym of VBI_SLICED_ANTIOPE.
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_A           0x00002000
+
+#define VBI_SLICED_TELETEXT_B_L10_625	0x00000001
+#define VBI_SLICED_TELETEXT_B_L25_625	0x00000002
+/**
+ * Teletext System B for 625 line systems
+ *
+ * Note this is separated into Level 1.0 and Level 2.5+ since the latter
+ * permits occupation of scan line 6 which is frequently out of
+ * range of raw VBI capture drivers. Clients should request decoding of both,
+ * may then verify Level 2.5 is covered. vbi_sliced id can be
+ * VBI_SLICED_TELETEXT_B, _B_L10_625 or _B_L25_625 regardless of line number.
+ *
+ * Reference: <a href="http://www.etsi.org">EN 300 706
+ * "Enhanced Teletext specification"</a>, <a href="http://www.itu.ch">
+ * ITU-R BT.653 "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 42 of the 45 byte Teletext packet, that is
+ * without clock run-in and framing code, lsb first transmitted.
+ */
+#define VBI_SLICED_TELETEXT_B		(VBI_SLICED_TELETEXT_B_L10_625 | \
+					 VBI_SLICED_TELETEXT_B_L25_625)
+/**
+ * Synonym of VBI_SLICED_TELETEXT_B.
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_B_625	VBI_SLICED_TELETEXT_B
+
+/**
+ * Teletext System C for 625 line systems
+ *
+ * Reference: <a href="http://www.itu.ch">ITU-R BT.653
+ * "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 33 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_C_625       0x00004000
+
+/**
+ * Teletext System D for 625 line systems
+ *
+ * Reference: <a href="http://www.itu.ch">ITU-R BT.653
+ * "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 34 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_D_625       0x00008000
+
+/**
+ * Video Program System
+ *
+ * Reference: <a href="http://www.etsi.org">ETS 300 231
+ * "Specification of the domestic video Programme
+ * Delivery Control system (PDC)"</a>, <a href="http://www.irt.de">
+ * IRT 8R2 "Video-Programm-System (VPS)"</a>.
+ *
+ * vbi_sliced payload: Byte number 3 to 15 according to ETS 300 231
+ * Figure 9, lsb first transmitted.
+ */
+#define VBI_SLICED_VPS                  0x00000004
+
+/**
+ * Pseudo-VPS signal transmitted on field 2
+ *
+ * vbi_sliced payload: 13 bytes.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_VPS_F2               0x00001000
+
+#define VBI_SLICED_CAPTION_625_F1       0x00000008
+#define VBI_SLICED_CAPTION_625_F2       0x00000010
+/**
+ * Closed Caption for 625 line systems
+ *
+ * Note this is split into field one and two services since for basic
+ * caption decoding only field one is required. vbi_sliced id can be
+ * VBI_SLICED_CAPTION_625, _625_F1 or _625_F2 regardless of line number.
+ *
+ * Reference: <a href="http://global.ihs.com">EIA 608
+ * "Recommended Practice for Line 21 Data Service"</a>.
+ *
+ * vbi_sliced payload: First and second byte including parity,
+ * lsb first transmitted.
+ */
+#define VBI_SLICED_CAPTION_625		(VBI_SLICED_CAPTION_625_F1 | \
+                                         VBI_SLICED_CAPTION_625_F2)
+
+/**
+ * Wide Screen Signalling for 625 line systems
+ *
+ * Reference: <a href="http://www.etsi.org">EN 300 294
+ * "625-line television Wide Screen Signalling (WSS)"</a>.
+ *
+ * vbi_sliced payload:
+ * <pre>
+ * Byte         0                  1
+ *       msb         lsb  msb             lsb
+ * bit   7 6 5 4 3 2 1 0  x x 13 12 11 10 9 8<br></pre>
+ * according to EN 300 294, Table 1, lsb first transmitted.
+ */
+#define VBI_SLICED_WSS_625              0x00000400
+
+#define VBI_SLICED_CAPTION_525_F1	0x00000020
+#define VBI_SLICED_CAPTION_525_F2	0x00000040
+/**
+ * Closed Caption for 525 line systems (NTSC).
+ *
+ * Note this is split into field one and two services since for basic
+ * caption decoding only field one is required. vbi_sliced id can be
+ * VBI_SLICED_CAPTION_525, _525_F1 or _525_F2 regardless of line number.
+ *
+ * VBI_SLICED_CAPTION_525 also covers XDS (Extended Data Service),
+ * V-Chip data and ITV / WebTV data.
+ *
+ * Reference: <a href="http://global.ihs.com">EIA 608
+ * "Recommended Practice for Line 21 Data Service"</a>.
+ *
+ * vbi_sliced payload: First and second byte including parity,
+ * lsb first transmitted.
+ */
+#define VBI_SLICED_CAPTION_525		(VBI_SLICED_CAPTION_525_F1 | \
+					 VBI_SLICED_CAPTION_525_F2)
+/**
+ * Closed Caption at double bit rate for 525 line systems.
+ *
+ * Reference: ?
+ *
+ * vbi_sliced payload: First to fourth byte including parity bit,
+ * lsb first transmitted.
+ */
+#define VBI_SLICED_2xCAPTION_525	0x00000080
+
+/**
+ * Teletext System B for 525 line systems
+ *
+ * Reference: <a href="http://www.itu.ch">ITU-R BT.653
+ * "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 34 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_B_525       0x00010000
+
+/**
+ * North American Basic Teletext Specification
+ * a.k.a. Teletext System C for 525 line systems
+ *
+ * Reference: <a href="http://global.ihs.com">EIA-516
+ * "North American Basic Teletext Specification (NABTS)"</a>,
+ * <a href="http://www.itu.ch">ITU-R BT.653 "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 33 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_NABTS                0x00000100
+
+/**
+ * Synonym of VBI_SLICED_NABTS.
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_C_525       0x00000100
+
+/**
+ * Misdefined.
+ *
+ * vbi_sliced payload: 34 bytes.
+ *
+ * @deprecated
+ * This service was misdefined.
+ * Use VBI_SLICED_TELETEXT_B_525 or VBI_SLICED_TELETEXT_D_525 in new code.
+ */
+#define VBI_SLICED_TELETEXT_BD_525	0x00000200
+
+/**
+ * Teletext System D for 525 line systems
+ *
+ * Reference: <a href="http://www.itu.ch">ITU-R BT.653
+ * "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 34 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_D_525       0x00020000
+
+/**
+ * Wide Screen Signalling for NTSC Japan
+ *
+ * Reference: <a href="http://www.jeita.or.jp">EIA-J CPR-1204</a>
+ *
+ * vbi_sliced payload:
+ * <pre>
+ * Byte         0                    1                  2
+ *       msb         lsb  msb               lsb  msb             lsb
+ * bit   7 6 5 4 3 2 1 0  15 14 13 12 11 10 9 8  x x x x 19 18 17 16
+ * </pre>
+ */
+
+#define VBI_SLICED_WSS_CPR1204		0x00000800
+
+/**
+ * No actual data service. This symbol is used to request capturing
+ * of all PAL/SECAM VBI data lines from the libzvbi driver interface,
+ * as opposed to just those lines used to transmit the requested
+ * data services.
+ */
+#define VBI_SLICED_VBI_625		0x20000000
+
+/**
+ * No actual data service. This symbol is used to request capturing
+ * of all NTSC VBI data lines from the libzvbi driver interface,
+ * as opposed to just those lines used to transmit the requested
+ * data services.
+ */
+#define VBI_SLICED_VBI_525		0x40000000
+
+/** @} */
+
+typedef unsigned int vbi_service_set;
+
+/**
+ * @ingroup Sliced
+ * @brief This structure holds one scan line of sliced vbi data.
+ *
+ * For example the contents of NTSC line 21, two bytes of Closed Caption
+ * data. Usually an array of vbi_sliced is used, covering all
+ * VBI lines of the two fields of a video frame.
+ */
+typedef struct {
+	/**
+	 * A @ref VBI_SLICED_ symbol identifying the data service. Under cirumstances
+	 * (see VBI_SLICED_TELETEXT_B) this can be a set of VBI_SLICED_ symbols.
+	 */
+	uint32_t		id;
+	/**
+	 * Source line number according to the ITU-R line numbering scheme,
+	 * a value of @c 0 if the exact line number is unknown. Note that some
+	 * data services cannot be reliable decoded without line number.
+	 *
+	 * @image html zvbi_625.gif "ITU-R PAL/SECAM line numbering scheme"
+	 * @image html zvbi_525.gif "ITU-R NTSC line numbering scheme"
+	 */
+	uint32_t		line;
+	/**
+	 * The actual payload. See the documentation of @ref VBI_SLICED_ symbols
+	 * for details.
+	 */
+	uint8_t			data[56];
+} vbi_sliced;
+
+/**
+ * @addtogroup Sliced
+ * @{
+ */
+extern const char *vbi_sliced_name (vbi_service_set service);
+extern unsigned int vbi_sliced_payload_bits (vbi_service_set service);
+/** @} */
+
+/* Private */
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* SLICED_H */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/zvbi_decoder.h

@@ -0,0 +1,404 @@
+/*
+ *  libzvbi -- Old raw VBI decoder
+ *
+ *  Copyright (C) 2000, 2001, 2002 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: decoder.h,v 1.11 2008/02/19 00:35:15 mschimek Exp $ */
+
+#ifndef DECODER_H
+#define DECODER_H
+
+#include "bcd.h"
+#include "sliced.h"
+
+
+/* Bit slicer */
+
+/**
+ * @ingroup Rawdec
+ * @brief Image format used as source to vbi_bit_slice() and vbi_raw_decode().
+ *
+ * @htmlonly
+<table border=1>
+<tr><th>Symbol</th><th>Byte&nbsp;0</th><th>Byte&nbsp;1</th><th>Byte&nbsp;2</th><th>Byte&nbsp;3</th></tr>
+<tr><td colspan=5>Planar YUV 4:2:0 data.</td></tr>
+<tr><td>VBI_PIXFMT_YUV420</td><td colspan=4>
+ <table>
+  <tr><th>Y plane</th><th>U plane</th><th>V plane</th></tr>
+  <tr><td><table border=1>
+   <tr><td>Y00</td><td>Y01</td><td>Y02</td><td>Y03</td></tr>
+   <tr><td>Y10</td><td>Y11</td><td>Y12</td><td>Y13</td></tr>
+   <tr><td>Y20</td><td>Y21</td><td>Y22</td><td>Y23</td></tr>
+   <tr><td>Y30</td><td>Y31</td><td>Y32</td><td>Y33</td></tr>
+  </table></td>
+  <td><table border=1>
+   <tr><td>Cb00</td><td>Cb01</td></tr>
+   <tr><td>Cb10</td><td>Cb11</td></tr>
+  </table></td>
+  <td><table border=1>
+   <tr><td>Cr00</td><td>Cr01</td></tr>
+   <tr><td>Cr10</td><td>Cr11</td></tr>
+  </table></td>
+ </tr></table></td>
+</tr>
+<tr><td colspan=5>Packed YUV 4:2:2 data.</td></tr>
+<tr><td>VBI_PIXFMT_YUYV</td><td>Y0</td><td>Cb</td><td>Y1</td><td>Cr</td></tr>
+<tr><td>VBI_PIXFMT_YVYU</td><td>Y0</td><td>Cr</td><td>Y1</td><td>Cb</td></tr>
+<tr><td>VBI_PIXFMT_UYVY</td><td>Cb</td><td>Y0</td><td>Cr</td><td>Y1</td></tr>
+<tr><td>VBI_PIXFMT_VYUY</td><td>Cr</td><td>Y0</td><td>Cb</td><td>Y1</td></tr>
+<tr><td colspan=5>Packed 32 bit RGB data.</td></tr>
+<tr><td>VBI_PIXFMT_RGBA32_LE VBI_PIXFMT_ARGB32_BE</td>
+<td>r7&nbsp;...&nbsp;r0</td><td>g7&nbsp;...&nbsp;g0</td>
+<td>b7&nbsp;...&nbsp;b0</td><td>a7&nbsp;...&nbsp;a0</td></tr>
+<tr><td>VBI_PIXFMT_BGRA32_LE VBI_PIXFMT_ARGB32_BE</td>
+<td>b7&nbsp;...&nbsp;b0</td><td>g7&nbsp;...&nbsp;g0</td>
+<td>r7&nbsp;...&nbsp;r0</td><td>a7&nbsp;...&nbsp;a0</td></tr>
+<tr><td>VBI_PIXFMT_ARGB32_LE VBI_PIXFMT_BGRA32_BE</td>
+<td>a7&nbsp;...&nbsp;a0</td><td>r7&nbsp;...&nbsp;r0</td>
+<td>g7&nbsp;...&nbsp;g0</td><td>b7&nbsp;...&nbsp;b0</td></tr>
+<tr><td>VBI_PIXFMT_ABGR32_LE VBI_PIXFMT_RGBA32_BE</td>
+<td>a7&nbsp;...&nbsp;a0</td><td>b7&nbsp;...&nbsp;b0</td>
+<td>g7&nbsp;...&nbsp;g0</td><td>r7&nbsp;...&nbsp;r0</td></tr>
+<tr><td colspan=5>Packed 24 bit RGB data.</td></tr>
+<tr><td>VBI_PIXFMT_RGBA24</td>
+<td>r7&nbsp;...&nbsp;r0</td><td>g7&nbsp;...&nbsp;g0</td>
+<td>b7&nbsp;...&nbsp;b0</td><td>&nbsp;</td></tr>
+<tr><td>VBI_PIXFMT_BGRA24</td>
+<td>b7&nbsp;...&nbsp;b0</td><td>g7&nbsp;...&nbsp;g0</td>
+<td>r7&nbsp;...&nbsp;r0</td><td>&nbsp;</td></tr>
+<tr><td colspan=5>Packed 16 bit RGB data.</td></tr>
+<tr><td>VBI_PIXFMT_RGB16_LE</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0</td>
+<td>b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g5&nbsp;g4&nbsp;g3</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_BGR16_LE</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0</td>
+<td>r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g5&nbsp;g4&nbsp;g3</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_RGB16_BE</td>
+<td>b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g5&nbsp;g4&nbsp;g3</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_BGR16_BE</td>
+<td>r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g5&nbsp;g4&nbsp;g3</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr>
+<tr><td colspan=5>Packed 15 bit RGB data.</td></tr>
+<tr><td>VBI_PIXFMT_RGBA15_LE</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0</td>
+<td>a0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g4&nbsp;g3</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_BGRA15_LE</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0</td>
+<td>a0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g4&nbsp;g3</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_ARGB15_LE</td>
+<td>g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;a0</td>
+<td>b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g4&nbsp;g3&nbsp;g2</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_ABGR15_LE</td>
+<td>g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;a0</td>
+<td>r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g4&nbsp;g3&nbsp;g2</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_RGBA15_BE</td>
+<td>a0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g4&nbsp;g3</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_BGRA15_BE</td>
+<td>a0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g4&nbsp;g3</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_ARGB15_BE</td>
+<td>b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g4&nbsp;g3&nbsp;g2</td>
+<td>g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;a0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_ABGR15_BE</td>
+<td>r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g4&nbsp;g3&nbsp;g2</td>
+<td>g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;a0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr>
+</table>
+@endhtmlonly */
+/* Attn: keep this in sync with rte, don't change order */
+typedef enum {
+	VBI_PIXFMT_YUV420 = 1,
+	VBI_PIXFMT_YUYV,
+	VBI_PIXFMT_YVYU,
+	VBI_PIXFMT_UYVY,
+	VBI_PIXFMT_VYUY,
+        VBI_PIXFMT_PAL8,
+	VBI_PIXFMT_RGBA32_LE = 32,
+	VBI_PIXFMT_RGBA32_BE,
+	VBI_PIXFMT_BGRA32_LE,
+	VBI_PIXFMT_BGRA32_BE,
+	VBI_PIXFMT_ABGR32_BE = 32, /* synonyms */
+	VBI_PIXFMT_ABGR32_LE,
+	VBI_PIXFMT_ARGB32_BE,
+	VBI_PIXFMT_ARGB32_LE,
+	VBI_PIXFMT_RGB24,
+	VBI_PIXFMT_BGR24,
+	VBI_PIXFMT_RGB16_LE,
+	VBI_PIXFMT_RGB16_BE,
+	VBI_PIXFMT_BGR16_LE,
+	VBI_PIXFMT_BGR16_BE,
+	VBI_PIXFMT_RGBA15_LE,
+	VBI_PIXFMT_RGBA15_BE,
+	VBI_PIXFMT_BGRA15_LE,
+	VBI_PIXFMT_BGRA15_BE,
+	VBI_PIXFMT_ARGB15_LE,
+	VBI_PIXFMT_ARGB15_BE,
+	VBI_PIXFMT_ABGR15_LE,
+	VBI_PIXFMT_ABGR15_BE
+} vbi_pixfmt;
+
+/* Private */
+
+typedef uint64_t vbi_pixfmt_set;
+
+#define VBI_MAX_PIXFMTS 64
+#define VBI_PIXFMT_SET(pixfmt) (((vbi_pixfmt_set) 1) << (pixfmt))
+#define VBI_PIXFMT_SET_YUV (VBI_PIXFMT_SET (VBI_PIXFMT_YUV420) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_YUYV) |		\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_YVYU) |		\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_UYVY) |		\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_VYUY))
+#define VBI_PIXFMT_SET_RGB (VBI_PIXFMT_SET (VBI_PIXFMT_RGBA32_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGBA32_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGRA32_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGRA32_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGB24) |		\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGR24) |		\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGB16_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGB16_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGR16_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGR16_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGBA15_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGBA15_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGRA15_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGRA15_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_ARGB15_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_ARGB15_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_ABGR15_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_ABGR15_BE))
+#define VBI_PIXFMT_SET_ALL (VBI_PIXFMT_SET_YUV |			\
+			    VBI_PIXFMT_SET_RGB)
+
+#define VBI_PIXFMT_BPP(fmt)						\
+	(((fmt) == VBI_PIXFMT_YUV420) ? 1 :				\
+	 (((fmt) >= VBI_PIXFMT_RGBA32_LE				\
+	   && (fmt) <= VBI_PIXFMT_BGRA32_BE) ? 4 :			\
+	  (((fmt) == VBI_PIXFMT_RGB24					\
+	    || (fmt) == VBI_PIXFMT_BGR24) ? 3 : 2)))
+
+/* Public */
+
+/**
+ * @ingroup Rawdec
+ * @brief Modulation used for VBI data transmission.
+ */
+typedef enum {
+	/**
+	 * The data is 'non-return to zero' coded, logical '1' bits
+	 * are described by high sample values, logical '0' bits by
+	 * low values. The data is last significant bit first transmitted.
+	 */
+	VBI_MODULATION_NRZ_LSB,
+	/**
+	 * 'Non-return to zero' coded, most significant bit first
+	 * transmitted.
+	 */
+	VBI_MODULATION_NRZ_MSB,
+	/**
+	 * The data is 'bi-phase' coded. Each data bit is described
+	 * by two complementary signalling elements, a logical '1'
+	 * by a sequence of '10' elements, a logical '0' by a '01'
+	 * sequence. The data is last significant bit first transmitted.
+	 */
+	VBI_MODULATION_BIPHASE_LSB,
+	/**
+	 * 'Bi-phase' coded, most significant bit first transmitted.
+	 */
+	VBI_MODULATION_BIPHASE_MSB
+} vbi_modulation;
+
+/**
+ * @ingroup Rawdec
+ * @brief Bit slicer context.
+ *
+ * The contents of this structure are private,
+ * use vbi_bit_slicer_init() to initialize.
+ */
+typedef struct vbi_bit_slicer {
+	int	(* func)(struct vbi_bit_slicer *slicer,
+				 uint8_t *raw, uint8_t *buf);
+	unsigned int	cri;
+	unsigned int	cri_mask;
+	int		thresh;
+	int		cri_bytes;
+	int		cri_rate;
+	int		oversampling_rate;
+	int		phase_shift;
+	int		step;
+	unsigned int	frc;
+	int		frc_bits;
+	int		payload;
+	int		endian;
+	int		skip;
+} vbi_bit_slicer;
+
+/**
+ * @addtogroup Rawdec
+ * @{
+ */
+extern void		vbi_bit_slicer_init(vbi_bit_slicer *slicer,
+					    int raw_samples, int sampling_rate,
+					    int cri_rate, int bit_rate,
+					    unsigned int cri_frc, unsigned int cri_mask,
+					    int cri_bits, int frc_bits, int payload,
+					    vbi_modulation modulation, vbi_pixfmt fmt);
+/**
+ * @param slicer Pointer to initialized vbi_bit_slicer object.
+ * @param raw Input data. At least the number of pixels or samples
+ *  given as @a raw_samples to vbi_bit_slicer_init().
+ * @param buf Output data. The buffer must be large enough to store
+ *   the number of bits given as @a payload to vbi_bit_slicer_init().
+ * 
+ * Decode one scan line of raw vbi data. Note the bit slicer tries
+ * to adapt to the average signal amplitude, you should avoid
+ * using the same vbi_bit_slicer object for data from different
+ * devices.
+ *
+ * @note As a matter of speed this function does not lock the
+ * @a slicer. When you want to share a vbi_bit_slicer object between
+ * multiple threads you must implement your own locking mechanism.
+ * 
+ * @return
+ * @c FALSE if the raw data does not contain the expected
+ * information, i. e. the CRI/FRC has not been found. This may also
+ * result from a too weak or noisy signal. Error correction must be
+ * implemented at a higher layer.
+ */
+inline int
+vbi_bit_slice(vbi_bit_slicer *slicer, uint8_t *raw, uint8_t *buf)
+{
+	return slicer->func(slicer, raw, buf);
+}
+/** @} */
+
+/**
+ * @ingroup Rawdec
+ * @brief Raw vbi decoder context.
+ *
+ * Only the sampling parameters are public. See
+ * vbi_raw_decoder_parameters() and vbi_raw_decoder_add_services()
+ * for usage.
+ */
+typedef struct vbi_raw_decoder {
+	/* Sampling parameters */
+
+	/**
+	 * Either 525 (M/NTSC, M/PAL) or 625 (PAL, SECAM), describing the
+	 * scan line system all line numbers refer to.
+	 */
+	int			scanning;
+	/**
+	 * Format of the raw vbi data.
+	 */
+	vbi_pixfmt		sampling_format;
+	/**
+	 * Sampling rate in Hz, the number of samples or pixels
+	 * captured per second.
+	 */
+	int			sampling_rate;		/* Hz */
+	/**
+	 * Number of samples or pixels captured per scan line,
+	 * in bytes. This determines the raw vbi image width and you
+	 * want it large enough to cover all data transmitted in the line (with
+	 * headroom).
+	 */
+	int			bytes_per_line;
+	/**
+	 * The distance from 0H (leading edge hsync, half amplitude point)
+	 * to the first sample (pixel) captured, in samples (pixels). You want
+	 * an offset small enough not to miss the start of the data
+	 * transmitted.
+	 */
+	int			offset;			/* 0H, samples */
+	/**
+	 * First scan line to be captured, first and second field
+	 * respectively, according to the ITU-R line numbering scheme
+	 * (see vbi_sliced). Set to zero if the exact line number isn't
+	 * known.
+	 */
+	int			start[2];		/* ITU-R numbering */
+	/**
+	 * Number of scan lines captured, first and second
+	 * field respectively. This can be zero if only data from one
+	 * field is required. The sum @a count[0] + @a count[1] determines the
+	 * raw vbi image height.
+	 */
+	int			count[2];		/* field lines */
+	/**
+	 * In the raw vbi image, normally all lines of the second
+	 * field are supposed to follow all lines of the first field. When
+	 * this flag is set, the scan lines of first and second field
+	 * will be interleaved in memory. This implies @a count[0] and @a count[1]
+	 * are equal.
+	 */
+	int		interlaced;
+	/**
+	 * Fields must be stored in temporal order, i. e. as the
+	 * lines have been captured. It is assumed that the first field is
+	 * also stored first in memory, however if the hardware cannot reliable
+	 * distinguish fields this flag shall be cleared, which disables
+	 * decoding of data services depending on the field number.
+	 */
+	int		synchronous;
+
+	unsigned int		services;
+	int			num_jobs;
+
+	int8_t *		pattern;
+	struct _vbi_raw_decoder_job {
+		unsigned int		id;
+		int			offset;
+		vbi_bit_slicer		slicer;
+	}			jobs[8];
+} vbi_raw_decoder;
+
+/**
+ * @addtogroup Rawdec
+ * @{
+ */
+extern void		vbi_raw_decoder_init(vbi_raw_decoder *rd);
+extern void		vbi_raw_decoder_reset(vbi_raw_decoder *rd);
+extern void		vbi_raw_decoder_destroy(vbi_raw_decoder *rd);
+extern unsigned int	vbi_raw_decoder_add_services(vbi_raw_decoder *rd,
+						     unsigned int services,
+						     int strict);
+extern unsigned int     vbi_raw_decoder_check_services(vbi_raw_decoder *rd,
+						     unsigned int services, int strict);
+extern unsigned int	vbi_raw_decoder_remove_services(vbi_raw_decoder *rd,
+							unsigned int services);
+extern void             vbi_raw_decoder_resize( vbi_raw_decoder *rd,
+						int * start, unsigned int * count );
+extern unsigned int	vbi_raw_decoder_parameters(vbi_raw_decoder *rd, unsigned int services,
+						   int scanning, int *max_rate);
+extern int		vbi_raw_decode(vbi_raw_decoder *rd, uint8_t *raw, vbi_sliced *out);
+/** @} */
+
+/* Private */
+
+#endif /* DECODER_H */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

windows/ccextractor.sln

@@ -8,13 +8,19 @@ EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Win32 = Debug|Win32
+		Debug-OCR|Win32 = Debug-OCR|Win32
 		Release|Win32 = Release|Win32
+		Release-OCR|Win32 = Release-OCR|Win32
 	EndGlobalSection
 	GlobalSection(ProjectConfigurationPlatforms) = postSolution
 		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Debug|Win32.ActiveCfg = Debug|Win32
 		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Debug|Win32.Build.0 = Debug|Win32
+		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Debug-OCR|Win32.ActiveCfg = Debug-OCR|Win32
+		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Debug-OCR|Win32.Build.0 = Debug-OCR|Win32
 		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Release|Win32.ActiveCfg = Release|Win32
 		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Release|Win32.Build.0 = Release|Win32
+		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Release-OCR|Win32.ActiveCfg = Release-OCR|Win32
+		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Release-OCR|Win32.Build.0 = Release-OCR|Win32
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE

windows/ccextractor.vcproj

@@ -1,744 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<VisualStudioProject
-	ProjectType="Visual C++"
-	Version="9.00"
-	Name="ccextractor"
-	ProjectGUID="{0F0063C4-BCBC-4379-A6D5-84A5669C940A}"
-	RootNamespace="ccextractor"
-	Keyword="Win32Proj"
-	TargetFrameworkVersion="0"
-	>
-	<Platforms>
-		<Platform
-			Name="Win32"
-		/>
-	</Platforms>
-	<ToolFiles>
-	</ToolFiles>
-	<Configurations>
-		<Configuration
-			Name="Debug|Win32"
-			OutputDirectory="Debug"
-			IntermediateDirectory="Debug"
-			ConfigurationType="1"
-			>
-			<Tool
-				Name="VCPreBuildEventTool"
-			/>
-			<Tool
-				Name="VCCustomBuildTool"
-			/>
-			<Tool
-				Name="VCXMLDataGeneratorTool"
-			/>
-			<Tool
-				Name="VCWebServiceProxyGeneratorTool"
-			/>
-			<Tool
-				Name="VCMIDLTool"
-			/>
-			<Tool
-				Name="VCCLCompilerTool"
-				Optimization="0"
-				AdditionalIncludeDirectories="../src/gpacmp4;../src/libpng;../src/zlib"
-				PreprocessorDefinitions="WIN32;_DEBUG;_CONSOLE;_FILE_OFFSET_BITS=64"
-				MinimalRebuild="true"
-				BasicRuntimeChecks="3"
-				RuntimeLibrary="3"
-				UsePrecompiledHeader="0"
-				WarningLevel="3"
-				Detect64BitPortabilityProblems="true"
-				DebugInformationFormat="4"
-			/>
-			<Tool
-				Name="VCManagedResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCPreLinkEventTool"
-			/>
-			<Tool
-				Name="VCLinkerTool"
-				AdditionalDependencies="WS2_32.Lib"
-				ShowProgress="0"
-				LinkIncremental="2"
-				AdditionalLibraryDirectories=""
-				GenerateDebugInformation="true"
-				SubSystem="1"
-				TargetMachine="1"
-			/>
-			<Tool
-				Name="VCALinkTool"
-			/>
-			<Tool
-				Name="VCManifestTool"
-			/>
-			<Tool
-				Name="VCXDCMakeTool"
-			/>
-			<Tool
-				Name="VCBscMakeTool"
-			/>
-			<Tool
-				Name="VCFxCopTool"
-			/>
-			<Tool
-				Name="VCAppVerifierTool"
-			/>
-			<Tool
-				Name="VCPostBuildEventTool"
-			/>
-		</Configuration>
-		<Configuration
-			Name="Release|Win32"
-			OutputDirectory="Release"
-			IntermediateDirectory="Release"
-			ConfigurationType="1"
-			>
-			<Tool
-				Name="VCPreBuildEventTool"
-			/>
-			<Tool
-				Name="VCCustomBuildTool"
-			/>
-			<Tool
-				Name="VCXMLDataGeneratorTool"
-			/>
-			<Tool
-				Name="VCWebServiceProxyGeneratorTool"
-			/>
-			<Tool
-				Name="VCMIDLTool"
-			/>
-			<Tool
-				Name="VCCLCompilerTool"
-				AdditionalIncludeDirectories="../src/gpacmp4;../src/libpng;../src/zlib"
-				PreprocessorDefinitions="WIN32;NDEBUG;_CONSOLE;_FILE_OFFSET_BITS=64"
-				RuntimeLibrary="2"
-				UsePrecompiledHeader="0"
-				WarningLevel="3"
-				Detect64BitPortabilityProblems="true"
-				DebugInformationFormat="3"
-			/>
-			<Tool
-				Name="VCManagedResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCPreLinkEventTool"
-			/>
-			<Tool
-				Name="VCLinkerTool"
-				AdditionalDependencies="WS2_32.Lib"
-				LinkIncremental="1"
-				AdditionalLibraryDirectories=""
-				GenerateDebugInformation="true"
-				SubSystem="1"
-				OptimizeReferences="2"
-				EnableCOMDATFolding="2"
-				TargetMachine="1"
-			/>
-			<Tool
-				Name="VCALinkTool"
-			/>
-			<Tool
-				Name="VCManifestTool"
-			/>
-			<Tool
-				Name="VCXDCMakeTool"
-			/>
-			<Tool
-				Name="VCBscMakeTool"
-			/>
-			<Tool
-				Name="VCFxCopTool"
-			/>
-			<Tool
-				Name="VCAppVerifierTool"
-			/>
-			<Tool
-				Name="VCPostBuildEventTool"
-			/>
-		</Configuration>
-	</Configurations>
-	<References>
-	</References>
-	<Files>
-		<Filter
-			Name="Header Files"
-			Filter="h;hpp;hxx;hm;inl;inc;xsd"
-			UniqueIdentifier="{93995380-89BD-4b04-88EB-625FBE52EBFB}"
-			>
-			<File
-				RelativePath="..\src\608.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_spupng.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\708.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\avilib.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\avparse.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\base_coding.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\bitstream.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\bitstream.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\ccextractor.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\configuration.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\constants.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\crc32.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\deflate.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\disable_warnings.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\gzguts.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\hamming.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\ietf.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inffast.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inffixed.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inflate.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inftrees.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\inttypes.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\isomedia.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\isomedia_dev.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\list.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\math.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\media_dev.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\media_tools.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\mpeg4_odf.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\network.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\odf_dev.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\odf_parse_common.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\ogg.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\png.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngconf.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\setup.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\stdint.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\stdintmsc.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\sync_layer.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\targetver.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\teletext.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\tools.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\trees.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\utf.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\version.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\zconf.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\zlib.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\zutil.h"
-				>
-			</File>
-		</Filter>
-		<Filter
-			Name="Resource Files"
-			Filter="rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx"
-			UniqueIdentifier="{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}"
-			>
-		</Filter>
-		<Filter
-			Name="Source Files"
-			Filter="cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx"
-			UniqueIdentifier="{4FC737F1-C7A5-4376-A066-2A32D752A2FF}"
-			>
-			<File
-				RelativePath="..\src\608.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_helpers.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_sami.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_smptett.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_spupng.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_srt.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\708.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\708_encoding.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\activity.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\adler32.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\asf_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\av_parsers.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\avc_ext.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\avc_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\avilib.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\base_encoding.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\bitstream.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_code_3gpp.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_code_apple.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_code_base.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_code_isma.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_code_meta.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_funcs.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\cc_bitstream.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\cc_decoders_common.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\ccextractor.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\constants.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\crc32.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\data_map.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\deflate.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\desc_private.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\descriptors.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\encoding.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\error.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\es_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\es_userdata.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\file_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\general_loop.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac_ogg.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\hinting.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inffast.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inflate.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inftrees.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\ipmpx_code.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\ipmpx_parse.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\isom_intern.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\isom_read.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\isom_store.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\isom_write.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\list.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\media.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\media_odf.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\meta.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\movie_fragments.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\mp4.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\myth.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\odf_code.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\odf_codec.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\odf_command.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\os_divers.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\output.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\params.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\params_dump.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\png.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngerror.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngmem.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngset.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngtrans.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngwio.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngwrite.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngwtran.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngwutil.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\qos.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\sample_descs.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\sequencing.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\slc.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\stbl_read.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\stbl_write.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\stream_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\telxcc.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\timing.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\track.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\trees.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\ts_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\tx3g.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\url.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\utf.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\utility.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\xds.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\zutil.c"
-				>
-			</File>
-		</Filter>
-	</Files>
-	<Globals>
-	</Globals>
-</VisualStudioProject>

windows/ccextractor.vcxproj

@@ -1,16 +1,25 @@
 <?xml version="1.0" encoding="utf-8"?>
 <Project DefaultTargets="Build" ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
   <ItemGroup Label="ProjectConfigurations">
+    <ProjectConfiguration Include="Debug-OCR|Win32">
+      <Configuration>Debug-OCR</Configuration>
+      <Platform>Win32</Platform>
+    </ProjectConfiguration>
     <ProjectConfiguration Include="Debug|Win32">
       <Configuration>Debug</Configuration>
       <Platform>Win32</Platform>
     </ProjectConfiguration>
+    <ProjectConfiguration Include="Release-OCR|Win32">
+      <Configuration>Release-OCR</Configuration>
+      <Platform>Win32</Platform>
+    </ProjectConfiguration>
     <ProjectConfiguration Include="Release|Win32">
       <Configuration>Release</Configuration>
       <Platform>Win32</Platform>
     </ProjectConfiguration>
   </ItemGroup>
   <ItemGroup>
+    <ClInclude Include="..\src\lib_ccx\compile_info.h" />
     <ClInclude Include="..\src\gpacmp4\gpac\avparse.h" />
     <ClInclude Include="..\src\gpacmp4\gpac\base_coding.h" />
     <ClInclude Include="..\src\gpacmp4\gpac\bitstream.h" />
@@ -66,6 +75,9 @@
     <ClInclude Include="..\src\lib_ccx\spupng_encoder.h" />
     <ClInclude Include="..\src\lib_ccx\teletext.h" />
     <ClInclude Include="..\src\lib_ccx\utility.h" />
+    <ClInclude Include="..\src\lib_hash\sha2.h" />
+    <ClInclude Include="..\src\win_iconv\iconv.h" />
+    <ClInclude Include="..\src\win_spec_incld\dirent.h" />
     <ClInclude Include="..\src\zlib\crc32.h" />
     <ClInclude Include="..\src\zlib\deflate.h" />
     <ClInclude Include="..\src\zlib\gzguts.h" />
@@ -77,8 +89,16 @@
     <ClInclude Include="..\src\zlib\zconf.h" />
     <ClInclude Include="..\src\zlib\zlib.h" />
     <ClInclude Include="..\src\zlib\zutil.h" />
-    <ClInclude Include="include\inttypes.h" />
-    <ClInclude Include="include\stdint.h" />
+    <ClInclude Include="..\src\zvbi\bcd.h" />
+    <ClInclude Include="..\src\zvbi\bit_slicer.h" />
+    <ClInclude Include="..\src\zvbi\macros.h" />
+    <ClInclude Include="..\src\zvbi\misc.h" />
+    <ClInclude Include="..\src\zvbi\raw_decoder.h" />
+    <ClInclude Include="..\src\zvbi\sampling_par.h" />
+    <ClInclude Include="..\src\zvbi\sliced.h" />
+    <ClInclude Include="..\src\zvbi\zvbi_decoder.h" />
+    <ClInclude Include="..\src\win_spec_incld\inttypes.h" />
+    <ClInclude Include="..\src\win_spec_incld\stdint.h" />
   </ItemGroup>
   <ItemGroup>
     <ClCompile Include="..\src\ccextractor.c" />
@@ -153,10 +173,12 @@
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_708_output.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_common.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_isdb.c" />
+    <ClCompile Include="..\src\lib_ccx\ccx_decoders_vbi.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_xds.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_demuxer.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_dtvcc.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_common.c" />
+    <ClCompile Include="..\src\lib_ccx\ccx_encoders_g608.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_helpers.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_sami.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_smptett.c" />
@@ -164,6 +186,7 @@
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_srt.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_webvtt.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_xds.c" />
+    <ClCompile Include="..\src\lib_ccx\ccx_gxf.c" />
     <ClCompile Include="..\src\lib_ccx\cc_bitstream.c" />
     <ClCompile Include="..\src\lib_ccx\configuration.c" />
     <ClCompile Include="..\src\lib_ccx\dvb_subtitle_decoder.c" />
@@ -188,6 +211,7 @@
     <ClCompile Include="..\src\lib_ccx\ts_tables_epg.c" />
     <ClCompile Include="..\src\lib_ccx\utility.c" />
     <ClCompile Include="..\src\lib_ccx\wtv_functions.c" />
+    <ClCompile Include="..\src\lib_hash\sha2.c" />
     <ClCompile Include="..\src\win_iconv\win_iconv.c" />
     <ClCompile Include="..\src\zlib\adler32.c" />
     <ClCompile Include="..\src\zlib\crc32.c" />
@@ -197,6 +221,10 @@
     <ClCompile Include="..\src\zlib\inftrees.c" />
     <ClCompile Include="..\src\zlib\trees.c" />
     <ClCompile Include="..\src\zlib\zutil.c" />
+    <ClCompile Include="..\src\zvbi\bit_slicer.c" />
+    <ClCompile Include="..\src\zvbi\decoder.c" />
+    <ClCompile Include="..\src\zvbi\raw_decoder.c" />
+    <ClCompile Include="..\src\zvbi\sampling_par.c" />
   </ItemGroup>
   <PropertyGroup Label="Globals">
     <ProjectGuid>{0F0063C4-BCBC-4379-A6D5-84A5669C940A}</ProjectGuid>
@@ -206,12 +234,20 @@
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
     <ConfigurationType>Application</ConfigurationType>
-    <PlatformToolset>v140</PlatformToolset>
+    <PlatformToolset>v120</PlatformToolset>
   </PropertyGroup>
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
     <ConfigurationType>Application</ConfigurationType>
     <PlatformToolset>v120</PlatformToolset>
   </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug-OCR|Win32'" Label="Configuration">
+    <ConfigurationType>Application</ConfigurationType>
+    <PlatformToolset>v120</PlatformToolset>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release-OCR|Win32'" Label="Configuration">
+    <ConfigurationType>Application</ConfigurationType>
+    <PlatformToolset>v120</PlatformToolset>
+  </PropertyGroup>
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
   <ImportGroup Label="ExtensionSettings">
   </ImportGroup>
@@ -221,6 +257,12 @@
   <ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="PropertySheets">
     <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
   </ImportGroup>
+  <ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug-OCR|Win32'" Label="PropertySheets">
+    <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
+  </ImportGroup>
+  <ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release-OCR|Win32'" Label="PropertySheets">
+    <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
+  </ImportGroup>
   <PropertyGroup Label="UserMacros" />
   <PropertyGroup>
     <_ProjectFileVersion>12.0.21005.1</_ProjectFileVersion>
@@ -229,16 +271,34 @@
     <OutDir>Debug\</OutDir>
     <IntDir>Debug\</IntDir>
     <LinkIncremental>true</LinkIncremental>
+    <TargetName>ccextractorwin</TargetName>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug-OCR|Win32'">
+    <OutDir>Debug-OCR\</OutDir>
+    <IntDir>Debug-OCR\</IntDir>
+    <LinkIncremental>true</LinkIncremental>
+    <TargetName>ccextractorwin</TargetName>
+    <IncludePath>$(ProjectDir)\libs\include;$(IncludePath)</IncludePath>
+    <LibraryPath>$(ProjectDir)\libs\lib;$(VC_LibraryPath_x86);$(WindowsSDK_LibraryPath_x86)</LibraryPath>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release-OCR|Win32'">
+    <OutDir>Release-OCR\</OutDir>
+    <IntDir>Release-OCR\</IntDir>
+    <LinkIncremental>false</LinkIncremental>
+    <TargetName>ccextractorwin</TargetName>
+    <IncludePath>$(ProjectDir)\libs\include;$(IncludePath)</IncludePath>
+    <LibraryPath>$(ProjectDir)\libs\lib;$(VC_LibraryPath_x86);$(WindowsSDK_LibraryPath_x86)</LibraryPath>
   </PropertyGroup>
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
     <OutDir>Release\</OutDir>
     <IntDir>Release\</IntDir>
     <LinkIncremental>false</LinkIncremental>
+    <TargetName>ccextractorwin</TargetName>
   </PropertyGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
     <ClCompile>
       <Optimization>Disabled</Optimization>
-      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/gpacmp4;../src/libpng;../src/zlib;../src;../src/lib_ccx;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/gpacmp4;../src/libpng;../src/zlib;../src;../src/lib_ccx;../src/zvbi;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
       <PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;_FILE_OFFSET_BITS=64;%(PreprocessorDefinitions)</PreprocessorDefinitions>
       <MinimalRebuild>true</MinimalRebuild>
       <BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
@@ -255,10 +315,58 @@
       <TargetMachine>MachineX86</TargetMachine>
     </Link>
   </ItemDefinitionGroup>
+  <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug-OCR|Win32'">
+    <ClCompile>
+      <Optimization>Disabled</Optimization>
+      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/gpacmp4;../src/libpng;../src/zlib;../src;../src/lib_ccx;../src/zvbi;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <PreprocessorDefinitions>ENABLE_OCR;WIN32;_DEBUG;_CONSOLE;_FILE_OFFSET_BITS=64;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <MinimalRebuild>true</MinimalRebuild>
+      <BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
+      <RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
+      <PrecompiledHeader>
+      </PrecompiledHeader>
+      <WarningLevel>Level3</WarningLevel>
+      <DebugInformationFormat>EditAndContinue</DebugInformationFormat>
+    </ClCompile>
+    <Link>
+      <AdditionalDependencies>WS2_32.Lib;liblept172.lib;libtesseract304d.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <ShowProgress>NotSet</ShowProgress>
+      <GenerateDebugInformation>true</GenerateDebugInformation>
+      <SubSystem>Console</SubSystem>
+      <TargetMachine>MachineX86</TargetMachine>
+    </Link>
+    <PostBuildEvent>
+      <Command>xcopy /y $(ProjectDir)libs\lib\liblept172.dll $(ProjectDir)$(OutDir)
+xcopy /y $(ProjectDir)libs\lib\libtesseract304d.dll $(ProjectDir)$(OutDir)</Command>
+    </PostBuildEvent>
+  </ItemDefinitionGroup>
+  <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release-OCR|Win32'">
+    <ClCompile>
+      <Optimization>Disabled</Optimization>
+      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/gpacmp4;../src/libpng;../src/zlib;../src;../src/lib_ccx;../src/zvbi;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <PreprocessorDefinitions>VERSION_FILE_PRESENT;ENABLE_OCR;WIN32;NDEBUG;_CONSOLE;_FILE_OFFSET_BITS=64;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
+      <PrecompiledHeader />
+      <WarningLevel>Level3</WarningLevel>
+      <DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
+    </ClCompile>
+    <Link>
+      <AdditionalDependencies>WS2_32.Lib;libtesseract302.lib;liblept168.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <GenerateDebugInformation>true</GenerateDebugInformation>
+      <SubSystem>Console</SubSystem>
+      <OptimizeReferences>true</OptimizeReferences>
+      <EnableCOMDATFolding>true</EnableCOMDATFolding>
+      <TargetMachine>MachineX86</TargetMachine>
+    </Link>
+    <PostBuildEvent>
+      <Command>xcopy /y $(ProjectDir)libs\leptonica\lib\liblept168.dll $(ProjectDir)$(OutDir)
+xcopy /y $(ProjectDir)libs\tesseract\lib\libtesseract302.dll $(ProjectDir)$(OutDir)</Command>
+    </PostBuildEvent>
+  </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
     <ClCompile>
-      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/lib_ccx;../src/gpacmp4;../src/libpng;../src/zlib;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
-      <PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;_FILE_OFFSET_BITS=64;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/lib_ccx;../src/lib_hash;../src/gpacmp4;../src/libpng;../src/zlib;../src/zvbi;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <PreprocessorDefinitions>VERSION_FILE_PRESENT;WIN32;NDEBUG;_CONSOLE;_FILE_OFFSET_BITS=64;%(PreprocessorDefinitions)</PreprocessorDefinitions>
       <RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary>
       <PrecompiledHeader />
       <WarningLevel>Level3</WarningLevel>
@@ -272,6 +380,9 @@
       <EnableCOMDATFolding>true</EnableCOMDATFolding>
       <TargetMachine>MachineX86</TargetMachine>
     </Link>
+    <PreBuildEvent>
+      <Command>pre-build.bat</Command>
+    </PreBuildEvent>
   </ItemDefinitionGroup>
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
   <ImportGroup Label="ExtensionTargets">

windows/ccextractor.vcxproj.filters

@@ -49,6 +49,27 @@
     <Filter Include="Source Files\ccx_common">
       <UniqueIdentifier>{2b841054-ad06-429d-9b44-fabd29fbeef0}</UniqueIdentifier>
     </Filter>
+    <Filter Include="Source Files\zvbi">
+      <UniqueIdentifier>{d86f4f94-df28-4fed-a455-c54bc521c86a}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Header Files\zvbi">
+      <UniqueIdentifier>{288f48c3-470a-45ad-a70a-8f062c51aeb1}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Header Files\lib_hash">
+      <UniqueIdentifier>{140e6ccb-2042-4ecc-9cba-42a04a5b0803}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Source Files\lib_hash">
+      <UniqueIdentifier>{dcb7e1ec-51eb-4f69-93db-b10133e98402}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Header Files\win_spec_incld">
+      <UniqueIdentifier>{de4fb954-0fd9-407a-8803-4dec4640296b}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Header Files\win_iconv">
+      <UniqueIdentifier>{1870287a-d318-4ef0-9bd1-11c965a1474c}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Source Files\win_iconv">
+      <UniqueIdentifier>{964b0a18-918d-4b08-ba92-5a84e3c48b43}</UniqueIdentifier>
+    </Filter>
   </ItemGroup>
   <ItemGroup>
     <ClInclude Include="..\src\gpacmp4\gpac\avparse.h">
@@ -177,12 +198,6 @@
     <ClInclude Include="..\src\lib_ccx\disable_warnings.h">
       <Filter>Header Files</Filter>
     </ClInclude>
-    <ClInclude Include="include\inttypes.h">
-      <Filter>Header Files</Filter>
-    </ClInclude>
-    <ClInclude Include="include\stdint.h">
-      <Filter>Header Files</Filter>
-    </ClInclude>
     <ClInclude Include="..\src\lib_ccx\ccx_common_char_encoding.h">
       <Filter>Header Files\ccx_common</Filter>
     </ClInclude>
@@ -255,6 +270,48 @@
     <ClInclude Include="..\src\lib_ccx\utility.h">
       <Filter>Header Files</Filter>
     </ClInclude>
+    <ClInclude Include="..\src\zvbi\bcd.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\bit_slicer.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\macros.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\misc.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\raw_decoder.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\sampling_par.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\sliced.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\zvbi_decoder.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\lib_ccx\compile_info.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\lib_hash\sha2.h">
+      <Filter>Header Files\lib_hash</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\win_spec_incld\stdint.h">
+      <Filter>Header Files\win_spec_incld</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\win_spec_incld\inttypes.h">
+      <Filter>Header Files\win_spec_incld</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\win_spec_incld\dirent.h">
+      <Filter>Header Files\win_spec_incld</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\win_iconv\iconv.h">
+      <Filter>Header Files\win_iconv</Filter>
+    </ClInclude>
   </ItemGroup>
   <ItemGroup>
     <ClCompile Include="..\src\ccextractor.c">
@@ -581,9 +638,6 @@
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_708_output.c">
       <Filter>Source Files\ccx_decoders</Filter>
     </ClCompile>
-    <ClCompile Include="..\src\win_iconv\win_iconv.c">
-      <Filter>Source Files</Filter>
-    </ClCompile>
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_708_encoding.c">
       <Filter>Source Files</Filter>
     </ClCompile>
@@ -617,5 +671,32 @@
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_isdb.c">
       <Filter>Source Files\ccx_decoders</Filter>
     </ClCompile>
+    <ClCompile Include="..\src\zvbi\bit_slicer.c">
+      <Filter>Source Files\zvbi</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\zvbi\decoder.c">
+      <Filter>Source Files\zvbi</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\zvbi\raw_decoder.c">
+      <Filter>Source Files\zvbi</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\zvbi\sampling_par.c">
+      <Filter>Source Files\zvbi</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\lib_ccx\ccx_gxf.c">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\lib_ccx\ccx_decoders_vbi.c">
+      <Filter>Source Files\ccx_decoders</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\lib_ccx\ccx_encoders_g608.c">
+      <Filter>Source Files\ccx_decoders</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\lib_hash\sha2.c">
+      <Filter>Source Files\lib_hash</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\win_iconv\win_iconv.c">
+      <Filter>Source Files\win_iconv</Filter>
+    </ClCompile>
   </ItemGroup>
 </Project>

windows/libs/include/alltypes.h

@@ -0,0 +1,64 @@
+/*====================================================================*
+ -  Copyright (C) 2001 Leptonica.  All rights reserved.
+ -
+ -  Redistribution and use in source and binary forms, with or without
+ -  modification, are permitted provided that the following conditions
+ -  are met:
+ -  1. Redistributions of source code must retain the above copyright
+ -     notice, this list of conditions and the following disclaimer.
+ -  2. Redistributions in binary form must reproduce the above
+ -     copyright notice, this list of conditions and the following
+ -     disclaimer in the documentation and/or other materials
+ -     provided with the distribution.
+ -
+ -  THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ -  ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ -  LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ -  A PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL ANY
+ -  CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+ -  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+ -  PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+ -  PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
+ -  OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+ -  NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+ -  SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *====================================================================*/
+
+#ifndef  LEPTONICA_ALLTYPES_H
+#define  LEPTONICA_ALLTYPES_H
+
+    /* Standard */
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+
+    /* General and configuration defs */
+#include "environ.h"
+
+    /* Generic and non-image-specific containers */
+#include "array.h"
+#include "bbuffer.h"
+#include "heap.h"
+#include "list.h"
+#include "ptra.h"
+#include "queue.h"
+#include "stack.h"
+
+    /* Imaging */
+#include "arrayaccess.h"
+#include "bmf.h"
+#include "ccbord.h"
+#include "dewarp.h"
+#include "gplot.h"
+#include "imageio.h"
+#include "jbclass.h"
+#include "morph.h"
+#include "pix.h"
+#include "recog.h"
+#include "regutils.h"
+#include "stringcode.h"
+#include "sudoku.h"
+#include "watershed.h"
+
+
+#endif /* LEPTONICA_ALLTYPES_H */