Updates CHANGES.TXT

- Add hack to ignore table_id=0xc0 (Program Information) in PMT without ignoring the full PMT.

.gitignore

@@ -9,8 +9,9 @@ CVS
 *.o
 linux/ccextractor
 linux/depend
-windows/debug/**
-windows/release/**
+windows/Debug/**
+windows/Release/**
+build/
 
 ####
 # Visual Studio project Ignored files
@@ -19,3 +20,14 @@ windows/release/**
 *.sdf
 *.opensdf
 *.user
+*.opendb
+
+####
+# Ignore the header file that is updated upon build
+src/lib_ccx/compile_info.h
+
+#### Ignore windows OCR libraries and folders
+windows/libs/leptonica/**
+windows/libs/tesseract/**
+windows/Release-OCR/**
+windows/Debug-OCR/**

README.md

@@ -4,7 +4,7 @@ ccextractor
 Carlos' version (mainstream) is the most stable branch.
 
 Extracting subtitles has never been so easy. Just type the following command:
-ccextrator "name of input"
+ccextractor "name of input"
 
 Gui lovers should download the Sorceforge version of CCExtractor, the Git Version is not your cup of tea.
 http://ccextractor.sourceforge.net/download-ccextractor.html

docs/CHANGES.TXT

@@ -1,3 +1,42 @@
+0.81 (2016-06-13)
+-----------------
+- New: --version parameter for extensive version information (version number, compile date, executable hash, git commit (if appropriate))
+- New: Add -sem (semaphore) to create a .sem file when an output file is open and delete it when it's closed.
+- New: Add --append parameter. This will prevent overwriting of exisiting files.
+- New: File Rotation support added. The user has to send a USR1 signal to rotate.
+- Fix: Issues with files <1 Mb
+- Fix: Preview of generated transcript.
+- Fix: Statistics were not generated anymore.
+- Fix: Correcting display of sub mode and info in transcripts.
+- Fix: Teletext page number displayed in -UCLA.
+- Fix: Removal of excessive XDS notices about aspect ratio info.
+- Fix: Force Flushing of file buffers works for all files now.
+- Fix: mp4 void atoms that was causing some .mp4 files to fail.
+- Fix: Memory usage caused by EPG processing was high due to many non-dynamic buffers.
+- Fix: Project files for Visual Studio now include OCR support in Windows.
+
+0.80 (2016-04-24)
+-----------------
+- Fix: "Premature end of file" (one of the scenarios)
+- Fix: XDS data is always parsed again (needed to extract information such as program name)
+- Fix: Teletext parsing: @ was incorrectly exported as * - X/26 packet specifications in ETS 300 706 v1.2.1 now better followed
+- Fix: Teletext parsing: Latin G2 subsets and accented characters not displaying properly
+- Fix: Timing in -ucla
+- Fix: Timing in ISDB (some instances)
+- Fix: "mfra" mp4 box weight changed to 1 (this helps with correct file format detection)
+- Fix: Fix for TARGET File is null. 
+- Fix: Fixed SegFaults while parsing parameters (if mandatory parameter is not present in -outinterval, -codec or -nocodec)
+- Fix: Crash when input small is too small
+- Fix: Update some URLs in code (references to docs)
+- Fix: -delay now updates final timestamp in ISDB, too
+- Fix: Removed minor compiler warnings
+- Fix: Visual Studio solution files working again
+- Fix: ffmpeg integration working again
+- New: Added --forceflush (-ff). If used, output file descriptors will be flushed immediately after being written to
+- New: Hexdump XDS packets that we cannot parse (shouldn't be many of those anyway)
+- New: If input file cannot be open, provide a decent human readable explanation
+- New: GXF support
+
 0.79 (2016-01-09)
 -----------------
 - Support for Grid Format (g608)

docs/README.TXT

@@ -1,4 +1,4 @@
-ccextractor, 0.79
+ccextractor, 0.81
 -----------------
 Authors: Carlos Fernández (cfsmp3), Volker Quetschke.
 Maintainer: cfsmp3
@@ -14,12 +14,12 @@ You can subscribe to new releases notifications at freshmeat:
 http://freshmeat.net/projects/ccextractor
 
 Google Summer of Code 2014 students
-- Willem van iseghem
+- Willem Van Iseghem
 - Ruslan KuchumoV
 - Anshul Maheshwari
 
 Google Summer of Code 2015 students
-- Willem van iseghem
+- Willem Van Iseghem
 - Ruslan Kuchumov
 - Anshul Maheshwari
 - Nurendra Choudhary

docs/gpl-2.0.txt

@@ -0,0 +1,339 @@
+                    GNU GENERAL PUBLIC LICENSE
+                       Version 2, June 1991
+
+ Copyright (C) 1989, 1991 Free Software Foundation, Inc.,
+ 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ Everyone is permitted to copy and distribute verbatim copies
+ of this license document, but changing it is not allowed.
+
+                            Preamble
+
+  The licenses for most software are designed to take away your
+freedom to share and change it.  By contrast, the GNU General Public
+License is intended to guarantee your freedom to share and change free
+software--to make sure the software is free for all its users.  This
+General Public License applies to most of the Free Software
+Foundation's software and to any other program whose authors commit to
+using it.  (Some other Free Software Foundation software is covered by
+the GNU Lesser General Public License instead.)  You can apply it to
+your programs, too.
+
+  When we speak of free software, we are referring to freedom, not
+price.  Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+this service if you wish), that you receive source code or can get it
+if you want it, that you can change the software or use pieces of it
+in new free programs; and that you know you can do these things.
+
+  To protect your rights, we need to make restrictions that forbid
+anyone to deny you these rights or to ask you to surrender the rights.
+These restrictions translate to certain responsibilities for you if you
+distribute copies of the software, or if you modify it.
+
+  For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must give the recipients all the rights that
+you have.  You must make sure that they, too, receive or can get the
+source code.  And you must show them these terms so they know their
+rights.
+
+  We protect your rights with two steps: (1) copyright the software, and
+(2) offer you this license which gives you legal permission to copy,
+distribute and/or modify the software.
+
+  Also, for each author's protection and ours, we want to make certain
+that everyone understands that there is no warranty for this free
+software.  If the software is modified by someone else and passed on, we
+want its recipients to know that what they have is not the original, so
+that any problems introduced by others will not reflect on the original
+authors' reputations.
+
+  Finally, any free program is threatened constantly by software
+patents.  We wish to avoid the danger that redistributors of a free
+program will individually obtain patent licenses, in effect making the
+program proprietary.  To prevent this, we have made it clear that any
+patent must be licensed for everyone's free use or not licensed at all.
+
+  The precise terms and conditions for copying, distribution and
+modification follow.
+
+                    GNU GENERAL PUBLIC LICENSE
+   TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
+
+  0. This License applies to any program or other work which contains
+a notice placed by the copyright holder saying it may be distributed
+under the terms of this General Public License.  The "Program", below,
+refers to any such program or work, and a "work based on the Program"
+means either the Program or any derivative work under copyright law:
+that is to say, a work containing the Program or a portion of it,
+either verbatim or with modifications and/or translated into another
+language.  (Hereinafter, translation is included without limitation in
+the term "modification".)  Each licensee is addressed as "you".
+
+Activities other than copying, distribution and modification are not
+covered by this License; they are outside its scope.  The act of
+running the Program is not restricted, and the output from the Program
+is covered only if its contents constitute a work based on the
+Program (independent of having been made by running the Program).
+Whether that is true depends on what the Program does.
+
+  1. You may copy and distribute verbatim copies of the Program's
+source code as you receive it, in any medium, provided that you
+conspicuously and appropriately publish on each copy an appropriate
+copyright notice and disclaimer of warranty; keep intact all the
+notices that refer to this License and to the absence of any warranty;
+and give any other recipients of the Program a copy of this License
+along with the Program.
+
+You may charge a fee for the physical act of transferring a copy, and
+you may at your option offer warranty protection in exchange for a fee.
+
+  2. You may modify your copy or copies of the Program or any portion
+of it, thus forming a work based on the Program, and copy and
+distribute such modifications or work under the terms of Section 1
+above, provided that you also meet all of these conditions:
+
+    a) You must cause the modified files to carry prominent notices
+    stating that you changed the files and the date of any change.
+
+    b) You must cause any work that you distribute or publish, that in
+    whole or in part contains or is derived from the Program or any
+    part thereof, to be licensed as a whole at no charge to all third
+    parties under the terms of this License.
+
+    c) If the modified program normally reads commands interactively
+    when run, you must cause it, when started running for such
+    interactive use in the most ordinary way, to print or display an
+    announcement including an appropriate copyright notice and a
+    notice that there is no warranty (or else, saying that you provide
+    a warranty) and that users may redistribute the program under
+    these conditions, and telling the user how to view a copy of this
+    License.  (Exception: if the Program itself is interactive but
+    does not normally print such an announcement, your work based on
+    the Program is not required to print an announcement.)
+
+These requirements apply to the modified work as a whole.  If
+identifiable sections of that work are not derived from the Program,
+and can be reasonably considered independent and separate works in
+themselves, then this License, and its terms, do not apply to those
+sections when you distribute them as separate works.  But when you
+distribute the same sections as part of a whole which is a work based
+on the Program, the distribution of the whole must be on the terms of
+this License, whose permissions for other licensees extend to the
+entire whole, and thus to each and every part regardless of who wrote it.
+
+Thus, it is not the intent of this section to claim rights or contest
+your rights to work written entirely by you; rather, the intent is to
+exercise the right to control the distribution of derivative or
+collective works based on the Program.
+
+In addition, mere aggregation of another work not based on the Program
+with the Program (or with a work based on the Program) on a volume of
+a storage or distribution medium does not bring the other work under
+the scope of this License.
+
+  3. You may copy and distribute the Program (or a work based on it,
+under Section 2) in object code or executable form under the terms of
+Sections 1 and 2 above provided that you also do one of the following:
+
+    a) Accompany it with the complete corresponding machine-readable
+    source code, which must be distributed under the terms of Sections
+    1 and 2 above on a medium customarily used for software interchange; or,
+
+    b) Accompany it with a written offer, valid for at least three
+    years, to give any third party, for a charge no more than your
+    cost of physically performing source distribution, a complete
+    machine-readable copy of the corresponding source code, to be
+    distributed under the terms of Sections 1 and 2 above on a medium
+    customarily used for software interchange; or,
+
+    c) Accompany it with the information you received as to the offer
+    to distribute corresponding source code.  (This alternative is
+    allowed only for noncommercial distribution and only if you
+    received the program in object code or executable form with such
+    an offer, in accord with Subsection b above.)
+
+The source code for a work means the preferred form of the work for
+making modifications to it.  For an executable work, complete source
+code means all the source code for all modules it contains, plus any
+associated interface definition files, plus the scripts used to
+control compilation and installation of the executable.  However, as a
+special exception, the source code distributed need not include
+anything that is normally distributed (in either source or binary
+form) with the major components (compiler, kernel, and so on) of the
+operating system on which the executable runs, unless that component
+itself accompanies the executable.
+
+If distribution of executable or object code is made by offering
+access to copy from a designated place, then offering equivalent
+access to copy the source code from the same place counts as
+distribution of the source code, even though third parties are not
+compelled to copy the source along with the object code.
+
+  4. You may not copy, modify, sublicense, or distribute the Program
+except as expressly provided under this License.  Any attempt
+otherwise to copy, modify, sublicense or distribute the Program is
+void, and will automatically terminate your rights under this License.
+However, parties who have received copies, or rights, from you under
+this License will not have their licenses terminated so long as such
+parties remain in full compliance.
+
+  5. You are not required to accept this License, since you have not
+signed it.  However, nothing else grants you permission to modify or
+distribute the Program or its derivative works.  These actions are
+prohibited by law if you do not accept this License.  Therefore, by
+modifying or distributing the Program (or any work based on the
+Program), you indicate your acceptance of this License to do so, and
+all its terms and conditions for copying, distributing or modifying
+the Program or works based on it.
+
+  6. Each time you redistribute the Program (or any work based on the
+Program), the recipient automatically receives a license from the
+original licensor to copy, distribute or modify the Program subject to
+these terms and conditions.  You may not impose any further
+restrictions on the recipients' exercise of the rights granted herein.
+You are not responsible for enforcing compliance by third parties to
+this License.
+
+  7. If, as a consequence of a court judgment or allegation of patent
+infringement or for any other reason (not limited to patent issues),
+conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License.  If you cannot
+distribute so as to satisfy simultaneously your obligations under this
+License and any other pertinent obligations, then as a consequence you
+may not distribute the Program at all.  For example, if a patent
+license would not permit royalty-free redistribution of the Program by
+all those who receive copies directly or indirectly through you, then
+the only way you could satisfy both it and this License would be to
+refrain entirely from distribution of the Program.
+
+If any portion of this section is held invalid or unenforceable under
+any particular circumstance, the balance of the section is intended to
+apply and the section as a whole is intended to apply in other
+circumstances.
+
+It is not the purpose of this section to induce you to infringe any
+patents or other property right claims or to contest validity of any
+such claims; this section has the sole purpose of protecting the
+integrity of the free software distribution system, which is
+implemented by public license practices.  Many people have made
+generous contributions to the wide range of software distributed
+through that system in reliance on consistent application of that
+system; it is up to the author/donor to decide if he or she is willing
+to distribute software through any other system and a licensee cannot
+impose that choice.
+
+This section is intended to make thoroughly clear what is believed to
+be a consequence of the rest of this License.
+
+  8. If the distribution and/or use of the Program is restricted in
+certain countries either by patents or by copyrighted interfaces, the
+original copyright holder who places the Program under this License
+may add an explicit geographical distribution limitation excluding
+those countries, so that distribution is permitted only in or among
+countries not thus excluded.  In such case, this License incorporates
+the limitation as if written in the body of this License.
+
+  9. The Free Software Foundation may publish revised and/or new versions
+of the General Public License from time to time.  Such new versions will
+be similar in spirit to the present version, but may differ in detail to
+address new problems or concerns.
+
+Each version is given a distinguishing version number.  If the Program
+specifies a version number of this License which applies to it and "any
+later version", you have the option of following the terms and conditions
+either of that version or of any later version published by the Free
+Software Foundation.  If the Program does not specify a version number of
+this License, you may choose any version ever published by the Free Software
+Foundation.
+
+  10. If you wish to incorporate parts of the Program into other free
+programs whose distribution conditions are different, write to the author
+to ask for permission.  For software which is copyrighted by the Free
+Software Foundation, write to the Free Software Foundation; we sometimes
+make exceptions for this.  Our decision will be guided by the two goals
+of preserving the free status of all derivatives of our free software and
+of promoting the sharing and reuse of software generally.
+
+                            NO WARRANTY
+
+  11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY
+FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW.  EXCEPT WHEN
+OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES
+PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED
+OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
+MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.  THE ENTIRE RISK AS
+TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU.  SHOULD THE
+PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING,
+REPAIR OR CORRECTION.
+
+  12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR
+REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING
+OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED
+TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY
+YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER
+PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGES.
+
+                     END OF TERMS AND CONDITIONS
+
+            How to Apply These Terms to Your New Programs
+
+  If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these terms.
+
+  To do so, attach the following notices to the program.  It is safest
+to attach them to the start of each source file to most effectively
+convey the exclusion of warranty; and each file should have at least
+the "copyright" line and a pointer to where the full notice is found.
+
+    <one line to give the program's name and a brief idea of what it does.>
+    Copyright (C) <year>  <name of author>
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 2 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License along
+    with this program; if not, write to the Free Software Foundation, Inc.,
+    51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program is interactive, make it output a short notice like this
+when it starts in an interactive mode:
+
+    Gnomovision version 69, Copyright (C) year name of author
+    Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+    This is free software, and you are welcome to redistribute it
+    under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate
+parts of the General Public License.  Of course, the commands you use may
+be called something other than `show w' and `show c'; they could even be
+mouse-clicks or menu items--whatever suits your program.
+
+You should also get your employer (if you work as a programmer) or your
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary.  Here is a sample; alter the names:
+
+  Yoyodyne, Inc., hereby disclaims all copyright interest in the program
+  `Gnomovision' (which makes passes at compilers) written by James Hacker.
+
+  <signature of Ty Coon>, 1 April 1989
+  Ty Coon, President of Vice
+
+This General Public License does not permit incorporating your program into
+proprietary programs.  If your program is a subroutine library, you may
+consider it more useful to permit linking proprietary applications with the
+library.  If this is what you want to do, use the GNU Lesser General
+Public License instead of this License.

linux/Makefile

@@ -3,7 +3,8 @@ SHELL = /bin/sh
 CC = gcc
 SYS := $(shell gcc -dumpmachine)
 CFLAGS = -O3 -std=gnu99
-INCLUDE = -I../src/gpacmp4/ -I../src/libpng -I../src/zlib -I../src/lib_ccx -I../src/.
+INCLUDE = -I../src/gpacmp4/ -I../src/libpng -I../src/lib_hash -I../src/zlib -I../src/lib_ccx -I../src/.
+INCLUDE += -I../src/zvbi
 ALL_FLAGS = -Wno-write-strings -D_FILE_OFFSET_BITS=64
 LDFLAGS = -lm
 
@@ -13,7 +14,7 @@ endif
 TARGET = ccextractor
 
 OBJS_DIR = objs
-VPATH = ../src:../src/gpacmp4:../src/libpng:../src/zlib:../src/lib_ccx
+VPATH = ../src:../src/gpacmp4:../src/libpng:../src/zlib:../src/lib_ccx:../src/zvbi:../src/lib_hash
 
 SRCS_DIR = ../src
 SRCS_C = $(wildcard $(SRCS_DIR)/*.c)
@@ -27,6 +28,10 @@ SRCS_PNG_DIR = $(SRCS_DIR)/libpng
 SRCS_PNG = $(wildcard $(SRCS_PNG_DIR)/*.c)
 OBJS_PNG = $(SRCS_PNG:$(SRCS_PNG_DIR)/%.c=$(OBJS_DIR)/%.o)
 
+SRCS_ZVBI_DIR = $(SRCS_DIR)/zvbi
+SRCS_ZVBI = $(wildcard $(SRCS_ZVBI_DIR)/*.c)
+OBJS_ZVBI = $(SRCS_ZVBI:$(SRCS_ZVBI_DIR)/%.c=$(OBJS_DIR)/%.o)
+
 SRCS_GPACMP4_DIR = $(SRCS_DIR)/gpacmp4
 SRCS_GPACMP4_C = $(wildcard $(SRCS_GPACMP4_DIR)/*.c)
 SRCS_GPACMP4_CPP = $(wildcard $(SRCS_GPACMP4_DIR)/*.cpp)
@@ -37,6 +42,10 @@ SRCS_ZLIB_DIR = $(SRCS_DIR)/zlib
 SRCS_ZLIB = $(wildcard $(SRCS_ZLIB_DIR)/*.c)
 OBJS_ZLIB = $(SRCS_ZLIB:$(SRCS_ZLIB_DIR)/%.c=$(OBJS_DIR)/%.o)
 
+SRCS_HASH_DIR = $(SRCS_DIR)/lib_hash
+SRCS_HASH = $(wildcard $(SRCS_HASH_DIR)/*.c)
+OBJS_HASH = $(SRCS_HASH:$(SRCS_HASH_DIR)/%.c=$(OBJS_DIR)/%.o)
+
 INSTLALL = cp -f -p
 INSTLALL_PROGRAM = $(INSTLALL)
 DESTDIR = /usr/bin
@@ -78,14 +87,14 @@ LDFLAGS+= $(shell pkg-config --libs libavutil )
 endif
 
 .PHONY: all
-all: objs_dir $(TARGET) 
+all: pre-build objs_dir $(TARGET) 
 
 .PHONY: objs_dir
 objs_dir: 
 	mkdir -p $(OBJS_DIR)
 
-$(TARGET): $(OBJS) $(OBJS_PNG) $(OBJS_GPACMP4) $(OBJS_ZLIB) $(OBJS_CCX)
-	$(CC) $(ALL_FLAGS) $(CFLAGS) $(OBJS) $(OBJS_CCX) $(OBJS_PNG) $(OBJS_GPACMP4) $(OBJS_ZLIB) $(LDFLAGS) -o $@
+$(TARGET): $(OBJS) $(OBJS_PNG) $(OBJS_GPACMP4) $(OBJS_ZVBI) $(OBJS_ZLIB) $(OBJS_HASH) $(OBJS_CCX)
+	$(CC) $(ALL_FLAGS) $(CFLAGS) $(OBJS) $(OBJS_CCX) $(OBJS_PNG) $(OBJS_ZVBI) $(OBJS_GPACMP4) $(OBJS_ZLIB) $(OBJS_HASH) $(LDFLAGS) -o $@
 
 $(OBJS_DIR)/%.o: %.c
 	$(CC) -c $(ALL_FLAGS) $(INCLUDE) $(CFLAGS) $< -o $@  
@@ -99,7 +108,7 @@ $(OBJS_DIR)/ccextractor.o: ccextractor.c
 .PHONY: clean
 clean:
 	rm -rf $(TARGET) 2>/dev/null || true
-	rm -rf $(OBJS_CCX) $(OBJS_PNG) $(OBJS_ZLIB) $(OBJS_GPACMP4) $(OBJS) 2>/dev/null || true
+	rm -rf $(OBJS_CCX) $(OBJS_PNG) $(OBJS_ZLIB) $(OBJS_GPACMP4) $(OBJS_HASH) $(OBJS) 2>/dev/null || true
 	rm -rdf $(OBJS_DIR) 2>/dev/null || true
 	rm -rf .depend 2>/dev/null || true
 
@@ -113,8 +122,12 @@ uninstall:
 
 .PHONY: depend dep
 depend dep:
-	$(CC) $(CFLAGS) $(INCLUDE) -E -MM $(SRCS_C) $(SRCS_PNG) $(SRCS_ZLIB) $(SRCS_CCX) \
+	$(CC) $(CFLAGS) $(INCLUDE) -E -MM $(SRCS_C) $(SRCS_PNG) $(SRCS_ZVBI) $(SRCS_ZLIB) $(SRCS_HASH) $(SRCS_CCX) \
 		$(SRCS_GPACMP4_C) $(SRCS_GPACMP4_CPP) |\
 		sed 's/^[a-zA-Z_0-9]*.o/$(OBJS_DIR)\/&/' > .depend
 
+.PHONY: pre-build
+pre-build:
+	./pre-build.sh
+
 -include .depend

linux/build

@@ -1,11 +1,14 @@
 #!/bin/bash
 BLD_FLAGS="-std=gnu99 -Wno-write-strings -DGPAC_CONFIG_LINUX -D_FILE_OFFSET_BITS=64"
-BLD_INCLUDE="-I../src/lib_ccx/ -I../src/gpacmp4/ -I../src/libpng/ -I../src/zlib/"
+BLD_INCLUDE="-I../src/lib_ccx/ -I../src/gpacmp4/ -I../src/libpng/ -I../src/zlib/ -I../src/zvbi -I../src/lib_hash"
 SRC_LIBPNG="$(find ../src/libpng/ -name '*.c')"
 SRC_ZLIB="$(find ../src/zlib/ -name '*.c')"
+SRC_ZVBI="$(find ../src/zvbi/ -name '*.c')"
 SRC_CCX="$(find ../src/lib_ccx/ -name '*.c')"
 SRC_GPAC="$(find ../src/gpacmp4/ -name '*.c')"
-BLD_SOURCES="../src/ccextractor.c $SRC_CCX $SRC_GPAC $SRC_ZLIB $SRC_LIBPNG"
+SRC_HASH="$(find ../src/lib_hash/ -name '*.c')"
+BLD_SOURCES="../src/ccextractor.c $SRC_CCX $SRC_GPAC $SRC_ZLIB $SRC_ZVBI $SRC_LIBPNG $SRC_HASH"
 BLD_LINKER="-lm -zmuldefs"
 
-gcc $BLD_FLAGS $BLD_INCLUDE -o ccextractor $BLD_SOURCES $BLD_LINKER
+./pre-build.sh
+gcc $BLD_FLAGS $BLD_INCLUDE -o ccextractor $BLD_SOURCES $BLD_LINKER

linux/builddebug

@@ -1,11 +1,14 @@
 #!/bin/bash
 BLD_FLAGS="-g -std=gnu99 -Wno-write-strings -DGPAC_CONFIG_LINUX -D_FILE_OFFSET_BITS=64"
-BLD_INCLUDE="-I../src/lib_ccx/ -I../src/gpacmp4/ -I../src/libpng/ -I../src/zlib/"
+BLD_INCLUDE="-I../src/lib_ccx/ -I../src/gpacmp4/ -I../src/libpng/ -I../src/zlib/ -I../src/zvbi -I../src/lib_hash"
 SRC_LIBPNG="$(find ../src/libpng/ -name '*.c')"
 SRC_ZLIB="$(find ../src/zlib/ -name '*.c')"
+SRC_ZVBI="$(find ../src/zvbi/ -name '*.c')"
 SRC_CCX="$(find ../src/lib_ccx/ -name '*.c')"
 SRC_GPAC="$(find ../src/gpacmp4/ -name '*.c')"
-BLD_SOURCES="../src/ccextractor.c $SRC_CCX $SRC_GPAC $SRC_ZLIB $SRC_LIBPNG"
+SRC_HASH="$(find ../src/lib_hash/ -name '*.c')"
+BLD_SOURCES="../src/ccextractor.c $SRC_CCX $SRC_GPAC $SRC_ZLIB  $SRC_ZVBI $SRC_LIBPNG $SRC_HASH"
 BLD_LINKER="-lm -zmuldefs"
 
-gcc $BLD_FLAGS $BLD_INCLUDE -o ccextractor $BLD_SOURCES $BLD_LINKER
+./pre-build.sh
+gcc $BLD_FLAGS $BLD_INCLUDE -o ccextractor $BLD_SOURCES $BLD_LINKER

linux/pre-build.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+echo "Obtaining Git commit"
+commit=(`git rev-parse HEAD 2>/dev/null`)
+if [ -z "$commit" ]; then
+	echo "Git command not present, trying folder approach"
+	if [ -d "../.git" ]; then 
+		echo "Git folder found, using HEAD file"
+		head="`cat ../.git/HEAD`"
+		ref_pos=(`expr match "$head" 'ref: '`)
+		if [ $ref_pos -eq 5 ]; then	
+			echo "HEAD file contains a ref, following"
+			commit=(`cat "../.git/${head:5}"`)
+			echo "Extracted commit: $commit"
+		else
+			echo "HEAD contains a commit, using it"
+			commit="$head"
+			echo "Extracted commit: $commit"	
+		fi 
+	fi
+fi
+if [ -z "$commit" ]; then
+	commit="Unknown"
+fi
+builddate=`date +%Y-%m-%d`
+echo "Storing variables in file"
+echo "Commit: $commit"
+echo "Date: $builddate"
+echo "#ifndef CCX_CCEXTRACTOR_COMPILE_H" > ../src/lib_ccx/compile_info.h
+echo "#define CCX_CCEXTRACTOR_COMPILE_H" >> ../src/lib_ccx/compile_info.h
+echo "#define GIT_COMMIT \"$commit\"" >> ../src/lib_ccx/compile_info.h
+echo "#define COMPILE_DATE \"$builddate\"" >> ../src/lib_ccx/compile_info.h
+echo "#endif" >> ../src/lib_ccx/compile_info.h
+echo "Stored all in compile.h"
+echo "Done."

mac/build.command

@@ -1,2 +1,17 @@
-gcc -std=gnu99 -Wno-write-strings -DGPAC_CONFIG_DARWIN -D_FILE_OFFSET_BITS=64 -Dfopen64=fopen -Dopen64=open -Dlseek64=lseek -I ../src/gpacmp4 -I ../src/lib_ccx -I ../src/libpng -I ../src/zlib -o ccextractor $(find ../src/ -name '*.cpp' \! -name 'win_*') $(find ../src/ -name '*.c' \! -name 'win_*') -liconv
+#!/bin/bash
+cd `dirname $0`
+BLD_FLAGS="-std=gnu99 -Wno-write-strings -DGPAC_CONFIG_DARWIN -D_FILE_OFFSET_BITS=64 -Dfopen64=fopen -Dopen64=open -Dlseek64=lseek"
+BLD_INCLUDE="-I../src/lib_ccx -I../src/gpacmp4 -I../src/libpng -I../src/zlib -I../src/zvbi -I../src/lib_hash"
+SRC_LIBPNG="$(find ../src/libpng -name '*.c')"
+SRC_ZLIB="$(find ../src/zlib -name '*.c')"
+SRC_ZVBI="$(find ../src/zvbi -name '*.c')"
+SRC_CCX="$(find ../src/lib_ccx -name '*.c')"
+SRC_GPAC="$(find ../src/gpacmp4 -name '*.c')"
+SRC_LIB_HASH="$(find ../src/lib_hash -name '*.c')"
+BLD_SOURCES="../src/ccextractor.c $SRC_CCX $SRC_GPAC $SRC_ZVBI $SRC_ZLIB $SRC_LIBPNG $SRC_LIB_HASH"
+BLD_LINKER="-lm -liconv"
+
+./pre-build.sh
+gcc $BLD_FLAGS $BLD_INCLUDE -o ccextractor $BLD_SOURCES $BLD_LINKER
+
 

mac/pre-build.sh

@@ -0,0 +1,34 @@
+#!/bin/bash
+echo "Obtaining Git commit"
+commit=(`git rev-parse HEAD 2>/dev/null`)
+if [ -z "$commit" ]; then
+	echo "Git command not present, trying folder approach"
+	if [ -d "../.git" ]; then 
+		echo "Git folder found, using HEAD file"
+		head="`cat ../.git/HEAD`"
+		ref_pos=(`expr match "$head" 'ref: '`)
+		if [ $ref_pos -eq 5 ]; then	
+			echo "HEAD file contains a ref, following"
+			commit=(`cat "../.git/${head:5}"`)
+			echo "Extracted commit: $commit"
+		else
+			echo "HEAD contains a commit, using it"
+			commit="$head"
+			echo "Extracted commit: $commit"	
+		fi 
+	fi
+fi
+if [ -z "$commit" ]; then
+	commit="Unknown"
+fi
+builddate=`date +%Y-%m-%d`
+echo "Storing variables in file"
+echo "Commit: $commit"
+echo "Date: $builddate"
+echo "#ifndef CCX_CCEXTRACTOR_COMPILE_H" > ../src/lib_ccx/compile_info.h
+echo "#define CCX_CCEXTRACTOR_COMPILE_H" >> ../src/lib_ccx/compile_info.h
+echo "#define GIT_COMMIT \"$commit\"" >> ../src/lib_ccx/compile_info.h
+echo "#define COMPILE_DATE \"$builddate\"" >> ../src/lib_ccx/compile_info.h
+echo "#endif" >> ../src/lib_ccx/compile_info.h
+echo "Stored all in compile.h"
+echo "Done."

src/CMakeLists.txt

@@ -20,6 +20,8 @@ configure_file (
 include_directories ("${PROJECT_SOURCE_DIR}")
 include_directories ("${PROJECT_SOURCE_DIR}/lib_ccx")
 include_directories ("${PROJECT_SOURCE_DIR}/gpacmp4/")
+include_directories ("${PROJECT_SOURCE_DIR}/zvbi")
+aux_source_directory ("${PROJECT_SOURCE_DIR}/zvbi" SOURCEFILE)
 
 #Adding some platform specific library path
 link_directories (/opt/local/lib)
@@ -44,9 +46,15 @@ if(PKG_CONFIG_FOUND)
     set (EXTRA_LIBS ${EXTRA_LIBS} png)
   else (PNG_FOUND)
     include_directories ("${PROJECT_SOURCE_DIR}/libpng/")
+    include_directories ("${PROJEXT_SOURCE_DIR}/zlib/")
     aux_source_directory ("${PROJECT_SOURCE_DIR}/libpng/" SOURCEFILE)
     aux_source_directory ("${PROJECT_SOURCE_DIR}/zlib/" SOURCEFILE)
   endif(PNG_FOUND)
+else(PKG_CONFIG_FOUND)
+    include_directories ("${PROJECT_SOURCE_DIR}/libpng/")
+    include_directories ("${PROJEXT_SOURCE_DIR}/zlib/")
+    aux_source_directory ("${PROJECT_SOURCE_DIR}/libpng/" SOURCEFILE)
+    aux_source_directory ("${PROJECT_SOURCE_DIR}/zlib/" SOURCEFILE)
 endif (PKG_CONFIG_FOUND)
 
 ########################################################

src/ccextractor.c

@@ -8,7 +8,6 @@ License: GPL 2.0
 #include <sys/types.h>
 #include <sys/stat.h>
 #include <signal.h>
-#include "ffmpeg_intgr.h"
 #include "ccx_common_option.h"
 #include "ccx_mp4.h"
 
@@ -24,9 +23,6 @@ void sigint_handler()
 struct ccx_s_options ccx_options;
 int main(int argc, char *argv[])
 {
-#ifdef ENABLE_FFMPEG
-	void *ffmpeg_ctx = NULL;
-#endif
 	struct lib_ccx_ctx *ctx;
 	struct lib_cc_decode *dec_ctx = NULL;
 	int ret = 0;
@@ -88,59 +84,32 @@ int main(int argc, char *argv[])
 	if (ccx_options.binary_concat)
 	{
 		ctx->total_inputsize=gettotalfilessize(ctx);
-		if (ctx->total_inputsize==-1)
-			fatal (EXIT_UNABLE_TO_DETERMINE_FILE_SIZE, "Failed to determine total file size.\n");
+		if (ctx->total_inputsize < 0)
+		{
+			switch (ctx->total_inputsize)
+			{
+			case -1*ENOENT:
+				fatal(EXIT_NO_INPUT_FILES, "Failed to open file: File not Exist");
+			case -1*EACCES:
+				fatal(EXIT_READ_ERROR, "Failed to open file: Unable to access");
+			case -1*EINVAL:
+				fatal(EXIT_READ_ERROR, "Failed to open file: Invalid opening flag.");
+			case -1*EMFILE:
+				fatal(EXIT_TOO_MANY_INPUT_FILES, "Failed to open file: Too many files are open.");
+			default:
+				fatal(EXIT_READ_ERROR, "Failed to open file: Reason unknown");
+			}
+		}
 	}
 
 #ifndef _WIN32
 	signal_ctx = ctx;
 	m_signal(SIGINT, sigint_handler);
+	create_signal();
 #endif
-
 	while (switch_to_next_file(ctx, 0))
 	{
 		prepare_for_new_file(ctx);
-#ifdef ENABLE_FFMPEG
-		close_input_file(ctx);
-		ffmpeg_ctx =  init_ffmpeg(ctx->inputfile[0]);
-		if(ffmpeg_ctx)
-		{
-			do
-			{
-				int ret = 0;
-				unsigned char *bptr = ctx->buffer;
-				int len = ff_get_ccframe(ffmpeg_ctx, bptr, 1024);
-                                int cc_count = 0;
-				if(len == AVERROR(EAGAIN))
-				{
-					continue;
-				}
-				else if(len == AVERROR_EOF)
-					break;
-				else if(len == 0)
-					continue;
-				else if(len < 0 )
-				{
-					mprint("Error extracting Frame\n");
-					break;
-
-				}
-                                else
-                                    cc_count = len/3;
-				ret = process_cc_data(dec_ctx, bptr, cc_count, &dec_ctx->dec_sub);
-				if(ret >= 0 && dec_ctx->dec_sub.got_output)
-				{
-					encode_sub(enc_ctx, &dec_ctx->dec_sub);
-					dec_ctx->dec_sub.got_output = 0;
-				}
-			}while(1);
-			continue;
-		}
-		else
-		{
-			mprint ("\rFailed to initialized ffmpeg falling back to legacy\n");
-		}
-#endif
 		stream_mode = ctx->demux_ctx->get_stream_mode(ctx->demux_ctx);
 		// Disable sync check for raw formats - they have the right timeline.
 		// Also true for bin formats, but -nosync might have created a
@@ -173,6 +142,10 @@ int main(int argc, char *argv[])
 			case CCX_SM_PROGRAM:
 			case CCX_SM_ASF:
 			case CCX_SM_WTV:
+			case CCX_SM_GXF:
+#ifdef ENABLE_FFMPEG
+			case CCX_SM_FFMPEG:
+#endif
 				if (!ccx_options.use_gop_as_pts) // If !0 then the user selected something
 					ccx_options.use_gop_as_pts = 0; 
 				mprint ("\rAnalyzing data in general mode\n");
@@ -194,7 +167,7 @@ int main(int argc, char *argv[])
 			case CCX_SM_MP4:
 				mprint ("\rAnalyzing data with GPAC (MP4 library)\n");
 				close_input_file(ctx); // No need to have it open. GPAC will do it for us
-				processmp4 (ctx, &ctx->mp4_cfg, ctx->inputfile[0]);
+				processmp4(ctx, &ctx->mp4_cfg, ctx->inputfile[ctx->current_file]);
 				if (ccx_options.print_file_reports)
 					print_file_report(ctx);
 				break;
@@ -209,68 +182,6 @@ int main(int argc, char *argv[])
 				break;
 		}
 
-#if 0
-		if (ctx->total_pulldownframes)
-			mprint ("incl. pulldown frames:  %s  (%u frames at %.2ffps)\n",
-					print_mstime( (LLONG)(ctx->total_pulldownframes*1000/current_fps) ),
-					ctx->total_pulldownframes, current_fps);
-		if (pts_set >= 1 && min_pts != 0x01FFFFFFFFLL)
-		{
-			LLONG postsyncms = (LLONG) (ctx->frames_since_last_gop*1000/current_fps);
-			mprint ("\nMin PTS:				%s\n",
-					print_mstime( min_pts/(MPEG_CLOCK_FREQ/1000) - fts_offset));
-			if (pts_big_change)
-				mprint ("(Reference clock was reset at some point, Min PTS is approximated)\n");
-			mprint ("Max PTS:				%s\n",
-					print_mstime( sync_pts/(MPEG_CLOCK_FREQ/1000) + postsyncms));
-
-			mprint ("Length:				 %s\n",
-					print_mstime( sync_pts/(MPEG_CLOCK_FREQ/1000) + postsyncms
-								  - min_pts/(MPEG_CLOCK_FREQ/1000) + fts_offset ));
-		}
-		// dvr-ms files have invalid GOPs
-		if (gop_time.inited && first_gop_time.inited && stream_mode != CCX_SM_ASF)
-		{
-			mprint ("\nInitial GOP time:	   %s\n",
-				print_mstime(first_gop_time.ms));
-			mprint ("Final GOP time:		 %s%+3dF\n",
-				print_mstime(gop_time.ms),
-				ctx->frames_since_last_gop);
-			mprint ("Diff. GOP length:	   %s%+3dF",
-				print_mstime(gop_time.ms - first_gop_time.ms),
-				ctx->frames_since_last_gop);
-			mprint ("	(%s)\n",
-				print_mstime(gop_time.ms - first_gop_time.ms
-				+(LLONG) ((ctx->frames_since_last_gop)*1000/29.97)) );
-		}
-
-		if (ctx->false_pict_header)
-			mprint ("\nNumber of likely false picture headers (discarded): %d\n",ctx->false_pict_header);
-
-		if (ctx->stat_numuserheaders)
-			mprint("\nTotal user data fields: %d\n", ctx->stat_numuserheaders);
-		if (ctx->stat_dvdccheaders)
-			mprint("DVD-type user data fields: %d\n", ctx->stat_dvdccheaders);
-		if (ctx->stat_scte20ccheaders)
-			mprint("SCTE-20 type user data fields: %d\n", ctx->stat_scte20ccheaders);
-		if (ctx->stat_replay4000headers)
-			mprint("ReplayTV 4000 user data fields: %d\n", ctx->stat_replay4000headers);
-		if (ctx->stat_replay5000headers)
-			mprint("ReplayTV 5000 user data fields: %d\n", ctx->stat_replay5000headers);
-		if (ctx->stat_hdtv)
-			mprint("HDTV type user data fields: %d\n", ctx->stat_hdtv);
-		if (ctx->stat_dishheaders)
-			mprint("Dish Network user data fields: %d\n", ctx->stat_dishheaders);
-		if (ctx->stat_divicom)
-		{
-			mprint("CEA608/Divicom user data fields: %d\n", ctx->stat_divicom);
-
-			mprint("\n\nNOTE! The CEA 608 / Divicom standard encoding for closed\n");
-			mprint("caption is not well understood!\n\n");
-			mprint("Please submit samples to the developers.\n\n\n");
-		}
-#endif
-
 		list_for_each_entry(dec_ctx, &ctx->dec_ctx_head, list, struct lib_cc_decode)
 		{
 			mprint("\n");
@@ -319,8 +230,76 @@ int main(int argc, char *argv[])
 			cb_field1 = 0; cb_field2 = 0; cb_708 = 0;
 			dec_ctx->timing->fts_now = 0;
 			dec_ctx->timing->fts_max = 0;
+		
+
+
+
+		if (dec_ctx->total_pulldownframes)
+			mprint ("incl. pulldown frames:  %s  (%u frames at %.2ffps)\n",
+					print_mstime( (LLONG)(dec_ctx->total_pulldownframes*1000/current_fps) ),
+					dec_ctx->total_pulldownframes, current_fps);
+		if (dec_ctx->timing->pts_set >= 1 && dec_ctx->timing->min_pts != 0x01FFFFFFFFLL)
+		{
+			LLONG postsyncms = (LLONG) (dec_ctx->frames_since_last_gop*1000/current_fps);
+			mprint ("\nMin PTS:				%s\n",
+					print_mstime( dec_ctx->timing->min_pts/(MPEG_CLOCK_FREQ/1000) - dec_ctx->timing->fts_offset));
+			if (pts_big_change)
+				mprint ("(Reference clock was reset at some point, Min PTS is approximated)\n");
+			mprint ("Max PTS:				%s\n",
+					print_mstime( dec_ctx->timing->sync_pts/(MPEG_CLOCK_FREQ/1000) + postsyncms));
+
+			mprint ("Length:				 %s\n",
+					print_mstime( dec_ctx->timing->sync_pts/(MPEG_CLOCK_FREQ/1000) + postsyncms
+								  - dec_ctx->timing->min_pts/(MPEG_CLOCK_FREQ/1000) + dec_ctx->timing->fts_offset ));
 		}
 
+
+		// dvr-ms files have invalid GOPs
+		if (gop_time.inited && first_gop_time.inited && stream_mode != CCX_SM_ASF)
+		{
+			mprint ("\nInitial GOP time:	   %s\n",
+				print_mstime(first_gop_time.ms));
+			mprint ("Final GOP time:		 %s%+3dF\n",
+				print_mstime(gop_time.ms),
+				dec_ctx->frames_since_last_gop);
+			mprint ("Diff. GOP length:	   %s%+3dF",
+				print_mstime(gop_time.ms - first_gop_time.ms),
+				dec_ctx->frames_since_last_gop);
+			mprint ("	(%s)\n\n",
+				print_mstime(gop_time.ms - first_gop_time.ms
+				+(LLONG) ((dec_ctx->frames_since_last_gop)*1000/29.97)) );
+		}
+
+		if (dec_ctx->false_pict_header)
+			mprint ("\nNumber of likely false picture headers (discarded): %d\n",dec_ctx->false_pict_header);
+
+		if (dec_ctx->stat_numuserheaders)
+			mprint("\nTotal user data fields: %d\n", dec_ctx->stat_numuserheaders);
+		if (dec_ctx->stat_dvdccheaders)
+			mprint("DVD-type user data fields: %d\n", dec_ctx->stat_dvdccheaders);
+		if (dec_ctx->stat_scte20ccheaders)
+			mprint("SCTE-20 type user data fields: %d\n", dec_ctx->stat_scte20ccheaders);
+		if (dec_ctx->stat_replay4000headers)
+			mprint("ReplayTV 4000 user data fields: %d\n", dec_ctx->stat_replay4000headers);
+		if (dec_ctx->stat_replay5000headers)
+			mprint("ReplayTV 5000 user data fields: %d\n", dec_ctx->stat_replay5000headers);
+		if (dec_ctx->stat_hdtv)
+			mprint("HDTV type user data fields: %d\n", dec_ctx->stat_hdtv);
+		if (dec_ctx->stat_dishheaders)
+			mprint("Dish Network user data fields: %d\n", dec_ctx->stat_dishheaders);
+		if (dec_ctx->stat_divicom)
+		{
+			mprint("CEA608/Divicom user data fields: %d\n", dec_ctx->stat_divicom);
+
+			mprint("\n\nNOTE! The CEA 608 / Divicom standard encoding for closed\n");
+			mprint("caption is not well understood!\n\n");
+			mprint("Please submit samples to the developers.\n\n\n");
+		}
+
+		}
+
+		
+
 		if(is_decoder_processed_enough(ctx) == CCX_TRUE)
 			break;
 	} // file loop
@@ -343,7 +322,6 @@ int main(int argc, char *argv[])
 			s1, s2);
 	}
 #endif
-	dbg_print(CCX_DMT_708, "[CEA-708] The 708 decoder was reset [%d] times.\n", ctx->freport.data_from_708->reset_count);
 
 	if (is_decoder_processed_enough(ctx) == CCX_TRUE)
 	{

src/gpacmp4/box_funcs.c

@@ -76,7 +76,14 @@ GF_Err gf_isom_parse_box_ex(GF_Box **outBox, GF_BitStream *bs, u32 parent_type)
 		if (!size) {
 			type = gf_bs_peek_bits(bs, 32, 0);
 			if (!isalnum((type>>24)&0xFF) || !isalnum((type>>16)&0xFF) || !isalnum((type>>8)&0xFF) || !isalnum(type&0xFF)) {
-				size = 4;
+				/* size = 4; */
+
+				/* In some files there are a single 0x00 at the end of atom. When the size is set to 4, */
+				/* but parent atom expects this atom to be 1 byte, it causes GF_ISOM_INCOMPLETE_FILE. */
+				/* In case there are multiple 0x00 bytes at the end, they will be handled individually, I hope :) */
+				size = 1;
+				hdr_size = 1;
+
 				type = GF_ISOM_BOX_TYPE_VOID;
 			} else {
 				goto proceed_box;
@@ -102,7 +109,7 @@ proceed_box:
 	}
 	
 	//handle large box
-	if (size == 1) {
+	if (size == 1 && type != GF_ISOM_BOX_TYPE_VOID) {
 		size = gf_bs_read_u64(bs);
 		hdr_size += 8;
 	}

src/lib_ccx/avc_functions.c

@@ -78,6 +78,12 @@ struct avc_ctx *init_avc(void)
 
 	ctx->last_pic_order_cnt_lsb = -1;
 	ctx->last_slice_pts = -1;
+
+	ctx->num_nal_unit_type_7 = 0;
+	ctx->num_vcl_hrd = 0;
+	ctx->num_nal_hrd = 0;
+	ctx->num_jump_in_frames = 0;
+	ctx->num_unexpected_sei_length = 0;	
 	return ctx;
 }
 
@@ -1023,18 +1029,14 @@ void slice_header (struct lib_cc_decode *ctx, unsigned char *heabuf, unsigned ch
 	}
 
 
-#if 0
-Leaving it for record,  after testing garbled_dishHD.mpg, and
-2014 SugarHouse Casino Mummers Parade Fancy Brigades_new.ts might need to come back
 	// Sometimes two P-slices follow each other, see garbled_dishHD.mpg,
 	// in this case we only treat the first as a reference pic
 	if (isref && ctx->frames_since_last_gop <= 3) // Used to be == 1, but the sample file
 	{ // 2014 SugarHouse Casino Mummers Parade Fancy Brigades_new.ts was garbled
 		// Probably doing a proper PTS sort would be a better solution.
-		//isref = 0;
+		isref = 0;
 		dbg_print(CCX_DMT_TIME, "Ignoring this reference pic.\n");
 	}
-#endif
 
 	// if slices are buffered - flush
 	if (isref)

src/lib_ccx/ccx_common_constants.h

@@ -162,7 +162,7 @@ enum ccx_output_format
 	CCX_OF_NULL = 5,
 	CCX_OF_SMPTETT = 6,
 	CCX_OF_SPUPNG = 7,
-	CCX_OF_DVDRAW = 8, // See -d at http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/SCC_TOOLS.HTML#CCExtract
+	CCX_OF_DVDRAW = 8, // See -d at http://www.theneitherworld.com/mcpoodle/SCC_TOOLS/DOCS/SCC_TOOLS.HTML#CCExtract
 	CCX_OF_WEBVTT = 9,
 	CCX_OF_SIMPLE_XML = 10,
 	CCX_OF_G608 = 11,
@@ -191,6 +191,10 @@ enum ccx_stream_mode_enum
 	CCX_SM_HEX_DUMP = 8, // Hexadecimal dump generated by wtvccdump
 #endif
 	CCX_SM_WTV = 9,
+#ifdef ENABLE_FFMPEG
+	CCX_SM_FFMPEG = 10,
+#endif
+	CCX_SM_GXF = 11,
 	CCX_SM_AUTODETECT = 16
 };
 
@@ -213,6 +217,8 @@ enum ccx_bufferdata_type
 	CCX_PRIVATE_MPEG2_CC = 6,
 	CCX_DVB_SUBTITLE = 7,
 	CCX_ISDB_SUBTITLE = 8,
+	/* BUffer where cc data contain 3 byte cc_valid ccdata 1 ccdata 2 */
+	CCX_RAW_TYPE = 9,
 };
 
 enum ccx_frame_type

src/lib_ccx/ccx_common_option.c

@@ -55,6 +55,10 @@ void init_options (struct ccx_s_options *options)
 	options->xmltvliveinterval=10; // interval in seconds between writting xmltv output files in live mode
 	options->xmltvoutputinterval=0; // interval in seconds between writting xmltv full file output
 	options->xmltvonlycurrent=0; // 0 off 1 on
+	options->keep_output_closed = 0; // By default just keep the file open.
+	options->force_flush = 0; // Don't flush whenever content is written.
+	options->append_mode = 0; //By default, files are overwritten.
+	options->ucla = 0; // By default, -UCLA not used
 
 	options->transcript_settings = ccx_encoders_default_transcript_settings;
 	options->millis_separator=',';
@@ -105,6 +109,7 @@ void init_options (struct ccx_s_options *options)
 	options->enc_cfg.no_bom = 0; // Use BOM by default.
 	options->enc_cfg.services_charsets = NULL;
 	options->enc_cfg.all_services_charset = NULL;
+	options->enc_cfg.with_semaphore = 0; 
 
 	options->settings_dtvcc.enabled = 0;
 	options->settings_dtvcc.active_services_count = 0;

src/lib_ccx/ccx_common_option.h

@@ -34,6 +34,10 @@ struct encoder_cfg
 	int gui_mode_reports; // If 1, output in stderr progress updates so the GUI can grab them
 	char *output_filename;
 	enum ccx_output_format write_format; // 0=Raw, 1=srt, 2=SMI
+	int keep_output_closed;
+	int force_flush; // Force flush on content write
+	int append_mode; //Append mode for output files
+	int ucla; // 1 if -UCLA used, 0 if not
 	
 	enum ccx_encoding_type encoding;
 	enum ccx_output_date_format date_format;
@@ -41,6 +45,7 @@ struct encoder_cfg
 	int autodash; // Add dashes (-) before each speaker automatically?
 	int trim_subs; // "    Remove spaces at sides?    "
 	int sentence_cap ; // FIX CASE? = Fix case?
+	int with_semaphore; // Write a .sem file on file open and delete it on close?
 	/* Credit stuff */
 	char *start_credits_text;
 	char *end_credits_text;
@@ -111,6 +116,10 @@ struct ccx_s_options // Options from user parameters
 	int xmltvliveinterval; // interval in seconds between writting xmltv output files in live mode
 	int xmltvoutputinterval; // interval in seconds between writting xmltv full file output
 	int xmltvonlycurrent; // 0 off 1 on
+	int keep_output_closed;
+	int force_flush; // Force flush on content write
+	int append_mode; //Append mode for output files
+	int ucla; // 1 if UCLA used, 0 if not
 
 	ccx_encoders_transcript_format transcript_settings; // Keeps the settings for generating transcript output files.
 	enum ccx_output_date_format date_format;

src/lib_ccx/ccx_decoders_608.c

@@ -1141,8 +1141,8 @@ int process608(const unsigned char *data, int length, void *private_data, struct
 			continue;
 		}
 		if (hi>=0x10 && hi<=0x1F) // Non-character code or special/extended char
-			// http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/CC_CODES.HTML
-			// http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/CC_CHARS.HTML
+			// http://www.theneitherworld.com/mcpoodle/SCC_TOOLS/DOCS/CC_CODES.HTML
+			// http://www.theneitherworld.com/mcpoodle/SCC_TOOLS/DOCS/CC_CHARS.HTML
 		{
 			if (!context || context->my_field == 2)
 				in_xds_mode=0; // Back to normal (CEA 608-8.6.2)

src/lib_ccx/ccx_decoders_708_output.c

@@ -149,7 +149,7 @@ void ccx_dtvcc_write_srt(ccx_dtvcc_writer_ctx *writer, dtvcc_tv_screen *tv, stru
 	sprintf(buf + strlen(buf), " --> ");
 	mstime_sprintf(tv->time_ms_hide + encoder->subs_delay,
 				   "%02u:%02u:%02u,%03u", buf + strlen(buf));
-	sprintf(buf + strlen(buf), (char *) encoder->encoded_crlf);
+	sprintf(buf + strlen(buf),"%s", (char *) encoder->encoded_crlf);
 
 	write(encoder->dtvcc_writers[tv->service_number - 1].fd, buf, strlen(buf));
 
@@ -422,4 +422,4 @@ void ccx_dtvcc_writer_output(ccx_dtvcc_writer_ctx *writer, dtvcc_tv_screen *tv,
 	}
 
 	_ccx_dtvcc_write(writer, tv, encoder);
-}
+}

src/lib_ccx/ccx_decoders_common.c

@@ -285,7 +285,19 @@ struct lib_cc_decode* init_cc_decode (struct ccx_decoders_common_settings_t *set
 	ctx->max_gop_length = 0;
 	ctx->has_ccdata_buffered = 0;
 	ctx->in_bufferdatatype = CCX_UNKNOWN;
-	ctx->frames_since_last_gop = 0;
+	ctx->frames_since_last_gop  = 0;
+	ctx->total_pulldownfields   = 0;
+	ctx->total_pulldownframes   = 0;
+	ctx->stat_numuserheaders    = 0;
+	ctx->stat_dvdccheaders      = 0;
+	ctx->stat_scte20ccheaders   = 0;
+	ctx->stat_replay5000headers = 0;
+	ctx->stat_replay4000headers = 0;
+	ctx->stat_dishheaders       = 0;
+	ctx->stat_hdtv              = 0;
+	ctx->stat_divicom           = 0;
+	ctx->false_pict_header = 0;
+
 	memcpy(&ctx->extraction_start, &setting->extraction_start,sizeof(struct ccx_boundary_time));
 	memcpy(&ctx->extraction_end, &setting->extraction_end,sizeof(struct ccx_boundary_time));
 
@@ -340,12 +352,13 @@ struct lib_cc_decode* init_cc_decode (struct ccx_decoders_common_settings_t *set
 
 	ctx->anchor_seq_number = -1;
 	// Init XDS buffers
-	if(setting->ignore_xds == CCX_TRUE)
-		ctx->xds_ctx = NULL;
-	else
-		ctx->xds_ctx = ccx_decoders_xds_init_library(ctx->timing);
-	//xds_cea608_test(ctx->xds_ctx);
+	if(setting->output_format!=CCX_OF_TRANSCRIPT)
+	{
+		setting->xds_write_to_file = 0;
+	}
+	ctx->xds_ctx = ccx_decoders_xds_init_library(ctx->timing, setting->xds_write_to_file);
 
+	ctx->vbi_decoder = NULL;
 	return ctx;
 }
 

src/lib_ccx/ccx_decoders_isdb.c

@@ -373,6 +373,7 @@ static int reserve_buf(struct ISDBText *text, size_t len)
 	text->buf = ptr;
 	text->len = blen;
 	isdb_log ("expanded ctx->text(%lu)\n", blen);
+	return CCX_OK;
 }
 
 static int append_char(ISDBSubContext *ctx, const char ch)

src/lib_ccx/ccx_decoders_structs.h

@@ -98,7 +98,7 @@ struct ccx_decoders_common_settings_t
 	unsigned int hauppauge_mode; // If 1, use PID=1003, process specially and so on
 	int program_number;
 	enum ccx_code_type codec;
-	int ignore_xds;
+	int xds_write_to_file;
 	void *private_data;
 };
 
@@ -166,6 +166,17 @@ struct lib_cc_decode
 	/* Reguired in es_function.c and es_userdata.c */
 	unsigned top_field_first; // Needs to be global
 
+    /* Stats. Modified in es_userdata.c*/
+    int stat_numuserheaders;
+    int stat_dvdccheaders;
+    int stat_scte20ccheaders;
+    int stat_replay5000headers;
+    int stat_replay4000headers;
+    int stat_dishheaders;
+    int stat_hdtv;
+    int stat_divicom;
+    int false_pict_header;
+
 	ccx_dtvcc_ctx *dtvcc;
 	int current_field;
 	// Analyse/use the picture information
@@ -181,6 +192,7 @@ struct lib_cc_decode
 	// B-Frames belong to this I- or P-frame.
 	int anchor_seq_number;
 	struct ccx_decoders_xds_context *xds_ctx;
+	struct ccx_decoder_vbi_ctx *vbi_decoder;
 
 	int (*writedata)(const unsigned char *data, int length, void *private_data, struct cc_subtitle *sub);
 };

src/lib_ccx/ccx_decoders_vbi.c

@@ -0,0 +1,100 @@
+#include "ccx_decoders_vbi.h"
+#include "ccx_decoders_common.h"
+#include "ccx_common_constants.h"
+#include "ccx_common_common.h"
+#include "utility.h"
+#include "stdlib.h"
+
+void delete_decoder_vbi(struct ccx_decoder_vbi_ctx** arg)
+{
+	struct ccx_decoder_vbi_ctx* ctx = *arg;
+	vbi_raw_decoder_destroy (&ctx->zvbi_decoder);
+
+	freep(arg);
+}
+struct ccx_decoder_vbi_ctx* init_decoder_vbi(struct ccx_decoder_vbi_cfg *cfg)
+{
+	struct ccx_decoder_vbi_ctx *vbi;
+
+	vbi = malloc(sizeof(*vbi));
+	if(!vbi)
+		return NULL;
+
+#ifdef VBI_DEBUG
+	vbi->vbi_debug_dump = fopen("dump_720.vbi","w");
+#endif
+	vbi_raw_decoder_init (&vbi->zvbi_decoder);
+
+	if(cfg == NULL)
+	{
+		/* Specify the image format. */
+		vbi->zvbi_decoder.scanning             = 525;
+
+		/* The decoder ignores chroma data. */
+		vbi->zvbi_decoder.sampling_format      = VBI_PIXFMT_YUV420;
+
+		/* You may have to adjust this. */
+		vbi->zvbi_decoder.sampling_rate        = 13.5e6; /* Hz */
+		vbi->zvbi_decoder.bytes_per_line       = 720;
+
+		/* Sampling starts 9.7 µs from the front edge of the
+		hor. sync pulse. You may have to adjust this. */
+		vbi->zvbi_decoder.offset               = 9.7e-6 * 13.5e6;
+
+		/* Which lines were captured from the first field.
+			You may have to adjust this. */
+		vbi->zvbi_decoder.start[0]             = 21;
+		vbi->zvbi_decoder.count[0]             = 1;
+
+		/* Second field. */
+		vbi->zvbi_decoder.start[1]             = 284;
+		vbi->zvbi_decoder.count[1]             = 1;
+
+		/* The image contains all lines of the first field,
+			followed by all lines of the second field. */
+		vbi->zvbi_decoder.interlaced           = CCX_TRUE;
+
+		/* The first field is always first in memory. */
+		vbi->zvbi_decoder.synchronous          = CCX_TRUE;
+
+		/* Specify the services you want. */
+		vbi_raw_decoder_add_services (&vbi->zvbi_decoder, VBI_SLICED_CAPTION_525, /* strict */ 0);
+
+	}
+	return vbi;
+}
+
+int decode_vbi(struct lib_cc_decode *dec_ctx, uint8_t field, unsigned char *buffer, size_t len,  struct cc_subtitle *sub)
+{
+	int i = 0;
+	unsigned int n_lines;
+	vbi_sliced sliced[52];
+	if(dec_ctx->vbi_decoder == NULL)
+	{
+		dec_ctx->vbi_decoder = init_decoder_vbi(NULL);
+	}
+
+	len -= 720;
+
+	n_lines = vbi_raw_decode(&dec_ctx->vbi_decoder->zvbi_decoder, buffer, sliced);
+	//n_lines = vbi3_raw_decoder_decode (&dec_ctx->vbi_decoder->zvbi_decoder, sliced, 2, buffer);
+	if (n_lines > 0)
+	{
+		for (i = 0; i < n_lines; ++i)
+		{
+			int index = 0;
+			//for(index = 0; index < 56; index += 2)
+			{
+				unsigned char data[3];
+				if (field == 1)
+					data[0] = 0x04;
+				else
+					data[0] = 0x05;
+				data[1] = sliced[i].data[index];
+				data[2] = sliced[i].data[index+1];
+				do_cb(dec_ctx, data, sub);
+			}
+		}
+	}
+	return CCX_OK;
+}

src/lib_ccx/ccx_decoders_vbi.h

@@ -0,0 +1,28 @@
+#ifndef CCX_DECODER_VBI
+#define CCX_DECODER_VBI
+
+#include <zvbi_decoder.h>
+#define VBI_DEBUG
+
+#include "ccx_decoders_structs.h"
+
+struct ccx_decoder_vbi_cfg
+{
+#ifdef VBI_DEBUG
+	char *debug_file_name;
+#endif
+};
+
+struct ccx_decoder_vbi_ctx
+{
+	int vbi_decoder_inited;
+	vbi_raw_decoder zvbi_decoder;
+	//vbi3_raw_decoder zvbi_decoder;
+#ifdef VBI_DEBUG
+	FILE *vbi_debug_dump;
+#endif
+};
+
+
+int decode_vbi(struct lib_cc_decode *dec_ctx, uint8_t field, unsigned char *buffer, size_t len,  struct cc_subtitle *sub);
+#endif

src/lib_ccx/ccx_decoders_xds.c

@@ -2,6 +2,7 @@
 #include "ccx_common_constants.h"
 #include "ccx_common_timing.h"
 #include "ccx_common_common.h"
+#include "utility.h" 
 
 LLONG ts_start_of_xds = -1; // Time at which we switched to XDS mode, =-1 hasn't happened yet
 
@@ -115,9 +116,14 @@ typedef struct ccx_decoders_xds_context
 	int cur_xds_packet_type;
 	struct ccx_common_timing_ctx *timing;
 
+	unsigned current_ar_start;
+	unsigned current_ar_end;
+
+	int xds_write_to_file; // Set to 1 if XDS data is to be written to file
+
 } ccx_decoders_xds_context_t;
 
-struct ccx_decoders_xds_context *ccx_decoders_xds_init_library(struct ccx_common_timing_ctx *timing)
+struct ccx_decoders_xds_context *ccx_decoders_xds_init_library(struct ccx_common_timing_ctx *timing, int xds_write_to_file)
 {
 	int i;
 	struct ccx_decoders_xds_context *ctx = NULL;
@@ -157,6 +163,8 @@ struct ccx_decoders_xds_context *ccx_decoders_xds_init_library(struct ccx_common
 	ctx->cur_xds_packet_type	= 0;
 	ctx->timing = timing;
 
+	ctx->xds_write_to_file = xds_write_to_file;
+
 	return ctx;
 }
 
@@ -192,6 +200,8 @@ int write_xds_string(struct cc_subtitle *sub, struct ccx_decoders_xds_context *c
 
 void xdsprint (struct cc_subtitle *sub, struct ccx_decoders_xds_context *ctx, const char *fmt,...)
 {
+	if(!ctx->xds_write_to_file)
+		return;
 	/* Guess we need no more than 100 bytes. */
 	int n, size = 100;
 	char *p, *np;
@@ -697,6 +707,7 @@ int xds_do_current_and_future (struct cc_subtitle *sub, struct ccx_decoders_xds_
 		case XDS_TYPE_ASPECT_RATIO_INFO:
 			{
 				unsigned ar_start, ar_end;
+				int changed = 0;
 				was_proc = 1;
 				if (ctx->cur_xds_payload_length < 5) // We need 2 data bytes
 					break;
@@ -709,8 +720,20 @@ int xds_do_current_and_future (struct cc_subtitle *sub, struct ccx_decoders_xds_
 				ar_end = 262 - (ctx->cur_xds_payload[3] & 0x1F);
 				unsigned active_picture_height = ar_end - ar_start;
 				float aspect_ratio = (float) 320 / active_picture_height;
-				ccx_common_logging.log_ftn("\rXDS Notice: Aspect ratio info, start line=%u, end line=%u\n", ar_start,ar_end);
-				ccx_common_logging.log_ftn("\rXDS Notice: Aspect ratio info, active picture height=%u, ratio=%f\n", active_picture_height, aspect_ratio);
+
+				if (ar_start != ctx->current_ar_start)
+				{	
+					ctx->current_ar_start = ar_start;
+					ctx->current_ar_end = ar_end;
+					changed = 1;
+					ccx_common_logging.log_ftn("\rXDS Notice: Aspect ratio info, start line=%u, end line=%u\n", ar_start, ar_end);
+					ccx_common_logging.log_ftn("\rXDS Notice: Aspect ratio info, active picture height=%u, ratio=%f\n", active_picture_height, aspect_ratio);
+				}
+				else
+				{
+					ccx_common_logging.debug_ftn(CCX_DMT_DECODER_XDS, "\rXDS Notice: Aspect ratio info, start line=%u, end line=%u\n", ar_start, ar_end);
+					ccx_common_logging.debug_ftn(CCX_DMT_DECODER_XDS, "\rXDS Notice: Aspect ratio info, active picture height=%u, ratio=%f\n", active_picture_height, aspect_ratio);
+				}
 
 			}
 		case XDS_TYPE_PROGRAM_DESC_1:
@@ -959,7 +982,8 @@ void do_end_of_xds (struct cc_subtitle *sub, struct ccx_decoders_xds_context *ct
 
 	if (!was_proc)
 	{
-		ccx_common_logging.log_ftn ("Note: We found an currently unsupported XDS packet.\n");
+		ccx_common_logging.log_ftn ("Note: We found a currently unsupported XDS packet.\n");
+		dump (CCX_DMT_DECODER_XDS,ctx->cur_xds_payload,ctx->cur_xds_payload_length,0,0);
 	}
 	clear_xds_buffer (ctx, ctx->cur_xds_buffer_idx);
 

src/lib_ccx/ccx_decoders_xds.h

@@ -7,7 +7,7 @@ struct ccx_decoders_xds_context;
 void process_xds_bytes (struct ccx_decoders_xds_context *ctx, const unsigned char hi, int lo);
 void do_end_of_xds (struct cc_subtitle *sub, struct ccx_decoders_xds_context *ctx, unsigned char expected_checksum);
 
-struct ccx_decoders_xds_context *ccx_decoders_xds_init_library(struct ccx_common_timing_ctx *timing);
+struct ccx_decoders_xds_context *ccx_decoders_xds_init_library(struct ccx_common_timing_ctx *timing, int xds_write_to_file);
 
 void xds_cea608_test();
 #endif

src/lib_ccx/ccx_demuxer.c

@@ -2,6 +2,7 @@
 #include "activity.h"
 #include "lib_ccx.h"
 #include "utility.h"
+#include "ffmpeg_intgr.h"
 
 static void ccx_demuxer_reset(struct ccx_demuxer *ctx)
 {
@@ -28,6 +29,25 @@ static int ccx_demuxer_isopen(struct ccx_demuxer *ctx)
 }
 static int ccx_demuxer_open(struct ccx_demuxer *ctx, const char *file)
 {
+	ctx->past = 0;
+	ctx->min_global_timestamp = 0;
+	ctx->global_timestamp_inited = 0;
+	ctx->last_global_timestamp = 0;
+	ctx->offset_global_timestamp = 0;
+
+#ifdef ENABLE_FFMPEG
+	ctx->ffmpeg_ctx =  init_ffmpeg(file);
+	if(ctx->ffmpeg_ctx)
+	{
+		ctx->stream_mode = CCX_SM_FFMPEG;
+		ctx->auto_stream = CCX_SM_FFMPEG;
+		return 0;
+	}
+	else
+	{
+		mprint ("\rFailed to initialized ffmpeg falling back to legacy\n");
+	}
+#endif
 	init_file_buffer(ctx);
 	if (ccx_options.input_source==CCX_DS_STDIN)
 	{
@@ -112,6 +132,9 @@ static int ccx_demuxer_open(struct ccx_demuxer *ctx, const char *file)
 			case CCX_SM_MP4:
 				mprint ("\rFile seems to be a MP4\n");
 				break;
+			case CCX_SM_GXF:
+				mprint ("\rFile seems to be a GXF\n");
+				break;
 #ifdef WTV_DEBUG
 			case CCX_SM_HEX_DUMP:
 				mprint ("\rFile seems to be an hexadecimal dump\n");					
@@ -145,7 +168,7 @@ static int ccx_demuxer_open(struct ccx_demuxer *ctx, const char *file)
 			{
 				case CCX_SM_ELEMENTARY_OR_NOT_FOUND:
 				case CCX_SM_PROGRAM:
-					if ( detect_myth(ctx->parent) )
+					if ( detect_myth(ctx) )
 					{
 						ctx->stream_mode=CCX_SM_MYTH;
 					}
@@ -156,12 +179,6 @@ static int ccx_demuxer_open(struct ccx_demuxer *ctx, const char *file)
 			}
 			break;					
 	}
-	ctx->past = 0;
-	ctx->min_global_timestamp = 0;
-	ctx->global_timestamp_inited = 0;
-	ctx->last_global_timestamp = 0;
-	ctx->offset_global_timestamp = 0;
-
 
 	return 0;
 }
@@ -374,6 +391,8 @@ struct demuxer_data* alloc_demuxer_data(void)
 	data->codec = CCX_CODEC_NONE;
 	data->len = 0;
 	data->pts = CCX_NOPTS;
+	data->tb.num = 1;
+	data->tb.den = 90000;
 	data->next_stream = 0;
 	data->next_program = 0;
 	return data;

src/lib_ccx/ccx_demuxer.h

@@ -5,6 +5,7 @@
 #include "ts_functions.h"
 #include "list.h"
 #include "activity.h"
+#include "utility.h"
 
 /* Report information */
 #define SUB_STREAMS_CNT 10
@@ -129,8 +130,14 @@ struct ccx_demuxer
 	// Remember if the last header was valid. Used to suppress too much output
 	// and the expected unrecognized first header for TiVo files.
 	int strangeheader;
+#ifdef ENABLE_FFMPEG
+	void *ffmpeg_ctx;
+#endif
 
 	void *parent;
+
+	//Will contain actual Demuxer Context
+	void *private_data;
 	void (*print_cfg)(struct ccx_demuxer *ctx);
 	void (*reset)(struct ccx_demuxer *ctx);
 	void (*close)(struct ccx_demuxer *ctx);
@@ -150,6 +157,7 @@ struct demuxer_data
 	unsigned char *buffer;
 	size_t len;
 	LLONG pts;
+	struct ccx_rational tb;
 	struct demuxer_data *next_stream;
 	struct demuxer_data *next_program;
 };

src/lib_ccx/ccx_encoders_common.c

@@ -10,6 +10,13 @@
 #include "ccx_encoders_xds.h"
 #include "ccx_encoders_helpers.h"
 
+#ifdef WIN32
+int fsync(int fd)
+{
+	return FlushFileBuffers((HANDLE)_get_osfhandle(fd)) ? 0 : -1;
+}
+#endif
+
 // These are the default settings for plain transcripts. No times, no CC or caption mode, and no XDS.
 ccx_encoders_transcript_format ccx_encoders_default_transcript_settings =
 {
@@ -591,7 +598,7 @@ int write_cc_subtitle_as_transcript(struct cc_subtitle *sub, struct encoder_ctx
 				char buf[80];
 				if (context->transcript_settings->relativeTimestamp)
 				{
-					millis_to_date(end_time, buf, context->date_format, context->millis_separator);
+					millis_to_date(end_time + context->subs_delay, buf, context->date_format, context->millis_separator);
 					fdprintf(context->out->fh, "%s|", buf);
 				}
 				else
@@ -609,12 +616,15 @@ int write_cc_subtitle_as_transcript(struct cc_subtitle *sub, struct encoder_ctx
 				if(context->in_fileformat == 1)
 					//TODO, data->my_field == 1 ? data->channel : data->channel + 2); // Data from field 2 is CC3 or 4
 					fdprintf(context->out->fh, "CC?|");
-				else
+				else if (!context->ucla || !strcmp(sub->mode,"TLT"))
 					fdprintf(context->out->fh, sub->info);
 			}
 			if (context->transcript_settings->showMode)
 			{
-				fdprintf(context->out->fh, "%s|", sub->mode);
+				if(context->ucla && strcmp(sub->mode,"TLT") == 0)
+					fdprintf(context->out->fh, "|");
+				else
+					fdprintf(context->out->fh, "%s|", sub->mode);
 			}
 			ret = write(context->out->fh, context->subline, length);
 			if(ret < length)
@@ -721,8 +731,8 @@ void write_cc_line_as_transcript2(struct eia608_screen *data, struct encoder_ctx
 				fdprintf(context->out->fh, "%s|", buf2);
 			}
 			else {
-				time_t end_time_int = (end_time + context->subs_delay) / 1000;
-				int end_time_dec = (end_time + context->subs_delay) % 1000;
+				time_t end_time_int = end_time / 1000;
+				int end_time_dec = end_time % 1000;
 				struct tm *end_time_struct = gmtime(&end_time_int);
 				strftime(buf2, sizeof(buf2), "%Y%m%d%H%M%S", end_time_struct);
 				fdprintf(context->out->fh, "%s%c%03d|", buf2,context->millis_separator,end_time_dec);
@@ -1058,6 +1068,9 @@ static int init_output_ctx(struct encoder_ctx *ctx, struct encoder_cfg *cfg)
 	if(!ctx->out)
 		return -1;
 	ctx->nb_out = nb_lang;
+	ctx->keep_output_closed = cfg->keep_output_closed;
+	ctx->force_flush = cfg->force_flush;
+	ctx->ucla = cfg->ucla;
 
 	if(cfg->cc_to_stdout == CCX_FALSE && cfg->send_to_srv == CCX_FALSE)
 	{
@@ -1071,19 +1084,14 @@ static int init_output_ctx(struct encoder_ctx *ctx, struct encoder_cfg *cfg)
 				basefilename = get_basename(cfg->output_filename);
 				extension = get_file_extension(cfg->write_format);
 
-				ret = init_write(&ctx->out[0], strdup(cfg->output_filename));
+				ret = init_write(&ctx->out[0], strdup(cfg->output_filename), cfg->with_semaphore);
 				check_ret(cfg->output_filename);
-				ret = init_write(&ctx->out[1], create_outfilename(basefilename, "_2", extension));
+				ret = init_write(&ctx->out[1], create_outfilename(basefilename, "_2", extension), cfg->with_semaphore);
 				check_ret(ctx->out[1].filename);
 			}
-			else if (cfg->extract == 1)
-			{
-				ret = init_write(ctx->out, strdup(cfg->output_filename));
-				check_ret(cfg->output_filename);
-			}
 			else
 			{
-				ret = init_write(ctx->out, strdup(cfg->output_filename));
+				ret = init_write(ctx->out, strdup(cfg->output_filename), cfg->with_semaphore );
 				check_ret(cfg->output_filename);
 			}
 		}
@@ -1094,19 +1102,14 @@ static int init_output_ctx(struct encoder_ctx *ctx, struct encoder_cfg *cfg)
 
 			if (cfg->extract == 12)
 			{
-				ret = init_write(&ctx->out[0], create_outfilename(basefilename, "_1", extension));
+				ret = init_write(&ctx->out[0], create_outfilename(basefilename, "_1", extension), cfg->with_semaphore);
 				check_ret(ctx->out[0].filename);
-				ret = init_write(&ctx->out[1], create_outfilename(basefilename, "_2", extension));
+				ret = init_write(&ctx->out[1], create_outfilename(basefilename, "_2", extension), cfg->with_semaphore);
 				check_ret(ctx->out[1].filename);
 			}
-			else if (cfg->extract == 1)
-			{
-				ret = init_write(ctx->out, create_outfilename(basefilename, NULL, extension));
-				check_ret(ctx->out->filename);
-			}
 			else
 			{
-				ret = init_write(ctx->out, create_outfilename(basefilename, NULL, extension));
+				ret = init_write(ctx->out, create_outfilename(basefilename, NULL, extension), cfg->with_semaphore);
 				check_ret(ctx->out->filename);
 			}
 		}
@@ -1245,6 +1248,9 @@ struct encoder_ctx *init_encoder(struct encoder_cfg *opt)
 	ctx->no_type_setting = opt->no_type_setting;
 	ctx->gui_mode_reports = opt->gui_mode_reports;
 	ctx->extract = opt->extract;
+	ctx->keep_output_closed = opt->keep_output_closed;
+	ctx->force_flush = opt->force_flush;
+	ctx->ucla = opt->ucla;
 
 	ctx->subline = (unsigned char *) malloc (SUBLINESIZE);
 	if(!ctx->subline)
@@ -1313,6 +1319,8 @@ int encode_sub(struct encoder_ctx *context, struct cc_subtitle *sub)
 	if(!context)
 		return CCX_OK;
 
+	context = change_filename(context);
+
 	if (sub->type == CC_608)
 	{
 		struct eia608_screen *data = NULL;
@@ -1374,7 +1382,17 @@ int encode_sub(struct encoder_ctx *context, struct cc_subtitle *sub)
 				wrote_something = write_cc_buffer_as_spupng(data, context);
 				break;
 			case CCX_OF_SIMPLE_XML:
+				if (ccx_options.keep_output_closed && context->out->temporarily_closed)
+				{
+					temporarily_open_output(context->out);
+					write_subtitle_file_header(context, context->out);
+				}
 				wrote_something = write_cc_buffer_as_simplexml(data, context);
+				if (ccx_options.keep_output_closed)
+				{
+					write_subtitle_file_footer(context, context->out);
+					temporarily_close_output(context->out);
+				}
 				break;
 			default:
 				break;
@@ -1479,6 +1497,8 @@ int encode_sub(struct encoder_ctx *context, struct cc_subtitle *sub)
 	}
 	if (!sub->nb_data)
 		freep(&sub->data);
+	if (wrote_something && context->force_flush)
+		fsync(context->out->fh); // Don't buffer
 	return wrote_something;
 }
 

src/lib_ccx/ccx_encoders_common.h

@@ -50,6 +50,12 @@ struct encoder_ctx
 	int nb_out;
 	/* Input file format used in Teletext for exceptional output */
 	unsigned int in_fileformat; //1 =Normal, 2=Teletext
+	/* Keep output file closed when not actually writing to it and start over each time (add headers, etc) */
+	unsigned int keep_output_closed; 
+	/* Force a flush on the file buffer whenever content is written */
+	int force_flush;
+	/* Keep track of whether -UCLA used */
+	int ucla;
 
 	/* Flag saying BOM to be written in each output file */
 	enum ccx_encoding_type encoding;

src/lib_ccx/ccx_encoders_helpers.c

@@ -54,6 +54,44 @@ void correct_case(int line_num, struct eia608_screen *data)
 	char *line = strdup(((char*)data->characters[line_num]));
 	char *oline = (char*)data->characters[line_num];
 	char *c = strtok(line, delim);
+	if (c == NULL)
+	{
+		free(line);
+		return;
+	}
+	do
+	{
+		char **index = bsearch(&c, spell_lower, spell_words, sizeof(*spell_lower), string_cmp);
+
+		if (index)
+		{
+			char *correct_c = *(spell_correct + (index - spell_lower));
+			size_t len = strlen(correct_c);
+			memcpy(oline + (c - line), correct_c, len);
+		}
+	} while ((c = strtok(NULL, delim)) != NULL);
+	free(line);
+}
+
+void telx_correct_case(char *sub_line)
+{
+	char delim[64] = {
+		' ', '\n', '\r', 0x89, 0x99,
+		'!', '"', '#', '%', '&',
+		'\'', '(', ')', ';', '<',
+		'=', '>', '?', '[', '\\',
+		']', '*', '+', ',', '-',
+		'.', '/', ':', '^', '_',
+		'{', '|', '}', '~', '\0' };
+
+	char *line = strdup(((char*)sub_line));
+	char *oline = (char*)sub_line;
+	char *c = strtok(line, delim);
+	if (c == NULL)
+	{
+		free(line);
+		return;
+	}
 	do
 	{
 		char **index = bsearch(&c, spell_lower, spell_words, sizeof(*spell_lower), string_cmp);

src/lib_ccx/ccx_encoders_helpers.h

@@ -24,6 +24,7 @@ struct ccx_encoders_helpers_settings_t {
 // Helper functions
 void correct_case(int line_num, struct eia608_screen *data);
 void capitalize(struct encoder_ctx *context, int line_num, struct eia608_screen *data);
+void telx_correct_case(char *sub_line);
 unsigned get_decoder_line_encoded_for_gui(unsigned char *buffer, int line_num, struct eia608_screen *data);
 unsigned get_decoder_line_encoded(struct encoder_ctx *ctx, unsigned char *buffer, int line_num, struct eia608_screen *data);
 

src/lib_ccx/ccx_encoders_srt.c

@@ -159,6 +159,7 @@ int write_cc_subtitle_as_srt(struct cc_subtitle *sub,struct encoder_ctx *context
 			ret = write_stringz_as_srt(sub->data, context, sub->start_time, sub->end_time);
 			freep(&sub->data);
 			sub->nb_data = 0;
+			ret = 1;
 		}
 		lsub = sub;
 		sub = sub->next;

src/lib_ccx/ccx_encoders_structs.h

@@ -16,8 +16,14 @@ typedef struct ccx_encoders_transcript_format {
 struct ccx_s_write
 {
 	int fh;
+	int temporarily_closed; // 1 means the file was created OK before but we released the handle
 	char *filename;
 	void* spupng_data;
+	int with_semaphore; // 1 means create a .sem file when the file is open and delete it when it's closed
+	char *semaphore_filename;
+	int renaming_extension; //Used for file rotations
+	int append_mode; /* Append the file. Prevent overwriting of files */
+
 };
 
 #endif

src/lib_ccx/ccx_gxf.h

@@ -0,0 +1,9 @@
+#ifndef CCX_GXF
+#define CCX_GXF
+
+#include "ccx_demuxer.h"
+
+int ccx_gxf_probe(unsigned char *buf, int len);
+int ccx_gxf_getmoredata(struct ccx_demuxer *ctx, struct demuxer_data **data);
+struct ccx_gxf *ccx_gxf_init(struct ccx_demuxer *arg);
+#endif

src/lib_ccx/compile_info.h

@@ -0,0 +1,5 @@
+#ifndef CCX_CCEXTRACTOR_COMPILE_H
+#define CCX_CCEXTRACTOR_COMPILE_H
+#define GIT_COMMIT "Unknown"
+#define COMPILE_DATE "Unknown"
+#endif

src/lib_ccx/es_userdata.c

@@ -1,4 +1,5 @@
 #include "lib_ccx.h"
+#include "ccx_decoders_vbi.h"
 
 
 // Parse the user data for captions. The udtype variable denotes
@@ -23,7 +24,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 	}
 
 	// Do something
-	//ctx->stat_numuserheaders++;
+	ctx->stat_numuserheaders++;
 	//header+=4;
 
 	unsigned char *ud_header = next_bytes(ustream, 4);
@@ -35,10 +36,10 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 	}
 
 	// DVD CC header, see
-	// <http://www.geocities.com/mcpoodle43/SCC_TOOLS/DOCS/SCC_FORMAT.HTML>
+	// <http://www.theneitherworld.com/mcpoodle/SCC_TOOLS/DOCS/SCC_FORMAT.HTML>
 	if ( !memcmp(ud_header,"\x43\x43", 2 ) )
 	{
-//		ctx->stat_dvdccheaders++;
+		ctx->stat_dvdccheaders++;
 
 		// Probably unneeded, but keep looking for extra caption blocks
 		int maybeextracb = 1;
@@ -144,7 +145,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 		{
 			unsigned char cc_data[3*31+1]; // Maximum cc_count is 31
 
-//			ctx->stat_scte20ccheaders++;
+			ctx->stat_scte20ccheaders++;
 			read_bytes(ustream, 2); // "03 01"
 
 			unsigned cc_count = (unsigned int) read_bits(ustream,5);
@@ -153,7 +154,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 			unsigned field_number;
 			unsigned cc_data1;
 			unsigned cc_data2;
-			/* unsigned marker; */
+            
 			for (unsigned j=0;j<cc_count;j++)
 			{
 				skip_bits(ustream,2); // priority - unused
@@ -161,7 +162,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 				skip_bits(ustream,5); // line_offset - unused
 				cc_data1 = (unsigned int) read_bits(ustream,8);
 				cc_data2 = (unsigned int) read_bits(ustream,8);
-				/* marker = (unsigned int)read_bits(ustream,1); // TODO: Add syntax check */
+				read_bits(ustream,1); // TODO: Add syntax check */
 
 				if (ustream->bitsleft < 0)
 					fatal(CCX_COMMON_EXIT_BUG_BUG, "Oops!");
@@ -205,12 +206,12 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 			&& ud_header[1] == 0x02 )
 	{
 		unsigned char data[3];
-#if 0
+
 		if (ud_header[0]==0xbb)
 			ctx->stat_replay4000headers++;
 		else
 			ctx->stat_replay5000headers++;
-#endif
+
 
 		read_bytes(ustream, 2); // "BB 02" or "99 02"
 		data[0]=0x05; // Field 2
@@ -226,7 +227,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 	// HDTV - see A/53 Part 4 (Video)
 	else if ( !memcmp(ud_header,"\x47\x41\x39\x34", 4 ) )
 	{
-//		ctx->stat_hdtv++;
+		ctx->stat_hdtv++;
 
 		read_bytes(ustream, 4); // "47 41 39 34"
 
@@ -287,7 +288,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 
 		dbg_print(CCX_DMT_VERBOSE, "Reading Dish Network user data\n");
 
-//		ctx->stat_dishheaders++;
+		ctx->stat_dishheaders++;
 
 		read_bytes(ustream, 2); // "05 02"
 
@@ -452,7 +453,7 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 	else if ( !memcmp(ud_header,"\x02\x09", 2 ) )
 	{
 		// Either a documentation or more examples are needed.
-//		ctx->stat_divicom++;
+		ctx->stat_divicom++;
 
 		unsigned char data[3];
 
@@ -465,6 +466,31 @@ int user_data(struct lib_cc_decode *ctx, struct bitstream *ustream, int udtype,
 		do_cb(ctx, data, sub);
 		// This is probably incomplete!
 	}
+	// GXF vbi OEM code
+	else if ( !memcmp(ud_header,"\x73\x52\x21\x06", 4 ) )
+	{
+		int udatalen = ustream->end - ustream->pos;
+		uint16_t line_nb;
+		uint8_t line_type;
+		uint8_t field = 1;
+		read_bytes(ustream, 4); //skip header code
+		read_bytes(ustream, 2); //skip data length 
+		line_nb = read_bits(ustream, 16);
+		line_type = read_u8(ustream);
+		field = (line_type & 0x03);
+		if(field == 0)
+			mprint("MPEG:VBI: Invalid field\n");
+
+		line_type = line_type >> 2;
+		if(line_type != 1)
+			mprint("MPEG:VBI: only support Luma line\n");
+
+		if (udatalen < 720)
+			mprint("MPEG:VBI: Minimum 720 bytes in luma line required\n");
+
+		decode_vbi(ctx, field, ustream->pos, 720, sub); 
+		dbg_print(CCX_DMT_VERBOSE, "GXF (vbi line %d) user data:\n", line_nb);
+	}
 	else
 	{
 		// Some other user data

src/lib_ccx/ffmpeg_intgr.c

@@ -21,6 +21,7 @@ struct ffmpeg_ctx
 	AVFrame *frame;
 	int stream_index;
 };
+
 /**
  * call back function to be registered for avlog
  */
@@ -52,13 +53,14 @@ static void log_cb(void* ptr, int level, const char* fmt, va_list vl)
 
 	vfprintf(flog, fmt, vl);
 }
+
 /**
  * @path this path could be relative or absolute path of static file
  * 	 this path could be path of device
  *
  * @return ctx Context of ffmpeg
  */
-void *init_ffmpeg(char *path)
+void *init_ffmpeg(const char *path)
 {
 	int ret = 0;
 	int stream_index = 0;
@@ -120,6 +122,7 @@ void *init_ffmpeg(char *path)
 fail:
 	return ctx;
 }
+
 /**
  * @param ctx context of ffmpeg
  * @param data preallocated buffer where data will be recieved
@@ -158,10 +161,10 @@ int ff_get_ccframe(void *arg, unsigned char*data, int maxlen)
 	{
 		return AVERROR(EAGAIN);
 	}
-	current_pts = av_frame_get_best_effort_timestamp(ctx->frame);
-	if(!pts_set)
-		pts_set = 1;
-	set_fts();
+//	current_pts = av_frame_get_best_effort_timestamp(ctx->frame);
+//	if(!pts_set)
+//		pts_set = 1;
+//	set_fts();
 	for(int i = 0;i< ctx->frame->nb_side_data;i++)
 	{
 		if(ctx->frame->side_data[i]->type == AV_FRAME_DATA_A53_CC)
@@ -184,4 +187,56 @@ int ff_get_ccframe(void *arg, unsigned char*data, int maxlen)
 
 	return len;
 }
+
+int ffmpeg_getmoredata(struct ccx_demuxer *ctx, struct demuxer_data **ppdata)
+{
+	struct demuxer_data *data;
+	int ret = 0;
+    if(!*ppdata)
+    {
+        *ppdata = alloc_demuxer_data();
+        if(!*ppdata)
+            return -1;
+        data = *ppdata;
+        //TODO Set to dummy, find and set actual value
+        data->program_number = 1;
+        data->stream_pid = 1;
+        data->codec  = CCX_CODEC_ATSC_CC;
+		data->bufferdatatype = CCX_RAW_TYPE;
+    }
+    else
+    {
+        data = *ppdata;
+    }
+
+	do
+	{
+		int len = ff_get_ccframe(ctx->ffmpeg_ctx, data->buffer, BUFSIZE);
+		if(len == AVERROR(EAGAIN))
+		{
+			continue;
+		}
+		else if(len == AVERROR_EOF)
+		{
+			ret = CCX_EOF;
+			break;
+		}
+		else if(len == 0)
+			continue;
+		else if(len < 0 )
+		{
+			mprint("Error extracting Frame\n");
+			break;
+
+		}
+		else
+		{
+			data->len = len;
+			break;
+		}
+
+	}while(1);
+
+	return ret;
+}
 #endif

src/lib_ccx/ffmpeg_intgr.h

@@ -5,19 +5,14 @@
 #include "libavutil/common.h"
 #include "libavutil/error.h"
 #endif
+#include "ccx_demuxer.h"
 /**
  * @path this path could be relative or absolute path of static file
  * 	 this path could be path of device
  *
  * @return ctx Context of ffmpeg
  */
-void *init_ffmpeg(char *path);
+void *init_ffmpeg(const char *path);
 
-/**
- * @param ctx context of ffmpeg
- * @param data preallocated buffer where data will be recieved
- * @param maxlen length of buffer, where data will be copied
- * @return number of bytes recieved as data
- */
-int ff_get_ccframe(void *arg, unsigned char*data, int maxlen);
+int ffmpeg_getmoredata(struct ccx_demuxer *ctx, struct demuxer_data **ppdata);
 #endif

src/lib_ccx/file_buffer.h

@@ -89,4 +89,6 @@ static size_t inline buffered_read_byte(struct ccx_demuxer *ctx, unsigned char *
 unsigned short buffered_get_be16(struct ccx_demuxer *ctx);
 unsigned char buffered_get_byte (struct ccx_demuxer *ctx);
 unsigned int buffered_get_be32(struct ccx_demuxer *ctx);
+unsigned short buffered_get_le16(struct ccx_demuxer *ctx);
+unsigned int buffered_get_le32(struct ccx_demuxer *ctx);
 #endif

src/lib_ccx/file_functions.c

@@ -37,11 +37,23 @@ LLONG gettotalfilessize (struct lib_ccx_ctx *ctx) // -1 if one or more files fai
 #else
 		h = OPEN (ctx->inputfile[i], O_RDONLY);
 #endif
-		if (h == -1)
-		{
-			mprint ("\rUnable to open %s\r\n", ctx->inputfile[i]);
-			return -1;
+
+		if (h == -1) {
+			switch (errno)
+			{
+			case ENOENT:
+				return -1 * ENOENT;
+			case EACCES:
+				return -1 * EACCES;
+			case EINVAL:
+				return -1 * EINVAL;
+			case EMFILE:
+				return -1 * EMFILE;
+			default:
+				return -1;
+			}
 		}
+
 		if (!ccx_options.live_stream)
 			ts += getfilesize (h);
 		close (h);
@@ -71,18 +83,21 @@ void prepare_for_new_file (struct lib_ccx_ctx *ctx)
 	pts_big_change              = 0;
 	firstcall                   = 1;
 
-	for(int x = 0; x < 0xfff; x++)
+	if(ctx->epg_inited)
 	{
-		ctx->epg_buffers[x].buffer   = NULL;
-		ctx->epg_buffers[x].ccounter = 0;
+		for(int x = 0; x < 0xfff; x++)
+		{
+			ctx->epg_buffers[x].buffer   = NULL;
+			ctx->epg_buffers[x].ccounter = 0;
+		}
+		for (int i = 0; i < TS_PMT_MAP_SIZE; i++)
+		{
+			ctx->eit_programs[i].array_len = 0;
+			ctx->eit_current_events[i] = -1;
+		}
+		ctx->epg_last_output      = -1;
+		ctx->epg_last_live_output = -1;
 	}
-	for (int i = 0; i < TS_PMT_MAP_SIZE; i++)
-	{
-		ctx->eit_programs[i].array_len = 0;
-		ctx->eit_current_events[i] = -1;
-	}
-	ctx->epg_last_output      = -1;
-	ctx->epg_last_live_output = -1;
 }
 
 /* Close input file if there is one and let the GUI know */
@@ -123,6 +138,7 @@ int switch_to_next_file (struct lib_ccx_ctx *ctx, LLONG bytesinbuffer)
 	/* Close current and make sure things are still sane */
 	if (ctx->demux_ctx->is_open(ctx->demux_ctx))
 	{
+		dbg_print(CCX_DMT_708, "[CEA-708] The 708 decoder was reset [%d] times.\n", ctx->freport.data_from_708->reset_count);
 		if (ccx_options.print_file_reports)
 			print_file_report(ctx);
 
@@ -286,6 +302,7 @@ void return_to_buffer (struct ccx_demuxer *ctx, unsigned char *buffer, unsigned
  */
 size_t buffered_read_opt (struct ccx_demuxer *ctx, unsigned char *buffer, size_t bytes)
 {
+	size_t origin_buffer_size = bytes;
 	size_t copied   = 0;
 	time_t seconds = 0;
 
@@ -378,7 +395,7 @@ size_t buffered_read_opt (struct ccx_demuxer *ctx, unsigned char *buffer, size_t
 				{
 					/* If live stream, don't try to switch - acknowledge eof here as it won't
 					   cause a loop end */
-					if (ccx_options.live_stream || !(ccx_options.binary_concat && switch_to_next_file(ctx->parent, copied)))
+					if (ccx_options.live_stream || ((struct lib_ccx_ctx *)ctx->parent)->inputsize <= origin_buffer_size || !(ccx_options.binary_concat && switch_to_next_file(ctx->parent, copied)))
 						eof = 1;
 				}
 				ctx->filebuffer_pos = keep;
@@ -484,3 +501,23 @@ unsigned int buffered_get_be32(struct ccx_demuxer *ctx)
 	val |= buffered_get_be16(ctx);
 	return val;
 }
+
+unsigned short buffered_get_le16(struct ccx_demuxer *ctx)
+{
+	unsigned char a,b;
+	unsigned char *a_p = &a; // Just to suppress warnings
+	unsigned char *b_p = &b;
+	buffered_read_byte(ctx, a_p);
+	ctx->past++;
+	buffered_read_byte(ctx, b_p);
+	ctx->past++;
+	return ( (unsigned short) (b<<8) )| ( (unsigned short) a);
+}
+
+unsigned int buffered_get_le32(struct ccx_demuxer *ctx)
+{
+	unsigned int val;
+	val = buffered_get_le16(ctx);
+	val |= buffered_get_le16(ctx) << 16;
+	return val;
+}

src/lib_ccx/general_loop.c

@@ -13,6 +13,8 @@
 #include "ccx_demuxer.h"
 #include "file_buffer.h"
 #include "ccx_decoders_isdb.h"
+#include "ffmpeg_intgr.h"
+#include "ccx_gxf.h"
 
 unsigned int rollover_bits = 0; // The PTS rolls over every 26 hours and that can happen in the middle of a stream.
 int end_of_file=0; // End of file?
@@ -580,7 +582,7 @@ int process_data(struct encoder_ctx *enc_ctx, struct lib_cc_decode *dec_ctx, str
 	else if (data_node->bufferdatatype == CCX_TELETEXT)
 	{
 		//telxcc_update_gt(dec_ctx->private_data, ctx->demux_ctx->global_timestamp);
-		ret = tlt_process_pes_packet (dec_ctx, data_node->buffer, data_node->len, dec_sub);
+		ret = tlt_process_pes_packet (dec_ctx, data_node->buffer, data_node->len, dec_sub, enc_ctx->sentence_cap);
 		if(ret == CCX_EINVAL)
 			return ret;
 		got = data_node->len;
@@ -632,6 +634,10 @@ int process_data(struct encoder_ctx *enc_ctx, struct lib_cc_decode *dec_ctx, str
 		dec_ctx->in_bufferdatatype = CCX_H264;
 		got = process_avc(dec_ctx, data_node->buffer, data_node->len, dec_sub);
 	}
+	else if (data_node->bufferdatatype == CCX_RAW_TYPE)
+	{
+		got = process_raw_with_field(dec_ctx, dec_sub, data_node->buffer, data_node->len);
+	}
 	else if (data_node->bufferdatatype == CCX_ISDB_SUBTITLE)
 	{
 		isdbsub_decode(dec_ctx, data_node->buffer, data_node->len, dec_sub);
@@ -739,6 +745,14 @@ void general_loop(struct lib_ccx_ctx *ctx)
 			case CCX_SM_WTV:
 				ret = wtv_getmoredata(ctx, &datalist);
 				break;
+			case CCX_SM_GXF:
+				ret = ccx_gxf_getmoredata(ctx->demux_ctx, &datalist);
+				break;
+#ifdef ENABLE_FFMPEG
+			case CCX_SM_FFMPEG:
+				ret = ffmpeg_getmoredata(ctx->demux_ctx, &datalist);
+				break;
+#endif
 			default:
 				fatal(CCX_COMMON_EXIT_BUG_BUG, "Impossible stream_mode");
 		}
@@ -774,14 +788,34 @@ void general_loop(struct lib_ccx_ctx *ctx)
 			dec_ctx = update_decoder_list_cinfo(ctx, cinfo);
 			dec_ctx->dtvcc->encoder = (void *)enc_ctx; //WARN: otherwise cea-708 will not work
 			if(data_node->pts != CCX_NOPTS)
-				set_current_pts(dec_ctx->timing, data_node->pts);
-
-			if( data_node->bufferdatatype == CCX_ISDB_SUBTITLE && ctx->demux_ctx->global_timestamp_inited)
 			{
-				uint64_t tstamp = ( ctx->demux_ctx->global_timestamp + ctx->demux_ctx->offset_global_timestamp )
-							- ctx->demux_ctx->min_global_timestamp;
+				struct ccx_rational tb = {1,MPEG_CLOCK_FREQ};
+				LLONG pts;
+				if(data_node->tb.num != 1 || data_node->tb.den != MPEG_CLOCK_FREQ)
+				{
+					pts = change_timebase(data_node->pts, data_node->tb, tb);
+				}
+				else
+					pts = data_node->pts;
+				set_current_pts(dec_ctx->timing, pts);
+				set_fts(dec_ctx->timing);
+			}
+
+			if( data_node->bufferdatatype == CCX_ISDB_SUBTITLE)
+			{
+				uint64_t tstamp;
+				if(ctx->demux_ctx->global_timestamp_inited)
+				{
+					tstamp = ( ctx->demux_ctx->global_timestamp + ctx->demux_ctx->offset_global_timestamp )
+								- ctx->demux_ctx->min_global_timestamp;
+				}
+				else
+				{
+					// Fix if global timestamp not inited
+					tstamp = get_fts(dec_ctx->timing, dec_ctx->current_field);
+				}
 				isdb_set_global_time(dec_ctx, tstamp);
-                        }
+			}
 
 			ret = process_data(enc_ctx, dec_ctx, data_node);
 			if( ret != CCX_OK)
@@ -853,7 +887,13 @@ void general_loop(struct lib_ccx_ctx *ctx)
 			telxcc_close(&dec_ctx->private_data, &dec_ctx->dec_sub);
 		// Flush remaining HD captions
 		if (dec_ctx->has_ccdata_buffered)
-                	process_hdcc(dec_ctx, &dec_ctx->dec_sub);
+					process_hdcc(dec_ctx, &dec_ctx->dec_sub);
+		
+	mprint ("\nNumber of NAL_type_7: %ld\n",dec_ctx->avc_ctx->num_nal_unit_type_7);
+	mprint ("Number of VCL_HRD: %ld\n",dec_ctx->avc_ctx->num_vcl_hrd);
+	mprint ("Number of NAL HRD: %ld\n",dec_ctx->avc_ctx->num_nal_hrd);
+	mprint ("Number of jump-in-frames: %ld\n",dec_ctx->avc_ctx->num_jump_in_frames);
+	mprint ("Number of num_unexpected_sei_length: %ld", dec_ctx->avc_ctx->num_unexpected_sei_length);
 	}
 
 	delete_datalist(datalist);

src/lib_ccx/lib_ccx.c

@@ -31,8 +31,8 @@ static struct ccx_decoders_common_settings_t *init_decoder_setting(
 	setting->cc_channel = opt->cc_channel;
 	setting->send_to_srv = opt->send_to_srv;
 	setting->hauppauge_mode = opt->hauppauge_mode;
-	/* if in transcript setting xds is not selected then set ignore xds flag */
-	setting->ignore_xds = !opt->transcript_settings.xds;
+	setting->xds_write_to_file = opt->transcript_settings.xds;
+	
 	return setting;
 }
 static void dinit_decoder_setting (struct ccx_decoders_common_settings_t **setting)
@@ -98,6 +98,29 @@ struct lib_ccx_ctx* init_libraries(struct ccx_s_options *opt)
 		ccx_common_logging.fatal_ftn(EXIT_NOT_ENOUGH_MEMORY, "lib_ccx_ctx");
 	memset(ctx, 0, sizeof(struct lib_ccx_ctx));
 
+	if(opt->xmltv)
+	{
+		ctx->epg_inited = 1;
+		ctx->epg_buffers = (struct PSI_buffer *)malloc(sizeof(struct PSI_buffer)*(0xfff+1));
+		ctx->eit_programs = (struct EIT_program *)malloc(sizeof(struct EIT_program)*(TS_PMT_MAP_SIZE+1));
+		ctx->eit_current_events = (int32_t *)malloc(sizeof(int32_t)*(TS_PMT_MAP_SIZE+1));
+		ctx->ATSC_source_pg_map = (int16_t *)malloc(sizeof(int16_t)*(0xffff));
+		memset(ctx->epg_buffers, 0, sizeof(struct PSI_buffer)*(0xfff+1));
+		memset(ctx->eit_programs, 0, sizeof(struct EIT_program)*(TS_PMT_MAP_SIZE+1));
+		memset(ctx->eit_current_events, 0, sizeof(int32_t)*(TS_PMT_MAP_SIZE+1));
+		memset(ctx->ATSC_source_pg_map, 0, sizeof(int16_t)*(0xffff));
+		if(!ctx->epg_buffers || !ctx->eit_programs || !ctx->eit_current_events || !ctx->ATSC_source_pg_map)
+			ccx_common_logging.fatal_ftn(EXIT_NOT_ENOUGH_MEMORY, "lib_ccx_ctx");
+	}
+	else
+	{
+		ctx->epg_inited = 0;
+		ctx->epg_buffers = NULL;
+		ctx->eit_programs = NULL;
+		ctx->eit_current_events = NULL;
+		ctx->ATSC_source_pg_map = NULL;
+	}
+
 	struct ccx_decoder_608_report *report_608 = malloc(sizeof(struct ccx_decoder_608_report));
 	if (!report_608)
 		ccx_common_logging.fatal_ftn(EXIT_NOT_ENOUGH_MEMORY, "report_608");
@@ -139,6 +162,7 @@ struct lib_ccx_ctx* init_libraries(struct ccx_s_options *opt)
 	if (ret < 0) {
 		goto end;
 	}
+	ctx->extension = get_file_extension(opt->write_format);
 
 	ctx->subs_delay = opt->subs_delay;
 

src/lib_ccx/lib_ccx.h

@@ -1,7 +1,7 @@
 #ifndef CCX_CCEXTRACTOR_H
 #define CCX_CCEXTRACTOR_H
 
-#define VERSION "0.79"
+#define VERSION "0.81"
 
 // Load common includes and constants for library usage
 #include "ccx_common_platform.h"
@@ -115,10 +115,11 @@ struct lib_ccx_ctx
 
 	unsigned teletext_warning_shown; // Did we detect a possible PAL (with teletext subs) and told the user already?
 
-	struct PSI_buffer epg_buffers[0xfff+1];
-	struct EIT_program eit_programs[TS_PMT_MAP_SIZE+1];
-	int32_t eit_current_events[TS_PMT_MAP_SIZE+1];
-	int16_t ATSC_source_pg_map[0xffff];
+	int epg_inited;
+	struct PSI_buffer *epg_buffers;
+	struct EIT_program *eit_programs;
+	int32_t *eit_current_events;
+	int16_t *ATSC_source_pg_map;
 	int epg_last_output; 
 	int epg_last_live_output; 
 	struct file_report freport;
@@ -199,7 +200,9 @@ void print_file_report(struct lib_ccx_ctx *ctx);
 
 // output.c
 void dinit_write(struct ccx_s_write *wb);
-int init_write (struct ccx_s_write *wb,char *filename);
+int temporarily_open_output(struct ccx_s_write *wb);
+int temporarily_close_output(struct ccx_s_write *wb);
+int init_write(struct ccx_s_write *wb, char *filename, int with_semaphore);
 int writeraw (const unsigned char *data, int length, void *private_data, struct cc_subtitle *sub);
 void flushbuffer (struct lib_ccx_ctx *ctx, struct ccx_s_write *wb, int closefile);
 void writercwtdata (struct lib_cc_decode *ctx, const unsigned char *data, struct cc_subtitle *sub);
@@ -238,6 +241,9 @@ void timestamp_to_srttime(uint64_t timestamp, char *buffer);
 void timestamp_to_smptetttime(uint64_t timestamp, char *buffer);
 int levenshtein_dist (const uint64_t *s1, const uint64_t *s2, unsigned s1len, unsigned s2len);
 void millis_to_date (uint64_t timestamp, char *buffer, enum ccx_output_date_format date_format, char millis_separator);
+void create_signal(void);
+void signal_handler(int sig_type);
+struct encoder_ctx* change_filename(struct encoder_ctx*);
 #ifndef _WIN32
 void m_signal(int sig, void (*func)(int));
 #endif
@@ -246,7 +252,7 @@ void m_signal(int sig, void (*func)(int));
 void buffered_seek (struct ccx_demuxer *ctx, int offset);
 extern void build_parity_table(void);
 
-int tlt_process_pes_packet(struct lib_cc_decode *dec_ctx, uint8_t *buffer, uint16_t size, struct cc_subtitle *sub);
+int tlt_process_pes_packet(struct lib_cc_decode *dec_ctx, uint8_t *buffer, uint16_t size, struct cc_subtitle *sub, int sentence_cap);
 void* telxcc_init(void);
 void telxcc_close(void **ctx, struct cc_subtitle *sub);
 void tlt_read_rcwt(void *codec, unsigned char *buf, struct cc_subtitle *sub);

src/lib_ccx/output.c

@@ -11,18 +11,72 @@ void dinit_write(struct ccx_s_write *wb)
 	if (wb->fh > 0)
 		close(wb->fh);
 	freep(&wb->filename);	
+	if (wb->with_semaphore && wb->semaphore_filename)
+		unlink(wb->semaphore_filename);
+	freep(&wb->semaphore_filename);
 }
-int init_write (struct ccx_s_write *wb, char *filename)
+
+int temporarily_close_output(struct ccx_s_write *wb)
 {
-	memset(wb, 0, sizeof(struct ccx_s_write));
-	wb->fh=-1;
-	wb->filename = filename;
-	mprint ("Creating %s\n", filename);
-	wb->fh = open (filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, S_IREAD | S_IWRITE);
+	close(wb->fh);
+	wb->fh = -1;
+	wb->temporarily_closed = 1;
+	return 0;
+}
+
+int temporarily_open_output(struct ccx_s_write *wb)
+{
+	int t = 0;
+	// Try a few times before giving up. This is because this close/open stuff exists for processes
+	// that demand exclusive access to the file, so often we'll find out that we cannot reopen the
+	// file immediately.
+	while (t < 5 && wb->fh == -1)
+	{
+		wb->fh = open(wb->filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, S_IREAD | S_IWRITE);
+		sleep(1);
+	}
 	if (wb->fh == -1)
 	{
 		return CCX_COMMON_EXIT_FILE_CREATION_FAILED;
 	}
+	wb->temporarily_closed = 0;
+	return EXIT_OK;
+}
+
+
+
+int init_write (struct ccx_s_write *wb, char *filename, int with_semaphore)
+{
+	memset(wb, 0, sizeof(struct ccx_s_write));
+	wb->fh=-1;
+	wb->temporarily_closed = 0; 
+	wb->filename = filename;
+	wb->with_semaphore = with_semaphore;
+	wb->append_mode = ccx_options.enc_cfg.append_mode;
+	mprint ("Creating %s\n", filename);
+	if(!(wb->append_mode))
+		wb->fh = open (filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, S_IREAD | S_IWRITE);
+	else
+		wb->fh = open (filename, O_RDWR | O_CREAT | O_APPEND | O_BINARY, S_IREAD | S_IWRITE);
+	wb->renaming_extension = 0;
+	if (wb->fh == -1)
+	{
+		return CCX_COMMON_EXIT_FILE_CREATION_FAILED;
+	}
+	if (with_semaphore)
+	{
+		wb->semaphore_filename = (char *)malloc(strlen(filename) + 6);
+		if (!wb->semaphore_filename)
+			return EXIT_NOT_ENOUGH_MEMORY;
+		sprintf(wb->semaphore_filename, "%s.sem", filename);
+		int t = open(wb->semaphore_filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, S_IREAD | S_IWRITE);
+		if (t == -1)
+		{
+			close(wb->fh);
+			return CCX_COMMON_EXIT_FILE_CREATION_FAILED;
+		}
+		close(t);
+	}
 	return EXIT_OK;
 }
 

src/lib_ccx/params.c

@@ -5,6 +5,8 @@
 #include "ccx_encoders_helpers.h"
 #include "ccx_common_common.h"
 #include "ccx_decoders_708.h"
+#include "compile_info.h"
+#include "../lib_hash/sha2.h"
 
 static int inputfile_capacity=0;
 
@@ -274,7 +276,7 @@ void usage (void)
 	mprint ("     .mp4, HDHomeRun are known to work).\n\n");
 	mprint ("  Syntax:\n");
 	mprint ("  ccextractor [options] inputfile1 [inputfile2...] [-o outputfilename]\n");
-	mprint ("               [-o1 outputfilename1] [-o2 outputfilename2]\n\n");
+	mprint ("\n");
 	mprint ("To see This Help Message: -h or --help\n\n");
 	mprint ("File name related options:\n");
 	mprint ("            inputfile: file(s) to process\n");
@@ -318,6 +320,7 @@ void usage (void)
 	mprint ("Options that affect what will be processed:\n");
 	mprint ("          -1, -2, -12: Output Field 1 data, Field 2 data, or both\n");
 	mprint ("                       (DEFAULT is -1)\n");
+	mprint ("Use --append to prevent overwriting of existing files. The output will be appended instead.");
 	mprint ("                 -cc2: When in srt/sami mode, process captions in channel 2\n");
 	mprint ("                       instead of channel 1.\n");
 	mprint ("-svc --service N1[cs1],N2[cs2]...:\n");
@@ -507,16 +510,22 @@ void usage (void)
 	mprint ("            -autodash: Based on position on screen, attempt to determine\n");
 	mprint ("                       the different speakers and a dash (-) when each\n");
 	mprint ("                       of them talks (.srt/.vtt only, -trim required).\n");
-	mprint ("         -xmltv mode:  produce an XMLTV file containing the EPG data from\n");
+	mprint ("          -xmltv mode: produce an XMLTV file containing the EPG data from\n");
 	mprint ("                       the source TS file. Mode: 1 = full output\n");
-	mprint ("                       2 = live output. 3 = both\n\n");
-
+	mprint ("                       2 = live output. 3 = both\n");
+	mprint ("                 -sem: Create a .sem file for each output file that is open\n");
+	mprint ("                       and delete it on file close.\n");
+	mprint ("\n");
 	mprint ("Options that affect how ccextractor reads and writes (buffering):\n");
 
 	mprint ("    -bi --bufferinput: Forces input buffering.\n");
 	mprint (" -nobi -nobufferinput: Disables input buffering.\n");
 	mprint (" -bs --buffersize val: Specify a size for reading, in bytes (suffix with K or\n");
 	mprint ("                       or M for kilobytes and megabytes). Default is 16M.\n");
+	mprint ("                 -koc: keep-output-close. If used then CCExtractor will close\n"); 
+	mprint ("                       the output file after writing each subtitle frame and\n");
+	mprint ("                       attempt to create it again when needed.\n");
+	mprint ("     -ff --forceflush: Flush the file buffer whenever content is written.\n");
 	mprint ("\n");
 
 	mprint ("Options that affect the built-in closed caption decoder:\n");
@@ -714,6 +723,42 @@ void usage (void)
 	mprint("    ...\n");
 }
 
+unsigned char sha256_buf[16384];
+
+char *calculateSHA256(char *location) {
+	int	size_read, bytes_read, fh = 0;
+	SHA256_CTX	ctx256;
+
+	SHA256_Init(&ctx256);
+
+	#ifdef _WIN32
+		fh = OPEN(location, O_RDONLY | O_BINARY);
+	#else
+		fh = OPEN(location, O_RDONLY);
+	#endif
+
+	if (fh < 0) {
+		return "Could not open file";
+	}
+	size_read = 0;
+	while ((bytes_read = read(fh, sha256_buf, 16384)) > 0) {
+		size_read += bytes_read;
+		SHA256_Update(&ctx256, (unsigned char*)sha256_buf, bytes_read);
+	}
+	close(fh);
+	SHA256_End(&ctx256, sha256_buf);
+	return sha256_buf;
+}
+
+void version(char *location) {
+	char *hash = calculateSHA256(location);
+	mprint("CCExtractor detailed version info\n");
+	mprint("	Version: %s\n", VERSION);
+	mprint("	Git commit: %s\n", GIT_COMMIT);
+	mprint("	Compilation date: %s\n", COMPILE_DATE);
+	mprint("	File SHA256: %s\n", hash);
+}
+
 void parse_708_services (struct ccx_s_options *opts, char *s)
 {
 	const char *all = "all";
@@ -838,6 +883,11 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			usage();
 			return EXIT_WITH_HELP;
 		}
+		if (!strcmp(argv[i], "--version"))
+		{
+			version(argv[0]);
+			return EXIT_WITH_HELP;
+		}
 		if (strcmp (argv[i], "-")==0 || strcmp(argv[i], "-stdin") == 0)
 		{
 
@@ -875,6 +925,21 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			opt->buffer_input = 0;
 			continue;
 		}
+		if (strcmp(argv[i], "-koc") == 0)
+		{
+			opt->keep_output_closed = 1;
+			continue;
+		}
+		if (strcmp(argv[i], "-ff") == 0 || strcmp(argv[i], "--forceflush") == 0)
+		{
+			opt->force_flush = 1;
+			continue;
+		}
+		if (strcmp(argv[i], "--append") == 0)
+		{
+			opt->append_mode = 1;
+			continue;
+		}
 		if ((strcmp (argv[i],"-bs")==0 || strcmp (argv[i],"--buffersize")==0) && i<argc-1)
 		{
 			FILEBUFFERSIZE = atol_size(argv[i+1]);
@@ -907,6 +972,10 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			opt->enc_cfg.no_bom = 1;
 			continue;
 		}
+		if (strcmp(argv[i], "-sem") == 0){
+			opt->enc_cfg.with_semaphore = 1;
+			continue;		
+		}	
 		if (strcmp (argv[i],"-nots")==0 ||
 				strcmp (argv[i],"--notypesetting")==0)
 		{
@@ -935,7 +1004,7 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 
 		/*user specified subtitle to be selected */
 
-		if(!strcmp (argv[i],"-codec"))
+		if(!strcmp (argv[i],"-codec") && i < argc - 1)
 		{
 			i++;
 			if(!strcmp (argv[i],"teletext"))
@@ -954,7 +1023,7 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 		}
 		/*user specified subtitle to be selected */
 
-		if(!strcmp (argv[i],"-nocodec"))
+		if(!strcmp (argv[i],"-nocodec") && i < argc - 1)
 		{
 			i++;
 			if(!strcmp (argv[i],"teletext"))
@@ -1133,7 +1202,7 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			opt->enc_cfg.trim_subs=1;
 			continue;
 		}
-		if (strcmp (argv[i],"-outinterval")==0)
+		if (strcmp (argv[i],"-outinterval")==0 && i<argc-1)
 		{
 			opt->out_interval = atoi(argv[i+1]);
 			i++;
@@ -1330,6 +1399,7 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 		}
 		if (strcmp (argv[i],"-xdsdebug")==0)
 		{
+			opt->transcript_settings.xds = 1;
 			opt->debug_mask |= CCX_DMT_DECODER_XDS;
 			continue;
 		}
@@ -1477,7 +1547,8 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 		}
 		if (strcmp (argv[i],"-UCLA")==0 || strcmp (argv[i],"-ucla")==0)
 		{
-			opt->millis_separator='.';
+			opt->ucla = 1;
+			opt->millis_separator = '.';
 			opt->enc_cfg.no_bom = 1;
 			if (!opt->transcript_settings.isFinal){
 				opt->transcript_settings.showStartTime = 1;
@@ -1504,6 +1575,11 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 			opt->enc_cfg.autodash = 1;
 			continue;
 		}
+		if (strcmp(argv[i], "-sem") == 0)
+		{
+			opt->enc_cfg.autodash = 1;
+			continue;
+		}
 		if (strcmp (argv[i],"-xmltv")==0)
 		{
 			if (i==argc-1 // Means no following argument
@@ -1807,13 +1883,15 @@ int parse_parameters (struct ccx_s_options *opt, int argc, char *argv[])
 	opt->enc_cfg.transcript_settings = opt->transcript_settings;
 	opt->enc_cfg.millis_separator = opt->millis_separator;
 	opt->enc_cfg.no_font_color = opt->nofontcolor;
+	opt->enc_cfg.force_flush = opt->force_flush;
+	opt->enc_cfg.append_mode = opt->append_mode;
+	opt->enc_cfg.ucla = opt->ucla;
 	opt->enc_cfg.no_type_setting = opt->notypesetting;
 	opt->enc_cfg.subs_delay = opt->subs_delay;
+	opt->enc_cfg.gui_mode_reports = opt->gui_mode_reports;
 	if(opt->output_filename && opt->multiprogram == CCX_FALSE)
 		opt->enc_cfg.output_filename = strdup(opt->output_filename);
 	else
 		opt->enc_cfg.output_filename = NULL;
 	return EXIT_OK;
-
 }
-

src/lib_ccx/params_dump.c

@@ -225,8 +225,7 @@ void print_cc_report(struct lib_ccx_ctx *ctx, struct cap_info* info)
 }
 void print_file_report(struct lib_ccx_ctx *ctx)
 {
-	struct lib_cc_decode *dec_ctx = NULL;
-	enum ccx_stream_mode_enum stream_mode;
+	struct lib_cc_decode *dec_ctx = NULL;	
 	struct ccx_demuxer *demux_ctx = ctx->demux_ctx;
 
 	printf("File: ");

src/lib_ccx/sequencing.c

@@ -22,8 +22,6 @@ void init_hdcc (struct lib_cc_decode *ctx)
 // Buffer caption blocks for later sorting/flushing.
 void store_hdcc(struct lib_cc_decode *ctx, unsigned char *cc_data, int cc_count, int sequence_number, LLONG current_fts_now, struct cc_subtitle *sub)
 {
-	enum ccx_stream_mode_enum stream_mode;
-
 	//stream_mode = ctx->demux_ctx->get_stream_mode(ctx->demux_ctx);
 	// Uninitialized?
 	if (ctx->anchor_seq_number < 0)

src/lib_ccx/stream_functions.c

@@ -9,6 +9,7 @@
 #include "utility.h"
 #include "ccx_common_timing.h"
 #include "file_buffer.h"
+#include "ccx_gxf.h"
 
 void detect_stream_type (struct ccx_demuxer *ctx)
 {
@@ -27,6 +28,14 @@ void detect_stream_type (struct ccx_demuxer *ctx)
 				ctx->startbytes[3]==0x75)
 			ctx->stream_mode=CCX_SM_ASF;
 	}
+	if (ctx->stream_mode == CCX_SM_ELEMENTARY_OR_NOT_FOUND)
+	{
+		if (ccx_gxf_probe(ctx->startbytes, ctx->startbytes_avail) == CCX_TRUE)
+		{
+			ctx->stream_mode = CCX_SM_GXF;
+			ctx->private_data = ccx_gxf_init(ctx);
+		}
+	}
 	if (ctx->stream_mode == CCX_SM_ELEMENTARY_OR_NOT_FOUND && ctx->startbytes_avail >= 4)
 	{
 		if(ctx->startbytes[0]==0xb7 &&
@@ -186,7 +195,7 @@ int detect_myth( struct ccx_demuxer *ctx )
 {
 	int vbi_blocks=0;
 	// VBI data? if yes, use myth loop
-	// STARTBTYTESLENGTH is 1MB, if the file is shorter we will never detect
+	// STARTBYTESLENGTH is 1MB, if the file is shorter we will never detect
 	// it as a mythTV file
 	if (ctx->startbytes_avail==STARTBYTESLENGTH)
 	{
@@ -409,18 +418,25 @@ typedef struct ccx_stream_mp4_box
 * An asterisk (*) marks a mandatory box for a regular file.
 * Box types that are on the second level or deeper are omitted.
 */
-ccx_stream_mp4_box ccx_stream_mp4_boxes[11] = {
+ccx_stream_mp4_box ccx_stream_mp4_boxes[16] = {
 		{ "ftyp", 6 }, // File type and compatibility*
 		{ "pdin", 1 }, // Progressive download information
 		{ "moov", 5 }, // Container for all metadata*
 		{ "moof", 4 }, // Movie fragment
-		{ "mfra", 3 }, // Movie fragment random access
+		{ "mfra", 1 }, // Movie fragment random access
 		{ "mdat", 2 }, // Media data container
 		{ "free", 1 }, // Free space
 		{ "skip", 1 }, // Free space
 		{ "meta", 1 }, // Metadata
 		{ "wide", 1 }, // For boxes that are > 2^32 bytes (https://developer.apple.com/library/mac/documentation/QuickTime/QTFF/QTFFChap1/qtff1.html)
-		{ "void", 1 }  // Unknown where this is from/for, assume free space.
+		{ "void", 1 },  // Unknown where this is from/for, assume free space.
+
+		// new ones in standard ISO/IEC 14496-12:2015
+		{ "meco", 1 }, // additional metadata container
+		{ "styp", 1 }, // segment type
+		{ "sidx", 1 }, // segment index
+		{ "ssix", 1 }, // subsegment index
+		{ "prft", 1 }  // producer reference time
 };
 
 /*
@@ -432,7 +448,7 @@ ccx_stream_mp4_box ccx_stream_mp4_boxes[11] = {
  */
 int isValidMP4Box(unsigned char *buffer, size_t position, size_t *nextBoxLocation, int *boxScore)
 {
-	for (int idx = 0; idx < 11; idx++)
+	for (int idx = 0; idx < 16; idx++)
 	{
 		if (buffer[position + 4] == ccx_stream_mp4_boxes[idx].boxType[0] && buffer[position + 5] == ccx_stream_mp4_boxes[idx].boxType[1] &&
 				buffer[position + 6] == ccx_stream_mp4_boxes[idx].boxType[2] && buffer[position + 7] == ccx_stream_mp4_boxes[idx].boxType[3]){

src/lib_ccx/telxcc.c

@@ -34,6 +34,7 @@ Werner Brückner -- Teletext in digital television
 #include "teletext.h"
 #include <signal.h>
 #include "activity.h"
+#include "ccx_encoders_helpers.h"
 
 #ifdef __MINGW32__
 // switch stdin and all normal files into binary mode -- needed for Windows
@@ -52,6 +53,7 @@ typedef struct {
 	uint64_t show_timestamp; // show at timestamp (in ms)
 	uint64_t hide_timestamp; // hide at timestamp (in ms)
 	uint16_t text[25][40]; // 25 lines x 40 cols (1 screen/page) of wide chars
+	uint8_t g2_char_present[25][40]; // 0- Supplementary G2 character set NOT used at this position 1-Supplementary G2 character set used at this position
 	uint8_t tainted; // 1 = text variable contains any data
 } teletext_page_t;
 
@@ -124,6 +126,9 @@ struct TeletextCtx
 	uint8_t using_pts;
 	int64_t delta;
 	uint32_t t0;
+
+	int sentence_cap;//Set to 1 if -sc is passed
+	int new_sentence;
 };
 typedef enum
 {
@@ -196,8 +201,9 @@ typedef enum
 	GREEK,
 	ARABIC,
 	HEBREW
-} g0_charsets_t;
+} g0_charsets_type;
 
+g0_charsets_type default_g0_charset; 
 
 // Note: All characters are encoded in UCS-2
 
@@ -344,7 +350,7 @@ const uint16_t G2[1][96] = {
 //	},
 //	{ // Greek G2 Supplementary Set
 //	},
-//	{ // Arabic G2 Supplementary Set
+//	{ // Arabic G2 Supplementary Set
 //	}
 };
 
@@ -503,6 +509,26 @@ uint32_t unham_24_18(uint32_t a)
 	return (a & 0x000004) >> 2 | (a & 0x000070) >> 3 | (a & 0x007f00) >> 4 | (a & 0x7f0000) >> 5;
 }
 
+//Default G0 Character Set 
+void set_g0_charset(uint32_t triplet)
+{
+	// ETS 300 706, Table 32
+	if((triplet & 0x3c00) == 0x1000)
+	{
+		if((triplet & 0x0380) == 0x0000)
+			default_g0_charset = CYRILLIC1;
+		else if((triplet & 0x0380) == 0x0200)
+			default_g0_charset = CYRILLIC2;
+		else if((triplet & 0x0380) == 0x0280)
+			default_g0_charset = CYRILLIC3;
+		else
+			default_g0_charset = LATIN;	
+	}
+	else
+		default_g0_charset = LATIN;
+}
+
+// Latin National Subset Selection
 void remap_g0_charset(uint8_t c)
 {
 	if (c != primary_charset.current)
@@ -523,7 +549,6 @@ void remap_g0_charset(uint8_t c)
 }
 
 
-
 // wide char (16 bits) to utf-8 conversion
 void ucs2_to_utf8(char *r, uint16_t ch)
 {
@@ -559,7 +584,7 @@ uint16_t telx_to_ucs2(uint8_t c)
 
 	uint16_t r = c & 0x7f;
 	if (r >= 0x20)
-		r = G0[LATIN][r - 0x20];
+		r = G0[default_g0_charset][r - 0x20];
 	return r;
 }
 
@@ -568,6 +593,36 @@ uint16_t bcd_page_to_int (uint16_t bcd)
 	return ((bcd&0xf00)>>8)*100 + ((bcd&0xf0)>>4)*10 + (bcd&0xf);
 }
 
+void telx_case_fix (struct TeletextCtx *context)
+{
+	//Capitalizing first letter of every sentence
+	int line_len = strlen(context->page_buffer_cur);
+	for(int i = 0; i < line_len; i++)
+	{
+		switch(context->page_buffer_cur[i])
+		{
+			case ' ':
+			//case 0x89: // This is a transparent space
+			case '-':
+				break;
+			case '.': // Fallthrough
+			case '?': // Fallthrough
+			case '!':
+			case ':':
+				context->new_sentence = 1;
+				break;
+			default:
+				if (context->new_sentence)
+					context->page_buffer_cur[i] = cctoupper(context->page_buffer_cur[i]);
+				else
+					context->page_buffer_cur[i] = cctolower(context->page_buffer_cur[i]);
+				context->new_sentence = 0;
+				break;
+		}
+	}
+	telx_correct_case(context->page_buffer_cur);
+}
+
 void telxcc_dump_prev_page (struct TeletextCtx *ctx, struct cc_subtitle *sub)
 {
 	char info[4]; 
@@ -871,6 +926,8 @@ void process_page(struct TeletextCtx *ctx, teletext_page_t *page, struct cc_subt
 			}
 			break;
 		default:
+			if (ctx->sentence_cap)
+				telx_case_fix(ctx);
 			add_cc_sub_text(sub, ctx->page_buffer_cur, page->show_timestamp,
 				page->hide_timestamp + 1, NULL, "TLT", CCX_ENC_UTF_8);
 	}
@@ -896,6 +953,7 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 
 	if (y == 0)
 	{
+
 		// CC map
 		uint8_t i = (unham_8_4(packet->data[1]) << 4) | unham_8_4(packet->data[0]);
 		uint8_t flag_subtitle = (unham_8_4(packet->data[5]) & 0x08) >> 3;
@@ -953,9 +1011,19 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 		if (page_number != tlt_config.page)
 			return;
 
+
 		// Now we have the begining of page transmission; if there is page_buffer pending, process it
 		if (ctx->page_buffer.tainted == YES)
 		{
+			// Convert telx to UCS-2 before processing
+			for(uint8_t yt = 1; yt <= 23; ++yt)
+			{
+				for(uint8_t it = 0; it < 40; it++)
+				{
+					if (ctx->page_buffer.text[yt][it] != 0x00 && ctx->page_buffer.g2_char_present[yt][it] == 0)
+						ctx->page_buffer.text[yt][it] = telx_to_ucs2(ctx->page_buffer.text[yt][it]);
+				}
+			}
 			// it would be nice, if subtitle hides on previous video frame, so we contract 40 ms (1 frame @25 fps)
 			ctx->page_buffer.hide_timestamp = timestamp - 40;
 			if (ctx->page_buffer.hide_timestamp > timestamp)
@@ -963,19 +1031,20 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 				ctx->page_buffer.hide_timestamp = 0;
 			}
 			process_page(ctx, &ctx->page_buffer, sub);
-
 		}
 
 		ctx->page_buffer.show_timestamp = timestamp;
 		ctx->page_buffer.hide_timestamp = 0;
 		memset(ctx->page_buffer.text, 0x00, sizeof(ctx->page_buffer.text));
+		memset(ctx->page_buffer.g2_char_present, 0x00, sizeof(ctx->page_buffer.g2_char_present));
 		ctx->page_buffer.tainted = NO;
 		ctx->receiving_data = YES;
-		primary_charset.g0_x28 = UNDEF;
-
-		c = (primary_charset.g0_m29 != UNDEF) ? primary_charset.g0_m29 : charset;
-		remap_g0_charset(c);
-
+		if(default_g0_charset == LATIN) // G0 Character National Option Sub-sets selection required only for Latin Character Sets
+		{
+			primary_charset.g0_x28 = UNDEF;
+			c = (primary_charset.g0_m29 != UNDEF) ? primary_charset.g0_m29 : charset;
+			remap_g0_charset(c);
+		}
 		/*
 		// I know -- not needed; in subtitles we will never need disturbing teletext page status bar
 		// displaying tv station name, current time etc.
@@ -995,7 +1064,7 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 		for (uint8_t i = 0; i < 40; i++)
 		{
 			if (ctx->page_buffer.text[y][i] == 0x00)
-				ctx->page_buffer.text[y][i] = telx_to_ucs2(packet->data[i]);
+				ctx->page_buffer.text[y][i] = packet->data[i];
 		}
 		ctx->page_buffer.tainted = YES;
 	}
@@ -1041,7 +1110,23 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 			if ((mode == 0x0f) && (row_address_group == NO))
 			{
 				x26_col = address;
-				if (data > 31) ctx->page_buffer.text[x26_row][x26_col] = G2[0][data - 0x20];
+				if (data > 31) 
+				{
+					ctx->page_buffer.text[x26_row][x26_col] = G2[0][data - 0x20];
+					ctx->page_buffer.g2_char_present[x26_row][x26_col] = 1;
+				}
+			}
+
+			// ETS 300 706 v1.2.1, chapter 12.3.4, Table 29: G0 character without diacritical mark (display '@' instead of '*')
+			if ((mode == 0x10) && (row_address_group == NO))
+			{
+				x26_col = address;
+				if (data == 64) // check for @ symbol
+				{
+					remap_g0_charset(0);
+					ctx->page_buffer.text[x26_row][x26_col] = 0x40;
+				}
+
 			}
 
 			// ETS 300 706, chapter 12.3.1, table 27: G0 character with diacritical mark
@@ -1058,6 +1143,8 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 				// other
 				else
 					ctx->page_buffer.text[x26_row][x26_col] = telx_to_ucs2(data);
+				
+				ctx->page_buffer.g2_char_present[x26_row][x26_col] = 1;
 			}
 		}
 	}
@@ -1081,9 +1168,14 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 			{
 				// ETS 300 706, chapter 9.4.2: Packet X/28/0 Format 1 only
 				if ((triplet0 & 0x0f) == 0x00)
-				{
-					primary_charset.g0_x28 = (triplet0 & 0x3f80) >> 7;
-					remap_g0_charset(primary_charset.g0_x28);
+				{					
+					// ETS 300 706, Table 32
+					set_g0_charset(triplet0); // Deciding G0 Character Set
+					if(default_g0_charset == LATIN)
+					{
+						primary_charset.g0_x28 = (triplet0 & 0x3f80) >> 7;
+						remap_g0_charset(primary_charset.g0_x28);
+					}
 				}
 			}
 		}
@@ -1098,7 +1190,7 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 			// ETS 300 706, chapter 9.5.1: Packet M/29/0
 			// ETS 300 706, chapter 9.5.3: Packet M/29/4
 			uint32_t triplet0 = unham_24_18((packet->data[3] << 16) | (packet->data[2] << 8) | packet->data[1]);
-
+	
 			if (triplet0 == 0xffffffff)
 			{
 				// invalid data (HAM24/18 uncorrectable error detected), skip group
@@ -1110,11 +1202,15 @@ void process_telx_packet(struct TeletextCtx *ctx, data_unit_t data_unit_id, tele
 				// ETS 300 706, table 13: Coding of Packet M/29/4
 				if ((triplet0 & 0xff) == 0x00)
 				{
-					primary_charset.g0_m29 = (triplet0 & 0x3f80) >> 7;
-					// X/28 takes precedence over M/29
-					if (primary_charset.g0_x28 == UNDEF)
+					set_g0_charset(triplet0);
+					if(default_g0_charset == LATIN)
 					{
-						remap_g0_charset(primary_charset.g0_m29);
+						primary_charset.g0_m29 = (triplet0 & 0x3f80) >> 7;
+						// X/28 takes precedence over M/29
+						if (primary_charset.g0_x28 == UNDEF)
+						{
+							remap_g0_charset(primary_charset.g0_m29);
+						}
 					}
 				}
 			}
@@ -1222,7 +1318,7 @@ int tlt_print_seen_pages(struct lib_cc_decode *dec_ctx)
 	}
 	return CCX_OK;
 }
-int tlt_process_pes_packet(struct lib_cc_decode *dec_ctx, uint8_t *buffer, uint16_t size, struct cc_subtitle *sub)
+int tlt_process_pes_packet(struct lib_cc_decode *dec_ctx, uint8_t *buffer, uint16_t size, struct cc_subtitle *sub, int sentence_cap)
 {
 	uint64_t pes_prefix;
 	uint8_t pes_stream_id;
@@ -1232,6 +1328,7 @@ int tlt_process_pes_packet(struct lib_cc_decode *dec_ctx, uint8_t *buffer, uint1
 	uint32_t t = 0;
 	uint16_t i;
 	struct TeletextCtx *ctx = dec_ctx->private_data;
+	ctx->sentence_cap = sentence_cap;
 
 	if(!ctx)
 	{
@@ -1406,6 +1503,9 @@ void* telxcc_init(void)
 	ctx->delta = 0;
 	ctx->t0 = 0;
 
+	ctx->sentence_cap = 0;
+	ctx->new_sentence = 0;
+
 	return ctx;
 }
 
@@ -1429,6 +1529,15 @@ void telxcc_close(void **ctx, struct cc_subtitle *sub)
 		// output any pending close caption
 		if (ttext->page_buffer.tainted == YES)
 		{
+			// Convert telx to UCS-2 before processing
+			for(uint8_t yt = 1; yt <= 23; ++yt)
+			{
+				for(uint8_t it = 0; it < 40; it++)
+				{
+					if (ttext->page_buffer.text[yt][it] != 0x00 && ttext->page_buffer.g2_char_present[yt][it] == 0)
+						ttext->page_buffer.text[yt][it] = telx_to_ucs2(ttext->page_buffer.text[yt][it]);
+				}
+			}
 			// this time we do not subtract any frames, there will be no more frames
 			ttext->page_buffer.hide_timestamp = ttext->last_timestamp;
 			process_page(ttext, &ttext->page_buffer, sub);
@@ -1440,4 +1549,4 @@ void telxcc_close(void **ctx, struct cc_subtitle *sub)
 	freep(&ttext->ucs2_buffer_cur);
 	freep(&ttext->page_buffer_cur);
 	freep(ctx);
-}
+}

src/lib_ccx/ts_functions.c

@@ -140,7 +140,7 @@ int ts_readpacket(struct ccx_demuxer* ctx, struct ts_payload *payload)
 		if (result != 4)
 		{
 			if (result>0)
-				mprint("Premature end of file!\n");
+				mprint("Premature end of file (incomplete TS packer header, expected 4 bytes, got %lld).\n",result);
 			return CCX_EOF;
 		}
 	}
@@ -150,7 +150,7 @@ int ts_readpacket(struct ccx_demuxer* ctx, struct ts_payload *payload)
 	if (result != 188)
 	{
 		if (result > 0)
-			mprint("Premature end of file!\n");
+			mprint("Premature end of file - Transport Stream packet is incomplete (expected 188 bytes, got %lld).\n", result);
 		return CCX_EOF;
 	}
 
@@ -518,8 +518,8 @@ int copy_payload_to_capbuf(struct cap_info *cinfo, struct ts_payload *payload)
 		if(payload->start[0] != 0x00 || payload->start[1] != 0x00 || 
 			payload->start[2] != 0x01)
 		{
-			mprint("Missing PES header!\n");
-			dump(CCX_DMT_GENERIC_NOTICES, payload->start, payload->length, 0, 0);
+			mprint("Notice: Missing PES header\n");
+			dump(CCX_DMT_DUMPDEF, payload->start, payload->length, 0, 0);
 			cinfo->saw_pesstart = 0;
 			errno = EINVAL;
 			return -1;

src/lib_ccx/ts_tables.c

@@ -108,6 +108,12 @@ int parse_PMT (struct ccx_demuxer *ctx, unsigned char *buf, int len,  struct pro
 	crc = (*(int32_t*)(sbuf+olen-4));
 	table_id = buf[0];
 
+	/* TO-DO: We're currently parsing the PMT making assumptions that there's only one section with table_id=2,
+	but that doesn't have to be the case. There's a sample (friends_tbs.ts) that shows a previous section with
+	table_id = 0xc0. I can't find any place that says that 0xc0 (Program Information Table) must come before
+	table_id = 2, so we should process sections in any order.
+	Check https://github.com/CCExtractor/ccextractor/issues/385 for more info
+	*/
 	if (table_id == 0xC0)
 	{
 		/*
@@ -116,7 +122,12 @@ int parse_PMT (struct ccx_demuxer *ctx, unsigned char *buf, int len,  struct pro
 		 * PROGRAM INFORMATION Table found in PMT
 		 */
 		dbg_print(CCX_DMT_PARSE, "PMT: PROGRAM INFORMATION Table need implementation");
-		return 0;
+		// For now, just parse its length and remove it from the buffer
+		unsigned c0length = (buf[1] << 8 | buf[2]) & 0xFFF; // 12 bytes
+		dbg_print(CCX_DMT_PARSE, "Program information table length: %u", c0length);
+		memmove(buf, buf + c0length + 3, len - c0length -3); // First 3 bytes are for the table_id and the length, don't count
+		table_id = buf[0];
+		// return 0;
 	}
 	else if (table_id == 0xC1)
 	{

src/lib_ccx/ts_tables_epg.c

@@ -81,7 +81,7 @@ void EPG_DVB_calc_start_time(struct EPG_event *event, uint64_t time)
 		y = y + k + 1900;
 		m = m - 1 - k*12;
 
-		sprintf(event->start_time_string, "%02ld%02ld%02ld%06llx +0000",y,m,d,time&0xffffff);
+		sprintf(event->start_time_string, "%02ld%02ld%02ld%06"PRIu64 "+0000",y,m,d,time&0xffffff);
 	}
 }
 
@@ -300,6 +300,11 @@ void EPG_output(struct lib_ccx_ctx *ctx)
 	memcpy(filename, ctx->basefilename, strlen(ctx->basefilename)+1);
 	strcat(filename, "_epg.xml");
 	f = fopen(filename, "w");
+	if(!f)
+	{
+		dbg_print (CCX_DMT_GENERIC_NOTICES, "\rUnable to open %s\n", filename);
+		return;
+	}
 	freep(&filename);
 
 	fprintf(f, "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<!DOCTYPE tv SYSTEM \"xmltv.dtd\">\n\n<tv>\n");
@@ -1091,37 +1096,19 @@ void parse_EPG_packet(struct lib_ccx_ctx *ctx)
 
 // Free all memory used for EPG parsing
 void EPG_free(struct lib_ccx_ctx *ctx)
-{
-	int i = 0, j;
-	if(ccx_options.xmltv==2 || ccx_options.xmltv==3 || ccx_options.send_to_srv)
+{	
+	if(ctx->epg_inited)
 	{
-		if (ccx_options.send_to_srv)
-			EPG_output_net(ctx);
-		else
-			EPG_output_live(ctx);
-	}
-	for (i = 0; i < TS_PMT_MAP_SIZE; i++)
-	{
-		for(j = 0; j < ctx->eit_programs[i].array_len; j++)
+		if(ccx_options.xmltv==2 || ccx_options.xmltv==3 || ccx_options.send_to_srv)
 		{
-			if(ctx->eit_programs[i].epg_events[j].has_simple)
-			{
-				free(ctx->eit_programs[i].epg_events[j].event_name);
-				free(ctx->eit_programs[i].epg_events[j].text);
-			}
-			if(ctx->eit_programs[i].epg_events[j].extended_text!=NULL)
-				free(ctx->eit_programs[i].epg_events[j].extended_text);
-			if(ctx->eit_programs[i].epg_events[j].num_ratings>0)
-				free(ctx->eit_programs[i].epg_events[j].ratings);
-			if(ctx->eit_programs[i].epg_events[j].num_categories>0)
-				free(ctx->eit_programs[i].epg_events[j].categories);
+			if (ccx_options.send_to_srv)
+				EPG_output_net(ctx);
+			else
+				EPG_output_live(ctx);
 		}
-		ctx->eit_programs[i].array_len=0;
-	}
-	
-	for (i = 0; i < 0xfff; i++)
-	{
-		if(ctx->epg_buffers[i].buffer!=NULL)
-			free(ctx->epg_buffers[i].buffer);
 	}
+	free(ctx->epg_buffers);
+	free(ctx->eit_programs);
+	free(ctx->eit_current_events);
+	free(ctx->ATSC_source_pg_map);
 }

src/lib_ccx/utility.c

@@ -1,10 +1,13 @@
+#include <stdio.h>
 #include <signal.h>
+#include <string.h>
 #include "lib_ccx.h"
 #include "ccx_common_option.h"
 #include "activity.h"
+#include "utility.h"
 
 int temp_debug = 0; // This is a convenience variable used to enable/disable debug on variable conditions. Find references to understand.
-
+volatile sig_atomic_t change_filename_requested = 0;
 
 
 static uint32_t crc32_table [] = {
@@ -371,7 +374,91 @@ void m_signal(int sig, void (*func)(int))
 
 	return;
 }
+
+void create_signal(void)
+{
+        if (signal(SIGUSR1, signal_handler) == SIG_ERR)
+		mprint("Can't catch SIGUSR1.\n");
+}
+
+void signal_handler(int sig_type)
+{
+	int ret;
+
+        if (sig_type == SIGUSR1)
+        {
+        	mprint("Caught SIGUSR1. Filename Change Requested\n");
+        	change_filename_requested = 1;
+        }
+}
 #endif
+
+struct encoder_ctx *change_filename(struct encoder_ctx *enc_ctx)
+{
+	if(change_filename_requested == 0)
+	{
+		return enc_ctx;
+	}
+	struct encoder_ctx *temp_encoder = malloc(sizeof(struct encoder_ctx));
+	*temp_encoder = *enc_ctx;
+	if (enc_ctx->out->fh != -1)
+	{
+		if (enc_ctx->out->fh > 0)
+			close(enc_ctx->out->fh);
+		enc_ctx->out->fh=-1;
+		int iter;
+		char str_number[15];
+		char *current_name = malloc(sizeof(char)*(strlen(enc_ctx->out->filename)+10));
+		strcpy(current_name,enc_ctx->out->filename);
+		mprint ("Creating %s\n", enc_ctx->out->filename);
+		if(enc_ctx->out->renaming_extension)
+		{
+			strcat(current_name,".");
+			sprintf(str_number, "%d", enc_ctx->out->renaming_extension);
+			strcat(current_name,str_number);
+		}
+		enc_ctx->out->renaming_extension++;
+		for (iter = enc_ctx->out->renaming_extension; iter >= 1; iter--)
+		{
+			int ret;
+			char new_extension[6];
+			sprintf(new_extension, ".%d", iter); 
+			char *newname = malloc(sizeof(char)*(strlen(enc_ctx->out->filename)+10));
+			strcpy(newname,enc_ctx->out->filename);
+			strcat(newname,new_extension);
+			ret = rename(current_name, newname);
+			if(ret)
+			{
+				mprint("Failed to rename the file\n");
+
+			}
+			mprint ("Creating %s\n", newname);
+			strcpy(current_name,enc_ctx->out->filename);
+			
+			if(iter-2>0)
+			{
+				strcat(current_name,".");
+				sprintf(str_number, "%d", iter-2);
+				strcat(current_name,str_number);
+			}
+			free(newname);
+		
+		}
+
+		enc_ctx->out->fh = open(enc_ctx->out->filename, O_RDWR | O_CREAT | O_TRUNC | O_BINARY, S_IREAD | S_IWRITE);
+		free(current_name);
+		if (enc_ctx->out->fh == -1)
+		{
+			mprint("Failed to create a new rotation file\n");
+			return temp_encoder;
+		}
+		free(temp_encoder);		
+		change_filename_requested = 0;
+		return enc_ctx;
+		
+	}
+	return temp_encoder;
+}
 char *get_basename(char *filename)
 {
 	char *c;
@@ -496,6 +583,14 @@ size_t utf16_to_utf8(unsigned short utf16_char, unsigned char *out)
 	return 0;
 }
 
+LLONG change_timebase(LLONG val, struct ccx_rational cur_tb, struct ccx_rational dest_tb)
+{
+	/* val = (value * current timebase) / destination timebase */
+	val = val * cur_tb.num * dest_tb.den;
+	val = val / ( cur_tb.den * dest_tb.num);
+	return val;
+}
+
 #ifdef _WIN32
 char *strndup(const char *s, size_t n)
 {
@@ -510,6 +605,6 @@ char *strndup(const char *s, size_t n)
 }
 char *strtok_r(char *str, const char *delim, char **saveptr)
 {
-	strtok_s(str, delim, saveptr);
+	return strtok_s(str, delim, saveptr);
 }
 #endif //_WIN32

src/lib_ccx/utility.h

@@ -1,6 +1,6 @@
 #ifndef CC_UTILITY_H
 #define CC_UTILITY_H
-
+#include <signal.h>
 #ifndef _WIN32
 	#include <arpa/inet.h>
 #endif
@@ -14,7 +14,13 @@
 
 #define CCX_NOPTS	((int64_t)UINT64_C(0x8000000000000000))
 
+struct ccx_rational
+{
+	int num;
+	int den;
+};
 extern int temp_debug;
+volatile extern sig_atomic_t change_filename_requested;
 void init_boundary_time (struct ccx_boundary_time *bt);
 void print_error (int mode, const char *fmt, ...);
 int stringztoms (const char *s, struct ccx_boundary_time *bt);
@@ -23,7 +29,9 @@ char *get_file_extension(enum ccx_output_format write_format);
 char *create_outfilename(const char *basename, const char *suffix, const char *extension);
 int verify_crc32(uint8_t *buf, int len);
 size_t utf16_to_utf8(unsigned short utf16_char, unsigned char *out);
+LLONG change_timebase(LLONG val, struct ccx_rational cur_tb, struct ccx_rational dest_tb);
 
+void dump (LLONG mask, unsigned char *start, int l, unsigned long abs_start, unsigned clear_high_bit);
 #ifdef _WIN32
 char *strndup(const char *s, size_t n);
 char *strtok_r(char *str, const char *delim, char **saveptr);

src/lib_hash/README

@@ -0,0 +1,277 @@
+VERSION:
+
+This is version 1.0.1 RELEASE
+
+While this is my "release" version, due to lack of additional
+official test vectors against which to verify this implementation's
+correctness, beware that there may be implementation bugs.  Also,
+it has not yet been tested on very many other architectures,
+big-endian machines in particular.
+
+
+LICENSE:
+
+This implementation is released freely under an open-source BSD
+license which appears at the top of each source code file.
+
+
+WHAT IT IS:
+
+The files sha2.h and sha2.c implement the SHA-256, SHA-384, and SHA-512
+hash algorithms as described in the PDF document found at the following
+web address:
+
+  http://csrc.nist.gov/cryptval/shs/sha256-384-512.pdf
+
+The interface is similar to the interface to SHA-1 found in the OpenSSL
+library.
+
+The file sha2prog.c is a simple program that accepts input from either
+STDIN or reads one or more files specified on the command line, and then
+generates the specified hash (either SHA-256, SHA-384, SHA-512, or any
+combination thereof, including all three at once).
+
+
+LIMITATIONS:
+
+This implementation has several limitations:
+
+ * Input data is only accepted in octet-length increments.  No sub-byte
+   data is handled.  The NIST document describes how to handle sub-byte
+   input data, but for ease of implementation this version will only
+   accept message data in multiples of bytes.
+ * This implementation utilizes 64-bit integer data types.  If your
+   system and compiler does not have a 64-bit integer data type, this
+   implementation will not work.
+ * Because of the use of 64-bit operations, many 32-bit architectures
+   that do have 64-bit data types but do operations most efficiently
+   on 32-bit words, this implementation may be slower than an
+   implementation designed to use only 32-bit words (emulating the
+   64-bit operations).
+ * On platforms with 128-bit integer data types, the SHA-384 and SHA-512
+   bit counters used by this implementation might be better off using
+   the 128-bit type instead of simulating it with two 64-bit integers.
+ * This implementation was written in C in hopes of portability and for
+   the fun of it during my spare time.  It is probably not the most
+   efficient or fastest C implementation.  I welcome suggestions,
+   however, that suggest ways to speed things up without breaking
+   portability.  I also welcome suggestions to improve portability.
+ * As mentioned above, this code has NOT been thoroughly tested.
+   This is perhaps the most severe limitation.
+
+
+BEFORE YOU COMPILE (OPTIONS):
+
+Each of the options described below may either be defined in the sha2.h
+header file (or in the sha2.c file in some cases), or on the command
+line at compile time if your compiler supports such things.  For
+example:
+
+  #define SHA2_USE_INTTYPES_H
+  #define SHA2_UNROLL_TRANSFORM
+
+Or:
+
+  cc -c -DSHA2_UNROLL_TRANSFORM sha2.c
+  cc -c -DBYTE_ORDER=4321 -DBIG_ENDIAN=4321 sha2.c
+
+Here are the available options.  Read on below for a description of
+each one:
+
+  SHA2_USE_INTTYPES_H
+  SHA2_USE_MEMSET_MEMCPY/SHA2_USE_BZERO_BCOPY
+  SHA2_UNROLL_TRANSFORM
+  BYTE_ORDER (LITTLE_ENDIAN/BIG_ENDIAN)
+
+* SHA2_USE_INTTYPES_H option:
+By default, this code uses u_intXX_t data types for 8 bit, 32 bit, and
+64 bit unsigned integer type definitions.  Most BSD systems define these,
+as does Linux.  However, some (like Compaq's Tru64 Unix) may instead
+use uintXX_t data types as defined by recent ANSI C standards and as
+included in the inttypes.h header file.  Those wanting to use inttypes.h
+need to define this either in sha.h or at compile time.
+
+On those systems where NEITHER definitions are available, you will need
+to edit both sha2.h and sha2.c and define things by hand in the appropriate
+sections.
+
+* BYTE_ORDER definitions:
+This code assumes that BYTE_ORDER will be defined by the system during
+compile to either equal LITTLE_ENDIAN or BIG_ENDIAN.  If your system
+does not define these, you may need to define them by hand in the sha.c
+file according to the byte ordering conventions of your system.
+
+* SHA2_USE_MEMSET_MEMCPY or SHA2_USE_BZERO_BCOPY
+The code in sha2.c can use either memset()/memcpy() for memory block
+operations, or bzero()/mcopy().  If you define neither of these, the
+code will default to memset()/memcpy().  You can define either at the
+command line or in sha2.h or in sha2.c.
+
+* SHA2_UNROLL_TRANSFORM
+By defining this either on the command line or in sha2.h or sha2.c,
+the code will use macros to partially "unroll" the SHA transform
+function.  This usually generates bigger executables.  It CAN (but
+not necessarily WILL) generate faster code when you tell your compiler
+to optimize things.  For example, on the FreeBSD and Linux x86 systems
+I tested things on (using gcc), when I optimized with just -O2 and
+unrolled the transform, the hash transform was faster by 15-30%.  On
+these same systems, if I did NO optimization, the unrolled transform
+was SLOWER, much slower (I'm guessing because the code was breaking
+the cache, but I'm not sure).  Your mileage may vary.
+
+
+PORTABILITY:
+
+The code in sha2.c and sha2.h is intended to be portable.  It may
+require that you do a few #definitions in the .h file.  I've successfully
+compiled and tested the sha2.c and sha2.h code on Apple's OS X (on
+a PPC), FreeBSD 4.1.1 on Intel, Linux on Intel, FreeBSD on the Alpha,
+and even under Windows98SE using Metrowerks C.  The utility/example
+programs (sha2prog.c, sha2test.c, and sha2speed.c) will very likely
+have more trouble in portability since they do I/O.
+
+To get sha2.c/sha2.h working under Windows, I had to define
+SHA2_USE_INTTYPES_H, BYTE_ORDER, LITTLE_ENDIAN, and had to comment
+out the include of <sys/types.h> in sha2.h.  With a bit more work
+I got the test program to run and verified that all the test
+cases passed.
+
+
+SUGGESTIONS/BUG FIXES:
+
+If you make changes to get it working on other architectures, if you fix
+any bugs, or if you make changes that improve this implementation's
+efficiency that would be relatively portable and you're willing to release
+your changes under the same license, please send them to me for possible
+inclusion in future versions.
+
+If you know where I can find some additional test vectors, please let me
+know.
+
+
+CHANGE HISTORY:
+
+0.8 to 0.9 	- Fixed spelling errors, changed to u_intXX_t type usage,
+		  removed names from prototypes, added prototypes to sha2.c,
+		  and a few things I can't recall.
+
+0.9 to 0.9.5	- Add a new define in sha2.c that permits one to compile
+		  it to either use memcpy()/memset() or bcopy()/bzero()
+		  for memory block copying and zeroing.  Added support
+		  for unrolled SHA-256/384/512 transform loops.  Just
+		  compile with SHA2_UNROLL_TRANSFORM to enable.  It takes
+		  longer to compile, but I hope it is a bit faster.  I
+		  need to do some test to see whether or not it is. Oh,
+		  in sha2.c, you either need to define SHA2_USE_BZERO_BCOPY
+		  or SHA2_USE_MEMSET_MEMCPY to choose which way you want
+		  to compile.  *Whew*  It's amazing how quickly something
+		  simple starts to grow more complex even in the span of
+		  just a few hours.  I didn't really intend to do this much.
+0.9.5 to 0.9.6  - Added a test program (sha2test) which tests against several
+                  known test vectors.  WARNING: Some of the test output
+                  hashes are NOT from NIST's documentation and are the
+                  output of this implementation and so may be incorrect.
+0.9.6 to 0.9.7  - Fixed a bug that could cause invalid output in certain
+		  cases and added an assumed scenario where zero-length
+		  data is hashed.  Also changed the rotation macros to use
+		  a temporary variable as this reduces the number of operations.
+		  When data is fed in blocks of the right length, copying of
+		  data is reduced in this version.  Added SHAYXZ_Data()
+		  functions for ease of hashing a set of data.  Added another
+		  file sha2speed.c for doing speed testing.  Added another test
+		  vector with a larger data size (16KB).  Fixed u_intXX_t and
+		  uintXX_t handling by adding a define for SHA2_USE_INTTYPES_H
+		  as well as made a few other minor changes to get rid of
+		  warnings when compiling on Compaq's Tru64 Unix.
+0.9.7 to 0.9.8  - The bug fix in 0.9.7 was incomplete and in some cases made
+                  things worse.  I believe that 0.9.8 fixes the bug completely
+                  so that output is correct.  I cannot verify this, however,
+                  because of the lack of test vectors against which to do such
+                  verification.  All versions correctly matched the very few
+                  NIST-provided vectors, but unfortunately the bug only
+                  appeared in longer message data sets.
+0.9.8 to 0.9.9  - Fixed some really bad typos and mistakes on my part that
+                  only affected big-endian systems.  I didn't have direct
+                  access for testing before this version.  Thanks to
+                  Lucas Marshall for giving me access to his OS X system.
+0.9.9 to 1.0.0b1  Added a few more test samples and made a few changes to
+                  make things easier compiling on several other platforms.
+                  Also I experimented with alternate macro definitions
+                  in the SHA2_UNROLL_TRANSFORM version (see sha2.slower.c)
+                  and eliminated the T1 temporary variable (the compiler
+                  would of course still use internal temporary storage
+                  during expression evaluation, but I'd hoped the compiler
+                  would be more efficient), but unfortunately under FreeBSD
+                  4.1.1-STABLE on an x86 platform, the change slowed things
+                  down.
+1.0.0b1 to 1.0 RELEASE  Fixed an off-by-one implementation bug that affected
+                  SHA-256 when hashed data length L = 55 + 64 * X where X is
+                  either zero or a positive integer, and another (basically
+                  the same bug) bug in SHA-384 and SHA-512 that showed up when
+                  hashed data lengths L = 111 + 128 * X.  Thanks to Rogier
+		  van de Pol for sending me test data that revealed the bug.
+                  The fix was very simple (just two tiny changes).  Also,
+                  I finally put the files into RCS so future changes will be
+                  easier to manage.  The sha2prog.c file was rewritten to
+                  be more useful to me, and I got rid of the old C testing
+                  program and now use a perl script with a subdirectory full
+                  of test data.  It's a more flexible test system.
+
+1.0 to 1.0.1    - Specified the specific *_CTX structure in the MEMSET_BZERO
+                  macro doing clean-up after hashing.  This should eliminate
+                  some warnings using Clang in version 3.0 (trunk 135348).
+                  Thanks, Stephane Leon for reporting this.
+
+
+LATEST VERSION:
+
+The latest version and documentation (if any ;) should always be available
+on the web at:
+
+  http://www.aarongifford.com/computers/sha.html
+
+
+CONTACT ME:
+
+I can be reached via email at:
+
+  Aaron Gifford   <m e @ a a r o n g i f f o r d . c o m>
+
+Please don't send support questions.  I don't have the time to answer and
+they'll probably be ignored.  Bug fixes, or patches that add something useful
+will be gratefully accepted, however.
+
+If you use this implementation, I would enjoy getting a brief email message
+letting me know who you are and what use to which it is being put.  There
+is no requirement to do so.  I just think it would be fun.
+
+
+EXAMPLES:
+
+Here's an example of compiling and using the sha2 program (in this example
+I build it using the unrolled transform version with -O2 optimizations),
+and then running the perl testing script:
+
+  cc -O2 -DSHA2_UNROLL_TRANSFORM -Wall -o sha2 sha2prog.c sha2.c
+  % ./sha2test.pl
+
+  [most of the perl script output deleted for brevity]
+
+  ===== RESULTS (18 VECTOR DATA FILES HASHED) =====
+
+  HASH TYPE       NO. OF TESTS    PASSED  FAILED
+  ---------       ------------    ------  ------
+  SHA-256                   18        18       0
+  SHA-384                   18        18       0
+  SHA-512                   18        18       0
+  ----------------------------------------------
+  TOTAL:                    54        54       0
+
+  NO ERRORS!  ALL TESTS WERE SUCCESSFUL!
+
+  ALL TEST VECTORS PASSED!
+
+That's all folks!  Have fun!
+
+Aaron out.
+

src/lib_hash/sha2.h

@@ -0,0 +1,193 @@
+/*
+ * FILE:	sha2.h
+ * AUTHOR:	Aaron D. Gifford - http://www.aarongifford.com/
+ * 
+ * Copyright (c) 2000-2001, Aaron D. Gifford
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ * 1. Redistributions of source code must retain the above copyright
+ *    notice, this list of conditions and the following disclaimer.
+ * 2. Redistributions in binary form must reproduce the above copyright
+ *    notice, this list of conditions and the following disclaimer in the
+ *    documentation and/or other materials provided with the distribution.
+ * 3. Neither the name of the copyright holder nor the names of contributors
+ *    may be used to endorse or promote products derived from this software
+ *    without specific prior written permission.
+ * 
+ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTOR(S) ``AS IS'' AND
+ * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+ * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ * ARE DISCLAIMED.  IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTOR(S) BE LIABLE
+ * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+ * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
+ * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
+ * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
+ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
+ * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
+ * SUCH DAMAGE.
+ *
+ * $Id: sha2.h,v 1.1 2001/11/08 00:02:01 adg Exp adg $
+ */
+
+#ifndef __SHA2_H__
+#define __SHA2_H__
+
+/*
+ * Import u_intXX_t size_t type definitions from system headers.  You
+ * may need to change this, or define these things yourself in this
+ * file.
+ */
+#include <sys/types.h>
+
+#ifdef _WIN32
+
+#include <inttypes.h>
+
+#define u_int8_t  uint8_t
+#define u_int32_t uint32_t
+#define u_int64_t uint64_t
+
+#define BYTE_ORDER LITTLE_ENDIAN
+
+#endif
+
+
+/*** SHA-256/384/512 Various Length Definitions ***********************/
+#define SHA256_BLOCK_LENGTH		64
+#define SHA256_DIGEST_LENGTH		32
+#define SHA256_DIGEST_STRING_LENGTH	(SHA256_DIGEST_LENGTH * 2 + 1)
+#define SHA384_BLOCK_LENGTH		128
+#define SHA384_DIGEST_LENGTH		48
+#define SHA384_DIGEST_STRING_LENGTH	(SHA384_DIGEST_LENGTH * 2 + 1)
+#define SHA512_BLOCK_LENGTH		128
+#define SHA512_DIGEST_LENGTH		64
+#define SHA512_DIGEST_STRING_LENGTH	(SHA512_DIGEST_LENGTH * 2 + 1)
+
+
+/*** SHA-256/384/512 Context Structures *******************************/
+/* NOTE: If your architecture does not define either u_intXX_t types or
+ * uintXX_t (from inttypes.h), you may need to define things by hand
+ * for your system:
+ */
+#if 0
+typedef unsigned char u_int8_t;		/* 1-byte  (8-bits)  */
+typedef unsigned int u_int32_t;		/* 4-bytes (32-bits) */
+typedef unsigned long long u_int64_t;	/* 8-bytes (64-bits) */
+#endif
+/*
+ * Most BSD systems already define u_intXX_t types, as does Linux.
+ * Some systems, however, like Compaq's Tru64 Unix instead can use
+ * uintXX_t types defined by very recent ANSI C standards and included
+ * in the file:
+ *
+ *   #include <inttypes.h>
+ *
+ * If you choose to use <inttypes.h> then please define: 
+ *
+ *   #define SHA2_USE_INTTYPES_H
+ *
+ * Or on the command line during compile:
+ *
+ *   cc -DSHA2_USE_INTTYPES_H ...
+ */
+#ifdef SHA2_USE_INTTYPES_H
+
+typedef struct _SHA256_CTX {
+	uint32_t	state[8];
+	uint64_t	bitcount;
+	uint8_t	buffer[SHA256_BLOCK_LENGTH];
+} SHA256_CTX;
+typedef struct _SHA512_CTX {
+	uint64_t	state[8];
+	uint64_t	bitcount[2];
+	uint8_t	buffer[SHA512_BLOCK_LENGTH];
+} SHA512_CTX;
+
+#else /* SHA2_USE_INTTYPES_H */
+
+typedef struct _SHA256_CTX {
+	u_int32_t	state[8];
+	u_int64_t	bitcount;
+	u_int8_t	buffer[SHA256_BLOCK_LENGTH];
+} SHA256_CTX;
+typedef struct _SHA512_CTX {
+	u_int64_t	state[8];
+	u_int64_t	bitcount[2];
+	u_int8_t	buffer[SHA512_BLOCK_LENGTH];
+} SHA512_CTX;
+
+#endif /* SHA2_USE_INTTYPES_H */
+
+typedef SHA512_CTX SHA384_CTX;
+
+
+/*** SHA-256/384/512 Function Prototypes ******************************/
+#ifndef NOPROTO
+#ifdef SHA2_USE_INTTYPES_H
+
+void SHA256_Init(SHA256_CTX *);
+void SHA256_Update(SHA256_CTX*, const uint8_t*, size_t);
+void SHA256_Final(uint8_t[SHA256_DIGEST_LENGTH], SHA256_CTX*);
+char* SHA256_End(SHA256_CTX*, char[SHA256_DIGEST_STRING_LENGTH]);
+char* SHA256_Data(const uint8_t*, size_t, char[SHA256_DIGEST_STRING_LENGTH]);
+
+void SHA384_Init(SHA384_CTX*);
+void SHA384_Update(SHA384_CTX*, const uint8_t*, size_t);
+void SHA384_Final(uint8_t[SHA384_DIGEST_LENGTH], SHA384_CTX*);
+char* SHA384_End(SHA384_CTX*, char[SHA384_DIGEST_STRING_LENGTH]);
+char* SHA384_Data(const uint8_t*, size_t, char[SHA384_DIGEST_STRING_LENGTH]);
+
+void SHA512_Init(SHA512_CTX*);
+void SHA512_Update(SHA512_CTX*, const uint8_t*, size_t);
+void SHA512_Final(uint8_t[SHA512_DIGEST_LENGTH], SHA512_CTX*);
+char* SHA512_End(SHA512_CTX*, char[SHA512_DIGEST_STRING_LENGTH]);
+char* SHA512_Data(const uint8_t*, size_t, char[SHA512_DIGEST_STRING_LENGTH]);
+
+#else /* SHA2_USE_INTTYPES_H */
+
+void SHA256_Init(SHA256_CTX *);
+void SHA256_Update(SHA256_CTX*, const u_int8_t*, size_t);
+void SHA256_Final(u_int8_t[SHA256_DIGEST_LENGTH], SHA256_CTX*);
+char* SHA256_End(SHA256_CTX*, char[SHA256_DIGEST_STRING_LENGTH]);
+char* SHA256_Data(const u_int8_t*, size_t, char[SHA256_DIGEST_STRING_LENGTH]);
+
+void SHA384_Init(SHA384_CTX*);
+void SHA384_Update(SHA384_CTX*, const u_int8_t*, size_t);
+void SHA384_Final(u_int8_t[SHA384_DIGEST_LENGTH], SHA384_CTX*);
+char* SHA384_End(SHA384_CTX*, char[SHA384_DIGEST_STRING_LENGTH]);
+char* SHA384_Data(const u_int8_t*, size_t, char[SHA384_DIGEST_STRING_LENGTH]);
+
+void SHA512_Init(SHA512_CTX*);
+void SHA512_Update(SHA512_CTX*, const u_int8_t*, size_t);
+void SHA512_Final(u_int8_t[SHA512_DIGEST_LENGTH], SHA512_CTX*);
+char* SHA512_End(SHA512_CTX*, char[SHA512_DIGEST_STRING_LENGTH]);
+char* SHA512_Data(const u_int8_t*, size_t, char[SHA512_DIGEST_STRING_LENGTH]);
+
+#endif /* SHA2_USE_INTTYPES_H */
+
+#else /* NOPROTO */
+
+void SHA256_Init();
+void SHA256_Update();
+void SHA256_Final();
+char* SHA256_End();
+char* SHA256_Data();
+
+void SHA384_Init();
+void SHA384_Update();
+void SHA384_Final();
+char* SHA384_End();
+char* SHA384_Data();
+
+void SHA512_Init();
+void SHA512_Update();
+void SHA512_Final();
+char* SHA512_End();
+char* SHA512_Data();
+
+#endif /* NOPROTO */
+
+#endif /* __SHA2_H__ */

src/zvbi/bcd.h

@@ -0,0 +1,216 @@
+/*
+ *  libzvbi -- BCD arithmetic for Teletext page numbers
+ *
+ *  Copyright (C) 2001, 2002 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: bcd.h,v 1.19 2008/02/21 07:18:52 mschimek Exp $ */
+
+#ifndef BCD_H
+#define BCD_H
+
+#include "ccx_common_platform.h"
+/**
+ * @addtogroup BCD BCD arithmetic for Teletext page numbers
+ * @ingroup HiDec
+ *
+ * Teletext page numbers are expressed as packed binary coded decimal
+ * numbers in range 0x100 to 0x8FF. The bcd format encodes one decimal
+ * digit in every hex nibble (four bits) of the number. Page numbers
+ * containing digits 0xA to 0xF are reserved for various system purposes
+ * and not intended for display.
+ */
+
+/* Public */
+
+/**
+ * @ingroup HiDec
+ * 
+ * Teletext or Closed Caption page number. For Teletext pages
+ * this is a packed bcd number in range 0x100 ... 0x8FF. Page
+ * numbers containing digits 0xA to 0xF are reserved for various
+ * system purposes, these pages are not intended for display.
+ * 
+ * Closed Caption page numbers between 1 ... 8 correspond
+ * to the four Caption and Text channels:
+ * <table>
+ * <tr><td>1</td><td>Caption 1</td><td>
+ *   "Primary synchronous caption service [English]"</td></tr>
+ * <tr><td>2</td><td>Caption 2</td><td>
+ *   "Special non-synchronous data that is intended to
+ *   augment information carried in the program"</td></tr>
+ * <tr><td>3</td><td>Caption 3</td><td>
+ *   "Secondary synchronous caption service, usually
+ *    second language [Spanish, French]"</td></tr>
+ * <tr><td>4</td><td>Caption 4</td><td>
+ *   "Special non-synchronous data similar to Caption 2"</td></tr>
+ * <tr><td>5</td><td>Text 1</td><td>
+ *   "First text service, data usually not program related"</td></tr>
+ * <tr><td>6</td><td>Text 2</td><td>
+ *   "Second text service, additional data usually not program related
+ *    [ITV data]"</td></tr>
+ * <tr><td>7</td><td>Text 3</td><td>
+ *   "Additional text channel"</td></tr>
+ * <tr><td>8</td><td>Text 4</td><td>
+ *   "Additional text channel"</td></tr>
+ * </table>
+ */
+/* XXX unsigned? */
+typedef int vbi_pgno;
+
+/**
+ * @ingroup HiDec
+ *
+ * This is the subpage number only applicable to Teletext pages,
+ * a packed bcd number in range 0x00 ... 0x99. On special 'clock' pages
+ * (for example listing the current time in different time zones)
+ * it can assume values between 0x0000 ... 0x2359 expressing
+ * local time. These are not actually subpages.
+ */
+typedef int vbi_subno;
+
+/**
+ * @ingroup HiDec
+ */
+#define VBI_ANY_SUBNO 0x3F7F
+/**
+ * @ingroup HiDec
+ */
+#define VBI_NO_SUBNO 0x3F7F
+
+/**
+ * @ingroup BCD
+ * @param dec Decimal number.
+ * 
+ * Converts a two's complement binary between 0 ... 999 to a
+ * packed bcd number in range  0x000 ... 0x999. Extra digits in
+ * the input will be discarded.
+ * 
+ * @return
+ * BCD number.
+ */
+inline unsigned int vbi_dec2bcd(unsigned int dec)
+{
+	return (dec % 10) + ((dec / 10) % 10) * 16 + ((dec / 100) % 10) * 256;
+}
+
+/**
+ * @ingroup BCD
+ * @since 0.2.28
+ */
+#define vbi_bin2bcd(n) vbi_dec2bcd(n)
+
+/**
+ * @ingroup BCD
+ * @param bcd BCD number.
+ * 
+ * Converts a packed bcd number between 0x000 ... 0xFFF to a two's
+ * complement binary in range 0 ... 999. Extra digits in the input
+ * will be discarded.
+ * 
+ * @return
+ * Decimal number. The result is undefined when the bcd number contains
+ * hex digits 0xA ... 0xF.
+ **/
+inline unsigned int vbi_bcd2dec(unsigned int bcd)
+{
+	return (bcd & 15) + ((bcd >> 4) & 15) * 10 + ((bcd >> 8) & 15) * 100;
+}
+
+/**
+ * @ingroup BCD
+ * @since 0.2.28
+ */
+#define vbi_bcd2bin(n) vbi_bcd2dec(n)
+
+/**
+ * @ingroup BCD
+ * @param a BCD number.
+ * @param b BCD number.
+ * 
+ * Adds two packed bcd numbers, returning a packed bcd sum. Arguments
+ * and result are in range 0xF000&nbsp;0000 ... 0x0999&nbsp;9999, that
+ * is -10**7 ... +10**7 - 1 in decimal notation. To subtract you can
+ * add the 10's complement, e. g. -1 = 0xF999&nbsp;9999.
+ * 
+ * @return
+ * Packed bcd number. The result is undefined when any of the arguments
+ * contain hex digits 0xA ... 0xF.
+ */
+inline unsigned int vbi_add_bcd(unsigned int a, unsigned int b)
+{
+	unsigned int t;
+
+	a += 0x06666666;
+	t  = a + b;
+	b ^= a ^ t;
+	b  = (~b & 0x11111110) >> 3;
+	b |= b * 2;
+
+	return t - b;
+}
+
+/**
+ * @ingroup BCD
+ * @param bcd BCD number.
+ * 
+ * Tests if @a bcd forms a valid BCD number. The argument must be
+ * in range 0x0000&nbsp;0000 ... 0x0999&nbsp;9999.
+ * 
+ * @return
+ * @c FALSE if @a bcd contains hex digits 0xA ... 0xF.
+ */
+inline int vbi_is_bcd(unsigned int bcd)
+{
+	static const unsigned int x = 0x06666666;
+
+	return (((bcd + x) ^ (bcd ^ x)) & 0x11111110) == 0;
+}
+
+/**
+ * @ingroup BCD
+ * @param bcd Unsigned BCD number.
+ * @param maximum Unsigned maximum value.
+ *
+ * Compares an unsigned packed bcd number digit-wise against a maximum
+ * value, for example 0x295959. @a maximum can contain digits 0x0
+ * ... 0xF.
+ *
+ * @return
+ * @c TRUE if any digit of @a bcd is greater than the
+ * corresponding digit of @a maximum.
+ *
+ * @since 0.2.28
+ */
+inline int vbi_bcd_digits_greater (unsigned int	bcd, unsigned int maximum)
+{
+	maximum ^= ~0;
+
+	return 0 != (((bcd + maximum) ^ bcd ^ maximum) & 0x11111110);
+}
+
+/* Private */
+
+#endif /* BCD_H */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/bit_slicer.h

@@ -0,0 +1,197 @@
+/*
+ *  libzvbi -- Bit slicer
+ *
+ *  Copyright (C) 2000-2007 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: bit_slicer.h,v 1.11 2008/02/24 14:18:13 mschimek Exp $ */
+
+#ifndef __ZVBI_BIT_SLICER_H__
+#define __ZVBI_BIT_SLICER_H__
+
+#include "sampling_par.h"
+
+VBI_BEGIN_DECLS
+
+/**
+ * @addtogroup BitSlicer
+ * @{
+ */
+
+/**
+ * @brief Modulation used for VBI data transmission.
+ */
+typedef enum {
+	/**
+	 * The data is 'non-return to zero' coded, logical '1' bits
+	 * are described by high sample values, logical '0' bits by
+	 * low values. The data is last significant bit first transmitted.
+	 */
+	VBI3_MODULATION_NRZ_LSB,
+
+	/**
+	 * 'Non-return to zero' coded, most significant bit first
+	 * transmitted.
+	 */
+	VBI3_MODULATION_NRZ_MSB,
+
+	/**
+	 * The data is 'bi-phase' coded. Each data bit is described
+	 * by two complementary signalling elements, a logical '1'
+	 * by a sequence of '10' elements, a logical '0' by a '01'
+	 * sequence. The data is last significant bit first transmitted.
+	 */
+	VBI3_MODULATION_BIPHASE_LSB,
+
+	/** 'Bi-phase' coded, most significant bit first transmitted. */
+	VBI3_MODULATION_BIPHASE_MSB
+} vbi3_modulation;
+
+/**
+ * @brief Bit slicer context.
+ *
+ * The contents of this structure are private.
+ * Call vbi3_bit_slicer_new() to allocate a bit slicer context.
+ */
+typedef struct _vbi3_bit_slicer vbi3_bit_slicer;
+
+typedef enum {
+	VBI3_CRI_BIT = 1,
+	VBI3_FRC_BIT,
+	VBI3_PAYLOAD_BIT
+} vbi3_bit_slicer_bit;
+
+/**
+ * @brief Bit slicer sampling point.
+ *
+ * This structure contains information about
+ * a bit sampled by the bit slicer.
+ */
+typedef struct {
+	/** Whether this struct refers to a CRI, FRC or payload bit. */
+	vbi3_bit_slicer_bit	kind;
+
+	/** Number of the sample times 256. */
+	unsigned int		index;
+
+	/** Signal amplitude at this sample, in range 0 to 65535. */
+	unsigned int		level;
+
+	/** 0/1 threshold at this sample, in range 0 to 65535. */
+	unsigned int		thresh;
+} vbi3_bit_slicer_point;
+
+extern vbi_bool
+vbi3_bit_slicer_slice_with_points
+				(vbi3_bit_slicer *	bs,
+				 uint8_t *		buffer,
+				 unsigned int		buffer_size,
+				 vbi3_bit_slicer_point *points,
+				 unsigned int *		n_points,
+				 unsigned int		max_points,
+				 const uint8_t *	raw)
+  _vbi_nonnull ((1, 2, 4, 5, 7));
+extern vbi_bool
+vbi3_bit_slicer_slice		(vbi3_bit_slicer *	bs,
+				 uint8_t *		buffer,
+				 unsigned int		buffer_size,
+				 const uint8_t *	raw)
+  _vbi_nonnull ((1, 2, 4));
+extern vbi_bool
+vbi3_bit_slicer_set_params	(vbi3_bit_slicer *	bs,
+				 vbi_pixfmt		sample_format,
+				 unsigned int		sampling_rate,
+				 unsigned int		sample_offset,
+				 unsigned int		samples_per_line,
+				 unsigned int		cri,
+				 unsigned int		cri_mask,
+				 unsigned int		cri_bits,
+				 unsigned int		cri_rate,
+				 unsigned int		cri_end,
+				 unsigned int		frc,
+				 unsigned int		frc_bits,
+				 unsigned int		payload_bits,
+				 unsigned int		payload_rate,
+				 vbi3_modulation	modulation)
+  _vbi_nonnull ((1));
+extern void
+vbi3_bit_slicer_set_log_fn	(vbi3_bit_slicer *	bs,
+				 vbi_log_mask		mask,
+				 vbi_log_fn *		log_fn,
+				 void *			user_data)
+  _vbi_nonnull ((1));
+extern void
+vbi3_bit_slicer_delete		(vbi3_bit_slicer *	bs);
+extern vbi3_bit_slicer *
+vbi3_bit_slicer_new		(void)
+  _vbi_alloc;
+
+/* Private */
+
+typedef vbi_bool
+_vbi3_bit_slicer_fn		(vbi3_bit_slicer *	bs,
+				 uint8_t *		buffer,
+				 vbi3_bit_slicer_point *points,
+				 unsigned int *		n_points,
+				 const uint8_t *	raw);
+
+/** @internal */
+struct _vbi3_bit_slicer {
+	_vbi3_bit_slicer_fn *	func;
+
+	vbi_pixfmt		sample_format;
+	unsigned int		cri;
+	unsigned int		cri_mask;
+	unsigned int		thresh;
+	unsigned int		thresh_frac;
+	unsigned int		cri_samples;
+	unsigned int		cri_rate;
+	unsigned int		oversampling_rate;
+	unsigned int		phase_shift;
+	unsigned int		step;
+	unsigned int		frc;
+	unsigned int		frc_bits;
+	unsigned int		total_bits;
+	unsigned int		payload;
+	unsigned int		endian;
+	unsigned int		bytes_per_sample;
+	unsigned int		skip;
+	unsigned int		green_mask;
+
+	_vbi_log_hook		log;
+};
+
+extern void
+_vbi3_bit_slicer_destroy	(vbi3_bit_slicer *	bs)
+  _vbi_nonnull ((1));
+extern vbi_bool
+_vbi3_bit_slicer_init		(vbi3_bit_slicer *	bs)
+  _vbi_nonnull ((1));
+
+/** @} */
+
+VBI_END_DECLS
+
+#endif /* __ZVBI_BIT_SLICER_H__ */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/decoder.c

@@ -0,0 +1,808 @@
+/*
+ *  libzvbi -- Old raw VBI decoder
+ *
+ *  Copyright (C) 2000, 2001, 2002 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: decoder.c,v 1.25 2008/02/19 00:35:15 mschimek Exp $ */
+
+/* Note this code is only retained for compatibility with older versions
+   of libzvbi. vbi_raw_decoder is now just a wrapper for the new raw
+   decoder (raw_decoder.c) and bit slicer (bit_slicer.c). We'll drop
+   the old API in libzvbi 0.3. Other modules (e.g. io-v4l2k.c) should
+   already use the new raw VBI decoder directly. */
+
+#include "misc.h"
+#include "zvbi_decoder.h"
+#include "raw_decoder.h"
+#include "ccx_common_common.h"
+
+/**
+ * @addtogroup Rawdec Raw VBI decoder
+ * @ingroup Raw
+ * @brief Converting raw VBI samples to bits and bytes.
+ *
+ * The libzvbi already offers hardware interfaces to obtain sliced
+ * VBI data for further processing. However if you want to write your own
+ * interface or decode data services not covered by libzvbi you can use
+ * these lower level functions.
+ */
+
+/*
+ *  Bit Slicer
+ */
+
+#define OVERSAMPLING 4		/* 1, 2, 4, 8 */
+#define THRESH_FRAC 9
+
+/*
+ * Note this is just a template. The code is inlined,
+ * with bpp and endian being const.
+ *
+ * This function translates from the image format to
+ * plain bytes, with linear interpolation of samples.
+ * Could be further improved with a lowpass filter.
+ */
+static inline unsigned int
+sample(uint8_t *raw, int offs, int bpp, int endian)
+{
+	unsigned char frac = offs;
+	int raw0, raw1;
+
+	switch (bpp) {
+	case 14: /* 1:5:5:5 LE/BE */
+		raw += (offs >> 8) * 2;
+		raw0 = (raw[0 + endian] + raw[1 - endian] * 256) & 0x07C0;
+		raw1 = (raw[2 + endian] + raw[3 - endian] * 256) & 0x07C0;
+		return (raw1 - raw0) * frac + (raw0 << 8);
+
+	case 15: /* 5:5:5:1 LE/BE */
+		raw += (offs >> 8) * 2;
+		raw0 = (raw[0 + endian] + raw[1 - endian] * 256) & 0x03E0;
+		raw1 = (raw[2 + endian] + raw[3 - endian] * 256) & 0x03E0;
+		return (raw1 - raw0) * frac + (raw0 << 8);
+
+	case 16: /* 5:6:5 LE/BE */
+		raw += (offs >> 8) * 2;
+		raw0 = (raw[0 + endian] + raw[1 - endian] * 256) & 0x07E0;
+		raw1 = (raw[2 + endian] + raw[3 - endian] * 256) & 0x07E0;
+		return (raw1 - raw0) * frac + (raw0 << 8);
+
+	default: /* 8 (intermediate bytes skipped by caller) */
+		raw += (offs >> 8) * bpp;
+		return (raw[bpp] - raw[0]) * frac + (raw[0] << 8);
+	}
+}
+
+/*
+ * Note this is just a template. The code is inlined,
+ * with bpp being const.
+ */
+static inline vbi_bool
+bit_slicer_tmpl(vbi_bit_slicer *d, uint8_t *raw,
+		uint8_t *buf, int bpp, int endian)
+{
+	unsigned int i, j, k;
+	unsigned int cl = 0, thresh0 = d->thresh, tr;
+	unsigned int c = 0, t;
+	unsigned char b, b1 = 0;
+	int raw0, raw1, mask;
+
+	raw += d->skip;
+
+	if (bpp == 14)
+		mask = 0x07C0;
+	else if (bpp == 15)
+		mask = 0x03E0;
+	else if (bpp == 16)
+		mask = 0x07E0;
+
+	for (i = d->cri_bytes; i > 0; raw += (bpp >= 14 && bpp <= 16) ? 2 : bpp, i--) {
+		if (bpp >= 14 && bpp <= 16) {
+			raw0 = (raw[0 + endian] + raw[1 - endian] * 256) & mask;
+			raw1 = (raw[2 + endian] + raw[3 - endian] * 256) & mask;
+			tr = d->thresh >> THRESH_FRAC;
+			d->thresh += ((raw0 - tr) * (int) ABS(raw1 - raw0)) >>
+				((bpp == 15) ? 2 : 3);
+			t = raw0 * OVERSAMPLING;
+		} else {
+			tr = d->thresh >> THRESH_FRAC;
+			d->thresh += ((int) raw[0] - tr) * (int) ABS(raw[bpp] - raw[0]);
+			t = raw[0] * OVERSAMPLING;
+		}
+
+		for (j = OVERSAMPLING; j > 0; j--) {
+			b = ((t + (OVERSAMPLING / 2)) / OVERSAMPLING >= tr);
+
+    			if (b ^ b1) {
+				cl = d->oversampling_rate >> 1;
+			} else {
+				cl += d->cri_rate;
+
+				if (cl >= (unsigned int) d->oversampling_rate) {
+					cl -= d->oversampling_rate;
+
+					c = c * 2 + b;
+
+					if ((c & d->cri_mask) == d->cri) {
+						i = d->phase_shift;
+						tr *= 256;
+						c = 0;
+
+						for (j = d->frc_bits; j > 0; j--) {
+							c = c * 2 + (sample(raw, i, bpp, endian) >= tr);
+    							i += d->step;
+						}
+
+						if (c ^= d->frc)
+							return CCX_FALSE;
+
+						/* CRI/FRC found, now get the
+						   payload and exit */
+						
+						switch (d->endian) {
+						case 3:
+							for (j = 0; j < (unsigned int) d->payload; j++) {
+					    			c >>= 1;
+								c += (sample(raw, i, bpp, endian) >= tr) << 7;
+			    					i += d->step;
+
+								if ((j & 7) == 7)
+									*buf++ = c;
+					    		}
+
+							*buf = c >> ((8 - d->payload) & 7);
+							break;
+
+						case 2:
+							for (j = 0; j < (unsigned int) d->payload; j++) {
+								c = c * 2 + (sample(raw, i, bpp, endian) >= tr);
+			    					i += d->step;
+
+								if ((j & 7) == 7)
+									*buf++ = c;
+					    		}
+
+							*buf = c & ((1 << (d->payload & 7)) - 1);
+							break;
+
+						case 1:
+							for (j = d->payload; j > 0; j--) {
+								for (k = 0; k < 8; k++) {
+						    			c >>= 1;
+									c += (sample(raw, i, bpp, endian) >= tr) << 7;
+			    						i += d->step;
+								}
+
+								*buf++ = c;
+					    		}
+
+							break;
+
+						case 0:
+							for (j = d->payload; j > 0; j--) {
+								for (k = 0; k < 8; k++) {
+									c = c * 2 + (sample(raw, i, bpp, endian) >= tr);
+			    						i += d->step;
+								}
+
+								*buf++ = c;
+					    		}
+
+							break;
+						}
+
+			    			return CCX_TRUE;
+					}
+				}
+			}
+
+			b1 = b;
+
+			if (OVERSAMPLING > 1) {
+				if (bpp >= 14 && bpp <= 16) {
+					t += raw1;
+					t -= raw0;
+				} else {
+					t += raw[bpp];
+					t -= raw[0];
+				}
+			}
+		}
+	}
+
+	d->thresh = thresh0;
+
+	return CCX_FALSE;
+}
+
+static vbi_bool
+bit_slicer_1(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 1, 0);
+}
+
+static vbi_bool
+bit_slicer_2(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 2, 0);
+}
+
+static vbi_bool
+bit_slicer_3(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 3, 0);
+}
+
+static vbi_bool
+bit_slicer_4(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 4, 0);
+}
+
+static vbi_bool
+bit_slicer_1555_le(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 14, 0);
+}
+
+static vbi_bool
+bit_slicer_5551_le(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 15, 0);
+}
+
+static vbi_bool
+bit_slicer_565_le(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 16, 0);
+}
+
+static vbi_bool
+bit_slicer_1555_be(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 14, 1);
+}
+
+static vbi_bool
+bit_slicer_5551_be(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 15, 1);
+}
+
+static vbi_bool
+bit_slicer_565_be(vbi_bit_slicer *d, uint8_t *raw, uint8_t *buf)
+{
+	return bit_slicer_tmpl(d, raw, buf, 16, 1);
+}
+
+/**
+ * @param slicer Pointer to vbi_bit_slicer object to be initialized. 
+ * @param raw_samples Number of samples or pixels in one raw vbi line
+ *   later passed to vbi_bit_slice(). This limits the number of
+ *   bytes read from the sample buffer.
+ * @param sampling_rate Raw vbi sampling rate in Hz, that is the number of
+ *   samples or pixels sampled per second by the hardware. 
+ * @param cri_rate The Clock Run In is a NRZ modulated
+ *   sequence of '0' and '1' bits prepending most data transmissions to
+ *   synchronize data acquisition circuits. This parameter gives the CRI bit
+ *   rate in Hz, that is the number of CRI bits transmitted per second.
+ * @param bit_rate The transmission bit rate of all data bits following the CRI
+ *   in Hz.
+ * @param cri_frc The FRaming Code usually following the CRI is a bit sequence
+ *   identifying the data service, and per libzvbi definition modulated
+ *   and transmitted at the same bit rate as the payload (however nothing
+ *   stops you from counting all nominal CRI and FRC bits as CRI).
+ *   The bit slicer compares the bits in this word, lsb last transmitted,
+ *   against the transmitted CRI and FRC. Decoding of payload starts
+ *   with the next bit after a match.
+ * @param cri_mask Of the CRI bits in @c cri_frc, only these bits are
+ *   actually significant for a match. For instance it is wise
+ *   not to rely on the very first CRI bits transmitted. Note this
+ *   mask is not shifted left by @a frc_bits.
+ * @param cri_bits 
+ * @param frc_bits Number of CRI and FRC bits in @a cri_frc, respectively.
+ *   Their sum is limited to 32.
+ * @param payload Number of payload <em>bits</em>. Only this data
+ *   will be stored in the vbi_bit_slice() output. If this number
+ *   is no multiple of eight, the most significant bits of the
+ *   last byte are undefined.
+ * @param modulation Modulation of the vbi data, see vbi_modulation.
+ * @param fmt Format of the raw data, see vbi_pixfmt.
+ * 
+ * Initializes vbi_bit_slicer object. Usually you will not use this
+ * function but vbi_raw_decode(), the vbi image decoder which handles
+ * all these details.
+ */
+void
+vbi_bit_slicer_init(vbi_bit_slicer *slicer,
+		    int raw_samples, int sampling_rate,
+		    int cri_rate, int bit_rate,
+		    unsigned int cri_frc, unsigned int cri_mask,
+		    int cri_bits, int frc_bits, int payload,
+		    vbi_modulation modulation, vbi_pixfmt fmt)
+{
+	unsigned int c_mask = (unsigned int)(-(cri_bits > 0)) >> (32 - cri_bits);
+	unsigned int f_mask = (unsigned int)(-(frc_bits > 0)) >> (32 - frc_bits);
+	int gsh = 0;
+
+	slicer->func = bit_slicer_1;
+
+	switch (fmt) {
+	case VBI_PIXFMT_RGB24:
+	case VBI_PIXFMT_BGR24:
+	        slicer->func = bit_slicer_3;
+		slicer->skip = 1;
+		break;
+
+	case VBI_PIXFMT_RGBA32_LE:
+	case VBI_PIXFMT_BGRA32_LE:
+		slicer->func = bit_slicer_4;
+		slicer->skip = 1;
+		break;
+
+	case VBI_PIXFMT_RGBA32_BE:
+	case VBI_PIXFMT_BGRA32_BE:
+		slicer->func = bit_slicer_4;
+		slicer->skip = 2;
+		break;
+
+	case VBI_PIXFMT_RGB16_LE:
+	case VBI_PIXFMT_BGR16_LE:
+		slicer->func = bit_slicer_565_le;
+		gsh = 3; /* (green << 3) & 0x07E0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_RGBA15_LE:
+	case VBI_PIXFMT_BGRA15_LE:
+		slicer->func = bit_slicer_5551_le;
+		gsh = 2; /* (green << 2) & 0x03E0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_ARGB15_LE:
+	case VBI_PIXFMT_ABGR15_LE:
+		slicer->func = bit_slicer_1555_le;
+		gsh = 3; /* (green << 2) & 0x07C0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_RGB16_BE:
+	case VBI_PIXFMT_BGR16_BE:
+		slicer->func = bit_slicer_565_be;
+		gsh = 3; /* (green << 3) & 0x07E0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_RGBA15_BE:
+	case VBI_PIXFMT_BGRA15_BE:
+		slicer->func = bit_slicer_5551_be;
+		gsh = 2; /* (green << 2) & 0x03E0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_ARGB15_BE:
+	case VBI_PIXFMT_ABGR15_BE:
+		slicer->func = bit_slicer_1555_be;
+		gsh = 3; /* (green << 2) & 0x07C0 */
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_YUV420:
+		slicer->func = bit_slicer_1;
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_YUYV:
+	case VBI_PIXFMT_YVYU:
+		slicer->func = bit_slicer_2;
+		slicer->skip = 0;
+		break;
+
+	case VBI_PIXFMT_UYVY:
+	case VBI_PIXFMT_VYUY:
+		slicer->func = bit_slicer_2;
+		slicer->skip = 1;
+		break;
+
+	default:
+		fprintf(stderr, "vbi_bit_slicer_init: unknown pixfmt %d\n", fmt);
+		exit(EXIT_FAILURE);
+	}
+
+	slicer->cri_mask		= cri_mask & c_mask;
+	slicer->cri		 	= (cri_frc >> frc_bits) & slicer->cri_mask;
+	/* We stop searching for CRI/FRC when the payload
+	   cannot possibly fit anymore. */
+	slicer->cri_bytes		= raw_samples
+		- ((long long) sampling_rate * (payload + frc_bits)) / bit_rate;
+	slicer->cri_rate		= cri_rate;
+	/* Raw vbi data is oversampled to account for low sampling rates. */
+	slicer->oversampling_rate	= sampling_rate * OVERSAMPLING;
+	/* 0/1 threshold */
+	slicer->thresh			= 105 << (THRESH_FRAC + gsh);
+	slicer->frc			= cri_frc & f_mask;
+	slicer->frc_bits		= frc_bits;
+	/* Payload bit distance in 1/256 raw samples. */
+	slicer->step			= (int)(sampling_rate * 256.0 / bit_rate);
+
+	if (payload & 7) {
+		slicer->payload	= payload;
+		slicer->endian	= 3;
+	} else {
+		slicer->payload	= payload >> 3;
+		slicer->endian	= 1;
+	}
+
+	switch (modulation) {
+	case VBI_MODULATION_NRZ_MSB:
+		slicer->endian--;
+	case VBI_MODULATION_NRZ_LSB:
+		slicer->phase_shift = (int)
+			(sampling_rate * 256.0 / cri_rate * .5
+			 + sampling_rate * 256.0 / bit_rate * .5 + 128);
+		break;
+
+	case VBI_MODULATION_BIPHASE_MSB:
+		slicer->endian--;
+	case VBI_MODULATION_BIPHASE_LSB:
+		/* Phase shift between the NRZ modulated CRI and the rest */
+		slicer->phase_shift = (int)
+			(sampling_rate * 256.0 / cri_rate * .5
+			 + sampling_rate * 256.0 / bit_rate * .25 + 128);
+		break;
+	}
+}
+
+/**
+ * @example examples/wss.c
+ * WSS capture example.
+ */
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param raw A raw vbi image as defined in the vbi_raw_decoder structure
+ *   (rd->sampling_format, rd->bytes_per_line, rd->count[0] + rd->count[1]
+ *    scan lines).
+ * @param out Buffer to store the decoded vbi_sliced data. Since every
+ *   vbi scan line may contain data, this must be an array of vbi_sliced
+ *   with the same number of entries as scan lines in the raw image
+ *   (rd->count[0] + rd->count[1]).
+ * 
+ * Decode a raw vbi image, consisting of several scan lines of raw vbi data,
+ * into sliced vbi data. The output is sorted by line number.
+ * 
+ * Note this function attempts to learn which lines carry which data
+ * service, or none, to speed up decoding. You should avoid using the same
+ * vbi_raw_decoder structure for different sources.
+ *
+ * @return
+ * The number of lines decoded, i. e. the number of vbi_sliced records
+ * written.
+ */
+int vbi_raw_decode (vbi_raw_decoder *	rd,
+				 uint8_t *		raw,
+				 vbi_sliced *		out)
+{
+	vbi3_raw_decoder *rd3;
+	unsigned int n_lines;
+
+	assert (NULL != rd);
+	assert (NULL != raw);
+	assert (NULL != out);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+	n_lines = rd->count[0] + rd->count[1];
+
+
+	n_lines = vbi3_raw_decoder_decode (rd3, out, n_lines, raw);
+
+	return n_lines;
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param start Array of start line indices for both fields
+ * @param count Array of line counts for both fields
+ * 
+ * Grows or shrinks the internal state arrays for VBI geometry changes
+ */
+void
+vbi_raw_decoder_resize		(vbi_raw_decoder *	rd,
+				 int *			start,
+				 unsigned int *		count)
+{
+	vbi_service_set service_set;
+	vbi3_raw_decoder *rd3;
+
+	assert (NULL != rd);
+	assert (NULL != start);
+	assert (NULL != count);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+
+
+	{
+		if ((rd->start[0] == start[0])
+		    && (rd->start[1] == start[1])
+		    && (rd->count[0] == (int) count[0])
+		    && (rd->count[1] == (int) count[1])) {
+			return;
+		}
+
+		rd->start[0] = start[0];
+		rd->start[1] = start[1];
+		rd->count[0] = count[0];
+		rd->count[1] = count[1];
+
+		service_set = vbi3_raw_decoder_set_sampling_par
+			(rd3, (vbi_sampling_par *) rd, /* strict */ 0);
+	}
+
+
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param services Set of @ref VBI_SLICED_ symbols.
+ * 
+ * Removes one or more data services to be decoded from the
+ * vbi_raw_decoder structure. This function can be called at any
+ * time and does not touch sampling parameters. 
+ * 
+ * @return 
+ * Set of @ref VBI_SLICED_ symbols describing the remaining data
+ * services that will be decoded.
+ */
+unsigned int
+vbi_raw_decoder_remove_services	(vbi_raw_decoder *	rd,
+				 unsigned int		services)
+{
+	vbi_service_set service_set;
+	vbi3_raw_decoder *rd3;
+
+	assert (NULL != rd);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+	service_set = services;
+
+	{
+		service_set = vbi3_raw_decoder_remove_services
+			(rd3, service_set);
+	}
+
+	return service_set;
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param services Set of @ref VBI_SLICED_ symbols.
+ * @param strict See description of vbi_raw_decoder_add_services()
+ *
+ * Check which of the given services can be decoded with current capture
+ * parameters at a given strictness level.
+ *
+ * @return
+ * Subset of services actually decodable.
+ */
+unsigned int
+vbi_raw_decoder_check_services	(vbi_raw_decoder *	rd,
+				 unsigned int		services,
+				 int			strict)
+{
+	vbi_service_set service_set;
+
+	assert (NULL != rd);
+
+	service_set = services;
+
+
+	{
+		service_set = vbi_sampling_par_check_services
+			((vbi_sampling_par *) rd, service_set, strict);
+	}
+
+
+	return (unsigned int) service_set;
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param services Set of @ref VBI_SLICED_ symbols.
+ * @param strict A value of 0, 1 or 2 requests loose, reliable or strict
+ *  matching of sampling parameters. For example if the data service
+ *  requires knowledge of line numbers while they are not known, @c 0
+ *  will accept the service (which may work if the scan lines are
+ *  populated in a non-confusing way) but @c 1 or @c 2 will not. If the
+ *  data service <i>may</i> use more lines than are sampled, @c 1 will
+ *  accept but @c 2 will not. If unsure, set to @c 1.
+ * 
+ * After you initialized the sampling parameters in @a rd (according to
+ * the abilities of your raw vbi source), this function adds one or more
+ * data services to be decoded. The libzvbi raw vbi decoder can decode up
+ * to eight data services in parallel. You can call this function while
+ * already decoding, it does not change sampling parameters and you must
+ * not change them either after calling this.
+ * 
+ * @return
+ * Set of @ref VBI_SLICED_ symbols describing the data services that actually
+ * will be decoded. This excludes those services not decodable given
+ * the sampling parameters in @a rd.
+ */
+unsigned int
+vbi_raw_decoder_add_services	(vbi_raw_decoder *	rd,
+				 unsigned int		services,
+				 int			strict)
+{
+	vbi_service_set service_set;
+	vbi3_raw_decoder *rd3;
+
+	assert (NULL != rd);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+	service_set = services;
+
+
+
+	{
+		vbi3_raw_decoder_set_sampling_par
+			(rd3, (vbi_sampling_par *) rd, strict);
+
+		service_set = vbi3_raw_decoder_add_services
+			(rd3, service_set, strict);
+	}
+
+	return service_set;
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * @param services Set of VBI_SLICED_ symbols. Here (and only here) you
+ *   can add @c VBI_SLICED_VBI_625 or @c VBI_SLICED_VBI_525 to include all
+ *   vbi scan lines in the calculated sampling parameters.
+ * @param scanning When 525 accept only NTSC services, when 625
+ *   only PAL/SECAM services. When scanning is 0, determine the scanning
+ *   from the requested services, an ambiguous set will pick
+ *   a 525 or 625 line system at random.
+ * @param max_rate If given, the highest data bit rate in Hz of all
+ *   services requested is stored here. (The sampling rate
+ *   should be at least twice as high; rd->sampling_rate will
+ *   be set to a more reasonable value of 27 MHz derived
+ *   from ITU-R Rec. 601.)
+ * 
+ * Calculate the sampling parameters in @a rd required to receive and
+ * decode the requested data @a services. rd->sampling_format will be
+ * @c VBI_PIXFMT_YUV420, rd->bytes_per_line set accordingly to a
+ * reasonable minimum. This function can be used to initialize hardware
+ * prior to calling vbi_raw_decoder_add_service().
+ * 
+ * @return
+ * Set of @ref VBI_SLICED_ symbols describing the data services covered
+ * by the calculated sampling parameters. This excludes services the libzvbi
+ * raw decoder cannot decode.
+ */
+unsigned int
+vbi_raw_decoder_parameters	(vbi_raw_decoder *	rd,
+				 unsigned int		services,
+				 int			scanning,
+				 int *			max_rate)
+{
+	vbi_videostd_set videostd_set;
+	vbi_service_set service_set;
+
+	switch (scanning) {
+	case 525:
+		videostd_set = VBI_VIDEOSTD_SET_525_60;
+		break;
+
+	case 625:
+		videostd_set = VBI_VIDEOSTD_SET_625_50;
+		break;
+
+	default:
+		videostd_set = 0;
+		break;
+	}
+
+	service_set = services;
+ 
+
+	{
+		service_set = vbi_sampling_par_from_services
+			((vbi_sampling_par *) rd,
+			 (unsigned int *) max_rate,
+			 videostd_set, service_set);
+	}
+
+	return (unsigned int) service_set;
+}
+
+/**
+ * @param rd Initialized vbi_raw_decoder structure.
+ * 
+ * Reset a vbi_raw_decoder structure. This removes
+ * all previously added services to be decoded (if any)
+ * but does not touch the sampling parameters. You are
+ * free to change the sampling parameters after calling this.
+ */
+void
+vbi_raw_decoder_reset		(vbi_raw_decoder *	rd)
+{
+	vbi3_raw_decoder *rd3;
+
+	if (!rd)
+		return; /* compatibility */
+
+	assert (NULL != rd);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+
+
+	{
+		vbi3_raw_decoder_reset (rd3);
+	}
+}
+
+/**
+ * @param rd Pointer to initialized vbi_raw_decoder
+ *  structure, can be @c NULL.
+ * 
+ * Free all resources associated with @a rd.
+ */
+void
+vbi_raw_decoder_destroy		(vbi_raw_decoder *	rd)
+{
+	vbi3_raw_decoder *rd3;
+
+	assert (NULL != rd);
+
+	rd3 = (vbi3_raw_decoder *) rd->pattern;
+
+	vbi3_raw_decoder_delete (rd3);
+
+	CLEAR (*rd);
+}
+
+/**
+ * @param rd Pointer to a vbi_raw_decoder structure.
+ * 
+ * Initializes a vbi_raw_decoder structure.
+ */
+void
+vbi_raw_decoder_init		(vbi_raw_decoder *	rd)
+{
+	vbi3_raw_decoder *rd3;
+
+	assert (NULL != rd);
+
+	CLEAR (*rd);
+
+
+	rd3 = vbi3_raw_decoder_new (/* sampling_par */ NULL);
+	assert (NULL != rd3);
+
+	rd->pattern = (int8_t *) rd3;
+}
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/macros.h

@@ -0,0 +1,156 @@
+/*
+ *  libzvbi -- Useful macros
+ *
+ *  Copyright (C) 2002, 2003, 2004, 2007 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: macros.h,v 1.12 2013/07/10 23:11:18 mschimek Exp $ */
+
+#ifndef __ZVBI_MACROS_H__
+#define __ZVBI_MACROS_H__
+
+#ifdef __cplusplus
+#  define VBI_BEGIN_DECLS extern "C" {
+#  define VBI_END_DECLS }
+#else
+#  define VBI_BEGIN_DECLS
+#  define VBI_END_DECLS
+#endif
+
+VBI_BEGIN_DECLS
+
+/* Public */
+
+#if __GNUC__ >= 4
+#  define _vbi_sentinel __attribute__ ((__sentinel__(0)))
+#  define _vbi_deprecated __attribute__ ((__deprecated__))
+#else
+#  define _vbi_sentinel
+#  define _vbi_deprecated
+#  define __restrict__
+#endif
+
+#if (__GNUC__ == 3 && __GNUC_MINOR__ >= 3) || __GNUC__ >= 4
+#  define _vbi_nonnull(params) __attribute__ ((__nonnull__ params))
+#  define _vbi_format(params) __attribute__ ((__format__ params))
+#else
+#  define _vbi_nonnull(params)
+#  define _vbi_format(params)
+#endif
+
+#if __GNUC__ >= 3
+#  define _vbi_pure __attribute__ ((__pure__))
+#  define _vbi_alloc __attribute__ ((__malloc__))
+#else
+#  define _vbi_pure
+#  define _vbi_alloc
+#endif
+
+#if __GNUC__ >= 2
+#  define _vbi_unused __attribute__ ((__unused__))
+#  define _vbi_const __attribute__ ((__const__))
+#  define _vbi_inline static __inline__
+#else
+#  define _vbi_unused
+#  define _vbi_const
+#  define _vbi_inline static
+#endif
+
+typedef int vbi_bool;
+/** @} */
+
+#ifndef NULL
+#  ifdef __cplusplus
+#    define NULL (0L)
+#  else
+#    define NULL ((void *) 0)
+#  endif
+#endif
+
+/* XXX Document me - for variadic funcs. */
+#define VBI_END ((void *) 0)
+
+#if 0
+typedef void
+vbi_lock_fn			(void *			user_data);
+typedef void
+vbi_unlock_fn			(void *			user_data);
+#endif
+
+/**
+ * @ingroup Basic
+ * @{
+ */
+typedef enum {
+	/** External error causes, for example lack of memory. */
+	VBI_LOG_ERROR		= 1 << 3,
+
+	/**
+	 * Invalid parameters and similar problems which suggest
+	 * a bug in the application using the library.
+	 */
+	VBI_LOG_WARNING		= 1 << 4,
+
+	/**
+	 * Causes of possibly undesired results, for example when a
+	 * data service cannot be decoded with the current video
+	 * standard setting.
+	 */
+	VBI_LOG_NOTICE		= 1 << 5,
+
+	/** Progress messages. */
+	VBI_LOG_INFO		= 1 << 6,
+
+	/** Information useful to debug the library. */
+	VBI_LOG_DEBUG		= 1 << 7,
+
+	/** Driver responses (strace). Not implemented yet. */
+	VBI_LOG_DRIVER		= 1 << 8,
+
+	/** More detailed debugging information. */
+	VBI_LOG_DEBUG2		= 1 << 9,
+	VBI_LOG_DEBUG3		= 1 << 10
+} vbi_log_mask;
+
+typedef void
+vbi_log_fn			(vbi_log_mask		level,
+				 const char *		context,
+				 const char *		message,
+				 void *			user_data);
+
+extern vbi_log_fn		vbi_log_on_stderr;
+/** @} */
+
+/* Private */
+
+typedef struct {
+	vbi_log_fn *		fn;
+	void *			user_data;
+	vbi_log_mask		mask;
+} _vbi_log_hook;
+
+VBI_END_DECLS
+
+#endif /* __ZVBI_MACROS_H__ */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/misc.h

@@ -0,0 +1,424 @@
+/*
+ *  libzvbi -- Miscellaneous cows and chickens
+ *
+ *  Copyright (C) 2000-2003 Iñaki García Etxebarria
+ *  Copyright (C) 2002-2007 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: misc.h,v 1.24 2013/07/02 02:32:31 mschimek Exp $ */
+
+#ifndef MISC_H
+#define MISC_H
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stddef.h>
+#include <stdarg.h>
+#include <string.h>
+#include <inttypes.h>		/* (u)intXX_t */
+#include <sys/types.h>		/* (s)size_t */
+#include <float.h>		/* DBL_MAX */
+#include <limits.h>		/* (S)SIZE_MAX */
+#include <assert.h>
+
+#include "macros.h"
+#include "ccx_common_platform.h"
+#include "ccx_common_constants.h"
+#include "ccx_common_structs.h"
+
+#define N_ELEMENTS(array) (sizeof (array) / sizeof (*(array)))
+
+#ifdef __GNUC__
+
+#if __GNUC__ < 3
+/* Expect expression usually true/false, schedule accordingly. */
+#  define likely(expr) (expr)
+#  define unlikely(expr) (expr)
+#else
+#  define likely(expr) __builtin_expect(expr, 1)
+#  define unlikely(expr) __builtin_expect(expr, 0)
+#endif
+
+#undef __i386__
+#undef __i686__
+/* FIXME #cpu is deprecated
+#if #cpu (i386)
+#  define __i386__ 1
+#endif
+#if #cpu (i686)
+#  define __i686__ 1
+#endif
+*/
+
+/* &x == PARENT (&x.tm_min, struct tm, tm_min),
+   safer than &x == (struct tm *) &x.tm_min. A NULL _ptr is safe and
+   will return NULL, not -offsetof(_member). */
+#undef PARENT
+#define PARENT(_ptr, _type, _member) ({					\
+	__typeof__ (&((_type *) 0)->_member) _p = (_ptr);		\
+	(_p != 0) ? (_type *)(((char *) _p) - offsetof (_type,		\
+	  _member)) : (_type *) 0;					\
+})
+
+/* Like PARENT(), to be used with const _ptr. */
+#define CONST_PARENT(_ptr, _type, _member) ({				\
+	__typeof__ (&((const _type *) 0)->_member) _p = (_ptr);		\
+	(_p != 0) ? (const _type *)(((const char *) _p) - offsetof	\
+	 (const _type, _member)) : (const _type *) 0;			\
+})
+
+/* Note the following macros have no side effects only when you
+   compile with GCC, so don't expect this. */
+
+/* Absolute value of int, long or long long without a branch.
+   Note ABS (INT_MIN) -> INT_MAX + 1. */
+#undef ABS
+#define ABS(n) ({							\
+	register __typeof__ (n) _n = (n), _t = _n;			\
+	if (-1 == (-1 >> 1)) { /* do we have signed shifts? */		\
+		_t >>= sizeof (_t) * 8 - 1;				\
+		_n ^= _t;						\
+		_n -= _t;						\
+	} else if (_n < 0) { /* also warns if n is unsigned type */	\
+		_n = -_n;						\
+	}								\
+	/* return */ _n;						\
+})
+
+#undef MIN
+#define MIN(x, y) ({							\
+	__typeof__ (x) _x = (x);					\
+	__typeof__ (y) _y = (y);					\
+	(void)(&_x == &_y); /* warn if types do not match */		\
+	/* return */ (_x < _y) ? _x : _y;				\
+})
+
+#undef MAX
+#define MAX(x, y) ({							\
+	__typeof__ (x) _x = (x);					\
+	__typeof__ (y) _y = (y);					\
+	(void)(&_x == &_y); /* warn if types do not match */		\
+	/* return */ (_x > _y) ? _x : _y;				\
+})
+
+/* Note other compilers may swap only int, long or pointer. */
+#undef SWAP
+#define SWAP(x, y)							\
+do {									\
+	__typeof__ (x) _x = x;						\
+	x = y;								\
+	y = _x;								\
+} while (0)
+
+#undef SATURATE
+#ifdef __i686__ /* has conditional move */
+#define SATURATE(n, min, max) ({					\
+	__typeof__ (n) _n = (n);					\
+	__typeof__ (n) _min = (min);					\
+	__typeof__ (n) _max = (max);					\
+	(void)(&_n == &_min); /* warn if types do not match */		\
+	(void)(&_n == &_max);						\
+	if (_n < _min)							\
+		_n = _min;						\
+	if (_n > _max)							\
+		_n = _max;						\
+	/* return */ _n;						\
+})
+#else
+#define SATURATE(n, min, max) ({					\
+	__typeof__ (n) _n = (n);					\
+	__typeof__ (n) _min = (min);					\
+	__typeof__ (n) _max = (max);					\
+	(void)(&_n == &_min); /* warn if types do not match */		\
+	(void)(&_n == &_max);						\
+	if (_n < _min)							\
+		_n = _min;						\
+	else if (_n > _max)						\
+		_n = _max;						\
+	/* return */ _n;						\
+})
+#endif
+
+#else /* !__GNUC__ */
+
+#define likely(expr) (expr)
+#define unlikely(expr) (expr)
+#undef __i386__
+#undef __i686__
+
+static char *
+PARENT_HELPER (char *p, unsigned int offset)
+{ return (0 == p) ? ((char *) 0) : p - offset; }
+
+static const char *
+CONST_PARENT_HELPER (const char *p, unsigned int offset)
+{ return (0 == p) ? ((char *) 0) : p - offset; }
+
+#define PARENT(_ptr, _type, _member)					\
+	((0 == offsetof (_type, _member)) ? (_type *)(_ptr)		\
+	 : (_type *) PARENT_HELPER ((char *)(_ptr), offsetof (_type, _member)))
+#define CONST_PARENT(_ptr, _type, _member)				\
+	((0 == offsetof (const _type, _member)) ? (const _type *)(_ptr)	\
+	 : (const _type *) CONST_PARENT_HELPER ((const char *)(_ptr),	\
+	  offsetof (const _type, _member)))
+
+#undef ABS
+#define ABS(n) (((n) < 0) ? -(n) : (n))
+
+#undef MIN
+#define MIN(x, y) (((x) < (y)) ? (x) : (y))
+
+#undef MAX
+#define MAX(x, y) (((x) > (y)) ? (x) : (y))
+
+#undef SWAP
+#define SWAP(x, y)							\
+do {									\
+	long _x = x;							\
+	x = y;								\
+	y = _x;								\
+} while (0)
+
+#undef SATURATE
+#define SATURATE(n, min, max) MIN (MAX (min, n), max)
+
+#endif /* !__GNUC__ */
+
+/* 32 bit constant byte reverse, e.g. 0xAABBCCDD -> 0xDDCCBBAA. */
+#define SWAB32(m)							\
+	(+ (((m) & 0xFF000000) >> 24)					\
+	 + (((m) & 0xFF0000) >> 8)					\
+	 + (((m) & 0xFF00) << 8)					\
+	 + (((m) & 0xFF) << 24))
+
+#ifdef HAVE_BUILTIN_POPCOUNT
+#  define popcnt(x) __builtin_popcount ((uint32_t)(x))
+#else
+#  define popcnt(x) _vbi_popcnt (x)
+#endif
+
+extern unsigned int
+_vbi_popcnt			(uint32_t		x);
+
+/* NB GCC inlines and optimizes these functions when size is const. */
+#define SET(var) memset (&(var), ~0, sizeof (var))
+
+#define CLEAR(var) memset (&(var), 0, sizeof (var))
+
+/* Useful to copy arrays, otherwise use assignment. */
+#define COPY(d, s)							\
+	(assert (sizeof (d) == sizeof (s)), memcpy (d, s, sizeof (d)))
+
+/* Copy string const into char array. */
+#define STRACPY(array, s)						\
+do {									\
+	/* Complain if s is no string const or won't fit. */		\
+	const char t_[sizeof (array) - 1] _vbi_unused = s;		\
+									\
+	memcpy (array, s, sizeof (s));					\
+} while (0)
+
+/* Copy bits through mask. */
+#define COPY_SET_MASK(dest, from, mask)					\
+	(dest ^= (from) ^ (dest & (mask)))
+
+/* Set bits if cond is TRUE, clear if FALSE. */
+#define COPY_SET_COND(dest, bits, cond)					\
+	 ((cond) ? (dest |= (bits)) : (dest &= ~(bits)))
+
+/* Set and clear bits. */
+#define COPY_SET_CLEAR(dest, set, clear)				\
+	(dest = (dest & ~(clear)) | (set))
+
+/* For applications, debugging and fault injection during unit tests. */
+
+#define vbi_malloc malloc
+#define vbi_realloc realloc
+#define vbi_strdup strdup
+#define vbi_free free
+
+#define vbi_cache_malloc vbi_malloc
+#define vbi_cache_free vbi_free
+
+/* Helper functions. */
+
+_vbi_inline int
+_vbi_to_ascii			(int			c)
+{
+	if (c < 0)
+		return '?';
+
+	c &= 0x7F;
+
+	if (c < 0x20 || c >= 0x7F)
+		return '.';
+
+	return c;
+}
+
+typedef struct {
+	const char *		key;
+	int			value;
+} _vbi_key_value_pair;
+
+extern vbi_bool
+_vbi_keyword_lookup		(int *			value,
+				 const char **		inout_s,
+				 const _vbi_key_value_pair * table,
+				 unsigned int		n_pairs)
+  _vbi_nonnull ((1, 2, 3));
+
+extern void
+_vbi_shrink_vector_capacity	(void **		vector,
+				 size_t *		capacity,
+				 size_t			min_capacity,
+				 size_t			element_size)
+  _vbi_nonnull ((1, 2));
+extern vbi_bool
+_vbi_grow_vector_capacity	(void **		vector,
+				 size_t *		capacity,
+				 size_t			min_capacity,
+				 size_t			element_size)
+  _vbi_nonnull ((1, 2));
+
+#define debug1 debug
+#define debug2 debug
+#define debug3 debug
+#define warning log
+#define error log
+#define info debug
+
+#define debug(a, fmt, ...)							\
+	ccx_common_logging.debug_ftn(CCX_DMT_PARSE, "VBI:%s:%d: "fmt , __FUNCTION__ ,__LINE__ , ##__VA_ARGS__)
+#define log(a, fmt, ...)							\
+	ccx_common_logging.log_ftn("VBI:%d: "fmt , __LINE__ , ##__VA_ARGS__)
+
+/* Portability stuff. */
+
+/* These should be defined in inttypes.h. */
+#ifndef PRId64
+#  define PRId64 "lld"
+#endif
+#ifndef PRIu64
+#  define PRIu64 "llu"
+#endif
+#ifndef PRIx64
+#  define PRIx64 "llx"
+#endif
+
+/* Should be defined in C99 limits.h? */
+#ifndef SIZE_MAX
+#  define SIZE_MAX ((size_t) -1)
+#endif
+
+#ifndef TIME_MIN
+#  define TIME_MIN (_vbi_time_min ())
+_vbi_inline time_t
+_vbi_time_min			(void)
+{
+	const time_t t = (time_t) -1.25;
+
+	if (t < -1) {
+		return (time_t)((sizeof (time_t) > 4) ? DBL_MIN : FLT_MIN);
+	} else if (t < 0) {
+		return ((uint64_t) 1) << (sizeof (time_t) * 8 - 1);
+	} else {
+		return 0;
+	}
+}
+#endif
+
+#ifndef TIME_MAX
+#  define TIME_MAX (_vbi_time_max ())
+_vbi_inline time_t
+_vbi_time_max			(void)
+{
+	const time_t t = (time_t) -1.25;
+
+	if (t < -1) {
+		return (time_t)((sizeof (time_t) > 4) ? DBL_MAX : FLT_MAX);
+	} else if (t < 0) {
+		/* Most likely signed 32 or 64 bit. */
+		return (((uint64_t) 1) << (sizeof (time_t) * 8 - 1)) - 1;
+	} else {
+		return -1;
+	}
+}
+#endif
+
+/* __va_copy is a GNU extension. */
+#ifndef __va_copy
+#  define __va_copy(ap1, ap2) do { ap1 = ap2; } while (0)
+#endif
+
+/* Use this instead of strncpy(). strlcpy() is a BSD extension. */
+#ifndef HAVE_STRLCPY
+#  define strlcpy _vbi_strlcpy
+#endif
+#undef strncpy
+#define strncpy use_strlcpy_instead
+
+extern size_t
+_vbi_strlcpy			(char *			dst,
+				 const char *		src,
+				 size_t			size)
+  _vbi_nonnull ((1, 2));
+
+/* strndup() is a BSD/GNU extension. */
+#ifndef HAVE_STRNDUP
+#  define strndup _vbi_strndup
+#endif
+
+extern char *
+_vbi_strndup			(const char *		s,
+				 size_t			len)
+  _vbi_nonnull ((1));
+
+/* vasprintf() is a GNU extension. */
+#ifndef HAVE_VASPRINTF
+#  define vasprintf _vbi_vasprintf
+#endif
+
+extern int
+_vbi_vasprintf			(char **		dstp,
+				 const char *		templ,
+				 va_list		ap)
+  _vbi_nonnull ((1, 2));
+
+/* asprintf() is a GNU extension. */
+#ifndef HAVE_ASPRINTF
+#  define asprintf _vbi_asprintf
+#endif
+
+extern int
+_vbi_asprintf			(char **		dstp,
+				 const char *		templ,
+				 ...)
+  _vbi_nonnull ((1, 2)) _vbi_format ((printf, 2, 3));
+
+#undef sprintf
+#define sprintf use_snprintf_or_asprintf_instead
+
+#endif /* MISC_H */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/raw_decoder.h

@@ -0,0 +1,212 @@
+/*
+ *  libzvbi -- Raw VBI decoder
+ *
+ *  Copyright (C) 2000-2004 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: raw_decoder.h,v 1.12 2008/02/19 00:35:21 mschimek Exp $ */
+
+#ifndef __ZVBI_RAW_DECODER_H__
+#define __ZVBI_RAW_DECODER_H__
+
+#include <stdio.h>
+
+#include "zvbi_decoder.h"
+#include "sampling_par.h"
+#include "bit_slicer.h"
+
+VBI_BEGIN_DECLS
+
+/*
+ * $ingroup RawDecoder
+ * $brief Raw VBI decoder.
+ *
+ * The contents of this structure are private.
+ * Call vbi3_raw_decoder_new() to allocate a raw VBI decoder.
+ */
+typedef struct _vbi3_raw_decoder vbi3_raw_decoder;
+
+/*
+ * $addtogroup RawDecoder
+ * ${
+ */
+extern vbi_bool
+vbi3_raw_decoder_sampling_point	(vbi3_raw_decoder *	rd,
+				 vbi3_bit_slicer_point *point,
+				 unsigned int		row,
+				 unsigned int		nth_bit);
+extern unsigned int
+vbi3_raw_decoder_decode		(vbi3_raw_decoder *	rd,
+				 vbi_sliced *		sliced,
+				 unsigned int		sliced_lines,
+				 const uint8_t *	raw);
+extern void
+vbi3_raw_decoder_reset		(vbi3_raw_decoder *	rd);
+extern vbi_service_set
+vbi3_raw_decoder_services	(vbi3_raw_decoder *	rd);
+extern vbi_service_set
+vbi3_raw_decoder_remove_services
+				(vbi3_raw_decoder *	rd,
+				 vbi_service_set	services);
+extern vbi_service_set
+vbi3_raw_decoder_add_services	(vbi3_raw_decoder *	rd,
+				 vbi_service_set	services,
+				 int			strict);
+extern vbi_bool
+vbi3_raw_decoder_debug		(vbi3_raw_decoder *	rd,
+				 vbi_bool		enable);
+extern vbi_service_set
+vbi3_raw_decoder_set_sampling_par
+				(vbi3_raw_decoder *	rd,
+				 const vbi_sampling_par *sp,
+				 int			strict);
+extern void
+vbi3_raw_decoder_get_sampling_par
+				(const vbi3_raw_decoder *rd,
+				 vbi_sampling_par *	sp);
+extern void
+vbi3_raw_decoder_set_log_fn	(vbi3_raw_decoder *	rd,
+				 vbi_log_fn *		log_fn,
+				 void *			user_data,
+				 vbi_log_mask		mask);
+extern void
+vbi3_raw_decoder_delete		(vbi3_raw_decoder *	rd);
+extern vbi3_raw_decoder *
+vbi3_raw_decoder_new		(const vbi_sampling_par *sp);
+
+/* $} */
+
+/* Private */
+
+/** @internal */
+#define _VBI3_RAW_DECODER_MAX_JOBS 8
+/** @internal */
+#define _VBI3_RAW_DECODER_MAX_WAYS 8
+
+/** @internal */
+typedef struct {
+	vbi_service_set		id;
+	vbi3_bit_slicer		slicer;
+} _vbi3_raw_decoder_job;
+
+/** @internal */
+typedef struct {
+	vbi3_bit_slicer_point   points[512];
+	unsigned int		n_points;
+} _vbi3_raw_decoder_sp_line;
+
+/**
+ * @internal
+ * Don't dereference pointers to this structure.
+ * I guarantee it will change.
+ */
+struct _vbi3_raw_decoder {
+	vbi_sampling_par	sampling;
+
+	vbi_service_set		services;
+
+	_vbi_log_hook		log;
+	vbi_bool		debug;
+
+	unsigned int		n_jobs;
+	unsigned int		n_sp_lines;
+	int			readjust;
+	int8_t *		pattern;	/* n scan lines * MAX_WAYS */
+	_vbi3_raw_decoder_job	jobs[_VBI3_RAW_DECODER_MAX_JOBS];
+	_vbi3_raw_decoder_sp_line *sp_lines;
+};
+
+/** @internal */
+typedef enum {
+	/** Requires field line numbers. */
+	_VBI_SP_LINE_NUM	= (1 << 0),
+	/** Requires field numbers. */
+	_VBI_SP_FIELD_NUM	= (1 << 1),
+} _vbi_service_par_flag;
+
+typedef struct _vbi_service_par _vbi_service_par;
+
+/** @internal */
+struct _vbi_service_par {
+	vbi_service_set		id;
+	const char *		label;
+
+	/**
+	 * Video standard
+	 * - 525 lines, FV = 59.94 Hz, FH = 15734 Hz
+	 * - 625 lines, FV = 50 Hz, FH = 15625 Hz
+	 */
+	vbi_videostd_set	videostd_set;
+
+	/**
+	 * Most scan lines used by the data service, first and last
+	 * line of first and second field. ITU-R numbering scheme.
+	 * Zero if no data from this field, requires field sync.
+	 */
+	unsigned int		first[2];
+        unsigned int		last[2];
+
+	/**
+	 * Leading edge hsync to leading edge first CRI one bit,
+	 * half amplitude points, in nanoseconds.
+	 */
+	unsigned int		offset;
+
+	unsigned int		cri_rate;	/**< Hz */
+	unsigned int		bit_rate;	/**< Hz */
+
+	/** Clock Run In and FRaming Code, LSB last txed bit of FRC. */
+	unsigned int		cri_frc;
+
+	/** CRI and FRC bits significant for identification. */
+	unsigned int		cri_frc_mask;
+
+	/**
+	 * Number of significat cri_bits (at cri_rate),
+	 * frc_bits (at bit_rate).
+	 */
+	unsigned int		cri_bits;
+	unsigned int		frc_bits;
+
+	unsigned int		payload;	/**< bits */
+	vbi_modulation		modulation;
+
+	_vbi_service_par_flag	flags;
+};
+
+extern const _vbi_service_par _vbi_service_table [];
+
+extern void
+_vbi3_raw_decoder_dump		(const vbi3_raw_decoder *rd,
+				 FILE *			fp);
+extern void
+_vbi3_raw_decoder_destroy	(vbi3_raw_decoder *	rd);
+extern vbi_bool
+_vbi3_raw_decoder_init		(vbi3_raw_decoder *	rd,
+				 const vbi_sampling_par *sp);
+
+VBI_END_DECLS
+
+#endif /* __ZVBI_RAW_DECODER_H__ */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/sampling_par.c

@@ -0,0 +1,589 @@
+/*
+ *  libzvbi -- Raw VBI sampling parameters
+ *
+ *  Copyright (C) 2000-2004 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: sampling_par.c,v 1.11 2009/02/18 15:37:11 mschimek Exp $ */
+
+#include <errno.h>
+
+#include "misc.h"
+#include "raw_decoder.h"
+#include "sampling_par.h"
+#include "sliced.h"
+#include "ccx_common_common.h" 
+
+#define vbi_pixfmt_bytes_per_pixel VBI_PIXFMT_BPP
+#define sp_sample_format sampling_format
+
+/**
+ * @addtogroup Sampling Raw VBI sampling
+ * @ingroup Raw
+ * @brief Raw VBI data sampling interface.
+ */
+
+/**
+ * @internal
+ * Compatibility.
+ */
+vbi_videostd_set
+_vbi_videostd_set_from_scanning
+				(int			scanning)
+{
+	switch (scanning) {
+	case 525:
+		return VBI_VIDEOSTD_SET_525_60;
+
+	case 625:
+		return VBI_VIDEOSTD_SET_625_50;
+
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+_vbi_inline vbi_bool
+range_check			(unsigned int		start,
+				 unsigned int		count,
+				 unsigned int		min,
+				 unsigned int		max)
+{
+	/* Check bounds and overflow. */
+	return (start >= min
+		&& (start + count) <= max
+		&& (start + count) >= start);
+}
+
+/**
+ * @internal
+ * @param sp Sampling parameters to verify.
+ * 
+ * @return
+ * CCX_TRUE if the sampling parameters are valid (as far as we can tell).
+ */
+vbi_bool
+_vbi_sampling_par_valid_log	(const vbi_sampling_par *sp,
+				 _vbi_log_hook *	log)
+{
+	vbi_videostd_set videostd_set;
+	unsigned int bpp;
+
+	assert (NULL != sp);
+
+	switch (sp->sp_sample_format) {
+	case VBI_PIXFMT_YUV420:
+		/* This conflicts with the ivtv driver, which returns an
+		   odd number of bytes per line.  The driver format is
+		   _GREY but libzvbi 0.2 has no VBI_PIXFMT_Y8. */
+		break;
+
+	default:
+		bpp = vbi_pixfmt_bytes_per_pixel (sp->sp_sample_format);
+		if (0 != (sp->bytes_per_line % bpp))
+			goto bad_samples;
+		break;
+	}
+
+	if (0 == sp->bytes_per_line)
+		goto no_samples;
+
+	if (0 == sp->count[0]
+	    && 0 == sp->count[1])
+		goto bad_range;
+
+	videostd_set = _vbi_videostd_set_from_scanning (sp->scanning);
+
+	if (VBI_VIDEOSTD_SET_525_60 & videostd_set) {
+		if (VBI_VIDEOSTD_SET_625_50 & videostd_set)
+			goto ambiguous;
+
+		if (0 != sp->start[0]
+		    && !range_check (sp->start[0], sp->count[0], 1, 262))
+			goto bad_range;
+
+		if (0 != sp->start[1]
+		    && !range_check (sp->start[1], sp->count[1], 263, 525))
+			goto bad_range;
+	} else if (VBI_VIDEOSTD_SET_625_50 & videostd_set) {
+		if (0 != sp->start[0]
+		    && !range_check (sp->start[0], sp->count[0], 1, 311))
+			goto bad_range;
+
+		if (0 != sp->start[1]
+		    && !range_check (sp->start[1], sp->count[1], 312, 625))
+			goto bad_range;
+	} else {
+	ambiguous:
+		info (log,
+		      "Ambiguous videostd_set 0x%lx.",
+		      (unsigned long) videostd_set);
+		return CCX_FALSE;
+	}
+
+	if (sp->interlaced
+	    && (sp->count[0] != sp->count[1]
+		|| 0 == sp->count[0])) {
+		info (log,
+			"Line counts %u, %u must be equal and "
+			"non-zero when raw VBI data is interlaced.",
+			sp->count[0], sp->count[1]);
+		return CCX_FALSE;
+	}
+
+	return CCX_TRUE;
+
+ no_samples:
+	info (log, "samples_per_line is zero.");
+	return CCX_FALSE;
+
+ bad_samples:
+	info (log,
+		"bytes_per_line value %u is no multiple of "
+		"the sample size %u.",
+		sp->bytes_per_line,
+		vbi_pixfmt_bytes_per_pixel (sp->sp_sample_format));
+	return CCX_FALSE;
+
+ bad_range:
+	info (log,
+		"Invalid VBI scan range %u-%u (%u lines), "
+		"%u-%u (%u lines).",
+		sp->start[0], sp->start[0] + sp->count[0] - 1,
+		sp->count[0],
+		sp->start[1], sp->start[1] + sp->count[1] - 1,
+		sp->count[1]);
+	return CCX_FALSE;
+}
+
+static vbi_bool
+_vbi_sampling_par_permit_service
+				(const vbi_sampling_par *sp,
+				 const _vbi_service_par *par,
+				 unsigned int		strict,
+				 _vbi_log_hook *	log)
+{
+	const unsigned int unknown = 0;
+	double signal;
+	unsigned int field;
+	unsigned int samples_per_line;
+	vbi_videostd_set videostd_set;
+
+	assert (NULL != sp);
+	assert (NULL != par);
+
+	videostd_set = _vbi_videostd_set_from_scanning (sp->scanning);
+	if (0 == (par->videostd_set & videostd_set)) {
+		info (log,
+		      "Service 0x%08x (%s) requires "
+		      "videostd_set 0x%lx, "
+		      "have 0x%lx.",
+		      par->id, par->label,
+		      (unsigned long) par->videostd_set,
+		      (unsigned long) videostd_set);
+		return CCX_FALSE;
+	}
+
+	if (par->flags & _VBI_SP_LINE_NUM) {
+                if ((par->first[0] > 0
+		     && unknown == (unsigned int) sp->start[0])
+                    || (par->first[1] > 0
+			&& unknown == (unsigned int) sp->start[1])) {
+			info (log,
+				"Service 0x%08x (%s) requires known "
+				"line numbers.",
+				par->id, par->label);
+			return CCX_FALSE;
+		}
+	}
+
+	{
+		unsigned int rate;
+
+		rate = MAX (par->cri_rate, par->bit_rate);
+
+		switch (par->id) {
+		case VBI_SLICED_WSS_625:
+			/* Effective bit rate is just 1/3 max_rate,
+			   so 1 * max_rate should suffice. */
+			break;
+
+		default:
+			rate = (rate * 3) >> 1;
+			break;
+		}
+
+		if (rate > (unsigned int) sp->sampling_rate) {
+			info (log,
+				"Sampling rate %f MHz too low "
+				"for service 0x%08x (%s).",
+				sp->sampling_rate / 1e6,
+				par->id, par->label);
+			return CCX_FALSE;
+		}
+	}
+
+	signal = par->cri_bits / (double) par->cri_rate
+		+ (par->frc_bits + par->payload) / (double) par->bit_rate;
+
+	samples_per_line = sp->bytes_per_line
+		/ VBI_PIXFMT_BPP (sp->sampling_format);
+
+	if (0 && sp->offset > 0 && strict > 0) {
+		double sampling_rate;
+		double offset;
+		double end;
+
+		sampling_rate = (double) sp->sampling_rate;
+
+		offset = sp->offset / sampling_rate;
+		end = (sp->offset + samples_per_line) / sampling_rate;
+
+		if (offset > (par->offset / 1e3 - 0.5e-6)) {
+			info (log,
+				"Sampling starts at 0H + %f us, too "
+				"late for service 0x%08x (%s) at "
+				"%f us.",
+				offset * 1e6,
+				par->id, par->label,
+				par->offset / 1e3);
+			return CCX_FALSE;
+		}
+
+		if (end < (par->offset / 1e9 + signal + 0.5e-6)) {
+			info (log,
+				"Sampling ends too early at 0H + "
+				"%f us for service 0x%08x (%s) "
+				"which ends at %f us",
+				end * 1e6,
+				par->id, par->label,
+				par->offset / 1e3
+				+ signal * 1e6 + 0.5);
+			return CCX_FALSE;
+		}
+	} else {
+		double samples;
+
+		samples = samples_per_line / (double) sp->sampling_rate;
+
+		if (strict > 0)
+			samples -= 1e-6; /* headroom */
+
+		if (samples < signal) {
+			info (log,
+				"Service 0x%08x (%s) signal length "
+				"%f us exceeds %f us sampling length.",
+				par->id, par->label,
+				signal * 1e6, samples * 1e6);
+			return CCX_FALSE;
+		}
+	}
+
+	if ((par->flags & _VBI_SP_FIELD_NUM)
+	    && !sp->synchronous) {
+		info (log,
+			"Service 0x%08x (%s) requires "
+			"synchronous field order.",
+			par->id, par->label);
+		return CCX_FALSE;
+	}
+
+	for (field = 0; field < 2; ++field) {
+		unsigned int start;
+		unsigned int end;
+
+		start = sp->start[field];
+		end = start + sp->count[field] - 1;
+
+		if (0 == par->first[field]
+		    || 0 == par->last[field]) {
+			/* No data on this field. */
+			continue;
+		}
+
+		if (0 == sp->count[field]) {
+			info (log,
+				"Service 0x%08x (%s) requires "
+				"data from field %u",
+				par->id, par->label, field + 1);
+			return CCX_FALSE;
+		}
+
+		/* (int) <= 0 for compatibility with libzvbi 0.2.x */
+		if ((int) strict <= 0 || 0 == sp->start[field])
+			continue;
+
+		if (1 == strict && par->first[field] > par->last[field]) {
+			/* May succeed if not all scanning lines
+			   available for the service are actually used. */
+			continue;
+		}
+		
+		if (start > par->first[field]
+		    || end < par->last[field]) {
+			info (log,
+				"Service 0x%08x (%s) requires "
+				"lines %u-%u, have %u-%u.",
+				par->id, par->label,
+				par->first[field],
+				par->last[field],
+				start, end);
+			return CCX_FALSE;
+		}
+	}
+
+	return CCX_TRUE;
+}
+
+/**
+ * @internal
+ */
+vbi_service_set
+_vbi_sampling_par_check_services_log
+				(const vbi_sampling_par *sp,
+				 vbi_service_set	services,
+				 unsigned int		strict,
+				 _vbi_log_hook *	log)
+{
+	const _vbi_service_par *par;
+	vbi_service_set rservices;
+
+	assert (NULL != sp);
+
+	rservices = 0;
+
+	for (par = _vbi_service_table; par->id; ++par) {
+		if (0 == (par->id & services))
+			continue;
+
+		if (_vbi_sampling_par_permit_service (sp, par, strict, log))
+			rservices |= par->id;
+	}
+
+	return rservices;
+}
+
+/**
+ * @internal
+ */
+vbi_service_set
+_vbi_sampling_par_from_services_log
+				(vbi_sampling_par *	sp,
+				 unsigned int *		max_rate,
+				 vbi_videostd_set	videostd_set_req,
+				 vbi_service_set	services,
+				 _vbi_log_hook *	log)
+{
+	const _vbi_service_par *par;
+	vbi_service_set rservices;
+	vbi_videostd_set videostd_set;
+	unsigned int rate;
+	unsigned int samples_per_line;
+
+	assert (NULL != sp);
+
+	videostd_set = 0;
+
+	if (0 != videostd_set_req) {
+		if (0 == (VBI_VIDEOSTD_SET_ALL & videostd_set_req)
+		    || ((VBI_VIDEOSTD_SET_525_60 & videostd_set_req)
+			&& (VBI_VIDEOSTD_SET_625_50 & videostd_set_req))) {
+			warning (log,
+				 "Ambiguous videostd_set 0x%lx.",
+				 (unsigned long) videostd_set_req);
+			CLEAR (*sp);
+			return 0;
+		}
+
+		videostd_set = videostd_set_req;
+	}
+
+	samples_per_line	= 0;
+	sp->sampling_rate	= 27000000;		/* ITU-R BT.601 */
+	sp->offset		= (int)(64e-6 * sp->sampling_rate);
+	sp->start[0]		= 30000;
+	sp->count[0]		= 0;
+	sp->start[1]		= 30000;
+	sp->count[1]		= 0;
+	sp->interlaced		= CCX_FALSE;
+	sp->synchronous		= CCX_TRUE;
+
+	rservices = 0;
+	rate = 0;
+
+	for (par = _vbi_service_table; par->id; ++par) {
+		double margin;
+		double signal;
+		int offset;
+		unsigned int samples;
+		unsigned int i;
+
+		if (0 == (par->id & services))
+			continue;
+
+		if (0 == videostd_set_req) {
+			vbi_videostd_set set;
+
+			set = par->videostd_set | videostd_set;
+
+			if (0 == (set & ~VBI_VIDEOSTD_SET_525_60)
+			    || 0 == (set & ~VBI_VIDEOSTD_SET_625_50))
+				videostd_set |= par->videostd_set;
+		}
+
+		if (VBI_VIDEOSTD_SET_525_60 & videostd_set)
+			margin = 1.0e-6;
+		else
+			margin = 2.0e-6;
+
+		if (0 == (par->videostd_set & videostd_set)) {
+			info (log,
+			      "Service 0x%08x (%s) requires "
+			      "videostd_set 0x%lx, "
+			      "have 0x%lx.",
+			      par->id, par->label,
+			      (unsigned long) par->videostd_set,
+			      (unsigned long) videostd_set);
+			continue;
+		}
+
+		rate = MAX (rate, par->cri_rate);
+		rate = MAX (rate, par->bit_rate);
+
+		signal = par->cri_bits / (double) par->cri_rate
+			+ ((par->frc_bits + par->payload) / (double) par->bit_rate);
+
+		offset = (int)((par->offset / 1e9) * sp->sampling_rate);
+		samples = (int)((signal + 1.0e-6) * sp->sampling_rate);
+
+		sp->offset = MIN (sp->offset, offset);
+
+		samples_per_line = MAX (samples_per_line + sp->offset,
+					samples + offset) - sp->offset;
+
+		for (i = 0; i < 2; ++i)
+			if (par->first[i] > 0
+			    && par->last[i] > 0) {
+				sp->start[i] = MIN
+					((unsigned int) sp->start[i],
+					 (unsigned int) par->first[i]);
+				sp->count[i] = MAX
+					((unsigned int) sp->start[i]
+					 + sp->count[i],
+					 (unsigned int) par->last[i] + 1)
+					- sp->start[i];
+			}
+
+		rservices |= par->id;
+	}
+
+	if (0 == rservices) {
+		CLEAR (*sp);
+		return 0;
+	}
+
+	if (0 == sp->count[1]) {
+		sp->start[1] = 0;
+
+		if (0 == sp->count[0]) {
+			sp->start[0] = 0;
+			sp->offset = 0;
+		}
+	} else if (0 == sp->count[0]) {
+		sp->start[0] = 0;
+	}
+
+	sp->scanning		= (videostd_set & VBI_VIDEOSTD_SET_525_60)
+		? 525 : 625;
+	sp->sp_sample_format	= VBI_PIXFMT_YUV420;
+
+	/* Note bpp is 1. */
+	sp->bytes_per_line = MAX (1440U, samples_per_line);
+
+	if (max_rate)
+		*max_rate = rate;
+
+	return rservices;
+}
+
+/**
+ * @param sp Sampling parameters to check against.
+ * @param services Set of data services.
+ * @param strict See description of vbi_raw_decoder_add_services().
+ *
+ * Check which of the given services can be decoded with the given
+ * sampling parameters at the given strictness level.
+ *
+ * @return
+ * Subset of @a services decodable with the given sampling parameters.
+ */
+vbi_service_set
+vbi_sampling_par_check_services
+				(const vbi_sampling_par *sp,
+				 vbi_service_set	services,
+				 unsigned int		strict)
+{
+	return _vbi_sampling_par_check_services_log (sp, services, strict,
+						      /* log_hook */ NULL);
+}
+
+/**
+ * @param sp Sampling parameters calculated by this function
+ *   will be stored here.
+ * @param max_rate If not NULL, the highest data bit rate in Hz of
+ *   all services requested will be stored here. The sampling rate
+ *   should be at least twice as high; @sp sampling_rate will
+ *   be set to a more reasonable value of 27 MHz, which is twice
+ *   the video sampling rate defined by ITU-R Rec. BT.601.
+ * @param videostd_set Create sampling parameters matching these
+ *   video standards. When 0 determine video standard from requested
+ *   services.
+ * @param services Set of VBI_SLICED_ symbols. Here (and only here) you
+ *   can add @c VBI_SLICED_VBI_625 or @c VBI_SLICED_VBI_525 to include all
+ *   vbi scan lines in the calculated sampling parameters.
+ *
+ * Calculate the sampling parameters required to receive and decode the
+ * requested data @a services. The @a sp sampling_format will be
+ * @c VBI_PIXFMT_Y8, offset and bytes_per_line will be set to
+ * reasonable minimums. This function can be used to initialize hardware
+ * prior to creating a vbi_raw_decoder object.
+ * 
+ * @return
+ * Subset of @a services covered by the calculated sampling parameters.
+ */
+vbi_service_set
+vbi_sampling_par_from_services	(vbi_sampling_par *	sp,
+				 unsigned int *		max_rate,
+				 vbi_videostd_set	videostd_set,
+				 vbi_service_set	services)
+{
+	return _vbi_sampling_par_from_services_log (sp, max_rate,
+						     videostd_set,
+						     services,
+						     /* log_hook */ NULL);
+}
+
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/sampling_par.h

@@ -0,0 +1,87 @@
+/*
+ *  libzvbi -- Raw VBI sampling parameters
+ *
+ *  Copyright (C) 2000-2004 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: sampling_par.h,v 1.9 2008/02/24 14:17:06 mschimek Exp $ */
+
+#ifndef __SAMPLING_PAR_H__
+#define __SAMPLING_PAR_H__
+
+#include "zvbi_decoder.h"
+
+VBI_BEGIN_DECLS
+
+/* Public */
+
+typedef vbi_raw_decoder vbi_sampling_par;
+
+#define VBI_VIDEOSTD_SET_EMPTY 0
+#define VBI_VIDEOSTD_SET_PAL_BG 1
+#define VBI_VIDEOSTD_SET_625_50 1
+#define VBI_VIDEOSTD_SET_525_60 2
+#define VBI_VIDEOSTD_SET_ALL 3
+typedef uint64_t vbi_videostd_set;
+
+/* Private */
+
+extern vbi_service_set
+vbi_sampling_par_from_services (vbi_sampling_par *    sp,
+				unsigned int *         max_rate,
+				vbi_videostd_set      videostd_set,
+				vbi_service_set       services);
+extern vbi_service_set
+vbi_sampling_par_check_services
+                                (const vbi_sampling_par *sp,
+                                 vbi_service_set       services,
+                                 unsigned int           strict)
+  _vbi_pure;
+
+extern vbi_videostd_set
+_vbi_videostd_set_from_scanning	(int			scanning);
+
+extern vbi_service_set
+_vbi_sampling_par_from_services_log
+                                (vbi_sampling_par *    sp,
+                                 unsigned int *         max_rate,
+                                 vbi_videostd_set      videostd_set,
+                                 vbi_service_set       services,
+                                 _vbi_log_hook *       log);
+extern vbi_service_set
+_vbi_sampling_par_check_services_log
+                                (const vbi_sampling_par *sp,
+                                 vbi_service_set       services,
+                                 unsigned int           strict,
+                                 _vbi_log_hook *       log)
+  _vbi_pure;
+extern vbi_bool
+_vbi_sampling_par_valid_log    (const vbi_sampling_par *sp,
+                                 _vbi_log_hook *       log)
+  _vbi_pure;
+
+VBI_END_DECLS
+
+#endif /* __SAMPLING_PAR_H__ */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/sliced.h

@@ -0,0 +1,363 @@
+/*
+ *  libzvbi -- Sliced VBI data
+ *
+ *  Copyright (C) 2000, 2001 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: sliced.h,v 1.11 2008/02/24 14:17:02 mschimek Exp $ */
+
+#ifndef SLICED_H
+#define SLICED_H
+
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+/* Public */
+
+#include <inttypes.h>
+
+/**
+ * @addtogroup Sliced Sliced VBI data
+ * @ingroup Raw
+ * @brief Definition of sliced VBI data.
+ *
+ * The output of the libzvbi raw VBI decoder, and input to the data
+ * service decoder, is VBI data in binary format as defined in this
+ * section. It is similar to the output of hardware VBI decoders
+ * and VBI data transmitted in digital TV streams.
+ */
+
+/**
+ * @name Data service symbols
+ * @ingroup Sliced
+ * @{
+ */
+
+/**
+ * @anchor VBI_SLICED_
+ * No data service, blank vbi_sliced structure.
+ */
+#define VBI_SLICED_NONE			0
+
+/**
+ * Unknown data service (vbi_dvb_demux).
+ * @since 0.2.10
+ */
+#define VBI_SLICED_UNKNOWN              0
+
+/**
+ * Antiope a.k.a. Teletext System A
+ *
+ * Reference: <a href="http://www.itu.ch">ITU-R BT.653
+ * "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 37 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_ANTIOPE              0x00002000
+/**
+ * Synonym of VBI_SLICED_ANTIOPE.
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_A           0x00002000
+
+#define VBI_SLICED_TELETEXT_B_L10_625	0x00000001
+#define VBI_SLICED_TELETEXT_B_L25_625	0x00000002
+/**
+ * Teletext System B for 625 line systems
+ *
+ * Note this is separated into Level 1.0 and Level 2.5+ since the latter
+ * permits occupation of scan line 6 which is frequently out of
+ * range of raw VBI capture drivers. Clients should request decoding of both,
+ * may then verify Level 2.5 is covered. vbi_sliced id can be
+ * VBI_SLICED_TELETEXT_B, _B_L10_625 or _B_L25_625 regardless of line number.
+ *
+ * Reference: <a href="http://www.etsi.org">EN 300 706
+ * "Enhanced Teletext specification"</a>, <a href="http://www.itu.ch">
+ * ITU-R BT.653 "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 42 of the 45 byte Teletext packet, that is
+ * without clock run-in and framing code, lsb first transmitted.
+ */
+#define VBI_SLICED_TELETEXT_B		(VBI_SLICED_TELETEXT_B_L10_625 | \
+					 VBI_SLICED_TELETEXT_B_L25_625)
+/**
+ * Synonym of VBI_SLICED_TELETEXT_B.
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_B_625	VBI_SLICED_TELETEXT_B
+
+/**
+ * Teletext System C for 625 line systems
+ *
+ * Reference: <a href="http://www.itu.ch">ITU-R BT.653
+ * "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 33 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_C_625       0x00004000
+
+/**
+ * Teletext System D for 625 line systems
+ *
+ * Reference: <a href="http://www.itu.ch">ITU-R BT.653
+ * "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 34 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_D_625       0x00008000
+
+/**
+ * Video Program System
+ *
+ * Reference: <a href="http://www.etsi.org">ETS 300 231
+ * "Specification of the domestic video Programme
+ * Delivery Control system (PDC)"</a>, <a href="http://www.irt.de">
+ * IRT 8R2 "Video-Programm-System (VPS)"</a>.
+ *
+ * vbi_sliced payload: Byte number 3 to 15 according to ETS 300 231
+ * Figure 9, lsb first transmitted.
+ */
+#define VBI_SLICED_VPS                  0x00000004
+
+/**
+ * Pseudo-VPS signal transmitted on field 2
+ *
+ * vbi_sliced payload: 13 bytes.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_VPS_F2               0x00001000
+
+#define VBI_SLICED_CAPTION_625_F1       0x00000008
+#define VBI_SLICED_CAPTION_625_F2       0x00000010
+/**
+ * Closed Caption for 625 line systems
+ *
+ * Note this is split into field one and two services since for basic
+ * caption decoding only field one is required. vbi_sliced id can be
+ * VBI_SLICED_CAPTION_625, _625_F1 or _625_F2 regardless of line number.
+ *
+ * Reference: <a href="http://global.ihs.com">EIA 608
+ * "Recommended Practice for Line 21 Data Service"</a>.
+ *
+ * vbi_sliced payload: First and second byte including parity,
+ * lsb first transmitted.
+ */
+#define VBI_SLICED_CAPTION_625		(VBI_SLICED_CAPTION_625_F1 | \
+                                         VBI_SLICED_CAPTION_625_F2)
+
+/**
+ * Wide Screen Signalling for 625 line systems
+ *
+ * Reference: <a href="http://www.etsi.org">EN 300 294
+ * "625-line television Wide Screen Signalling (WSS)"</a>.
+ *
+ * vbi_sliced payload:
+ * <pre>
+ * Byte         0                  1
+ *       msb         lsb  msb             lsb
+ * bit   7 6 5 4 3 2 1 0  x x 13 12 11 10 9 8<br></pre>
+ * according to EN 300 294, Table 1, lsb first transmitted.
+ */
+#define VBI_SLICED_WSS_625              0x00000400
+
+#define VBI_SLICED_CAPTION_525_F1	0x00000020
+#define VBI_SLICED_CAPTION_525_F2	0x00000040
+/**
+ * Closed Caption for 525 line systems (NTSC).
+ *
+ * Note this is split into field one and two services since for basic
+ * caption decoding only field one is required. vbi_sliced id can be
+ * VBI_SLICED_CAPTION_525, _525_F1 or _525_F2 regardless of line number.
+ *
+ * VBI_SLICED_CAPTION_525 also covers XDS (Extended Data Service),
+ * V-Chip data and ITV / WebTV data.
+ *
+ * Reference: <a href="http://global.ihs.com">EIA 608
+ * "Recommended Practice for Line 21 Data Service"</a>.
+ *
+ * vbi_sliced payload: First and second byte including parity,
+ * lsb first transmitted.
+ */
+#define VBI_SLICED_CAPTION_525		(VBI_SLICED_CAPTION_525_F1 | \
+					 VBI_SLICED_CAPTION_525_F2)
+/**
+ * Closed Caption at double bit rate for 525 line systems.
+ *
+ * Reference: ?
+ *
+ * vbi_sliced payload: First to fourth byte including parity bit,
+ * lsb first transmitted.
+ */
+#define VBI_SLICED_2xCAPTION_525	0x00000080
+
+/**
+ * Teletext System B for 525 line systems
+ *
+ * Reference: <a href="http://www.itu.ch">ITU-R BT.653
+ * "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 34 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_B_525       0x00010000
+
+/**
+ * North American Basic Teletext Specification
+ * a.k.a. Teletext System C for 525 line systems
+ *
+ * Reference: <a href="http://global.ihs.com">EIA-516
+ * "North American Basic Teletext Specification (NABTS)"</a>,
+ * <a href="http://www.itu.ch">ITU-R BT.653 "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 33 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_NABTS                0x00000100
+
+/**
+ * Synonym of VBI_SLICED_NABTS.
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_C_525       0x00000100
+
+/**
+ * Misdefined.
+ *
+ * vbi_sliced payload: 34 bytes.
+ *
+ * @deprecated
+ * This service was misdefined.
+ * Use VBI_SLICED_TELETEXT_B_525 or VBI_SLICED_TELETEXT_D_525 in new code.
+ */
+#define VBI_SLICED_TELETEXT_BD_525	0x00000200
+
+/**
+ * Teletext System D for 525 line systems
+ *
+ * Reference: <a href="http://www.itu.ch">ITU-R BT.653
+ * "Teletext Systems"</a>
+ *
+ * vbi_sliced payload: Last 34 bytes, without clock run-in and
+ * framing code, lsb first transmitted.
+ *
+ * @since 0.2.10
+ */
+#define VBI_SLICED_TELETEXT_D_525       0x00020000
+
+/**
+ * Wide Screen Signalling for NTSC Japan
+ *
+ * Reference: <a href="http://www.jeita.or.jp">EIA-J CPR-1204</a>
+ *
+ * vbi_sliced payload:
+ * <pre>
+ * Byte         0                    1                  2
+ *       msb         lsb  msb               lsb  msb             lsb
+ * bit   7 6 5 4 3 2 1 0  15 14 13 12 11 10 9 8  x x x x 19 18 17 16
+ * </pre>
+ */
+
+#define VBI_SLICED_WSS_CPR1204		0x00000800
+
+/**
+ * No actual data service. This symbol is used to request capturing
+ * of all PAL/SECAM VBI data lines from the libzvbi driver interface,
+ * as opposed to just those lines used to transmit the requested
+ * data services.
+ */
+#define VBI_SLICED_VBI_625		0x20000000
+
+/**
+ * No actual data service. This symbol is used to request capturing
+ * of all NTSC VBI data lines from the libzvbi driver interface,
+ * as opposed to just those lines used to transmit the requested
+ * data services.
+ */
+#define VBI_SLICED_VBI_525		0x40000000
+
+/** @} */
+
+typedef unsigned int vbi_service_set;
+
+/**
+ * @ingroup Sliced
+ * @brief This structure holds one scan line of sliced vbi data.
+ *
+ * For example the contents of NTSC line 21, two bytes of Closed Caption
+ * data. Usually an array of vbi_sliced is used, covering all
+ * VBI lines of the two fields of a video frame.
+ */
+typedef struct {
+	/**
+	 * A @ref VBI_SLICED_ symbol identifying the data service. Under cirumstances
+	 * (see VBI_SLICED_TELETEXT_B) this can be a set of VBI_SLICED_ symbols.
+	 */
+	uint32_t		id;
+	/**
+	 * Source line number according to the ITU-R line numbering scheme,
+	 * a value of @c 0 if the exact line number is unknown. Note that some
+	 * data services cannot be reliable decoded without line number.
+	 *
+	 * @image html zvbi_625.gif "ITU-R PAL/SECAM line numbering scheme"
+	 * @image html zvbi_525.gif "ITU-R NTSC line numbering scheme"
+	 */
+	uint32_t		line;
+	/**
+	 * The actual payload. See the documentation of @ref VBI_SLICED_ symbols
+	 * for details.
+	 */
+	uint8_t			data[56];
+} vbi_sliced;
+
+/**
+ * @addtogroup Sliced
+ * @{
+ */
+extern const char *vbi_sliced_name (vbi_service_set service);
+extern unsigned int vbi_sliced_payload_bits (vbi_service_set service);
+/** @} */
+
+/* Private */
+
+#ifdef __cplusplus
+}
+#endif
+
+#endif /* SLICED_H */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

src/zvbi/zvbi_decoder.h

@@ -0,0 +1,404 @@
+/*
+ *  libzvbi -- Old raw VBI decoder
+ *
+ *  Copyright (C) 2000, 2001, 2002 Michael H. Schimek
+ *
+ *  This library is free software; you can redistribute it and/or
+ *  modify it under the terms of the GNU Library General Public
+ *  License as published by the Free Software Foundation; either
+ *  version 2 of the License, or (at your option) any later version.
+ *
+ *  This library is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ *  Library General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Library General Public
+ *  License along with this library; if not, write to the 
+ *  Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, 
+ *  Boston, MA  02110-1301  USA.
+ */
+
+/* $Id: decoder.h,v 1.11 2008/02/19 00:35:15 mschimek Exp $ */
+
+#ifndef DECODER_H
+#define DECODER_H
+
+#include "bcd.h"
+#include "sliced.h"
+
+
+/* Bit slicer */
+
+/**
+ * @ingroup Rawdec
+ * @brief Image format used as source to vbi_bit_slice() and vbi_raw_decode().
+ *
+ * @htmlonly
+<table border=1>
+<tr><th>Symbol</th><th>Byte&nbsp;0</th><th>Byte&nbsp;1</th><th>Byte&nbsp;2</th><th>Byte&nbsp;3</th></tr>
+<tr><td colspan=5>Planar YUV 4:2:0 data.</td></tr>
+<tr><td>VBI_PIXFMT_YUV420</td><td colspan=4>
+ <table>
+  <tr><th>Y plane</th><th>U plane</th><th>V plane</th></tr>
+  <tr><td><table border=1>
+   <tr><td>Y00</td><td>Y01</td><td>Y02</td><td>Y03</td></tr>
+   <tr><td>Y10</td><td>Y11</td><td>Y12</td><td>Y13</td></tr>
+   <tr><td>Y20</td><td>Y21</td><td>Y22</td><td>Y23</td></tr>
+   <tr><td>Y30</td><td>Y31</td><td>Y32</td><td>Y33</td></tr>
+  </table></td>
+  <td><table border=1>
+   <tr><td>Cb00</td><td>Cb01</td></tr>
+   <tr><td>Cb10</td><td>Cb11</td></tr>
+  </table></td>
+  <td><table border=1>
+   <tr><td>Cr00</td><td>Cr01</td></tr>
+   <tr><td>Cr10</td><td>Cr11</td></tr>
+  </table></td>
+ </tr></table></td>
+</tr>
+<tr><td colspan=5>Packed YUV 4:2:2 data.</td></tr>
+<tr><td>VBI_PIXFMT_YUYV</td><td>Y0</td><td>Cb</td><td>Y1</td><td>Cr</td></tr>
+<tr><td>VBI_PIXFMT_YVYU</td><td>Y0</td><td>Cr</td><td>Y1</td><td>Cb</td></tr>
+<tr><td>VBI_PIXFMT_UYVY</td><td>Cb</td><td>Y0</td><td>Cr</td><td>Y1</td></tr>
+<tr><td>VBI_PIXFMT_VYUY</td><td>Cr</td><td>Y0</td><td>Cb</td><td>Y1</td></tr>
+<tr><td colspan=5>Packed 32 bit RGB data.</td></tr>
+<tr><td>VBI_PIXFMT_RGBA32_LE VBI_PIXFMT_ARGB32_BE</td>
+<td>r7&nbsp;...&nbsp;r0</td><td>g7&nbsp;...&nbsp;g0</td>
+<td>b7&nbsp;...&nbsp;b0</td><td>a7&nbsp;...&nbsp;a0</td></tr>
+<tr><td>VBI_PIXFMT_BGRA32_LE VBI_PIXFMT_ARGB32_BE</td>
+<td>b7&nbsp;...&nbsp;b0</td><td>g7&nbsp;...&nbsp;g0</td>
+<td>r7&nbsp;...&nbsp;r0</td><td>a7&nbsp;...&nbsp;a0</td></tr>
+<tr><td>VBI_PIXFMT_ARGB32_LE VBI_PIXFMT_BGRA32_BE</td>
+<td>a7&nbsp;...&nbsp;a0</td><td>r7&nbsp;...&nbsp;r0</td>
+<td>g7&nbsp;...&nbsp;g0</td><td>b7&nbsp;...&nbsp;b0</td></tr>
+<tr><td>VBI_PIXFMT_ABGR32_LE VBI_PIXFMT_RGBA32_BE</td>
+<td>a7&nbsp;...&nbsp;a0</td><td>b7&nbsp;...&nbsp;b0</td>
+<td>g7&nbsp;...&nbsp;g0</td><td>r7&nbsp;...&nbsp;r0</td></tr>
+<tr><td colspan=5>Packed 24 bit RGB data.</td></tr>
+<tr><td>VBI_PIXFMT_RGBA24</td>
+<td>r7&nbsp;...&nbsp;r0</td><td>g7&nbsp;...&nbsp;g0</td>
+<td>b7&nbsp;...&nbsp;b0</td><td>&nbsp;</td></tr>
+<tr><td>VBI_PIXFMT_BGRA24</td>
+<td>b7&nbsp;...&nbsp;b0</td><td>g7&nbsp;...&nbsp;g0</td>
+<td>r7&nbsp;...&nbsp;r0</td><td>&nbsp;</td></tr>
+<tr><td colspan=5>Packed 16 bit RGB data.</td></tr>
+<tr><td>VBI_PIXFMT_RGB16_LE</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0</td>
+<td>b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g5&nbsp;g4&nbsp;g3</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_BGR16_LE</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0</td>
+<td>r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g5&nbsp;g4&nbsp;g3</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_RGB16_BE</td>
+<td>b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g5&nbsp;g4&nbsp;g3</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_BGR16_BE</td>
+<td>r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g5&nbsp;g4&nbsp;g3</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr>
+<tr><td colspan=5>Packed 15 bit RGB data.</td></tr>
+<tr><td>VBI_PIXFMT_RGBA15_LE</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0</td>
+<td>a0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g4&nbsp;g3</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_BGRA15_LE</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0</td>
+<td>a0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g4&nbsp;g3</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_ARGB15_LE</td>
+<td>g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;a0</td>
+<td>b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g4&nbsp;g3&nbsp;g2</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_ABGR15_LE</td>
+<td>g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;a0</td>
+<td>r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g4&nbsp;g3&nbsp;g2</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_RGBA15_BE</td>
+<td>a0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g4&nbsp;g3</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_BGRA15_BE</td>
+<td>a0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g4&nbsp;g3</td>
+<td>g2&nbsp;g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_ARGB15_BE</td>
+<td>b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;g4&nbsp;g3&nbsp;g2</td>
+<td>g1&nbsp;g0&nbsp;r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;a0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr><tr><td>VBI_PIXFMT_ABGR15_BE</td>
+<td>r4&nbsp;r3&nbsp;r2&nbsp;r1&nbsp;r0&nbsp;g4&nbsp;g3&nbsp;g2</td>
+<td>g1&nbsp;g0&nbsp;b4&nbsp;b3&nbsp;b2&nbsp;b1&nbsp;b0&nbsp;a0</td>
+<td>&nbsp;</td><td>&nbsp;</td></tr>
+</table>
+@endhtmlonly */
+/* Attn: keep this in sync with rte, don't change order */
+typedef enum {
+	VBI_PIXFMT_YUV420 = 1,
+	VBI_PIXFMT_YUYV,
+	VBI_PIXFMT_YVYU,
+	VBI_PIXFMT_UYVY,
+	VBI_PIXFMT_VYUY,
+        VBI_PIXFMT_PAL8,
+	VBI_PIXFMT_RGBA32_LE = 32,
+	VBI_PIXFMT_RGBA32_BE,
+	VBI_PIXFMT_BGRA32_LE,
+	VBI_PIXFMT_BGRA32_BE,
+	VBI_PIXFMT_ABGR32_BE = 32, /* synonyms */
+	VBI_PIXFMT_ABGR32_LE,
+	VBI_PIXFMT_ARGB32_BE,
+	VBI_PIXFMT_ARGB32_LE,
+	VBI_PIXFMT_RGB24,
+	VBI_PIXFMT_BGR24,
+	VBI_PIXFMT_RGB16_LE,
+	VBI_PIXFMT_RGB16_BE,
+	VBI_PIXFMT_BGR16_LE,
+	VBI_PIXFMT_BGR16_BE,
+	VBI_PIXFMT_RGBA15_LE,
+	VBI_PIXFMT_RGBA15_BE,
+	VBI_PIXFMT_BGRA15_LE,
+	VBI_PIXFMT_BGRA15_BE,
+	VBI_PIXFMT_ARGB15_LE,
+	VBI_PIXFMT_ARGB15_BE,
+	VBI_PIXFMT_ABGR15_LE,
+	VBI_PIXFMT_ABGR15_BE
+} vbi_pixfmt;
+
+/* Private */
+
+typedef uint64_t vbi_pixfmt_set;
+
+#define VBI_MAX_PIXFMTS 64
+#define VBI_PIXFMT_SET(pixfmt) (((vbi_pixfmt_set) 1) << (pixfmt))
+#define VBI_PIXFMT_SET_YUV (VBI_PIXFMT_SET (VBI_PIXFMT_YUV420) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_YUYV) |		\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_YVYU) |		\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_UYVY) |		\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_VYUY))
+#define VBI_PIXFMT_SET_RGB (VBI_PIXFMT_SET (VBI_PIXFMT_RGBA32_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGBA32_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGRA32_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGRA32_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGB24) |		\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGR24) |		\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGB16_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGB16_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGR16_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGR16_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGBA15_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_RGBA15_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGRA15_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_BGRA15_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_ARGB15_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_ARGB15_BE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_ABGR15_LE) |	\
+			    VBI_PIXFMT_SET (VBI_PIXFMT_ABGR15_BE))
+#define VBI_PIXFMT_SET_ALL (VBI_PIXFMT_SET_YUV |			\
+			    VBI_PIXFMT_SET_RGB)
+
+#define VBI_PIXFMT_BPP(fmt)						\
+	(((fmt) == VBI_PIXFMT_YUV420) ? 1 :				\
+	 (((fmt) >= VBI_PIXFMT_RGBA32_LE				\
+	   && (fmt) <= VBI_PIXFMT_BGRA32_BE) ? 4 :			\
+	  (((fmt) == VBI_PIXFMT_RGB24					\
+	    || (fmt) == VBI_PIXFMT_BGR24) ? 3 : 2)))
+
+/* Public */
+
+/**
+ * @ingroup Rawdec
+ * @brief Modulation used for VBI data transmission.
+ */
+typedef enum {
+	/**
+	 * The data is 'non-return to zero' coded, logical '1' bits
+	 * are described by high sample values, logical '0' bits by
+	 * low values. The data is last significant bit first transmitted.
+	 */
+	VBI_MODULATION_NRZ_LSB,
+	/**
+	 * 'Non-return to zero' coded, most significant bit first
+	 * transmitted.
+	 */
+	VBI_MODULATION_NRZ_MSB,
+	/**
+	 * The data is 'bi-phase' coded. Each data bit is described
+	 * by two complementary signalling elements, a logical '1'
+	 * by a sequence of '10' elements, a logical '0' by a '01'
+	 * sequence. The data is last significant bit first transmitted.
+	 */
+	VBI_MODULATION_BIPHASE_LSB,
+	/**
+	 * 'Bi-phase' coded, most significant bit first transmitted.
+	 */
+	VBI_MODULATION_BIPHASE_MSB
+} vbi_modulation;
+
+/**
+ * @ingroup Rawdec
+ * @brief Bit slicer context.
+ *
+ * The contents of this structure are private,
+ * use vbi_bit_slicer_init() to initialize.
+ */
+typedef struct vbi_bit_slicer {
+	int	(* func)(struct vbi_bit_slicer *slicer,
+				 uint8_t *raw, uint8_t *buf);
+	unsigned int	cri;
+	unsigned int	cri_mask;
+	int		thresh;
+	int		cri_bytes;
+	int		cri_rate;
+	int		oversampling_rate;
+	int		phase_shift;
+	int		step;
+	unsigned int	frc;
+	int		frc_bits;
+	int		payload;
+	int		endian;
+	int		skip;
+} vbi_bit_slicer;
+
+/**
+ * @addtogroup Rawdec
+ * @{
+ */
+extern void		vbi_bit_slicer_init(vbi_bit_slicer *slicer,
+					    int raw_samples, int sampling_rate,
+					    int cri_rate, int bit_rate,
+					    unsigned int cri_frc, unsigned int cri_mask,
+					    int cri_bits, int frc_bits, int payload,
+					    vbi_modulation modulation, vbi_pixfmt fmt);
+/**
+ * @param slicer Pointer to initialized vbi_bit_slicer object.
+ * @param raw Input data. At least the number of pixels or samples
+ *  given as @a raw_samples to vbi_bit_slicer_init().
+ * @param buf Output data. The buffer must be large enough to store
+ *   the number of bits given as @a payload to vbi_bit_slicer_init().
+ * 
+ * Decode one scan line of raw vbi data. Note the bit slicer tries
+ * to adapt to the average signal amplitude, you should avoid
+ * using the same vbi_bit_slicer object for data from different
+ * devices.
+ *
+ * @note As a matter of speed this function does not lock the
+ * @a slicer. When you want to share a vbi_bit_slicer object between
+ * multiple threads you must implement your own locking mechanism.
+ * 
+ * @return
+ * @c FALSE if the raw data does not contain the expected
+ * information, i. e. the CRI/FRC has not been found. This may also
+ * result from a too weak or noisy signal. Error correction must be
+ * implemented at a higher layer.
+ */
+inline int
+vbi_bit_slice(vbi_bit_slicer *slicer, uint8_t *raw, uint8_t *buf)
+{
+	return slicer->func(slicer, raw, buf);
+}
+/** @} */
+
+/**
+ * @ingroup Rawdec
+ * @brief Raw vbi decoder context.
+ *
+ * Only the sampling parameters are public. See
+ * vbi_raw_decoder_parameters() and vbi_raw_decoder_add_services()
+ * for usage.
+ */
+typedef struct vbi_raw_decoder {
+	/* Sampling parameters */
+
+	/**
+	 * Either 525 (M/NTSC, M/PAL) or 625 (PAL, SECAM), describing the
+	 * scan line system all line numbers refer to.
+	 */
+	int			scanning;
+	/**
+	 * Format of the raw vbi data.
+	 */
+	vbi_pixfmt		sampling_format;
+	/**
+	 * Sampling rate in Hz, the number of samples or pixels
+	 * captured per second.
+	 */
+	int			sampling_rate;		/* Hz */
+	/**
+	 * Number of samples or pixels captured per scan line,
+	 * in bytes. This determines the raw vbi image width and you
+	 * want it large enough to cover all data transmitted in the line (with
+	 * headroom).
+	 */
+	int			bytes_per_line;
+	/**
+	 * The distance from 0H (leading edge hsync, half amplitude point)
+	 * to the first sample (pixel) captured, in samples (pixels). You want
+	 * an offset small enough not to miss the start of the data
+	 * transmitted.
+	 */
+	int			offset;			/* 0H, samples */
+	/**
+	 * First scan line to be captured, first and second field
+	 * respectively, according to the ITU-R line numbering scheme
+	 * (see vbi_sliced). Set to zero if the exact line number isn't
+	 * known.
+	 */
+	int			start[2];		/* ITU-R numbering */
+	/**
+	 * Number of scan lines captured, first and second
+	 * field respectively. This can be zero if only data from one
+	 * field is required. The sum @a count[0] + @a count[1] determines the
+	 * raw vbi image height.
+	 */
+	int			count[2];		/* field lines */
+	/**
+	 * In the raw vbi image, normally all lines of the second
+	 * field are supposed to follow all lines of the first field. When
+	 * this flag is set, the scan lines of first and second field
+	 * will be interleaved in memory. This implies @a count[0] and @a count[1]
+	 * are equal.
+	 */
+	int		interlaced;
+	/**
+	 * Fields must be stored in temporal order, i. e. as the
+	 * lines have been captured. It is assumed that the first field is
+	 * also stored first in memory, however if the hardware cannot reliable
+	 * distinguish fields this flag shall be cleared, which disables
+	 * decoding of data services depending on the field number.
+	 */
+	int		synchronous;
+
+	unsigned int		services;
+	int			num_jobs;
+
+	int8_t *		pattern;
+	struct _vbi_raw_decoder_job {
+		unsigned int		id;
+		int			offset;
+		vbi_bit_slicer		slicer;
+	}			jobs[8];
+} vbi_raw_decoder;
+
+/**
+ * @addtogroup Rawdec
+ * @{
+ */
+extern void		vbi_raw_decoder_init(vbi_raw_decoder *rd);
+extern void		vbi_raw_decoder_reset(vbi_raw_decoder *rd);
+extern void		vbi_raw_decoder_destroy(vbi_raw_decoder *rd);
+extern unsigned int	vbi_raw_decoder_add_services(vbi_raw_decoder *rd,
+						     unsigned int services,
+						     int strict);
+extern unsigned int     vbi_raw_decoder_check_services(vbi_raw_decoder *rd,
+						     unsigned int services, int strict);
+extern unsigned int	vbi_raw_decoder_remove_services(vbi_raw_decoder *rd,
+							unsigned int services);
+extern void             vbi_raw_decoder_resize( vbi_raw_decoder *rd,
+						int * start, unsigned int * count );
+extern unsigned int	vbi_raw_decoder_parameters(vbi_raw_decoder *rd, unsigned int services,
+						   int scanning, int *max_rate);
+extern int		vbi_raw_decode(vbi_raw_decoder *rd, uint8_t *raw, vbi_sliced *out);
+/** @} */
+
+/* Private */
+
+#endif /* DECODER_H */
+
+/*
+Local variables:
+c-set-style: K&R
+c-basic-offset: 8
+End:
+*/

windows/ccextractor.sln

@@ -8,13 +8,19 @@ EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Win32 = Debug|Win32
+		Debug-OCR|Win32 = Debug-OCR|Win32
 		Release|Win32 = Release|Win32
+		Release-OCR|Win32 = Release-OCR|Win32
 	EndGlobalSection
 	GlobalSection(ProjectConfigurationPlatforms) = postSolution
 		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Debug|Win32.ActiveCfg = Debug|Win32
 		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Debug|Win32.Build.0 = Debug|Win32
+		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Debug-OCR|Win32.ActiveCfg = Debug-OCR|Win32
+		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Debug-OCR|Win32.Build.0 = Debug-OCR|Win32
 		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Release|Win32.ActiveCfg = Release|Win32
 		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Release|Win32.Build.0 = Release|Win32
+		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Release-OCR|Win32.ActiveCfg = Release-OCR|Win32
+		{0F0063C4-BCBC-4379-A6D5-84A5669C940A}.Release-OCR|Win32.Build.0 = Release-OCR|Win32
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE

windows/ccextractor.vcproj

@@ -1,744 +0,0 @@
-<?xml version="1.0" encoding="UTF-8"?>
-<VisualStudioProject
-	ProjectType="Visual C++"
-	Version="9.00"
-	Name="ccextractor"
-	ProjectGUID="{0F0063C4-BCBC-4379-A6D5-84A5669C940A}"
-	RootNamespace="ccextractor"
-	Keyword="Win32Proj"
-	TargetFrameworkVersion="0"
-	>
-	<Platforms>
-		<Platform
-			Name="Win32"
-		/>
-	</Platforms>
-	<ToolFiles>
-	</ToolFiles>
-	<Configurations>
-		<Configuration
-			Name="Debug|Win32"
-			OutputDirectory="Debug"
-			IntermediateDirectory="Debug"
-			ConfigurationType="1"
-			>
-			<Tool
-				Name="VCPreBuildEventTool"
-			/>
-			<Tool
-				Name="VCCustomBuildTool"
-			/>
-			<Tool
-				Name="VCXMLDataGeneratorTool"
-			/>
-			<Tool
-				Name="VCWebServiceProxyGeneratorTool"
-			/>
-			<Tool
-				Name="VCMIDLTool"
-			/>
-			<Tool
-				Name="VCCLCompilerTool"
-				Optimization="0"
-				AdditionalIncludeDirectories="../src/gpacmp4;../src/libpng;../src/zlib"
-				PreprocessorDefinitions="WIN32;_DEBUG;_CONSOLE;_FILE_OFFSET_BITS=64"
-				MinimalRebuild="true"
-				BasicRuntimeChecks="3"
-				RuntimeLibrary="3"
-				UsePrecompiledHeader="0"
-				WarningLevel="3"
-				Detect64BitPortabilityProblems="true"
-				DebugInformationFormat="4"
-			/>
-			<Tool
-				Name="VCManagedResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCPreLinkEventTool"
-			/>
-			<Tool
-				Name="VCLinkerTool"
-				AdditionalDependencies="WS2_32.Lib"
-				ShowProgress="0"
-				LinkIncremental="2"
-				AdditionalLibraryDirectories=""
-				GenerateDebugInformation="true"
-				SubSystem="1"
-				TargetMachine="1"
-			/>
-			<Tool
-				Name="VCALinkTool"
-			/>
-			<Tool
-				Name="VCManifestTool"
-			/>
-			<Tool
-				Name="VCXDCMakeTool"
-			/>
-			<Tool
-				Name="VCBscMakeTool"
-			/>
-			<Tool
-				Name="VCFxCopTool"
-			/>
-			<Tool
-				Name="VCAppVerifierTool"
-			/>
-			<Tool
-				Name="VCPostBuildEventTool"
-			/>
-		</Configuration>
-		<Configuration
-			Name="Release|Win32"
-			OutputDirectory="Release"
-			IntermediateDirectory="Release"
-			ConfigurationType="1"
-			>
-			<Tool
-				Name="VCPreBuildEventTool"
-			/>
-			<Tool
-				Name="VCCustomBuildTool"
-			/>
-			<Tool
-				Name="VCXMLDataGeneratorTool"
-			/>
-			<Tool
-				Name="VCWebServiceProxyGeneratorTool"
-			/>
-			<Tool
-				Name="VCMIDLTool"
-			/>
-			<Tool
-				Name="VCCLCompilerTool"
-				AdditionalIncludeDirectories="../src/gpacmp4;../src/libpng;../src/zlib"
-				PreprocessorDefinitions="WIN32;NDEBUG;_CONSOLE;_FILE_OFFSET_BITS=64"
-				RuntimeLibrary="2"
-				UsePrecompiledHeader="0"
-				WarningLevel="3"
-				Detect64BitPortabilityProblems="true"
-				DebugInformationFormat="3"
-			/>
-			<Tool
-				Name="VCManagedResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCResourceCompilerTool"
-			/>
-			<Tool
-				Name="VCPreLinkEventTool"
-			/>
-			<Tool
-				Name="VCLinkerTool"
-				AdditionalDependencies="WS2_32.Lib"
-				LinkIncremental="1"
-				AdditionalLibraryDirectories=""
-				GenerateDebugInformation="true"
-				SubSystem="1"
-				OptimizeReferences="2"
-				EnableCOMDATFolding="2"
-				TargetMachine="1"
-			/>
-			<Tool
-				Name="VCALinkTool"
-			/>
-			<Tool
-				Name="VCManifestTool"
-			/>
-			<Tool
-				Name="VCXDCMakeTool"
-			/>
-			<Tool
-				Name="VCBscMakeTool"
-			/>
-			<Tool
-				Name="VCFxCopTool"
-			/>
-			<Tool
-				Name="VCAppVerifierTool"
-			/>
-			<Tool
-				Name="VCPostBuildEventTool"
-			/>
-		</Configuration>
-	</Configurations>
-	<References>
-	</References>
-	<Files>
-		<Filter
-			Name="Header Files"
-			Filter="h;hpp;hxx;hm;inl;inc;xsd"
-			UniqueIdentifier="{93995380-89BD-4b04-88EB-625FBE52EBFB}"
-			>
-			<File
-				RelativePath="..\src\608.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_spupng.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\708.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\avilib.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\avparse.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\base_coding.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\bitstream.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\bitstream.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\ccextractor.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\configuration.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\constants.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\crc32.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\deflate.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\disable_warnings.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\gzguts.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\hamming.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\ietf.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inffast.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inffixed.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inflate.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inftrees.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\inttypes.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\isomedia.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\isomedia_dev.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\list.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\math.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\media_dev.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\media_tools.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\mpeg4_odf.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\network.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\odf_dev.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\odf_parse_common.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\internal\ogg.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\png.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngconf.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\setup.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\stdint.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\stdintmsc.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\sync_layer.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\targetver.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\teletext.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\tools.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\trees.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\utf.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac\version.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\zconf.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\zlib.h"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\zutil.h"
-				>
-			</File>
-		</Filter>
-		<Filter
-			Name="Resource Files"
-			Filter="rc;ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe;resx"
-			UniqueIdentifier="{67DA6AB6-F800-4c08-8B7A-83BB121AAD01}"
-			>
-		</Filter>
-		<Filter
-			Name="Source Files"
-			Filter="cpp;c;cc;cxx;def;odl;idl;hpj;bat;asm;asmx"
-			UniqueIdentifier="{4FC737F1-C7A5-4376-A066-2A32D752A2FF}"
-			>
-			<File
-				RelativePath="..\src\608.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_helpers.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_sami.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_smptett.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_spupng.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\608_srt.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\708.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\708_encoding.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\activity.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\adler32.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\asf_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\av_parsers.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\avc_ext.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\avc_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\avilib.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\base_encoding.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\bitstream.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_code_3gpp.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_code_apple.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_code_base.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_code_isma.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_code_meta.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\box_funcs.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\cc_bitstream.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\cc_decoders_common.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\ccextractor.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\constants.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\crc32.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\data_map.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\deflate.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\desc_private.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\descriptors.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\encoding.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\error.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\es_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\es_userdata.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\file_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\general_loop.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\gpac_ogg.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\hinting.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inffast.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inflate.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\inftrees.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\ipmpx_code.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\ipmpx_parse.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\isom_intern.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\isom_read.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\isom_store.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\isom_write.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\list.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\media.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\media_odf.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\meta.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\movie_fragments.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\mp4.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\myth.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\odf_code.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\odf_codec.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\odf_command.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\os_divers.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\output.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\params.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\params_dump.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\png.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngerror.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngmem.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngset.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngtrans.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngwio.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngwrite.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngwtran.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\libpng\pngwutil.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\qos.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\sample_descs.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\sequencing.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\slc.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\stbl_read.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\stbl_write.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\stream_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\telxcc.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\timing.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\track.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\trees.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\ts_functions.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\tx3g.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\url.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\gpacmp4\utf.c"
-				>
-			</File>
-			<File
-				RelativePath="..\src\utility.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\xds.cpp"
-				>
-			</File>
-			<File
-				RelativePath="..\src\zlib\zutil.c"
-				>
-			</File>
-		</Filter>
-	</Files>
-	<Globals>
-	</Globals>
-</VisualStudioProject>

windows/ccextractor.vcxproj

@@ -1,16 +1,25 @@
 <?xml version="1.0" encoding="utf-8"?>
 <Project DefaultTargets="Build" ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
   <ItemGroup Label="ProjectConfigurations">
+    <ProjectConfiguration Include="Debug-OCR|Win32">
+      <Configuration>Debug-OCR</Configuration>
+      <Platform>Win32</Platform>
+    </ProjectConfiguration>
     <ProjectConfiguration Include="Debug|Win32">
       <Configuration>Debug</Configuration>
       <Platform>Win32</Platform>
     </ProjectConfiguration>
+    <ProjectConfiguration Include="Release-OCR|Win32">
+      <Configuration>Release-OCR</Configuration>
+      <Platform>Win32</Platform>
+    </ProjectConfiguration>
     <ProjectConfiguration Include="Release|Win32">
       <Configuration>Release</Configuration>
       <Platform>Win32</Platform>
     </ProjectConfiguration>
   </ItemGroup>
   <ItemGroup>
+    <ClInclude Include="..\src\lib_ccx\compile_info.h" />
     <ClInclude Include="..\src\gpacmp4\gpac\avparse.h" />
     <ClInclude Include="..\src\gpacmp4\gpac\base_coding.h" />
     <ClInclude Include="..\src\gpacmp4\gpac\bitstream.h" />
@@ -66,6 +75,9 @@
     <ClInclude Include="..\src\lib_ccx\spupng_encoder.h" />
     <ClInclude Include="..\src\lib_ccx\teletext.h" />
     <ClInclude Include="..\src\lib_ccx\utility.h" />
+    <ClInclude Include="..\src\lib_hash\sha2.h" />
+    <ClInclude Include="..\src\win_iconv\iconv.h" />
+    <ClInclude Include="..\src\win_spec_incld\dirent.h" />
     <ClInclude Include="..\src\zlib\crc32.h" />
     <ClInclude Include="..\src\zlib\deflate.h" />
     <ClInclude Include="..\src\zlib\gzguts.h" />
@@ -77,8 +89,16 @@
     <ClInclude Include="..\src\zlib\zconf.h" />
     <ClInclude Include="..\src\zlib\zlib.h" />
     <ClInclude Include="..\src\zlib\zutil.h" />
-    <ClInclude Include="include\inttypes.h" />
-    <ClInclude Include="include\stdint.h" />
+    <ClInclude Include="..\src\zvbi\bcd.h" />
+    <ClInclude Include="..\src\zvbi\bit_slicer.h" />
+    <ClInclude Include="..\src\zvbi\macros.h" />
+    <ClInclude Include="..\src\zvbi\misc.h" />
+    <ClInclude Include="..\src\zvbi\raw_decoder.h" />
+    <ClInclude Include="..\src\zvbi\sampling_par.h" />
+    <ClInclude Include="..\src\zvbi\sliced.h" />
+    <ClInclude Include="..\src\zvbi\zvbi_decoder.h" />
+    <ClInclude Include="..\src\win_spec_incld\inttypes.h" />
+    <ClInclude Include="..\src\win_spec_incld\stdint.h" />
   </ItemGroup>
   <ItemGroup>
     <ClCompile Include="..\src\ccextractor.c" />
@@ -153,10 +173,12 @@
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_708_output.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_common.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_isdb.c" />
+    <ClCompile Include="..\src\lib_ccx\ccx_decoders_vbi.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_xds.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_demuxer.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_dtvcc.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_common.c" />
+    <ClCompile Include="..\src\lib_ccx\ccx_encoders_g608.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_helpers.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_sami.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_smptett.c" />
@@ -164,6 +186,7 @@
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_srt.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_webvtt.c" />
     <ClCompile Include="..\src\lib_ccx\ccx_encoders_xds.c" />
+    <ClCompile Include="..\src\lib_ccx\ccx_gxf.c" />
     <ClCompile Include="..\src\lib_ccx\cc_bitstream.c" />
     <ClCompile Include="..\src\lib_ccx\configuration.c" />
     <ClCompile Include="..\src\lib_ccx\dvb_subtitle_decoder.c" />
@@ -188,6 +211,7 @@
     <ClCompile Include="..\src\lib_ccx\ts_tables_epg.c" />
     <ClCompile Include="..\src\lib_ccx\utility.c" />
     <ClCompile Include="..\src\lib_ccx\wtv_functions.c" />
+    <ClCompile Include="..\src\lib_hash\sha2.c" />
     <ClCompile Include="..\src\win_iconv\win_iconv.c" />
     <ClCompile Include="..\src\zlib\adler32.c" />
     <ClCompile Include="..\src\zlib\crc32.c" />
@@ -197,6 +221,10 @@
     <ClCompile Include="..\src\zlib\inftrees.c" />
     <ClCompile Include="..\src\zlib\trees.c" />
     <ClCompile Include="..\src\zlib\zutil.c" />
+    <ClCompile Include="..\src\zvbi\bit_slicer.c" />
+    <ClCompile Include="..\src\zvbi\decoder.c" />
+    <ClCompile Include="..\src\zvbi\raw_decoder.c" />
+    <ClCompile Include="..\src\zvbi\sampling_par.c" />
   </ItemGroup>
   <PropertyGroup Label="Globals">
     <ProjectGuid>{0F0063C4-BCBC-4379-A6D5-84A5669C940A}</ProjectGuid>
@@ -206,12 +234,20 @@
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.Default.props" />
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'" Label="Configuration">
     <ConfigurationType>Application</ConfigurationType>
-    <PlatformToolset>v140</PlatformToolset>
+    <PlatformToolset>v120</PlatformToolset>
   </PropertyGroup>
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="Configuration">
     <ConfigurationType>Application</ConfigurationType>
     <PlatformToolset>v120</PlatformToolset>
   </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug-OCR|Win32'" Label="Configuration">
+    <ConfigurationType>Application</ConfigurationType>
+    <PlatformToolset>v120</PlatformToolset>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release-OCR|Win32'" Label="Configuration">
+    <ConfigurationType>Application</ConfigurationType>
+    <PlatformToolset>v120</PlatformToolset>
+  </PropertyGroup>
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
   <ImportGroup Label="ExtensionSettings">
   </ImportGroup>
@@ -221,6 +257,12 @@
   <ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'" Label="PropertySheets">
     <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
   </ImportGroup>
+  <ImportGroup Condition="'$(Configuration)|$(Platform)'=='Debug-OCR|Win32'" Label="PropertySheets">
+    <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
+  </ImportGroup>
+  <ImportGroup Condition="'$(Configuration)|$(Platform)'=='Release-OCR|Win32'" Label="PropertySheets">
+    <Import Project="$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props" Condition="exists('$(UserRootDir)\Microsoft.Cpp.$(Platform).user.props')" Label="LocalAppDataPlatform" />
+  </ImportGroup>
   <PropertyGroup Label="UserMacros" />
   <PropertyGroup>
     <_ProjectFileVersion>12.0.21005.1</_ProjectFileVersion>
@@ -229,16 +271,34 @@
     <OutDir>Debug\</OutDir>
     <IntDir>Debug\</IntDir>
     <LinkIncremental>true</LinkIncremental>
+    <TargetName>ccextractorwin</TargetName>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug-OCR|Win32'">
+    <OutDir>Debug-OCR\</OutDir>
+    <IntDir>Debug-OCR\</IntDir>
+    <LinkIncremental>true</LinkIncremental>
+    <TargetName>ccextractorwin</TargetName>
+    <IncludePath>$(ProjectDir)libs\tesseract\include\tesseract;$(ProjectDir)libs\leptonica\include\leptonica;$(IncludePath)</IncludePath>
+    <LibraryPath>$(ProjectDir)libs\tesseract\lib;$(ProjectDir)libs\leptonica\lib;$(VC_LibraryPath_x86);$(WindowsSDK_LibraryPath_x86)</LibraryPath>
+  </PropertyGroup>
+  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release-OCR|Win32'">
+    <OutDir>Release-OCR\</OutDir>
+    <IntDir>Release-OCR\</IntDir>
+    <LinkIncremental>false</LinkIncremental>
+    <TargetName>ccextractorwin</TargetName>
+    <IncludePath>$(ProjectDir)libs\tesseract\include\tesseract;$(ProjectDir)libs\leptonica\include\leptonica;$(IncludePath)</IncludePath>
+    <LibraryPath>$(ProjectDir)libs\tesseract\lib;$(ProjectDir)libs\leptonica\lib;$(VC_LibraryPath_x86);$(WindowsSDK_LibraryPath_x86)</LibraryPath>
   </PropertyGroup>
   <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
     <OutDir>Release\</OutDir>
     <IntDir>Release\</IntDir>
     <LinkIncremental>false</LinkIncremental>
+    <TargetName>ccextractorwin</TargetName>
   </PropertyGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
     <ClCompile>
       <Optimization>Disabled</Optimization>
-      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/gpacmp4;../src/libpng;../src/zlib;../src;../src/lib_ccx;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/gpacmp4;../src/libpng;../src/zlib;../src;../src/lib_ccx;../src/zvbi;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
       <PreprocessorDefinitions>WIN32;_DEBUG;_CONSOLE;_FILE_OFFSET_BITS=64;%(PreprocessorDefinitions)</PreprocessorDefinitions>
       <MinimalRebuild>true</MinimalRebuild>
       <BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
@@ -255,9 +315,57 @@
       <TargetMachine>MachineX86</TargetMachine>
     </Link>
   </ItemDefinitionGroup>
+  <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug-OCR|Win32'">
+    <ClCompile>
+      <Optimization>Disabled</Optimization>
+      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/gpacmp4;../src/libpng;../src/zlib;../src;../src/lib_ccx;../src/zvbi;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <PreprocessorDefinitions>ENABLE_OCR;WIN32;_DEBUG;_CONSOLE;_FILE_OFFSET_BITS=64;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <MinimalRebuild>true</MinimalRebuild>
+      <BasicRuntimeChecks>EnableFastChecks</BasicRuntimeChecks>
+      <RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
+      <PrecompiledHeader>
+      </PrecompiledHeader>
+      <WarningLevel>Level3</WarningLevel>
+      <DebugInformationFormat>EditAndContinue</DebugInformationFormat>
+    </ClCompile>
+    <Link>
+      <AdditionalDependencies>WS2_32.Lib;libtesseract302.lib;liblept168.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <ShowProgress>NotSet</ShowProgress>
+      <GenerateDebugInformation>true</GenerateDebugInformation>
+      <SubSystem>Console</SubSystem>
+      <TargetMachine>MachineX86</TargetMachine>
+    </Link>
+    <PostBuildEvent>
+      <Command>xcopy /y $(ProjectDir)libs\leptonica\lib\liblept168.dll $(ProjectDir)$(OutDir)
+xcopy /y $(ProjectDir)libs\tesseract\lib\libtesseract302.dll $(ProjectDir)$(OutDir)</Command>
+    </PostBuildEvent>
+  </ItemDefinitionGroup>
+  <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release-OCR|Win32'">
+    <ClCompile>
+      <Optimization>Disabled</Optimization>
+      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/gpacmp4;../src/libpng;../src/zlib;../src;../src/lib_ccx;../src/zvbi;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <PreprocessorDefinitions>ENABLE_OCR;WIN32;NDEBUG;_CONSOLE;_FILE_OFFSET_BITS=64;%(PreprocessorDefinitions)</PreprocessorDefinitions>
+      <RuntimeLibrary>MultiThreadedDebugDLL</RuntimeLibrary>
+      <PrecompiledHeader />
+      <WarningLevel>Level3</WarningLevel>
+      <DebugInformationFormat>ProgramDatabase</DebugInformationFormat>
+    </ClCompile>
+    <Link>
+      <AdditionalDependencies>WS2_32.Lib;libtesseract302.lib;liblept168.lib;%(AdditionalDependencies)</AdditionalDependencies>
+      <GenerateDebugInformation>true</GenerateDebugInformation>
+      <SubSystem>Console</SubSystem>
+      <OptimizeReferences>true</OptimizeReferences>
+      <EnableCOMDATFolding>true</EnableCOMDATFolding>
+      <TargetMachine>MachineX86</TargetMachine>
+    </Link>
+    <PostBuildEvent>
+      <Command>xcopy /y $(ProjectDir)libs\leptonica\lib\liblept168.dll $(ProjectDir)$(OutDir)
+xcopy /y $(ProjectDir)libs\tesseract\lib\libtesseract302.dll $(ProjectDir)$(OutDir)</Command>
+    </PostBuildEvent>
+  </ItemDefinitionGroup>
   <ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
     <ClCompile>
-      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/lib_ccx;../src/gpacmp4;../src/libpng;../src/zlib;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
+      <AdditionalIncludeDirectories>../src/win_spec_incld;../src/lib_ccx;../src/lib_hash;../src/gpacmp4;../src/libpng;../src/zlib;../src/zvbi;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
       <PreprocessorDefinitions>WIN32;NDEBUG;_CONSOLE;_FILE_OFFSET_BITS=64;%(PreprocessorDefinitions)</PreprocessorDefinitions>
       <RuntimeLibrary>MultiThreadedDLL</RuntimeLibrary>
       <PrecompiledHeader />
@@ -272,6 +380,9 @@
       <EnableCOMDATFolding>true</EnableCOMDATFolding>
       <TargetMachine>MachineX86</TargetMachine>
     </Link>
+    <PreBuildEvent>
+      <Command>pre-build.bat</Command>
+    </PreBuildEvent>
   </ItemDefinitionGroup>
   <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
   <ImportGroup Label="ExtensionTargets">

windows/ccextractor.vcxproj.filters

@@ -49,6 +49,27 @@
     <Filter Include="Source Files\ccx_common">
       <UniqueIdentifier>{2b841054-ad06-429d-9b44-fabd29fbeef0}</UniqueIdentifier>
     </Filter>
+    <Filter Include="Source Files\zvbi">
+      <UniqueIdentifier>{d86f4f94-df28-4fed-a455-c54bc521c86a}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Header Files\zvbi">
+      <UniqueIdentifier>{288f48c3-470a-45ad-a70a-8f062c51aeb1}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Header Files\lib_hash">
+      <UniqueIdentifier>{140e6ccb-2042-4ecc-9cba-42a04a5b0803}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Source Files\lib_hash">
+      <UniqueIdentifier>{dcb7e1ec-51eb-4f69-93db-b10133e98402}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Header Files\win_spec_incld">
+      <UniqueIdentifier>{de4fb954-0fd9-407a-8803-4dec4640296b}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Header Files\win_iconv">
+      <UniqueIdentifier>{1870287a-d318-4ef0-9bd1-11c965a1474c}</UniqueIdentifier>
+    </Filter>
+    <Filter Include="Source Files\win_iconv">
+      <UniqueIdentifier>{964b0a18-918d-4b08-ba92-5a84e3c48b43}</UniqueIdentifier>
+    </Filter>
   </ItemGroup>
   <ItemGroup>
     <ClInclude Include="..\src\gpacmp4\gpac\avparse.h">
@@ -177,12 +198,6 @@
     <ClInclude Include="..\src\lib_ccx\disable_warnings.h">
       <Filter>Header Files</Filter>
     </ClInclude>
-    <ClInclude Include="include\inttypes.h">
-      <Filter>Header Files</Filter>
-    </ClInclude>
-    <ClInclude Include="include\stdint.h">
-      <Filter>Header Files</Filter>
-    </ClInclude>
     <ClInclude Include="..\src\lib_ccx\ccx_common_char_encoding.h">
       <Filter>Header Files\ccx_common</Filter>
     </ClInclude>
@@ -255,6 +270,48 @@
     <ClInclude Include="..\src\lib_ccx\utility.h">
       <Filter>Header Files</Filter>
     </ClInclude>
+    <ClInclude Include="..\src\zvbi\bcd.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\bit_slicer.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\macros.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\misc.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\raw_decoder.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\sampling_par.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\sliced.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\zvbi\zvbi_decoder.h">
+      <Filter>Header Files\zvbi</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\lib_ccx\compile_info.h">
+      <Filter>Header Files</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\lib_hash\sha2.h">
+      <Filter>Header Files\lib_hash</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\win_spec_incld\stdint.h">
+      <Filter>Header Files\win_spec_incld</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\win_spec_incld\inttypes.h">
+      <Filter>Header Files\win_spec_incld</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\win_spec_incld\dirent.h">
+      <Filter>Header Files\win_spec_incld</Filter>
+    </ClInclude>
+    <ClInclude Include="..\src\win_iconv\iconv.h">
+      <Filter>Header Files\win_iconv</Filter>
+    </ClInclude>
   </ItemGroup>
   <ItemGroup>
     <ClCompile Include="..\src\ccextractor.c">
@@ -581,9 +638,6 @@
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_708_output.c">
       <Filter>Source Files\ccx_decoders</Filter>
     </ClCompile>
-    <ClCompile Include="..\src\win_iconv\win_iconv.c">
-      <Filter>Source Files</Filter>
-    </ClCompile>
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_708_encoding.c">
       <Filter>Source Files</Filter>
     </ClCompile>
@@ -617,5 +671,32 @@
     <ClCompile Include="..\src\lib_ccx\ccx_decoders_isdb.c">
       <Filter>Source Files\ccx_decoders</Filter>
     </ClCompile>
+    <ClCompile Include="..\src\zvbi\bit_slicer.c">
+      <Filter>Source Files\zvbi</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\zvbi\decoder.c">
+      <Filter>Source Files\zvbi</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\zvbi\raw_decoder.c">
+      <Filter>Source Files\zvbi</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\zvbi\sampling_par.c">
+      <Filter>Source Files\zvbi</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\lib_ccx\ccx_gxf.c">
+      <Filter>Source Files</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\lib_ccx\ccx_decoders_vbi.c">
+      <Filter>Source Files\ccx_decoders</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\lib_ccx\ccx_encoders_g608.c">
+      <Filter>Source Files\ccx_decoders</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\lib_hash\sha2.c">
+      <Filter>Source Files\lib_hash</Filter>
+    </ClCompile>
+    <ClCompile Include="..\src\win_iconv\win_iconv.c">
+      <Filter>Source Files\win_iconv</Filter>
+    </ClCompile>
   </ItemGroup>
 </Project>

windows/libs/installation.txt

@@ -0,0 +1,21 @@
+In order to compile CCExtractor with OCR support (the (Release|Debug)-OCR configurations), 
+we need the header files and .lib files for two libraries: Tesseract and Leptonica.
+
+The project expects the following directory structure:
+
+leptonica
+	-> include
+		 * library header files which leptonica depends on (gif_lib, zlib, png, ...)
+		-> leptonica
+			* leptonica headers (most importantly allheaders.h)
+	-> lib
+		 * the .lib & .dll generated by compiling Leptonica
+tesseract
+	-> include
+		-> tesseract
+			* tesseract headers (most importantly capi.h)
+	-> lib
+			* the .lib & .dll generated by compiling Tesseract
+
+
+Currently CCExtractor makes use of liblept168 (Leptonica 1.68) and libtesseract302 (Tesseract 3.02).

windows/pre-build.bat

@@ -0,0 +1,47 @@
+@echo OFF
+setlocal EnableDelayedExpansion
+echo Obtaining Git commit
+git rev-parse HEAD 2>NUL > temp.txt
+set /p commit=<temp.txt
+if not defined commit (
+	echo Git command not present, trying folder approach
+	if exist "..\.git" (
+		echo Git folder found, using HEAD file
+		type "..\.git\HEAD" 2>NUL > temp.txt
+		for /f "delims=" %%a in (temp.txt) do set head=%%a&goto next
+		:next
+		set start=%head:~0,5%
+		set rest=%head:~5%
+		if "%start%"=="ref: " (
+			echo HEAD file contains a ref, following the ref
+			echo ..\.git\%rest% > temp.txt
+			for /f "delims=" %%a in (temp.txt) do set ref_path=%%a&goto nexttt
+			:nexttt
+			set ref_path=%ref_path:/=\%
+			type "%ref_path%" 2>NUL > temp.txt
+			for /f "delims=" %%a in (temp.txt) do set commit=%%a&goto nextt
+			:nextt
+			echo Extracted commit: %commit%
+			goto final
+		) else (
+			echo HEAD contains a commit, using it (%head%)
+			set commit = %head%
+			goto final
+		)		
+	)
+)
+:final
+if not defined commit set commit=Unknown
+for /F "usebackq tokens=1,2 delims==" %%i in (`wmic os get LocalDateTime /VALUE 2^>NUL`) do if '.%%i.'=='.LocalDateTime.' set ldt=%%j
+set builddate=%ldt:~0,4%-%ldt:~4,2%-%ldt:~6,2%
+echo storing variables in file
+echo commit: %commit%
+echo date: %builddate%
+echo #ifndef CCX_CCEXTRACTOR_COMPILE_H > ../src/lib_ccx/compile_info.h
+echo #define CCX_CCEXTRACTOR_COMPILE_H >> ../src/lib_ccx/compile_info.h
+echo #define GIT_COMMIT "%commit%" >> ../src/lib_ccx/compile_info.h
+echo #define COMPILE_DATE "%builddate%" >> ../src/lib_ccx/compile_info.h
+echo #endif >> ../src/lib_ccx/compile_info.h
+echo stored all in compile_info.h
+del temp.txt
+echo done