mirror of
https://github.com/CCExtractor/ccextractor.git
synced 2026-02-03 21:23:48 +00:00
Random failures with buffered input on stdin in certain files. #195
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @mackworth on GitHub (Oct 12, 2016).
The symptom is
Error: Syntax problem: Final 0xFF marker missing.Same video file, same command. Fails with the -bi; otherwise it works fine.Here's a sample file.
https://dl.dropboxusercontent.com/u/21507587/RobotChicken-RandomFailuresBI.mpg
Here's a bad pass:
RobotChickenBad2.txt
and here's a good pass:
RobotChickenGood.txt
When I run it 20 times, I get this:
When I turn on
-debug and -videsthe log looks like thisRobotChickenBad.txt
The difference is here (3912 lines in). In multiple passes, I've seen the prev/curr numbers becomes 4/15, 4/9 and 4/2 as well.
@mackworth commented on GitHub (Oct 12, 2016):
Let me add a question, now that you fixed the other issue, I could just drop the bufferedInput flag and avoid this issue. What is the reason for the flag? Is there better performance with it? (Just FYI, I'm the developer for cTiVo, which downloads shows from the TiVo to your Mac, coordinating tivodecode, ffmpeg, mencoder, handbrake, ccextractor, and comskip. So I've got a steady stream of these files coming over, and construct a pipeline for each one.)
And thank you for an excellent program!
@cfsmp3 commented on GitHub (Oct 12, 2016):
This one is going to be a tough one I think. For starters I can't reproduce:
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
OK - RobotChicken-RandomFailuresBI.mpg
About your question, you probably don't want to buffer in Mac. Best thing you can do is just run CCExtractor with and without the flag and compare execution time. But I remember that in linux the internal buffer actually made things slower while in Windows it seemed to help which is why default behavior is different in each platform.
On the other hand, even if you don't care about the buffer anymore we shouldn't have a bug running around :-) You never know when it's going to bite.
@mackworth commented on GitHub (Oct 13, 2016):
so, and I have the opposite problem! I can run it, somewhat reliably get it to fail, load it in the debugger stop at the error point, and look around. I just have no idea what it all means. Let me play with it some and see if I can identify the place where good and bad runs diverge and see what I can find. Looks to me like
TS continuity counter not incremented prev/curris the critical spot.@mackworth commented on GitHub (Oct 13, 2016):
Oh, even stranger. If I redirect stdIn to the file (
<$f), it works fine; but if I pipe it in from cat (`cat $f |'), it fails. I was unaware these could be different. My only guess so far is there's an uninitialized something somewhere, and putting it in a pipe causes it to initialized differently?@cfsmp3 commented on GitHub (Oct 13, 2016):
I ran it with valgrind and it didn't report any incorrect use of memory.
I can't reproduce no matter what I try, but I'm using linux, not Mac... so
it's not the same thing.
On Thu, Oct 13, 2016 at 9:17 AM, Hugh Mackworth notifications@github.com
wrote:
@MatejMecka commented on GitHub (Jan 8, 2017):
Invalid! Sample missing. Cannot Reproduce
@cfsmp3 commented on GitHub (Jan 8, 2017):
@mackworth can you make that file (or another one with the same problem) available for download? We want to take another look.
@mackworth commented on GitHub (Jan 8, 2017):
I have restored that file to my dropbox again.
https://dl.dropboxusercontent.com/u/21507587/RobotChicken-RandomFailuresBI.mpg
@MatejMecka commented on GitHub (Jan 8, 2017):
The Download failed multiple times for me. Can you post it on a different site.
@mackworth commented on GitHub (Jan 8, 2017):
Hmm, maybe cause it's treating it as a video file. I renamed it to .jnk; please try it again.
If not, I'm not currently subscribed to any filesharing site that shares by URL, just by email-based distribution, can you suggest one?
@mackworth commented on GitHub (Jan 8, 2017):
so revised URL is https://dl.dropboxusercontent.com/u/21507587/RobotChicken-RandomFailuresBI.jnk
@mackworth commented on GitHub (Jan 9, 2017):
Did that work?
@cfsmp3 commented on GitHub (Jan 9, 2017):
@mackworth it did for me - don't know about @MatejMecka who is going to look into it. But anyway it's not a dropbox problem for sure.
@MatejMecka commented on GitHub (Jan 9, 2017):
@mackworth it did this time!
@MatejMecka commented on GitHub (Jan 9, 2017):
I'm not seeing any problems at all:
Tested on Mac Os X El Capitan on a Macbook Air 2011(11-inch, Mid 2011).
Also the subtitles look ok. Used the second URL you gave me with the .jnk
@MatejMecka commented on GitHub (Jan 9, 2017):
Which Version are you using? I used the latest one from GitHub
@mackworth commented on GitHub (Jan 9, 2017):
I just ran it again, and got almost identical results to before: it worked the first time, failed the second. Overall worked 5 times out of 20. I've included a bad run's logs as well as a good one.
The version I'm using is compiled a couple days ago from the github release, using the build.command. I've put my binary copy at https://dl.dropboxusercontent.com/u/21507587/ccextractor
I note your log is identical to mine except yours says "100% | 15:02" versus my "Streaming | 15:00". Are you piping
cat $filenameinto ccextractor? If not, as I discussed above, if you're even do a stdin redirect, it doesn't trigger this problem.Here's my output:
Here's a successful run (almost identical to yours, except as noted above)...
@mackworth commented on GitHub (Jan 13, 2017):
Were you able to recreate?
@cfsmp3 commented on GitHub (Jan 20, 2017):
GSoC qualification: This issues gives 2 points.
@cfsmp3 commented on GitHub (Nov 20, 2017):
Sample link dead. Closing - reopen if the issue is still present (but with a permanent link).
@mackworth commented on GitHub (Nov 20, 2017):
Sorry, hadn't heard anything in so long, I cleaned up my dropbox directory to save space. Does @MatejMecka still have a copy?
@mackworth commented on GitHub (Jan 24, 2018):
aha! Found it on a backup:
https://www.dropbox.com/s/zo6ta9d8ppl093p/RobotChicken-RandomFailuresBI.mpg?dl=0
@cfsmp3: It appears I don't have the ability to reopen a closed issue.
@cfsmp3 commented on GitHub (Oct 15, 2018):
Closing this - not sure if it's active or if the original report is still accurate (maybe something was fixed).
@mackworth if this is still causing you problems could you open a new ticket with current details? We'll add it a a GCI task.
@mackworth commented on GitHub (Oct 15, 2018):
Yes, I can confirm it is still a problem.
Just updated my copy to the latest master from github, recompiled on my Mac and have the exact same symptoms.
To be specific, when run through stdin pipe, the sample file available here, has a high probability of failing (averaging 7 successes out of 20) with
Error: Syntax problem: Final 0xFF marker missing. The log file saysTS continuity counter not incremented prev/curr 4/15. But depending on the run, I've seen 2, 9, or 15 for the "curr" number.To recreate in Terminal, set
fto test file andccextractorto compiled exe, then runfor i in $(seq 1 1 20); do cat "$f" | $ccextractor -bi - -o junk3.srt > "junkNew$i.log"; if [ $? -eq 0 ]; then echo "OK "; else echo "Fail"; fi; done