Request: Add support to extract telemetry data (stored as subtitles) from recordings #408

Closed
opened 2026-01-29 16:43:15 +00:00 by claunia · 15 comments
Owner

Originally created by @cfsmp3 on GitHub (Mar 9, 2018).

This file is a MP4 recorded from a drone (specifically: "Zenmuse camera on a DJI Inspire 2 Drone"). The telemetry data is saved as subtitles and in fact VLC can display it.

CCExtractor doesn't find it - possible it's stored in a MP4 box we still don't handle, or something like that. We do show a number of "unknown box type", even though we're using a fairly new libGPAC.

GSoC qualification: 6 points

Originally created by @cfsmp3 on GitHub (Mar 9, 2018). [This file](https://drive.google.com/file/d/1MyABc8cByEWcW9wQkl_Y_JIUYGC-Kd_5/view?usp=sharing) is a MP4 recorded from a drone (specifically: "Zenmuse camera on a DJI Inspire 2 Drone"). The telemetry data is saved as subtitles and in fact VLC can display it. CCExtractor doesn't find it - possible it's stored in a MP4 box we still don't handle, or something like that. We do show a number of "unknown box type", even though we're using a fairly new libGPAC. **GSoC qualification: 6 points**
claunia added the GSoC-related label 2026-01-29 16:43:15 +00:00
Author
Owner

@cfsmp3 commented on GitHub (Mar 12, 2018):

After some digging - this is just EXIF data. Not sure CCExtractor is the right tool to actually extract it (since it has nothing to do with subtitles after all), since exiftool does it perfectly already. We could however detect its presence and display some notification.

Anyway this tool processes the sample file just fine and reports everything:
https://www.sno.phy.queensu.ca/~phil/exiftool/

@cfsmp3 commented on GitHub (Mar 12, 2018): After some digging - this is just EXIF data. Not sure CCExtractor is the right tool to actually extract it (since it has nothing to do with subtitles after all), since exiftool does it perfectly already. We could however detect its presence and display some notification. Anyway this tool processes the sample file just fine and reports everything: https://www.sno.phy.queensu.ca/~phil/exiftool/
Author
Owner

@thetransformerr commented on GitHub (Mar 13, 2018):

This was the dump I got from atom "moov/udta" present in above said file where the data was stored,

udta(?xyz?+26.483061-80.124142+38.000?xsp?-0.40?ysp?+0.10?zsp?+0.00?fpt?+2.80?fyw?+82.90?frl?-9.50?gpt?-27.70?gyw?+83.50?grl?+0.00?rec ?repPmeta hdlrmdir?ilst̩cmt?dataDE=TrueColor, Mode=P, DSW=0001Xtra?
           WM/Categorv01.07.2091.1.v1.0.0?uid??QPd ?mdlFC65100?csndb5c42c8e7d3048193df558fbdc77dfb(?aud??mux????free 
@thetransformerr commented on GitHub (Mar 13, 2018): This was the dump I got from atom "moov/udta" present in above said file where the data was stored, ``` udta(?xyz?+26.483061-80.124142+38.000?xsp?-0.40?ysp?+0.10?zsp?+0.00?fpt?+2.80?fyw?+82.90?frl?-9.50?gpt?-27.70?gyw?+83.50?grl?+0.00?rec ?repPmeta hdlrmdir?ilst̩cmt?dataDE=TrueColor, Mode=P, DSW=0001Xtra? WM/Categorv01.07.2091.1.v1.0.0?uid??QPd ?mdlFC65100?csndb5c42c8e7d3048193df558fbdc77dfb(?aud??mux????free ```
Author
Owner

@thetransformerr commented on GitHub (Mar 13, 2018):

They might be the location from where we can make some changes:

39b96cc544/src/gpacmp4/mp4.c (L631)

or

39b96cc544/src/gpacmp4/mp4.c (L420)

@thetransformerr commented on GitHub (Mar 13, 2018): They might be the location from where we can make some changes: https://github.com/CCExtractor/ccextractor/blob/39b96cc54493d7462702b31195be84c1936551f2/src/gpacmp4/mp4.c#L631 or https://github.com/CCExtractor/ccextractor/blob/39b96cc54493d7462702b31195be84c1936551f2/src/gpacmp4/mp4.c#L420
Author
Owner

@thealphadollar commented on GitHub (Mar 19, 2018):

@cfsmp3 What changes are we looking forward to? I might be able to finish this with some lead and more information.

@thealphadollar commented on GitHub (Mar 19, 2018): @cfsmp3 What changes are we looking forward to? I might be able to finish this with some lead and more information.
Author
Owner

@cfsmp3 commented on GitHub (Mar 19, 2018):

I'd start with detecting the presence of EXIF data, which shouldn't be too
hard.

On Sun, Mar 18, 2018 at 9:36 PM, Shivam Kumar Jha notifications@github.com
wrote:

@cfsmp3 https://github.com/cfsmp3 What changes are we looking forward
to? I might be able to finish this with some lead and more information.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/CCExtractor/ccextractor/issues/961#issuecomment-374099246,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFrJ2YIjW9yNBbGoRg7paZ-LWjr9azvDks5tfzXKgaJpZM4Sjm9C
.

@cfsmp3 commented on GitHub (Mar 19, 2018): I'd start with detecting the presence of EXIF data, which shouldn't be too hard. On Sun, Mar 18, 2018 at 9:36 PM, Shivam Kumar Jha <notifications@github.com> wrote: > @cfsmp3 <https://github.com/cfsmp3> What changes are we looking forward > to? I might be able to finish this with some lead and more information. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/CCExtractor/ccextractor/issues/961#issuecomment-374099246>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AFrJ2YIjW9yNBbGoRg7paZ-LWjr9azvDks5tfzXKgaJpZM4Sjm9C> > . >
Author
Owner

@thealphadollar commented on GitHub (Mar 20, 2018):

@cfsmp3 When ccextractor is installed with -FFMPEG, MP4 files are also processed in general mode. Is that desirable? Or it should go with GPAC only?

This
d23cb8571d/src/ccextractor.c (L202)
comes before
d23cb8571d/src/ccextractor.c (L228)
which leads to the problem I mentioned.

The sample provided in this discussion gives no output nonetheless but the box information is also not provided (which is very obvious).

@thealphadollar commented on GitHub (Mar 20, 2018): @cfsmp3 When ccextractor is installed with -FFMPEG, MP4 files are also processed in general mode. Is that desirable? Or it should go with GPAC only? This https://github.com/CCExtractor/ccextractor/blob/d23cb8571d87f912393de8b2f8ff4a3f04404233/src/ccextractor.c#L202 comes before https://github.com/CCExtractor/ccextractor/blob/d23cb8571d87f912393de8b2f8ff4a3f04404233/src/ccextractor.c#L228 which leads to the problem I mentioned. The sample provided in this discussion gives no output nonetheless but the box information is also not provided (which is very obvious).
Author
Owner

@thealphadollar commented on GitHub (Mar 20, 2018):

@cfsmp3 What I think we can do is when we detect the first error, we stop the loop and tell the user that the file doesn't contain subtitles but EXIF data and point it to some tool that can extract it.

What do you say?

@thealphadollar commented on GitHub (Mar 20, 2018): @cfsmp3 What I think we can do is when we detect the first error, we stop the loop and tell the user that the file doesn't contain subtitles but EXIF data and point it to some tool that can extract it. What do you say?
Author
Owner

@cfsmp3 commented on GitHub (Mar 20, 2018):

Well, EXIF data being present is not an error :-)
What if the file contains both EXIF and subtitles? Why would you stop?

On Tue, Mar 20, 2018 at 12:43 AM, Shivam Kumar Jha <notifications@github.com

wrote:

@cfsmp3 https://github.com/cfsmp3 What I think we can do is when we
detect the first error, we stop the loop and tell the user that the file
doesn't contain subtitles but EXIF data and point it to some tool that can
extract it.

What do you say?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/CCExtractor/ccextractor/issues/961#issuecomment-374503804,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFrJ2ehjBSAOEBK204gHU_kp-lWi-f3Uks5tgLMjgaJpZM4Sjm9C
.

@cfsmp3 commented on GitHub (Mar 20, 2018): Well, EXIF data being present is not an error :-) What if the file contains both EXIF and subtitles? Why would you stop? On Tue, Mar 20, 2018 at 12:43 AM, Shivam Kumar Jha <notifications@github.com > wrote: > @cfsmp3 <https://github.com/cfsmp3> What I think we can do is when we > detect the first error, we stop the loop and tell the user that the file > doesn't contain subtitles but EXIF data and point it to some tool that can > extract it. > > What do you say? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/CCExtractor/ccextractor/issues/961#issuecomment-374503804>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AFrJ2ehjBSAOEBK204gHU_kp-lWi-f3Uks5tgLMjgaJpZM4Sjm9C> > . >
Author
Owner

@thealphadollar commented on GitHub (Apr 8, 2018):

Digging a little deeper into the problem, I found that using the latest gpac's MP4Box, I'm unable to extract the information from the mp4 file and it is throwing the same errors as CCExtractor.

screenshot from 2018-04-09 01-19-34

This could mean that the issue lies with GPAC and not our program. I'm thinking about raising this issue in the gpac repository, what do you say @cfsmp3 ?

@thealphadollar commented on GitHub (Apr 8, 2018): Digging a little deeper into the problem, I found that using the latest gpac's MP4Box, I'm unable to extract the information from the mp4 file and it is throwing the same errors as CCExtractor. ![screenshot from 2018-04-09 01-19-34](https://user-images.githubusercontent.com/32812320/38471837-84d7bcc8-3b94-11e8-9cd9-b2b04b22ffac.png) This could mean that the issue lies with GPAC and not our program. I'm thinking about raising this issue in the gpac repository, what do you say @cfsmp3 ?
Author
Owner

@cfsmp3 commented on GitHub (Apr 9, 2018):

Feel free to get in touch with them...

On Sun, Apr 8, 2018 at 12:55 PM, Shivam Kumar Jha notifications@github.com
wrote:

Digging a little deeper into the problem, I found that using the latest
gpac's MP4Box, I'm unable to extract the information from the mp4 file and
it is throwing the same errors as CCExtractor.

[image: screenshot from 2018-04-09 01-19-34]
https://user-images.githubusercontent.com/32812320/38471837-84d7bcc8-3b94-11e8-9cd9-b2b04b22ffac.png

This could mean than the issue lies with GPAC and not our program. I'm
thinking about raising this issue in the gpac repository, what do you say
@cfsmp3 https://github.com/cfsmp3 ?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/CCExtractor/ccextractor/issues/961#issuecomment-379577693,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFrJ2XJnVppvvG41prbTKc1InWnuKhypks5tmms3gaJpZM4Sjm9C
.

@cfsmp3 commented on GitHub (Apr 9, 2018): Feel free to get in touch with them... On Sun, Apr 8, 2018 at 12:55 PM, Shivam Kumar Jha <notifications@github.com> wrote: > Digging a little deeper into the problem, I found that using the latest > gpac's MP4Box, I'm unable to extract the information from the mp4 file and > it is throwing the same errors as CCExtractor. > > [image: screenshot from 2018-04-09 01-19-34] > <https://user-images.githubusercontent.com/32812320/38471837-84d7bcc8-3b94-11e8-9cd9-b2b04b22ffac.png> > > This could mean than the issue lies with GPAC and not our program. I'm > thinking about raising this issue in the gpac repository, what do you say > @cfsmp3 <https://github.com/cfsmp3> ? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/CCExtractor/ccextractor/issues/961#issuecomment-379577693>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AFrJ2XJnVppvvG41prbTKc1InWnuKhypks5tmms3gaJpZM4Sjm9C> > . >
Author
Owner

@thealphadollar commented on GitHub (Apr 16, 2018):

@cfsmp3 libGPAC has been updated to handle the above file (thanks to @jeanlf), and I think we should update our version also in order to correct the issue with this sample. Can you please guide me how we update an entire library in our source? I can change the files manually but if there's a better way I would like to learn :)

Below is the updated MP4Box output for the video.

screenshot from 2018-04-16 23-41-49

@thealphadollar commented on GitHub (Apr 16, 2018): @cfsmp3 libGPAC has been updated to handle the above file (thanks to @jeanlf), and I think we should update our version also in order to correct the issue with this sample. Can you please guide me how we update an entire library in our source? I can change the files manually but if there's a better way I would like to learn :) Below is the updated MP4Box output for the video. ![screenshot from 2018-04-16 23-41-49](https://user-images.githubusercontent.com/32812320/38827228-e133721e-41cf-11e8-915e-9e47d2563261.png)
Author
Owner

@cfsmp3 commented on GitHub (Apr 16, 2018):

I didn't do the migration last time, but basically you have two options -
compare the changes they made in libGPAC and apply then to our code base,
or just copy the changed files. I mean, assuming you're OK with a "blind"
migration. You could also dig into their code but it's probably unneeded
here.

If overwriting files make sure however we didn't make any change though.I
believe few made some really minor changes (but changes) in a few of them.

Thanks @jeanlf, always willing to assist however approaches him :-)

On Mon, Apr 16, 2018 at 11:10 AM, Shivam Kumar Jha <notifications@github.com

wrote:

@cfsmp3 https://github.com/cfsmp3 libGPAC has been updated to handle
the above file (thanks to @jeanlf https://github.com/jeanlf), and I
think we should update our version also in order to correct the issue with
this sample. Can you please guide me how we update the entire library? I
can change the files manually but if there's a better way I would like to
learn :)


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/CCExtractor/ccextractor/issues/961#issuecomment-381698083,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFrJ2cL6OB9F4ze__lgb5j7Z-HwVnj5Wks5tpN6wgaJpZM4Sjm9C
.

@cfsmp3 commented on GitHub (Apr 16, 2018): I didn't do the migration last time, but basically you have two options - compare the changes they made in libGPAC and apply then to our code base, or just copy the changed files. I mean, assuming you're OK with a "blind" migration. You could also dig into their code but it's probably unneeded here. If overwriting files make sure however we didn't make any change though.I believe few made some really minor changes (but changes) in a few of them. Thanks @jeanlf, always willing to assist however approaches him :-) On Mon, Apr 16, 2018 at 11:10 AM, Shivam Kumar Jha <notifications@github.com > wrote: > @cfsmp3 <https://github.com/cfsmp3> libGPAC has been updated to handle > the above file (thanks to @jeanlf <https://github.com/jeanlf>), and I > think we should update our version also in order to correct the issue with > this sample. Can you please guide me how we update the entire library? I > can change the files manually but if there's a better way I would like to > learn :) > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/CCExtractor/ccextractor/issues/961#issuecomment-381698083>, > or mute the thread > <https://github.com/notifications/unsubscribe-auth/AFrJ2cL6OB9F4ze__lgb5j7Z-HwVnj5Wks5tpN6wgaJpZM4Sjm9C> > . >
Author
Owner

@animesh-chouhan commented on GitHub (Jan 21, 2019):

I want to work on this issue,
Can you brief me about the current state of this issue
as I can see there have been few merges in past

@animesh-chouhan commented on GitHub (Jan 21, 2019): I want to work on this issue, Can you brief me about the current state of this issue as I can see there have been few merges in past
Author
Owner

@thealphadollar commented on GitHub (Jan 23, 2019):

@cfsmp3 I think we should close this as we had concluded these are not subtitles and information is provided by GPAC.

@thealphadollar commented on GitHub (Jan 23, 2019): @cfsmp3 I think we should close this as we had concluded these are not subtitles and information is provided by GPAC.
Author
Owner

@cfsmp3 commented on GitHub (Jan 25, 2020):

Closing - not our business.

@cfsmp3 commented on GitHub (Jan 25, 2020): Closing - not our business.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/ccextractor#408