[PATCH] Add optional "&exact=1" CGI arg to show video between keyframes

[Tracey Jaquith]

Greetings and hello Maxim,

Thanks for the reply and excellent question and points.

So.. yes, certainly enough information (worst case scenario, the entire moov atom) could be sent down to the client to figure out what best keyframes to jump to (and then range request it, etc.)

However:
That’s potentially a lot of data in the full moov atom to go down the wire, just to know where to start playing from, for a single .mp4 file.  For example, for a 2-3 hour HD TV show, that can easily exceed 10MB of moov atom that needs to head to the client before it can even start playing a single first video frame.  Compared to, say, HLS/DASH where playback can begin very quickly.
We’re relying on the server side “clipping” for our TV News Archive ( https://tv.archive.org <https://tv.archive.org/> ) to restrict the portion of the videos we permit our users to view.  So we’re always only permitting streaming of relatively short clips (~15 to 120s) since we consider this as viewing samples from a larger TV news show (ie: we aren’t a VOD type Archive — we’re serving search results and trying to help researchers, etc.)

It seems to me that either way, the value in the mp4 module is primarily in that server-side “magic” of opening and rewriting the moov atom, and then losslessly sending the A/V packets down to the client.  It saves a huge amount of bandwidth for smaller clips, seeking, and even jumping mid-way into a film or TV show — for archives and playback services that have more constraints on internal storage.  The Internet Archive, in general, only makes a single .mp4 web-friendly video file from uploaded materials.  We don’t have the storage to make 2-5 streaming qualities for all our videos, and chunk them into small HLS/DASH pieces with an ideal adaptive/feedback constantly changing quality system (like bigger players like Netflix, YouTube, etc.)

I figure the “begin video immediately” patch (this patch) could be useful to others, writ-large.  It certainly has been of keen interest to our TV viewers and researchers and data visualization folks.
I suspect that most of the users of this mp4 module might find the “being video immediately” (perceptively) a nice optional improvement.

I suppose a middle ground (but would also be a change/patch) could be to always start at the keyframe before the wanted start-point — and have the client, with JS, always seek to the desired start point.  However, it’s definitely nice magic to request a specific start/end point, and get exactly that, down to 1/10th of a second, with no need for client JS to seek.  They can play directly in all browsers and mobile as a legit .mp4 file that’s been efficiently served up just for you, on demand, losslessly.  Magic!  :) 


And as a sidenote, I must confess, that the 3500 lines of C code (in a single file!), would normally be a run-away-screaming situation for me.  However,  never in my life (and early start with C) have I seen such well-structured and clever C code like that.  I was pretty speechless exploring it!

Thanks for the consideration,
-Tracey


> On Jun 4, 2021, at 10:38 AM, Maxim Dounin <mdounin at mdounin.ru> wrote:
> 
> Hello!
> 
> On Thu, Jun 03, 2021 at 07:54:49PM +0000, Tracey Jaquith wrote:
> 
>> # HG changeset patch
>> # User Tracey Jaquith <tracey at archive.org>
>> # Date 1622678642 0
>> #      Thu Jun 03 00:04:02 2021 +0000
>> # Node ID 5da9c62fa61016600f2c59982ae184e2811be427
>> # Parent  5f765427c17ac8cf753967387562201cf4f78dc4
>> Add optional "&exact=1" CGI arg to show video between keyframes.
>> 
>> archive.org has been using mod_h264_streaming with an "&exact=1" patch from me since 2013.
>> We just moved to nginx mp4 module and are using this patch.
>> The technique is to find the keyframe just before the desired "start" time, and send
>> that down the wire so video playback can start immediately.
>> Next calculate how many video samples are between the keyframe and desired "start" time
>> and update the STTS atom where those samples move the duration from (typically) 1001 to 1.
>> This way, initial unwanted video frames play at ~1/30,000s -- so visually the
>> video & audio start playing immediately.
> 
> So, effectively, this makes invalid streaming requests (not 
> starting at a keyframe) to return the same data as valid ones 
> (starting at a keyframe).  A properly written player should be 
> able to achieve the same result by making correct streaming 
> requests.
> 
> Any specific reasons to do this within nginx instead of fixing the 
> player?  Or, given the fact that Flash is effectively dead, by 
> simply using HTML5 video with appropriate seeking?
> 
> -- 
> Maxim Dounin
> http://mdounin.ru/
> _______________________________________________
> nginx-devel mailing list
> nginx-devel at nginx.org
> http://mailman.nginx.org/mailman/listinfo/nginx-devel


@tracey_pooh
TV Architect  https://archive.org/tv <https://archive.org/tv>





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx-devel/attachments/20210604/22da9459/attachment.htm>