Re: Media & Entertainment IG conference call on 360-degree video

Thanks for sharing this Chris! I'm the lead web engineer at WITHIN where we
have built our own 360 player. e.g:
https://with.in/watch/evolution-of-verse/ ( I've personally been
maintaining this player for 2.5 years )

One thing I noticed from developing our own player and seeing others
implement 360 video players is that the interaction paradigms tend to vary.
Is there inertial drift when dragging / viewing with a mouse? Are there key
events bound to control the orientation? What's the optimal field of view
for the camera's perspective? Do you use an absolute mapping or relative
mapping when invoking the device orientation API for magic window
experiences?

Some of these would be great properties on the video element for a
developer to decide, but others aren't so obvious.

In addition to this, the main feedback we get from our QA team when
comparing WebVR and native builds is that the texture uploading from the
video element to WebGL can impact frame rate. We purposefully remove 4K
quality videos on mobile VR for this reason. Though our GearVR and Daydream
apps happily handle 4K videos.

I don't posit a bias on which way things should or shouldn't operate. But,
your post inspired me to share some insights we've gained over the course
of our development.

-Jono

On Tue, Jan 9, 2018, 4:23 AM Chris Needham <chris.needham@bbc.co.uk> wrote:

> Hi all,
>
> The Media & Entertainment Interest Group had a conference call last
> Thursday discussing 360-degree video support on the web. The goal was to
> start a discussion to identify any gaps in the web platform APIs needed to
> better support 360-degree video.
>
> Stefan Pham from Fraunhofer Fokus presented their 360-degree video playout
> system [1], which has identified some issues with Media Source Extensions:
>
> - The need for a low latency buffering capability
> - Reliability of replacing segments using the 'segment' AppendMode in a
> single SourceBuffer
> - Switching between multiple SourceBuffers attached to a single MediaSource
>
> He also mentioned that using WebGL and Canvas for video transformations is
> not possible with protected content, and so a secure media path is needed.
>
> Nell Waliczek from Microsoft presented the current state of work in the
> WebVR CG, which has been renamed to the Immersive Web CG. This is the place
> to continue discussion of 360-degree video support. Nell will send
> information to the IG on how to contribute to the Immersive Web CG. Topics
> that are specific to WebXR should be followed up there, e.g., support for
> protected content.
>
> We briefly discussed metadata, in particular projection information, and
> David Dorwin from Google pointed out their existing work [2]. See related
> work on OMAF at MPEG [3]. There seemed to be consensus to favour a metadata
> based solution. David also mentioned the need for extensibility to support
> different projections.
>
> More general topics, not specific to WebXR, e.g., the MSE related issues
> mentioned should be handled separately. David Dorwin will report back on
> the current status of discussion on updating Media Source Extensions at
> WICG, and I recommend that Stephan follows up with the MSE related issues
> they have found.
>
> I'd also like to take this opportunity to point out some relevant work
> that BBC R&D had done regarding the challenges of designing subtitle
> behaviours for 360-degree video content [4, 5].
>
> Minutes from the meeting are available at [6].
>
> I look forward to continuing to collaborate on this topic.
>
> Kind regards,
>
> Chris (Co-chair, W3C Media & Entertainment Interest Group)
>
> [1]
> https://www.w3.org/2011/webtv/wiki/images/f/fb/2018-01-04-Fraunhofer-FOKUS-360-for-W3C-M%26E.pdf
> [2]
> https://github.com/google/spatial-media/blob/master/docs/spherical-video-v2-rfc.md
> [3]
> https://mpeg.chiariglione.org/standards/mpeg-a/omnidirectional-media-application-format
> [4]
> http://www.bbc.co.uk/rd/blog/2017-03-subtitles-360-video-virtual-reality
> [5]
> http://www.bbc.co.uk/rd/blog/2017-10-subtitles-360-video-virtual-reality-vr
> [6] https://www.w3.org/2018/01/04-me-minutes.html
>
>
> --
http://jonobr1.com/