Media perspectives in Machine Learning workshop presentations

Hello Media & Entertainment Interest Group,

W3C is organizing a virtual workshop on Web and Machine Learning. 
Workshop presentations have been published last week (a few of them are 
still pending):
https://www.w3.org/2020/06/machine-learning-workshop/presentations.html

Some of the presentations touch on machine learning applied to media 
scenarios. I thought I would mention them here (order is that of the 
presentations page). Note that the list below does not include talks 
that mention "usual" augmented reality scenarios (gesture/object 
recognition) although, by definition, they also need to process media 
streams.


Real-time ML Process of media in-browser
-----
by Bernard Aboba (Microsoft)
... still pending!


Media processing hooks for the Web
-----
https://www.w3.org/2020/06/machine-learning-workshop/talks/media_processing_hooks_for_the_web.html

This one's from me. Roughly contains the points raised by Paul and Chris 
when they presented WebCodecs to the IG back in July [1], and actually 
does not talk about machine learning per se, so nothing new for this 
group. I just found it useful to draw the encoding/decoding pipeline as 
a way to explain why more work is needed to enable efficient media 
processing on the Web.


Accelerated graphics and compute API for Machine Learning - DirectML
-----
https://www.w3.org/2020/06/machine-learning-workshop/talks/accelerated_graphics_and_compute_api_for_machine_learning_directml.html
by Chai Chaoweeraprasit (Microsoft)

Media related demos include the Auto Reframe feature in Adobe Premier 
Pro and up-sampling of textures in games.


Mobile-first web-based Machine Learning
-----
https://www.w3.org/2020/06/machine-learning-workshop/talks/mobile_first_web_based_machine_learning.html
by Josh Meyer & Lindy Rauchenstein (Artie)

Contains considerations about model sizes for audio (speech) processing 
and video processing for mobile games.


Interactive ML - Powered Music Applications on the Web
-----
https://www.w3.org/2020/06/machine-learning-workshop/talks/interactive_ml_powered_music_applications_on_the_web.html
by Tero Parviainen (Counterpoint)

Mentions the difficulty to process audio in real-time with machine 
learning algorithms for musical purpose. Also mentions performance 
measurements for video processing using the GPU, where half of the time 
gets spent actually getting the data to the ML model running on the GPU.

Thanks,
Francois.

[1] https://www.w3.org/2020/07/07-me-minutes.html#item02

Received on Monday, 24 August 2020 15:10:36 UTC