Re: [mediaqueries4] Media queries for physical resolution

On Sat, 23 Mar 2013 18:23:29 +0100, Mark Watson <watsonm@netflix.com>  
wrote:

>> And even then, it isn't that simple, as the video element may be scaled  
>> up or down using CSS transforms, or even rotated, possibly in 3D. Or  
>> maybe the user agent has a zooming in and out mechanism either for page  
>> navigation, or for tab switching.
>
> True - those things make dynamic determination of the actual rendering  
> area device pixel count very difficult. Perhaps knowing the maximum is  
> the best we can do for now.

This assumes that the typical case of videos on the web is playing them  
fullscreen, but that's far from my experience.

>> No matter what is happening to the video rendering surface, the UA has  
>> to know about it. If we want a javascript based selecting system to be  
>> able to make an equally inteligent choice, all that information (and  
>> more: the timing of the animation if any is applied to the transform,  
>> network conditions...). Exposing all that through media queries in not  
>> possible.
>
> IIUC, it is presently possible to define quite complex sets of  
> conditions with media queries and get notifications when the match state  
> of a given query changes (through the mediaMatch function). A script  
> could use those notifications to determine whether it was or was not  
> appropriate to change resource selection. Note that you usually have  
> some media buffered, so resource selection changes don't take place  
> immediately.

That wouldn't work, because media queries can get reevaluated when the  
environment changes, but not to changes inside the webpage driven by other  
css rules. Quoting the spec:
   "To avoid circular dependencies, it is never necessary to apply the  
style sheet in order to evaluate expressions."

This imply that media queries cannot be used to inform about the size of  
an element, since that depends on applying styles, and certainly not about  
changes in the size of the element due to transitions or animations, or  
about geometric distortions due to transforms.

>> You could try crafting some javascript API independently, but I don't  
>> think this is an effective approach.
>>
>> Whether you do it in markup or via scripts, describing the media to the  
>> UA and letting it pick the ideal variant seems a lot easier, so that's  
>> the direction I would look into.
>
> So I'm confused why you advocate this approach for this particular  
> aspect (resolution) when the video element resource selection algorithm  
> and the existing system of media queries takes the opposite approach for  
> many other device properties (size, aspect ratio, orientation, color  
> depth, etc.)

I don't think that was a great design either. I would advocate declaring  
the properties of the resource, and letting the UA figure it out there too.

> I can see how a declarative approach for the HTML Media Element could  
> work, by adding attributes such as resolution and frame rate to the  
> <source> element, but for a script-based approach with Media Source we  
> would need a completely new API with which to declare the available  
> streams and new events for the UA to request selection and re-selection  
> of the streams. We run into problems where the script is _also_ trying  
> to switch streams for reasons the UA does not understand (throughput,  
> user preferences and selections). It seems a lot more work and a lot  
> less promising.

It is indeed a fair bit of work, but I am not sure anything simpler would  
actually be adequate to solve the problem you're trying to solve. If I  
understand correctly, there are two parts to what you are trying to do:
1) Using scripts to fetch the video stream
2) Using scripts to decide which stream to fetch

My impression is that the primary goal is 1 rather than 2, and I believe  
that you don't need to do 2 to be able to do 1. As 2 depends on a lot of  
hard to expose information already known by the UA, I am not convinced the  
complexity of exposing that information is worth it. If the UA currently  
does not know enough about the resource to do a good choice, I'd rather  
focus on exposing resource info to the UA, as it seems to me to be a  
simpler problem.

> By contrast, simply defining a length unit of 'device pixels', say dpx,  
> for use in media queries, seems relatively simple and addresses this  
> use-case in a way that is consistent with existing resource selection,  
> both declarative and script-based, for other properties. For example
>
> <source media='min-display-height: 1080dpx' ... />

There is no doubt that this is simple, but I do doubt that it actually  
solves the problem well. As I mentioned above, full screening everything  
is not the typical situation. It does happen, but not most of the time. In  
all other situations, it doesn't really help. And the introduction of the  
dpx unit, which is tempting due to its apparent simplicity, but the wrong  
tool in almost every situation, would do more harm than good, as has been  
discussed multiple times in this working group.

  - Florian

Received on Friday, 29 March 2013 09:44:14 UTC