Re: an interoperable object (fallback and context menus)

Boris Zbarsky 2009-03-22 20.01:
> Leif Halvard Silli wrote:
>>> I thought we were talking about <object> here.  <video> is a totally 
>>> different kettle of fish.
>>
>> Same fallback expectations.
> 
> Except the spec for <video> currently says that the fallback for <video> 
> works completely differently.
> 
> So apparently at least some people have different fallback expectations.

Hm - I was under the strong impression that <video> could contain 
nested <video> elements? But the draft says that

"Content may be provided inside the video element. User agents 
should not show this content to the user; it is intended for older 
Web browsers which do not support video,"

I realise that this means that the intent is not to contain nested 
video elements.

However, there is nothing in the draft that *forbids* a <video> 
elment nested in the outer <video> element. Also, Henri's 
validator.nu accepts it as valid. And it seems logical to nest 
<video> elements, if a UA do not support the specific video format.

>>>> What if one UA supports one format - but badly, how do authors get 
>>>> the UA to serve the fallback instead? 
>>>
>>> That's a problem in general, no?  Doesn't seem to be <object>-specific.
>>
>> This is one of the spesific difficulties when using images with <object>.
> 
> Sure.  But it's not limited to that special case.

Yes, it has wider implications - as wide as one wants.

>> E.g. both Opera and Firefox 3 know that they do not support image/tiff
> 
> They do not support it with their image library.  The can render it 
> using plug-ins, of course.  <object> does not specify a rendering 
> method; it just asks to render the content; plug-ins are explicitly 
> included in ways to render the content.  If the content cannot be 
> rendered at all, you fall back.

HTML has two attributes, @classid and @codetype, which let authors 
specify which implementation to use. HTML 4 about the latter:

     "codetype = content-type [CI]  This attribute specifies the 
content type of data expected when downloading the object 
specified by classid. This attribute is optional but recommended 
when classid is specified since it allows the user agent to avoid 
loading information for unsupported content types. When absent, it 
defaults to the value of the type attribute."

Principally codetype="image/png" seems equal to type="image/png", 
with the difference that it is linked to the classid resource. If 
that reourced do not support image/png, the browser will use the 
fallback instead (fingers crossed - needs to be documented, but I 
have seen behaviour that is similar to this, in IE. And oiutside 
Internet Explorer at least the Flip4Mac[1] QuickTime plug-in for 
Windows Media files looks at the classid - and even reports about 
unnsupported microsoft CLSID's.

What is classid support in Opera, Firefox, Safari?

>> but instead of doing fallback, they hand the file over to quicktime - 
>> preventing the author from getting the fallback displayed. This 
>> beavior is excellent for elements that do not support fallback - such 
>> as <img>. But when Firefox treats <object data="tiff"> the same way, 
>> then it defeats the idea of how <object> - or fallback - is supposed 
>> to work.
> 
> You're supposed to fall back if you can't render the image.  The image 
> is being shown.  Hence no fallback.
> 
> Now I realize your issue is that you want the UA to realize that the 
> image is an image in this case and provide the appropriate context menu, 
> etc.  So in fact, you would prefer that your fallback GIF be rendered by 
> the UA to the TIFF being rendered by a plug-in.  This is by no means the 
> most common use case with <object> however...  

The very *attemt* to use <object> for images is quite common, in 
the "community". This fails - typically due to IE-before-8's bad 
support for images in <object>. The IE7 script contains a 
workaroudn for it.

> Perhaps we need an 
> attribute (or set of attributes) on <object> that allows author control 
> over the methods the UA uses for rendering the content?  Not sure what 
> form this would take; a start here would be trying to figure out what 
> the different rendering methods are and how one differentiates them.

Yes, more documentation would be needed.

As for solution, how about just implementing classid?

Another option is something a la CSS. For example, if one had an 
attribute called features="", then to tell the UA to display 
nested content instead, if it doesn't support animation with the 
designated resource, one could do this:

<object features="animation" data="animatedPNG">
    <object data="animatedGIF">
    [Text fallback.] </object></object>

Users could eventually have a method for highten or lower the 
preferred support level, to avoid or force fallback. We allready 
have User CSS - may be it could be implemented in a similar way.

W.r.t. plug-ins, if the UA doesn't know whether the plug-in - e.g. 
QuickTime - can present't *animated* PNGs, then the UA must show 
the fallback instead.

What is needed, then, is to define some keywords - such as 
animation, transparency, multipage-images etc - and get vendors to 
check if they support what the keywords means.

>> Even if UAs should give the same user experience regardless of whether 
>> the image is inside <object> or <img> (same interface to them), they 
>> should still be treated differently when it comes to fallback. Or else 
>> <img> is a very fine element ...
> 
> They _are_ treated differently when it comes to fallback: <object> 
> allows much richer fallback than <img>.

Of course: IMG do not allow same media fallback. That's the 
difference. And that is also why it is logical that <img> loads 
the image via plug-in.

> And <img> is in fact fine element.

Sure. :-) But <object> becomes superfluos - from a graphical media 
point of view - if it just duplicates how <img> behaves with 
regard to loading plug-ins.

In that regard: it is problematic to have <img> as last fallback 
of <object data="IMAGE">. E.g imagine this:

<object data=unsuppImage><img src=JPEG alt="Short txt."></object>

Here the "fine element" <img> trumphs the advantage of the 
<object>, namley the rich fallback (unless the author adds extra 
CSS to hide any possible text that is inserted after the image). 
But even this is of course useful if <img> provides enough space 
for the fallback.

This shows that for <object> to be able to play its role as media 
element with richer fallback than <img>, it is necessary that the 
media part works. The claim that "you can simply use <object> 
instead of @longdesc" becomes as de-rooted in sound reality as 
that famous poster child, as long as <object> doesn't really work 
as it should.

>>> Well... if there were a separate MIME type for animated PNG, you 
>>> could do this with content negotiation.  This seems like a poster 
>>> child example of it, in fact.
>>
>> Cliches and deaf ears ...
> 
> I'm not sure what you're trying to say here.

Cliches doesn't make anyone listen, one only closes one's ears. It 
was a reaction to that poster child argument, once again.

My point with the animated PNG-s example was just to point out 
that an <object> with a well defined fallback model could also 
help vendors who want to support new file formats, as well as 
authors who want to use them. Even if it sounds as a nice thing, I 
do not have any special love for animated PNGs (not sure there is 
a offical format for it animated PNGs yet, either).

>> Let Opera and Firefox handle <img src="TIFF"> via QuickTime *at 
>> fallback level*. But not in the first level of <object> - when there 
>> is fallbak!
> 
> That would break existing content (due to breaking how <object> works in 
> HTML4).  As I said above I could see the need for a off-by-default flag 
> authors could toggle to get this behavior.

Right, it must be something that authors can switch on when needed.

>> I am curious to see how this will look like for <video>: when the UA 
>> doesn't support the preferred format natively - but still supports it 
>> via a plug-in, while a native format appars in a nested <video> 
>> element. What happens then?
> 
> Per current <video> spec, the outer <video> is rendered.  In fact, all 
> content inside <video> is ignored by a UA that implements <video>, per 
> current spec.
> 
> If this was a hypothetical question about some other proposal, I'd like 
> to see the proposal; it's impossible to answer otherwise.

Not sure what you meant here, but I would like to propose that 
<video> should *formally* allow nested <video> elements (see top 
of letter), and that it should display the nested <video> element 
if the format in the outer <video> element isn't supported.

And, yes, I would also suggest that such an attribute for 
specifying supportlevel/features, could be useful for <video>. 
Forexample,  a video might be available in two formats - one being 
HD quality.  Then, if the UA doesn't support HD quality (or has it 
disabled), it may not be any point in loading that version (it 
becomes a litle like <canvas> - if script is disabled, then, 
naturally, the fallback is displayed instead):

<video features="HD-support" src="VideoHD">
    <video src="SimplerFormat">
    [Text fallback.] </video></video>


>> A better fallback model could allow users to specify
> 
> Users or authors?

Not sure why I said 'users'. I primarely had authors in mind. But 
with a CSS model, it could work both ways. E.g. the !important 
keyword to override author choices etc ...

>> - whether UA should count in plug-in support when it looks at
>>   whether it supports the resource or not.
> 
> Define "plug-in"?

Most UAs have some things they support natively, and some things 
they support via plug-ins only. Plug-ins are usually added after 
the browser was installed. Or it is something that interacts 
closely with the UA, but not close enought to be part of it. 
QuickTime is a plug-in, even if it exist on 99,999% of all 
Macintosh machines.

May be you hint that it can be hard to define what it is? May be 
it is more important to focus on necessary support, rather than 
plug-in or not. If QuickTime can display a transparent PNG,  then 
fine. But it must not be theoretical support - it must work inside 
the browser.

>> - whether it is acceptable that the UA treats the resource as
>>   something else than what it is specfied to be.
> 
> "specified" where?

I had in mind that IE and Safari under certain circumstances 
presents <object> content as a frame because they think the 
content is a web page even when it is an image. If one could then 
say e.g.

<object features="graphic" data="objectImage" >

then Safari would not load that image if, due to its own 
confusion, it did not consider that resources as an image file.

>> - whether lack of support for a particular resource feature
>>   (such as transparencey) should lead to fallback.
> 
> This involves enumerating all possible features of all possible 
> resources?  Or something else?

Yes. Or a well defined set ..., for a start.

>> - whether an XML error in should lead to fallback or parsing
>>   errors being displayed or lead to fallback.
> 
> This sentence seems to have gotten mangled.  Not sure what it was trying 
> to say.

I'm not sure it is possible or even a good idea. But if the 
<object> contains a SVG image, and if the <object> also contains 
fallback, and if the <svg> fails to display due to non-conformant 
code, then the UA could display the fallback instead rather than 
displaying the broken SVG image.

  <object features="XML" data="image.svg">

This suggests a double nature of a such @features attribute: the 
UA shoud know if it supports the announced feature, and go for the 
fallback if it knows that it doesn't. But also, if the UA 
knows/understand that the resource doesn't live up to the 
announced feature(s), the UA could perform fallback. (This doesn't 
seem any more different than false URIs, which also should cause 
fallback.)

>> Better fallback requires that fallback is governed by more than MIME 
>> and URIs. Authors must also be able to specify the resource features 
>> that one are after - so the UA can go for fallback if it doesn't 
>> support those features.
> 
> This is very difficult to do across the space of all possible content...

It probably is. Like with everything else, one must begin with the 
beginning ...

[1] http://www.telestream.net/flip4mac-wmv/overview.htm
-- 
leif halvard silli

Received on Monday, 23 March 2009 01:18:46 UTC