Re: an interoperable object (fallback and context menus)

Boris Zbarsky 2009-03-23 03.00:
> Leif Halvard Silli wrote:

> The same draft says that a single <video> element can specify multiple 
> versions of the data in different formats, with fallback happening along 
> that list.

Blush - forgot about <source>.

    [ ... @classid ... ]

> In practice, this mechanism is not suitable for cross-browser authoring, [...]

> Of course we could redefine what @classid means to mean something much 
> more general than what it means now, but there might be compat issues 
> with existing content if we do that.

Right.

>>> Now I realize your issue is that you want the UA to realize that the 
>>> image is an image in this case and provide the appropriate context 
>>> menu, etc.  So in fact, you would prefer that your fallback GIF be 
>>> rendered by the UA to the TIFF being rendered by a plug-in.  This is 
>>> by no means the most common use case with <object> however...  
>>
>> The very *attemt* to use <object> for images is quite common, in the 
>> "community". This fails - typically due to IE-before-8's bad support 
>> for images in <object>. The IE7 script contains a workaroudn for it.
> 
> Yes.  But the desire to fall back to a different type of content 
> altogether for an <object>, in preference to handling the type of 
> content the author prefers but with a plug-in, doesn't seem common.  For 
> the most part, those who are in that position just use the more broadly 
> implemented content format to start with.

Hm - well, I would not say that: When serving "moving images" via 
<object>, the trouble is that some UAs needs <embed> instead. And 
so an <embed> is nested inside <object>. Wheras for <object>, the 
trouble is that some needs <img> instead, which could be inserted 
as fallback. Quite similar. It is just that much more energy has 
been spent on getting "moving images" to function cross browser 
via <objeect> than on static images via <object>.

The rather new trend to use a transparent 8bit PNG format instead 
of applying filters to 32bit transparent PNG could also have been 
solved through fallback.

>>> Perhaps we need an attribute (or set of attributes) on <object> that 
>>> allows author control over the methods the UA uses for rendering the 
>>> content?  Not sure what form this would take; a start here would be 
>>> trying to figure out what the different rendering methods are and how 
>>> one differentiates them.

>> Another option is something a la CSS. For example, if one had an 
>> attribute called features=""
> 
> Right; the problem then becomes defining the set of all possible 
> features for all possible content one might serve, no?  Also not really 
> tenable.

There is some carpet to roll out, yes.

>> What is needed, then, is to define some keywords - such as animation, 
>> transparency, multipage-images etc - and get vendors to check if they 
>> support what the keywords means.
> 
> Why are these keywords more important than, say "ES4" (for flash-like 
> things), "3d" (for 3d acceleration of rendering for things like vrml), 
> or any of a variety of other keywords people might care about for 
> non-image content?
 >
> That is, why exactly are we trying to customize <object> to work well 
> for images in particular?

Assuming that this is the clue:

>  Is the issue that we want the rich fallback 
> capabilities that <img> doesn't provide but at the same time don't want 
> the rich rendering capabilities (past using the built-in image renderer) 
> that <object> provides?

Yes, to the first part of the question. (Too me, the rich 
rendering capabilities are hampered by the, in practise, not so 
rich fallback capabilities.)

Anyway, a designated image element with rich fallback would of 
course have been fine.

To me, the two isssues are connected: The most basic thing is that 
it is simple to use <object>. With IE8 it is actually starting to 
become simple. But it should also have been simple to take 
advantage of the fallback - without it, it becomes difficult to 
take advantage of the simplicity that IE8 brings.

>  If we really need such a "just make it act like 
> <img> but with richer fallback" mode for <object>, maybe we should just 
> be introducing a single attribute that means that?

These thoughts are motivated by the wish to serve images with 
<object>, but it could of course help other formats as well.

I would be positive towards such an @thisObjectIsAnImage attribute 
that you suggest - it would be second best to a designated, new 
image element. If that could help better support, then of course 
fallback would be less needed. And it would also be simpler to 
eventually add fallback, one should think. And to provide advice 
on fallback. And to get good support in screen readerse etc. A 
good proposal, I think.

>>> They _are_ treated differently when it comes to fallback: <object> 
>>> allows much richer fallback than <img>.
>>
>> Of course: IMG do not allow same media fallback. That's the 
>> difference. And that is also why it is logical that <img> loads the 
>> image via plug-in.
> 
> I'm not sure what that last sentence is saying.

For <img src="TIFF" alt="Txt."> it is likely better to try to 
render the image, even in some mediocre quality, than it would do 
the same for this OBJECT to use the same mediocre plug-in:

<object data="TIFF">
	<object data="JPG">[Txt fallback]</object></object>

>>> And <img> is in fact fine element.
>>
>> Sure. :-) But <object> becomes superfluos - from a graphical media 
>> point of view - if it just duplicates how <img> behaves with regard to 
>> loading plug-ins.
> 
> Um.  Except of course that it precisely does NOT do that right now. Your 
> whole problem is that it doesn't, as far as I can tell.

One one side: In Internet Explorer (until version 8) and (to a 
much lesser degree) Safari, the problem is that images served with 
<object> are not giving  users the same experience as images 
served with <img>.

On the other side: W.r.t. fallback, the problem is that both 
<object data="TIFF"> and <img src="TIFF"> are loaded via plug-ins 
in Firefox and Opera. Thus neither author or user get to 
experience the fallback advantage that OBJECT is supposed to have.

>> In that regard: it is problematic to have <img> as last fallback of 
>> <object data="IMAGE">. E.g imagine this:
>>
>> <object data=unsuppImage><img src=JPEG alt="Short txt."></object>
>>
>> Here the "fine element" <img> trumphs the advantage of the <object>, 
>> namley the rich fallback (unless the author adds extra CSS to hide any 
>> possible text that is inserted after the image). But even this is of 
>> course useful if <img> provides enough space for the fallback.
> 
> I have no idea what those three paragraphs are trying to say.

There were an error in what I said above. I believed that if you 
have this:

     <object data="uri" ...> Text fallback 1.
           <object data="uri"> Text fallback 2.
     </object></object>

Then only "Text fallback 2", as the innermost object, would 
constitute the textual fallback. But I now see that when looking 
at it from the textual fallback perspective, then the second 
object is more like a <source> element. (The only purpose of 
keeping the fallback in the innermost OBJECT is to hide it better 
from graphical UAs.)

	The above is something the draft should make clearer.

Thus, having said that, you should understand the paragraphs you 
asked about above better: Uf the fallback of the <object> is an 
<img>, then the <img> a) requires an alt="", b) the alt has very 
little room for fallback, c) if you place text outside the <img>, 
as a child of the <object>, then even GUI browsers will display 
that content (unless - of course, you apply some CSS). That was 
the entire point. Letting <object> fall back to an <img> is not 
optimal use of <object>.

There is the same problem with letting <object> fall back to 
<embed>: You cannot add fallback text, or else the fallback text 
will "spill over" - at least in Firefox and Opera (which follows 
the spec (HTML 4) and the draft (HTML 5).

Safari has taken a differnt approach - it seems - where it uses 
the <embed> element more as if it was a <source> element. Thus it 
hides the content of <object>. The Safari behaviour makes some 
sense - it is easier to author with. But of course not how 
<object> was planned.

>> This shows that for <object> to be able to play its role as media 
>> element with richer fallback than <img>, it is necessary that the 
>> media part works.
> 
> What about it doesn't work exactly?  You serve media of a particular 
> type; the browser tries its best to render media of that type.  If it 
> can't, it falls back.  You want it to try less hard, as far as I can tell.

Yes. I would like the UA to try less hard when the <object> 
contains another <object> which also contains an image.

>> The claim that "you can simply use <object> instead of @longdesc" 
>> becomes as de-rooted in sound reality as that famous poster child, as 
>> long as <object> doesn't really work as it should.
> 
> So is your issue really with the way <object> is defined to work, or 
> with particular browser bugs?  If the latter, what evidence is there 
> that changing the spec in any way will help?

Issues with object that defeats it fallback capabilities:

1. Embed being used as "the fallback" = hinders that textual 
fallback can be added withotu disturing sighted users (unless one 
fiddles with CSS).

2. Same issue if <img> is used as fallback.

Why *are* <embed> and <img> used as fallback - instead of adding a 
  second <object> as fallback? The answer is simple - at least 
when it comes to images: Because fallback to a second <object> is 
difficult to get to work. (Or is is it simply habit? I am not 
quite certain.)

So, with the choice you gives me, the problem is browser bugs. 
(Though one could also dicuss treating <embed> effectively a 
<source> element, like Safari does.) Together with author 
practises. However "browser bugs" is a broad term. Those who are 
pusing <embed> as part of <object> are working against the 
fallback features of <object>.

The second part - the evidence - requires more thought.

>>>>> Well... if there were a separate MIME type for animated PNG, you 
>>>>> could do this with content negotiation.  This seems like a poster 
>>>>> child example of it, in fact.
>>>>
>>>> Cliches and deaf ears ...
>>>
>>> I'm not sure what you're trying to say here.
>>
>> Cliches doesn't make anyone listen, one only closes one's ears. It was 
>> a reaction to that poster child argument, once again.
> 
> Other than the phrase "poster child", which part was a cliche, exactly?

Nothing. That was the cliche.

> Your problem is that you want animated and non-animated PNGs treated 
> differently by the browser.  That means you need to tell the browser 
> that they're different.  Browsers do content dispatch by MIME type, so 
> you need to either give them different types or change how browsers do 
> content dispatch.  You seem to be trying to do the latter; I think the 
> former makes a lot more sense, personally.

MIME was the first thing I tried. The draft says that if the 
object has a @type that begins with the 5 characters "image/", 
then the UA should perform image sniffing.  So there doesn't seem 
to be much room for MIME to have any role.

>> My point with the animated PNG-s example was just to point out that an 
>> <object> with a well defined fallback model could also help vendors 
>> who want to support new file formats, as well as authors who want to 
>> use them.
> 
> All that already works fine for _new_ file formats.  The PNG issue is 
> that a new format was created, but always claims to be the same as an 
> old format.  That's just poor format design, imo.

I guess we must say the same about animated GIFs also then? 
(Since, when we get more animated image formats, one would like 
animated images to fallback to other animated images.)

So what to do? Register new mimetypes? Plus change what HTML 5 
says about looking at "image/"?

>> Not sure what you meant here, but I would like to propose that <video> 
>> should *formally* allow nested <video> elements (see top of letter)
> 
> I'm going to assume that you've read the long thread on precisely this 
> topic in this newsgroup and that further posts from you in that thread 
> will shed new light on the subject.

Yup. ;-)

>> And, yes, I would also suggest that such an attribute for specifying 
>> supportlevel/features, could be useful for <video>. Forexample,  a 
>> video might be available in two formats - one being HD quality.
> 
> It's still not clear to me why this cannot be handled via MIME types... 
>  What am I missing?

If it can be handled via MIME types, that would be fine.

>>>> - whether UA should count in plug-in support when it looks at
>>>>   whether it supports the resource or not.
>>>
>>> Define "plug-in"?

>> May be you hint that it can be hard to define what it is?
> 
> "Is", not "can be".
> 
>> May be it is more important to focus on necessary support, rather than 
>> plug-in or not.
> 
> Imo, yes.
> 
>> If QuickTime can display a transparent PNG,  then fine. But it must 
>> not be theoretical support - it must work inside the browser.
> 
> Is that a MUST in 2119 terms?  Or something else?  Are you expecting a 
> browser to somehow know whether there are bugs in various dynamic 
> libraries it's linking to?

No, not about bugs. Of course, we risk that an UA is classified as 
supporting, but still doesn't support fallback, as speficied.

>>>> - whether it is acceptable that the UA treats the resource as
>>>>   something else than what it is specfied to be.
>>>
>>> "specified" where?
>>
>> I had in mind that IE and Safari under certain circumstances presents 
>> <object> content as a frame because they think the content is a web 
>> page even when it is an image.
> 
> That's just a bug, no?  I believe that there was a post earlier in this 
> thread linking to the relevant Safari bug reports.

Sure, that was a bug. But you know that CSS is used to hide bugs. 
So that is not a contradiction, IMHO.

>>> This involves enumerating all possible features of all possible 
>>> resources?  Or something else?
>>
>> Yes. Or a well defined set ..., for a start.
> 
> I just think that this is the wrong way to approach the problem, since 
> any set will be too limited for some people, and since you _still_ have 
> buggy libraries claiming to support stuff that they don't... from 
> someone's point of view.  E.g. maybe they support 8-bit alpha in PNGs in 
> some cases, but not others...

OK. But then, what do you suggest? A special MIME for 8-bit alpha 
PNGs?

>>>> - whether an XML error in should lead to fallback or parsing
>>>>   errors being displayed or lead to fallback.
>>>
>>> This sentence seems to have gotten mangled.  Not sure what it was 
>>> trying to say.
>>
>> I'm not sure it is possible or even a good idea. But if the <object> 
>> contains a SVG image, and if the <object> also contains fallback, and 
>> if the <svg> fails to display due to non-conformant code, then the UA 
>> could display the fallback instead rather than displaying the broken 
>> SVG image.
> 
> Sure.  This could be a very reasonable thing to do, in fact.  Worth 
> bringing up as a standalone suggestion.

May be the draft allready agrees? From the OBJECT algorithm:

«If the image cannot be rendered, e.g. because it is malformed or 
in an unsupported format, jump to the last step in the overall set 
of steps (fallback).»

I guess it could be specified stronger? After all, there has so 
much discussion about draconian errors etc.

(Though we could also simply say that the draft is very unspesific 
about the fallback. )

>>  <object features="XML" data="image.svg">
> 
> Meaning what?  I think the behavior you describe above for SVG would be 
> a reasonable default behavior.  That's what happens for malformed raster 
> images, after all, assuming the error is "fatal" (happens before the UA 
> has gotten any data out of it).

Agreed.

>> (This doesn't seem any more different than false URIs, which also 
>> should cause fallback.)
> 
> Agreed.
> 
>>>> Better fallback requires that fallback is governed by more than MIME 
>>>> and URIs. Authors must also be able to specify the resource features 
>>>> that one are after - so the UA can go for fallback if it doesn't 
>>>> support those features.
>>>
>>> This is very difficult to do across the space of all possible content...
>>
>> It probably is. Like with everything else, one must begin with the 
>> beginning ...
> 
> Not if there's no hope of getting to the end, or even to a reasonable 
> stopping point.

So then we are left with what we have: MIME and @classid. Classid 
is possible to use, at least in IE, to some extent. MIME has some, 
but limited use. For the common formats, JPEG, GIF, PNG, I don't 
see that it has any use (by use I mean fallback use). Not unless 
the OBJECT algorithm changes, at least. The change one could 
introduce could be to say that before trying plug-ins (and the UA 
must find it's own definition fo that - but I think we coul agree 
in general),  the UA should try the nested OBJECT, if there is any 
and if contains an image. If there were such an image attribute 
that you talked about, this could be simpler, may be.
-- 
leif halvard silli

Received on Tuesday, 24 March 2009 04:59:19 UTC