Re: an interoperable object (fallback and context menus)

Leif Halvard Silli wrote:
> However, there is nothing in the draft that *forbids* a <video> elment 
> nested in the outer <video> element. Also, Henri's validator.nu accepts 
> it as valid. And it seems logical to nest <video> elements, if a UA do 
> not support the specific video format.

The same draft says that a single <video> element can specify multiple 
versions of the data in different formats, with fallback happening along 
that list.

> What is classid support in Opera, Firefox, Safari?

I can't speak to Opera or Safari.  Firefox recognizes a few 
"java:something" classid values that mean to invoke the Java plug-in; 
what it does after that is up to it.  Past that, any classid value means 
we fall back.  This is particularly true for clsid: values, since those 
imply a particular ActiveX component implementation.

In practice, this mechanism is not suitable for cross-browser authoring, 
since a classid really implies a particular implementation as defined in 
the spec, so any given classid value could be implemented by at most one 
implementation (e.g. one browser or one plug-in).  Any change to the 
implementation means the classid needs to change.  For example, if 
Firefox were to have a classid value that meant "render as png", it 
would need to change the value that means that any time anything changed 
in its png implementation (and either start falling back on the old 
values, or ship bug-compatible versions of the png implemenation for the 
old values).

Note that @type tells the browser what the data is but does not depend 
on any particular implementation.  This leads to the issues you have, 
but means that you don't have to have one <object> for every revision of 
every browser you target...

Of course we could redefine what @classid means to mean something much 
more general than what it means now, but there might be compat issues 
with existing content if we do that.

>> Now I realize your issue is that you want the UA to realize that the 
>> image is an image in this case and provide the appropriate context 
>> menu, etc.  So in fact, you would prefer that your fallback GIF be 
>> rendered by the UA to the TIFF being rendered by a plug-in.  This is 
>> by no means the most common use case with <object> however...  
> 
> The very *attemt* to use <object> for images is quite common, in the 
> "community". This fails - typically due to IE-before-8's bad support for 
> images in <object>. The IE7 script contains a workaroudn for it.

Yes.  But the desire to fall back to a different type of content 
altogether for an <object>, in preference to handling the type of 
content the author prefers but with a plug-in, doesn't seem common.  For 
the most part, those who are in that position just use the more broadly 
implemented content format to start with.

>> Perhaps we need an attribute (or set of attributes) on <object> that 
>> allows author control over the methods the UA uses for rendering the 
>> content?  Not sure what form this would take; a start here would be 
>> trying to figure out what the different rendering methods are and how 
>> one differentiates them.
> 
> Yes, more documentation would be needed.
> 
> As for solution, how about just implementing classid?

It's too fine-grained for sane use in the way you envision; see above.

> Another option is something a la CSS. For example, if one had an 
> attribute called features=""

Right; the problem then becomes defining the set of all possible 
features for all possible content one might serve, no?  Also not really 
tenable.

> What is needed, then, is to define some keywords - such as animation, 
> transparency, multipage-images etc - and get vendors to check if they 
> support what the keywords means.

Why are these keywords more important than, say "ES4" (for flash-like 
things), "3d" (for 3d acceleration of rendering for things like vrml), 
or any of a variety of other keywords people might care about for 
non-image content?

That is, why exactly are we trying to customize <object> to work well 
for images in particular?  Is the issue that we want the rich fallback 
capabilities that <img> doesn't provide but at the same time don't want 
the rich rendering capabilities (past using the built-in image renderer) 
that <object> provides?  If we really need such a "just make it act like 
<img> but with richer fallback" mode for <object>, maybe we should just 
be introducing a single attribute that means that?

>> They _are_ treated differently when it comes to fallback: <object> 
>> allows much richer fallback than <img>.
> 
> Of course: IMG do not allow same media fallback. That's the difference. 
> And that is also why it is logical that <img> loads the image via plug-in.

I'm not sure what that last sentence is saying.

>> And <img> is in fact fine element.
> 
> Sure. :-) But <object> becomes superfluos - from a graphical media point 
> of view - if it just duplicates how <img> behaves with regard to loading 
> plug-ins.

Um.  Except of course that it precisely does NOT do that right now. 
Your whole problem is that it doesn't, as far as I can tell.

> In that regard: it is problematic to have <img> as last fallback of 
> <object data="IMAGE">. E.g imagine this:
> 
> <object data=unsuppImage><img src=JPEG alt="Short txt."></object>
> 
> Here the "fine element" <img> trumphs the advantage of the <object>, 
> namley the rich fallback (unless the author adds extra CSS to hide any 
> possible text that is inserted after the image). But even this is of 
> course useful if <img> provides enough space for the fallback.

I have no idea what those three paragraphs are trying to say.

> This shows that for <object> to be able to play its role as media 
> element with richer fallback than <img>, it is necessary that the media 
> part works.

What about it doesn't work exactly?  You serve media of a particular 
type; the browser tries its best to render media of that type.  If it 
can't, it falls back.  You want it to try less hard, as far as I can tell.

> The claim that "you can simply use <object> instead of 
> @longdesc" becomes as de-rooted in sound reality as that famous poster 
> child, as long as <object> doesn't really work as it should.

So is your issue really with the way <object> is defined to work, or 
with particular browser bugs?  If the latter, what evidence is there 
that changing the spec in any way will help?

>>>> Well... if there were a separate MIME type for animated PNG, you 
>>>> could do this with content negotiation.  This seems like a poster 
>>>> child example of it, in fact.
>>>
>>> Cliches and deaf ears ...
>>
>> I'm not sure what you're trying to say here.
> 
> Cliches doesn't make anyone listen, one only closes one's ears. It was a 
> reaction to that poster child argument, once again.

Other than the phrase "poster child", which part was a cliche, exactly?

Your problem is that you want animated and non-animated PNGs treated 
differently by the browser.  That means you need to tell the browser 
that they're different.  Browsers do content dispatch by MIME type, so 
you need to either give them different types or change how browsers do 
content dispatch.  You seem to be trying to do the latter; I think the 
former makes a lot more sense, personally.

> My point with the animated PNG-s example was just to point out that an 
> <object> with a well defined fallback model could also help vendors who 
> want to support new file formats, as well as authors who want to use 
> them.

All that already works fine for _new_ file formats.  The PNG issue is 
that a new format was created, but always claims to be the same as an 
old format.  That's just poor format design, imo.

> Not sure what you meant here, but I would like to propose that <video> 
> should *formally* allow nested <video> elements (see top of letter)

I'm going to assume that you've read the long thread on precisely this 
topic in this newsgroup and that further posts from you in that thread 
will shed new light on the subject.

> And, yes, I would also suggest that such an attribute for specifying 
> supportlevel/features, could be useful for <video>. Forexample,  a video 
> might be available in two formats - one being HD quality.

It's still not clear to me why this cannot be handled via MIME types... 
  What am I missing?

>>> - whether UA should count in plug-in support when it looks at
>>>   whether it supports the resource or not.
>>
>> Define "plug-in"?
> 
> Most UAs have some things they support natively, and some things they 
> support via plug-ins only.

That's not a definition.

> Plug-ins are usually added after the browser was installed.

This is pretty much false for Java and Flash, for example (both ship as 
part of operating system installs and are often installed before the 
browser that will use them).

On the other hand, for Firefox one could write an extension that hooks 
into the "native" image rendering code and adds support for a new image 
format.  Its behavior will not be distinguishable from the existing 
behavior for GIF and JPEG.  Is that a "plug-in" per your definition?

> Or it is something that interacts closely with the UA, 
> but not close enought to be part of it. QuickTime is a plug-in, even if 
> it exist on 99,999% of all Macintosh machines.

This is still not a definition.  What do you mean by "part of it"? 
NPAPI plug-ins in Gecko (something well-defined: dynamically loaded 
libraries that talk to the gklayout library using the NPAPI APIs and can 
be called into via those APIs to handle data in various ways) are "part 
of the browser" in all sorts of senses once the browser has started up. 
  libimg2.so and libflashplayer.so are both dynamic libraries that 
gklayout.so loads and then talks to via well-defined APIs (XPCOM for the 
former, NPAPI for the latter) and hands content off to to be rendered. 
How do they differ, really?  What makes you treat one of them as a 
second-class citizen compared to the other?

> May be you hint that it can be hard to define what it is?

"Is", not "can be".

> May be it is more important to focus on necessary support, rather than plug-in or 
> not.

Imo, yes.

> If QuickTime can display a transparent PNG,  then fine. But it must 
> not be theoretical support - it must work inside the browser.

Is that a MUST in 2119 terms?  Or something else?  Are you expecting a 
browser to somehow know whether there are bugs in various dynamic 
libraries it's linking to?

>>> - whether it is acceptable that the UA treats the resource as
>>>   something else than what it is specfied to be.
>>
>> "specified" where?
> 
> I had in mind that IE and Safari under certain circumstances presents 
> <object> content as a frame because they think the content is a web page 
> even when it is an image.

That's just a bug, no?  I believe that there was a post earlier in this 
thread linking to the relevant Safari bug reports.

>> This involves enumerating all possible features of all possible 
>> resources?  Or something else?
> 
> Yes. Or a well defined set ..., for a start.

I just think that this is the wrong way to approach the problem, since 
any set will be too limited for some people, and since you _still_ have 
buggy libraries claiming to support stuff that they don't... from 
someone's point of view.  E.g. maybe they support 8-bit alpha in PNGs in 
some cases, but not others...

>>> - whether an XML error in should lead to fallback or parsing
>>>   errors being displayed or lead to fallback.
>>
>> This sentence seems to have gotten mangled.  Not sure what it was 
>> trying to say.
> 
> I'm not sure it is possible or even a good idea. But if the <object> 
> contains a SVG image, and if the <object> also contains fallback, and if 
> the <svg> fails to display due to non-conformant code, then the UA could 
> display the fallback instead rather than displaying the broken SVG image.

Sure.  This could be a very reasonable thing to do, in fact.  Worth 
bringing up as a standalone suggestion.

>  <object features="XML" data="image.svg">

Meaning what?  I think the behavior you describe above for SVG would be 
a reasonable default behavior.  That's what happens for malformed raster 
images, after all, assuming the error is "fatal" (happens before the UA 
has gotten any data out of it).

> This suggests a double nature of a such @features attribute: the UA 
> shoud know if it supports the announced feature, and go for the fallback 
> if it knows that it doesn't. But also, if the UA knows/understand that 
> the resource doesn't live up to the announced feature(s), the UA could 
> perform fallback.

I don't think the SVG XML thing is a good example of that.

> (This doesn't seem any more different than false URIs, 
> which also should cause fallback.)

Agreed.

>>> Better fallback requires that fallback is governed by more than MIME 
>>> and URIs. Authors must also be able to specify the resource features 
>>> that one are after - so the UA can go for fallback if it doesn't 
>>> support those features.
>>
>> This is very difficult to do across the space of all possible content...
> 
> It probably is. Like with everything else, one must begin with the 
> beginning ...

Not if there's no hope of getting to the end, or even to a reasonable 
stopping point.

-Boris

Received on Monday, 23 March 2009 02:01:20 UTC