Re: comments on draft-barth-mime-sniffing

>> There, I meant renderer to mean the player or whatever that 
>> friendly thing behind the <audio> element is: the thingee that 
>> plays me.ogg and knows literallyl nothing else about anything but 
>> .ogg and .wav.:)

> OK.  So you don't mean the two different things one of which knows 
> nothing about anything but .ogg and the other of which knows nothing 
> about anything but .wav.  Gotcha.

Yes, my depth of vision into the process is text of the <audio> 
element in this case. I neither know about nor care about the 
machinery behind it. All I have is a list of file extensions and MIMEs 
that are acceptable. I dutifully prepare my files with that 
information in my user code.

> So the "renderer" in your definition still needs to decide whether 
> to handle the data as an OGG or a WAV.  With this definition, the UA 
> does not indeed need to know anything about the kind of data it's 
> handing the renderer.  The renderer does need to know what sort of 
> data it's looking at, though.

I told it what the file extension and type. I got them from the spec 
list then I made my content in some tool that gave me the right file 
extension. I put it the file on the server. What else does it need to 
know?

>> No, in my little fantasy of this case, there is only one renderer 
>> for .ogg and .wav and that is all the host browser knows about for 
>> <audio>. There is no choice, If I <audio my.ogg> all I care about 
>> is that the host browser picks the 'built-in'  or native one that 
>> only knows about these file types.

> The "native renderer" as you call it then has to delegate to two 
> completely different subsystems for the OGG and WAV cases. How do 
> you envision it picking which one to use?

I told it the file extension and the type in genuine HTML 5 
keystrokes. What do I have to do, hold its hand?


>> What can me.ogg do to corrupt the system using the simple native 
>> player. ogg or .wav does not have and active form, I think.

> You're assuming a completely non-buggy player.  That's desirable, 
> but unlikely in practice.

That is too bad and not really acceptable at introduction. Seems like 
all the browser makers should join forces on this one and make it 
work. It is the right thing to do.

>> 'I see in the general sniffing algoritm that the file is read to 
>> look for the internal contenttype in the header. If it finds this, 
>> then that is the mime. Yes, you can find the data structures in the 
>> detailed specs for the streams but lookie here mr browser maker, I 
>> don't want to fool with all that stuff.

> No one is asking you to.

Good. I need a break.

>> I put me.ogg out there on my server somehow and then I have an HTML 
>> 5 web page that says play what I put out there, please.

> Sure.  And the browser will take that data, and treat it as what you 
> tell it that data is.  If you lie to it... then it'll still treat it 
> as what you tell it that data is.  Ideally, at least.

I won't lie, at least I will really try. If I lie, maybe it shouldn't 
work but I don't think you are going to find that out by looking at 
the served MIME.

>> The browser should let this happen with  no built-in security or 
>> other delays except perhaps some user preferrence settings about 
>> multiple media, on the level of a show images choice somdwhere.

> I have no idea what you're trying to say here.

I mean such as a warning or caution that hey lookout we are starting a 
player that could damage your system. In this web the <audio> and 
<video> are first class and are rendered just like other first class 
content like <p> and <em> and as easy to use if you stay in the spec.

>> Oh, I will look at that. If type is there, that should be clue 
>> enough to start the player and start fetching the content
>
> What's "the player"?  OGG and WAV need different "players".

Another implementation detail There is no <audioogg ...> <audiowav 
...>, the player is whatever the UA sets up for the <audio> element 
which may be requested to play some wav or hopefully some ogg.

"4.8.8.1 Audio codecs for audio elements

User agents may support any audio codecs and container formats.

User agents must support the WAVE container format with audio encoded 
using the 16 bit PCM (LE) codec, at sampling frequencies of 11.025kHz, 
22.050kHz, and 44.100kHz, and for both mono and stereo. [WAVE]"

I think it is more reasonable to drop the first sentence and update 
this spec as the proven cross-browser capability increases. Ogg next. 
Also, what is the need to restrict the kHz? wav also does 12/24/48/96, 
and ogg does a fixed Q or variable rates. and all rates should work 
fine, I think.

>> There should be no choice.
> Choice by whom of what?

I think that had something to do with <video>

>> Anyway, the correct thing to do is for the host browser not to 
>> worry about it all. Just send the url to the player behind the 
>> <audio> or <video> element and let it figure it out and when it is 
>> ready, let it ask you for the content by url and really, just let 
>> the handler deal witht the loading. I would hope the host has 
>> better things to do than spoonfeed content to the audio or video 
>> handler.
>
> What sort of media player are you using, exactly, that has a rich 
> network stack built in?  In practice, it is exactly the browser's 
> job to get the data and "spoonfeed" it to the player; the player 
> just knows what to do with bytes, not with URLs.

OK, I guess this player shouldn't need to go get URLs by itself. Maybe 
spoonfeeding it is best, but I really don't know. Yes, I do get to 
play with a couple of 'em that expose a very rich network stack in 
some environments. Nothing really proprietaryd, though. .

>> 'OK, let's make it like that in the spec. Right now are listing the 
>> acceptable file types identified mainly by file extension.

> Where, exactly?

Well, now that I look again, it seems like <audio> can be anything 
(must have wav) and <video> can be anything so hey, I guess browser 
makers will just have to keep doing what they were doing in the old 
days before HTML 5.  I hope it works out better.

>> That is very enlightened and forward looking. However in this case 
>> where all I want to do is play me.ogg I don't really care about all 
>> that. I know what I put out there. Trust me.

> Sure. We do! "You" in this case is the web server.  We'll handle the 
> data based on what you tell us it is.

That is fine. If the server is right.


>> Anyway, with these file extensions, what is the problem?

> With which ones?

Well, for <audio> the only thing we tell them is that at least certain 
wav is OK.
Anything else is browsers dependent.

>> OK, I must step back a notch and focus just on <audio>.

> OK...

>> No problem for audio, right. Send the url or the file content as 
>> instructed in the user code no sniffing needed because we trust the 
>> author to have done the right thing.

> Uh...  The sniffing is just needed to pick the right player, in the 
> case when the author hasn't bothered to tell us what kind of media 
> he's sending.  Assuming sniffing is done at all.

>> I will have to look at that in more detail. Are you saying there 
>> are .wev files out there that this simple player is not going to 
>> play.

> Sure.  Firefox, or example, is shipping a WAV player in version 3.5 
> that plays only a subset of the WAV files out there. 
> https://bugzilla.mozilla.org/show_bug.cgi?id=475110 has some example 
> WAV files attached that will not play in that player.

Great saga. Looks like the right goals. Where is ogg?
That should be the <audio> player for everyone. All you browser makers 
get that thing working right and turn it loose as a simple handler 
that works in <audio> everywhere for that certain set of required 
support files. Certain wav(s) but if you want anything else, go to 
'house' player which the user may need to get from someplace like QT 
or Win or Real and use <object>. That is the secret to doing something 
good with <audio>  -  a limited easy to use player that works with 
these file types in all the browsers.

>> However, in either of these, there is nothing that would give the 
>> player the idea that there was some script it was supposed to 
>> execute or anything else hazardous.

> I'm not sure what script has to do with this whole thing....

Right, I meant a hidden script in the <audio> content. That shouldn't 
have anything to do with it because it can't happen.

>> For me it is easy to draw the line: I'm just the simple little 
>> <audio> player. I know I don't do much, but give me a chance. Host, 
>> get out of the way and let me do it. For these file types, give me 
>> the bits straight for the server as fast as you can and I will 
>> handle it.

> OK. So you are a chunk of code that will take a byte stream and the 
> MIME type and then do the right thing.  Fine.

Not just any one. Only the byte stream(s) required by the spec. I just 
know that if you put a wav in an <audio> I will get picked first.

>> Great. But how is the server going to screw it up. I ask for the 
>> file and he sends it, but uhoh he sends the wrong contenttype.

> Well, define "wrong".  Not the one the author intended?  The browser 
> can't read your mind; it can't tell whether the server is sending 
> what you intended or not.  We can only assume that it is or isn't. 
> I'm arguing we should assume it is.  Others want to assume it 
> isn't...

It is, it is. It is what I told you it was going to be in my <audio> 
or <source> statement.
If the server does not agree, the fact that the server doesn't agree 
with the user request (I think I requested it by file name and 
extension) that is just a sad coincidence

>> Maybe not, maybe he is up to date or the webmaster has used 
>> .htaccess corretly. For <audio> why do I care?

> Because you need to decide whether to play it as a WAV or an OGG?

So the host is going to tell me here comes a wav? If I can only do 
wav, i think I can figure out what to do when I start getting the 
stream.

>>  Somebody else may  be torched because the mime has not passed ietf 
>> or whoever and/or the server has not been updated. Do I gotta take 
>> the wrap for that?

> You're sort of responsible for what you put on the wire, yes.

Well, I want to be. That is why I do it, I guess

>> ... as we see, mostly a user can manage to get .html served 
>> correctly as text/html or whatever the recommendation.

> Sure.  And CSS as text/css (and if he doesn't, a number of modern 
> browsers won't treat it as CSS).  Why is this case different?
> I can't tell what your point is, honestly....

I think it was about how sniffing can help solve problems, mainly 
security issues.

>> But how do I fix my server?

> That depends on the exact server you're using; I suggest contacting 
> your hosting service (or server vendor if you configure your own 
> server).

>> I pick a file extension, like .x3d for instance, then I tell my 
>> server (maybe via .htaccess) to serve that file extension as 
>> model/x3d+xml.

> That's up to you.  Some servers use this model for configuration; 
> others use other models.  Some servers support more than one 
> approach to this problem (e.g. Apache).

Above. linking up a mime with a file extension is one way to do it for 
Apache. specifically, like
AddType model/x3d+xml .x3d

>> Why not fix that in the host browser by me telling it that if you 
>> see a .x3d always treat it as being served as model/x3d+xml.
>
> Because the host browser is not being run by you.  It's being run by 
> the user...  Which means that you can't tell it anything directly. 
> All it knows is what the web server tells it.

Well, I meant in some W3C browsers that I personally use I can sort of 
experiment around in some of that.

>> But really, from the server, as far as I know, it keys on file 
>> extension.

> Some servers, sometimes, do that.  I can assure you that if you have 
> a PHP script that serves an OGG file you do NOT want your server 
> keying off the ".php" extension for the type.  Nor does it.

I am glad. I hoped I would never see or hear of anything like that 
again.
But really,  the file reqested from somewhere, maybe home, as .ogg. If 
home did not request it and does not know which guest is coming, then 
maybe the served ct will tell, or maybe the file meta ct will tell. If 
the request is from home, you may know more about what to expect back. 
Please note that if that request ends up wanting a new audio([src]) 
then wherefore the MIME?

This is why the types of files played at intro of this feature has to 
be a known subset of the avworld with some good hints about creation 
tool config.

Another message close to this advised trying to find out what the 
actual situation is out there with the server settings.
So, if I give the UA
<audio autobuffer autoplay><source src='me.ogg' type='audio/ogg' 
 ></audio>
are you going out the the server to look for MIME or name.extension?

The sniffing routine as shown in the referenced barth-mime-sniffing 
text is fine and it mentions looking for the the content type coming 
from the server with the content and also possibly looking inside the 
file at the meta content type string. Maybe I should look again, but 
the step of looking inside in that way is generally not needed for 
<audio> and <video> media types because something different but 
equally identfiable is probably in there.
This does remind me that in X3D we had decided not to include a 
requirement or even suggestion to include a specific meta content type 
declaration. Maybe because we did not have an approved MIME at that 
time. With this mime-sniffing algorithm that finally gets down into 
the internal meta, then that might be the last chance to get on board 
if the server ct messes up.
Anyway, I have seen mimes at work. All browsers are using them better 
and better all the time. I think I even know in a special example, how 
to test whether or not the W3C browser even cares or knows. 
Fortunately, bad or non-existent behavior is diminishing.

What I probably want to do is start it like this:
<audio autobuffer autoplay><source type='audio/ogg' ></audio>
and wait to hear back from the DOM when the handler is up and running 
and ready to be sent a src URL. At this point, the handler still may 
not know what it is going to play, it is just ready with currentSrc; 
empty (or null) and HAVE_NOTHING. When it is ready and I am ready, 
then maybe try to send it something to play. Then the player will 
somehow ask its parent context for some content by name URL; does it 
need to include a type in the request?

So as I look at the HTMLMediaElement in more detail I remember again 
how much fun this multiple media stuff really is. It should be as 
simple and reliable as me filling a div with new text. Looks like all 
the controls and stati are in there and it only remains to actually be 
able to stuff the thing with some content. By golly I might be ready 
to actually look around and find an <audio> element to work on.

Thanks Again and Best Regards,
Joe

Received on Friday, 19 June 2009 10:59:00 UTC