[webapps-component] Web and Canvas Components, select, audio controls and ARIA

Regarding UI components in web applications

For me it's been an uphill battle to have Canvas surfaces recognized as 
a legitimate surface for interactive UI controls. I know XBL advocates 
have had quite a battle in developing a markup syntax for packaging UI 
components. I suppose I should be grateful. Without these battles, I'd 
have nothing to talk about...

What say you all about recognizing this situation as a class of problems 
and addressing it as such?

I've yet to reach a shared reality with Dimitri Glazkov, as he authors 
web components, I've yet to reach a compromise with Tab Atkins and Ian 
Hickson, the editor and assistant person-thing of HTML5, and I can 
barely hold a consensus with Rich Schwerdtfeger, a previous editor and 
prime mover of ARIA. Yet, these all live in the same realm of "Web Apps 
UI". They are somewhat separable from Web Apps APIs.

While I seem to have connected with Richard on the concept that ARIA 
should be applied to UI components (not that difficult, since that 
matches his world view), I'm struggling with others to gain consensus on 
the importance of UI APIs.

Tab and Ian have gone with a "pure" markup model, where the UA 
could/should and all the work declarative. Dimitri is in between, 
providing a markup language and shadow DOM, but not yet addressing other 
imperative APIs.


That's my situation with spec contributors. And now onto element cases:

<select /> has been around a long long time. It's a unique element in 
that it may display outside of the browser window. When I tap on a 
select element, the options list may go beyond the bounds of the client 
window. Neat stuff. It's one of the untouchable elements when it comes 
to CSS. Everyone and their grandmother has programmed some form of 
select via <div>. That's a big part of the motivation behind ARIA's 
creation.

<select /> hasn't changed much. We can now style the font and 
background, and maybe the scroll bars if we use some vendor-prefixed 
CSS. That's about it. <select /> [<option>] is a prime example of 
CSS+HTML Forms failing to work together.

<audio controls>, now that's an interesting case. This media element was 
designed with flexibility in mind. The flexibility being that, you use 
@controls, or you roll your own UI and we'll give you everything in the 
API you need to do it.

It's great, but the built-in UI suffers from the all-or-nothing 
phenomenon. And that sucks, because I have made CSS hacks with with the 
audio tag. I re-discovered that the css @zoom attribute is important for 
web components. I've used CSS transforms and had as much fun with audio 
@controls as I have with repainting the canvas tag to meet my visibility 
needs. That's all great, but I came up against an obstacle in 
implementations. I can't push and pull the <audio> tag out of the DOM 
without causing a stutter in the audio playing in Chrome. I filed a bug, 
sure, but I lost my ability to really use the audio tag in a flexible 
manner with @controls. I'm forced to implement my own @controls UI, even 
though I really would prefer to use the one packaged with the browser. 
All or nothing, and when something, something minor breaks, I'm back to 
nothing. I'd really like to use the progress meter, and play and volume 
buttons that are implemented for <audio controls>. But I can't, because 
it's all or nothing.

Then there's ARIA. There are some great techniques to expose data with 
ARIA. Now that CSS selectors are working with ARIA, I tell you my world 
is getting easier each day. But even ARIA can't help me with audio. I 
can't markup my custom play, pause, and I am indeed playing audio right 
now, elements in ARIA. So the moment I remove my <audio controls> tag 
from the DOM, because it fails on me, I also remove the opportunity for 
any standard method of pausing or detecting that audio is playing in my 
window.

While we can argue these are about bugs, or these are just shortcomings 
of browser vendors who haven't reached an ideal situation: they just 
haven't made that "final convergence" that the HTML5 editor is hoping 
for, I'd prefer to have a focused discussion on how we can meet 
obligations of programmatic access and of re-usability. I'd like to find 
a means of testing, talking about and solving issues with these UI 
components. ARIA has done a lot for that. I can prove that an AT is 
getting sufficient access to a component. I can prove that the scripting 
environment also has sufficient access. I want to carry that work forward.

To do that, I need to have a consensus of some sort in webapps that we 
are talking about the same thing.

Can I provide details to an <audio controls> UI that allows the progress 
meter to monitor progress? I've got everything but a means to say, yeah 
dude, you're at minute 1 and 20 seconds. Can I provide details to the UA 
and AT that an audio element is currently playing and spouting out some 
volume? Can I write a <select> element in canvas where I have indeed 
applied style to individual <option> elements, and have it work as well 
as a native element; browser zoom and all? If I can do that, then I've 
got myself a real application.

This is a problem that I believe can and should be addressed as a whole, 
without negativity, without exclusions. It is being addressed, but it's 
being addressed in parts where various spec editors are fundamentally 
opposed to each other. IME API on one side and WHATWG on the other. It's 
a mess. It doesn't need to be. I want to get a real, lasting solution to 
the issues that keep on battering me as I try to write web applications.

I am frustrated with <audio controls>, I am disappointed with <select>, 
I am desperate to get <canvas> working with the OS and I am hoping that 
we can talk on a real technical level about how we can make these things 
happen. Because, at present, I've just heard a lot about how we 
shouldn't make them work. How we should discourage authors like myself. 
Or about how we should just wait, because eventually, a vendor will get 
things right, they'll have the perfect input element.

Please consider these UI issues as a whole, find a coherent narrative 
about the problem. If and when we can get that far, maybe we can have a 
webapps-ui group, and discuss the particulars. Because I don't think 
these particulars are relevant to public-webapps proper. I think they 
should be hashed out elsewhere, but I haven't found an appropriate 
elsewhere to do it.

-Charles

Received on Friday, 24 February 2012 02:09:50 UTC