Re: [css-shaders] CSS shaders for custom filters (ACTION-3072)

On Thu, Oct 6, 2011 at 11:51 AM, Chris Marrin <cmarrin@apple.com> wrote:

>
> On Oct 6, 2011, at 10:19 AM, Gregg Tavares (wrk) wrote:
>
> >
> >
> > On Thu, Oct 6, 2011 at 9:44 AM, Chris Marrin <cmarrin@apple.com> wrote:
> >
> > On Oct 5, 2011, at 9:27 PM, Robert O'Callahan wrote:
> >
> > > On Thu, Oct 6, 2011 at 11:10 AM, Chris Marrin <cmarrin@apple.com>
> wrote:
> > > We do that by reverse mapping the mouse positions from global into
> local coordinates. That's easy with a single (or composite) transform. It's
> impossible for pixels that are altered by a vertex or fragment shader. You'd
> essentially have to run the shaders backwards to get them to tell you where
> a given point is in the local coordinate space. I know of no hardware that
> can do that. I haven't seen it anyway...
> > >
> > > I think we can give up on having event handling account for the
> geometric effects of fragment shaders. We should encourage authors to change
> geometry using vertex shaders instead.
> > >
> > > In principle, we can have event handling account for the effects of a
> vertex shader. Once you've computed the transformed mesh, you can figure out
> which point on the transformed mesh is topmost under the event position, and
> map that point back to the pre-transform mesh.
> >
> > But imagine an author who wants to use a very dense mesh to get smooth
> curves to closely match the curve effect you can get from a fragment shader.
> Mapping mouse positions with such a mesh would require thousands of matrix
> operations, including inverse which is very expensive. Especially on mobile
> hardware, this would be too expensive to be practical.
> >
> > When native apps need to do this sort of thing they often will construct
> a second, simpler set of geometry which animates along with the finer mesh
> and is used for things like picking and collision detection. We could do
> something like that, but it would probably be hard to come up with the right
> heuristics. And it would still require running the shader in software, which
> would be quite a big impediment to many implementations.
> >
> > It seems to me if you can't click the results then this proposal
> shouldn't be implemented as specified. All the CSS to date has been about
> styling the content, not moving it such that's it's no longer interactive.
>  Offset and displace are there soley to do things like drop shadows or
> pseudo 3d text.
> >
> > The CSS shader proposal breaks all that. If all it was doing was allowing
> you to generate 2d filters (pixel shaders only?) then it might make sense
> because its uses would mostly be limited to effects current CSS provides
> just with more flexibility.
> >
> > The current CSS shader proposal is taking CSS in an entirely new
> direction and it arguably breaks everything CSS was designed for. To me that
> requires either solving the problems or limiting its functionality so as not
> to break things. If all you want is non-interactive 2d content warped into
> 3d and then re-flattened into 2d then just use WebGL
>
> "Entirely" and "everything" are strong words. The Adobe demos clearly show
> some compelling use cases where pixel accurate interaction of filtered
> content is not needed. Other filters in the SVG filter spec have the same
> issue (e.g., feOffset and feDisplacementMap). So I think there is some
> precedent here.
>
> I understand your desire to do your Offscreen Tabs demo (
> http://dev.chromium.org/developers/design-documents/extensions/offscreen-tabs)
> using CSS shaders. But the fact that it doesn't fully solve that use case
> doesn't make it useless for others.
>
>
Possibly. I think though that if you check most interactive 3d interfaces to
date you'll find they are interactive. Heck, even Apple's Coverflow allows
you to click on the images off to the left and right. With this proposal, if
I was to implement coverflow using CSS shaders but change it to use slightly
warped cards then hit testing would no longer be accurate.

The current CSS shader proposal seems to be trying to be step toward
allowing 3d UIs but as you have pointed out it's nearly impossible to make
it work and still be interactive.

As an actually workable counter suggestion, what about being able to specify
meshes in css and morph between them, something like

@mesh folded-map {
   vertices:   x, y, z, u, v, x, y, z, u, v, ....
}

@mesh unfolded-map {
   vertices:   x, y, z, u, v, x, y, z, u, v, ....
}

@keyframes fold-unfold {
   from { mesh: folded-map }
   to { mesh: unfolded-map }
}

That seems in some ways more flexible than vertex shaders since you can
supply meshes instead having to compute them algorithmically and it's easily
solvable for keeping things interactive.





> -----
> ~Chris
> cmarrin@apple.com
>
>
>
>
>