Re: CSS shaders: Cinematic effects for the web | Adobe Developer Connection

Hi,

[Remark: It became rather long than short, but it's a hot topic and we 
should get into this discussion]

I had a deeper look into CSS shaders and Filter Effects. Let me try to 
give a short overview and relations to our work:

1. Filter Effects allows to define 2D filter effect graphs. One filter 
graph can be composed from several predefined filter functions. These 
can be complex and thus are encoded in XML/DOM. A 'filter' property is 
introduced to assign a filter to geometry. This filter property can 
reference one filter function or a concatenation of filter functions. A 
filter function is either:
- a reference to a filter graph defined inside the same or in an 
external document (via url() syntax)
- a shorthand to one of the predefined filters

2. CSS Shaders
CSS Shaders extends the filter functions by a 'custom' filter (might be 
renamed to shader later). This filter adds programmability to the Filter 
Effects graph. The input is the rendered SVG/HTML box as image plus an 
implicit mesh geometry. The granularity of the mesh geometry can be 
configured. Also a selection of other maybe useful shader parameters are 
passed to the custom shaders (projectionMatrix, texture coordinates 
etc). The examples are based on OpenGL ES Shading Languages and mappings 
between shader parameter types and OpenGL ES types are given.

I see some similarities to our approaches, but also some issues. In any 
case, this brings us directly into the "shading system" discussion, we 
wanted to start with first results until TPAC.

So let me start with the interesting points:
- The combination of FE and CSS Shaders fits to the approach "simple 
things simple, complex things possible)
- Assigning the shader (I use shader here as a synonym for filter) to 
the geometry via CSS makes absolutely sense, has been discussed and is 
done e.g. in XML3D ;)
- Interestingly enough, Filter Effects allows the definition of complex 
graphs in DOM/XML. It seem CSS alone was not expressive enough?! @Chris: 
Do you know some backgrounds here? Anyway, this could be an approach for 
Dec3D, too
- CSS Shaders introduces generic parameter definition in CSS. Since we 
probably will need generic parameters for Dec3D, this is very interesting
- The Filter Effects is very similar to Shader Trees in 3D. Composing 
complex shaders from simple predefined operations is very common in DCC 
Tools. It also gives an abstraction layer above the used rendering 
technqiue; a compiler can merge the operation into a single shader 
object in the best case. The predefined operations create some kind of 
"sandbox". It's seems to be a good tradeoff between full flexibilty (and 
security issues) using arbritary shader code and single fixed function 
shaders on the other side

Some issues/incompatibilities is see:
- The approach is tied to the bounding box model of SVG/HTML
- For the simple bounding box model, the implicit shader parameters are 
straightforward. For 3D objects, the number of implicit shader 
parameters (lights, transformations etc) and vertex attributes are far 
more complex
- I think in the 3D case, we should distinguish between functions to 
apply on the mesh and shading functions. The vertex/fragment shader 
model mixes these things ups
- Generic parameters in CSS are nice, but I wonder if this string based 
mechanism is enough. Unfortunately there is no connection to Typed Arrays.
- The authors of CSS shaders emphasize the compatibility with CSS 
Animations and Transitions. On the other hand, shader parameters can not 
be animated individually (s. Note 11, 
https://dvcs.w3.org/hg/FXTF/raw-file/tip/custom/index.html#shader-params-interpolation). 
This

These are just my observations, no conclusion yet ;)

Best,
   Kristian


> Hi,
>
> Just skimmed over, but looks very interesting.
> Sme questions at @Chris:
> - Do you know these guys?
> - The Filter Effects are encoded as elements/attrbutes only, right? For
> backwards compatibity to SVG agents without CSS functionality or are
> there other reasons?
>
> Thanks,
> Kristian
>
>
>
> Am 03.10.2011 23:17, schrieb Chris Lilley:
>> On Monday, October 3, 2011, 11:11:31 PM, Johannes wrote:
>>
>>>> On Monday, October 3, 2011, 10:07:32 PM, Sandy wrote:
>>>> SR> This seems interesting...do we need to interact with these
>>>> folks?? How??
>>>> SR> Sandy
>>>> SR> http://www.adobe.com/devnet/html5/articles/css-shaders.html
>>>> Yes, its interesting. Also, clearly extensible to 3D vertex and
>>>> pixel shaders.
>> JB> I didn't read the full spec yet but this looks very promising.
>> JB> It's not predefined fragment effects (like the svg filter)
>> JB> but supports full custom vertex, fragment and mesh setup. Even
>> JB> glsl is the vertex/fragment language of choice.
>>
>> Yes. Note that this is planned to be implemented as a 'custom' filter
>> in the context of svg filter effects.
>>
>> Only rectilinear trimeshes are supported (but I think that is a
>> reasonable trade off).
>>
>> JB> Does this run in any browser yet?
>>
>> Apparently Adobe has an experimental Webkit build that does this, and
>> will be demo-ing it as their developer conference this week.
>>
>> JB> best regards
>> JB> johannes
>>
>>
>>>> As to how: the work is being done in the FX taskforce, which is a
>>>> joint taskforce of the SVG and CSS Working groups.
>>>> public-fx@w3.org
>>>> http://lists.w3.org/Archives/Public/public-fx/
>>
>>>> --
>>>> Chris Lilley Technical Director, Interaction Domain
>>>> W3C Graphics Activity Lead, Fonts Activity Lead
>>>> Co-Chair, W3C Hypertext CG
>>>> Member, CSS, WebFonts, SVG Working Groups
>>
>>
>> JB> --------
>> JB> Dr.-Ing. Johannes Behr tel: +49-6151-155-510
>> JB> Fraunhoferstr. 5 fax: +49-6151-155-196
>> JB> D-64283 Darmstadt skype: johannesbehr
>> JB> Germany web: www.igd.fhg.de/www/igd-a4/
>>
>>
>>
>>
>
>


-- 
________________________________________________________

Kristian Sons
Deutsches Forschungszentrum für Künstliche Intelligenz GmbH, DFKI
Agenten und Simulierte Realität
Campus, Geb. D 3 2, Raum 0.77
66123 Saarbrücken, Germany

Phone: +49 681 85775-3833
Phone: +49 681 302-3833
Fax:   +49 681 85775–2235
kristian.sons@dfki.de
http://www.xml3d.org

Geschäftsführung: Prof. Dr. Dr. h.c. mult. Wolfgang Wahlster (Vorsitzender)
Dr. Walter Olthoff

Vorsitzender des Aufsichtsrats: Prof. Dr. h.c. Hans A. Aukes
Amtsgericht Kaiserslautern, HRB 2313
________________________________________________________

Received on Tuesday, 4 October 2011 09:43:00 UTC