Re: [filter-effects] New syntax proposal for 'custom' filter function

On Tue, Dec 4, 2012 at 10:09 PM, Dirk Schulze <dschulze@adobe.com> wrote:
> Hi David,

Hi Dirk,

> You seem to mix up the proposals, and based on that make wrong assumptions and conclusions about the last proposal. Lets make some examples:

I don't think I have mixed up the proposals and I'm uncertain what you
believe I have assumed wrongly as you have not actually responded to
the concerns in my previous email (Message-ID:
<CAAWM5Tzmi73JW7y01c5rY5MZc3=VvMVPYnTt59CN9OBb=yBKcw@mail.gmail.com>).
If you respond to those questions, I believe we would all learn more.

> #0 The proposal in the working draft
>
> This proposal has just the custom function, with one or two url's, a vertex mesh and a list of parameters, which could look like this example:
>
>         filter: custom(url(fragmentShader.fs) mix(url(vertexShader.vs) source-in multiply), 4 4, identityMatrix mat3(1,0,0,0,1,0,0,0,1) )
>
> The problem with this proposal is, that it is not future proof and does not allow different shading languages or fallback shaders.

I never refer to this proposal. It is inadequate as you point out.

> #1 custom-filter at-rule
>
> The syntax for the at-rule would look like in the following example:
>
>         @custom-filter curlGLSL {
>                 format: "glsl-1"; /* we can find a better string for it */
>                 fragment: url(fragmentShader.fs);
>                 vertex: url(vertexShader.vs);
>                 mesh: 4 4;
>                 mix: source-in multiply;
>                 parameters: identityMatrix mat3(1,0,0,0,1,0,0,0,1);
>         }
>
>         @custom-filter curlHLSL {
>                 format: "hlsl";
>                 fragment: url(fragmentShader.fs);
>                 vertex: url(vertexShader.vs);
>                 geometry: url(geometryShader.gs);
>                 mesh: 4 4;
>                 parameters: identityMatrix mat3(1,0,0,0,1,0,0,0,1);
>         }
>
>         filter: custom(curlHLSL, curlGLSL);
>
> The custom filter provides a list of fallback shaders. Every fallback would need another at-rule definition. This will blow-up the CSS code, but may could be solved with media queries to other possibilities. The problem are the descriptors. We can not define descriptors for all shaders today, nor can we predict descriptors for future languages. We would need to allow other shading languages to extend the CSS syntax for their needs.

I refer to this proposal as an effective starting point as it provides
a complete representation for the present operational model in the
working draft.

> #2 a generic filter at rule
>
> The syntax could look like the following:
>
>         @filter curl {
>                 src: url(shaderGLSL) format("glsl-1"), url(shaderHLSL) format("hlsl");
>                 parameters: identityMatrix mat3(1,0,0,0,1,0,0,0,1);
>         }
>
>         filter: custom(curl);

This proposal totally neglects the operational model in the working
draft. The parameters and semantics of the previous proposal are now
hidden inside of some new effect format.

> I actually do not assume that every web developer will write the shader code their own. This is where your idea with the shader server comes into the game. The syntax is very clear, very short and can be adapted and integrated by web developers easily.

Writing shader code is not required for customizing shader code or its
parameterization. This proposal is clear and short by hiding of
semantics inside of the effect formats. Where does the author specify
mesh dimensions?

Many developers will not want to use a new effect format. There are
similarly easy solutions that offer more flexibility to developers and
users and require less support by browsers.

> The fallback is integrated as well, and the CSS syntax does not need to be changed by the supported shading languages if there is a need to. The syntax is a lot more future proof and allows the extension by other formats.

Fallback is easily integrated into the previous proposal. CSS syntax
only needs to be changed to allow a new @-rule and associated schema
for each individual operational model. As the operational model for
filter effects is part of filter effects and not necessarily tied to a
specific language, this coupling seems justified.

See my proposal at the end of this mail for a future-proof and
extensible solution.

> The syntax is very similar to @font-face and shares some concepts. @font-face does allow more then one format (OpenType, TrueType and WOFF as container).

font-face is both more monolithic and less customizable than custom
filter effects functions.

> The down side is that it just hides the parametrization. With the current syntax, just one URI would be possible. The parametrization and the shaders would need to be request able with just this one URI. There are different ideas how to solve this issue. XML is just one idea.

I don't see anything about the parameterization that doesn't fit in
either the reference from CSS or the shader source itself. What does a
new place to encode parameterization win us?

> It looks like some of your arguments are based on proposal #1 (your reference of the geometry descriptor for example). In this case I would understand the proposals with fragments and <script> elements.

Proposal #1 contains the model that proposal #2 hides. I referred to a
geometry descriptor in response to your assertion that such an
extension would require a new 'format'. I believe both #1 and #2 can
handle this issue but #2 does not require standards to use consistent
semantics (bad) as #1 does (good).

Using fragids and associated machinery is one example of reusing an
existing system to achieve similar goals as bundled effects.

> Or I don't understand how fragments can solve the problem of proposal #2.

See <http://dsheets.github.com/custom-fx-modules/> which may answer
your question (@import + fragids).

> I don't see your point that we would break existing web technologies with any of the above proposals, but am happy to adapt the proposal if it does.

I don't believe I ever asserted that any of these proposals broke
existing web technologies. Proposal #0 does not satisfy extensibility
requirements. Proposal #1 could be refined into a powerful tool
<http://dsheets.github.com/custom-fx-modules/>. Proposal #2 ignores
features of HTTP, URI, HTML, XML, and CSS in favor of introducing a
specific new XML vocabulary which will not have the same flexibility
as easily as the rest of the platform already has.

Regards,

David

> Greetings,
> Dirk
>
> PS: I just used HLSL as example here, but it would also be possible to use another dialect of GLSL instead, as David mentioned.
>
>
> On Dec 4, 2012, at 6:24 PM, David Sheets <kosmo.zb@gmail.com> wrote:
>
>> Hi Dirk,
>>
>> On Tue, Dec 4, 2012 at 1:33 PM, Dirk Schulze <dschulze@adobe.com> wrote:
>>> On Dec 3, 2012, at 5:39 PM, David Sheets <kosmo.zb@gmail.com> wrote:
>>>> On Mon, Dec 3, 2012 at 3:46 PM, Dirk Schulze <dschulze@adobe.com> wrote:
>>>>> On Dec 3, 2012, at 2:30 PM, David Sheets <kosmo.zb@gmail.com> wrote:
>>>>>> Once a media subtype is chosen for the shading language, I believe
>>>>>> this parameter should mirror that string (i.e. "application/webglsl"
>>>>>> => "webglsl" or "text/essl" => "essl" or "application/glsl" => "glsl"
>>>>>> (yikes)).
>>>>>
>>>>> Dependent on the file format, it might not be necessary to rely on the MIME type. MIME types can also be a burden if the server does not support certain MIME types. The string for the format would be to determined. Any format needs to make sure that the requirements of the used language are fulfilled.
>>>>
>>>> I don't understand your objection to alignment with a media type. HTML
>>>> has a media type, "text/html". CSS has a media type, "text/css".
>>>> Javascript has several media types. WebGLSL should have a media type
>>>> to refer to shading language resources that use WebGLSL's syntax and
>>>> semantics. If a package format is served, the serving host needs to
>>>> know what shading language should be served in the package. Each
>>>> language may differ in its CSS parameters.
>>>
>>> It is not an objection per se. Requesting documents with MIME type from a server is always a risk. I have problems with servers that don't accept "image/svg+xml" for SVG documents today. The specification is more then 12 years old. For interpretation on the client side it may work, since the server can still send it as text.
>>
>> What server has the following behavior?
>> You request URI Q with "Accept: image/svg+xml" and the server responds
>> with code 406 with a body that isn't image/svg+xml.
>> You request URI Q with "Accept: text/plain" and the server responds
>> with code 200 with a body that is image/svg+xml but marked as
>> text/plain and interpreted by the client as image/svg+xml.
>>
>> If this actually occurs, the server is clearly broken by
>> simultaneously being too strict and not strict enough. This does not
>> prohibit the specification of reasonable media type behavior, however.
>> E.g:
>>
>> You request URI Q with "Accept: application/webglsl, text/plain" and
>> the server responds with something you can interpret with
>> "Content-type: application/webglsl" or "Content-type: text/plain".
>> Ideally, you request URI Q with "Accept: application/webglsl,
>> application/xml" or however many types you can interpret in this
>> context and if the server responds with "Content-type: text/plain" you
>> have to sniff as always.
>>
>> Please explain the risk of requesting the specific media types that
>> the client is capable of interpreting.
>>
>>>> If an author controls both page and shader server, this doesn't matter
>>>> very much. Hopefully, it is a goal of FXTF to consider third-party
>>>> shader effect hosts for which knowing as much as possible
>>>> declaratively about the interpretive capabilities of the client is
>>>> critically important.
>>>
>>> In a lot of cases the author is not under control of the server. Therefore it is hard to rely on this assumption. This actually does not block your idea of a shader server.
>>
>> What assumption? I am not concerned with being unable to implement a
>> shader server. I am very concerned that the CSS FX specification will
>> not specify network requests detailing what specific languages the
>> client can interpret. What are the downsides of alignment with the
>> design principles of HTTP?
>>
>>> Furthermore, the shader files are not fully compatible with GLSL or WebGL, since we have more security requirements and predefined variables.
>>
>> Every valid CSS FX WebGLSL shader is also a valid WebGLSL shader. Your
>> security requirements may be satisfied in one of two ways:
>>
>> 1. a language extension to introduce new annotations for secure
>> textures which are easily removed by non-sensitive processors
>> 2. a language coextension to declare certain normally-allowed texture
>> manipulations as globally disabled
>>
>> All existing processors (WebGL and ESSL), to my knowledge, simply
>> ignore the coextension declaration as they have no problem "disabling"
>> extensions of which they are ignorant. This solution is the least
>> disruptive and clearly indicates CSS FX shaders are a subset of all
>> WebGLSL shaders which can be used interchangeably without requiring
>> insensitive WebGLSL processors understand a new security restriction
>> extension just so they can ignore it. See
>> <https://www.khronos.org/webgl/public-mailing-list/archives/1211/msg00132.html>
>> for more info.
>>
>> As for predefined variables, they constitute a shading pipeline
>> interface just like non-CSS FX shaders have interfaces. Everything
>> that CSS FX mandates for shaders is emulable with WebGLSL.
>>
>>>> How can accurate media types be a burden? The server is free to ignore
>>>> them. The browser should only send what it can understand in a given
>>>> context.
>>>
>>> This depends how you request the document and how flexible the client is on the MIME type send by the server. With our proposal the user does not need to worry about it, since he defines the vertex and fragment shader on his own. This is already the case with the syntax in the working draft of Filter Effects.
>>
>> There will be a type interpretation task for the client under all
>> circumstances. This does not affect the client's request. Simple
>> servers may simply read a file and push it down with an
>> extension-based or default media type. Nothing I suggest prevents or
>> spoils this behavior.
>>
>> In most cases, for any proposal, the client will not have to worry
>> about it. In the cases where it does matter (translations of a single
>> shader into multiple languages, translations of a package into
>> multiple formats), your proposal will still have to address this
>> issue. A request for "application/xml" is insufficient if the response
>> is expected to contain tags with specific semantics or encapsulate
>> shaders in a specific language. A request for "application/xml,
>> application/webglsl" is appropriate if the response may be interpreted
>> using generic XML features like @xml:id to yield WebGLSL source.
>>
>>>>>>> The 'src' descriptor is very generic and allows "packaging" of a feature. The feature is identified by the format passed as a string. This makes Filter Effects "extensible" by other specifications. We don't need to adapt Filter Effects to extend the support. Furthermore, we just have one HTTP request instead of multiple requests as before, which is a great benefit.
>>>>>>
>>>>>> A pipeline package/effect concept would definitely be useful in the
>>>>>> future but I am concerned about this direction for a number of
>>>>>> reasons:
>>>>>>
>>>>>> 1. A new mandatory, bespoke package format
>>>>>> 2. A new XML vocabulary that deserves careful design to support more
>>>>>> than just this use case
>>>>>> 3. Mixing and matching vertex and fragment effects becomes harder
>>>>>>
>>>>>> Can I package only a vertex effect and use a default fragment shader?
>>>>>> Custom fragment with default vertex?
>>>>>>
>>>>>> As an extension to custom CSS shaders, browser vendors may certainly
>>>>>> wish to implement package format support which may be indicated by the
>>>>>> appropriate Accept header.
>>>>>
>>>>> Ideally the package format should be determined by the string passed to the 'format()' function.
>>>>
>>>> Why is a package format required? Why would the proposed XML
>>>> vocabulary be referenced by "glsl"? The platform already supports
>>>> means to request equivalent representations and extract fragments
>>>> thereof.
>>>
>>> I speak about a package format in a general manner. An XML file does not need to be a package of course. But a package could still be an option. Future formats can define other ways to define the feature. For "glsl" it could be XML. Again, XML is just a suggestion, since browsers already support this format natively and no extra parsing instruction is required.
>>
>> I don't see what "glsl" has to do with XML. What extra parsing was
>> required in the previous proposal?
>>
>> Definition and implementation of the specific XML vocabulary is still
>> required. CSS @filter rules syntax is still required. I propose the
>> use of fragment identifiers which removes the requirement to implement
>> a specific XML vocabulary and removes the mandate that authors use a
>> specific XML vocabulary even when it is inappropriate to their use
>> case.
>>
>> For packaging purposes, CSS @-rule + @import can package effects
>> without duplicating semantics between CSS and XML.
>>
>>>> Here are some examples to illustrate my proposed generic approach to
>>>> resource reference:
>>>>
>>>> <html>
>>>> <head>
>>>> <script type="application/webglsl" id="my-vs">
>>>> // some VS code
>>>> </script>
>>>> <script type="application/webglsl" id="my-fs">
>>>> // some FS code
>>>> </script>
>>>> <script type="application/webglsl" id="other-fs">
>>>> // FS code
>>>> </script>
>>>> <style>
>>>> // @filter referencing url("#my-vs") and url("#my-fs") in vertex: and
>>>> fragment: rules for type("webglsl")
>>>> </style>
>>>> </head>
>>>> ...
>>>> </html>
>>>>
>>>
>>> I am not objecting to this ideas either and happy to discuss them as well. But I am skeptical because it does not cleanly separate between style and content in the document. Even if a lot of WebGL content already uses this way, it is not cleanly defined to it this way.
>>
>> My proposal allows authors to use ANY of these approaches (except
>> maybe the WebGLSL macro approach which requires fragment microsyntax
>> definition).
>>
>> This use case is a result of reuse of existing platform features. Of
>> course using in-line shader is not the best way to design large
>> applications but neither is <style> or @style and still they are
>> extremely useful under certain circumstances (e.g. dynamic insertion).
>>
>> Could you please elaborate on how the definition of the functionality
>> is not clean?
>>
>>>> OR
>>>>
>>>> serve a resource called "shader-pkg":
>>>>
>>>> <my-app-specific-package-vocab>
>>>> <vertex xml:id="vs1">
>>>> // vs
>>>> </vertex>
>>>> <vertex xml:id="vs2">
>>>> // vs
>>>> </vertex>
>>>> <fragment xml:id="fs1">
>>>> // fs
>>>> </fragment>
>>>> <fragment xml:id="fs2">
>>>> // fs
>>>> </fragment>
>>>> <other-tag />
>>>> </my-app-specific-package-vocab>
>>>>
>>>> with CSS referencing "shader-pkg#vs1" or "shader-pkg#fs2" to select
>>>> which shader.
>>>
>>> Would that expose a separate call for fragment and vertex shader in the CSS again?
>>
>> Mesh and pixel transforms are concepts that Filter Effects must
>> address. These concepts have direct analogs in every dialect of GLSL,
>> as well.
>>
>>> That is something that we tried to avoid with the last syntax, so that Filter Effects and different "extensions" (like GLSL shaders) can be updated independent of each other. This would make Filter Effects more future proof for changes in GLSL as well, since a new format would be used to differ between fundamental changes in GLSL. Who know if GLSL gets a geometry shader in the future as well? If so, the format can be updated and new string for the format() function would be introduced by the new format.
>>
>> This is a great reason to not call this language "glsl" as some
>> dialects of GLSL already have geometry shaders. I love the idea of a
>> language type parameter in CSS. I think it really frees Filter Effects
>> from a lot of lower-level details. I see this language type parameter
>> as analogous to the language's media type. Is it not?
>>
>> I don't see any problem here with directly specifying the mesh and
>> pixel programs. If geometry shaders are introduced, a new "format()"
>> may still be minted which allows "geometry:" in the @-rule. The @-rule
>> is the package. The @-rule has an IDENT for the name of the applicable
>> effect.
>>
>> If, in future, the browser understands a fragment identifier syntax
>> for "application/foo+json", this type may be used as a package format
>> and requested in a CSS FX context without changing Filter Effects or
>> defining a new "format()".
>>
>>>> OR
>>>>
>>>> serve a resource called "combined-shader":
>>>>
>>>> #ifdef VS
>>>> // some VS code
>>>> #elif defined(FS)
>>>> // some FS code
>>>> #endif
>>>>
>>>> with references to "combined-shader#?D=VS" and "combined-shader#?D=FS"
>>>> for vertex shader and fragment shader respectively at the author's
>>>> discretion for their particular needs. More implicit approaches are
>>>> also possible, of course ("combined-shader#VS" might mean the macro
>>>> "VS" should be defined if no name "VS" exists in the source), but
>>>> begin to suffer from lack of flexibility and namespace collisions.
>>>
>>> I am not sure if we want to expose "compiler flags" here. This seems not to be very intuitive. Do you see it as an alternative for serving an XML file (or other web based format)?
>>
>> Not every "compiler flag" would be exposed in this fragment
>> microsyntax. What do you find unintuitive about specifying a portion
>> of a shader constructed after preprocessor partial evaluation?
>>
>> Fragment macros are an alternative to serving many nearly identical
>> shaders at significant and unnecessary bandwidth cost. Because macros
>> act at the syntactic level, they change the semantic interpretation of
>> the shader resource and it makes sense to address them as different
>> aspects, with different meanings, of the same identified resource.
>>
>> The WebGLSL preprocessor is a fundamental part of the shading language
>> and should always be supported by a consuming specification at some
>> level.
>>
>> An alternative is for the FXTF to define macro definition facilities
>> along-side the uniform/attribute facilities. I find this solution to
>> be worse for several reasons:
>>
>> 1. Macros may change the pipeline stage type of a single shader
>> resource (VS <-> FS as above).
>> 2. Macros must be applied prior to uniforms/attributes as they may
>> change the shader program's interface fields and types
>> 3. Macros can contain WebGLSL strings
>>
>> For these reasons, I believe macro definition should be done as early
>> as possible (i.e. in the reference to a subresource of a shader).
>>
>>>> Support for this fragment identifier macro definition syntax is more
>>>> involved than the HTML/XML fragment support but has a number of useful
>>>> properties for shader customization (exactly the purpose of the
>>>> preprocessor) without sending nearly identical shaders or using
>>>> inefficient run-time conditionals in shaders.
>>>>
>>>> Each of these options is specifiable and implementable separate from
>>>> the others and still allows loading each shader from a different URL
>>>> by just supplying that URL. No intermediary XML manifest format is
>>>> required (though, one may be easily supported with the xml:id method).
>>>>
>>>> This approach additionally supports things like serving COLLADA files
>>>> as shader package resources with embedded shaders identified by
>>>> @xml:id.
>>>
>>> The idea is to make the @filter rule generic enough, so that it could be extended with techniques like COLLADA (even if I did not thought about COLLADA).
>>
>> Adobe's present proposal does not achieve this genericity by requiring
>> a bespoke effect format. An explicit extension for COLLADA is heavier
>> than necessary.
>>
>> By using XML's generic @xml:id and fragment identifiers, both Adobe's
>> effect package format as well as COLLADA (and head/script shaders and
>> custom XML vocab shaders and pure GLSL shaders) may be supported with
>> a single "format()" definition and optional mesh/pixel processor URIs.
>>
>>>>> An XML file seems to be easier to support by browsers, since browsers already have a way to parse XML documents. This would avoid defining a parser for the description document. But we would indeed meet to define a schema for the XML file.
>>>>
>>>> I am proposing to reuse the browser's CSS parsing for shader
>>>> composition, the browser's session cache for request deduplication,
>>>> the browser's HTML parser for embedding WebGLSL source, and the
>>>> browser's generic XML parser for selecting XML document fragments that
>>>> correspond to shader source. Implementation of a shader source macro
>>>> fragment query syntax would reuse the browser's shader analysis system
>>>> to insert name-value pairs into the shader's macro namespace. This
>>>> would allow functional specialization of generic shaders at load time,
>>>> something not presently possible with only uniform/attribute
>>>> parameters. This also requires coordination with Khronos in their
>>>> media type registration work to mandate macro fragment query syntax
>>>> support in resource resolution.
>>>
>>> I would like to limit exposing features as much as possible. Obviously shaders in CSS have needs that need to be addressed differently then for WebGL as example. In general, our positions seem not to be to far away, just the how differs.
>>
>> What exposed features are you referring to here?
>>
>>>> A new, mandatory package format appears to reduce author freedom,
>>>> increase vocabulary implementation count in browsers, and require
>>>> redefinition of concepts already present in the platform.
>>>
>>> No, we do not redefine any concept already used by browsers and reuse all existing concepts. We are using established technologies and formatting, like @font-face, shaders and shader parsing from WebGL and possibly XML.
>>
>> @font-face src refers to a monolithic non-text format with a
>> well-defined interface and limited end-user re-application or
>> consumption. Shaders are variable modules of textual source code
>> without a fixed interface that are easily read, modified, and
>> re-appropriated by end-users.
>>
>> Adobe's proposed format redefines default parameters and effect
>> packaging that are already required in CSS.
>>
>> If this format is going to support manifests with multiple alternate
>> representations, Adobe's format will redefine content negotiation.
>>
>> If this format is going to support selecting only a fragment shader or
>> only a vertex shader or a VS/FS pair that's not already paired,
>> Adobe's format will redefine resource fragment naming and a syntax
>> will need to be devised for CSS to address these subresources.
>>
>> If Adobe's format does not support mix-n-match due to coupling vertex
>> and fragment programs, duplicate data will be transmitted.
>>
>> What use cases does Adobe's proposed format support that are not
>> supported by URI references?
>>
>>> There is nothing that is not supported by at least an alpha version of three different browsers.
>>
>> Is the proposed XML vocabulary currently supported? I believe all the
>> semantics of CSS properties and generic URI references are already
>> standard.
>>
>>>>> At this point we didn't thought about the details of the file format, since we would like to agree on the CSS syntax first.
>>>>
>>>> My point is that the details of the file format do not matter if you
>>>> totally decouple it from the CSS syntax through the use of URLs with
>>>> fragment identifiers.
>>>
>>> I was not talking about fragments in URLs so far. You introduced fragments in your concepts. Fragments are valid as well, but don't need to matter here.
>>
>> Why don't they need to matter? They appear to solve the network
>> request consolidation problem as well as the semantic grouping
>> problem. Their use heavily reuses concepts and implementations in the
>> rest of the platform.
>>
>>>> The format of the manifest file can be directly represented in the CSS
>>>> syntax as with previous proposals. Instead of two places you can
>>>> specify parameters to override (XML and CSS), you only have one (CSS
>>>> at-rule). The proposed XML format will only be used by CSS unless you
>>>> are wanting to specify it for wider use among non-CSS/WebGL shader
>>>> programmers. If so, the specification task is much larger and has a
>>>> number of new constituencies. That work appears to be out of scope of
>>>> this TF and I don't see what necessary use cases it would address than
>>>> reusing the existing fragid system doesn't.
>>>
>>> The idea was to decouple the CSS parsing from the current version of GLSL or WebGL. We don't know how these technologies are evolving in the future, while CSS should be downwards compatible. Introducing keywords that might be outdated in a couple of years for instance is suboptimal.
>>
>> What keywords will be outdated? "vertex"? "fragment"? The Filter
>> Effects spec must already contain analogous concepts to define mesh
>> filters and pixel filters. In what way do you foresee WebGLSL evolving
>> enough but staying the same enough to cause a problem? How would this
>> evolution not be addressed with a new "format()", a new media type or
>> media type parameter, or a built-in shader compatibility wrapper?
>>
>>>> Instead of making a new vocabulary for version 1, why not reuse the
>>>> existing languages we already have: HTML, generic XML, CSS, WebGLSL?
>>>
>>> That is what our proposal did.
>>
>> I'm sorry if I misunderstood your proposal. Does your proposal define
>> semantics for new XML tags? Does your proposal use only features that
>> generic XML processors understand? How do I refer to a specific
>> fragment shader inside of an Adobe CSS Effect?
>>
>>>> If you'd like to transport metadata for shaders, why not propose new
>>>> #pragma directives to declare default parameters, copyright licenses,
>>>> etc so the entire WebGLSL and ESSL ecosystem can benefit regardless of
>>>> their XML manifest buy-in.
>>>
>>> I am not stopping the evolution of WegGL or GLSL, and the ecosystem of these specifications is not in danger either.
>>
>> I am not asserting either your interference or global damage. However,
>> I am asserting that Adobe's present proposal incorporates features in
>> a wrapper format that could profitably be included directly in shaders
>> with a little cooperation. The incorporation of these features would
>> benefit everyone and not just those who choose to use this specific
>> XML vocabulary. Rather than having each package format redefine these
>> metadata properties, we could specify a way to embed metadata directly
>> into programs.
>>
>> Consider the case of copyright/license. This information is directly
>> related to the actual program expression. Why should it only be
>> included in one XML wrapper format? A simple xpath expression would
>> strip this information from the program code. In fact, I believe that
>> in-page submission of open source licensed shaders to a WebGLSL
>> processor without a license declaration inline (reaching the WebGL
>> API) constitutes source redistribution without copyright/license
>> notice. Use of the proposed XML format with license ONLY in the
>> manifest would then uniformly violate the license for every manifest
>> user.
>>
>> This is one reason why copyright declarations and licenses are often
>> placed in source code header comments.
>>
>> If you would like to solve this problem, I would love to collaborate
>> with you to define a way to embed programmatically actionable metadata
>> directly into shader code. I have been studying related problems quite
>> intensely for over a year now.
>>
>>>> Using CSS for the effect composition format gets you:
>>>>
>>>> 1. built-in modularity, cascading, grouping just like any other CSS stylesheet
>>>
>>> This is the case with all proposals.
>>
>> The present XML vocabulary proposal duplicates grouping as it is
>> already present in CSS.
>>
>>>> 2. familiar syntax for designers without requiring XML (that also
>>>> works with their CSS preprocessing tools)
>>>
>>> If we can decouple it from the current state of WebGL and GLSL, then I am fine with it.
>>
>> What aspects of WebGL and GLSL, that you consider to be part of the
>> current state, might leak?
>>
>>>> 3. a single name to refer to an effect (the IDENT for the @-rule)
>>>> instead of both the @-rule IDENT as well as the shader package URL
>>>
>>> You need to have the possibility of fallback formats, I don't see this with just providing an IDENT, without a URL.
>>
>> Yes, you absolutely need a URL. I am concerned that the @-rule and the
>> shader package URL effectively identify the same thing. There exists a
>> trivial @-rule that simply embeds the shader package into CSS. Why do
>> I need both the XML and the trivial injection?
>>
>>>> 4. trivial generation from XML with XSLT if you desire an XML source format
>>>
>>> Indeed.
>>>
>>>> 5. no new format semantics questions or hastily designed vocabularies
>>>> that only solve 50% of the problem and have slow uptake
>>>
>>> This won't happen if we have a simple and clean definition of the format.
>>
>> Buy-in for a new XML format in addition to a CSS module is by
>> definition equal to or less than buy-in for simply a new CSS module.
>>
>> The format will have to handle naming, mix-n-match, and multiple
>> content types. How each of these are supported will require decisions
>> on the semantics of the XML vocabulary. These issues have already been
>> resolved for the platform as a whole in the form of URI references,
>> CSS properties, and media types.
>>
>> Some people will want to use different root elements and transmit
>> non-shader data alongside their shaders (e.g. COLLADA).
>>
>>>> 6. fewer files, less duplication
>>>
>>> Ideally yes. This is already a benefit to the new concept in comparison to the old one.
>>
>> New:
>>
>> 1 XML file, 1 CSS file
>> XML vocabulary can define default parameters, CSS can override default
>> parameters
>> XML vocabulary can pair VS/FS, CSS can't pair VS/FS?
>>
>> Old:
>>
>> 1 shader file (unless embedded), 1 CSS file
>> CSS defines default parameters
>> CSS can pair VS/FS (and only reference 1 shader file using a generic
>> fragment id)
>>
>>>> At one time, flexibility was also a goal of the CSS WG by allowing
>>>> cascading styles at various levels of a page. Only use #1 bundles
>>>> shader source into the document and should probably only be used with
>>>> inline styles. This would enable 1 network request for the entire
>>>> HTML+CSS+WebGLSL which could then stand alone. JavaScript may also be
>>>> used inline. Without fragment identifier support, I do not see how to
>>>> supply a single document which uses CSS effects (inline CSS + manifest
>>>> fetch for every VS/FS pair unless you use data blocks but then why
>>>> mandate the XML vocab?).
>>>
>>> If you worry about this, this is valid as well:
>>>
>>> @filter curl {
>>>        src: url(data:application/xml;base64,iVBORw…) format("glsl");
>>> }
>>>
>>> Which could be done with a minimizer.
>>
>> Now I'm transmitting base64-encoded text which is:
>> A. Unreadable
>> B. Splattered inside my CSS
>> C. Larger than necessary
>>
>> Proposed steps to load a custom dynamic filter:
>> 1. Unpack application format
>> 2. Wrap shaders with XML
>> 3. Base64 encode XML
>> 4. Build @filter
>> ---
>> 5. Browser Base64 decodes XML
>> 6. Browser parses XML
>> 7. Browser parses SL
>>
>> Rather than:
>> 1. Unpack application format
>> 2. Base64 encode shaders
>> 3. Build @filter
>> ---
>> 4. Browser Base64 decodes shader
>> 5. Browser parses shader
>>
>> Or:
>> 1. Unpack application format
>> 2. Insert shaders into DOM with @id
>> 3. Build @filter
>> ---
>> 4. Browser parses shader
>>
>> What does wrapping with XML win the developer?
>>
>>>> Uses #2, #3, and #4 decouple style from structure and do so in a way
>>>> that gives authors maximum flexibility to decide how they wish to
>>>> serve their shaders (everything from a single XML bundle with 50
>>>> shaders and a custom vocab to a single GLSL file with preprocessor
>>>> switches to a future manifest or module format).
>>>
>>> Ok, I got your point. Now I understand your fragment request, in this case we just define multiple shaders in the XML file:
>>>
>>> <shader id="curl">
>>>        <fragment/>
>>>        <vertex/>
>>> </shader>
>>> <shader id="wave">
>>>        <fragment/>
>>>        <vertex/>
>>> </shader>
>>>
>>> This sounds reasonable.
>>
>> What is the root element? What if I only want to use a fragment with
>> default vertex or vertex with default fragment? What does it mean to
>> refer to an XML package resource which uses @id without a fragid
>> specified?
>>
>> What does using this specific vocabulary allow that using any XML
>> vocabulary with @xml:id does not?
>>
>> After all of that, I would like to reiterate that effect formats are
>> immensely useful and I'm sure the community would appreciate a common
>> effect format. However, I don't think they should be mandated or
>> favored and I don't see how they fall under CSS WG's mission.
>>
>> Warm regards,
>>
>> David
>

Received on Monday, 10 December 2012 11:26:41 UTC