Re: Shader Language Form Discussion

With dev tools open I actually get much less variance in my results (maybe
performance.now behaves differently).

Now I'm getting 32-37 ms for "void main() {}" and 46-49 ms to compile the
big shadertoy from earlier.

On Wed, Oct 25, 2017 at 4:13 PM Kai Ninomiya <kainino@google.com> wrote:

> On Wed, Oct 25, 2017 at 11:39 AM Maciej Stachowiak <mjs@apple.com> wrote:
>
>>
>> On Oct 25, 2017, at 10:49 AM, Corentin Wallez <cwallez@google.com> wrote:
>>
>> Thanks Kai for putting this together, a couple more trade-offs would be
>> the following:
>>
>>    - Interoperability. It might be more difficult to have to have
>>    multiple implementations be interoperable with a SL.
>>
>> I know you said "it might", but I'm not sure this is true. There are many
>> human-readable languages normally have high interoperability, arguably
>> higher than SPIR-V does. The main factor affecting interop is precision of
>> the spec, not whether the formal is text or binary.
>>
>>
>>    - Browser engineering. How much do the compiler add to the browser
>>    attack surface and binary size.
>>
>> Note that whether we use an "IR" or "SL" format, we need to include in
>> this complexity all the required validation and translation logic. I
>> mention this because human-readable text formats are often easier to
>> validate than SSA-level formats, since the parser proves many of the
>> desired well-formedness constraints automatically.
>>
>>
>>    - Governance. How do we make changes to the SL / IR, who's involved.
>>    - Portability to MSL, HLSL and SPIR-V. This so important that we
>>    forgot it ^^.
>>
>>
>> On Tue, Oct 24, 2017 at 2:21 AM, Kai Ninomiya <kainino@google.com> wrote:
>>
>>> Hey all,
>>>
>>> We said we would resume the shader language "levels" discussion via
>>> email, so here we are. (Apologies that it's so late.) I'll try to start
>>> this off with some concrete ideas before we dive into the rabbit hole.
>>> Please add suggestions as necessary. We'll be talking about shading
>>> languages again this Wednesday, so I'd ask everyone to spend some time to
>>> think about the topic before then.
>>>
>>> The forms of language we've discussed:
>>>
>>> * "SL". High-level, human-authored, highly-structured, imperative,
>>> whatever-you-want-to-call-it language, like WSL, HLSL, or GLSL.
>>> * "IR". Machine-generated, typically-SSA, lower-level,
>>> canonically-binary representation, like SPIR-V or DXIL.
>>>
>>> The alternatives we've discussed:
>>>
>>> * WebGPU ingests SL only.
>>> * WebGPU ingests both SL and IR. We bless an offline SL->IR tool.
>>> * WebGPU ingests IR only, but we have a Web API to compile unsecured SL
>>> to unsecured IR. We bless an offline SL->IR tool.
>>> * WebGPU ingests IR only. We bless both online and offline SL->IR tools
>>> - an online compiler is required for some applications.
>>>
>>
>> It seems like the second and third options are nearly equivalent in
>> capability. The third makes it less convenient and potentially less
>> efficient to use the SL form. But both expose the additional
>> interoperability considerations and security attack surface inherent in
>> consuming two different formats to about the same degree.
>>
>> [1] A shader compiler may, for example, be a WebAssembly module on a CDN.
>>>
>>> In any alternative, we seem to agree to bless an SL. We bless/provide
>>> tools for that SL, and documentation is written using that SL. The browser
>>> compiles its ingested language to a secured shader in the native API's
>>> ingested language.
>>>
>>> Some trade-offs we've discussed:
>>>
>>> * Compilation performance. SLs may be more expensive to compile. IRs may
>>> be expensive to compile securely - we're working on this one.
>>> * Runtime performance. Could a practical SL be more "performantly
>>> securable" than an IR?
>>> * View Source. Vocal web developers have complained about the View
>>> Source story for WebAssembly. (And how well could we mitigate this if
>>> needed?)
>>>
>> Apologies for not doing this before our meeting today, but here are some
> shaders compiled from GLSL to SPIRV, optimized, and decompiled. I hope
> these help everyone understand the level of abstraction that is provided by
> SPIR-V (and as a bonus, the quality of a potential "decompiling View
> Source" dev tool). They are quite educational for me as well.
>
> An example of a small but not-quite-trivial shader we used in the very
> first NXT demo. I inserted extra line breaks to make the rows line up.
>
> https://cdn.rawgit.com/kainino0x/7df254f5d3a2343fd1cab5f9c09e3354/raw/fbcb4815877c6344d587106879be6b10aa10155c/boids.comp.opt.spv.html
>
> Here is a much larger example, iq's famous "Raymarching - Primitives"
> resource for the ShaderToy community. (I grabbed it from Dev Tools by
> breaking at gl.shaderSource, and modified it slightly to compile with
> shaderc.) The code got reordered, so I didn't pretty it up.
>
> https://cdn.rawgit.com/kainino0x/2504dc196b04c8db6487b2d9050329fd/raw/483e7ae1d8383aa8c61ac751fbe45c0f5931e5f4/shadertoy-Xds3zN.opt.html
>
> * Compiler maintenance. SL compilers may be more prone to bugs. (Critical
>>> for any compiler/translator that must enforce security boundaries.)
>>> * Download/initialization time. Probably not a significant issue for
>>> either format after compression, but we also have to consider the download
>>> and compile time of, e.g., runtime shader compiler modules.
>>> * Ecosystem bootstrapping. There a lot of HLSL and GLSL out there, in
>>> the form of libraries, sample code, and existing engines (including WebGL).
>>> * Development time for the ingested format. Probably not an issue if we
>>> start out with HLSL or GLSL.
>>> * Soundness/provability. We differ on whether this is important at all,
>>> and whether it's easier for SL or IR. Probably not worth discussing right
>>> now?
>>>
>>> Please add topics to this list. I have one more:
>>>
>>> * Code reuse. Some engines (like Three.js) manipulate shader strings to
>>> generate shaders at runtime; if we only ingest IR, they would have to
>>> either use a new method or add a shader compiler. On the other hand, some
>>> engines may(?) prefer to use concepts like SPIRV's specialization and
>>> linking.
>>>
>>> Relevant notes for review:
>>>
>>> 2017-09-22 (F2F):
>>> https://docs.google.com/document/d/1VridLAmC05h80_d-FGmwyI7On0_AY5y8pVGI-TT4ysQ/edit
>>> 2017-10-11:
>>> https://docs.google.com/document/d/1-ciiWbGletoOXOGrBBZpOhhWEWJcIqK0Bb-dtHJ8ffE/edit
>>>
>>> -Kai
>>>
>>
>>