Re: Notes about Secure HLSL

On Mon, Dec 11, 2017 at 2:34 PM, Myles C. Maxfield <mmaxfield@apple.com>
wrote:

> Replies inline.
>
> > On Dec 8, 2017, at 8:17 PM, Dzmitry Malyshau <dmalyshau@mozilla.com>
> wrote:
> >
> > Hi Myles,
> >
> > Thanks for publishing the Secure HLSL wiki page!
> > Here are my comments, split into sections:
> >
> > ## Compatibility
> >
> > > Secure HLSL aspires to be source-compatible with almost all existing
> HLSL shaders.
> >
> > Why do you want to support the case of having multiple main() functions
> for different shader stages?
>
> This is a pattern that we’ve seen used often in Metal Shading Language.
> It’s pretty common for a bunch of source code (like helper functions) to be
> shared between two different shader stages. This is even more important
> because of the lack of #include.
>
>
Sorry to butt in. I hope this comment is helpful.

I don't really understand the argument for #include or #include like
behavior. AFAICT the driver is just loading a bunch of files and
concatenating the strings. Why should the driver/api do that when you can
do it yourself?
If my point wasn't clear the lack of #include seems like it should have
zero influence on API design. Writing the 5-10 lines of JS or even C++ to
regex replace '#include "url"' would seems to show that #include issues are
really best left at the user level to deal with not the api/shader-language
level. I get that the user using a regex or string search for "#include" is
not a perfect solution (not actually parsing the shader) but I doubt edge
cases would affect 99% of programs. For those projects that want a perfect
pre-processor they can implement one.

In fact in JS replicating #include it's even easier

const helper1 = `
somecode
`;
const helper2 = `
some other code
`;

const shader1 = `
...
${helper1}
${helper2}
...
`;

const shader2 = `
...
${helper1}
${helper2}
...
`;

You can load helper1 and helper2 any way you choose. (XHR with promises,
other strings, script tags, pulled out of a zip file, etc...)



> >
> > > The #include preprocessor directive in HLSL is not present in Secure
> HLSL
> >
> > Most of the shader codebases (HLSL and whatnot) that I had a pleasure to
> work with used some sort of include mechanism to share common parts of
> code. Is your idea to require the existing HLSL adopters to have a
> pre-processor step and concatenate the code on their side first before
> passing to WebGPU?
>
> The specific behavior of #include doesn’t work because there is no direct
> filesystem access on the Web. The most natural way we thought of fixing
> this would be to allow placing multiple shader stages in a single file.
>
> You are right, though - modularity is good. We would love to come up with
> a more elegant way of making shader source code modular. Suggestions are
> welcome :)
>

See above. Not sure why the API / shader language should care about this.
That seems a user level detail. If you want modular shaders you write your
own shader generation library and make it as simple or complex as you want.
If you want something like #include it will take all of 5-10 mins to
implement in JS. If you want something more sophisticated you're free to do
that too.


>
> >
> > > The texture HLSL type is not present in Secure HLSL, nor is the
> sampler HLSL type present in Secure HLSL.
> >
> > I don't understand this part. It says neither texture or sampler types
> are present, yet it doesn't mention them in "The following HLSL types are
> not present in Secure HLSL" list.
>
> The idea here is that the old-style HLSL texture sampling is not present.
> Similarly, textures in Secure HLSL are typed (so you know when you sample
> if you’re getting a float4 or whatever). DX10-style texturing is present.
>
> I’ll update the document to include those two types in the list.
>
> >
> > ## Scope
> >
> > What I would expect from Secure HLSL is to restrict and specify the
> existing HLSL syntax, so that we can securely adopt it. However, the
> document lists a number of extra features that don't appear to be strictly
> necessary for securing HLSL:
> >
> > - generics
> > - pointers
> > - array references
> > - operator overloading
> >
> > These features are nice to have for a language in general, but they
> appear to be secondary (if not tangential) to the primary goal of the spec
> and WebGPU effort in general.
>
> Secure HLSL is an idea for a future evolution of HLSL, and includes
> features which are not present in HLSL.
>
> Two weeks ago, there was consensus to promote HLSL proper (not Secure
> HLSL) in WebGPU. We’re still trying to figure out what “HLSL proper” would
> look like in the context of WebGPU. We hope that we land on something which
> can be extended in the future to incorporate the ideas in this document.
>
> >
> > ## Efficiency
> >
> > > All variables live for the lifetime of the program
> >
> > Wouldn't this blow up the register occupancy of such programs? If the
> shader compiler is supposed to optimize this requirement out for variables
> not visible outside of their scope, then it appears to me that this would
> complicate the compiler and/or hinder compiling performance.
>
> The idea here is to disallow dangling pointers. By making variables have
> infinite lifetimes, it is now impossible to have a dangling pointer.
>
> The compiler is responsible for narrowing the lifetimes of variables such
> that register pressure is reduced. It does this by using inductive proofs
> to determine points after which the variable is impossible to access.
>
> This is an “as-if” situation, where code must act “as-if” all variables
> are static. However, if a variable never has its address taken, there’s no
> reason for its implementation to be static. Similarly, even if a pointer to
> it is created, if that pointer has the same lifetime as the variable, the
> variable’s implementation still doesn’t have to be static. In fact, the
> only occasion where the compiler would have to do the lifetime analysis of
> the pointer is when the pointer outlives the variable, such as when
> returning a pointer to a local. Given that pointers don’t exist in any
> existing HLSL code, and returning a pointer to a local is a super rare
> occurrence in any programming language, it is extremely unlikely for this
> to be a problem in real-world code.
>
> >
> > Couldn't you only require global lifetimes for the variables being
> pointed to?
> >
> > > If an argument to any built-in function is NaN
> >
> > Built-in functions are expected to run fast. I'm worried that putting
> checks in all the arguments would heavily penalize arithmetic-heavy
> programs.
>
> Yeah, we’ve reverted this because it is needlessly more restrictive than
> WebAssembly. I’ve since updated the document.
>
> >
> > > Denormalized floats must not exist
> >
> > Is that also the approach WebGL group has taken?
>
> I don’t think the CG has discussed this.
>
> We can change this if we find a popular GPU that requires denormalized
> numbers; however, we aren’t aware of any yet.
>
> >
> > ## Other
> >
> > > As such, Secure HLSL includes a "Logical Mode" which conforms to the
> requirements of these languages.
> >
> > There should be no need to avoid mentioning the elephant in the room by
> name ;)
>
> On the contrary, the language itself doesn’t need to have the same
> restrictions that logical mode has. Indeed, the language is naturally more
> powerful than Logical Mode, but it needs to have these restrictions in
> order to be compatible with Vulkan. We have to do additional work to make
> it compatible; so this additional work is behind a “Logical Mode” flag.
>
> It is our hope that, one day, we will no longer need Logical Mode.
>
> >
> > Thanks,
> > Dzmitry
>
>
>