Minutes for the 2017-08-09 meeting

GPU Web 2017-08-09

Chair: Corentin

Scribe: Corentin / Dean / Ken

Location: Google Hangout
Minutes from last meeting
<https://docs.google.com/document/d/166apsbaF4suHkbXC5MJC-OD8159mNQRaZoyadB2j0Q4>Tentative
agenda

   -

   Administrative stuff (if any)


   -

   Individual design and prototype status


   -

   Shading languages
   -

   Agenda for next meeting

Attendance

   -

   Dean Jackson (Apple)


   -

   Filip Pizlo (Apple)


   -

   JF Bastien (Apple)


   -

   Myles C. Maxfield (Apple)
   -

   Keith Miller (Apple)
   -

   Jon Lee (Apple)
   -

   Austin Eng (Google)
   -

   Corentin Wallez (Google)
   -

   David Neto (Google)
   -

   John Kessenich (Google)
   -

   Kai Ninomiya (Google)
   -

   Ken Russell (Google)
   -

   Lei Zhang (Google)
   -

   Zhenyao Mo (Google)
   -

   Chas Boyd (Microsoft)
   -

   Rafael Cintron (Microsoft)
   -

   Dzmitry Malyshau (Mozilla)
   -

   Jeff Gilbert (Mozilla)

Administrative items

   -

   CW: News from the lawyers, on our side they promised they’d come back to
   us but didn’t yet.
   -

      DJ: Not on our side either.

Individual design and prototype status

   -

   CW: (Google) We’ve added blend state. Using swap chain objects. Clears
   are next.
   -

   DM: (Mozilla) Nothing particular to report. Looking into resource copies
   and clears (issue on github)
   -

   RC: Nothing from Microsoft. Ben has been contributing to NXT - depth
   stencil state.
   -

   DJ: Nothing to update.
   -

   MM: I wanted to mention that on the github wiki page I started a roadmap
   page, and went through the minutes to look for things I remember that we
   vaguely decided on.
   -

   CW: Yes, we should go through the minutes officially and record the
   decisions there.

Shading languages

   -

   CW: let’s agree or disagree on the criteria to use
   -

      Some sort of constraint on security. We’re on the web and we don’t
      want compute shaders to access CPU ram, or some other web page.
      -

      Want something that can be translated to all of Metal/D3D12/Vulkan
      -

      Hopefully non-controversial: probably don’t want a high-level
      language but instead an intermediate one
      -

         DJ: Apple’s not completely sold on that at the moment
         -

         It’s a trend in one part of the industry but not everywhere --
         using IRs rather than a language.
         -

         CB: Microsoft also not completely sold on the idea of using an IR
         rather than high-level language
         -

      Has to work well with the rest of the API, in particular the binding
      model
      -

         DJ: agree with this -- the shading language had a big impact on
         Metal’s binding API. The two have a big impact on each other. Shading
         language has to evolve with the API, and vice versa. (Also MM
commenting)
         Can’t decide on one without the other.
         -

   JK: shall we discuss reasons for using an IL?
   -

      Obfuscation
      -

      Abstracting driver details and high-level languages from source format
      -

      Portability
      -

   CB: what tier of the stack are we talking about?
   -

      Here at the W3C we’re assuming we’re building a C++ API and then
      layering the web exposure on top of it
      -

      Maybe the lower layer uses an IR but the thing that’s exposed to the
      browser is a high-level representation of some kind
      -

   CW: Google being heavily invested in Vulkan, going to suggest SPIR-V as
   an intermediate language
   -

      Proven that it’s cross-compilable to HLSL and MSL
      -

      Open standard guided by a foundation not tied to one company in
      particular
      -

   MM: thought HLSL doesn’t have pointers, but SPIR-V does? How will they
   map?
   -

   JK: SPIR-V has a “logical” addressing mode where the pointers map 1:1 to
   objects + offsets
   -

      When OpenCL translates to SPIR-V it uses more C-like pointers, but
      when translating HLSL it uses the logical addressing mode
      -

   DJ: Concerned about SPIRV->High level language. Will SPIRV using
   pointers by translatable back to high-level.
   -

   JK: has a logical addressing mode. If the intent is that you be able to
   translate to high-level languages that don’t have pointers, you can do
   this. Pointers are basically handles the objects.
   -

      Obvious example: starting with HLSL or GLSL, you end up with SPIRV
      that uses logical addressing mode.
      -

      We could enforce that GPUWeb shaders use logical addressing mode.
      -

   MM: SPIRV spec says stuff about addressing capabilities, for example
   converting integers to pointers. Is this the same thing you’re talking
   about?
   -

   JK: if you’re in the logical addressing mode, you can’t do that
   conversion.
   -

   FP: if I wanted to write a function that accepts a pointer as argument,
   can you do that in this addressing mode?
   -

   JK: Yes you can, passing pointers to function is like passing a
   reference to the object. Using static analysis you can figure out which
   objects are passed to the function. It’s isomorphic.
   -

   CB: might end up with multiple security requirements depending on the
   part of the stack you’re talking about. Loading a shader’s similar to
   loading an x86 executable in your process. May need a restricted subset.
   -

   JK: Think SPIRV lot safer than x86 especially in logical mode. You know
   where the objects are and you aren’t dereferencing any pointer.
   -

   CB: do we have a set of customers like Adobe who would be satisfied with
   this logical addressing mode?
   -

   JK: not quite sure which Adobe workloads you mean
   -

   CB: Adobe wanted a baseline C++ layer underneath. They may have stuff
   that goes beyond the logical addressing mode’s capabilities
   -

   JK: it’s possible you’d need more.
   -

      Added a variable pointers extension to vulkan for SPIRV. It’s still
      pretty safe. You can select between objects when pointing to different
      ones, but you can still keep tabs on the necessary bounds checks.
      -

   CB: Is that in logical addressing mode too?
   -

   JK: Yes it exposes selects on pointers but doesn’t allow bit casts for
   example.
   -

   MM: compiling arbitrary C/C++ programs to the shading language should be
   a non-goal.
   -

   JK: Portability should also be on the list. For high-level languages, it
   implies not having a bunch of frontends each interpreting the language in a
   slightly different way.
   -

   MM: of apps that ship today using SPIRV, what’s the original language
   the shaders were written in?
   -

   CW: GLSL
   -

   JK: And more and more HLSL and OpenCL C as well.
   -

   MM: the language that the author writes their shader in should match the
   languages they use the API in. If we add / remove features from SPIRV,
   we’ll need to make it match more the language the API is used in.
   -

   JK: For Vulkan having multiple high level languages is a strength, no
   one complained about it.
   -

   MM: not trying to restrict to using a particular high level language.
   but if we’re using a set of features from an IR, that set of features has
   to be applied to the high-level language the author would be writing in.
   -

   CB: would need to be some reference implementation of translation from
   HL to IR. So Javascript.
   -

   JK: People would use GLSL
   -

   CB: Web people don’t know GLSL
   -

   KR: Except WebGL people, it would have been super hard to make a new HLL
   for WebGL.
   -

   JG: We don’t want the API to be usable by “just any” Web developers.
   -

   MM: Javascript is a problem because one instruction becomes many
   instructions. But we still want to allow Web developers to pick up the
   language easily (very behind not sure if correct notes)
   -

   KR: Take the experience learned from WebGL. Significant translations
   from WebGL “GLSL ES” to different backends (HLSL, GLSL, GLSL ES). Needed to
   change things to avoid corner cases badly supported by backends.
   -

   DJ: Aren’t we going to have the same issue with SPIRV?
   -

   JK: It is much more narrowly defined.
   -

   JG: We are going to have a translation layer, the goal is to make it as
   easy as possible to make that translator. Example: we use SPIRV as IR, if
   you want to use a HL, then you can use GLSL compile it, then use SPIRV at
   the API level.
   -

   KN: Also can compile shaderc in WebAssembly.
   -

   KR: SPIRV is the input language, anything else is a library. Having a
   tightly defined IR is a create idea seen the crazy bugs we’ve had in the
   GLSL and HLSL frontends for WebGL.
   -

   JG: There is a SPIRV reference validator that exists
   -

   DN: yes, there’s a validator and we maintain it. Contributions welcome.
   -

   FP: If you take SPIRV and use logical addressing mode, which language
   wouldn’t be able to target SPIRV.
   -

   JK: HLSL and GLSL do target this mode and do work. With variable
   pointers, opencl C can use that mode.
   -

   FP: would be worthwhile to understand in Adobe’s use case whether their
   use of pointers would work. Also, conflating two things: how high level is
   the representation, and how tightly defined it is. You can define a tightly
   defined textual human-readable representation of code.
   -

   CB: we’ve been talking about stacks used by the web developer. Would be
   good to have some alignment at that level so they can share assets and
   skill bases. Important to have a reference implementation so the ecosystem
   can have portability.
   -

   FP: Agreed, that’s true of any good IR effort, for WASM we have
   reference implementations independent of all browsers. Should probably be
   part of what this group does.
   -

   JK: one aspect of a high-level language that it’s flexible. Declare a
   variable in GLSL: there are tons of defaults and simplifications. This is
   where variability in frontends come in (what the defaults are, for example).
   -

   JG: perhaps we are conflating low-level and high-level with
   human-readable and machine-readable.
   -

   CB: The choice we need to make is at what level of the stack do we want
   consistency, and where do we want to allow for innovation?
   -

      JK: agree
      -

   DJ: is the assertion that we could use SPIRV unmodified?
   -

   JK: SPIRV is a single IR serving 3 APIs. There is a core SPIRV spec, and
   the client APIs specify an execution environment with builtin variable, a
   binding model, ...
   -

      Canonical built-in variables
      -

      OpenGL doesn’t have descriptor sets but Vulkan does
      -

      Arrays of objects
      -

      There’s a separate spec about how a particular client API interfaces
      with SPIRV
      -

      Vulkan defines rules about defining locations for outputs and inputs
      (and how they link up)
      -

      For any new client API we’d expect to write this spec. Could likely
      be written without any changes to SPIRV. But if something could
be added to
      SPIRV to make things easier on a client it can be added.
      -

   JG: there’s definite value in taking something existing and using it,
   than creating something new.
   -

   CB: Brings up the industry trend on basing things on LLVM IR. Apple and
   Microsoft base things more on LLVM IR than SPIRV does. Should we make
   something based on a recent LLVM IR?
   -

   DN: Khronos went down that path with OpenCL 4-5 years ago. LLVM 3.2,
   then 3.6. Fantastic tooling, but is on a fork which gets frozen. LLVM
   evolves quickly, as it should. Then have a huge pile of tooling that
   everyone has to carry around.
   -

      Everyone working on tooling wants to work with top-of-tree LLVM.
      -

      Causes a fair amount of difficulty with the multiple versions of LLVM.
      -

      That’s why Khronos went with an independently-specified spec that’s
      split from LLVM.
      -

      Lose built-in tooling but avoid a whole lot of pain.
      -

   CB: should we potentially reset the clock and use a recent version of
   LLVM?
   -

   JG: We don’t have three years.
   -

   JFB: want to echo what David says. The experience with PNaCl was exactly
   the same. No advantage to using LLVM’s IR.
   -

   JK: LLVM ir a snapshot of an SDK instead of a specified IR.
   -

   MM: our discussion has been about two IRs. But there are multiple
   options. One would be to use WebAssembly. Or DXIL. Or go high-level, HLSL,
   GLSL, C++.
   -

   KR: Should take into consideration whether we should have pointer or
   pairs of object, offsets in the shader. We are going to rely on the GPU MMU
   for some robustness but believe the security implications remove the
   possibility of raw pointers.
   -

   CB: probably going to have to run some validator on these incoming
   shaders and restrict it to some set of intrinsics, similar to SPIRV’s
   logical addressing mode.
   -

   MM: On at least Metal, there is a way to have context have their own
   address space so they can’t touch library they don’t own.
   -

      These are per-process, not per-library.
      -

      The WebGPU / Metal device has the same page tables on the GPU as the
      Core Animation compositing engine.
      -

      Either: make a bazillion different processes, or validate the shaders
      and make sure statically that they don’t reference stuff they
shouldn’t be
      able to.
      -

   KR: Ensuring things statically for compute shaders will be much harder
   than for VS/FS like in WebGL. Looking at WebCL they managed to make
   pointers checked but 10% perf cost and
   -

   CW: could make a Vulkan extension that has multiple logical address
   spaces, but it doesn’t exist yet.
   -

   JK: There’s the robustness extension that OpenGL and Vulkan have to some
   extent. If you add robustness on top of GLSL / SPIRV think you are in good
   shape. Compute with fanky pointer stuff would be more complicated.
   -

   MM: should this new API only run on hardware with the robustness
   extension?
   -

   JG: Would phrase it as a requirement on safety instead of requiring an
   extension.
   -

   MM: but if you could do static analysis of the shader and prove that
   it’s safe?
   -

   JFB: Three criteria
   -

      Security, which is non-negotiable
      -

      Expressiveness of the source language, supporting pointers like Ken
      mentioned.
      -

      Performance. On some hardware might have extensions to give you the
      first two for free. Or some hardware might not support them. Trying to
      understand: MM asked whether we’re OK restricting WebGPU to the devices
      that don’t have these extensions. Is that OK? Ken quoted a 10%
perf hit for
      WebCL’s bounds checking. That sounds OK.
      -

   KM: At least for WebAssembly there is bounds-checking in some context
   where we don’t get the free virtual addressing. We don’t worry about it
   because it doesn’t happen that often. Legacy would eat the 10% cost.
   -

   CW: one major feature of compute shaders is unsized arrays. There’s no
   way to remove the runtime cost of the bounds checks in this case.
   -

   DJ: you couldn’t use logical addressing mode in this case anyway?
   -

   JK: You can, you know the what object it is and you can check you stay
   in it.
   -

   JFB: Can you change the size of the array during the execution of the
   shader? WebAssembly has free resizing.
   -

   KR: This works in WebAssembly because CPU but not GPU. CPU can do trap
   based range checking but GPU can’t. We can’t induce a recompile when you
   change which array the shader uses.
   -

   JK: Can give the size of the arrays as uniforms.
   -

   KR: Question is not just “are we ok with a 10% performance hit” first
   because we aren’t sure about this number. Unclear if SPIRV pointer
   extension would even allow runtime checks. Need more investigation, ask
   Adobe what kind of kernels they’d like to run. Possibility of doing static
   analysis. Thought about relying on static analysis in WebGL to prove that
   certain shaders were safe to run, but rejected relying on it because it was
   too big a constraint on the WebGL implementation -- depending on how smart
   the shader validator was, the shader might load, or might not.

Agenda for next meeting

Talk about  shading languages more.