Re: Ill-defined programs

Hey Maciej,

WebGL already has parts that follow (2.5) when accesses to arbitrary
offsets are possible. In WebGL 2 this is only limited to vertex buffers and
arrays in uniform buffers. In WebGPU with compute functionality, accesses
to arbitrary offset become more common with the "shader storage buffers"
and "texture read-write" features. When possible we would like to use
hardware features to secure these accesses as this will get us the most
performance. This is where (2.5) comes from because different HW implement
the "robust access" feature differently.

This interop problem hasn't been a problem because shaders that hits any of
the cases of (2.5) will usually produce very incorrect and noticeable
results.

Corentin

On Mon, Nov 13, 2017 at 6:03 PM, Maciej Stachowiak <mjs@apple.com> wrote:

>
> Hi Jeff,
>
> I think we might be making different assumptions about the amount of
> undefinedness that is being proposed:
>
> (1) Approximately as interoperable as WebGL.
>
> (2) if a shader program reads out of bounds or violates any other
> correctness constraint, then all bets are off.
>
> (2.5)  if a shader program reads out of bounds or violates any other
> correctness constraint,, then it might do any one of a fixed list of
> behaviors (e.g. terminate, clamp to bounds, or return constant zero).
>
>
> I'm reluctantly ok with (1). I'd hope we could do better, but it might be
> that this is as good as it gets. Real-World interop for WebGL is ok, even
> if it's not as good as some other older web technologies.
>
> My impression is that what Dzhmitry was advocating was more like (2): "We
> don't care about the computed results or performance of an ill-behaved
> application."
>
> My impression is that WebGL does not follow approach (2). The behavior for
> ill-defined applications is specified and covered by tests, and when there
> are behavior differences, it's due to things like GPUs not doing floating
> point arithmetic consistently. It's not due to completely different
> behavior for blown bounds checks.
>
> I don't think (2) or even (2.5) is ok. It seems likely to create a much
> bigger interop problem than we have with WebGL today. It's not ok to
> completely abandon interop for programs with a certain class of bugs. Most
> nontrivial programs will be buggy, and many will com to inadvertently
> depend on their bugs.
>
> Regards,
> Maciej
>
>
>
> > On Nov 13, 2017, at 4:37 PM, Jeff Gilbert <jgilbert@mozilla.com> wrote:
> >
> > We (Mozilla at least) definitely feel that WebGL 1 and 2 are in a
> > great place portability-wise, and this is also the feedback we get
> > from devs and partners we've worked with. WebGL has real
> > implementation differences, with real portability concerns, but these
> > have not become malignant. Interop is a very real concern, but there
> > is middle ground between wild-wild-west (which absolutely no one here
> > is proposing) and absolute portability for all apps. We have been very
> > successful in striking that middle ground with WebGL, and I don't see
> > that changing with WebGPU. Centralized test suites ensure all points
> > of portability we require, and prevent implementations from deviating
> > from the spec, and guiding them back into compliance.
> >
> > People write browser-specific code all the time, both on accident and
> > on purpose. There's no panacea for it. We should make smart choices
> > and properly weight the costs, as well as the benefits, of various
> > approaches. Flatly refusing to consider slight deviations is naive,
> > and not supported by our existing experience in this area.
> >
> > I'm trying to avoid rehashing years of API decisions here, and largely
> > decisions we're happy with. I understand your concerns, but they are
> > largely (and fortunately) not reflected in our experience with WebGL,
> > which is the closest thing to prior art that we have here.
> >
> > I do not think we disagree as much as you fear, but I simply cannot
> > agree with how hard you are pushing on excruciating portability. There
> > is a balance to strike, and we've been fairly successful at doing so.
> >
> > On Mon, Nov 13, 2017 at 4:02 PM, Maciej Stachowiak <mjs@apple.com>
> wrote:
> >>
> >>
> >>> On Nov 13, 2017, at 3:44 PM, Jeff Gilbert <jgilbert@mozilla.com>
> wrote:
> >>>
> >>> First off, those using these APIs (WebGL and WebGPU) are not writing
> >>> geocities websites. They are engineers, and we should optimize for
> >>> these engineers, not for the layman.
> >>
> >> The worst interop problems come from extremely popular sites created by
> professional engineers, not geocities sites made by laymen.
> >>
> >>>
> >>> In WebGL, we are aggressive about issuing descriptive errors and
> >>> warnings on malformed or questionable API use. We've found this
> >>> quickly surfaces portability issues, and makes them very quick to
> >>> solve. I anticipate continuing this behavior during implementation of
> >>> WebGPU. We have a ton of tools for giving feedback to devs who are
> >>> doing the wrong thing, particularly if we have a less-perf-sensitive
> >>> debug mode. (which can even be a JS library, not needing to be baked
> >>> into the browser)
> >>>
> >>> I think good validation is important for development, but its
> >>> performance impact can hurt outside development, once a product is
> >>> released. In our usage of APIs inside the browsers, outside of DEBUG
> >>> builds, we try to turn off as much validation code as we can, given
> >>> its performance impact. If we find it useful to do this, why would web
> >>> apps not find it useful to do similar, and be able to run with a
> >>> minimum (but not zero!) level of validation?
> >>
> >> They'll definitely want to do this. It's just a bad idea to let them,
> because they will accidentally code sites that only work in one browser, or
> only on one OS, or only one CPU architecture, or only one GPU. Some of this
> may be inevitable but we should try our hardest to limit it.
> >>
> >>>
> >>> In the six years I've been a part of WebGL, we have not seen the
> >>> divergence in behaviors that you are afraid of, so I'm not sure why
> >>> you think they will happen with this new API. Please appreciate our
> >>> experience with this field, and the differences it has as compared to
> >>> other areas.
> >>
> >> According to Ken, it seems like WebGL has tried a lot harder to
> guarantee interop than the standard suggested here.
> >>
> >> Even so, we have seen WebGL sites that only work in Chrome. So WebGL
> doesn't prove that interop isn't a concern for graphics.
> >>
> >>
> >>>
> >>> On Mon, Nov 13, 2017 at 3:29 PM, Maciej Stachowiak <mjs@apple.com>
> wrote:
> >>>>
> >>>>
> >>>>> On Nov 13, 2017, at 3:21 PM, Jeff Gilbert <jgilbert@mozilla.com>
> wrote:
> >>>>>
> >>>>> On Mon, Nov 13, 2017 at 12:27 PM, Maciej Stachowiak <mjs@apple.com>
> wrote:
> >>>>>> On Nov 13, 2017, at 11:44 AM, Dzmitry Malyshau <
> dmalyshau@mozilla.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>> Yes. Strong objection. If behavior of any programs is not fully
> specified,
> >>>>>> then web developers will start to accidentally depend on the
> behavior of one
> >>>>>> browser (usually whichever is most popular), and then browsers will
> have to
> >>>>>> reverse-engineer each others' behavior. This has happened so many
> times in
> >>>>>> the course of web standards development that it's almost a running
> joke.
> >>>>>> Every once in a while someone says "hey, let's just not define error
> >>>>>> handling, we only need to define the behavior for valid content" it
> happens.
> >>>>>> The first time was HTML, Browsers ended up reverse-engineering each
> other's
> >>>>>> error handling until finally they got sick of the W3C not defining
> this and
> >>>>>> formed the WHATWG to create HTML5, which fully specified parsing
> behavior
> >>>>>> for all invalid documents. CSS, JavaScript and WebAssembly also
> have fully
> >>>>>> interoperable behavior by spec, even in "invalid" or "error" or
> >>>>>> "ill-defined" cases.
> >>>>>>
> >>>>>> Let's not make this rookie mistake. We must fully define the
> behavior of all
> >>>>>> programs.
> >>>>>
> >>>>> We can fully define it without requiring exact behavior.
> >>>>
> >>>> I'm not sure what that means. Are you suggesting a menu choice of one
> of N behaviors? Then the most popular behavior will become a de facto
> standard.
> >>>>
> >>>>> It's not a rookie mistake to make compromises here.
> >>>>
> >>>> It totally is. I can't think of a time we've done this on the web and
> it has been ok in the long run. I gave you a very notable failure example.
> Can you cite any past successes for the strategy of sacrificing
> interoperability on the web?
> >>>>
> >>>>> There are a variety of ways
> >>>>> in which our API here, like WebGL, differs greatly from the parsing
> >>>>> deviations in early HTML. It is not white and black. Short of proved
> >>>>> code, there will always be accidental portability problems, even if
> >>>>> from nothing other than cargo-culting. We should make smart
> >>>>> compromises here, using all the tools and avenues available to us,
> >>>>> rather than have a knee-jerk requirement for maximum portability.
> >>>>> (absolute portability is not even possible with our base graphics
> >>>>> APIs)
> >>>>
> >>>> It's my impression that, historically, graphics programmers have
> accepted a much lower bar for portability than the standard we strive for
> on the web. I believe we should shoot for something much closer to the web
> bar for portability. Accidental portability problems from cargo-culting are
> exactly the kind of problems I am worried about. The more we create the
> opportunity for such problems, the more likely that in 5-10 years we'll all
> have to reverse engineer Chrome for Windows and Safari for iOS behavior for
> desktop and mobile respectively.
> >>>>
> >>>> Regards,
> >>>> Maciej
> >>>>
> >>>
> >>
>
>
>