Re: When it is appropriate to use wpt...

On Thu, Apr 21, 2016 at 5:14 PM, Geoffrey Sneddon <me@gsnedders.com> wrote:

> On Thu, Apr 21, 2016 at 7:57 PM, Shane McCarron <shane@spec-ops.io> wrote:
> >
> > On Thu, Apr 21, 2016 at 1:39 PM, Geoffrey Sneddon <me@gsnedders.com>
> wrote:
> >>
> >> I wonder if a WebDriver API would make sense (WebDriver API use
> >> inevitably already has security consequences, so I don't think it's
> >> too unreasonable); thoughts, anyone, or any better ideas?
> >
> >
> > I have a view of this where vendors could supply their own web driver
> based
> > "scripts" to drive the parts of a transaction that might require human
> > intervention.  There is a similar problem with Web Annotation - where
> there
> > is no defined UI per se, but of course each client will have one and
> there
> > needs to be a way to prod that client until they spit out an annotation
> > object.
>
> I think Andreas's comments about WebDriver likely in future supporting
> a permissions API deals with most of the need for that (though not in
> the Web Annotation case!).
>

Yep!


>
> >> > Is it appropriate to incorporate tests and tools into WPT that will
> >> > potentially utilize and/or exercise resources outside of the test
> server
> >> > (e.g., pulling in information from a tester-defined remote resource or
> >> > testing the conformance of a tester-defined remote resource)?
> >>
> >> Yes, I think it is, though I think we should add tools to wpt-tools to
> >> keep everything self-contained.
> >
> >
> > Can you expand on this?  My examples of external resources would be
> things
> > like the Visa(tm) Web Payment service.  That's not something we can put
> into
> > a self-contained environment.  Maybe *they* can, and I suppose we could
> > require that.  But it feels like a high bar to make them leap over.
> Also,
> > if the Visa(tm) Web Payment service is what is supposed to be under test,
> > and we test something different, aren't we violating the basic principle
> of
> > testing ("If it's not the same, it's different.")?
>
> From the browser testing side: I was imagining a dummy payment service
> (given there's no way we can test against live payment services!) in
> wpt-tools.
>

Me too.  Although it might not be in wpt-tools.  I mean, it could be.  But
it could just be in the "test suite" couldn't it?  Or are there sandboxing
restrictions that mean plugins / extensions to the server.py script (pipes)
need to be outside of the test suite folder?


>
> I expect most of the code needed to test the server-side stuff (e.g.,
> the Visa(tm) Web Payment service) would live in wpt.
>

Perfect.  That's what I was hoping.