Re: Status report re: WebAuth extension interop reporting

On 11/20/18 11:27 AM, Brett McDowell wrote:
> Sam,
> If a public thread is the best method for you, I am happy to continue 
> our dialog here.  But please confirm that you have the following 
> individuals subscribed to this list as they are critical to closing out 
> all the current open issues:
> 
>   * Yuriy Ackermann <yuriy@fidoalliance.org <mailto:yuriy@fidoalliance.org>>
>   * Rae Hayward <rae@fidoalliance.org <mailto:rae@fidoalliance.org>>

Yuriy is a member of the working group (and hence on the list).  I'll 
send Rae instructions for joining the WG.

...
> We have obviously seen hundreds of products tested and publicly listed 
> as having passed those tests so this is simply a matter of providing the 
> documentation in a format you need to see it in, and perhaps that 
> requires some leg work on our part to get permission to share anything 
> currently protected by test-event-NDAs.
> 
> Once you have Yuriy and Rae on this mailing list, would you send around 
> a specific example of the kind of test mapping documentation you are 
> accustomed to seeing?  That would be helpful.

Rather than try to reformat the data FIDO has, I encourage you to focus 
first on the specific question I asked on November 7th.  That question, 
which I managed to phrase as a yes/no, boils down to "would you please 
clarify the minimum requirements for certification, so we can see if 
certification necessarily would prove extension interop?".

I'm happy to ask Ralph for examples of what he's used to seeing, but I 
suspect it will be sufficient just to answer the specific questions we 
asked a couple of weeks ago or, perhaps, to share documents you already 
have.  Hopefully this will be easy...

-- Sam

> Best regards,
> *
> *
> *Brett McDowell***| Executive Director | FIDO Alliance 
> <https://fidoalliance.org>
> brett@fidoalliance.org <mailto:brett@fidoalliance.org> | M: 
> +1.413.404.5593 | @FIDOalliance <https://twitter.com/FIDOalliance>
> 
> 
> 
> 
> 
> 
> On Thu, Nov 15, 2018 at 2:48 PM Samuel Weiler <weiler@w3.org 
> <mailto:weiler@w3.org>> wrote:
> 
>     Colleagues,
> 
>     There are multiple threads going around with long - and different - CC
>     lists re: interop testing for the WebAuth extensions.
> 
>     This has left many people - including this W3C Team Contact - feeling
>     confused.  In the interest of improving matters, I'm starting a public
>     thread.  Hopefully, to the extent that the matters are not covered by
>     NDA, we can quit using CC lists that forget important players.
> 
>     Below are:
> 
>     1) where I think we're at, and
> 
>     2) some questions I sent to various people last week, edited to remove
>     some context to protect the innocent.
> 
>     Feel free to correct my understanding as needed.
> 
> 
>     Where I think we're at:
> 
>     The the extent that the extensions in the base WebAuth spec were
>     implemented in UAF, Ralph has agreed to accept interop testing of those
>     from the UAF context - rather than require new interop testing specific
>     to WebAuth.
> 
>     Ralph remains willing to publish any or all of the extensions marked as
>     informative (non-normative).  They could also be split into a separate
>     doc and pushed through at a later time.
> 
>     W3C has received some documentation of a) which extensions have been
>     implemented by multiple UAF devices and b) the names of certified UAF
>     implementations.  We do not have a detailed mapping of which
>     implementations were shown, though testing, to have interoperable
>     versions of which extensions.
> 
> 
>     I have asked for some more detail about the testing - or the
>     certification criteria - to reassure us that the extensions have, in
>     fact, been tested.
> 
>     I understand that FIDO, the W3C WG chairs, and others are assembling
>     such details.
> 
>     I urge patience - I think we're in relatively uncharted territory here,
>     partly because W3C proposes accepting interop testing based on another
>     spec and, more significantly, because FIDO has not provided interop
>     reports of the sort we're accustomed to seeing.
> 
>     Below are the clarifying questions I sent last week.
> 
>     -- Sam
> 
> 
>     -------- Forwarded Message --------
>     Date: Wed, 7 Nov 2018 08:40:48 -0500
>     From: Samuel Weiler <weiler@w3.org <mailto:weiler@w3.org>>
> 
>     Colleagues,
> 
>     ...
> 
>     ... forwarded this thread (or at least a portion of it) and asked me to
>     formulate some questions that may help clarify things:
> 
>     If a product had a non-interoperable implementation of one (or more) of
>     these extensions, could it still have been certified by FIDO?
> 
>     I am concerned that while a product may advertise that it implements an
>     extension, FIDO's specific certification requirements are unclear - for
>     example, if a product supporting no optional extensions would be
>     certified, I can imagine a certification program allowing that product
>     to still be certified if it contained an "early" or "pre-release"
>     extension implementation that was not (yet) interoperable.  (Perhaps
>     related: if a product did not ask for certification re: a particular
>     extension, did you test to make sure that extension was not present?)
> 
>     I think it would help to share specifics: e.g. "implementation X was
>     shown to have an interoperable implementation of extension foo".
>     Perhaps
>     you have a chart of which implementations were shown to have
>     interoperable implementations of which extensions?
> 
> 
>     ...
> 
>     ... I expect an interop report to contain more detail.
> 
>     Here are some examples that look more like what I expect.  I'm not
>     suggesting you mimic any one of these - they have their own flaws and,
>     of course, their methodology may not be applicable - but perhaps you
>     already have something more like this that you could share?
> 
>     https://datatracker.ietf.org/meeting/101/materials/slides-101-dots-ietf-101-hackathon-dots-interop-01
>     https://tools.ietf.org/html/rfc6984
>     https://tools.ietf.org/html/draft-rosen-megaco-interop-1-report-00
> 
>     -- Sam
> 
>

Received on Tuesday, 20 November 2018 19:37:27 UTC