Status report re: WebAuth extension interop reporting

Colleagues,

There are multiple threads going around with long - and different - CC 
lists re: interop testing for the WebAuth extensions.

This has left many people - including this W3C Team Contact - feeling 
confused.  In the interest of improving matters, I'm starting a public 
thread.  Hopefully, to the extent that the matters are not covered by 
NDA, we can quit using CC lists that forget important players.

Below are:

1) where I think we're at, and

2) some questions I sent to various people last week, edited to remove 
some context to protect the innocent.

Feel free to correct my understanding as needed.


Where I think we're at:

The the extent that the extensions in the base WebAuth spec were 
implemented in UAF, Ralph has agreed to accept interop testing of those 
from the UAF context - rather than require new interop testing specific 
to WebAuth.

Ralph remains willing to publish any or all of the extensions marked as 
informative (non-normative).  They could also be split into a separate 
doc and pushed through at a later time.

W3C has received some documentation of a) which extensions have been 
implemented by multiple UAF devices and b) the names of certified UAF 
implementations.  We do not have a detailed mapping of which 
implementations were shown, though testing, to have interoperable 
versions of which extensions.


I have asked for some more detail about the testing - or the 
certification criteria - to reassure us that the extensions have, in 
fact, been tested.

I understand that FIDO, the W3C WG chairs, and others are assembling 
such details.

I urge patience - I think we're in relatively uncharted territory here, 
partly because W3C proposes accepting interop testing based on another 
spec and, more significantly, because FIDO has not provided interop 
reports of the sort we're accustomed to seeing.

Below are the clarifying questions I sent last week.

-- Sam


-------- Forwarded Message --------
Date: Wed, 7 Nov 2018 08:40:48 -0500
From: Samuel Weiler <weiler@w3.org>

Colleagues,

...

... forwarded this thread (or at least a portion of it) and asked me to 
formulate some questions that may help clarify things:

If a product had a non-interoperable implementation of one (or more) of 
these extensions, could it still have been certified by FIDO?

I am concerned that while a product may advertise that it implements an 
extension, FIDO's specific certification requirements are unclear - for 
example, if a product supporting no optional extensions would be 
certified, I can imagine a certification program allowing that product 
to still be certified if it contained an "early" or "pre-release" 
extension implementation that was not (yet) interoperable.  (Perhaps 
related: if a product did not ask for certification re: a particular 
extension, did you test to make sure that extension was not present?)

I think it would help to share specifics: e.g. "implementation X was 
shown to have an interoperable implementation of extension foo". Perhaps 
you have a chart of which implementations were shown to have 
interoperable implementations of which extensions?


...

... I expect an interop report to contain more detail.

Here are some examples that look more like what I expect.  I'm not 
suggesting you mimic any one of these - they have their own flaws and, 
of course, their methodology may not be applicable - but perhaps you 
already have something more like this that you could share?

https://datatracker.ietf.org/meeting/101/materials/slides-101-dots-ietf-101-hackathon-dots-interop-01
https://tools.ietf.org/html/rfc6984
https://tools.ietf.org/html/draft-rosen-megaco-interop-1-report-00

-- Sam

Received on Thursday, 15 November 2018 19:47:53 UTC